TechPrep
Building blocksReadPracticeCheck

How LLM features fail

A working mental model of tokens, context and sampling, organised around the ways production features break.

You do not need to derive attention to pass an AI Engineer loop. You do need to predict how an LLM feature will misbehave and design around it. Most failures fall into five groups.

FailureWhat causes itFirst line of defence
Confident wrong answers (hallucination)The model generates plausible text without groundingRetrieval with citations; instruct it to say "not in the sources"; check faithfulness in evals
Ignored instructionsLong or conflicting context; instruction buried mid-promptClear structure, important rules at the start or end, fewer competing instructions
Format breaksFree-text generation where a parser expects JSONStructured outputs / tool schemas; validate and retry
InconsistencySampling and prompt sensitivityLower temperature for deterministic tasks; evals over many samples, not one
Cost and latency blow-upsContext grows with history and retrieved text; agents loopBudgets per request, summarisation/compaction, caching, step limits

Words to use precisely

Interview signal: when asked "how would you build X with an LLM", start with how you will know it works (the eval), then build. Both labs' Applied AI posts put evaluation right next to building, and OpenAI's says success is "production systems ... not simply ... successful demonstrations" (Sources A6, O7).

Practice

Written for this site to drill the lesson. Not reported interview questions.

  1. A summarisation feature works in demos but users report it invents action items. Walk through diagnosis and fixes.
  2. Explain to a product manager why doubling the context window doubled the bill but not the quality.

Check yourself

What should you design first for an LLM feature?

How you will measure that it works: the eval.

Which latency metric do chat users feel most?

Time to first token.

Saved in this browser.