TechPrep
Production AI systemsReadLabPracticeCheck

Evals: how you know it works

A versioned test set, metrics that match the task, graders you have checked, and a gate on every prompt or model change.

Evals are the most consistent requirement across the lab job posts we read: Applied AI roles at Anthropic ask candidates to "help customers develop evaluation frameworks", and OpenAI's FDE post defines success through "evaluation loops that establish customer-specific benchmarks, acceptance criteria, and launch readiness" (official, Sources A7, O6).

Build one in a day

  1. Collect 50 to 200 real inputs, stratified: common cases, known hard cases, adversarial ones, and the cases that caused incidents.
  2. Define what good means per input: an exact answer, required facts, a rubric, or a reference document.
  3. Choose graders: code checks first (format, exact match, required fields, citations present), then model-graded rubrics for qualities code cannot check, then human review on a sample.
  4. Calibrate any LLM judge against 30 to 50 human labels; report agreement. An unchecked judge is a guess.
  5. Run it on every change to prompts, models, retrieval or tools, and track results over time. Block releases that regress.
  6. Feed production back in: user thumbs-down, escalations and support tickets become new eval cases.
TaskUseful metrics
Classification / extractionAccuracy, precision/recall per class, exact-match per field
RAG Q&ARetrieval recall@k, faithfulness, answer relevance, refusal correctness
AgentsTask success rate, steps and cost per success, tool-call error rate, unsafe-action rate
GenerationRubric scores (pairwise preference is often more reliable than absolute scores)
Report variance. LLM outputs are stochastic: run each case several times or at least report confidence intervals before claiming a 2-point improvement.

Eval readiness checklist

Tick what your current or most recent LLM project has. The gaps are your talking points: what you would add next.

Practice

Written for this site to drill the lesson. Not reported interview questions.

  1. Design the eval for an assistant that drafts responses to procurement security questionnaires.
  2. Your LLM judge says the new prompt is 8% better; humans disagree. What do you do?
  3. Explain to an executive, in three sentences, why the team cannot ship the new model until the eval runs.

Check yourself

What must you do before trusting an LLM judge?

Measure its agreement with human labels on a sample.

Why prefer code checks first?

They are cheap, deterministic and unambiguous; use model grading only for what code cannot check.

Sources

Saved in this browser.