TechPrep
Building blocksReadPracticeCheck

Retrieval-augmented generation, end to end

Ingest, chunk, embed, retrieve, rerank, generate with citations, and evaluate retrieval separately from generation.

RAG gives a model facts it was not trained on, at answer time. Most RAG quality problems are retrieval problems, so measure retrieval on its own before touching the prompt.

The pipeline

StageDecisionsTypical mistake
IngestConnectors, change detection, permissions captured per documentForgetting deletes and permission changes
ChunkBy structure (headings, sections), with overlap; keep titles and breadcrumbs in each chunkFixed 500-token chunks that split tables and lose context
EmbedModel choice, dimension, language coverageChanging embedding model without re-embedding everything
IndexHybrid: keyword (BM25) plus vector; metadata filtersVector-only search that misses exact codes, names and acronyms
RetrieveQuery rewriting, top-k, filters (tenant, ACL, date)No permission filter at query time
RerankCross-encoder or LLM reranker on the top 20 to 50Skipping it, then raising top-k until the context is noise
GenerateCite chunk ids; allow "I don't know"No instruction to refuse when sources do not answer

Evaluate the two halves separately

When not to use RAG

If the corpus fits comfortably in context and changes rarely, put it in the prompt and cache it. If the task needs the model to learn a style or format, fine-tuning or examples may fit better. If the answer requires computation over structured data, give the model a SQL or API tool instead of retrieved text.

Practice

Written for this site to drill the lesson. Not reported interview questions.

  1. Your RAG bot answers questions about product codes badly. Which stage do you suspect, and what do you change?
  2. Design the golden set for an HR policy assistant: where do questions come from, how many, and who labels them?
  3. Explain hybrid search to an architect who only knows SQL LIKE queries.

Check yourself

Why hybrid search?

Vectors capture meaning; keyword search catches exact names, codes and acronyms that embeddings blur.

What two things do RAG evals measure separately?

Retrieval quality (recall@k) and generation quality (faithfulness, relevance).

Saved in this browser.