Worked prompt: enterprise document Q&A
Retrieval-augmented generation at enterprise scale, where permissions, freshness and evaluation are harder than the model call.
Prompt. A 30,000-person company wants an assistant that answers questions from SharePoint, Confluence and a ticketing system, with citations, respecting existing permissions.
1. Requirements
Users see answers only from documents they can already open. Answers cite sources. New and changed documents are searchable within an hour. Target: answer begins streaming in under 3 s.
Ask: which languages? Are there documents that must never be indexed? Who owns quality?
2. Ingestion
Connectors per source pull changes (webhooks or delta queries), extract text, chunk by structure (headings, sections) rather than fixed size, embed, and write to a hybrid index (keyword plus vector) with the document's ACL principals stored on every chunk.
Deletions and permission changes are events too; they are the ones teams forget.
3. Query path
Rewrite the question (expand acronyms, resolve the conversation). Retrieve with hybrid search filtered by the user's group memberships at query time, rerank, pack the top chunks into context with source ids, generate with an instruction to cite and to say when the sources do not answer.
4. Deep dive: permissions
Security trimming at query time is mandatory; never rely on the model to withhold. Group memberships can be large; cache them per user with a short TTL and accept a small staleness window, or resolve through the identity provider for sensitive sources.
Prompt injection through documents is a real risk: a wiki page can contain instructions. Keep retrieved text clearly delimited as data, and give the assistant no tools with side effects in this version.
5. Deep dive: evaluation
Build a golden set of ~200 questions with expected source documents, from real user questions. Track retrieval recall@k separately from answer faithfulness (is every claim supported by a cited chunk?) and answer relevance. Run it on every change to chunking, embeddings, prompts or models.
6. Principal framing
Phase it: one department and one source first, measured against a baseline (time to answer, ticket deflection), then widen. Name the operating model: who curates sources, who handles "the bot was wrong" reports, and what the cost per active user is.
Practice
Written for this site to drill the lesson. Not reported interview questions.
- Users complain answers cite outdated policy documents. Diagnose using the pipeline stages.
- Estimate the index size for 5M documents averaging 8 pages, at an assumed chunk size and embedding dimension you state.
- The CISO asks how you prevent an intern from seeing board minutes through the assistant. Answer in three sentences.
Check yourself
Where must permission filtering happen?
At retrieval time, using ACLs stored with each chunk, before anything reaches the model.
Name two metrics you would track separately in RAG evals.
Retrieval recall@k and answer faithfulness (also answer relevance).