Data protection, tenancy and retention
Know what data enters prompts, logs, indexes and caches, and keep each tenant's data in its own lane everywhere it goes.
- Inventory every copy: prompts, responses, logs, traces, eval datasets, vector indexes, caches, fine-tuning data.
- Classify each and set retention. Eval datasets built from production traffic are production data.
- Isolate tenants: tenant id on every chunk and cache key; filter at query time; separate indexes or keys for regulated customers.
- Minimise: redact PII before it reaches the model where the task allows.
- Encrypt with keys you control where customers require it; know how deletion requests propagate to indexes and logs.
Response caches keyed only on the prompt text will eventually serve one customer's answer to another. Include tenant and user scope in every cache key.
Practice
Written for this site to drill the lesson. Not reported interview questions.
- A customer invokes their right to deletion. List every store in an LLM support system that must be purged.
- Design tenant isolation for a vector index serving 2,000 business customers, three of them regulated.
Check yourself
Are eval datasets built from user traffic production data?
Yes; classify, protect and retain them as such.
What must every cache key include in a multi-tenant LLM system?
Tenant (and where relevant user) scope.
Saved in this browser.