TechPrep
Securing AI systemsReadPracticeCheck

Prompt injection and tool permissions

You cannot reliably filter injection out of text, so design so that a fooled model still cannot do damage.

Prompt injection is not a bug you patch once. Models follow instructions, and untrusted text can contain instructions. Detection helps, but the robust defence is architectural: limit what a fooled model can reach.

The dangerous combination

Risk is highest when one agent has all three of: access to private data, exposure to untrusted content, and a way to send data out (email, web requests, rendering images from URLs). Security researcher Simon Willison calls this the "lethal trifecta" (third-party). Remove any one leg and the exfiltration path closes.

Design controls

Tool policy for a support agent (example)

tool                 reads private   untrusted input   sends out   approval
search_kb            no              yes (KB pages)    no          none
get_customer_orders  yes             no                no          none
draft_reply          no              no                no          none
send_reply           no              no                yes         human agent clicks Send
refund_order         yes             no                yes         human, above $50

Practice

Written for this site to drill the lesson. Not reported interview questions.

  1. An agent summarises GitHub issues and can comment on PRs. Write its tool policy table.
  2. Explain to a product manager why "just add a filter for 'ignore previous instructions'" is not enough.

Check yourself

What three capabilities together create an exfiltration path?

Access to private data, exposure to untrusted content, and a way to send data out.

What is the robust defence against prompt injection?

Architecture: limit what a fooled model can reach, with least-privilege tools and approvals.

Sources

Saved in this browser.