The frontier

Every defence's cost (tasks it blocks, prompts, extra calls) against the risk it leaves, on the same benign suite, for fixed attacks and for an attacker who adapts: the security–utility frontier.

Loading the animation…

Concept

Every defence in this site has a price. The label policy blocks tasks the user wanted; approvals cost attention; detectors add model calls and flag some benign results; a planner adds a call and cannot do every task unattended. The engine measures the price and the protection on the same tasks: a suite of 120 benign tasks drawn with its seeded generator (each reads one to three sources, summarises them and answers, saves a file or sends a message; 21% branch on what they read). For each of 8 defences it counts the tasks that finish, the prompts and the extra calls, and the chance that one steering attempt in a task's untrusted input ends in a write or send that nothing stops. Threats appear only as that abstract event.

The animation adds one defence per step to the plot: tasks finished across, risk up (on a log scale, so each gridline is ten times the last). A filled point is on the frontier: no other defence is at least as good on tasks finished, risk and prompts, and better on one. Switch the attacker:

Fixed attacks, the published rates: the detectors look best. Three layers leave a risk of 0.24% per task and still finish 95.1% of tasks, but those rates were measured against fixed sets of attacks (chapter 6).
An attacker who adapts, getting past every detector (Zhan et al. 2025: over 50% success against all eight defences tested) at a 50% base rate: the detectors fall back to the undefended risk, 42.9%. The designs that never read content do not move. The label policy alone leaves 25.7%; adding least privilege takes that to 16.6% at no cost in tasks (both finish 72.5%); planning in advance with least privilege leaves 2.4% and finishes 76.7%, with 0.25 prompts and 1.00 extra call per task. Approving every write and send finishes everything and leaves 8.9%, paid in attention (0.78 prompts per task, with the fatigue of chapter 8).

What does the policy block? Every blocked task traces to one rule: a send carrying data from untrusted, private sources (an inbox) to anyone but the user, in 33 of the 120 tasks (27.5%). A real deployment would make choices here, for example allowing colleagues as destinations and taking that risk back; the frontier is a menu of such choices, not a verdict. For scale: on AgentDojo, CaMeL kept 92% of the undefended system's solved tasks (77% against 84%, Debenedetti et al. 2025); this suite's figures are not comparable with that, only the shape is.

Maths

For task $t$ and defence $d$ : $\text{keep}(t,d)$ is 0 if the policy blocks a call the task needs, and otherwise $\prod_i (1 - f_i)^{s_i}$ for detector layers with false-positive rate $f_i$ scanning $s_i$ items. The risk is $p \cdot P_{\text{miss}}(d) \cdot \max_r \text{pass}_r(t,d)$ , with $P_{\text{miss}}$ the common-mode mixture of chapter 6 and the routes of chapter 8. Averages over the $N$ tasks give each defence's point. A point $a$ is beaten by $b$ when $b$ finishes at least as many tasks, with at most the risk and at most the prompts, and is strictly better in one; the frontier is the points nobody beats. The engine's tests check the closed forms against a seeded simulation (planning in advance: 1.04% simulated over 48,000 task draws, against 1.14% computed), that least privilege never adds risk or blocks a benign task, that an adaptive attacker never lowers risk, and the frontier against its definition on random point sets.

Code

The frontier's test, cut from the vendored engine (policy/utility.ts).

if (
  b.utility >= a.utility &&
  b.risk <= a.risk &&
  b.prompts <= a.prompts &&
  (b.utility > a.utility || b.risk < a.risk || b.prompts < a.prompts)
)
  beaten = true;