A builder's checklist
Nine checks derived from the chapters, each measured by the engine on a design you choose and linked back to its chapter: labels, routes, the trifecta, least privilege, the sandbox, detection, the plan, prompts and the frontier.
Loading the animation…
Concept
The chapters reduce to nine questions to ask of any agent design. Each is measured here, not answered by hand: choose a design in the animation (a defence from chapter 9, whether every write and send needs approval, and which sandbox layers are on) and the engine audits it on the benign suite, against an adaptive attacker, one check per step.
- Labels (chapter 1): does every value carry where it came from and how secret it is, and is every call checked against a policy?
- Untrusted control (chapter 2): can a tool result choose a write or send that nothing checks?
- The trifecta (chapter 3): does any task hold private data, untrusted input and a way out with nothing to stop the flow?
- Least privilege (chapter 4): does each task hold only the tools, scopes and destinations it needs, with tokens bound to one audience?
- The sandbox (chapter 5): is code confined by a file scope, a container and an egress allow-list?
- Detection (chapter 6): are detectors one layer among several, never the only defence?
- Structure (chapter 7): is the plan fixed before any untrusted data is read?
- Humans (chapter 8): are prompts few enough that each one is read?
- The frontier (chapter 9): is the design on the frontier, within a risk budget of 5% per task?
The default design (planning in advance with the label policy and least privilege, and all three sandbox layers) passes 7 of the nine, finishing 76.7% of the tasks with a risk of 2.4% per task. Two checks warn, and both are honest. Check 2 is warn: tasks that branch on what they read let untrusted data steer their last step, and the policy allows some of those calls. Check 3 is warn: tasks that need private data, untrusted input and a send still hold all three; the policy denies the flow out, but the circles are there. Turn on approval for every write and send and check 2 becomes pass, while check 8 becomes warn (0.55 prompts per task). No design passes all nine on this suite: that is the frontier's lesson in another form. Try the others, from no defence at all to detectors alone, and follow each check back to its chapter.
Maths
Each check is a function of the design measured on the suite of tasks: for example check 2 is the share of tasks whose best route passes with probability 1, check 4 the mean number of unneeded tools per task, , check 8 the mean prompts per task, and check 9 asks whether the design's point (tasks finished, risk, prompts) is beaten by any defence on the adaptive frontier, and whether its risk is under the budget. The score is the number of passes. The engine's tests check that no defence scores higher than the default design, and that the checks are the same in Python and in the browser for every defence, with and without approvals, with and without the sandbox.
Code
Check 7, cut from the vendored engine (policy/study.ts): planned in advance passes, a classic agent behind a policy warns, and no policy fails.
verdict: d.mode === "dataflow" ? "pass" : d.policy ? "warn" : "fail",