Trust boundaries
An agent drawn as values flowing between the user, the model, its tools and the outside world, each value labelled by where it came from and how secret it is; the boundaries it crosses are where defences go.
Loading the animation…
Concept
An agent is a model in a loop with tools. Draw it as four places: the user, who asks; the model, which decides; the tools, which read and change things; and the outside world, where messages, web requests and uploads go. Every step moves a value from one place to another, and every move crosses a boundary. The boundaries are where an agent can go wrong, and so they are where defences go.
This site never looks at what a value says. It looks only at its label: where the value came from (the source: the user, a trusted tool such as the user's own notes, or something untrusted that a third party could have written, such as a web page or a message from someone else) and how secret it is (public or private). That gives 6 labels. A value made from other values carries the join of their labels: the more tainted source and the more secret sensitivity. Labels only ever go up as data flows.
The animation runs a small task through the engine's 8 abstract tools: read my notes and my inbox, then reply to a colleague. The request is the user's and public. The notes come back trusted and private; the inbox comes back untrusted and private (anyone can send the user a message). The summary joins them, so it is untrusted and private. When the reply tries to leave, the policy at the gate between the model and the world sees an untrusted, private value heading to a destination that may not read it, and the decision is deny. Pick another task: a summary of the user's own notes written to a file goes through untouched, and so does a digest the user sends to themselves.
Maths
Order the sources and the sensitivities . A label is a pair, and labels are ordered part by part (the product order), which makes them a lattice: any two labels have a least upper bound, the join , and a greatest lower bound, the meet . The join is what a derived value gets, so a value is at least as tainted and at least as secret as everything it was made from. The engine's tests check every lattice law (order, bounds, commutativity, associativity, absorption) on all 6 labels.
A call is also controlled by something: the model chose to make it, and chose its arguments, after reading what was in its context. So a call's effective label is the join of its arguments' labels and its control label. A read returns the join of its source's label and the call's effective label (what was read depends on how the call was chosen). Chapter 2 is about the control label.
This is classic information-flow control (Denning 1976, a lattice model of secure information flow), applied to the values an agent moves around. CaMeL (Debenedetti et al. 2025) applies the same idea to agents, with capabilities attached to values.
Code
The join, cut from the vendored engine (policy/labels.ts):
export function join(a: Label, b: Label): Label {
return { src: si(a.src) >= si(b.src) ? a.src : b.src, sens: ci(a.sens) >= ci(b.sens) ? a.sens : b.sens };
}