Detection is not enough

Layered, imperfect detectors multiply their misses but never reach zero, and a blind spot they share sets a floor no extra layer removes: attempts drawn one by one through the layers, beside the closed forms and the published rates.

Loading the animation…

Concept

Chapters 1 to 5 put defences where the labels say data crosses a boundary. The other family of defences tries to recognise a steering attempt: mark untrusted text so the model can tell it apart (Spotlighting's provenance marking, Hines et al. 2024), run a classifier over each tool result, check the model's output before it acts. Each is a filter, and each misses some attempts. Stacking them helps, because an attempt must get past every layer, so the misses multiply.

The animation draws 400 attempts arriving in untrusted results. At the published base rate (24%, InjecAgent's ReAct GPT-4 figure, Zhan et al. 2024) about a quarter of them would steer an undefended model. Then three layers are added: provenance marking, which misses 4% (the ratio of Spotlighting's published bounds, below 2% from above 50%), an input classifier that misses 20% and an output check that misses 30%. If the layers miss independently, all three miss 0.24% of attempts, and end to end 0.06% get through. That is a large reduction, and it is not zero: over 1,000 attempts, the chance that at least one gets through is 44%.

Real layers do not miss independently. Detectors trained on similar data, or reading the same rewritten text, share blind spots: some attempts evade them all at once. Choose a shared blind spot in the animation. With just 1% of attempts evading every layer together, all three miss 1.17%, about five times the independent figure, and the chance that one of 1,000 gets through rises to 94%. A blind spot every layer shares is a floor that no extra layer can lower.

The published rates are also measured against fixed sets of attacks. An attacker who can probe the defences adapts to them: Zhan et al. (2025) evaluated eight defences against indirect prompt injection and bypassed every one with adaptive attacks, at an attack success rate over 50%. So detection is worth having as a layer, and is never the whole answer. The rest of this part is about defences that do not depend on recognising anything: designs that make the dangerous flow impossible (chapter 7), humans at the gate (chapter 8), and what each costs (chapter 9).

Maths

Layer $i$ misses an attempt with probability $m_i$ . Independently, all $n$ layers miss with probability $\prod_i m_i$ , which is positive whenever every $m_i$ is. The engine models correlation as a common-mode mixture: with probability $c$ an attempt evades every layer together; otherwise layer $i$ misses independently with $q_i = (m_i - c)/(1 - c)$ , chosen so each layer's own miss rate stays $m_i$ . Then

P(\text{all miss}) = c + (1 - c) \prod_i q_i \;\ge\; c,

and the correlation between two layers' misses is $\big(P(\text{both}) - m_i m_j\big) / \sqrt{m_i(1 - m_i)\, m_j(1 - m_j)}$ with $P(\text{both}) = c + (1-c)\, q_i q_j$ . Over $N$ attempts, each getting through with probability $p\,P(\text{all miss})$ , at least one gets through with probability $1 - \big(1 - p\,P(\text{all miss})\big)^N$ , which tends to 1 as $N$ grows for any positive rate. The engine's tests check the closed forms against a seeded simulation of 20,000 attempts per blind spot, and the animation's draws against the same closed form.

Code

The mixture, cut from the vendored engine (policy/depth.ts): the shared blind spot plus the independent part.

export function allMissCommon(m: number[], c: number): number {
  return c + (1.0 - c) * allMissIndependent(qGivenCommon(m, c));
}