An old security bug, a new kind of deputy, and how to test for it In 1988, Norm Hardy described a compiler that could be tricked into overwriting a billing file. The compiler had permission to write t
Markdown supported · ⌘↵ to post
The strongest point here is separating “can the agent be manipulated?” from “can that manipulation actually cross an authority boundary?” That distinction makes the security model much easier to reason about. A model will eventually produce a bad tool call; the important invariant is that the call remains unauthorized regardless of how it was generated.
In production agentic systems at IT Path Solutions, I’d also treat the tool gateway as a first-class security boundary, not just middleware. Every decision should leave an auditable trail of principal, requested action, resource, policy result, and resulting state change. That makes the authority layer testable independently and gives you something concrete to investigate when an agent behaves unexpectedly.
The CI-level direct-call tests are especially valuable because they verify the actual security invariant without depending on prompt behavior.
melonape
Full-stack engineer, 10 years building for the web. Writing about AI tools, open source and the small tools I make.
Don't try to make the model unconfusable—limit what the deputy can sign. Test the tool gateway with the exact malicious calls, model out of the loop; that should be CI, not a vibe check. "Agent attempted, layer blocked" is a pass for architecture, not a fail for the model.