Google DeepMind has introduced the AI Control Roadmap, a framework for building and managing advanced AI agents within Google. This 'defense-in-depth' approach adds system-level security beyond traditional model alignment, providing assurance even if alignment is imperfect.
Understanding AI Control
The roadmap builds on traditional safeguards like sandboxing and prompt injection resistance, then adds model alignment as a primary defense. It treats internal agents as potentially misaligned, using a threat-modeling framework based on MITRE ATT&CK to break down potential attacks into tactics and techniques. This allows systematic risk tracking and testing of defenses.
Key Components
- Detection: Trusted AI 'supervisors' monitor agent reasoning, actions, and plans to identify off-track behavior.
- Prevention and Response: Supervisors can block harmful actions in real-time or after the fact, depending on risk level.
- Metrics: Coverage, recall, and time-to-response are constantly measured.
Scaling Security
As AI capabilities advance, defenses must strengthen. The roadmap maps security protocols to milestones in detection evasion (D1-D4) and attack execution (R1-R3). For low-risk actions, asynchronous response suffices; for high-risk actions like major cyber attacks, synchronous real-time prevention is required.
Research Insights
An internal prototype analyzed a million coding agent trajectories, moving beyond keyword filtering to identify high-signal behavioral patterns. This informed a live monitor for the Gemini Spark agent, enabling real-time response to issues like unintentional data deletion. Most flagged events stem from misinterpretation or overeagerness, not adversarial intent.
Shared Responsibility
DeepMind also published 'Three Layers of Agent Security' for policymakers, emphasizing security at individual agent, multi-agent, and ecosystem levels. The roadmap is part of a multilayered approach to agent security, intended as a model for the wider industry.
Images
