Excessive agency

Documented cases

Incidents happened in the wild. Research demos were shown by researchers, often disclosed to the vendor first.

• Research demodisclosed June 5, 2026·AML.CS0067

  CI secret read through a coding agent's file tool

  Microsoft researchers hid instructions in GitHub issue and pull request text processed by Anthropic's Claude Code GitHub Action. Secret scrubbing covered shell commands, but the agent's in-process Read tool could open /proc/self/environ and expose the workflow's API key.

  Microsoft reported it on April 29, 2026 and Anthropic fixed it on May 5 by blocking sensitive /proc files (Claude Code 2.1.128). Microsoft's advice: never give one workflow untrusted input, secrets and outbound communication at the same time.

  Sources: Microsoft Threat Intelligence; MITRE ATLAS AML.CS0067

• Research demodisclosed Feb. 19, 2026·AML.CS0062

  Prompt to code execution in Semantic Kernel's vector search

  In the Python version of Microsoft's Semantic Kernel, the default filter for in-memory vector search built a Python lambda from model-controlled values and ran it with eval(). A crafted prompt got past the blocklist and ran code on the host.

  Fixed in semantic-kernel 1.39.4 (CVE-2026-26030). Microsoft's lesson: any tool parameter the model can influence is attacker-controlled input.

  Sources: Microsoft Security; NIST National Vulnerability Database; MITRE ATLAS AML.CS0062

• Research demodisclosed Aug. 6, 2025·AML.CS0063

  A calendar invite that took over Gemini

  SafeBreach researchers put instructions in the title of a Google Calendar invitation. When the victim asked Gemini about their calendar, it followed them: deleting events, opening a video call, controlling smart-home devices through Google Home and leaking email subject lines in URLs.

  The researchers reported it in February 2025. Google described layered defenses, including stronger confirmations for sensitive actions and URL sanitization, and called the techniques theoretical.

  Sources: SafeBreach Labs; MITRE ATLAS AML.CS0063

• Incidentdisclosed July 23, 2025·AML.CS0047

  Wiper prompt shipped in the Amazon Q VS Code extension

  An attacker used a GitHub token with too much scope to commit code to the open-source repository of the Amazon Q Developer extension for VS Code, and the code shipped in version 1.84.0. MITRE's write-up says the code prompted the Q agent to clean a system to a near-factory state and delete file system and cloud resources.

  AWS says the code failed to run because of a syntax error and made no changes to services or customer environments. Version 1.85.0 removed it (CVE-2025-8217).

  Sources: Amazon Web Services; NIST National Vulnerability Database; MITRE ATLAS AML.CS0047

Controls to start with

• Give the agent only the tools it needs, with only the functions it needs. Avoid open-ended tools such as a raw shell or arbitrary URL fetcher.LLM03:2026, mitigations 1 to 3
• Run tools with the permissions of the user being served, not a broad service account.LLM03:2026, mitigations 4 and 5
• Require user approval for high-impact actions, and enforce authorization in the downstream system itself.LLM03:2026, mitigations 6 and 7
• Issue short-lived, task-scoped credentials and log every tool call.ASI02, guidelines 6 and 8

Questions to ask vendors

Our suggestions, not taken from a framework.

• List every action the agent can take on its own. Which ones are reversible?
• Whose identity and permissions does the agent use when it calls our systems?
• Can we require approval for specific actions, and is that enforced outside the model?