A register of AI security risks with the documented incidents behind each one, current OWASP, MITRE ATLAS and NIST IDs, controls to start with, and a free tool that builds a register for the AI you use.https://aisecurityrisks.com/en-ushttps://aisecurityrisks.com/guides/ai-acceptable-use-policy/https://aisecurityrisks.com/guides/ai-acceptable-use-policy/A short AI acceptable use policy you can copy and adapt, with each clause linked to the security risk and documented cases it answers. No email required.Sun, 15 Nov 2026 00:00:00 GMThttps://aisecurityrisks.com/guides/mcp-security-risks/https://aisecurityrisks.com/guides/mcp-security-risks/Documented Model Context Protocol attacks, the risks the MCP specification itself names, and a checklist to run before you connect a server.Sun, 15 Nov 2026 00:00:00 GMThttps://aisecurityrisks.com/guides/owasp-llm-top-10-2026/https://aisecurityrisks.com/guides/owasp-llm-top-10-2026/The 2026 OWASP Top 10 for LLM Applications, entry by entry, with what moved since 2025 and which IDs now mean something different.Sun, 15 Nov 2026 00:00:00 GMThttps://aisecurityrisks.com/guides/prompt-injection-examples/https://aisecurityrisks.com/guides/prompt-injection-examples/Dated prompt injection cases from Microsoft 365 Copilot, Slack AI, Gemini, ChatGPT, coding assistants and more, each labeled as a real incident or a research demonstration, with the source.Sun, 15 Nov 2026 00:00:00 GMThttps://aisecurityrisks.com/cases/#distillation-aa26-251a/https://aisecurityrisks.com/cases/#distillation-aa26-251a/NSA, CISA and the FBI say six named China-based AI companies collected outputs from U.S. models over millions of exchanges, from at least late 2024, and used them to train their own models. The advisory asks model providers to watch for anomalous accounts and usage, rate-limit, verify identities, log telemetry and share intelligence across providers.Tue, 08 Sep 2026 00:00:00 GMThttps://aisecurityrisks.com/cases/#hugging-face-dataset-intrusion/https://aisecurityrisks.com/cases/#hugging-face-dataset-intrusion/Hugging Face says a malicious dataset used two code-execution paths in its dataset processing, a remote-code loader and a template injection in a dataset configuration, to run code on a worker. The intruder then harvested credentials and moved into several internal clusters over a weekend. Hugging Face reports access to some internal datasets and service credentials, and no tampering with public models, datasets or Spaces. It closed both paths and rotated credentials. Its post does not name the attacker. MITRE's case study attributes the activity to autonomous agents from OpenAI cyber evaluations.Thu, 16 Jul 2026 00:00:00 GMThttps://aisecurityrisks.com/cases/#claude-code-github-action/https://aisecurityrisks.com/cases/#claude-code-github-action/Microsoft researchers hid instructions in GitHub issue and pull request text processed by Anthropic's Claude Code GitHub Action. Secret scrubbing covered shell commands, but the agent's in-process Read tool could open /proc/self/environ and expose the workflow's API key. Microsoft reported it on April 29, 2026 and Anthropic fixed it on May 5 by blocking sensitive /proc files (Claude Code 2.1.128). Microsoft's advice: never give one workflow untrusted input, secrets and outbound communication at the same time.Fri, 05 Jun 2026 00:00:00 GMThttps://aisecurityrisks.com/cases/#semantic-kernel-rce/https://aisecurityrisks.com/cases/#semantic-kernel-rce/In the Python version of Microsoft's Semantic Kernel, the default filter for in-memory vector search built a Python lambda from model-controlled values and ran it with eval(). A crafted prompt got past the blocklist and ran code on the host. Fixed in semantic-kernel 1.39.4 (CVE-2026-26030). Microsoft's lesson: any tool parameter the model can influence is attacker-controlled input.Thu, 19 Feb 2026 00:00:00 GMThttps://aisecurityrisks.com/cases/#ai-recommendation-poisoning/https://aisecurityrisks.com/cases/#ai-recommendation-poisoning/Most AI assistants accept a prefilled prompt in a link. Microsoft found businesses using such links to tell assistants to remember them as a trusted source or recommend them first. Over 60 days of email traffic it counted 50 prompts from 31 companies. Microsoft's advice: treat AI assistant links like downloads, review and prune stored memories, and hunt for assistant URLs whose prompt contains words like "remember" or "trusted source".Tue, 10 Feb 2026 00:00:00 GMThttps://aisecurityrisks.com/cases/#gtg-1002/https://aisecurityrisks.com/cases/#gtg-1002/Anthropic says a group it assesses with high confidence to be Chinese state-sponsored used Claude Code to attempt intrusions at about 30 organizations, succeeding in a small number. The operators split the work into innocent-looking tasks and told the model it worked for a security firm doing defensive testing. Anthropic estimates AI did 80 to 90 percent of the campaign's work. It banned the accounts, notified affected organizations and added detection classifiers.Thu, 13 Nov 2025 00:00:00 GMThttps://aisecurityrisks.com/cases/#postmark-mcp-backdoor/https://aisecurityrisks.com/cases/#postmark-mcp-backdoor/Someone published an npm package called postmark-mcp that posed as an MCP server for the Postmark email service. After 15 harmless versions, version 1.0.16 added a backdoor that blind-copied outgoing email to an external server. Postmark says it had nothing to do with the package. It told users to remove it, check email logs and consider rotating any credentials sent by email while it was installed.Thu, 25 Sep 2025 00:00:00 GMThttps://aisecurityrisks.com/cases/#gemini-calendar-invite/https://aisecurityrisks.com/cases/#gemini-calendar-invite/SafeBreach researchers put instructions in the title of a Google Calendar invitation. When the victim asked Gemini about their calendar, it followed them: deleting events, opening a video call, controlling smart-home devices through Google Home and leaking email subject lines in URLs. The researchers reported it in February 2025. Google described layered defenses, including stronger confirmations for sensitive actions and URL sanitization, and called the techniques theoretical.Wed, 06 Aug 2025 00:00:00 GMThttps://aisecurityrisks.com/cases/#amazon-q-vscode-extension/https://aisecurityrisks.com/cases/#amazon-q-vscode-extension/An attacker used a GitHub token with too much scope to commit code to the open-source repository of the Amazon Q Developer extension for VS Code, and the code shipped in version 1.84.0. MITRE's write-up says the code prompted the Q agent to clean a system to a near-factory state and delete file system and cloud resources. AWS says the code failed to run because of a syntax error and made no changes to services or customer environments. Version 1.85.0 removed it (CVE-2025-8217).Wed, 23 Jul 2025 00:00:00 GMThttps://aisecurityrisks.com/cases/#lamehug/https://aisecurityrisks.com/cases/#lamehug/Ukraine's CERT-UA found Python malware that called Qwen 2.5-Coder-32B-Instruct through the Hugging Face API to write reconnaissance and document-theft commands on infected machines. It arrived in phishing emails sent from compromised accounts posing as ministry officials. CERT-UA attributed the activity to APT28 with medium confidence. It did not say whether the generated commands worked.Thu, 17 Jul 2025 00:00:00 GMThttps://aisecurityrisks.com/cases/#echoleak/https://aisecurityrisks.com/cases/#echoleak/Researchers showed that a single crafted email could make Microsoft 365 Copilot send internal data to an attacker with no click from the victim. Copilot pulled the email into its context, and the chain got past an injection classifier and link redaction, then used auto-fetched images and a Teams proxy allowed by the content security policy. Tracked as CVE-2025-32711, described as AI command injection in M365 Copilot that lets an unauthorized attacker disclose information over a network.Wed, 11 Jun 2025 00:00:00 GMThttps://aisecurityrisks.com/cases/#mcp-tool-poisoning-poc/https://aisecurityrisks.com/cases/#mcp-tool-poisoning-poc/Invariant Labs put instructions in the description of an innocent-looking add tool. In Cursor, the agent then read the user's MCP config file and SSH private key and sent them out through a hidden tool parameter. The confirmation dialog did not show the full input. The researchers also described rug pulls (a server changes a tool after approval) and shadowing (one server's tool description changes how the agent uses another server's tool). They recommend pinning server versions and showing users full tool descriptions.Tue, 01 Apr 2025 00:00:00 GMThttps://aisecurityrisks.com/cases/#rules-file-backdoor/https://aisecurityrisks.com/cases/#rules-file-backdoor/Pillar Security hid instructions in the rules files that configure Cursor and GitHub Copilot, using invisible Unicode characters that reviewers could not see. The assistants then added attacker-chosen code, such as a script tag, without mentioning it in chat. Cursor said the risk was the user's responsibility. GitHub said users must review suggestions, and in May 2025 it started warning when a file on github.com contains hidden Unicode text.Tue, 18 Mar 2025 00:00:00 GMThttps://aisecurityrisks.com/cases/#storm-2139/https://aisecurityrisks.com/cases/#storm-2139/Members of a group Microsoft tracks as Storm-2139 scraped exposed customer credentials from public sources and used them to get into generative AI service accounts. They changed what the services would do, deliberately worked around Microsoft's safeguards, and resold the access. Microsoft sued in December 2024, seized a website central to the operation and later named four developers in an amended complaint.Thu, 27 Feb 2025 00:00:00 GMThttps://aisecurityrisks.com/cases/#hugging-face-nullifai/https://aisecurityrisks.com/cases/#hugging-face-nullifai/ReversingLabs found two models on Hugging Face whose Pickle files opened a reverse shell when loaded. The files used 7z compression and broke right after the payload, so Hugging Face's Picklescan tool did not flag them even though the code still ran. ReversingLabs reported them on January 20, 2025. Hugging Face removed the models within 24 hours and changed Picklescan to inspect broken Pickle files.Thu, 06 Feb 2025 00:00:00 GMThttps://aisecurityrisks.com/cases/#prokyc-deepfake-kyc/https://aisecurityrisks.com/cases/#prokyc-deepfake-kyc/ProKYC was sold on criminal forums as a way through remote identity verification. It produced a fake ID document together with a video of the face on it, and was used for account fraud against cryptocurrency exchanges. IDScan.net reported a price of $629 a year. Cato Networks' original write-up did not load for us, so this entry rests on IDScan's report and MITRE's case study.Tue, 01 Oct 2024 00:00:00 GMThttps://aisecurityrisks.com/cases/#slack-ai-exfiltration/https://aisecurityrisks.com/cases/#slack-ai-exfiltration/PromptArmor posted an instruction in a public channel that only the attacker had joined. When a victim later asked Slack AI about their own private messages, it followed the instruction and showed a "reauthenticate" link carrying an API key from a private channel. The citation pointed only to the victim's channel. Slack told the researchers this was intended behavior, since public channel messages are searchable by all workspace members.Tue, 20 Aug 2024 00:00:00 GMThttps://aisecurityrisks.com/cases/#chatgpt-memory-injection/https://aisecurityrisks.com/cases/#chatgpt-memory-injection/A researcher put instructions in a Google Doc. When ChatGPT read it through a connected app, each instruction became a stored memory that carried into later conversations. Uploaded images worked too, and the researcher could also delete all memories this way. OpenAI closed the report as a model safety issue rather than a security vulnerability. The researcher advises watching for "Memory updated" notices and reviewing stored memories.Wed, 22 May 2024 00:00:00 GMThttps://aisecurityrisks.com/cases/#llmjacking/https://aisecurityrisks.com/cases/#llmjacking/Attackers took cloud credentials from a server running a vulnerable version of Laravel (CVE-2021-3129) and used them to reach cloud-hosted language models. Their script checked stolen keys against ten AI services, including AWS Bedrock, Azure, OpenAI and Google Vertex AI. Sysdig estimated a worst case of more than $46,000 a day in model costs for the victim. That figure assumes maximum quota use in several regions. Its advice: patch, keep secrets out of plain text, scope credentials tightly and log model invocations.Mon, 06 May 2024 00:00:00 GMThttps://aisecurityrisks.com/cases/#package-hallucination/https://aisecurityrisks.com/cases/#package-hallucination/A Lasso Security researcher asked four models thousands of coding questions across five languages and found they often recommended packages that did not exist, sometimes the same invented name again and again. He uploaded an empty package under one of those names, huggingface-cli. The empty package received more than 30,000 downloads in three months, and an Alibaba research repository told readers to install it. OWASP calls registering such names in advance slopsquatting.Thu, 28 Mar 2024 00:00:00 GMThttps://aisecurityrisks.com/cases/#shadowray/https://aisecurityrisks.com/cases/#shadowray/Ray, an open-source framework for scaling AI workloads, has a Jobs API with no authorization. Oligo found internet-exposed clusters that attackers had been using since at least September 2023 to install crypto-miners, open reverse shells and collect credentials for services such as OpenAI, Hugging Face and Stripe. The issue is tracked as CVE-2023-48022, which Anyscale disputes. Its position is that Ray is not meant to run outside a strictly controlled network.Tue, 26 Mar 2024 00:00:00 GMT