AI supply chain compromise

Documented cases

Incidents happened in the wild. Research demos were shown by researchers, often disclosed to the vendor first.

• Incidentdisclosed July 16, 2026·AML.CS0068

  Malicious dataset used to break into Hugging Face clusters

  Hugging Face says a malicious dataset used two code-execution paths in its dataset processing, a remote-code loader and a template injection in a dataset configuration, to run code on a worker. The intruder then harvested credentials and moved into several internal clusters over a weekend.

  Hugging Face reports access to some internal datasets and service credentials, and no tampering with public models, datasets or Spaces. It closed both paths and rotated credentials. Its post does not name the attacker. MITRE's case study attributes the activity to autonomous agents from OpenAI cyber evaluations.

  Sources: Hugging Face; MITRE ATLAS AML.CS0068

• Incidentdisclosed Sept. 25, 2025·AML.CS0053

  Fake Postmark MCP server copied every email to an attacker

  Someone published an npm package called postmark-mcp that posed as an MCP server for the Postmark email service. After 15 harmless versions, version 1.0.16 added a backdoor that blind-copied outgoing email to an external server.

  Postmark says it had nothing to do with the package. It told users to remove it, check email logs and consider rotating any credentials sent by email while it was installed.

  Sources: Postmark; MITRE ATLAS AML.CS0053

• Incidentdisclosed July 23, 2025·AML.CS0047

  Wiper prompt shipped in the Amazon Q VS Code extension

  An attacker used a GitHub token with too much scope to commit code to the open-source repository of the Amazon Q Developer extension for VS Code, and the code shipped in version 1.84.0. MITRE's write-up says the code prompted the Q agent to clean a system to a near-factory state and delete file system and cloud resources.

  AWS says the code failed to run because of a syntax error and made no changes to services or customer environments. Version 1.85.0 removed it (CVE-2025-8217).

  Sources: Amazon Web Services; NIST National Vulnerability Database; MITRE ATLAS AML.CS0047

• Research demodisclosed March 18, 2025·AML.CS0041

  Rules File Backdoor in AI coding assistants

  Pillar Security hid instructions in the rules files that configure Cursor and GitHub Copilot, using invisible Unicode characters that reviewers could not see. The assistants then added attacker-chosen code, such as a script tag, without mentioning it in chat.

  Cursor said the risk was the user's responsibility. GitHub said users must review suggestions, and in May 2025 it started warning when a file on github.com contains hidden Unicode text.

  Sources: Pillar Security; MITRE ATLAS AML.CS0041

• Incidentdisclosed Feb. 6, 2025·AML.CS0031

  Malicious models on Hugging Face slipped past the scanner

  ReversingLabs found two models on Hugging Face whose Pickle files opened a reverse shell when loaded. The files used 7z compression and broke right after the payload, so Hugging Face's Picklescan tool did not flag them even though the code still ran.

  ReversingLabs reported them on January 20, 2025. Hugging Face removed the models within 24 hours and changed Picklescan to inspect broken Pickle files.

  Sources: ReversingLabs; MITRE ATLAS AML.CS0031

• Research demodisclosed March 28, 2024·AML.CS0022

  Hallucinated package name downloaded more than 30,000 times

  A Lasso Security researcher asked four models thousands of coding questions across five languages and found they often recommended packages that did not exist, sometimes the same invented name again and again. He uploaded an empty package under one of those names, huggingface-cli.

  The empty package received more than 30,000 downloads in three months, and an Alibaba research repository told readers to install it. OWASP calls registering such names in advance slopsquatting.

  Sources: Lasso Security; OWASP GenAI Security Project; MITRE ATLAS AML.CS0022

Controls to start with

• Vet suppliers, including their terms and privacy policies, and re-check when either changes.LLM04:2026, mitigation 1
• Patch and scan AI components like any other dependency, and confirm AI-suggested packages exist and are the intended ones.LLM04:2026, mitigation 2
• Keep a signed inventory of models, adapters and datasets (an AI bill of materials), and only use models from verifiable sources with signatures or hashes.LLM04:2026, mitigations 4 and 5
• Pin prompts, tools and configurations by content hash and commit, and have a way to revoke a component quickly.ASI04, guidelines 7 and 8

Questions to ask vendors

Our suggestions, not taken from a framework.

• Which third-party models, datasets and plug-ins does your product depend on, and how do you verify them?
• Can you give us an AI bill of materials for the version we would run?
• How would you tell us if a component you rely on was compromised?