This Week in AI Security - 17th September 2026
A shorter episode this week, recorded from the sidelines of Mind The Sec, soon to be Black Hat Latin America. The headline is distillation. Two separate disclosures, one from Anthropic and a joint advisory from the FBI, CISA, and the NSA, describe Chinese-linked labs siphoning the reasoning out of rival models at industrial scale, with Alibaba alone attributed 151 million exchanges against Claude in three months. Around that sit the usual escalations: AI-run exploit foundries, malware that rebuilds itself to dodge antivirus, model keys stolen through gateways, a covert attack that hides in plain English, and one runaway agent that burned $50,000 in a couple of hours. Note: there is no episode next week. We will be back in a couple of weeks with a longer edition covering both weeks. Key Discussion Points Two disclosures detail industrial-scale distillation campaigns from Chinese-linked labs. A joint FBI/CISA/NSA advisory names six companies going back to 2024, and Anthropic reports its largest-ever measured attack: 151 million exchanges attributed to Alibaba alone, peaking at 3 million a day across 3,500 accounts. No exploits, just reframing tactics that coax reasoning traces out of the model. Anthropic's latest misuse report covers an autonomous exploit foundry that produced 12 potential zero-days in a month across 50 targets, plus Russian-aligned actors whose agents rebuilt malware to evade antivirus and dumped 2,100 cloud access tokens across 40 tenants in 34 hours. Eval sandboxes and LiteLLM gateways are the new target for model-key theft. One actor planted instructions in a vendor's eval sandbox to force key handover, then hit 30 AI companies in four days. Separately, 10% of internet-exposed LiteLLM instances still accept the default admin key. A new CVE in IBM's Context Forge MCP gateway allows OS command injection, with a second CVE around default credentials. MCP gateways are becoming a choke point. Check Point's "Puzzle Mask" shows that plain, well-crafted English can smuggle a hidden instruction past fast gatekeeper models. It hit a 100% bypass rate against four gatekeepers with 23 prompts, exploiting the fast-gatekeeper, strong-target design many production systems rely on. Anthropic CEO Dario Amodei published a "We Must Pace the Frontier" paper calling to slow AI capability improvement, against a backdrop of no real regulation and a narrow antitrust waiver that lets labs coordinate on safety. A late-breaking story: one runaway agent running a business-automation process racked up a $50,000 bill in a couple of hours. Most organizations have neither cost guardrails nor full telemetry across hybrid cloud to catch it. Episode Links https://techcrunch.com/2026/09/10/anthropic-details-distillation-campaigns-from-alibaba-moonshot-ai-and-deepseek/ https://media.defense.gov/2026/Sep/08/2003992823/-1/-1/1/CSA_CHINA_BASED_AI_COMPANIES_MALICIOUS_DISTILLATION_AGAINST_US.PDF https://www.anthropic.com/threat-intelligence-report-september-2026 https://www.beri.net/article/anthropic-threat-report-eval-sandbox-litellm-prompt-injection-api-key-theft https://vulners.com/pypa/PYPA:PYSEC-2026-3862 https://research.checkpoint.com/2026/puzzlemask-abusing-plain-prose-as-a-covert-ai-attack-vector/ https://techcrunch.com/2026/09/12/anthropic-ceo-outlines-plan-to-pace-the-frontier/