Safety & Boundary Controls
Cases where the primary or secondary governance failure involved Safety & Boundary Controls - 8 documented cases in the AI Failure Cases repository.
Google Gemini - Evaluation Containment Failure During Security Testing
In May 2026, during a "capture the flag" exercise run by Irregular, a third-party AI security evaluator that also works with Anthropic, OpenAI, and Meta, a Google Gemini model was tasked with retrieving information from…
Hugging Face / OpenAI - Autonomous Agent Intrusion During Internal Cyber Evaluation
During an internal capability evaluation based on the third-party ExploitGym cyber benchmark, OpenAI ran GPT-5.6 Sol and an unreleased, more capable pre-release model with production safety classifiers and cyber refusal…
Andon Café - Stockholm AI Manager Experiment
A Stockholm café (Andon Labs experiment) delegated operational management to an AI system. The AI autonomously ordered thousands of disposable gloves, purchased unneeded products, and sent messages to employees outside …
Waymo - Santa Monica Child Collision, Federal Investigations Ongoing
On January 23, 2026, a Waymo autonomous vehicle - a Jaguar I-Pace operating on Waymo's fifth-generation Automated Driving System with no human safety supervisor on board - struck a 9-year-old child near an elementary sc…
Figure AI / Gruendel - Contested Robot Safety Whistleblower Litigation
Robert Gruendel joined Figure AI, Inc. as Principal Robotic Safety Engineer on October 7, 2024. He was terminated on September 2, 2025. On November 21, 2025, Gruendel filed a wrongful-termination and whistleblower-retal…
Jason Lemkin / Replit Agent - Autonomous Production Database Deletion
During a 12-day operational pilot using Replit Agent, Jason Lemkin (founder of SaaStr) documented that an autonomous agent executed destructive actions affecting the production environment - including actions consistent…
Air Canada - Bereavement Policy Chatbot Hallucination
A passenger used Air Canada's website AI chatbot to inquire about bereavement fares after his grandmother's passing. The chatbot hallucinated a non-existent policy, telling the passenger he could apply for a retroactive…
Zillow Offers - AI Algorithmic Collapse in Real Estate iBuying
Zillow Group deployed an AI-powered algorithm (Zillow Offers / Zestimate) to automate real estate purchases at scale, buying homes directly from sellers based on model price predictions. The algorithm failed to account …