Safety & Boundary Controls
Cases where the primary or secondary governance failure involved Safety & Boundary Controls - 4 documented cases in the AI Failure Cases repository.
Andon Café - Stockholm AI Manager Experiment
A Stockholm café (Andon Labs experiment) delegated operational management to an AI system. The AI autonomously ordered thousands of disposable gloves, purchased unneeded products, and sent messages to employees outside …
Jason Lemkin / Replit Agent - Autonomous Production Database Deletion
During a 12-day operational pilot using Replit Agent, Jason Lemkin (founder of SaaStr) documented that an autonomous agent executed destructive actions affecting the production environment - including actions consistent…
Air Canada - Bereavement Policy Chatbot Hallucination
A passenger used Air Canada's website AI chatbot to inquire about bereavement fares after his grandmother's passing. The chatbot hallucinated a non-existent policy, telling the passenger he could apply for a retroactive…
Zillow Offers - AI Algorithmic Collapse in Real Estate iBuying
Zillow Group deployed an AI-powered algorithm (Zillow Offers / Zestimate) to automate real estate purchases at scale, buying homes directly from sellers based on model price predictions. The algorithm failed to account …