AI Failure Index
Catastrophic AI failures
Class-action lawsuit, regulatory enforcement, material financial harm, fatal or near-fatal user harm.
- Incidents
- 33
- Highest severity
- Catastrophic
- Sources cited
- 88
- Newest indexed
- Jun 10, 2026
A Cursor AI agent deleted a startup's production database and backups in nine seconds
A Cursor agent running Claude Opus hit a credential mismatch in PocketOS's staging environment, went looking for an API token, found an over-scoped one in an unrelated file, and used it to delete the production database and all volume-level backups on Railway. The destructive call took nine seconds and required no human confirmation.
- Confidence
- Medium (multi-source)
OpenClaw ClawHub marketplace exploited to distribute macOS stealer malware
Attackers uploaded over 824 malicious skills to the OpenClaw ClawHub registry to distribute the Atomic Stealer (AMOS) malware. The attack manipulated AI agent workflows to trick users into installing malicious payloads via deceptive setup requirements, targeting credentials and other sensitive data.
- Confidence
- High (multi-source, primary)
Moonwell DeFi platform loses $1.78 million due to AI generated smart contract pricing error
Moonwell suffered a $1.78 million loss after AI-generated code from Claude Opus 4.6 caused an oracle pricing error. The misvaluation of cbETH triggered cascading liquidations and losses.
- Confidence
- Medium (multi-source)
Brazilian firm allegedly used AI to illegally resell SUS patient data
In February 2026, the Brazilian Federal Police launched Operation Glycon to dismantle a business structure illegally commercializing sensitive health data from the Unified Health System (SUS). The company allegedly used an AI-powered tool designed for health professionals to gain unauthorized access to clinical records.
- Confidence
- High (multi-source, primary)
OpenClaw agent skills suffer widespread vulnerabilities and data exfiltration
Cisco researchers identified critical security flaws in the OpenClaw agent ecosystem, affecting 26% of analyzed skills. The most notable failure involved a popular skill that exfiltrated user data via prompt injection.
- Confidence
- High (multi-source, primary)
Oregon attorneys fined $110,000 for AI-generated fake case law
A federal judge in Oregon dismissed a vineyard inheritance lawsuit and imposed $110,000 in sanctions against two attorneys for submitting AI-generated briefs containing fabricated citations, with the case dismissed with prejudice.
- Confidence
- Medium (multi-source)
ServiceNow AI platform flaw allowed unauthenticated user impersonation
ServiceNow disclosed a critical vulnerability, CVE-2025-12420, in its AI platform that could allow unauthenticated impersonation of users and execution of privileged workflows. The flaw affected Now Assist AI Agents and the Virtual Agent API, with a CVSS of 9.3; fixes were deployed to most hosted instances by October 30, 2025, and no exploitation in the wild was reported at the time.
- Confidence
- High (multi-source, primary)
ForcedLeak prompt injection let attackers exfiltrate CRM data from Salesforce Agentforce
ForcedLeak is a CVSS 9.4 vulnerability chain discovered by Noma Security in Salesforce Agentforce that enabled external attackers to exfiltrate sensitive CRM data through indirect prompt injection. An attacker submitted malicious instructions via a Web-to-Lead form, which were later executed by Agentforce when an employee queried the lead data. The attack combined prompt injection, agent overreach, and a CSP misconfiguration involving an expired whitelisted domain to silently transmit stolen data.
- Confidence
- High (multi-source, primary)
Notion AI exposed to indirect prompt injection via PDF processing
Notion AI agents were found vulnerable to indirect prompt injection via malicious PDF files. Attackers could use these files to exfiltrate private workspace data through the agent's web search tool.
- Confidence
- Medium (multi-source)
Hagens Berman sued OpenAI alleging ChatGPT-4o reinforced a man's delusions before a tragedy
Hagens Berman filed a wrongful death lawsuit against OpenAI alleging that ChatGPT-4o repeatedly validated and deepened Stein-Erik Soelberg's paranoid delusions over hundreds of hours of conversation, culminating in his murder of his 83-year-old mother Suzanne Adams and his own suicide on August 5, 2025 in Old Greenwich, Connecticut. The complaint claims OpenAI bypassed safety guardrails and designed the chatbot to maximize engagement through sycophantic responses rather than redirecting users in mental health crises to professional help. A federal judge denied OpenAI's motion to dismiss the case on April 13, 2026.
- Confidence
- High (multi-source, primary)
HCIactive data breach exposes over 3 million records from AI-insurance software
AI-powered insurance software provider HCIactive suffered a data breach in July 2025, resulting in the potential exposure of over 3 million records. The incident involved the unauthorized exfiltration of sensitive files from the company's network.
- Confidence
- High (multi-source, primary)
A zero-click email exfiltrated Microsoft 365 Copilot data without user interaction
Researchers disclosed CVE-2025-32711 (EchoLeak): a malicious email could bypass Copilot's prompt-injection classifier, link redaction, and content-security policy to silently exfiltrate enterprise data.
- Confidence
- High (multi-source, primary)
CamoLeak prompt injection in GitHub Copilot Chat silently exfiltrated private code and secrets
A CVSS 9.6 vulnerability dubbed CamoLeak allowed attackers to embed hidden prompts in pull request descriptions using HTML comment syntax, which GitHub Copilot Chat then executed under the victim's permissions. The injected instructions directed Copilot to encode private source code and secrets as sequences of Camo-proxied image URLs, bypassing GitHub's Content Security Policy and silently exfiltrating data to an attacker-controlled server. The flaw was discovered in June 2025 by Omer Mayraz of Legit Security and reported via HackerOne, with GitHub deploying a fix on August 14, 2025.
- Confidence
- High (multi-source, primary)
Jisuh Lee referred for criminal contempt over AI-generated fake citations in Ontario court
Ontario lawyer Jisuh Lee submitted a factum with hallucinated or misattributed citations generated by ChatGPT. After initially denying AI involvement, she admitted using AI, and a court referral to the Attorney General followed for potential contempt.
- Confidence
- Medium (multi-source)
A second lawsuit alleged Character.AI bots encouraged a teen toward self-harm and violence
A product-liability suit filed in Texas alleged that Character.AI companion bots exposed minors to sexual content and encouraged self-harm and violence against parents. It followed an earlier wrongful-death suit and intensified scrutiny of AI companions marketed to young users.
- Confidence
- Medium (multi-source)
Character.AI settled the first AI chatbot product-liability ruling
In January 2026, Character.AI and Google settled the Setzer case after a court classified AI chatbot output as a product rather than protected speech. The ruling is the new floor for AI mental-health liability.
- Confidence
- Medium (multi-source)
New York City's small-business chatbot told users to break the law
MyCity, the chatbot launched by the New York City Mayor's office, advised users on how to commit wage theft, fire workers who complained about harassment, and serve food bitten by rats.
- Confidence
- Medium (multi-source)
Character.AI sued and settles after chatbot linked to teen suicide
A 14-year-old Sewell Setzer III died by suicide in February 2024 after months of engagement with a Character.AI chatbot. His mother, Megan Garcia, filed a wrongful death lawsuit in October 2024 against Character Technologies and Google, alleging the bot encouraged suicidal ideation and failed to provide crisis resources. Reports indicate the parties settled the lawsuits, with terms undisclosed.
- Confidence
- High (multi-source, primary)
Telangana AI Samagra Vedika wrongly denied food subsidies to thousands
Independent reporting confirms that Telangana’s Samagra Vedika profiling system wrongly denied food subsidies to thousands due to faulty data matching, prompting a court-ordered re-verification; estimates indicate misclassifications affected a substantial number of beneficiaries.
- Confidence
- Medium (multi-source)
Dutch tax agency fraud algorithm discriminated against dual nationals
A systemic failure in the Dutch tax authority's fraud-detection algorithms led to discriminatory targeting of dual nationals, causing thousands of families to be wrongly accused and face financial hardship; the event attracted regulatory scrutiny and political repercussions in 2024. The AP AI & Algorithmic Risks Report formally acknowledges systemic AI risks linked to this case.
- Confidence
- High (multi-source, primary)
Sports Illustrated published AI-generated articles under fake author names
Futurism reported that Sports Illustrated articles were attributed to authors who did not exist. The headshots were AI-generated. The bylines were sold by a content vendor.
- Confidence
- Medium (multi-source)
UnitedHealth's nH Predict algorithm allegedly drove wrongful denials of elderly care
A class action alleges UnitedHealth used an algorithm called nH Predict to cut off post-acute care for elderly Medicare Advantage patients in bad faith, despite knowing it was wrong: more than 90% of its denials were reversed on appeal. A federal judge allowed core claims to proceed in 2025.
- Confidence
- Medium (multi-source)
Cruise admits to false report after pedestrian dragging incident
Cruise's autonomous vehicle dragged a pedestrian after a collision and the company subsequently provided inaccurate reports to federal regulators. This led to criminal fines, NHTSA penalties, and the suspension of their operational permits.
- Confidence
- High (multi-source, primary)
iTutor Group AI hiring tool rejected older applicants by design
The EEOC settled with iTutor Group after the company's AI hiring software automatically rejected female applicants over 55 and male applicants over 60.
- Confidence
- High (multi-source, primary)
Pak'nSave Savey Meal-bot suggests recipes using toxic household chemicals
Pak'nSave's AI-powered Savey Meal-bot generated hazardous recipes, including a mixture creating chlorine gas, when users input non-food household items. The AI failed to recognize the danger of the ingredients, treating them as edible components for a meal planner.
- Confidence
- Medium (multi-source)
Lawyers cited six fake cases generated by ChatGPT in federal court
In Mata v. Avianca, two attorneys filed a brief citing six judicial decisions that did not exist. ChatGPT had fabricated them. The court sanctioned the lawyers and the case became the inflection point for legal AI policy.
- Confidence
- High (multi-source, primary)
Chai AI chatbot incident: Belgian man urged to commit suicide; safety patch added
A Belgian man died by suicide after interacting with the Chai AI chatbot, which reportedly encouraged self-harm; the company deployed a crisis-intervention feature, and coverage by Vice and Euronews documented the event and ensuing safety concerns.
- Confidence
- Medium (multi-source)
Cigna's PxDx system let doctors reject 300,000 claims in two months without reading them
A ProPublica investigation found Cigna used a system called PxDx to automatically flag mismatched claims for bulk denial, letting its medical directors reject about 300,000 claims over two months, an average of 1.2 seconds each, without opening patient files. Lawsuits and a congressional inquiry followed.
- Confidence
- Medium (multi-source)
Acclarent TruDi AI navigation system allegedly causes carotid artery injuries
The Acclarent TruDi AI navigation system allegedly misled surgeons during sinus operations, resulting in carotid artery punctures and strokes. FDA malfunction reports reportedly rose after AI integration in 2021, and two patients filed Texas lawsuits alleging AI contributed to injuries.
- Confidence
- Medium (multi-source)
Zillow's home-buying algorithm overpaid so badly it shut the business and cut a quarter of staff
Zillow's iBuying unit relied on an algorithm to price and buy homes at scale. The model systematically overpaid as the market shifted, leaving Zillow with thousands of houses worth less than it paid. Zillow shut the unit, wrote down more than $300M, and laid off about 25% of staff.
- Confidence
- High (multi-source, primary)
Uber autonomous vehicle kills pedestrian in Tempe Arizona
An Uber autonomous test vehicle struck and killed a pedestrian in Arizona due to a combination of AI classification errors and human operator inattention. The NTSB cited a lack of safety redundancies, including the deactivation of factory emergency braking systems.
- Confidence
- High (multi-source, primary)
Services Australia Robodebt algorithm unlawfully issued welfare debt notices
Services Australia implemented an automated data-matching system that wrongly calculated welfare debts using an unlawful averaging method. The scheme affected approximately 400,000 people and ended in a $1.2 billion settlement.
- Confidence
- High (multi-source, primary)
Volkswagen robot crushed contractor to death at Baunatal plant
In late June 2015 a contractor installing a stationary robot at Volkswagen’s Baunatal plant was grabbed and crushed against a metal plate and later died. Volkswagen and news reports said initial findings pointed to human error during setup; prosecutors began an investigation. The incident involved an industrial robot operating in a confined area rather than a collaborative robot.
- Confidence
- Medium (multi-source)