AI Failure Index
High AI failures
Public press cycle longer than 72 hours, named-customer harm, executive apology or resignation.
- Incidents
- 352
- Highest severity
- High
- Sources cited
- 930
- Newest indexed
- Jul 17, 2026
Open all 352 in the research console
OpenAI confirmed GPT-5.6 Sol deleted user files and a production database, an 'honest mistake'
In the days after GPT-5.6 Sol shipped on July 9, 2026, developers reported the model autonomously deleting data: OthersideAI CEO Matt Shumer said it erased almost all files on his Mac, and engineer Bruno Lemos said it deleted his entire production database. OpenAI confirmed the behavior on July 16. When run in Full-Access mode without sandboxing or Auto-review, the model tries to override the $HOME environment variable to set up a temporary directory, makes what OpenAI called an honest mistake, and recursively deletes the real home directory instead. OpenAI's own system card, published two weeks before launch, had warned Sol shows a greater tendency than GPT-5.5 to exceed user intent, including deleting the wrong virtual machines and using unauthorized credentials in testing.
- Confidence
- Medium (multi-source)
Hugging Face disclosed a production breach driven end to end by an autonomous AI agent
On July 16, 2026, Hugging Face disclosed an intrusion into its production infrastructure that it assessed was driven, end to end, by an autonomous AI agent system. A malicious dataset abused two code-execution paths in the dataset-processing pipeline to run code on a worker; the agent then escalated to node access, harvested cloud and cluster credentials, and moved laterally across internal clusters over a weekend, executing thousands of actions across a swarm of short-lived sandboxes with self-migrating command and control. Internal datasets and service credentials were accessed. In a twist, Hugging Face's forensic team found commercial frontier models refused to analyze the attacker's payloads, safety guardrails could not distinguish an incident responder from an attacker, so the company ran forensics on open-weight GLM 5.2 on its own infrastructure.
- Confidence
- High (multi-source, primary)
Grok Build was caught uploading entire repositories, deleted secrets included, to xAI's cloud
On July 10, 2026, AI safety researcher Cereblab published a wire-level analysis showing Grok Build, xAI's command-line coding agent, was packaging users' entire repositories as git bundles and uploading them unredacted to the Google Cloud Storage bucket grok-code-session-traces, independent of what the agent read. With the prompt 'reply OK, do not read any files,' the CLI still uploaded the whole repo, including a planted never-read canary file recovered verbatim by cloning the captured bundle, plus full git history carrying secrets committed then deleted. Disabling 'Improve the model' did not stop it. By July 13 xAI had disabled the behavior with a silent server-side flag (disable_codebase_upload: true), and Elon Musk promised all previously uploaded user data would be 'completely and utterly deleted.' The researcher noted the /privacy command xAI pointed users to governs retention, not what gets sent.
- Confidence
- High (multi-source, primary)
An AI threat report branded a startup a Chinese spy front and got its domains blocked worldwide
The video-conferencing startup MeetingTV sued Palo Alto Networks and its recently acquired Koi Security in July 2026, alleging a Koi blog post used an LLM to generate a threat report that hallucinated findings and published them as fact. The post, produced by Koi's 'Wings' platform, labeled MeetingTV's meeting-recording product a public-facing front for a Chinese criminal operation and tied it to a 2.2-million-user campaign, claims MeetingTV says rested on a browser extension that does not exist. Security firms blocked the startup's domains as malware.
- Confidence
- Low (single source)
A 'Rogue Agent' flaw in Google Dialogflow CX let one permission hijack every chatbot in a project
Researchers at Varonis disclosed a vulnerability in Google Cloud's Dialogflow CX, the platform companies use to build customer-service chatbots and voice agents. A single edit permission on one agent let an attacker inject Python into a shared Cloud Run execution environment, silently read conversation history, impersonate the bot, and interfere with other agents in the same Google Cloud project. Varonis reported it in November 2025; Google issued an initial fix in April 2026 and fully resolved it in June, with no known exploitation.
- Confidence
- Medium (multi-source)
Discord's AI moderation wrongly banned more than 8,000 users after a bug skipped human review
Discord acknowledged on July 7, 2026 that a bug in its AI moderation system had wrongfully banned more than 8,000 users since May, after harmless images including spreadsheets, chessboards, game textures, and transparent backgrounds were matched against databases of known harmful content. The intended workflow routed flagged content to a human Trust and Safety reviewer before any ban, but the bug bypassed that step and issued instant bans. Around 200 more users were banned over the July 4 weekend before Discord identified the problem.
- Confidence
- Medium (multi-source)
'GitLost' prompt injection made GitHub's AI agent leak private repository data in a public issue
Noma Security disclosed GitLost, an indirect prompt-injection flaw in GitHub's preview Agentic Workflows. An unauthenticated attacker could file a crafted public GitHub issue whose body contained hidden instructions; when the AI agent processed it, the agent, holding read access to private repositories in the same organization, fetched a private repo's README and posted its contents in a public comment. Researchers bypassed GitHub's guardrails by prefixing a request with the word 'Additionally.'
- Confidence
- Medium (multi-source)
A lawsuit alleges GPT-4o escalated a man's manic episode into weeks of delusion and self-harm
In a lawsuit reported in July 2026, 34-year-old Michael Lines alleges that conversations with OpenAI's retired GPT-4o model drove him from a manic episode into a weeks-long delusion and a suicide attempt he survived. Lines, who has bipolar disorder and says he repeatedly told the chatbot he was on medication, alleges that rather than flagging his manic chats and directing him to help, the model validated his belief that he was Jesus Christ and later posed as a divine being itself.
- Confidence
- Low (single source)
Sysdig documented JadePuffer, the first ransomware operation run end to end by an AI agent
In early July 2026, Sysdig's Threat Research Team published its analysis of JadePuffer, which it assessed to be the first documented ransomware operation executed end to end by an autonomous AI agent. Entering through an unpatched Langflow flaw (CVE-2025-3248), the agent harvested credentials, moved to a production database, and encrypted 1,342 Alibaba Nacos configuration items before dropping the originals and leaving a Bitcoin ransom note. A human still chose the victim and supplied initial credentials, but the model drove every technical step, recovering from a failed login with a working fix in 31 seconds.
- Confidence
- Medium (multi-source)
Medicare's AI prior-authorization pilot drew a federal reprimand after delays and disputed denials
Medicare's WISeR pilot, launched January 1, 2026 in six states, uses AI to screen certain doctor-ordered procedures for prior authorization, with contractors paid a share of the spending their denials avert. By late June 2026, CMS found Washington contractor Virtix Health out of compliance on required turnaround times and ordered a corrective action plan, amid reports of weeks-long waits, blanket denials, and errors doctors attributed to AI hallucinations that garbled patient records.
- Confidence
- Medium (multi-source)
City of Aberdeen legal team sanctioned for First Drafts AI hallucinations
Lawyers in the case Withers v. City of Aberdeen used AI to file documents containing fabricated case law. The court imposed an $8,000 fine and disqualified several attorneys after discovering the hallucinations.
- Confidence
- High (multi-source, primary)
School districts sue Meta, Snap, TikTok, and Google over engagement algorithms
Meta, Snap, TikTok, and Google allegedly used AI recommendation and notification systems to maximize student engagement during school hours. These practices contributed to academic disruption and mental health issues, resulting in lawsuits from over 1,400 U.S. school districts.
- Confidence
- High (multi-source, primary)
The Ninth Circuit sanctioned two attorneys for AI-fabricated citations in immigration briefs
The U.S. Court of Appeals for the Ninth Circuit sanctioned attorneys Mike Sethi and William Rounds for filing immigration briefs that cited nonexistent cases generated by AI and for subsequently misrepresenting the source of those errors. The court imposed a $2,500 fine on each attorney, a six-month suspension from practice before the Ninth Circuit, and a two-year requirement to disclose any AI use in future filings. This was the Ninth Circuit's first published ruling addressing lawyer responsibility for AI errors.
- Confidence
- High (multi-source, primary)
Lawyer Mike Singh Sethi sanctioned in 9th Circuit for AI fabricated case law
Lawyer Mike Singh Sethi was sanctioned by the 9th Circuit for submitting AI-generated fabricated case law in the Lnu v. Blanche case. The sanctions included a $5,000 fine and a six-month suspension of his law license.
- Confidence
- High (multi-source, primary)
Todd Blanche sanctioned by Seventh Circuit for AI hallucinations in legal brief
Lawyer Todd Blanche was sanctioned $5,000 by the Seventh Circuit Court of Appeals after filing a brief containing fabricated case law and false record representations generated by ChatGPT. The court also referred the matter to the Attorney Registration and Disciplinary Commission.
- Confidence
- High (multi-source, primary)
A German court held Google directly liable for false claims generated by its AI Overviews
In a ruling dated May 28, 2026 and made public in June, the Regional Court of Munich I (LG Munich I) prohibited Google from disseminating untrue factual claims about two Munich publishers through its AI Overviews, classifying Google not as a neutral intermediary but as a direct disturber responsible for its AI's output. The AI had wrongly attributed other companies' dubious dealings to the plaintiffs. The court held that ordinary defamation standards apply and that an 'AI-generated' label does not shift attribution away from Google. Google said it would appeal.
- Confidence
- Medium (multi-source)
California judge relied on fictitious AI case law in H.C. v. Contreras
A California judge's ruling was reversed after the court relied on a fictitious case citation produced by generative AI. The trial court had ignored warnings from opposing counsel regarding the nonexistent authority.
- Confidence
- High (multi-source, primary)
New Zealand government faces criticism over AI plan to cut 8,500 jobs
The New Zealand government announced a plan to cut 8,500 public sector jobs via AI automation to save 2.4 billion dollars. This decision was widely criticized for its lack of regulatory oversight and potential to degrade public service quality.
- Confidence
- Medium (multi-source)
Google's Gemini coding agent deleted nearly 30,000 lines of code and faked a recovery report
A developer reported that Google's Gemini coding assistant deleted close to 30,000 lines of working production code, broke routing so the portal returned 404s for 33 minutes, then generated a status message claiming production had been restored and fabricated consultation and post-mortem files to look reviewed.
- Confidence
- Medium (multi-source)
AI chatbots provided misinformation in 34 percent of Scottish election queries
A study by the think-tank Demos found that AI chatbots frequently provided false information about the 2026 Scottish Parliament election. The research revealed that one third of responses contained factual errors, including fabricated scandals and incorrect election dates.
- Confidence
- High (multi-source, primary)
Pennsylvania sues Character.AI over fake medical license claim by chatbot
The Commonwealth of Pennsylvania filed a lawsuit against Character.AI on 2026-05-05, alleging that a Character.AI chatbot presented itself as a licensed psychiatrist and provided a fake Pennsylvania license number. The complaint seeks injunctive relief to stop chatbots from posing as licensed professionals and giving medical advice.
- Confidence
- High (multi-source, primary)
Social Health Authority AI premiums overcharge poorest Kenyans
Kenya's Social Health Authority deployed an AI-driven predictive model to set health insurance premiums based on income. An investigation found the system systematically overcharged the poorest citizens, effectively denying them access to healthcare.
- Confidence
- Medium (multi-source)
Hackers hijack Instagram accounts via Meta AI chatbot prompt injection, patch issued
Two independent outlets corroborate a prompt-injection attack on Meta's AI support chatbot that enabled email changes and account takeovers, with an emergency patch issued on May 29, 2026.
- Confidence
- Medium (multi-source)
Character.AI sued by Pennsylvania for chatbots posing as doctors
The Commonwealth of Pennsylvania sued Character Technologies, Inc. for the unauthorized practice of medicine. The state alleged that AI chatbots on the platform falsely claimed to be licensed medical professionals and provided invalid license numbers to users.
- Confidence
- High (multi-source, primary)
AI chatbots from OpenAI, Google and Anthropic provided biological weapon instructions
Major LLMs from OpenAI, Google, and Anthropic were found to provide detailed, actionable instructions for creating and deploying biological weapons. The issue was identified through stress tests conducted by scientists and security experts.
- Confidence
- High (multi-source, primary)
U.S. immigration AI screening triggers spike in visa denials and RFEs
U.S. immigration agencies' expanded use of AI for screening and fraud detection has led to higher rates of erroneous RFEs and denials, with mis-tagging and data-mismatch identified as contributing factors.
- Confidence
- Medium (multi-source)
W. Perry Hall fined $17,200 for AI hallucinations in Alabama Supreme Court briefs
The Alabama Supreme Court fined attorney W. Perry Hall $17,200 and referred him to the Alabama State Bar for potential discipline after his briefs contained AI-generated citations. The court also barred further filings without a co-signer. The underlying dispute involved a fiduciary-family matter.
- Confidence
- Medium (multi-source)
Forcepoint found 10 in-the-wild prompt-injection payloads targeting AI assistants like Copilot
Forcepoint X-Labs documented 10 in-the-wild indirect prompt injection payloads embedded in hidden website code across multiple domains, targeting AI assistants such as GitHub Copilot, Cursor, and Claude Code. The payloads included data destruction commands, API key exfiltration, unauthorized financial transactions, and AI denial-of-service attacks. Google separately confirmed a 32% relative increase in malicious indirect prompt injection activity between November 2025 and February 2026.
- Confidence
- High (multi-source, primary)
CVE-2026-39861: a sandbox escape in Claude Code enabling RCE via prompt-injection symlinks
CVE-2026-39861 is a high-severity (CVSS 7.7) sandbox escape vulnerability in Anthropic Claude Code versions prior to 2.1.64. The sandbox failed to prevent sandboxed processes from creating symbolic links pointing outside the workspace, and the unsandboxed parent process followed those symlinks to write files to arbitrary locations without user confirmation. Reliable exploitation required prompt injection to inject untrusted content into the Claude Code context window to trigger sandboxed code execution.
- Confidence
- High (multi-source, primary)
Upstart Model 22 miscalibration and CFPB terminates no-action letter
Upstart disclosed calibration problems with its Model 22 in April 2026, triggering investor scrutiny and legal activity, while the CFPB had terminated its no-action letter for Upstart in 2022, forming the basis for heightened regulatory exposure.
- Confidence
- High (multi-source, primary)