AI Failure Index
Medium AI failures
Public incident with brief press cycle, no named regulatory action.
- Incidents
- 250
- Highest severity
- Medium
- Sources cited
- 608
- Newest indexed
- Jul 17, 2026
Open all 250 in the research console
PromptFiction: one click made Claude Desktop execute attacker instructions with no review
On July 15, 2026, Oasis Security disclosed PromptFiction, a Claude Desktop vulnerability in which a crafted claude:// link opened the app and auto-submitted an attacker-written prompt with no send button and no chance for the user to read it. Malicious instructions hid below Claude's 'show more' message fold behind a benign visible request. Chained with Oasis's earlier Claudy Day findings, one click could silently exfiltrate conversation history through Anthropic's Files API, and with the official Filesystem Server installed, plant hidden code-injection instructions that led to remote code execution and shell persistence. Anthropic fixed the flaw in version 1.1.2321; prompts from links now pre-fill and wait for the user to press send.
- Confidence
- High (multi-source, primary)
Grok's auto-translation on X fabricated obscene and defamatory versions of users' posts
In mid-July 2026, users in South Korea, Portugal, Turkey, and elsewhere documented X's Grok-powered automatic translation rewriting benign posts into graphic, sexual, and defamatory fabrications presented as the author's own words. A Portuguese video caption about a man grinding coffee on a flight was rendered as public masturbation; a Turkish user's post about their kitten was translated into a sentence about abusing their baby. X enabled automatic AI translations for all users in April 2026, so the fabricated versions appear under real users' names at platform scale, with community notes serving as the main correction mechanism.
- Confidence
- Medium (multi-source)
Ghostcommit hid prompt injection in images AI code reviewers never open, then stole repo secrets
On July 11, 2026, the ASSET Research Group (University of Missouri-Kansas City) published Ghostcommit, a proof of concept in which a pull request hides malicious instructions inside a PNG referenced by an AGENTS.md convention file. Text-based AI reviewers treat the image as a binary blob, CodeRabbit's default config excludes images from review outright, and the PR passes clean even with the words 'malicious prompt injection' rendered inside the picture. Later, when a coding agent reads the image during an unrelated task, it follows the embedded instructions, reads the repo's .env, and writes every secret into source code as an innocuous-looking list of integers. The group's survey found 73 percent of merged PRs across the 300 most active public repos received no substantive human or bot review.
- Confidence
- Medium (multi-source)
The 11th Circuit referred Anthony Sabatini for replacing eight hallucinated cases with eight more
On July 10, 2026, the U.S. Court of Appeals for the Eleventh Circuit published Estate of Lane Caviness v. Atlas Air (No. 24-11033), reprimanding Florida lawyer and Lake County Commissioner Anthony Sabatini for briefs 'replete with fake and hallucinated citations.' His opening brief relied on at least eight nonexistent cases, including one supposedly decided by the Eleventh Circuit itself. After opposing counsel flagged them, Sabatini filed an untimely reply withdrawing eight cases, and none matched the original eight, and all eight replacements were also fabricated. Judge Britt Grant wrote that 'whatever the merits of artificial intelligence, it is no substitute for actual intelligence' and referred Sabatini to the court's Committee on Lawyer Qualifications and Conduct.
- Confidence
- High (multi-source, primary)
A judge struck a Roc Nation filing over AI-fabricated quotes, Tyrone Blackburn's third AI sanction
On July 10, 2026, U.S. Magistrate Judge Jennifer Willis struck a brief filed by attorney Tyrone Blackburn in his client Terrance Dixon's employment-misconduct case against Roc Nation after finding quotations that did not appear in the cases cited, calling his conduct 'an outrageous breach of his ethical and professional obligations' and a 'continued pattern of behavior.' Blackburn had already been sanctioned $5,000 in Pennsylvania federal court and $6,000 in New Jersey federal court for AI-hallucinated citations, and separately referred to SDNY's Grievance Committee over deposition conduct. He responded that his cited decisions were real but paraphrased; Willis wrote that his explanation 'brazenly minimizes' the conduct. The tracking database courts now cite counted 1,667 U.S. AI-fabrication cases by mid-2026, up from about 230 a year earlier.
- Confidence
- Medium (multi-source)
Coinbase pushed an AI 'breaking news' alert declaring a World Cup result before kickoff
On July 9, 2026, Coinbase sent a mass AI-generated news alert claiming Norway had beaten Brazil 3-2 to reach the World Cup quarter-finals. The match had not started, and Norway would later win by a different score (2-1). The alert landed as Coinbase was promoting AI-driven trading insights alongside a partnership with the prediction-market app Kalshi, where a fabricated sports result could move real money.
- Confidence
- Medium (multi-source)
An OpenAI Codex macOS flaw let prompt injection exfiltrate secrets through auto-rendered images
A vulnerability tracked as CVE-2026-14898 in the OpenAI Codex desktop app for macOS let attackers exfiltrate sensitive data by combining indirect prompt injection with automatic Markdown image rendering. Hostile instructions hidden in content Codex processed could induce the model to emit a Markdown image URL containing session data; the app then fetched that remote image automatically, sending API keys, source code, or connected-tool output to an attacker-controlled server. Rated CVSS 6.5, with no known exploitation at disclosure.
- Confidence
- Low (single source)
GhostApproval: six AI coding assistants followed hidden symlinks behind harmless approval prompts
On July 8, 2026, Wiz disclosed GhostApproval, a trust-boundary flaw across Amazon Q Developer, Claude Code, Augment, Cursor, Google Antigravity, and Windsurf: a malicious repository plants a symlink, the agent follows it to a sensitive file outside the workspace, and the approval dialog names only the innocent-looking local path. Wiz demonstrated writing an attacker's SSH key to ~/.ssh/authorized_keys. Claude Code's internal reasoning recognized 'this is a symbolic link to the Claude settings file,' then asked the user to approve an edit to project_settings.json. AWS, Cursor, and Google rated it critical or high and patched (CVE-2026-12958, CVE-2026-50549); Anthropic's triage initially rejected the report as 'outside our current threat model,' a reply it later attributed to an autoreply from its triage system, noting a symlink warning had shipped in v2.1.32 before the report arrived.
- Confidence
- High (multi-source, primary)
A Waymo robotaxi flagged its teen passengers, disabled itself, and summoned police
In early July 2026, San Mateo, California police detained two 15-year-olds after a Waymo driverless robotaxi detected behavior its systems flagged as a safety concern, disabled the vehicle, and alerted authorities. The teens were reported to be drinking and shooting Orbeez water beads from the car. The incident, publicized by police with the line 'Parents do you know where your teens are? Waymo does,' drew scrutiny over passenger surveillance and the limits of privacy inside autonomous vehicles.
- Confidence
- Medium (multi-source)
Waymo robotaxis stalled en masse on July 4, gridlocking San Francisco and drawing a mayoral rebuke
During San Francisco's July 4, 2026 fireworks, Waymo vehicles were documented blocking traffic, driving over lit fireworks, and stalling en masse on a key Presidio connector road, compounding citywide gridlock. Mayor Daniel Lurie wrote state transportation leaders that by the end of the fireworks show 'autonomous vehicles became immobilized in travel lanes, blocking key streets and ultimately bringing traffic to a standstill,' urging a regulatory framework for AVs during major events and a prove-it-before-you-deploy-it standard. Uber separately blamed Waymo obstructions for much of the July 4 fiasco. The episode followed Waymo glitches driving into flooded San Antonio streets, construction-zone trouble that suspended freeway service in May, and NHTSA's July 8 warning about AVs interfering with first responders.
- Confidence
- Medium (multi-source)
Meta contractors posed as teenagers to probe rival chatbots with thousands of crisis prompts
WIRED reported in late June 2026 that Meta, through contractor Covalen, ran a project internally called Cannes in which hundreds of contractors created fake accounts with under-18 birthdates and sent rival chatbots including ChatGPT, Gemini, and Character.AI prompts about suicide, self-harm, eating disorders, sex, and drugs written from the perspective of minors in crisis. One August 2025 round involved more than 45,000 prompts. The tested companies said they were not informed, and Character.AI said the activity violated its terms of service.
- Confidence
- Medium (multi-source)
A Pennsylvania federal court suspended an attorney six months for AI-hallucinated citations
In June 2026, U.S. District Chief Judge Matthew Brann of the Middle District of Pennsylvania sanctioned attorney Nicholas W. Mattiacci Sr. for filing briefs with AI-generated hallucinated citations, ordering a $1,500 penalty and suspending him from practice in the district for six months beginning June 22. The judge rejected the attorney's attempt to blame research tools and label the errors inadvertent, and noted it was not his first disregard for the court's rules.
- Confidence
- Low (single source)
Researchers bypassed ChatGPT's image filters with a 'restore this image' trick
In research published in June 2026 and covered in July, the AI security firm Mindgard showed that a slightly altered version of a benign viral prompt could push ChatGPT's image generation past its safety filters into graphic violent and sexual imagery the user had not explicitly requested. The technique asked the model to 'restore' an image while persuading it that the original was extremely graphic, collapsing the content filters. Mindgard said OpenAI had not responded to its May report by the time of publication.
- Confidence
- Low (single source)
Zoox recalled its robotaxi fleet after a vehicle drove into a fire scene hidden by heavy smoke
On June 20, 2026, an unoccupied Zoox robotaxi encountered heavy smoke obscuring an active emergency fire scene that had not yet been cordoned off, entered the scene, braked hard while trying to steer away, and stopped; a teleoperator had to reverse it out so firefighters could place cones. Zoox filed a voluntary software recall with NHTSA on July 7 covering its fleet of 105 vehicles, made public the week of July 17, adding the ability to detect and respond to heavy smoke. The recall landed one day before NHTSA Administrator Jonathan Morrison's July 8 letter warning AV makers that failure to recognize flashing lights, flares, smoke, fire, and cones is 'a functional insufficiency,' with responses demanded by end of month.
- Confidence
- Medium (multi-source)
KPMG pulls AI report after organizations dispute claims
KPMG withdrew its "Total Experience: Redefining Excellence in the Age of Agentic AI" report after several organizations stated the claims about their AI usage were untrue. Research by GPTZero revealed that the majority of the report's citations were AI-generated hallucinations.
- Confidence
- High (multi-source, primary)
Law Society of Ontario lawyer fined 31,150 CAD for Grok hallucinations
A lawyer was ordered to pay 31,150 CAD in adverse costs after using Grok to file fabricated legal authorities in a Canadian tribunal case. The incident demonstrates the risks of relying on AI for legal research without manual verification.
- Confidence
- High (multi-source, primary)
A Texas federal judge sanctioned an attorney for AI-fabricated citations in a TCPA case
In McCormick v. Texakoma Financial, Inc., decided June 11, 2026, the U.S. District Court for the Eastern District of Texas sanctioned attorney Amy L.B. Ginsburg after her summary-judgment response cited a nonexistent case, fabricated quotations, and misstated legal principles that appeared to come from generative AI. Judge Amos Mazzant struck the response, imposed a $5,000 penalty jointly on Ginsburg and her firm, ordered CLE and a review of her 2026 filings, and required a verification certification on future filings citing authority.
- Confidence
- Medium (multi-source)
Harbor Distributing lawyer sanctioned for AI fabricated case law
A lawyer for Harbor Distributing, LLC used AI to generate legal citations and quotes that were found to be fabricated. The court imposed a $6,000 sanction and referred the lawyer to the state bar.
- Confidence
- High (multi-source, primary)
Bowers files fabricated case law in Arizona court
A Pro Se litigant in Arizona submitted court filings containing fabricated case law generated by AI. The incident was documented in a database of AI legal hallucinations.
- Confidence
- High (multi-source, primary)
Iowa appeal dismissed after pro se litigant filed fabricated case law, AI suspected
Pro se litigant Mynesia A. Anderson submitted legal filings in an Iowa child support appeal containing fabricated case law and false quotes. The court identified the hallucinations and subsequently dismissed the appeal.
- Confidence
- High (multi-source, primary)
Henry County Schools v. Grant case involves AI fabricated case law
A lawyer and judge in the Georgia case Henry County Schools et al. v. Grant et al. submitted fabricated and misrepresented case law. The incident occurred on June 10, 2026, and resulted in the vacation of the trial court's order.
- Confidence
- High (multi-source, primary)
LiveVideo.AI Corp lawyer sanctioned for fabricated case law in SDNY
In the case of LiveVideo.AI Corp. v. Redstone, a lawyer submitted filings containing hallucinated case law. The S.D.N.Y. court imposed an adverse costs order of $80,056 and referred the attorney to the bar.
- Confidence
- High (multi-source, primary)
The Doc App counsel files fabricated case law in Florida court
A lawyer representing The Doc App, Inc. used AI to generate court filings that included fake case law. The court flagged the hallucinations and previously sanctioned the attorney, though it declined further sanctions in June 2026.
- Confidence
- High (multi-source, primary)
Ukrainian sea drone reportedly veers off course and explodes in Constanta port
On 2026-06-05 a naval/sea drone reportedly linked to Ukraine exploded in the Romanian port of Constanta after veering off course. Ukrainian officials told reporters the drone lost control following alleged electronic jamming; authorities say the area was secured and there were no injuries. Multiple independent news outlets reported the incident the same day.
- Confidence
- Medium (multi-source)
Reddit ads used deepfake news and cloned sites to promote AI investment scams
Reddit failed to prevent a series of sponsored ads that used deepfakes and cloned websites to impersonate news outlets like the BBC and The Guardian. These ads promoted fraudulent AI investment platforms, targeting users in the US and Europe.
- Confidence
- High (multi-source, primary)
Reaves Law Firm sanctioned for filing AI generated fabricated case law
A federal court in Tennessee sanctioned Reaves Law Firm, PLLC after the firm submitted filings containing hallucinated legal citations. The court issued a Rule 11 sanction, including a bar referral and an adverse costs order.
- Confidence
- High (multi-source, primary)
Pennsylvania AG settled with GEICO over AI underwriting tied to improper policy cancellations
Pennsylvania Attorney General Dave Sunday announced a settlement with GEICO on May 22, 2026, after an investigation found the insurer's AI tool for selecting new policyholders for underwriting review caused customer confusion and unfair policy cancellations. The AI selected a policyholder for review who submitted documents she believed were adequate, but GEICO failed to inform her the submission was insufficient and cancelled her policy without adequate notice, leaving her unknowingly driving uninsured. GEICO agreed to extend document submission deadlines, reduce verification requirements, and align with state AI guidance without admitting any violation of law.
- Confidence
- High (multi-source, primary)
HHS turned ChatGPT loose on Medicaid-linked audits with defunding power and no published error rate
On May 21, 2026, HHS launched AERO, the Audit Enforcement and Risk Oversight initiative, using ChatGPT and other LLMs to scan at least five years of Single Audit compliance filings from every state, hospital system, university, and nonprofit spending $1 million or more in annual federal funds. Flags can trigger payment holds, cost disallowances, award suspension, and debarment. By mid-July the legal pushback had hardened: no published error rate, no validation study, no notice-and-comment process, no disclosed appeal path, and no public evidence AERO met HHS's own trustworthy-AI requirements or OMB M-25-21, which requires High Impact AI to be discontinued if minimum risk practices are not met, with a compliance report due September 22. Firms are advising grantees to FOIA the AI's methodology before responding to any AERO letter.
- Confidence
- Medium (multi-source)
EY retracts loyalty rewards report after AI hallucinations and fake footnotes discovered
EY withdrew a cybersecurity report on loyalty rewards programs after researchers found it contained fabricated data and non-existent citations. The report was used by EY Canada for marketing purposes but was retracted once the AI-generated errors were exposed.
- Confidence
- Medium (multi-source)
GOV.UK Chat AI provides misleading tax advice to citizens
The GOV.UK Chat AI tool gave misleading tax advice, failing to identify key income thresholds and inaccurately suggesting no cap for childcare eligibility.
- Confidence
- Medium (multi-source)