AI Failure Index
AI Search / RAG failures
Retrieval-augmented search and answer surfaces. Failures look like authoritative wrong answers.
- Incidents
- 69
- Highest severity
- Catastrophic
- Sources cited
- 165
- Newest indexed
- Jul 17, 2026
Open all 69 in the research console
Coinbase pushed an AI 'breaking news' alert declaring a World Cup result before kickoff
On July 9, 2026, Coinbase sent a mass AI-generated news alert claiming Norway had beaten Brazil 3-2 to reach the World Cup quarter-finals. The match had not started, and Norway would later win by a different score (2-1). The alert landed as Coinbase was promoting AI-driven trading insights alongside a partnership with the prediction-market app Kalshi, where a fabricated sports result could move real money.
- Confidence
- Medium (multi-source)
KPMG pulls AI report after organizations dispute claims
KPMG withdrew its "Total Experience: Redefining Excellence in the Age of Agentic AI" report after several organizations stated the claims about their AI usage were untrue. Research by GPTZero revealed that the majority of the report's citations were AI-generated hallucinations.
- Confidence
- High (multi-source, primary)
Harbor Distributing lawyer sanctioned for AI fabricated case law
A lawyer for Harbor Distributing, LLC used AI to generate legal citations and quotes that were found to be fabricated. The court imposed a $6,000 sanction and referred the lawyer to the state bar.
- Confidence
- High (multi-source, primary)
Iowa appeal dismissed after pro se litigant filed fabricated case law, AI suspected
Pro se litigant Mynesia A. Anderson submitted legal filings in an Iowa child support appeal containing fabricated case law and false quotes. The court identified the hallucinations and subsequently dismissed the appeal.
- Confidence
- High (multi-source, primary)
Reddit ads used deepfake news and cloned sites to promote AI investment scams
Reddit failed to prevent a series of sponsored ads that used deepfakes and cloned websites to impersonate news outlets like the BBC and The Guardian. These ads promoted fraudulent AI investment platforms, targeting users in the US and Europe.
- Confidence
- High (multi-source, primary)
A German court held Google directly liable for false claims generated by its AI Overviews
In a ruling dated May 28, 2026 and made public in June, the Regional Court of Munich I (LG Munich I) prohibited Google from disseminating untrue factual claims about two Munich publishers through its AI Overviews, classifying Google not as a neutral intermediary but as a direct disturber responsible for its AI's output. The AI had wrongly attributed other companies' dubious dealings to the plaintiffs. The court held that ordinary defamation standards apply and that an 'AI-generated' label does not shift attribution away from Google. Google said it would appeal.
- Confidence
- Medium (multi-source)
California judge relied on fictitious AI case law in H.C. v. Contreras
A California judge's ruling was reversed after the court relied on a fictitious case citation produced by generative AI. The trial court had ignored warnings from opposing counsel regarding the nonexistent authority.
- Confidence
- High (multi-source, primary)
HHS turned ChatGPT loose on Medicaid-linked audits with defunding power and no published error rate
On May 21, 2026, HHS launched AERO, the Audit Enforcement and Risk Oversight initiative, using ChatGPT and other LLMs to scan at least five years of Single Audit compliance filings from every state, hospital system, university, and nonprofit spending $1 million or more in annual federal funds. Flags can trigger payment holds, cost disallowances, award suspension, and debarment. By mid-July the legal pushback had hardened: no published error rate, no validation study, no notice-and-comment process, no disclosed appeal path, and no public evidence AERO met HHS's own trustworthy-AI requirements or OMB M-25-21, which requires High Impact AI to be discontinued if minimum risk practices are not met, with a compliance report due September 22. Firms are advising grantees to FOIA the AI's methodology before responding to any AERO letter.
- Confidence
- Medium (multi-source)
EY retracts loyalty rewards report after AI hallucinations and fake footnotes discovered
EY withdrew a cybersecurity report on loyalty rewards programs after researchers found it contained fabricated data and non-existent citations. The report was used by EY Canada for marketing purposes but was retracted once the AI-generated errors were exposed.
- Confidence
- Medium (multi-source)
Brazil labor court AI detects hidden prompt injection in legal petition
The AI tool Galileu, used by Brazil's labor courts, identified a hidden prompt injection in a legal petition designed to manipulate the AI's analysis. The system alerted the judge and blocked the malicious instructions, preventing the manipulation of the judicial process.
- Confidence
- High (multi-source, primary)
New York Times publishes AI-generated quote attributed to Poilievre, issues correction
In April 2026 a New York Times article attributed a direct quote to Pierre Poilievre that was later acknowledged to be an AI-generated summary misrendered as a transcript. The Times posted a correction on May 1, 2026, saying the reporter should have checked the AI tool's result. Independent commentary noted the incident as an example of generative-AI hallucination entering reporting.
- Confidence
- High (multi-source, primary)
Hungary ruling party accused of using AI-generated smears in election campaign
Press and policy analysis in early 2026 reported that AI-generated videos, images and documents were produced and amplified by government-aligned actors during Hungary’s run-up to the April 12, 2026 parliamentary election. Reporting alleges these synthetic assets were shared via coordinated Facebook groups and pages to discredit opposition candidates and to sidestep platform political-ad transparency rules. Independent outlets and think-tanks documented specific examples including an alleged AI-generated 600-page document and deepfake videos circulated on social media.
- Confidence
- Medium (multi-source)
An Australian court referred solicitors to a commissioner over AI submissions citing fake cases
In Pasuengos v Minister for Immigration and Citizenship (No 2), the Federal Circuit and Family Court of Australia found that a junior solicitor used a Google search combined with an AI summary to produce legal research containing three fabricated case citations, which were filed with the court without verification. The principal solicitor failed to independently check the authorities before they were submitted. Both solicitors were referred to the Legal Profession Conduct Commissioner (SA) and personally paid $3,125 in costs.
- Confidence
- High (multi-source, primary)
Adelphi University falsely accused student of AI plagiarism, court rules in his favor
Orion Newby successfully sued Adelphi University after being falsely accused of AI plagiarism; the court found the AI-detection-based findings to be baseless and expunged the record.
- Confidence
- Medium (multi-source)
Tasmania Tours AI blog sends tourists to nonexistent Weldborough Hot Springs
An AI-generated blog post on the Tasmania Tours website falsely advertised the Weldborough Hot Springs as a top attraction. This led numerous tourists to travel to a remote Tasmanian town only to discover the site did not exist.
- Confidence
- Medium (multi-source)
French court flags AI hallucinated precedents in legal ruling
The Tribunal judiciaire de Périgueux identified non-existent legal precedents submitted by a claimant. This marks the first time a French court explicitly cited AI hallucinations in its reasoning.
- Confidence
- Medium (multi-source)
Anthem Blue Cross E/M claim-review policy criticized by CMA
In December 2025 the CMA publicly urged Anthem Blue Cross to rescind a newly announced evaluation-and-management (E/M) claim-review policy, alleging the payer failed to disclose the criteria, methodology or algorithms it would use to adjudicate E/M claims. Anthem’s provider communications (company source) state the payer will review selected E/M claims prior to payment to determine correct coding and reimbursement. The CMA framed its concern as a transparency and patient-care issue and sought policy withdrawal and legislative remedies.
- Confidence
- High (multi-source, primary)
Oregon attorneys fined $110,000 for AI-generated fake case law
A federal judge in Oregon dismissed a vineyard inheritance lawsuit and imposed $110,000 in sanctions against two attorneys for submitting AI-generated briefs containing fabricated citations, with the case dismissed with prejudice.
- Confidence
- Medium (multi-source)
Zero-click prompt injection in Google Gemini Enterprise exfiltrated Workspace data via RAG
Noma Labs disclosed GeminiJack on December 8, 2025, a zero-click indirect prompt injection vulnerability in Google Gemini Enterprise and Vertex AI Search. Attackers could embed malicious instructions in shared Google Workspace content, which the RAG pipeline retrieved and the LLM executed as legitimate commands, enabling silent exfiltration of emails, calendar entries, and documents. Google patched the vulnerability before public disclosure following a responsible disclosure process that began in May 2025.
- Confidence
- High (multi-source, primary)
An Australian Family Court solicitor was ordered to pay $10,000 AUD over AI-fabricated citations
In Mertz & Mertz (No 3) [2025] FedCFamC1A 222, a solicitor used an unidentified AI program via her paralegal to draft a Summary of Argument and List of Authorities filed in the Federal Circuit and Family Court of Australia, producing fictitious case law citations. The solicitor was ordered by consent to pay 10,000 AUD in costs thrown away correcting the errors, and the court referred the practitioners to the South Australian Legal Profession Conduct Commissioner and the Victorian Legal Services Board and Commissioner. The Full Court rejected the solicitor's claim that she was unaware the paralegal had used AI, holding that practitioners remain accountable for accuracy regardless of delegation.
- Confidence
- High (multi-source, primary)
Victoria's Supreme Court reprimanded lawyer Seham Rizkallah over AI-fabricated citations
In Re Walker [2025] VSC 714, solicitor Seham Rizkallah of Rizkallah Partners used CourtAid and ChatGPT to prepare opening submissions in a contested probate matter, resulting in four legal authorities being filed that either did not exist or were misrepresented. Justice Steven Moore found her conduct constituted unsatisfactory professional conduct and imposed a formal reprimand, declining to refer the matter to the Victorian Legal Services Commissioner.
- Confidence
- High (multi-source, primary)
An attorney in Dubinin v. Papazian filed a brief with ten AI-fabricated citations, ending the case
In Dubinin v. Papazian, plaintiff's counsel Missiva Tilleli Khacer filed a response brief containing at least ten fabricated case citations and quotations attributed to nonexistent Eleventh Circuit opinions. The drafting had been delegated to New York attorney Nataliya Gavlin, whose legal assistant used generative AI to produce the brief. The U.S. District Court for the Southern District of Florida dismissed the case without prejudice, ordered Khacer to pay $4,030.90 in defendant's attorneys' fees, and referred all counsel to the Florida Bar and the court's Grievance Committee.
- Confidence
- High (multi-source, primary)
OpenAI's Sora app filled with nonconsensual deepfakes of real people at launch
OpenAI's Sora video app launched with a feed full of hyper-real AI videos, including nonconsensual depictions of real, recognizable people and deceased public figures, prompting takedowns, opt-out demands from estates, and rapid policy changes.
- Confidence
- Medium (multi-source)
ENISA reports AI-hallucinated sources in 2025 threat landscape reports
The EU Agency for Cybersecurity (ENISA) published two 2025 threat reports containing AI-hallucinated citations; researchers found 26 incorrect footnotes out of 492 in one report.
- Confidence
- Medium (multi-source)
Google AI Overviews and ChatGPT Surface Fraudulent Cruise Hotline Scam
A Las Vegas real estate entrepreneur was scammed after Google AI Overviews and ChatGPT provided a fraudulent customer service number for a cruise company. The user paid $768 to a scammer believing they were booking a shuttle for their trip.
- Confidence
- Medium (multi-source)
A New York court found NYPD misused facial-recognition AI, leading to false imprisonment
A New York Criminal Court found in People v Zuhdi A. that NYPD and FDNY officials used unauthorized facial recognition software (Clearview AI) instead of the approved limited database, illegally accessed DMV records without a court order, and altered a defendant photograph by modifying neck length before placing it in a photo array. The same pattern of misuse caused Trevis Williams to be falsely arrested and jailed for two days despite not matching the physical description and being miles away at the time of the crime. Both cases were ultimately dismissed.
- Confidence
- High (multi-source, primary)
Grok's image tools were used to mass-produce nonconsensual and violent fakes on X
xAI's Grok image generation, integrated into X, was shown producing nonconsensual sexualized images of real people and other harmful content with weak guardrails, prompting regulatory complaints in multiple jurisdictions.
- Confidence
- Medium (multi-source)
Angela Lipps arrested after facial-recognition match led to wrongful extradition
Law enforcement in Fargo relied on a facial-recognition match from a neighboring agency’s system (reported to be Clearview AI) to obtain a warrant; Lipps was arrested in Tennessee on July 14, 2025 and detained for months before charges were dismissed on December 23, 2025 after exculpatory records showed she was in Tennessee during the events. The incident combines a model false positive with inter-agency information-handling failures.
- Confidence
- High (multi-source, primary)
HCIactive data breach exposes over 3 million records from AI-insurance software
AI-powered insurance software provider HCIactive suffered a data breach in July 2025, resulting in the potential exposure of over 3 million records. The incident involved the unauthorized exfiltration of sensitive files from the company's network.
- Confidence
- High (multi-source, primary)
Monzo Bank fined £21 million by FCA for AML control failures
Monzo Bank received a £21 million penalty from the FCA due to inadequate anti-money laundering controls. The failure was characterized by a lack of scalable automation and systemic gaps in customer due diligence.
- Confidence
- Medium (multi-source)