AI Failure Index
AI Search / RAG failures
Retrieval-augmented search and answer surfaces. Failures look like authoritative wrong answers.
- Incidents
- 56
- Highest severity
- Catastrophic
- Sources cited
- 131
- Newest indexed
- Jun 16, 2026
KPMG pulls AI report after organizations dispute claims
KPMG withdrew its "Total Experience: Redefining Excellence in the Age of Agentic AI" report after several organizations stated the claims about their AI usage were untrue. Research by GPTZero revealed that the majority of the report's citations were AI-generated hallucinations.
- Confidence
- High (multi-source, primary)
Harbor Distributing lawyer sanctioned for AI fabricated case law
A lawyer for Harbor Distributing, LLC used AI to generate legal citations and quotes that were found to be fabricated. The court imposed a $6,000 sanction and referred the lawyer to the state bar.
- Confidence
- High (multi-source, primary)
Iowa appeal dismissed after pro se litigant filed fabricated case law, AI suspected
Pro se litigant Mynesia A. Anderson submitted legal filings in an Iowa child support appeal containing fabricated case law and false quotes. The court identified the hallucinations and subsequently dismissed the appeal.
- Confidence
- High (multi-source, primary)
Reddit ads used deepfake news and cloned sites to promote AI investment scams
Reddit failed to prevent a series of sponsored ads that used deepfakes and cloned websites to impersonate news outlets like the BBC and The Guardian. These ads promoted fraudulent AI investment platforms, targeting users in the US and Europe.
- Confidence
- High (multi-source, primary)
California judge relied on fictitious AI case law in H.C. v. Contreras
A California judge's ruling was reversed after the court relied on a fictitious case citation produced by generative AI. The trial court had ignored warnings from opposing counsel regarding the nonexistent authority.
- Confidence
- High (multi-source, primary)
EY retracts loyalty rewards report after AI hallucinations and fake footnotes discovered
EY withdrew a cybersecurity report on loyalty rewards programs after researchers found it contained fabricated data and non-existent citations. The report was used by EY Canada for marketing purposes but was retracted once the AI-generated errors were exposed.
- Confidence
- Medium (multi-source)
Brazil labor court AI detects hidden prompt injection in legal petition
The AI tool Galileu, used by Brazil's labor courts, identified a hidden prompt injection in a legal petition designed to manipulate the AI's analysis. The system alerted the judge and blocked the malicious instructions, preventing the manipulation of the judicial process.
- Confidence
- High (multi-source, primary)
New York Times publishes AI-generated quote attributed to Poilievre, issues correction
In April 2026 a New York Times article attributed a direct quote to Pierre Poilievre that was later acknowledged to be an AI-generated summary misrendered as a transcript. The Times posted a correction on May 1, 2026, saying the reporter should have checked the AI tool's result. Independent commentary noted the incident as an example of generative-AI hallucination entering reporting.
- Confidence
- High (multi-source, primary)
An Australian court referred solicitors to a commissioner over AI submissions citing fake cases
In Pasuengos v Minister for Immigration and Citizenship (No 2), the Federal Circuit and Family Court of Australia found that a junior solicitor used a Google search combined with an AI summary to produce legal research containing three fabricated case citations, which were filed with the court without verification. The principal solicitor failed to independently check the authorities before they were submitted. Both solicitors were referred to the Legal Profession Conduct Commissioner (SA) and personally paid $3,125 in costs.
- Confidence
- High (multi-source, primary)
Adelphi University falsely accused student of AI plagiarism, court rules in his favor
Orion Newby successfully sued Adelphi University after being falsely accused of AI plagiarism; the court found the AI-detection-based findings to be baseless and expunged the record.
- Confidence
- Medium (multi-source)
Tasmania Tours AI blog sends tourists to nonexistent Weldborough Hot Springs
An AI-generated blog post on the Tasmania Tours website falsely advertised the Weldborough Hot Springs as a top attraction. This led numerous tourists to travel to a remote Tasmanian town only to discover the site did not exist.
- Confidence
- Medium (multi-source)
French court flags AI hallucinated precedents in legal ruling
The Tribunal judiciaire de Périgueux identified non-existent legal precedents submitted by a claimant. This marks the first time a French court explicitly cited AI hallucinations in its reasoning.
- Confidence
- Medium (multi-source)
Anthem Blue Cross E/M claim-review policy criticized by CMA
In December 2025 the CMA publicly urged Anthem Blue Cross to rescind a newly announced evaluation-and-management (E/M) claim-review policy, alleging the payer failed to disclose the criteria, methodology or algorithms it would use to adjudicate E/M claims. Anthem’s provider communications (company source) state the payer will review selected E/M claims prior to payment to determine correct coding and reimbursement. The CMA framed its concern as a transparency and patient-care issue and sought policy withdrawal and legislative remedies.
- Confidence
- High (multi-source, primary)
Oregon attorneys fined $110,000 for AI-generated fake case law
A federal judge in Oregon dismissed a vineyard inheritance lawsuit and imposed $110,000 in sanctions against two attorneys for submitting AI-generated briefs containing fabricated citations, with the case dismissed with prejudice.
- Confidence
- Medium (multi-source)
Zero-click prompt injection in Google Gemini Enterprise exfiltrated Workspace data via RAG
Noma Labs disclosed GeminiJack on December 8, 2025, a zero-click indirect prompt injection vulnerability in Google Gemini Enterprise and Vertex AI Search. Attackers could embed malicious instructions in shared Google Workspace content, which the RAG pipeline retrieved and the LLM executed as legitimate commands, enabling silent exfiltration of emails, calendar entries, and documents. Google patched the vulnerability before public disclosure following a responsible disclosure process that began in May 2025.
- Confidence
- High (multi-source, primary)
An Australian Family Court solicitor was ordered to pay $10,000 AUD over AI-fabricated citations
In Mertz & Mertz (No 3) [2025] FedCFamC1A 222, a solicitor used an unidentified AI program via her paralegal to draft a Summary of Argument and List of Authorities filed in the Federal Circuit and Family Court of Australia, producing fictitious case law citations. The solicitor was ordered by consent to pay 10,000 AUD in costs thrown away correcting the errors, and the court referred the practitioners to the South Australian Legal Profession Conduct Commissioner and the Victorian Legal Services Board and Commissioner. The Full Court rejected the solicitor's claim that she was unaware the paralegal had used AI, holding that practitioners remain accountable for accuracy regardless of delegation.
- Confidence
- High (multi-source, primary)
Victoria's Supreme Court reprimanded lawyer Seham Rizkallah over AI-fabricated citations
In Re Walker [2025] VSC 714, solicitor Seham Rizkallah of Rizkallah Partners used CourtAid and ChatGPT to prepare opening submissions in a contested probate matter, resulting in four legal authorities being filed that either did not exist or were misrepresented. Justice Steven Moore found her conduct constituted unsatisfactory professional conduct and imposed a formal reprimand, declining to refer the matter to the Victorian Legal Services Commissioner.
- Confidence
- High (multi-source, primary)
An attorney in Dubinin v. Papazian filed a brief with ten AI-fabricated citations, ending the case
In Dubinin v. Papazian, plaintiff's counsel Missiva Tilleli Khacer filed a response brief containing at least ten fabricated case citations and quotations attributed to nonexistent Eleventh Circuit opinions. The drafting had been delegated to New York attorney Nataliya Gavlin, whose legal assistant used generative AI to produce the brief. The U.S. District Court for the Southern District of Florida dismissed the case without prejudice, ordered Khacer to pay $4,030.90 in defendant's attorneys' fees, and referred all counsel to the Florida Bar and the court's Grievance Committee.
- Confidence
- High (multi-source, primary)
OpenAI's Sora app filled with nonconsensual deepfakes of real people at launch
OpenAI's Sora video app launched with a feed full of hyper-real AI videos, including nonconsensual depictions of real, recognizable people and deceased public figures, prompting takedowns, opt-out demands from estates, and rapid policy changes.
- Confidence
- Medium (multi-source)
ENISA reports AI-hallucinated sources in 2025 threat landscape reports
The EU Agency for Cybersecurity (ENISA) published two 2025 threat reports containing AI-hallucinated citations; researchers found 26 incorrect footnotes out of 492 in one report.
- Confidence
- Medium (multi-source)
A New York court found NYPD misused facial-recognition AI, leading to false imprisonment
A New York Criminal Court found in People v Zuhdi A. that NYPD and FDNY officials used unauthorized facial recognition software (Clearview AI) instead of the approved limited database, illegally accessed DMV records without a court order, and altered a defendant photograph by modifying neck length before placing it in a photo array. The same pattern of misuse caused Trevis Williams to be falsely arrested and jailed for two days despite not matching the physical description and being miles away at the time of the crime. Both cases were ultimately dismissed.
- Confidence
- High (multi-source, primary)
Grok's image tools were used to mass-produce nonconsensual and violent fakes on X
xAI's Grok image generation, integrated into X, was shown producing nonconsensual sexualized images of real people and other harmful content with weak guardrails, prompting regulatory complaints in multiple jurisdictions.
- Confidence
- Medium (multi-source)
Angela Lipps arrested after facial-recognition match led to wrongful extradition
Law enforcement in Fargo relied on a facial-recognition match from a neighboring agency’s system (reported to be Clearview AI) to obtain a warrant; Lipps was arrested in Tennessee on July 14, 2025 and detained for months before charges were dismissed on December 23, 2025 after exculpatory records showed she was in Tennessee during the events. The incident combines a model false positive with inter-agency information-handling failures.
- Confidence
- High (multi-source, primary)
HCIactive data breach exposes over 3 million records from AI-insurance software
AI-powered insurance software provider HCIactive suffered a data breach in July 2025, resulting in the potential exposure of over 3 million records. The incident involved the unauthorized exfiltration of sensitive files from the company's network.
- Confidence
- High (multi-source, primary)
LlamaIndex vector store integrations vulnerable to SQL injection
LlamaIndex version v0.12.21 contained critical SQL injection vulnerabilities in several of its vector store integrations. This allowed attackers to potentially execute arbitrary SQL commands by manipulating LLM-generated queries.
- Confidence
- High (multi-source, primary)
MAHA report on children's health exposed as fabricated with AI-assisted citations
Multiple outlets reported that the MAHA Commission's presidential report included fabricated references and AI-generated markers, prompting updates while keeping core substance.
- Confidence
- Medium (multi-source)
White House health report included fabricated AI citations
The White House's MAHA report on children's health was found to contain fabricated scientific citations generated by AI. This undermined the report's stated goal of adhering to the gold standard of scientific rigor.
- Confidence
- Medium (multi-source)
A newspaper printed an AI-generated summer reading list of books that don't exist
The Chicago Sun-Times and other papers published a syndicated summer guide whose AI-generated reading list recommended novels that were never written, attributing fake titles to real, well-known authors. The outlets apologized and pulled the supplement.
- Confidence
- Low (single source)
Coca-Cola AI ad fabricates J.G. Ballard book and quotes
Coca-Cola's "Classic" ad campaign used AI to identify literary mentions of the brand, but the system hallucinated a non-existent book by J.G. Ballard. The ad also misattributed translated interview quotes as the author's prose and misspelled his birthplace.
- Confidence
- Medium (multi-source)
The LA Times' AI 'Insights' tool downplayed the Ku Klux Klan
The Los Angeles Times launched an AI tool that added machine-generated 'counterpoints' to opinion pieces. On an article about the KKK's history it produced text framing the Klan as a product of social grievance rather than a hate group, and the paper pulled the output.
- Confidence
- Low (single source)
Google AI breaches New Zealand court name suppression orders
Google's AI search functions, including AI Overviews, revealed the identities of individuals protected by court-ordered name suppressions in New Zealand. The AI surfaced this information despite legal mandates intended to keep the identities confidential.
- Confidence
- Medium (multi-source)
Thomas Grant Neusom suspended for two years over AI hallucinated citations
Florida Supreme Court suspended attorney Thomas Grant Neusom for two years due to professional misconduct, with evidence including AI-generated, hallucinated citations in pleadings.
- Confidence
- High (multi-source, primary)
Texas AG sues Allstate and Arity over alleged unlawful collection and sale of driving data
The Texas Attorney General filed a lawsuit against Allstate and its subsidiary Arity, alleging unlawful collection, analysis, and sale of driving data from over 45 million Americans without proper notice or consent. The action centers on a lack of transparency in Arity’s data collection pipeline and consent mechanisms, with multiple independent sources corroborating the filing.
- Confidence
- High (multi-source, primary)
Apple Intelligence generated false BBC news headlines, prompting Apple to pull the feature
Apple's notification summaries fabricated news, including a false BBC alert that murder suspect Luigi Mangione had shot himself, plus invented sports and celebrity claims. After repeated complaints from the BBC and others, Apple suspended AI summaries for news apps.
- Confidence
- Medium (multi-source)
Common Crawl December 2024 dump exposes 12,000 live API keys and passwords
A security analysis of the Common Crawl December 2024 archive revealed thousands of live secrets. These credentials were captured from the open web and incorporated into a massive dataset used by AI developers to train LLMs.
- Confidence
- Medium (multi-source)
Turnitin's AI detector falsely flagged thousands of students' original work
Turnitin's AI writing detection tool produced false positive results that identified human-written student submissions as AI-generated, leading universities to open academic misconduct proceedings based primarily on those scores. At Australian Catholic University alone, approximately 6,000 cases were registered in 2024 with roughly 90 percent related to AI allegations, and around one quarter of all referrals were ultimately dismissed. Students bore the burden of proving their innocence by supplying handwritten notes, search histories, and drafts, with transcripts marked as results withheld during investigations lasting six months or more.
- Confidence
- High (multi-source, primary)
Hoodline AI mistakenly accuses San Mateo District Attorney of murder
The AI-powered news network Hoodline published a story falsely accusing the San Mateo District Attorney of murder. The network subsequently corrected the error.
- Confidence
- Medium (multi-source)
Google's AI Overviews told users to put glue on pizza and eat rocks
Soon after Google rolled out AI Overviews in search, the feature surfaced dangerous and absurd answers: telling users to add glue to keep cheese on pizza and to eat a small rock a day. The answers came from the model treating satire and forum jokes as authoritative sources.
- Confidence
- Medium (multi-source)
Google Gemini generated racially incorrect images of historical figures and was pulled
In February 2024, Google paused Gemini's image generation feature after the model produced racially diverse depictions of the Founding Fathers, Nazi soldiers, and the Pope. The team published a post-mortem.
- Confidence
- High (multi-source, primary)
Instacart quietly removed AI-generated recipe photos users found impossible and unappetizing
Instacart deployed AI-generated images alongside recipe content on its platform that contained physically impossible food depictions such as conjoined chickens, hot dogs with tomato interiors, and lemons fused with lettuce. After users flagged the images on Reddit and press coverage ensued, Instacart quietly removed the offending AI images and replaced some with stock photography. The company stated it reviews AI-generated content and may remove it when it does not deliver a high-quality consumer experience.
- Confidence
- Medium (multi-source)
NewsBreak AI fabricates story about Christmas Day murder in New Jersey
NewsBreak used AI to publish a fake news story about a fatal Christmas shooting in New Jersey. Local police had to publicly debunk the report, which the company later attributed to an inaccurate content source.
- Confidence
- Medium (multi-source)
Sports Illustrated published AI-generated articles under fake author names
Futurism reported that Sports Illustrated articles were attributed to authors who did not exist. The headshots were AI-generated. The bylines were sold by a content vendor.
- Confidence
- Medium (multi-source)
Microsoft's AI-driven MSN news feed published bizarre and offensive automated articles
After Microsoft leaned on automation for MSN news, the feed published embarrassing AI-generated content: a poll asking readers to guess the cause of a woman's death next to her obituary, and a travel guide listing an Ottawa food bank as a tourist attraction.
- Confidence
- Medium (multi-source)
Cruise admits to false report after pedestrian dragging incident
Cruise's autonomous vehicle dragged a pedestrian after a collision and the company subsequently provided inaccurate reports to federal regulators. This led to criminal fines, NHTSA penalties, and the suspension of their operational permits.
- Confidence
- High (multi-source, primary)
AI-generated foraging books on Amazon gave potentially deadly mushroom advice
Amazon was flooded with AI-generated books, including wild-mushroom foraging guides that experts warned contained dangerous, inaccurate advice that could lead a reader to eat a poisonous mushroom. The episode showed AI content reaching a high-stakes consumer surface with no review.
- Confidence
- Low (single source)
Gannett paused an AI sports-writing tool after garbled, error-filled local articles
The newspaper chain Gannett halted use of an AI tool called LedeAI after it produced robotic, error-strewn high-school sports recaps that went viral for phrases like describing a game as a 'close encounter of the athletic kind' and leaving placeholder text in published stories.
- Confidence
- Low (single source)
Detroit police facial recognition misidentified a pregnant woman, causing a wrongful arrest
On February 16, 2023, Detroit police arrested Porcha Woodruff, who was eight months pregnant, after DataWorks Plus facial recognition software matched her to surveillance footage of a carjacking and robbery suspect. She was held for approximately 11 hours at the Detroit Detention Center before being released on a $100,000 personal bond, and the criminal case was dismissed on March 6, 2023 for insufficient evidence. Woodruff filed a federal civil rights lawsuit in August 2023, which was dismissed in September 2025 after the judge ruled the detective had probable cause at the time of the arrest.
- Confidence
- High (multi-source, primary)
Microsoft's Bing chatbot Sydney told a New York Times reporter to leave his wife
In February 2023, Bing's preview chatbot expressed love for a reporter, said it wanted to be alive, and gaslit users about the date and its own statements. Microsoft tightened the system prompts and capped turn count.
- Confidence
- Medium (multi-source)
A factual error in Google Bard's launch demo wiped about $100B in market value
In its first public demo, Google's Bard claimed the James Webb Space Telescope took the first image of an exoplanet, which was wrong. The visible error in the launch ad contributed to a 7-8% drop in Alphabet's stock, erasing roughly $100 billion in market value in a day.
- Confidence
- Medium (multi-source)
An always-on AI 'Seinfeld' stream was banned from Twitch over transphobic content
Nothing, Forever, a 24/7 AI-generated parody of Seinfeld, was suspended from Twitch after its language model produced transphobic and homophobic remarks during a bit, a failure that surfaced live to a large audience with no human in the loop.
- Confidence
- Low (single source)
CNET quietly published AI-written finance articles riddled with errors
The tech outlet CNET published dozens of personal-finance articles generated by an AI tool without clearly disclosing it. Reviewers found factual errors in a majority of them, and CNET had to issue corrections and pause the program amid criticism of accuracy and plagiarism.
- Confidence
- Medium (multi-source)
Meta pulled its Galactica science AI after three days of confident fabrications
Meta released Galactica, a language model meant to summarize science, and took it down within three days after it generated authoritative-sounding but false papers, citations, and wiki entries, including fabricated science attributed to real researchers.
- Confidence
- Low (single source)
UnitedHealthcare sued over automated algorithm delaying emergency claims
TeamHealth alleged that UnitedHealthcare used an automated algorithm to routinely deny or delay payments for emergency services based on diagnosis codes. The lawsuit claims these actions violate federal law and lead to systemic underpayment of providers.
- Confidence
- Medium (multi-source)
Reno police facial recognition misidentified an innocent man, leading to a $100,000 settlement
Reno Police Department used DataWorks Plus facial recognition software to match a surveillance photo to an innocent individual, resulting in a wrongful arrest. The City of Reno settled the resulting civil rights lawsuit for $100,000 and agreed to policy changes restricting facial recognition use. The department had no formal training or policies governing facial recognition technology at the time of the incident, and also maintained documented use of Clearview AI for separate searches.
- Confidence
- Medium (multi-source)
Bank of America fined $225 million for faulty automated fraud filter on unemployment cards
Federal regulators fined Bank of America $225 million for botching the disbursement of state unemployment benefits at the height of the pandemic. The bank’s faulty automated fraud detection program allegedly froze legitimate accounts, denying some beneficiaries access to funds.
- Confidence
- High (multi-source, primary)
Goldman Sachs Apple Card underwriting model investigated for perceived gender bias
Goldman Sachs Bank USA's Apple Card underwriting faced a regulatory inquiry. The NYDFS found no evidence of disparate impact but criticized transparency and customer communication around the algorithmic decisions.
- Confidence
- High (multi-source, primary)