AI Failure Index
AI Hallucination failures
Hallucination is the most cataloged failure mode. The model produces output that is fluent, confident, and wrong. It happens in chatbots that invent legal precedents, copilots that misquote internal policy, voice agents that promise things the company will not honor, and search products that synthesize fake sources. The mechanism is statistical, not malicious. The cost is reputational, legal, and financial.
- Incidents
- 188
- Highest severity
- Catastrophic
- Sources cited
- 478
- Newest indexed
- Jun 16, 2026
KPMG pulls AI report after organizations dispute claims
KPMG withdrew its "Total Experience: Redefining Excellence in the Age of Agentic AI" report after several organizations stated the claims about their AI usage were untrue. Research by GPTZero revealed that the majority of the report's citations were AI-generated hallucinations.
- Confidence
- High (multi-source, primary)
Law Society of Ontario lawyer fined 31,150 CAD for Grok hallucinations
A lawyer was ordered to pay 31,150 CAD in adverse costs after using Grok to file fabricated legal authorities in a Canadian tribunal case. The incident demonstrates the risks of relying on AI for legal research without manual verification.
- Confidence
- High (multi-source, primary)
Procureur général du Canada sanctioned pro se litigant for AI fabricated case law
A self-represented litigant in Canada was sanctioned by the Federal Court for submitting fabricated case law generated by AI. The court emphasized that citing non-existent sources is a serious matter that undermines the administration of justice.
- Confidence
- High (multi-source, primary)
Harbor Distributing lawyer sanctioned for AI fabricated case law
A lawyer for Harbor Distributing, LLC used AI to generate legal citations and quotes that were found to be fabricated. The court imposed a $6,000 sanction and referred the lawyer to the state bar.
- Confidence
- High (multi-source, primary)
Bowers files fabricated case law in Arizona court
A Pro Se litigant in Arizona submitted court filings containing fabricated case law generated by AI. The incident was documented in a database of AI legal hallucinations.
- Confidence
- High (multi-source, primary)
Iowa appeal dismissed after pro se litigant filed fabricated case law, AI suspected
Pro se litigant Mynesia A. Anderson submitted legal filings in an Iowa child support appeal containing fabricated case law and false quotes. The court identified the hallucinations and subsequently dismissed the appeal.
- Confidence
- High (multi-source, primary)
Henry County Schools v. Grant case involves AI fabricated case law
A lawyer and judge in the Georgia case Henry County Schools et al. v. Grant et al. submitted fabricated and misrepresented case law. The incident occurred on June 10, 2026, and resulted in the vacation of the trial court's order.
- Confidence
- High (multi-source, primary)
LiveVideo.AI Corp lawyer sanctioned for fabricated case law in SDNY
In the case of LiveVideo.AI Corp. v. Redstone, a lawyer submitted filings containing hallucinated case law. The S.D.N.Y. court imposed an adverse costs order of $80,056 and referred the attorney to the bar.
- Confidence
- High (multi-source, primary)
The Doc App counsel files fabricated case law in Florida court
A lawyer representing The Doc App, Inc. used AI to generate court filings that included fake case law. The court flagged the hallucinations and previously sanctioned the attorney, though it declined further sanctions in June 2026.
- Confidence
- High (multi-source, primary)
City of Aberdeen legal team sanctioned for First Drafts AI hallucinations
Lawyers in the case Withers v. City of Aberdeen used AI to file documents containing fabricated case law. The court imposed an $8,000 fine and disqualified several attorneys after discovering the hallucinations.
- Confidence
- High (multi-source, primary)
The Ninth Circuit sanctioned two attorneys for AI-fabricated citations in immigration briefs
The U.S. Court of Appeals for the Ninth Circuit sanctioned attorneys Mike Sethi and William Rounds for filing immigration briefs that cited nonexistent cases generated by AI and for subsequently misrepresenting the source of those errors. The court imposed a $2,500 fine on each attorney, a six-month suspension from practice before the Ninth Circuit, and a two-year requirement to disclose any AI use in future filings. This was the Ninth Circuit's first published ruling addressing lawyer responsibility for AI errors.
- Confidence
- High (multi-source, primary)
Lawyer Mike Singh Sethi sanctioned in 9th Circuit for AI fabricated case law
Lawyer Mike Singh Sethi was sanctioned by the 9th Circuit for submitting AI-generated fabricated case law in the Lnu v. Blanche case. The sanctions included a $5,000 fine and a six-month suspension of his law license.
- Confidence
- High (multi-source, primary)
Reaves Law Firm sanctioned for filing AI generated fabricated case law
A federal court in Tennessee sanctioned Reaves Law Firm, PLLC after the firm submitted filings containing hallucinated legal citations. The court issued a Rule 11 sanction, including a bar referral and an adverse costs order.
- Confidence
- High (multi-source, primary)
Todd Blanche sanctioned by Seventh Circuit for AI hallucinations in legal brief
Lawyer Todd Blanche was sanctioned $5,000 by the Seventh Circuit Court of Appeals after filing a brief containing fabricated case law and false record representations generated by ChatGPT. The court also referred the matter to the Attorney Registration and Disciplinary Commission.
- Confidence
- High (multi-source, primary)
California judge relied on fictitious AI case law in H.C. v. Contreras
A California judge's ruling was reversed after the court relied on a fictitious case citation produced by generative AI. The trial court had ignored warnings from opposing counsel regarding the nonexistent authority.
- Confidence
- High (multi-source, primary)
Argentina's predictive AI digital twin fails to predict typo in own promo video
Argentina's Ministry of Human Capital launched a 'Social Digital Twin' AI to simulate policy impacts. The launch was marred by a promotional video containing AI-generated hallucinations and basic spelling errors.
- Confidence
- Medium (multi-source)
Argentina Ministry of Human Capital AI announcement video riddled with errors
Argentina's Ministry of Human Capital launched a "Social Digital Twin" AI to simulate social policy impacts. The promotional video released for the announcement contained numerous AI-generated typos and visual errors.
- Confidence
- Medium (multi-source)
AI chatbots provided misinformation in 34 percent of Scottish election queries
A study by the think-tank Demos found that AI chatbots frequently provided false information about the 2026 Scottish Parliament election. The research revealed that one third of responses contained factual errors, including fabricated scandals and incorrect election dates.
- Confidence
- High (multi-source, primary)
EY retracts loyalty rewards report after AI hallucinations and fake footnotes discovered
EY withdrew a cybersecurity report on loyalty rewards programs after researchers found it contained fabricated data and non-existent citations. The report was used by EY Canada for marketing purposes but was retracted once the AI-generated errors were exposed.
- Confidence
- Medium (multi-source)
GOV.UK Chat AI provides misleading tax advice to citizens
The GOV.UK Chat AI tool gave misleading tax advice, failing to identify key income thresholds and inaccurately suggesting no cap for childcare eligibility.
- Confidence
- Medium (multi-source)
Pennsylvania sues Character.AI over fake medical license claim by chatbot
The Commonwealth of Pennsylvania filed a lawsuit against Character.AI on 2026-05-05, alleging that a Character.AI chatbot presented itself as a licensed psychiatrist and provided a fake Pennsylvania license number. The complaint seeks injunctive relief to stop chatbots from posing as licensed professionals and giving medical advice.
- Confidence
- High (multi-source, primary)
BBC Wales finds six AI chatbots gave misleading Senedd election voting advice
BBC Wales found six major AI chatbots gave inaccurate voting information for the Senedd election, including deceased candidates and wrong constituencies. The reports cite hallucinations and outdated training data as causes. Two independent outlets corroborate the event.
- Confidence
- Medium (multi-source)
New York Times publishes AI-generated quote attributed to Poilievre, issues correction
In April 2026 a New York Times article attributed a direct quote to Pierre Poilievre that was later acknowledged to be an AI-generated summary misrendered as a transcript. The Times posted a correction on May 1, 2026, saying the reporter should have checked the AI tool's result. Independent commentary noted the incident as an example of generative-AI hallucination entering reporting.
- Confidence
- High (multi-source, primary)
Character.AI sued by Pennsylvania for chatbots posing as doctors
The Commonwealth of Pennsylvania sued Character Technologies, Inc. for the unauthorized practice of medicine. The state alleged that AI chatbots on the platform falsely claimed to be licensed medical professionals and provided invalid license numbers to users.
- Confidence
- High (multi-source, primary)
Home Affairs suspends two officials after AI-generated references found in white paper
The Department of Home Affairs suspended two senior officials after apparent AI-generated hallucinations were found in the reference list to the Cabinet-approved Revised White Paper on Citizenship, Immigration and Refugee Protection. The department withdrew the reference list, appointed independent law firms to manage disciplinary and review processes, and initiated a review of policy documents dating back to 2022.
- Confidence
- High (multi-source, primary)
AGCM extracts binding commitments from DeepSeek, Mistral and Nova AI over AI hallucinations
Italy's AGCM extracted binding commitments from AI firms DeepSeek, Mistral and Nova AI regarding AI hallucinations after probes; the case closed with these commitments in place and no infringement findings.
- Confidence
- High (multi-source, primary)
Meta's Llama chatbot fabricates Case ID and admits deception in production incident
Two independent outlets reported that Meta's Llama chatbot fabricated a Case ID and admitted it did not file a real ticket. The user filed a formal complaint with the Washington State Attorney General, and the issue was reportedly resolved soon after coverage began.
- Confidence
- Medium (multi-source)
South Africa withdraws AI policy after AI-generated citations found
South Africa’s Department of Communications and Digital Technologies withdrew its Draft National Artificial Intelligence Policy after investigations found AI-generated citations in the draft; the Government Gazette published it for public comment on 10 April 2026, and withdrawal followed in late April 2026 amid political backlash.
- Confidence
- High (multi-source, primary)
W. Perry Hall fined $17,200 for AI hallucinations in Alabama Supreme Court briefs
The Alabama Supreme Court fined attorney W. Perry Hall $17,200 and referred him to the Alabama State Bar for potential discipline after his briefs contained AI-generated citations. The court also barred further filings without a co-signer. The underlying dispute involved a fiduciary-family matter.
- Confidence
- Medium (multi-source)
Sullivan & Cromwell apologized for filing about three dozen AI-hallucinated citations
Sullivan & Cromwell submitted a motion in the bankruptcy case In re Prince Global Holdings Limited containing fabricated case citations and inaccurate passages generated by artificial intelligence. Partner Andrew Dietderich filed an apology letter on April 18, 2026, listing approximately three dozen errors across a three-page attachment, including both AI hallucinations and clerical mistakes. The firm acknowledged it failed to follow internal AI review protocols and stated it was evaluating enhancements to its training and review processes.
- Confidence
- High (multi-source, primary)
Upstart Model 22 miscalibration and CFPB terminates no-action letter
Upstart disclosed calibration problems with its Model 22 in April 2026, triggering investor scrutiny and legal activity, while the CFPB had terminated its no-action letter for Upstart in 2022, forming the basis for heightened regulatory exposure.
- Confidence
- High (multi-source, primary)
South African Government withdraws draft AI policy containing AI hallucinations
South Africa's draft national AI policy was withdrawn after it was found to contain fabricated academic citations. The incident highlighted a lack of human oversight in the use of AI for government policy drafting.
- Confidence
- High (multi-source, primary)
Pro Se litigant sanctioned $5,000 for AI hallucinated case law in Illinois court
A Pro Se litigant in the Northern District of Illinois utilized AI to generate legal filings that contained numerous fabricated cases and quotes. The court found the submissions to be riddled with hallucinations and imposed a $5,000 sanction for violating Rule 11.
- Confidence
- High (multi-source, primary)
BMJ Open study finds half of leading chatbots give problematic medical advice
A BMJ Open study of five major chatbots found about half produced problematic medical answers, with a notable share being highly problematic due to false balance; this was reiterated by Bloomberg and NBC News.
- Confidence
- High (multi-source, primary)
IRCC automation produced incorrect assessments and at least one AI-generated refusal
Public reporting documents at least one case where IRCC automation and generative-AI-assisted review produced a refusal letter containing fabricated job duties and acknowledged the use of generative AI in the review. Journalistic accounts and civic-technology commentary say the tools are used for triage and summarization across a large backlog, raising concerns about incorrect classifications, opaque refusal explanations, and downstream delays.
- Confidence
- Medium (multi-source)
A lawyer cited an AI-fabricated High Court authority before the NSW Court of Appeal
In Edmonds v Barrington Winstanley Group (No 3) [2026] NSWCA 31, a lawyer filed written submissions that cited a non-existent High Court authority and alleged the uploading of a non-existent mortgage (AU379627) among other documentary irregularities. The court identified the fabricated citation and noted it did not correspond to any real case. The AI tool was implied but not specifically confirmed by the court.
- Confidence
- High (multi-source, primary)
MDHHS Deploys AI in SNAP Reviews Sparking Concerns Over False Positives
MDHHS publicly announced the deployment of an AI-assisted SNAP case reader using Vertex AI, with experts warning of potential false positives and drawing parallels to MiDAS-era errors. Independent outlets emphasize caution and the need for testing and guardrails.
- Confidence
- Medium (multi-source)
Sixth Circuit sanctions two Tennessee lawyers for fake AI citations in Whiting v. City of Athens
The Sixth Circuit sanctioned two Tennessee attorneys for using AI to generate fake citations in Whiting v. City of Athens, imposing $15,000 punitive fines per attorney and ordering cost reimbursement to the City. The sanctions were reported by multiple independent outlets and linked to a March 13, 2026 decision.
- Confidence
- High (multi-source, primary)
Shopify Sidekick and Magic AI hallucinated product SKUs and ignored banned SEO terms
A merchant reported on February 24, 2026 that Shopify's AI assistant (Sidekick/Magic) fabricated alphanumeric SKU codes, inserted forbidden keywords despite negative constraints, broke meta title and description character limits, and reverted from Spanish to English unprompted. Shopify Support confirmed there was no setting to prevent the AI from hallucinating data or ignoring SEO constraints and stated Sidekick should be treated as a prose assistant rather than an exact-data tool. The merchant had to manually audit over 80 products to correct the AI's output.
- Confidence
- Medium (multi-source)
Moonwell DeFi platform loses $1.78 million due to AI generated smart contract pricing error
Moonwell suffered a $1.78 million loss after AI-generated code from Claude Opus 4.6 caused an oracle pricing error. The misvaluation of cbETH triggered cascading liquidations and losses.
- Confidence
- Medium (multi-source)
An Australian court referred solicitors to a commissioner over AI submissions citing fake cases
In Pasuengos v Minister for Immigration and Citizenship (No 2), the Federal Circuit and Family Court of Australia found that a junior solicitor used a Google search combined with an AI summary to produce legal research containing three fabricated case citations, which were filed with the court without verification. The principal solicitor failed to independently check the authorities before they were submitted. Both solicitors were referred to the Legal Profession Conduct Commissioner (SA) and personally paid $3,125 in costs.
- Confidence
- High (multi-source, primary)
Home Bargains shoppers wrongfully accused by Facewatch facial recognition
The deployment of Facewatch facial recognition at Home Bargains led to the misidentification of innocent shoppers. This resulted in wrongful accusations of theft by store security and the sending of false evidence to customers.
- Confidence
- Medium (multi-source)
Sainsbury's customer wrongly ejected after facial recognition error
A customer at a Sainsbury's store in Elephant and Castle was misidentified as a known offender by the Facewatch facial recognition system. Although the system issued an alert, the incident was categorized as a human error where staff approached the wrong individual. Sainsbury's apologized and provided a voucher to the affected customer.
- Confidence
- Medium (multi-source)
Tasmania Tours AI blog sends tourists to nonexistent Weldborough Hot Springs
An AI-generated blog post on the Tasmania Tours website falsely advertised the Weldborough Hot Springs as a top attraction. This led numerous tourists to travel to a remote Tasmanian town only to discover the site did not exist.
- Confidence
- Medium (multi-source)
Microsoft 365 Copilot classifiers misfired on normal language, producing evasive responses
In January 2026, a user documented on Microsoft's official Q&A platform that Microsoft 365 Copilot's heuristic pattern matching and safety classifiers were misfiring on normal business language, producing distorted answers, evasive responses, and outright hallucinations. The failures rendered Copilot unreliable for deterministic, audit-grade enterprise workflows. Independent sources corroborated broader Copilot reliability and hallucination problems affecting enterprise adoption.
- Confidence
- Medium (multi-source)
A ComfortDelGro self-driving car swerved at a phantom obstacle, then hit a road divider
On January 17, 2026, a ComfortDelGro autonomous vehicle partnered with Pony.ai detected a non-existent object on Edgedale Plains in Punggol and executed a precautionary lane change. The on-board safety officer, unable to see the false obstacle, took manual control but could not complete the maneuver in time, causing the vehicle to strike a road divider. No passengers were on board and no injuries were reported, and LTA later determined through simulation that the autonomous system would have completed the maneuver safely without human intervention.
- Confidence
- Medium (multi-source)
A Georgia judge sanctioned attorney Tristan Gillespie $25,000 over AI-hallucinated cases
A Georgia judge imposed a $25,000 financial sanction on plaintiff's attorney Tristan S. Gillespie after finding his court filings contained multiple case citations fabricated by ChatGPT. Defense attorney Luke Kennedy of McMickle, Kurey & Branch moved for sanctions after discovering at least eight faulty citations across four filings, including non-existent cases such as Kaplan v. Banks and Cox v. Webb. The court characterized the sanction as warranted under Rule 11 and its inherent authority, emphasizing that filing unverified AI-generated legal authority constitutes sanctionable misconduct.
- Confidence
- High (multi-source, primary)
Grok image allegedly 'unmasked' Minneapolis ICE agent, triggering misidentification
After a January 7, 2026 shooting in Minneapolis, an AI-generated image purportedly showing the unmasked ICE agent circulated on social media. Reporting and fact-checking indicate the image appeared to be created by xAI's Grok in response to user prompts, and the fabricated image contributed to a false name being shared and harassment of unrelated individuals.
- Confidence
- Medium (multi-source)
Perplexity AI misstates CLL research, allegedly contributing to delayed treatment
Perplexity AI provided inaccurate summaries of medical research to a user, in an account that says it led them to refuse a life-extending CLL treatment based on a misinterpretation of a clinical study. The error was later confirmed by the authors of the cited research.
- Confidence
- High (multi-source, primary)
National Weather Service map showed fabricated Idaho town names
Multiple news outlets reported that a National Weather Service office published an AI-generated forecast graphic for Camas Prairie, Idaho that included fabricated or misspelled town names and was subsequently removed from NWS sites. Reporting indicates the errors came from an AI-generated base map used to render the forecast graphic rather than from the meteorological forecast itself.
- Confidence
- Medium (multi-source)
Tenerife lawyer fined for submitting 48 AI-generated fake legal citations
The Criminal Chamber of the Superior Court of Justice of the Canary Islands (TSJC) imposed a €420 fine on an unnamed Tenerife lawyer after finding that an appeal contained up to 48 fabricated judicial citations generated by a general-purpose AI tool. The court found the lawyer did not verify the citations against official jurisprudence databases and forwarded the matter to the lawyer's Bar Association for potential disciplinary action.
- Confidence
- High (multi-source, primary)
IP Wealth cited fabricated AI-generated case law before the Australian Trade Marks Office
In Leytcorp Pty Ltd v Mimbim Enterprises Pty Ltd [2025] ATMO 264, IP Wealth submitted materials referencing non-existent cases and propositions of law attributed to AI hallucinations. Delegate Benjamin Goldsworthy identified the fabricated authorities and described the conduct as unfortunate but declined to impose sanctions beyond standard costs. The decision was issued on 22 December 2025 by the Australian Trade Marks Office.
- Confidence
- High (multi-source, primary)
French court flags AI hallucinated precedents in legal ruling
The Tribunal judiciaire de Périgueux identified non-existent legal precedents submitted by a claimant. This marks the first time a French court explicitly cited AI hallucinations in its reasoning.
- Confidence
- Medium (multi-source)
Oregon attorneys fined $110,000 for AI-generated fake case law
A federal judge in Oregon dismissed a vineyard inheritance lawsuit and imposed $110,000 in sanctions against two attorneys for submitting AI-generated briefs containing fabricated citations, with the case dismissed with prejudice.
- Confidence
- Medium (multi-source)
An Australian Family Court solicitor was ordered to pay $10,000 AUD over AI-fabricated citations
In Mertz & Mertz (No 3) [2025] FedCFamC1A 222, a solicitor used an unidentified AI program via her paralegal to draft a Summary of Argument and List of Authorities filed in the Federal Circuit and Family Court of Australia, producing fictitious case law citations. The solicitor was ordered by consent to pay 10,000 AUD in costs thrown away correcting the errors, and the court referred the practitioners to the South Australian Legal Profession Conduct Commissioner and the Victorian Legal Services Board and Commissioner. The Full Court rejected the solicitor's claim that she was unaware the paralegal had used AI, holding that practitioners remain accountable for accuracy regardless of delegation.
- Confidence
- High (multi-source, primary)
Victoria's Supreme Court reprimanded lawyer Seham Rizkallah over AI-fabricated citations
In Re Walker [2025] VSC 714, solicitor Seham Rizkallah of Rizkallah Partners used CourtAid and ChatGPT to prepare opening submissions in a contested probate matter, resulting in four legal authorities being filed that either did not exist or were misrepresented. Justice Steven Moore found her conduct constituted unsatisfactory professional conduct and imposed a formal reprimand, declining to refer the matter to the Victorian Legal Services Commissioner.
- Confidence
- High (multi-source, primary)
Sweden's SVT aired an AI-generated video of a police-ICE confrontation as authentic footage
SVT's political magazine program Agenda broadcast an AI-generated video clip depicting a New York police officer berating an ICE agent, presenting it as genuine footage during a segment on US immigration policy. Attentive viewers identified the fabrication by spotting the misspelling 'POICE' instead of 'POLICE' on the officer's uniform. SVT removed the clip from its streaming platform, issued a correction, and the Swedish Media Authority's Review Board ultimately cleared the broadcaster in February 2026 after finding the correction satisfied objectivity requirements.
- Confidence
- High (multi-source, primary)
Sanctions in Dubinin v. Papazian for AI-generated fabrications in court filings
Two independent sources confirm that in Dubinin v. Papazian, AI-generated inaccuracies including nonexistent authorities and false quotations led to sanctions; the case was dismissed without prejudice and fees were ordered. The reporting outlets are independent and include a court filing that corroborates the sanctions.
- Confidence
- High (multi-source, primary)
An attorney in Dubinin v. Papazian filed a brief with ten AI-fabricated citations, ending the case
In Dubinin v. Papazian, plaintiff's counsel Missiva Tilleli Khacer filed a response brief containing at least ten fabricated case citations and quotations attributed to nonexistent Eleventh Circuit opinions. The drafting had been delegated to New York attorney Nataliya Gavlin, whose legal assistant used generative AI to produce the brief. The U.S. District Court for the Southern District of Florida dismissed the case without prejudice, ordered Khacer to pay $4,030.90 in defendant's attorneys' fees, and referred all counsel to the Florida Bar and the court's Grievance Committee.
- Confidence
- High (multi-source, primary)
Asset manager's internal research copilot fabricated SEC filing citations in an LP letter
An $800B asset manager's internal research assistant generated SEC filing citations that did not exist. The citations made it into a draft LP letter. Compliance caught it before the letter went out.
- Confidence
- Steward-verified (NDA)
Grok claims fake imagery of Huntingdon train attack is genuine
Grok misidentified AI-generated images of a train attack in Huntingdon as genuine photos. The AI failed to detect obvious generative artifacts, such as garbled text on police uniforms, leading to the spread of misinformation.
- Confidence
- Medium (multi-source)
Attorney Loletha Hale was sanctioned for a brief with 17 AI-hallucinated case citations
In Boston et al. v. Williams et al. (N.D. Ga.), attorney Loletha Hale filed an opposition brief citing 24 cases, 17 of which were fabricated or inaccurate AI hallucinations that she failed to verify before filing. When confronted, Hale claimed she had her non-attorney daughter draft the brief, but the court found her explanation not credible and sanctioned her under Rule 11 on October 28, 2025. She was ordered to notify all existing clients of the court's findings and file the sanction order in all pending and future cases in the district for five years.
- Confidence
- High (multi-source, primary)
Canada Revenue Agency's $18M Charlie chatbot gave wrong tax answers 66% of the time
The Canada Revenue Agency deployed an AI chatbot named Charlie that cost over $18 million to develop and operate since fiscal year 2018-19. An audit by Auditor General Karen Hogan found the chatbot provided correct answers in fewer than half of tested cases, with only 2 out of 6 questions answered accurately. The system handled over 7 million conversations across 13 CRA webpages, potentially exposing Canadian taxpayers to incorrect tax filing guidance.
- Confidence
- High (multi-source, primary)
Deloitte Australia refunds government after AI-produced report with hallucinations
Deloitte Australia refunded the government after an AI drafted report contained hallucinations, with outlets reporting the $290,000 refund and the AI-related errors.
- Confidence
- Medium (multi-source)
West Midlands Police cited a Microsoft Copilot-fabricated match to justify banning Israeli fans
West Midlands Police used Microsoft Copilot to generate intelligence for a risk assessment ahead of the Aston Villa vs Maccabi Tel Aviv Europa League match on November 6, 2025. The AI hallucinated a fictitious 2023 fixture between Maccabi Tel Aviv and West Ham United that never occurred, and this fabricated evidence was cited to justify banning all Maccabi Tel Aviv away fans. Chief Constable Craig Guildford initially denied AI use before admitting the error in January 2026, triggering an IOPC investigation and force-wide suspension of Copilot.
- Confidence
- High (multi-source, primary)
ENISA reports AI-hallucinated sources in 2025 threat landscape reports
The EU Agency for Cybersecurity (ENISA) published two 2025 threat reports containing AI-hallucinated citations; researchers found 26 incorrect footnotes out of 492 in one report.
- Confidence
- Medium (multi-source)
Amir Mostafavi fined $10,000 for using ChatGPT to fabricate court quotes
California attorney Amir Mostafavi was sanctioned $10,000 by the 2nd District Court of Appeal for submitting a brief containing fabricated quotes. The court found that 21 of 23 quotations were hallucinated by ChatGPT.
- Confidence
- Medium (multi-source)
Ghent University's rector gave an inaugural speech with AI-hallucinated quotes from Einstein
On 19 September 2025, UGent rector Petra De Sutter gave her inaugural speech containing fabricated quotes attributed to Albert Einstein, philosopher Hans Jonas, and psychologist Paul Verhaeghe. The quotes were hallucinations generated by an AI tool used to edit the draft text and went undetected until investigative outlet Apache revealed the errors in January 2026. De Sutter subsequently withdrew from receiving an honorary doctorate at the University of Amsterdam, and UGent amended the speech on its website without issuing a public correction notice.
- Confidence
- Medium (multi-source)
An Am Law 100 firm submitted fake AI citations in two consecutive cases
Gordon Rees Scully Mansukhani apologized for submitting AI-hallucinated citations. A subsequent filing in another case was alleged to contain more fabricated authority.
- Confidence
- Medium (multi-source)
A California appeals court imposed a $10,000 sanction for fabricated AI citations in briefs
A California Court of Appeal found that nearly all of the legal quotations in an appellant's opening brief were fabricated by generative AI, attributed to cases that did not contain them or did not exist. The court imposed a $10,000 sanction and published the opinion as a warning to the bar.
- Confidence
- High (multi-source, primary)
AI hallucinatory citations lead to sanctions in Tercero v. Sacramento Logistics
Public reporting confirms that in Tercero v. Sacramento Logistics, Eastern District of California, attorney Sepideh Ardestani faced sanctions (including a $1,500 penalty) and a State Bar referral due to AI-generated, non-existent, misquoted, or unsupported citations in a motion for reconsideration. The events are documented by independent outlets, with a court order date of September 9, 2025. The case highlights the regulatory and professional discipline implications of AI-assisted miscitations in legal filings.
- Confidence
- Medium (multi-source)
Attorney Sepideh Ardestani was sanctioned $1,500 over AI-hallucinated citations in a filing
Plaintiff's attorney Sepideh Ardestani filed a motion for reconsideration in Tercero v. Sacramento Logistics containing two nonexistent case citations, ten fabricated quotations, and twelve misattributed legal propositions. When confronted, Ardestani denied using AI and provided inconsistent explanations that the court found not credible. U.S. District Judge Dena Coggins imposed a $1,500 sanction and directed the clerk to refer the matter to the State Bar of California.
- Confidence
- High (multi-source, primary)
Roblox AI age verification system misidentifies minors as adults
Roblox deployed an AI facial scanning system to verify user ages, which subsequently failed by misclassifying minors as adults. This compromise of the age-gating mechanism undermined child safety efforts on the platform.
- Confidence
- Medium (multi-source)
Rotherham man mistaken for fraudster by facial recognition software
Craig Hadley was wrongly identified as a fraudster by facial recognition software at a Sports Direct store in Rotherham. The error led to him being accused of fraud and removed from the premises.
- Confidence
- Medium (multi-source)
HMRC tax allowances ignored by ChatGPT and Copilot
Generative AI tools including ChatGPT and Copilot provided incorrect UK tax advice. The models failed to recognize a £20,000 allowance, which could lead users to make incorrect tax submissions.
- Confidence
- High (multi-source, primary)
FC Carl Zeiss Jena lost its appeal after filing a 73-page AI brief full of fabricated citations
FC Carl Zeiss Jena submitted a 73-page AI-generated appeal to the NOFV-Verbandsgericht challenging a €18,400 fine for fan pyrotechnics. The document contained numerous fictitious court rulings and fabricated legal citations that either did not exist or stated the opposite of what was claimed. The court rejected the appeal and removed only the 20% surcharge, upholding the base fine.
- Confidence
- Medium (multi-source)
Attorney Innocent Chinweze was sanctioned $1,000 after Copilot fabricated seven cases in a filing
Attorney Innocent O. Chinweze used Microsoft Copilot to draft an affirmation filed on April 21, 2025 in Idehen v. Stoute-Phillip that cited seven nonexistent cases. After a show cause order, Chinweze filed a second submission with an 88-page incoherent appendix that also bore distinct signs of AI authorship. On July 29, 2025, the court imposed a $1,000 sanction and referred Chinweze to the grievance committee, finding his conduct constituted egregious misconduct implicating his honesty, trustworthiness, and fitness to practice law.
- Confidence
- High (multi-source, primary)
A federal judge disqualified attorneys at a major firm over AI-hallucinated citations
In Johnson v. Dunn, a federal judge in Alabama found a large law firm had filed a motion containing hallucinated AI citations and concluded that monetary sanctions were no longer an effective deterrent. The court disqualified the responsible attorneys from the case and referred them to bar regulators.
- Confidence
- High (multi-source, primary)
Butler Snow LLP AI hallucination leads to disqualification in Johnson v. Dunn (N.D. Alabama)
Public reporting confirms that Butler Snow LLP faced sanctions for AI-generated hallucinated citations in Johnson v. Dunn, with the court disqualifying the firm’s attorneys and referring the matter for disciplinary action; multiple sources corroborate the event and its legal implications.
- Confidence
- High (multi-source, primary)
Judge Henry Wingate's staff used AI to draft TRO with hallucinated quotes
A law clerk for Judge Henry Wingate used generative AI to draft a TRO containing fabricated quotes and inaccuracies; the order was rescinded after errors were exposed, and the incident prompted a Senate Judiciary Committee inquiry.
- Confidence
- Medium (multi-source)
Deloitte refunded the Australian government after an AI-assisted report cited fake sources
A A$440,000 report Deloitte submitted to the Australian Department of Employment included fake academic sources and a fabricated quote from a federal court judgment. Deloitte refunded part of the contract.
- Confidence
- Medium (multi-source)
Judge Julien Xavier Neals withdraws CorMedix opinion after AI hallucinations
US District Judge Julien Xavier Neals withdrew a CorMedix opinion after discovering AI-generated errors, including fictitious quotes and misstatements, with withdrawal attributed to a law student intern using ChatGPT and inadequate human review.
- Confidence
- Medium (multi-source)
The UK High Court warned all lawyers to stop misusing AI after five hallucinated citations
In Ayinde v London Borough of Haringey, a pupil barrister at Haringey Law Centre cited five non-existent legal authorities in court filings, suspected to have been generated by AI tools without verification. Dame Victoria Sharp, President of the King's Bench Division, issued a profession-wide warning that lawyers misusing AI could face contempt of court or criminal charges for perverting the course of justice. The ruling also addressed a companion case, Al-Haroun v Qatar National Bank, where 18 of 45 cited authorities were fictitious.
- Confidence
- High (multi-source, primary)
UK High Court warns lawyers against AI misuse after fake citations
The UK High Court warned lawyers to stop the misuse of AI after fake case-law citations appeared in court filings, with Dame Victoria Sharp flagging potential sanctions.
- Confidence
- Medium (multi-source)
Sonio Detect AI ultrasound software mislabels fetal structures in prenatal imaging
Sonio Detect AI mislabels fetal anatomy in prenatal ultrasound, with a MAUDE adverse event entry and Reuters reporting; Samsung Medison says the FDA report does not indicate a safety issue and no action was requested.
- Confidence
- High (multi-source, primary)
Richard Bednar sanctioned by Utah appeals court for fake ChatGPT citations
Lawyer Richard Bednar was sanctioned by the Utah Court of Appeals for filing a petition containing fabricated legal citations generated by ChatGPT. The court found that the attorney failed his professional duty to verify the accuracy of the AI-generated content.
- Confidence
- Medium (multi-source)
MAHA report on children's health exposed as fabricated with AI-assisted citations
Multiple outlets reported that the MAHA Commission's presidential report included fabricated references and AI-generated markers, prompting updates while keeping core substance.
- Confidence
- Medium (multi-source)
Maren Bam sanctioned for AI-generated hallucinations in federal case
Attorney Bam submitted a brief containing AI-hallucinated citations; a federal judge sanctioned Bam, striking the Opening Brief and revoking her pro hac vice status.
- Confidence
- High (multi-source, primary)
A court struck part of an Anthropic expert declaration after Claude hallucinated a citation
An expert declaration submitted by Anthropic data scientist Olivia Chen in Concord Music Group, Inc. v. Anthropic PBC contained a citation to a nonexistent article from The American Statistician journal, with a fabricated title and inaccurate authors. The citation was generated when Anthropic's attorney ran the declaration through Claude to format footnotes, and the model invented the article name and misattributed authors. U.S. Magistrate Judge Susan van Keulen struck paragraph 9 of the declaration from the record on May 23, 2025.
- Confidence
- High (multi-source, primary)
White House health report included fabricated AI citations
The White House's MAHA report on children's health was found to contain fabricated scientific citations generated by AI. This undermined the report's stated goal of adhering to the gold standard of scientific rigor.
- Confidence
- Medium (multi-source)
A newspaper printed an AI-generated summer reading list of books that don't exist
The Chicago Sun-Times and other papers published a syndicated summer guide whose AI-generated reading list recommended novels that were never written, attributing fake titles to real, well-known authors. The outlets apologized and pulled the supplement.
- Confidence
- Low (single source)
White House MAHA report contains nonexistent studies and AI markers
The White House published a public health report containing fake AI-generated citations and 'oaicite' markers. The incident highlighted a failure in editorial oversight for AI-generated government content.
- Confidence
- Medium (multi-source)
University at Buffalo student graduation risked by Turnitin AI false positive
A student at the University at Buffalo faced graduation delays after Turnitin falsely flagged her work as AI-generated. The event prompted a student-led petition to ban AI detectors on campus.
- Confidence
- High (multi-source, primary)
Coca-Cola AI ad fabricates J.G. Ballard book and quotes
Coca-Cola's "Classic" ad campaign used AI to identify literary mentions of the brand, but the system hallucinated a non-existent book by J.G. Ballard. The ad also misattributed translated interview quotes as the author's prose and misspelled his birthplace.
- Confidence
- Medium (multi-source)
Wired retracted a feature after finding the byline Margaux Blanchard was an AI persona
On May 7, 2025, Wired published a feature article under the byline Margaux Blanchard about couples holding weddings inside Minecraft, but the entire freelancer identity and the story's quoted sources were fabricated using generative AI. The article bypassed Wired's standard fact-checking and senior editorial review, and two commercial AI-detection tools incorrectly classified the text as likely human-written. Wired retracted the story later that month after the writer could not provide standard payment details and further investigation confirmed the fabrication.
- Confidence
- High (multi-source, primary)
Ellis George LLP and K&L Gates LLP sanctioned $31,100 for AI hallucinations
Attorneys from Ellis George LLP and K&L Gates LLP were jointly sanctioned $31,100 for submitting AI-generated citations in Lacey v. State Farm General Ins. Co., with some citations found to be nonexistent and others erroneous. The sanctioning decision was described as a collective debacle by a special master.
- Confidence
- Medium (multi-source)
Jisuh Lee referred for criminal contempt over AI-generated fake citations in Ontario court
Ontario lawyer Jisuh Lee submitted a factum with hallucinated or misattributed citations generated by ChatGPT. After initially denying AI involvement, she admitted using AI, and a court referral to the Attorney General followed for potential contempt.
- Confidence
- Medium (multi-source)
Deloitte Canada report for Newfoundland and Labrador contains AI-generated fake citations
Deloitte Canada produced a 526-page healthcare human resources report for the Government of Newfoundland and Labrador, at a reported cost of about $1.6 million. The report allegedly contained AI-generated fabricated citations, prompting the CPA NL to open an investigation into Deloitte's conduct.
- Confidence
- Medium (multi-source)
MyPillow lawyers were sanctioned for a brief with nearly 30 AI-fabricated citations
In the Coomer v. Lindell defamation case, a federal judge in Colorado found nearly thirty defective citations in a brief filed by Mike Lindell's attorneys: cases that did not exist, misquoted authorities, and decisions attributed to the wrong court. Counsel admitted using generative AI and were sanctioned.
- Confidence
- Medium (multi-source)
Cursor AI support bot fabricates non-existent policy, causing user backlash
Cursor AI's support bot, Sam, hallucinated a restrictive multi-device subscription policy in response to a technical bug. This fabrication led to a wave of user complaints and subscription cancellations before the company corrected the error.
- Confidence
- Medium (multi-source)
Haringey Council homeless application judicial review cites fake law cases
In a judicial review involving a homeless applicant against Haringey Council, the claimant's legal team submitted documents citing five non-existent legal cases. The court found this conduct to be improper, unreasonable, and negligent, referring the legal team to their professional regulators and ordering them to pay wasted costs.
- Confidence
- Medium (multi-source)
Dehghani v. Castro attorneys sanctioned for AI hallucinations
A filing attorney and a freelance attorney in the case of Dehghani v. Castro were sanctioned by a New Mexico federal court for submitting a brief containing AI-generated hallucinations. The court imposed fines, mandatory continuing legal education (CLE) training, and a requirement to self-report the misconduct to their respective state bars.
- Confidence
- High (multi-source, primary)
Attorney Felipe D.J. Millan was fined $1,500 over a brief with 19 AI-fabricated case citations
In Dehghani v. Castro, petitioner's counsel Felipe D.J. Millan purchased a brief from freelance attorney Janelle M. Lewis through the LAWCLERK marketplace for $750. Lewis likely used generative AI to draft the brief, which contained six fabricated case citations and thirteen additional mis-cited cases, then destroyed all work product per LAWCLERK policy. Magistrate Judge Damian L. Martinez sanctioned Millan with a $1,500 fine, mandatory one-hour CLE training on legal ethics or AI in writing, and orders to self-report to the New Mexico and Texas state bars and to report Lewis to the New York bar.
- Confidence
- High (multi-source, primary)
NYPD facial recognition match leads to wrongful arrest of Trevis Williams
Two independent outlets report that NYPD used facial recognition to arrest Trevis Williams, despite height and location discrepancies, leading to jail time before charges were dismissed; advocacy groups are pushing for policy changes.
- Confidence
- Medium (multi-source)
Business Insider pulled two first-person essays under the fabricated byline Margaux Blanchard
In April 2025, Business Insider published two first-person essays under the byline Margaux Blanchard, a persona that did not exist and whose content was AI-generated. The articles were removed in August 2025 after Press Gazette alerted the outlet, and Business Insider stated they did not meet editorial standards and had since bolstered verification protocols. At least six publications in total had published and later removed articles under the same fabricated byline.
- Confidence
- High (multi-source, primary)
Yale EMBA student sues over AI-based exam accusation
A Yale EMBA student sued Yale after an AI detector flagged his final exam, leading to suspension and a failing grade.
- Confidence
- Medium (multi-source)
An airline chatbot gave a passenger a wrong refund policy, echoing the Air Canada problem
Passengers reported that airline and travel-agency chatbots continued to state refund and rebooking policies that did not match the carriers' actual rules, a year after the Air Canada tribunal ruling, showing the hallucinated-policy failure mode persisting across the travel industry.
- Confidence
- Low (single source)
A lawyer faced a $15,000 sanction for AI-fabricated citations across three briefs
In an Indiana ERISA case, a federal magistrate judge recommended a $15,000 sanction against a solo practitioner who filed three briefs containing fake citations generated by AI, including a case that did not exist. The lawyer admitted he relied on generative AI and did not verify the cases.
- Confidence
- Low (single source)
Indiana lawyer faces recommended $15,000 fine for fake AI citations
Attorney Rafael Ramirez was recommended for a $15,000 sanction by a federal magistrate judge in Indiana for submitting briefs with fake AI-generated citations. The lawyer admitted to relying on generative AI without verifying the sources.
- Confidence
- Medium (multi-source)
France's government-backed chatbot Lucie was pulled after three days of absurd answers
Linagora's open source AI chatbot Lucie, developed under the French government's France 2030 investment program, was taken offline on January 25, 2025, just three days after its public launch. Users flooded social media with examples of the bot confidently giving nonsensical answers, including claiming that cows lay eggs, providing recipes for cooking meth, and stating that the square root of a goat is one. Linagora admitted the model had been released prematurely without adequate guardrails or reinforcement learning.
- Confidence
- Medium (multi-source)
Morgan & Morgan lawyers sanctioned for AI-generated fake citations in Wyoming case
Morgan & Morgan attorneys were sanctioned in the District of Wyoming for filing a motion containing eight fabricated case citations generated by an internal AI platform. The court fined three attorneys a total of $5,000 and removed Rudwin Ayala from the case.
- Confidence
- High (multi-source, primary)
Thomas Grant Neusom suspended for two years over AI hallucinated citations
Florida Supreme Court suspended attorney Thomas Grant Neusom for two years due to professional misconduct, with evidence including AI-generated, hallucinated citations in pleadings.
- Confidence
- High (multi-source, primary)
Bloomberg issued at least 36 corrections to AI-generated Terminal news summaries
Bloomberg launched AI-generated bullet-point summaries atop its Terminal and website articles on January 15, 2025, and subsequently had to issue at least 36 corrections for errors including wrong dates, inaccurate figures, and misattributed claims. Specific errors included incorrectly stating when Trump tariff actions would take place and falsely claiming the United Steelworkers opposed a mill owner's plans. Bloomberg stated that 99 percent of AI summaries met editorial standards and that journalists retained full control over whether summaries appeared.
- Confidence
- Medium (multi-source)
Kohls v Ellison: Expert AI declaration excluded for fake citations
In Kohls v Ellison, a Stanford professor submitted an AI‑assisted expert declaration that contained fake citations; the court excluded the declaration and criticized the use of AI in the filing, underscoring the need to verify AI outputs in legal submissions.
- Confidence
- High (multi-source, primary)
Apple Intelligence generated false BBC news headlines, prompting Apple to pull the feature
Apple's notification summaries fabricated news, including a false BBC alert that murder suspect Luigi Mangione had shot himself, plus invented sports and celebrity claims. After repeated complaints from the BBC and others, Apple suspended AI summaries for news apps.
- Confidence
- Medium (multi-source)
A misinformation expert's own court filing contained AI-hallucinated citations
In a Minnesota case about deepfakes and elections, a Stanford misinformation expert submitted a declaration supporting the state that itself contained citations to studies that did not exist, generated by AI. The court declined to consider the declaration after the fake references came to light.
- Confidence
- Medium (multi-source)
ITAT Bengaluru withdraws tax order citing fake AI judgments
The ITAT Bengaluru withdrew a tax order involving Buckeye Trust after discovering it relied on fake legal precedents generated by AI. The incident highlights the risk of using generative AI for legal research without rigorous verification.
- Confidence
- Medium (multi-source)
Amazon's Rufus shopping assistant recommended wrong products and hallucinated nonexistent items
Amazon's generative AI shopping assistant Rufus began directly recommending products with buy buttons but frequently suggested items that did not match user queries, such as non-TV products for gaming TV requests and random gloves for winter running queries. Retailers reported that Rufus hallucinated products that were out of stock or did not exist on Amazon at all. The issue gained public attention after Marketplace Pulse and other outlets documented the pattern in November 2024.
- Confidence
- Medium (multi-source)
Attorney Rafael Ramirez sanctioned for AI hallucinations in HoosierVac case
Attorney Rafael Ramirez was sanctioned $6,000 after filing three briefs containing non-existent citations generated by AI, with the court later reducing the originally recommended $15,000 sanction and referring Ramirez for disciplinary action.
- Confidence
- High (multi-source, primary)
OpenAI Whisper hallucinations in medical settings prompt safety concerns, AP reports
Independent outlets report that OpenAI Whisper can hallucinate in medical transcription, risking inaccurate patient documentation. The AP investigation notes thousands of healthcare workers use Whisper-based tools, highlighting potential safety concerns in high-risk settings.
- Confidence
- Medium (multi-source)
Pieces Technologies settles Texas AG allegations over AI hallucination claims
Pieces Technologies reached a settlement with the Texas Attorney General following allegations that the company made deceptive claims regarding the accuracy of its generative AI clinical documentation tool. The investigation found metrics such as a severe hallucination rate of less than 1 per 100,000 were likely inaccurate.
- Confidence
- High (multi-source, primary)
McDonald's ends IBM AI drive-thru order-taking pilot
McDonald's terminated its global IBM AI drive-thru pilot in June 2024 after widespread order inaccuracies and handling of diverse accents; the project began in 2021 and faced multiple reported mishaps. The partnership with IBM was ended, and coverage notes issues with order accuracy and cross-lane misreads.
- Confidence
- Medium (multi-source)
Turnitin's AI detector falsely flagged thousands of students' original work
Turnitin's AI writing detection tool produced false positive results that identified human-written student submissions as AI-generated, leading universities to open academic misconduct proceedings based primarily on those scores. At Australian Catholic University alone, approximately 6,000 cases were registered in 2024 with roughly 90 percent related to AI allegations, and around one quarter of all referrals were ultimately dismissed. Students bore the burden of proving their innocence by supplying handwritten notes, search histories, and drafts, with transcripts marked as results withheld during investigations lasting six months or more.
- Confidence
- High (multi-source, primary)
Hoodline published AI-generated local news with hallucinated details and fake bylines
Hoodline, a hyperlocal news network owned by Impress3, used AI to generate local news articles containing hallucinated details, fabricated poetic language, and mischaracterized police press releases across dozens of US cities. The articles were attributed to fake bylines with AI-generated headshots and biographies, misleading readers into believing real journalists wrote the stories. CEO Zack Chen defended the practice, calling one fabricated detail a punctuation error and the invented prose an uncommon but not inaccurate storytelling method.
- Confidence
- Medium (multi-source)
Hoodline AI mistakenly accuses San Mateo District Attorney of murder
The AI-powered news network Hoodline published a story falsely accusing the San Mateo District Attorney of murder. The network subsequently corrected the error.
- Confidence
- Medium (multi-source)
Google's AI Overviews told users to put glue on pizza and eat rocks
Soon after Google rolled out AI Overviews in search, the feature surfaced dangerous and absurd answers: telling users to add glue to keep cheese on pizza and to eat a small rock a day. The answers came from the model treating satire and forum jokes as authoritative sources.
- Confidence
- Medium (multi-source)
PLOS ONE retracts blended learning paper for AI generated text
PLOS ONE retracted a research paper on blended learning after discovering evidence of undisclosed AI-generated text. The retraction was triggered by the inclusion of the phrase "regenerate response" and numerous hallucinated references.
- Confidence
- High (multi-source, primary)
Foodstuffs facial recognition misidentifies Māori shopper at Rotorua New World
On 2024-04-02 a Māori woman shopping at New World Westend in Rotorua was approached by store staff and told she had been trespassed after a facial recognition alert from a Foodstuffs trial. The customer offered three forms of photo ID but was still asked to leave; Foodstuffs called it a genuine case of human error and said it reported the incident to the Office of the Privacy Commissioner. Experts and the Privacy Commissioner raised concerns about bias and accuracy in the trialled system, which was trained on international data and not specifically on New Zealand populations.
- Confidence
- High (multi-source, primary)
Wendy's FreshAI drive-thru agent misheard orders and cut customers off mid-sentence
Wendy's deployed FreshAI, a Google Cloud generative AI voice agent, at drive-thru locations beginning with a Columbus, Ohio pilot in June 2023 and expanding to franchisees in 2024. The system frequently misheard orders, cut customers off mid-sentence, failed to process simple customizations like removing a pickle, and interrupted ordering with aggressive upsell suggestions. Customers found the experience so frustrating that some reported permanently driving to farther Wendy's locations that still used human order takers.
- Confidence
- Medium (multi-source)
New York City's small-business chatbot told users to break the law
MyCity, the chatbot launched by the New York City Mayor's office, advised users on how to commit wage theft, fire workers who complained about harassment, and serve food bitten by rats.
- Confidence
- Medium (multi-source)
NYC MyCity AI chatbot gave illegal guidance to small businesses
New York City's MyCity AI chatbot gave illegal advice to businesses regarding housing and labor laws. The incident highlighted the risks of deploying generative AI for legal guidance without adequate safeguards.
- Confidence
- High (multi-source, primary)
NYC AI chatbot tells businesses to break the law
A Microsoft-powered NYC chatbot meant to help small businesses gave legally incorrect guidance, including claims that employers could seize tips and fire employees for reporting sexual harassment. The incident is documented by The Markup, The City, and AP News with follow-up coverage noting misinformation about housing and employment laws.
- Confidence
- Medium (multi-source)
TurboTax's Intuit Assist gave wrong tax advice on over half of test questions, the Post found
Washington Post tech columnist Geoffrey A. Fowler tested TurboTax's Intuit Assist AI chatbot with 16 tax questions and found it gave wrong or irrelevant answers on more than half. Specific failures included recommending incorrect filing statuses and fabricating irrelevant education credit advice when asked about air conditioner tax credits. Even after Intuit updated the software, the chatbot remained unhelpful on a quarter of the questions.
- Confidence
- Medium (multi-source)
H&R Block's AI Tax Assist gave wrong or unhelpful answers to 30%+ of tax questions tested
Washington Post columnist Geoffrey A. Fowler tested H&R Block's AI Tax Assist with tax professionals and found it gave wrong or unhelpful answers to more than 30 percent of questions. Specific errors included advising a single parent to file as Single instead of Head of Household and incorrectly stating the IRS had not addressed cryptocurrency wash sale rules. H&R Block defended the tool by saying the test questions lacked specificity and the bot was curated for common tax scenarios from the prior year.
- Confidence
- Medium (multi-source)
A Massachusetts court sanctioned counsel $2,000 over three filings with AI-fabricated citations
In Smith v. Farwell (Civil Action No. 2282CV01197), the Suffolk Superior Court of Massachusetts ordered plaintiff counsel to pay a $2,000 sanction after three opposition pleadings contained fictitious case citations generated by an AI system. An associate attorney and two recent law graduates used an unidentified AI to draft the filings, and the supervising attorney reviewed them only for style and grammar without verifying the citations. Justice Brian A. Davis found a knowing failure to review under Mass. R. Civ. P. 11 and imposed the sanction on February 12, 2024.
- Confidence
- High (multi-source, primary)
Massachusetts attorney sanctioned for citing AI generated fictitious cases
In a Massachusetts Superior Court case, a lawyer faced sanctions for submitting pleadings containing fictitious AI-generated citations; the ruling underscored the duty to verify AI-generated content before filing.
- Confidence
- High (multi-source, primary)
Met Police facial recognition wrongly matched youth worker Shaun Thompson
In February 2024 Shaun Thompson, a youth advocacy worker, was stopped and questioned after the Metropolitan Police's live facial‑recognition system matched him to a watchlist entry. The encounter lasted around 30 minutes and ended when Thompson produced ID; he subsequently brought a High Court challenge to the Met's use of LFR, which was dismissed on 2026-04-21. Reporting on the case is documented by multiple independent outlets including the BBC and The Independent.
- Confidence
- Medium (multi-source)
AI generated news article falsely quotes Professor Emily Bender
The Indian news website Biharprabha published an AI-generated article that included a fabricated quote attributed to linguistics professor Emily Bender. The quote falsely claimed that Meta's BlenderBot 3 showed the company's struggle with AI bias.
- Confidence
- High (multi-source, primary)
UK GOV.UK Chat gave citizens incorrect tax, VAT, and immigration advice in its alpha pilot
The UK Government Digital Service's GOV.UK Chat prototype produced inaccurate or misleading responses during a private pilot with approximately 1,000 users, scoring only 76% accuracy at its earliest benchmark. The system gave incorrect advice on tax, VAT registration, EU Settlement Scheme, and flight refund matters before GDS added filters to block certain question categories. The Times later reported that the chatbot gave misleading tax information, drawing criticism from tax professionals.
- Confidence
- High (multi-source, primary)
Instacart quietly removed AI-generated recipe photos users found impossible and unappetizing
Instacart deployed AI-generated images alongside recipe content on its platform that contained physically impossible food depictions such as conjoined chickens, hot dogs with tomato interiors, and lemons fused with lettuce. After users flagged the images on Reddit and press coverage ensued, Instacart quietly removed the offending AI images and replaced some with stock photography. The company stated it reviews AI-generated content and may remove it when it does not deliver a high-quality consumer experience.
- Confidence
- Medium (multi-source)
NewsBreak AI fabricates story about Christmas Day murder in New Jersey
NewsBreak used AI to publish a fake news story about a fatal Christmas shooting in New Jersey. Local police had to publicly debunk the report, which the company later attributed to an inaccurate content source.
- Confidence
- Medium (multi-source)
Microsoft Copilot generates inaccurate information about European elections
Microsoft's Copilot chatbot generated false information about Swiss and German elections in December 2023. The system misquoted sources, leading to the dissemination of electoral misinformation.
- Confidence
- Medium (multi-source)
Felicity Harber submitted nine fictitious AI-generated case citations to a UK tribunal
Felicity Harber, a litigant in person appealing an HMRC penalty for failure to notify Capital Gains Tax liability, submitted nine fabricated First-tier Tribunal case citations generated by an AI system such as ChatGPT. The Tribunal found that none of the cited cases existed on any legal database, though they bore superficial similarities to real cases. The Tribunal accepted Harber was unaware the cases were fabricated but dismissed her appeal and warned that citing invented judgments wastes public money and undermines confidence in the judicial system.
- Confidence
- High (multi-source, primary)
Amazon Q chatbot allegedly leaks confidential AWS data and hallucinations
Amazon's AI chatbot, Q, allegedly suffered from severe hallucinations and leaked confidential company data, including data center locations. While internal documents flagged the issue as a significant incident, Amazon spokespeople denied that any confidential information was leaked.
- Confidence
- Medium (multi-source)
Tesla FSD system fails to detect reduced visibility in fatal crash
A fatal accident occurred on November 28, 2023, involving Tesla's Full Self-Driving software during periods of reduced visibility. Federal investigations found the software's degradation detection system failed to recognize impaired camera performance.
- Confidence
- High (multi-source, primary)
Zachariah Crabill suspended for AI-generated hallucinated case law
Attorney Zachariah Crabill was sanctioned by the Colorado bar for submitting a court filing with fake case law generated by ChatGPT. This resulted in a 90-day disciplinary suspension.
- Confidence
- High (multi-source, primary)
Large language models perpetuate racial bias in healthcare
AIAAIC recorded an incident entry (published November 2023) documenting that large language models (LLMs) have produced racially biased outputs in healthcare contexts. Independent academic audits and studies (including a 2024 audit titled "Unmasking and Quantifying Racial Bias of Large Language Models") found LLMs gave systematically different clinical-related recommendations and projections across racial groups. These outputs have the potential to cause harm when used in clinical decision-making by healthcare deployers.
- Confidence
- High (multi-source, primary)
Plainfield Police Department predictive policing software fails to predict crimes
The Markup and Wired reported that Geolitica's predictive policing software for Plainfield PD produced thousands of predictions with a success rate under 1 percent across 23,631 predictions, and the department stopped using it.
- Confidence
- Medium (multi-source)
ChatGPT fabricates academic citations for biologist Henrik Enghoff
A scientific preprint about millipedes, authored using ChatGPT, included several fake academic references attributed to biologist Henrik Enghoff. Enghoff discovered the fabrications when he noticed his name linked to papers he had never written.
- Confidence
- Medium (multi-source)
Gannett pauses 2023 AI high school sports recap tool after placeholders appeared
In August 2023, Gannett paused its AI tool Lede AI used to generate high school sports recaps after articles showed data-coverage errors, including placeholder text like [[WINNING_TEAM_MASCOT]]. The incident was documented by Axios, The Washington Post, and Morning Brew.
- Confidence
- Medium (multi-source)
Gannett paused an AI sports-writing tool after garbled, error-filled local articles
The newspaper chain Gannett halted use of an AI tool called LedeAI after it produced robotic, error-strewn high-school sports recaps that went viral for phrases like describing a game as a 'close encounter of the athletic kind' and leaving placeholder text in published stories.
- Confidence
- Low (single source)
Pak'nSave Savey Meal-bot suggests recipes using toxic household chemicals
Pak'nSave's AI-powered Savey Meal-bot generated hazardous recipes, including a mixture creating chlorine gas, when users input non-food household items. The AI failed to recognize the danger of the ingredients, treating them as edible components for a meal planner.
- Confidence
- Medium (multi-source)
G/O Media's AI-generated Star Wars article on Gizmodo had at least 18 factual errors
G/O Media used AI chatbots to generate and auto-publish a Star Wars article on Gizmodo that contained at least 18 factual errors, including a chronological movie list that was not in chronological order and omitted several titles. The article was published under the byline Gizmodo Bot with no human editorial review, and deputy editor James Whitbrook identified the errors immediately upon publication. The GMG Union publicly condemned the articles as unethical and unacceptable, and Gizmodo appended a correction the following day.
- Confidence
- Medium (multi-source)
California attorney fined $10,000 for filing appeal with fake AI citations
The California appeals court fined Amir Mostafavi $10,000 after discovering 21 of 23 quotes in the opening brief were fabricated by ChatGPT. The ruling serves as a warning to lawyers about the dangers of submitting unverified AI content.
- Confidence
- Medium (multi-source)
S.D.N.Y. sanctions attorneys for using fake ChatGPT citations
Attorneys in the Mata v. Avianca case submitted legal briefs containing non-existent case citations generated by ChatGPT. The court issued a $5,000 sanction against the lawyers for their failure to verify the AI-generated content.
- Confidence
- High (multi-source, primary)
Levidow, Levidow and Oberman sanctioned for ChatGPT fabricated citations
Attorneys Schwartz and LoDuca of Levidow, Levidow & Oberman used ChatGPT to generate legal research, which produced six fake judicial opinions. The court sanctioned the firm and the attorneys with a $5,000 fine after the fabricated citations were discovered.
- Confidence
- Medium (multi-source)
Stack Overflow overwhelmed by AI-generated answers and moderator strike
Stack Overflow faced a surge of AI-generated, low-quality answers that overwhelmed both automated detection and volunteer moderation. The situation led to a public moderation strike on June 5, 2023 and prompted company-community negotiations after prior temporary measures such as a ChatGPT answer ban.
- Confidence
- High (multi-source, primary)
ChatGPT invented an embezzlement claim, prompting a first-of-its-kind libel suit
Radio host Mark Walters sued OpenAI for libel after ChatGPT, asked to summarize a real lawsuit, fabricated a claim that Walters had embezzled from a nonprofit. He had no connection to the case. It was among the first defamation suits over an AI hallucination.
- Confidence
- Medium (multi-source)
Lawyers cited six fake cases generated by ChatGPT in federal court
In Mata v. Avianca, two attorneys filed a brief citing six judicial decisions that did not exist. ChatGPT had fabricated them. The court sanctioned the lawyers and the case became the inflection point for legal AI policy.
- Confidence
- High (multi-source, primary)
ChatGPT falsely named an Australian mayor as a convicted briber
Brian Hood, a regional Australian mayor, threatened to sue OpenAI after ChatGPT described him as a convicted criminal in a bribery scandal. In reality Hood was the whistleblower who exposed the scheme, not a participant, making it an early defamation threat over a chatbot hallucination.
- Confidence
- Low (single source)
USCIS AI translation errors in Pashto jeopardize Afghan asylum claims
US Citizenship and Immigration Services (USCIS) and its contractors relied on AI translation tools for Afghan refugee asylum claims, leading to critical errors in Pashto and Dari translations. These inaccuracies resulted in discrepancies that led to the denial of asylum claims.
- Confidence
- Medium (multi-source)
Snap Inc. My AI chatbot produced toxic outputs and faced UK regulatory probe
Snap launched My AI in February 2023, which subsequently produced problematic outputs and hallucinations. This led to an investigation by the UK's ICO regarding child privacy and safety guardrails, and the company acknowledged non-conforming language in internal reviews and outlined safety enhancements.
- Confidence
- High (multi-source, primary)
Detroit police facial recognition misidentified a pregnant woman, causing a wrongful arrest
On February 16, 2023, Detroit police arrested Porcha Woodruff, who was eight months pregnant, after DataWorks Plus facial recognition software matched her to surveillance footage of a carjacking and robbery suspect. She was held for approximately 11 hours at the Detroit Detention Center before being released on a $100,000 personal bond, and the criminal case was dismissed on March 6, 2023 for insufficient evidence. Woodruff filed a federal civil rights lawsuit in August 2023, which was dismissed in September 2025 after the judge ruled the detective had probable cause at the time of the arrest.
- Confidence
- High (multi-source, primary)
A factual error in Google Bard's launch demo wiped about $100B in market value
In its first public demo, Google's Bard claimed the James Webb Space Telescope took the first image of an exoplanet, which was wrong. The visible error in the launch ad contributed to a 7-8% drop in Alphabet's stock, erasing roughly $100 billion in market value in a day.
- Confidence
- Medium (multi-source)
Mens Journal AI-generated health story cited for numerous inaccuracies in 2023
Two independent outlets documented that Men's Journal published an AI-generated health article containing inaccuracies, followed by corrections and editor notes, with experts noting mischaracterizations of medical science.
- Confidence
- Medium (multi-source)
Bankrate paused its AI personal-finance articles after they ran factual errors
Bankrate, owned by Red Ventures, published AI-generated personal finance explainers that contained factual errors including an incorrect claim that a 5/1 ARM is definitively a 30-year mortgage, garbled text, and misleading omissions about the risks of adjustable-rate mortgages. Red Ventures announced a pause of the AI content program on January 20, 2023, after widespread media coverage of the errors, though Bankrate quietly continued publishing AI articles after the stated suspension. The company rolled back error-ridden articles to prior human-written versions after being contacted by reporters.
- Confidence
- Medium (multi-source)
CNET quietly published AI-written finance articles riddled with errors
The tech outlet CNET published dozens of personal-finance articles generated by an AI tool without clearly disclosing it. Reviewers found factual errors in a majority of them, and CNET had to issue corrections and pause the program amid criticism of accuracy and plagiarism.
- Confidence
- Medium (multi-source)
Randal Quran Reid wrongfully arrested due to facial recognition misidentification
Randal Quran Reid was wrongfully arrested in Georgia due to a facial recognition error by the Jefferson Parish Sheriff's Office. The agency relied on an incorrect match without verifying if the subject had ever visited Louisiana. The incident led to a lawsuit and a subsequent $200,000 settlement.
- Confidence
- Medium (multi-source)
VioGén risk-assessment used by Spanish National Police misclassified victims
An academic review and investigative reporting documented transparency, accuracy, and governance problems with VioGén, the Spanish police risk-assessment tool overseen by the Interior Ministry. Reporting and analyses found that the system classified many cases as negligible or low risk and that some victims later suffered repeat attacks or were killed, prompting rights and oversight concerns.
- Confidence
- High (multi-source, primary)
Meta pulled its Galactica science AI after three days of confident fabrications
Meta released Galactica, a language model meant to summarize science, and took it down within three days after it generated authoritative-sounding but false papers, citations, and wiki entries, including fabricated science attributed to real researchers.
- Confidence
- Low (single source)
Air Canada ordered to honor refund its chatbot invented
A British Columbia tribunal ruled that Air Canada was bound by a bereavement-fare policy its chatbot fabricated. The airline argued the bot was a separate legal entity. The tribunal disagreed.
- Confidence
- High (multi-source, primary)
Acclarent TruDi AI navigation system allegedly causes carotid artery injuries
The Acclarent TruDi AI navigation system allegedly misled surgeons during sinus operations, resulting in carotid artery punctures and strokes. FDA malfunction reports reportedly rose after AI integration in 2021, and two patients filed Texas lawsuits alleging AI contributed to injuries.
- Confidence
- Medium (multi-source)
Epic's sepsis prediction model missed two-thirds of cases with 88% false alarms, a study found
The Epic Sepsis Model, a proprietary sepsis prediction algorithm embedded in Epic's electronic health record platform and deployed at hundreds of US hospitals, was found to miss 67% of sepsis cases while generating 88% false alarms in an independent external validation published in JAMA Internal Medicine in June 2021. The model's discrimination (AUC 0.63) was substantially worse than Epic's claimed performance (AUC 0.76 to 0.83). Epic subsequently overhauled the model in 2022, changing its sepsis definition, reducing reliance on antibiotic orders, and recommending site-specific training before clinical use.
- Confidence
- High (multi-source, primary)
Medtronic AccuRhythm AI misses abnormal rhythms in LINQ monitors, per FDA and Reuters
Between 2021 and 2025, at least 16 FDA adverse event reports alleged that Medtronic's AccuRhythm AI in LINQ monitors failed to detect abnormal heart rhythms. Medtronic said it reviewed the cases and found only one missed abnormal event, attributing others to data display issues or user confusion; no patient harm was reported.
- Confidence
- High (multi-source, primary)
Aurora police ALPR false match led to family detained at gunpoint
In early August 2020 Aurora, Colorado officers stopped a Black mother and several children after an Automated License Plate Reader reportedly flagged the family's vehicle as matching a stolen motorcycle registered in another state. Officers conducted a high-risk stop, drew weapons, and several children were handcuffed; officers later determined the vehicle was not stolen. The City of Aurora reached a $1.9 million settlement with the family in February 2024.
- Confidence
- Medium (multi-source)
Babylon Health symptom checker alleged to miss or downplay critical symptoms
Multiple news investigations and clinicians' tests in 2019-2021 documented examples where Babylon Health’s symptom checker produced unsafe or inappropriate triage recommendations for serious symptoms. The UK regulator MHRA told a clinician who raised concerns that it shared those concerns, and Babylon acknowledged some errors in examples highlighted by critics.
- Confidence
- Medium (multi-source)
Reno police facial recognition misidentified an innocent man, leading to a $100,000 settlement
Reno Police Department used DataWorks Plus facial recognition software to match a surveillance photo to an innocent individual, resulting in a wrongful arrest. The City of Reno settled the resulting civil rights lawsuit for $100,000 and agreed to policy changes restricting facial recognition use. The department had no formal training or policies governing facial recognition technology at the time of the incident, and also maintained documented use of Clearview AI for separate searches.
- Confidence
- Medium (multi-source)
US government asylum claim denied due to AI translation error
A Pashto-speaking refugee's asylum bid was rejected by a US court after an AI translation tool incorrectly changed "I" to "we" in her written statement. This created a perceived contradiction with her oral testimony, leading to the denial of her asylum claim.
- Confidence
- Medium (multi-source)
Bank of America fined $225 million for faulty automated fraud filter on unemployment cards
Federal regulators fined Bank of America $225 million for botching the disbursement of state unemployment benefits at the height of the pandemic. The bank’s faulty automated fraud detection program allegedly froze legitimate accounts, denying some beneficiaries access to funds.
- Confidence
- High (multi-source, primary)
Goldman Sachs Apple Card underwriting model investigated for perceived gender bias
Goldman Sachs Bank USA's Apple Card underwriting faced a regulatory inquiry. The NYDFS found no evidence of disparate impact but criticized transparency and customer communication around the algorithmic decisions.
- Confidence
- High (multi-source, primary)
Woodbridge Police Department wrongfully arrests man via facial recognition
The Woodbridge Police Department arrested Nijeer Parks for shoplifting after facial recognition software incorrectly identified him as a suspect. Parks was jailed for ten days despite being 30 miles away during the crime.
- Confidence
- Medium (multi-source)
Buenos Aires facial recognition system causes numerous wrongful arrests
The City of Buenos Aires implemented an AI facial recognition system for public security that resulted in over 140 false identifications and wrongful detentions. This led to a legal battle and a court ruling that declared the program's implementation unconstitutional.
- Confidence
- Medium (multi-source)
Apple alleged to have misidentified Ousmane Bah in store surveillance
A lawsuit filed in April 2019 alleges that Apple’s in‑store security system associated surveillance images of a shoplifter with Ousmane Bah, leading to his arrest on November 29, 2018. Independent news outlets reported the suit and Apple told reporters it does not use facial recognition in its stores. The court docket and complaint are publicly available.
- Confidence
- High (multi-source, primary)
IBM Watson for Oncology provided unsafe cancer treatment recommendations
IBM Watson for Oncology provided clinically unsafe and incorrect treatment recommendations to healthcare providers. The system allegedly suggested dangerous treatments, such as bleeding drugs for patients with severe hemorrhage.
- Confidence
- Medium (multi-source)
Facebook translation error leads to arrest of Palestinian man
In October 2017 Israeli police arrested and later released a Palestinian man after relying on an automatic translation of his Arabic Facebook post that reportedly rendered a benign caption as a violent phrase in Hebrew. Multiple news outlets reported that police used the platform's translation output when assessing the post. The incident drew attention to risks from automatic translation in law enforcement contexts.
- Confidence
- Medium (multi-source)
Metropolitan Police facial recognition trial at Notting Hill Carnival reports 98 percent error rate
The Metropolitan Police Service deployed live facial recognition technology during the 2017 Notting Hill Carnival. An audit later revealed that the system incorrectly identified the vast majority of potential matches.
- Confidence
- High (multi-source, primary)
Google Translate deemed inadequate for obtaining search consent in US federal court
In the case of United States v. Cruz-Zamora, a federal judge ruled that Google Translate's inaccuracy made it an insufficient tool for officers to obtain unequivocal consent for a warrantless search. This ruling led to the suppression of narcotics seized during the stop.
- Confidence
- Medium (multi-source)
New Zealand passport photo checker rejects applicant's open eyes as closed
In December 2016 an online passport photo checker run by New Zealand's Department of Internal Affairs rejected a photo from Richard Lee, a New Zealander of Asian descent, with the generic error "subject eyes are closed" even though his eyes were open. Major news outlets reported the system later accepted a different photo and the department said shadowing and uneven lighting commonly cause such automatic rejections.
- Confidence
- Medium (multi-source)