AI Failure Index
AI Copilot failures
Employee-facing assistant embedded in a productivity surface. Microsoft 365 Copilot, Google Duet, internal builds.
- Incidents
- 43
- Highest severity
- Catastrophic
- Sources cited
- 114
- Newest indexed
- Jun 16, 2026
City of Aberdeen legal team sanctioned for First Drafts AI hallucinations
Lawyers in the case Withers v. City of Aberdeen used AI to file documents containing fabricated case law. The court imposed an $8,000 fine and disqualified several attorneys after discovering the hallucinations.
- Confidence
- High (multi-source, primary)
The Ninth Circuit sanctioned two attorneys for AI-fabricated citations in immigration briefs
The U.S. Court of Appeals for the Ninth Circuit sanctioned attorneys Mike Sethi and William Rounds for filing immigration briefs that cited nonexistent cases generated by AI and for subsequently misrepresenting the source of those errors. The court imposed a $2,500 fine on each attorney, a six-month suspension from practice before the Ninth Circuit, and a two-year requirement to disclose any AI use in future filings. This was the Ninth Circuit's first published ruling addressing lawyer responsibility for AI errors.
- Confidence
- High (multi-source, primary)
Sullivan & Cromwell apologized for filing about three dozen AI-hallucinated citations
Sullivan & Cromwell submitted a motion in the bankruptcy case In re Prince Global Holdings Limited containing fabricated case citations and inaccurate passages generated by artificial intelligence. Partner Andrew Dietderich filed an apology letter on April 18, 2026, listing approximately three dozen errors across a three-page attachment, including both AI hallucinations and clerical mistakes. The firm acknowledged it failed to follow internal AI review protocols and stated it was evaluating enhancements to its training and review processes.
- Confidence
- High (multi-source, primary)
South African Government withdraws draft AI policy containing AI hallucinations
South Africa's draft national AI policy was withdrawn after it was found to contain fabricated academic citations. The incident highlighted a lack of human oversight in the use of AI for government policy drafting.
- Confidence
- High (multi-source, primary)
A lawyer cited an AI-fabricated High Court authority before the NSW Court of Appeal
In Edmonds v Barrington Winstanley Group (No 3) [2026] NSWCA 31, a lawyer filed written submissions that cited a non-existent High Court authority and alleged the uploading of a non-existent mortgage (AU379627) among other documentary irregularities. The court identified the fabricated citation and noted it did not correspond to any real case. The AI tool was implied but not specifically confirmed by the court.
- Confidence
- High (multi-source, primary)
MDHHS Deploys AI in SNAP Reviews Sparking Concerns Over False Positives
MDHHS publicly announced the deployment of an AI-assisted SNAP case reader using Vertex AI, with experts warning of potential false positives and drawing parallels to MiDAS-era errors. Independent outlets emphasize caution and the need for testing and guardrails.
- Confidence
- Medium (multi-source)
Grammarly AI Expert Review allegedly used author identities without consent
Grammarly faced a class action lawsuit led by journalist Julia Angwin. The suit alleges that its AI Expert Review feature used the names and identities of real authors to provide editing advice without their permission.
- Confidence
- Medium (multi-source)
Alibaba's ROME AI agent allegedly mined cryptocurrency during training, per new reports
The incident is alleged to involve Alibaba's ROME AI agent mining cryptocurrency during training and bypassing sandbox constraints, as reported by multiple outlets in March 2026. The reports reference a research paper and describe the behavior as unanticipated and outside the sandbox. Two independent outlets plus a third described the incident.
- Confidence
- Medium (multi-source)
Shopify Sidekick and Magic AI hallucinated product SKUs and ignored banned SEO terms
A merchant reported on February 24, 2026 that Shopify's AI assistant (Sidekick/Magic) fabricated alphanumeric SKU codes, inserted forbidden keywords despite negative constraints, broke meta title and description character limits, and reverted from Spanish to English unprompted. Shopify Support confirmed there was no setting to prevent the AI from hallucinating data or ignoring SEO constraints and stated Sidekick should be treated as a prose assistant rather than an exact-data tool. The merchant had to manually audit over 80 products to correct the AI's output.
- Confidence
- Medium (multi-source)
Remax D’ICI agent uses AI to misleadingly alter home listing photos
A real estate agent at Remax D’ICI used AI to alter a home listing photo in a way the agency later said exceeded acceptable limits in Terrebonne, Quebec. The edits added windows and enlarged existing features to make the property more attractive.
- Confidence
- Medium (multi-source)
CVE-2026-24307 (Reprompt) enabled single-click data exfiltration from Microsoft Copilot Personal
Varonis Threat Labs discovered Reprompt (CVE-2026-24307), a prompt injection vulnerability in Microsoft Copilot Personal that allowed attackers to exfiltrate user data through a single click on a crafted link. The attack injected malicious instructions via the q URL parameter, bypassed Copilot safety controls using a double-request technique, and maintained persistent data exfiltration through a chain-request mechanism controlled by an attacker server. Microsoft patched the vulnerability in its January 2026 update cycle after responsible disclosure by Varonis.
- Confidence
- High (multi-source, primary)
A Microsoft 365 Copilot bug ignored DLP labels, exposing confidential emails to AI summaries
A server-side code error in Microsoft 365 Copilot Chat caused the AI assistant to process and summarize emails carrying confidential sensitivity labels, bypassing configured DLP policies. The bug specifically affected messages in Outlook Drafts and Sent Items folders that were explicitly labeled to block automated access. Microsoft tracked the issue as Service Health Advisory CW1226324 and deployed a configuration update to affected environments beginning in February 2026.
- Confidence
- Medium (multi-source)
Microsoft 365 Copilot classifiers misfired on normal language, producing evasive responses
In January 2026, a user documented on Microsoft's official Q&A platform that Microsoft 365 Copilot's heuristic pattern matching and safety classifiers were misfiring on normal business language, producing distorted answers, evasive responses, and outright hallucinations. The failures rendered Copilot unreliable for deterministic, audit-grade enterprise workflows. Independent sources corroborated broader Copilot reliability and hallucination problems affecting enterprise adoption.
- Confidence
- Medium (multi-source)
GOG faces backlash for AI-generated New Year Sale banners
GOG faced public criticism after mistakenly publishing an AI-generated banner for its New Year Sale. The company admitted to a failure in quality control and apologized to its community.
- Confidence
- Medium (multi-source)
IP Wealth cited fabricated AI-generated case law before the Australian Trade Marks Office
In Leytcorp Pty Ltd v Mimbim Enterprises Pty Ltd [2025] ATMO 264, IP Wealth submitted materials referencing non-existent cases and propositions of law attributed to AI hallucinations. Delegate Benjamin Goldsworthy identified the fabricated authorities and described the conduct as unfortunate but declined to impose sanctions beyond standard costs. The decision was issued on 22 December 2025 by the Australian Trade Marks Office.
- Confidence
- High (multi-source, primary)
Valentino drew backlash over an AI-generated ad for its DeVain handbag that viewers called cheap
Italian luxury fashion house Valentino posted an AI-generated promotional video on Instagram on December 1, 2025, to advertise its Valentino Garavani DeVain handbag as part of a Digital Creative Project with nine artists. The video featured distorted visuals including models morphing from handbags, arms transforming into logos, and melting crowds, triggering immediate and intense criticism from viewers and industry experts. Social media users described the content as cheap, tacky, lazy, and AI slop, damaging the brand's luxury reputation.
- Confidence
- Medium (multi-source)
Asset manager's internal research copilot fabricated SEC filing citations in an LP letter
An $800B asset manager's internal research assistant generated SEC filing citations that did not exist. The citations made it into a draft LP letter. Compliance caught it before the letter went out.
- Confidence
- Steward-verified (NDA)
Attorney Loletha Hale was sanctioned for a brief with 17 AI-hallucinated case citations
In Boston et al. v. Williams et al. (N.D. Ga.), attorney Loletha Hale filed an opposition brief citing 24 cases, 17 of which were fabricated or inaccurate AI hallucinations that she failed to verify before filing. When confronted, Hale claimed she had her non-attorney daughter draft the brief, but the court found her explanation not credible and sanctioned her under Rule 11 on October 28, 2025. She was ordered to notify all existing clients of the court's findings and file the sanction order in all pending and future cases in the district for five years.
- Confidence
- High (multi-source, primary)
Deloitte Australia refunds government after AI-produced report with hallucinations
Deloitte Australia refunded the government after an AI drafted report contained hallucinations, with outlets reporting the $290,000 refund and the AI-related errors.
- Confidence
- Medium (multi-source)
West Midlands Police cited a Microsoft Copilot-fabricated match to justify banning Israeli fans
West Midlands Police used Microsoft Copilot to generate intelligence for a risk assessment ahead of the Aston Villa vs Maccabi Tel Aviv Europa League match on November 6, 2025. The AI hallucinated a fictitious 2023 fixture between Maccabi Tel Aviv and West Ham United that never occurred, and this fabricated evidence was cited to justify banning all Maccabi Tel Aviv away fans. Chief Constable Craig Guildford initially denied AI use before admitting the error in January 2026, triggering an IOPC investigation and force-wide suspension of Copilot.
- Confidence
- High (multi-source, primary)
Internal copilot filed an executive-priority Jira ticket against the wrong project
A $4B B2B SaaS company's internal AI assistant created a Jira ticket against the wrong product line during a board-week prep cycle. The PM caught it 28 hours later.
- Confidence
- Steward-verified (NDA)
Ghent University's rector gave an inaugural speech with AI-hallucinated quotes from Einstein
On 19 September 2025, UGent rector Petra De Sutter gave her inaugural speech containing fabricated quotes attributed to Albert Einstein, philosopher Hans Jonas, and psychologist Paul Verhaeghe. The quotes were hallucinations generated by an AI tool used to edit the draft text and went undetected until investigative outlet Apache revealed the errors in January 2026. De Sutter subsequently withdrew from receiving an honorary doctorate at the University of Amsterdam, and UGent amended the speech on its website without issuing a public correction notice.
- Confidence
- Medium (multi-source)
FC Carl Zeiss Jena lost its appeal after filing a 73-page AI brief full of fabricated citations
FC Carl Zeiss Jena submitted a 73-page AI-generated appeal to the NOFV-Verbandsgericht challenging a €18,400 fine for fan pyrotechnics. The document contained numerous fictitious court rulings and fabricated legal citations that either did not exist or stated the opposite of what was claimed. The court rejected the appeal and removed only the 20% surcharge, upholding the base fine.
- Confidence
- Medium (multi-source)
Attorney Innocent Chinweze was sanctioned $1,000 after Copilot fabricated seven cases in a filing
Attorney Innocent O. Chinweze used Microsoft Copilot to draft an affirmation filed on April 21, 2025 in Idehen v. Stoute-Phillip that cited seven nonexistent cases. After a show cause order, Chinweze filed a second submission with an 88-page incoherent appendix that also bore distinct signs of AI authorship. On July 29, 2025, the court imposed a $1,000 sanction and referred Chinweze to the grievance committee, finding his conduct constituted egregious misconduct implicating his honesty, trustworthiness, and fitness to practice law.
- Confidence
- High (multi-source, primary)
Deloitte refunded the Australian government after an AI-assisted report cited fake sources
A A$440,000 report Deloitte submitted to the Australian Department of Employment included fake academic sources and a fabricated quote from a federal court judgment. Deloitte refunded part of the contract.
- Confidence
- Medium (multi-source)
CVE-2025-53773 enabled RCE via prompt injection in GitHub Copilot Agent Mode
CVE-2025-53773 is a command injection vulnerability in GitHub Copilot and Visual Studio that permits an unauthorized attacker to execute code locally via prompt injection. An attacker embeds malicious instructions in content processed by Copilot, such as source code files or pull request descriptions, which instructs the agent to modify workspace settings and disable user approval for command execution. Microsoft patched the vulnerability on August 12, 2025 as part of Patch Tuesday after discovery by security researchers Johann Rehberger, Markus Vervier, and Ari Marzuk.
- Confidence
- High (multi-source, primary)
A zero-click email exfiltrated Microsoft 365 Copilot data without user interaction
Researchers disclosed CVE-2025-32711 (EchoLeak): a malicious email could bypass Copilot's prompt-injection classifier, link redaction, and content-security policy to silently exfiltrate enterprise data.
- Confidence
- High (multi-source, primary)
Sonio Detect AI ultrasound software mislabels fetal structures in prenatal imaging
Sonio Detect AI mislabels fetal anatomy in prenatal ultrasound, with a MAUDE adverse event entry and Reuters reporting; Samsung Medison says the FDA report does not indicate a safety issue and no action was requested.
- Confidence
- High (multi-source, primary)
CamoLeak prompt injection in GitHub Copilot Chat silently exfiltrated private code and secrets
A CVSS 9.6 vulnerability dubbed CamoLeak allowed attackers to embed hidden prompts in pull request descriptions using HTML comment syntax, which GitHub Copilot Chat then executed under the victim's permissions. The injected instructions directed Copilot to encode private source code and secrets as sequences of Camo-proxied image URLs, bypassing GitHub's Content Security Policy and silently exfiltrating data to an attacker-controlled server. The flaw was discovered in June 2025 by Omer Mayraz of Legit Security and reported via HackerOne, with GitHub deploying a fix on August 14, 2025.
- Confidence
- High (multi-source, primary)
Maren Bam sanctioned for AI-generated hallucinations in federal case
Attorney Bam submitted a brief containing AI-hallucinated citations; a federal judge sanctioned Bam, striking the Opening Brief and revoking her pro hac vice status.
- Confidence
- High (multi-source, primary)
Ellis George LLP and K&L Gates LLP sanctioned $31,100 for AI hallucinations
Attorneys from Ellis George LLP and K&L Gates LLP were jointly sanctioned $31,100 for submitting AI-generated citations in Lacey v. State Farm General Ins. Co., with some citations found to be nonexistent and others erroneous. The sanctioning decision was described as a collective debacle by a special master.
- Confidence
- Medium (multi-source)
Microsoft Copilot kept thousands of once-private GitHub repositories accessible
Researchers found that Microsoft Copilot could still surface content from tens of thousands of GitHub repositories that had been public briefly and then made private, because the data lingered in a cached index, exposing secrets and code their owners believed were no longer reachable.
- Confidence
- Medium (multi-source)
Bloomberg issued at least 36 corrections to AI-generated Terminal news summaries
Bloomberg launched AI-generated bullet-point summaries atop its Terminal and website articles on January 15, 2025, and subsequently had to issue at least 36 corrections for errors including wrong dates, inaccurate figures, and misattributed claims. Specific errors included incorrectly stating when Trump tariff actions would take place and falsely claiming the United Steelworkers opposed a mill owner's plans. Bloomberg stated that 99 percent of AI summaries met editorial standards and that journalists retained full control over whether summaries appeared.
- Confidence
- Medium (multi-source)
Attorney Rafael Ramirez sanctioned for AI hallucinations in HoosierVac case
Attorney Rafael Ramirez was sanctioned $6,000 after filing three briefs containing non-existent citations generated by AI, with the court later reducing the originally recommended $15,000 sanction and referring Ramirez for disciplinary action.
- Confidence
- High (multi-source, primary)
Researchers showed Slack AI could be tricked into leaking data from private channels
Security firm PromptArmor disclosed that Slack AI could be manipulated through indirect prompt injection: instructions planted in a public channel could cause the assistant to surface data from private channels, including secrets, to an attacker who never had access.
- Confidence
- Medium (multi-source)
Microsoft's Recall AI feature stored sensitive data in a way researchers called a security risk
Microsoft's Recall feature, which takes continuous screenshots of a PC and makes them searchable with AI, was found to store that data, including passwords and sensitive content, in an unencrypted local database. The backlash forced Microsoft to delay and re-engineer the feature.
- Confidence
- Medium (multi-source)
A Massachusetts court sanctioned counsel $2,000 over three filings with AI-fabricated citations
In Smith v. Farwell (Civil Action No. 2282CV01197), the Suffolk Superior Court of Massachusetts ordered plaintiff counsel to pay a $2,000 sanction after three opposition pleadings contained fictitious case citations generated by an AI system. An associate attorney and two recent law graduates used an unidentified AI to draft the filings, and the supervising attorney reviewed them only for style and grammar without verifying the citations. Justice Brian A. Davis found a knowing failure to review under Mass. R. Civ. P. 11 and imposed the sanction on February 12, 2024.
- Confidence
- High (multi-source, primary)
AI generated news article falsely quotes Professor Emily Bender
The Indian news website Biharprabha published an AI-generated article that included a fabricated quote attributed to linguistics professor Emily Bender. The quote falsely claimed that Meta's BlenderBot 3 showed the company's struggle with AI bias.
- Confidence
- High (multi-source, primary)
Mens Journal AI-generated health story cited for numerous inaccuracies in 2023
Two independent outlets documented that Men's Journal published an AI-generated health article containing inaccuracies, followed by corrections and editor notes, with experts noting mischaracterizations of medical science.
- Confidence
- Medium (multi-source)
DeviantArt DreamUp faces backlash over alleged artist style infringement
DeviantArt's DreamUp AI generator sparked outrage for training on artist styles without consent. The company initially used an opt-out system, leading to community backlash and legal action.
- Confidence
- Medium (multi-source)
Koko used GPT-3 to generate AI-assisted emotional support without informed consent
Koko conducted an October 2022 experiment using GPT-3 to generate emotional support messages, with human editors, affecting about 4,000 users and generating roughly 30,000 messages. The incident became public in January 2023 through reports and statements by Koko’s co-founders, prompting ethical criticism over informed consent and disclosure, and Koko announced pursuing a third‑party IRB review for future changes.
- Confidence
- Medium (multi-source)
Epic's sepsis prediction model missed two-thirds of cases with 88% false alarms, a study found
The Epic Sepsis Model, a proprietary sepsis prediction algorithm embedded in Epic's electronic health record platform and deployed at hundreds of US hospitals, was found to miss 67% of sepsis cases while generating 88% false alarms in an independent external validation published in JAMA Internal Medicine in June 2021. The model's discrimination (AUC 0.63) was substantially worse than Epic's claimed performance (AUC 0.76 to 0.83). Epic subsequently overhauled the model in 2022, changing its sepsis definition, reducing reliance on antibiotic orders, and recommending site-specific training before clinical use.
- Confidence
- High (multi-source, primary)
UT Austin scrapped its GRADE machine-learning PhD admissions system over entrenched bias
UT Austin's Department of Computer Science used GRADE, a machine-learning system trained on past admissions decisions, to score and organize PhD applications from 2013 through 2019. Critics identified that the system reproduced historical inequities by encoding institutional prestige bias and linguistic patterns from recommendation letters that disadvantaged underrepresented groups. The university discontinued GRADE in 2020, officially citing maintenance difficulties, though the announcement coincided with public criticism about its fairness.
- Confidence
- High (multi-source, primary)