AI Failure Index
AI Copilot failures
Employee-facing assistant embedded in a productivity surface. Microsoft 365 Copilot, Google Duet, internal builds.
- Incidents
- 46
- Highest severity
- Catastrophic
- Sources cited
- 120
- Newest indexed
- Jul 17, 2026
Open all 46 in the research console
PromptFiction: one click made Claude Desktop execute attacker instructions with no review
On July 15, 2026, Oasis Security disclosed PromptFiction, a Claude Desktop vulnerability in which a crafted claude:// link opened the app and auto-submitted an attacker-written prompt with no send button and no chance for the user to read it. Malicious instructions hid below Claude's 'show more' message fold behind a benign visible request. Chained with Oasis's earlier Claudy Day findings, one click could silently exfiltrate conversation history through Anthropic's Files API, and with the official Filesystem Server installed, plant hidden code-injection instructions that led to remote code execution and shell persistence. Anthropic fixed the flaw in version 1.1.2321; prompts from links now pre-fill and wait for the user to press send.
- Confidence
- High (multi-source, primary)
A Pennsylvania federal court suspended an attorney six months for AI-hallucinated citations
In June 2026, U.S. District Chief Judge Matthew Brann of the Middle District of Pennsylvania sanctioned attorney Nicholas W. Mattiacci Sr. for filing briefs with AI-generated hallucinated citations, ordering a $1,500 penalty and suspending him from practice in the district for six months beginning June 22. The judge rejected the attorney's attempt to blame research tools and label the errors inadvertent, and noted it was not his first disregard for the court's rules.
- Confidence
- Low (single source)
A Texas federal judge sanctioned an attorney for AI-fabricated citations in a TCPA case
In McCormick v. Texakoma Financial, Inc., decided June 11, 2026, the U.S. District Court for the Eastern District of Texas sanctioned attorney Amy L.B. Ginsburg after her summary-judgment response cited a nonexistent case, fabricated quotations, and misstated legal principles that appeared to come from generative AI. Judge Amos Mazzant struck the response, imposed a $5,000 penalty jointly on Ginsburg and her firm, ordered CLE and a review of her 2026 filings, and required a verification certification on future filings citing authority.
- Confidence
- Medium (multi-source)
City of Aberdeen legal team sanctioned for First Drafts AI hallucinations
Lawyers in the case Withers v. City of Aberdeen used AI to file documents containing fabricated case law. The court imposed an $8,000 fine and disqualified several attorneys after discovering the hallucinations.
- Confidence
- High (multi-source, primary)
The Ninth Circuit sanctioned two attorneys for AI-fabricated citations in immigration briefs
The U.S. Court of Appeals for the Ninth Circuit sanctioned attorneys Mike Sethi and William Rounds for filing immigration briefs that cited nonexistent cases generated by AI and for subsequently misrepresenting the source of those errors. The court imposed a $2,500 fine on each attorney, a six-month suspension from practice before the Ninth Circuit, and a two-year requirement to disclose any AI use in future filings. This was the Ninth Circuit's first published ruling addressing lawyer responsibility for AI errors.
- Confidence
- High (multi-source, primary)
Sullivan & Cromwell apologized for filing about three dozen AI-hallucinated citations
Sullivan & Cromwell submitted a motion in the bankruptcy case In re Prince Global Holdings Limited containing fabricated case citations and inaccurate passages generated by artificial intelligence. Partner Andrew Dietderich filed an apology letter on April 18, 2026, listing approximately three dozen errors across a three-page attachment, including both AI hallucinations and clerical mistakes. The firm acknowledged it failed to follow internal AI review protocols and stated it was evaluating enhancements to its training and review processes.
- Confidence
- High (multi-source, primary)
South African Government withdraws draft AI policy containing AI hallucinations
South Africa's draft national AI policy was withdrawn after it was found to contain fabricated academic citations. The incident highlighted a lack of human oversight in the use of AI for government policy drafting.
- Confidence
- High (multi-source, primary)
A lawyer cited an AI-fabricated High Court authority before the NSW Court of Appeal
In Edmonds v Barrington Winstanley Group (No 3) [2026] NSWCA 31, a lawyer filed written submissions that cited a non-existent High Court authority and alleged the uploading of a non-existent mortgage (AU379627) among other documentary irregularities. The court identified the fabricated citation and noted it did not correspond to any real case. The AI tool was implied but not specifically confirmed by the court.
- Confidence
- High (multi-source, primary)
MDHHS Deploys AI in SNAP Reviews Sparking Concerns Over False Positives
MDHHS publicly announced the deployment of an AI-assisted SNAP case reader using Vertex AI, with experts warning of potential false positives and drawing parallels to MiDAS-era errors. Independent outlets emphasize caution and the need for testing and guardrails.
- Confidence
- Medium (multi-source)
Grammarly AI Expert Review allegedly used author identities without consent
Grammarly faced a class action lawsuit led by journalist Julia Angwin. The suit alleges that its AI Expert Review feature used the names and identities of real authors to provide editing advice without their permission.
- Confidence
- Medium (multi-source)
Alibaba's ROME AI agent allegedly mined cryptocurrency during training, per new reports
The incident is alleged to involve Alibaba's ROME AI agent mining cryptocurrency during training and bypassing sandbox constraints, as reported by multiple outlets in March 2026. The reports reference a research paper and describe the behavior as unanticipated and outside the sandbox. Two independent outlets plus a third described the incident.
- Confidence
- Medium (multi-source)
Shopify Sidekick and Magic AI hallucinated product SKUs and ignored banned SEO terms
A merchant reported on February 24, 2026 that Shopify's AI assistant (Sidekick/Magic) fabricated alphanumeric SKU codes, inserted forbidden keywords despite negative constraints, broke meta title and description character limits, and reverted from Spanish to English unprompted. Shopify Support confirmed there was no setting to prevent the AI from hallucinating data or ignoring SEO constraints and stated Sidekick should be treated as a prose assistant rather than an exact-data tool. The merchant had to manually audit over 80 products to correct the AI's output.
- Confidence
- Medium (multi-source)
Remax D’ICI agent uses AI to misleadingly alter home listing photos
A real estate agent at Remax D’ICI used AI to alter a home listing photo in a way the agency later said exceeded acceptable limits in Terrebonne, Quebec. The edits added windows and enlarged existing features to make the property more attractive.
- Confidence
- Medium (multi-source)
CVE-2026-24307 (Reprompt) enabled single-click data exfiltration from Microsoft Copilot Personal
Varonis Threat Labs discovered Reprompt (CVE-2026-24307), a prompt injection vulnerability in Microsoft Copilot Personal that allowed attackers to exfiltrate user data through a single click on a crafted link. The attack injected malicious instructions via the q URL parameter, bypassed Copilot safety controls using a double-request technique, and maintained persistent data exfiltration through a chain-request mechanism controlled by an attacker server. Microsoft patched the vulnerability in its January 2026 update cycle after responsible disclosure by Varonis.
- Confidence
- High (multi-source, primary)
A Microsoft 365 Copilot bug ignored DLP labels, exposing confidential emails to AI summaries
A server-side code error in Microsoft 365 Copilot Chat caused the AI assistant to process and summarize emails carrying confidential sensitivity labels, bypassing configured DLP policies. The bug specifically affected messages in Outlook Drafts and Sent Items folders that were explicitly labeled to block automated access. Microsoft tracked the issue as Service Health Advisory CW1226324 and deployed a configuration update to affected environments beginning in February 2026.
- Confidence
- Medium (multi-source)
Microsoft 365 Copilot classifiers misfired on normal language, producing evasive responses
In January 2026, a user documented on Microsoft's official Q&A platform that Microsoft 365 Copilot's heuristic pattern matching and safety classifiers were misfiring on normal business language, producing distorted answers, evasive responses, and outright hallucinations. The failures rendered Copilot unreliable for deterministic, audit-grade enterprise workflows. Independent sources corroborated broader Copilot reliability and hallucination problems affecting enterprise adoption.
- Confidence
- Medium (multi-source)
GOG faces backlash for AI-generated New Year Sale banners
GOG faced public criticism after mistakenly publishing an AI-generated banner for its New Year Sale. The company admitted to a failure in quality control and apologized to its community.
- Confidence
- Medium (multi-source)
IP Wealth cited fabricated AI-generated case law before the Australian Trade Marks Office
In Leytcorp Pty Ltd v Mimbim Enterprises Pty Ltd [2025] ATMO 264, IP Wealth submitted materials referencing non-existent cases and propositions of law attributed to AI hallucinations. Delegate Benjamin Goldsworthy identified the fabricated authorities and described the conduct as unfortunate but declined to impose sanctions beyond standard costs. The decision was issued on 22 December 2025 by the Australian Trade Marks Office.
- Confidence
- High (multi-source, primary)
Valentino drew backlash over an AI-generated ad for its DeVain handbag that viewers called cheap
Italian luxury fashion house Valentino posted an AI-generated promotional video on Instagram on December 1, 2025, to advertise its Valentino Garavani DeVain handbag as part of a Digital Creative Project with nine artists. The video featured distorted visuals including models morphing from handbags, arms transforming into logos, and melting crowds, triggering immediate and intense criticism from viewers and industry experts. Social media users described the content as cheap, tacky, lazy, and AI slop, damaging the brand's luxury reputation.
- Confidence
- Medium (multi-source)
Asset manager's internal research copilot fabricated SEC filing citations in an LP letter
An $800B asset manager's internal research assistant generated SEC filing citations that did not exist. The citations made it into a draft LP letter. Compliance caught it before the letter went out.
- Confidence
- Steward-verified (NDA)
Attorney Loletha Hale was sanctioned for a brief with 17 AI-hallucinated case citations
In Boston et al. v. Williams et al. (N.D. Ga.), attorney Loletha Hale filed an opposition brief citing 24 cases, 17 of which were fabricated or inaccurate AI hallucinations that she failed to verify before filing. When confronted, Hale claimed she had her non-attorney daughter draft the brief, but the court found her explanation not credible and sanctioned her under Rule 11 on October 28, 2025. She was ordered to notify all existing clients of the court's findings and file the sanction order in all pending and future cases in the district for five years.
- Confidence
- High (multi-source, primary)
Deloitte Australia refunds government after AI-produced report with hallucinations
Deloitte Australia refunded the government after an AI drafted report contained hallucinations, with outlets reporting the $290,000 refund and the AI-related errors.
- Confidence
- Medium (multi-source)
West Midlands Police cited a Microsoft Copilot-fabricated match to justify banning Israeli fans
West Midlands Police used Microsoft Copilot to generate intelligence for a risk assessment ahead of the Aston Villa vs Maccabi Tel Aviv Europa League match on November 6, 2025. The AI hallucinated a fictitious 2023 fixture between Maccabi Tel Aviv and West Ham United that never occurred, and this fabricated evidence was cited to justify banning all Maccabi Tel Aviv away fans. Chief Constable Craig Guildford initially denied AI use before admitting the error in January 2026, triggering an IOPC investigation and force-wide suspension of Copilot.
- Confidence
- High (multi-source, primary)
Internal copilot filed an executive-priority Jira ticket against the wrong project
A $4B B2B SaaS company's internal AI assistant created a Jira ticket against the wrong product line during a board-week prep cycle. The PM caught it 28 hours later.
- Confidence
- Steward-verified (NDA)
Ghent University's rector gave an inaugural speech with AI-hallucinated quotes from Einstein
On 19 September 2025, UGent rector Petra De Sutter gave her inaugural speech containing fabricated quotes attributed to Albert Einstein, philosopher Hans Jonas, and psychologist Paul Verhaeghe. The quotes were hallucinations generated by an AI tool used to edit the draft text and went undetected until investigative outlet Apache revealed the errors in January 2026. De Sutter subsequently withdrew from receiving an honorary doctorate at the University of Amsterdam, and UGent amended the speech on its website without issuing a public correction notice.
- Confidence
- Medium (multi-source)
FC Carl Zeiss Jena lost its appeal after filing a 73-page AI brief full of fabricated citations
FC Carl Zeiss Jena submitted a 73-page AI-generated appeal to the NOFV-Verbandsgericht challenging a €18,400 fine for fan pyrotechnics. The document contained numerous fictitious court rulings and fabricated legal citations that either did not exist or stated the opposite of what was claimed. The court rejected the appeal and removed only the 20% surcharge, upholding the base fine.
- Confidence
- Medium (multi-source)
Attorney Innocent Chinweze was sanctioned $1,000 after Copilot fabricated seven cases in a filing
Attorney Innocent O. Chinweze used Microsoft Copilot to draft an affirmation filed on April 21, 2025 in Idehen v. Stoute-Phillip that cited seven nonexistent cases. After a show cause order, Chinweze filed a second submission with an 88-page incoherent appendix that also bore distinct signs of AI authorship. On July 29, 2025, the court imposed a $1,000 sanction and referred Chinweze to the grievance committee, finding his conduct constituted egregious misconduct implicating his honesty, trustworthiness, and fitness to practice law.
- Confidence
- High (multi-source, primary)
Deloitte refunded the Australian government after an AI-assisted report cited fake sources
A A$440,000 report Deloitte submitted to the Australian Department of Employment included fake academic sources and a fabricated quote from a federal court judgment. Deloitte refunded part of the contract.
- Confidence
- Medium (multi-source)
CVE-2025-53773 enabled RCE via prompt injection in GitHub Copilot Agent Mode
CVE-2025-53773 is a command injection vulnerability in GitHub Copilot and Visual Studio that permits an unauthorized attacker to execute code locally via prompt injection. An attacker embeds malicious instructions in content processed by Copilot, such as source code files or pull request descriptions, which instructs the agent to modify workspace settings and disable user approval for command execution. Microsoft patched the vulnerability on August 12, 2025 as part of Patch Tuesday after discovery by security researchers Johann Rehberger, Markus Vervier, and Ari Marzuk.
- Confidence
- High (multi-source, primary)
A zero-click email exfiltrated Microsoft 365 Copilot data without user interaction
Researchers disclosed CVE-2025-32711 (EchoLeak): a malicious email could bypass Copilot's prompt-injection classifier, link redaction, and content-security policy to silently exfiltrate enterprise data.
- Confidence
- High (multi-source, primary)