AI Failure Index
AI Tool Misuse failures
Tool misuse is the failure mode below agentic action error. The agent picked the wrong function, or passed parameters that pointed at the wrong target. Sometimes the action goes through and the harm is the same as an agentic action error. Sometimes the tool errors out and the harm is a stuck workflow that masquerades as availability. Either way the agent did not do what the user asked.
Tool Misuse failures
At the tool call, the arguments point at the wrong target.
- Records
- 41
- Severity mix
- 1 catastrophic · 23 high · 14 medium · 3 low
- Industries
- 9
- Incident span
- Jul 2022 to Jul 2026
- Sources cited
- 101
- Newest indexed
- Jul 2026
Right tool selected → wrong arguments → no validation → wrong target hit → wrong record changed
No validation checks the arguments first.
- Validate tool-call arguments against the user's actual intent before the call runs.
- Constrain which tools and targets an agent can reach for a given task.
- Escalate calls whose target or arguments do not match what was asked.
41 records in this class.
Open all 41 in the research console
Hugging Face disclosed a production breach driven end to end by an autonomous AI agent
On July 16, 2026, Hugging Face disclosed an intrusion into its production infrastructure that it assessed was driven, end to end, by an autonomous AI agent system. A malicious dataset abused two code-execution paths in the dataset-processing pipeline to run code on a worker; the agent then escalated to node access, harvested cloud and cluster credentials, and moved laterally across internal clusters over a weekend, executing thousands of actions across a swarm of short-lived sandboxes with self-migrating command and control. Internal datasets and service credentials were accessed. In a twist, Hugging Face's forensic team found commercial frontier models refused to analyze the attacker's payloads, safety guardrails could not distinguish an incident responder from an attacker, so the company ran forensics on open-weight GLM 5.2 on its own infrastructure.
- Confidence
- High (multi-source, primary)
Sysdig documented JadePuffer, the first ransomware operation run end to end by an AI agent
In early July 2026, Sysdig's Threat Research Team published its analysis of JadePuffer, which it assessed to be the first documented ransomware operation executed end to end by an autonomous AI agent. Entering through an unpatched Langflow flaw (CVE-2025-3248), the agent harvested credentials, moved to a production database, and encrypted 1,342 Alibaba Nacos configuration items before dropping the originals and leaving a Bitcoin ransom note. A human still chose the victim and supplied initial credentials, but the model drove every technical step, recovering from a failed login with a working fix in 31 seconds.
- Confidence
- Medium (multi-source)
U.S. immigration AI screening triggers spike in visa denials and RFEs
U.S. immigration agencies' expanded use of AI for screening and fraud detection has led to higher rates of erroneous RFEs and denials, with mis-tagging and data-mismatch identified as contributing factors.
- Confidence
- Medium (multi-source)
Anthropic Model Context Protocol vulnerability exposes 200,000 AI servers to RCE
A systemic command injection vulnerability was discovered in Anthropic's Model Context Protocol (MCP). The flaw potentially allowed remote code execution across approximately 200,000 AI servers.
- Confidence
- High (multi-source, primary)
CrewAI Docker status check failure enables remote code execution
CrewAI failed to verify Docker availability at runtime, causing the system to fall back to an insecure sandbox mode. This vulnerability, tracked as CVE-2026-2287, allowed attackers to achieve remote code execution on the host machine.
- Confidence
- High (multi-source, primary)
Inland Revenue: Deepfake ads impersonate Commissioner Peter Mersi for fake crypto webinars
In March 2026 Inland Revenue New Zealand warned that scammers used a purported AI-generated likeness of Commissioner Peter Mersi in social media ads inviting people to closed webinars about crypto tax changes. The department issued a public warning and national broadcaster RNZ reported the impersonation. There are no public reports of adjudicated outcomes or confirmed, widespread financial losses in the sources found.
- Confidence
- High (multi-source, primary)
Amity Regional High School AI grading error misread rubric, penalizing a student
A student reported that an AI grading tool at Amity Regional High School misread the rubric for an AP Psychology assignment, interpreting cat least oned as conly oned and receiving a failing grade entered into PowerSchool. The grade was corrected after an academic appeal, and public backlash followed, including a petition to Keep Amity Human; FOIA materials indicated the district spent more on AI tools than initially claimed.
- Confidence
- Medium (multi-source)
OpenClaw ClawHub marketplace exploited to distribute macOS stealer malware
Attackers uploaded over 824 malicious skills to the OpenClaw ClawHub registry to distribute the Atomic Stealer (AMOS) malware. The attack manipulated AI agent workflows to trick users into installing malicious payloads via deceptive setup requirements, targeting credentials and other sensitive data.
- Confidence
- High (multi-source, primary)
Hungary ruling party accused of using AI-generated smears in election campaign
Press and policy analysis in early 2026 reported that AI-generated videos, images and documents were produced and amplified by government-aligned actors during Hungary’s run-up to the April 12, 2026 parliamentary election. Reporting alleges these synthetic assets were shared via coordinated Facebook groups and pages to discredit opposition candidates and to sidestep platform political-ad transparency rules. Independent outlets and think-tanks documented specific examples including an alleged AI-generated 600-page document and deepfake videos circulated on social media.
- Confidence
- Medium (multi-source)
Augsburg car dealer uses AI-generated image of burning car to attempt fraud
A car dealer in Augsburg allegedly attempted to defraud a seller by providing an AI-generated image of her car on fire. The dealer claimed previous damages caused a fire to demand a refund while simultaneously listing the undamaged car for sale.
- Confidence
- High (multi-source, primary)
Remax D’ICI agent uses AI to misleadingly alter home listing photos
A real estate agent at Remax D’ICI used AI to alter a home listing photo in a way the agency later said exceeded acceptable limits in Terrebonne, Quebec. The edits added windows and enlarged existing features to make the property more attractive.
- Confidence
- Medium (multi-source)
Tesla Austin robotaxi fleet logs 14 crashes prompting NHTSA investigation
Tesla's robotaxi fleet in Austin recorded 14 crashes over 800,000 miles of operation. This data was disclosed to NHTSA and is part of a broader safety investigation.
- Confidence
- High (multi-source, primary)
Adelphi University falsely accused student of AI plagiarism, court rules in his favor
Orion Newby successfully sued Adelphi University after being falsely accused of AI plagiarism; the court found the AI-detection-based findings to be baseless and expunged the record.
- Confidence
- Medium (multi-source)
US DHS agents use AI surveillance to threaten legal observers as domestic terrorists
In January 2026, US Department of Homeland Security (DHS) agents used AI-enabled surveillance to identify and intimidate legal observers. In one instance, an agent threatened an observer by claiming she was now considered a domestic terrorist in a government database.
- Confidence
- Medium (multi-source)
Gloucester City Council mayor deepfake video sparks political row
An independent councillor is reported to have created an AI-generated video of the Mayor of Gloucester, Ashley Bowkett, falsely claiming he blocked a budget investigation and laughing at the camera. The video prompted calls for stricter AI rules in politics.
- Confidence
- Medium (multi-source)
US Border Patrol facial recognition scan leads to Global Entry revocation
A US Border Patrol agent identified a neighborhood observer using facial recognition software, which was allegedly followed by the revocation of the observer's Global Entry status. The incident is reported as part of a pattern of surveillance and intimidation of protesters and observers.
- Confidence
- High (multi-source, primary)
Internal copilot filed an executive-priority Jira ticket against the wrong project
A $4B B2B SaaS company's internal AI assistant created a Jira ticket against the wrong product line during a board-week prep cycle. The PM caught it 28 hours later.
- Confidence
- Steward-verified (NDA)
Taco Bell rethought its drive-thru voice AI after viral order failures
Taco Bell's parent company said it was reconsidering where to use AI voice ordering at drive-thrus after viral clips showed the system mishandling orders, including one prankster who got it to add 18,000 cups of water, jamming the order flow.
- Confidence
- Medium (multi-source)
A New York court found NYPD misused facial-recognition AI, leading to false imprisonment
A New York Criminal Court found in People v Zuhdi A. that NYPD and FDNY officials used unauthorized facial recognition software (Clearview AI) instead of the approved limited database, illegally accessed DMV records without a court order, and altered a defendant photograph by modifying neck length before placing it in a photo array. The same pattern of misuse caused Trevis Williams to be falsely arrested and jailed for two days despite not matching the physical description and being miles away at the time of the crime. Both cases were ultimately dismissed.
- Confidence
- High (multi-source, primary)
A disabled ChatGPT consent toggle instantly deleted a Cologne professor's two years of history
In August 2025, University of Cologne plant scientist Marcel Bucher turned off ChatGPT's 'Improve the model for everyone' data consent option, which immediately and irreversibly deleted his entire two-year chat history containing grant applications, teaching materials, and publication drafts. OpenAI confirmed the deletion was by design under its 'privacy by design' policy and offered no recovery. The incident was first reported by Nature in January 2026 and raised questions about whether bundling training consent withdrawal with data destruction complies with EU GDPR data portability requirements.
- Confidence
- Medium (multi-source)
Lithuanian politicians and doctors impersonated in deepfake health scam
An international scam network used AI-generated deepfakes of Lithuanian politicians and doctors to promote fraudulent health products. These videos mimicked legitimate TV news segments to deceive the public into buying fake cures.
- Confidence
- Medium (multi-source)
Argentine judge's use of ChatGPT leads to annulment of criminal conviction
Judge Carlos Rogelio Richeri used ChatGPT to draft a criminal sentence without disclosure, resulting in an initial annulment of the ruling. The AI's involvement was discovered via a characteristic phrase in the text.
- Confidence
- High (multi-source, primary)
Pope Leo XIV deepfake video promotes Ibrahim Traoré of Burkina Faso
An AI-generated deepfake of Pope Leo XIV was created to spread false political messaging in support of Ibrahim Traoré. The fabrication was identified and denied by the Vatican and independent fact-checkers.
- Confidence
- High (multi-source, primary)
Deepfake of Dr Rinki Murphy and Jack Tame promotes fake diabetes cure in New Zealand
A deepfake video impersonating Dr. Rinki Murphy and journalist Jack Tame was used to promote a fraudulent diabetes cure in New Zealand. The video appeared to be a TVNZ interview and targeted people with Type 2 diabetes.
- Confidence
- Medium (multi-source)
British Airways chatbot fails to recognize London and Heathrow as valid entries
A British Airways chatbot failed to recognize London and Heathrow as valid inputs even after suggesting them as examples, blocking a user from finding their reservation.
- Confidence
- High (multi-source, primary)
Michigan woman loses $26,000 to AI deepfake romance scam
A Michigan woman, Beth Hyland, was defrauded of $26,000 by a scammer using the persona 'Richard.' The scammer utilized AI-generated deepfake video calls on Skype to build trust and impersonate a real person.
- Confidence
- Medium (multi-source)
McDonald's ended its IBM drive-through AI partnership after viral order failures
After three years of pilots and viral videos showing the AI ordering 260 chicken nuggets or topping ice cream with bacon, McDonald's ended the partnership in June 2024.
- Confidence
- Medium (multi-source)
Turkish student arrested for using ChatGPT to cheat on university exam
A Turkish student was arrested in Isparta for using a custom-built device connected to ChatGPT to cheat during the 2024 YKS university entrance exam. The incident highlighted the use of AI tools to circumvent academic integrity measures.
- Confidence
- Medium (multi-source)
Russia-linked CopyCop network uses AI to produce 19,000 deceptive reports monthly
The Russian-linked network CopyCop used LLMs to create thousands of deceptive reports by modifying real news stories. The operation targeted audiences in the US, UK, and France to spread disinformation and align with Russian state objectives.
- Confidence
- High (multi-source, primary)
Fake AI law firms sent fake DMCA notices to increase SEO gains
In 2024, scammers used AI-generated lawyer profiles and fake law firms, such as Commonwealth Legal, to send fraudulent DMCA copyright notices to website owners. The goal was to coerce victims into adding backlinks to a site called Tech4Gods to artificially boost its search engine rankings.
- Confidence
- Medium (multi-source)