AI Failure Index
AI Failures in Public Sector
Government and public-sector AI deployments are subject to public records laws. The failures are unusually visible.
- Incidents
- 120
- Highest severity
- Catastrophic
- Sources cited
- 319
- Newest indexed
- Jun 16, 2026
Procureur général du Canada sanctioned pro se litigant for AI fabricated case law
A self-represented litigant in Canada was sanctioned by the Federal Court for submitting fabricated case law generated by AI. The court emphasized that citing non-existent sources is a serious matter that undermines the administration of justice.
- Confidence
- High (multi-source, primary)
City of Aberdeen legal team sanctioned for First Drafts AI hallucinations
Lawyers in the case Withers v. City of Aberdeen used AI to file documents containing fabricated case law. The court imposed an $8,000 fine and disqualified several attorneys after discovering the hallucinations.
- Confidence
- High (multi-source, primary)
Ukrainian sea drone reportedly veers off course and explodes in Constanta port
On 2026-06-05 a naval/sea drone reportedly linked to Ukraine exploded in the Romanian port of Constanta after veering off course. Ukrainian officials told reporters the drone lost control following alleged electronic jamming; authorities say the area was secured and there were no injuries. Multiple independent news outlets reported the incident the same day.
- Confidence
- Medium (multi-source)
Argentina's predictive AI digital twin fails to predict typo in own promo video
Argentina's Ministry of Human Capital launched a 'Social Digital Twin' AI to simulate policy impacts. The launch was marred by a promotional video containing AI-generated hallucinations and basic spelling errors.
- Confidence
- Medium (multi-source)
Argentina Ministry of Human Capital AI announcement video riddled with errors
Argentina's Ministry of Human Capital launched a "Social Digital Twin" AI to simulate social policy impacts. The promotional video released for the announcement contained numerous AI-generated typos and visual errors.
- Confidence
- Medium (multi-source)
AI chatbots provided misinformation in 34 percent of Scottish election queries
A study by the think-tank Demos found that AI chatbots frequently provided false information about the 2026 Scottish Parliament election. The research revealed that one third of responses contained factual errors, including fabricated scandals and incorrect election dates.
- Confidence
- High (multi-source, primary)
GOV.UK Chat AI provides misleading tax advice to citizens
The GOV.UK Chat AI tool gave misleading tax advice, failing to identify key income thresholds and inaccurately suggesting no cap for childcare eligibility.
- Confidence
- Medium (multi-source)
Pennsylvania sues Character.AI over fake medical license claim by chatbot
The Commonwealth of Pennsylvania filed a lawsuit against Character.AI on 2026-05-05, alleging that a Character.AI chatbot presented itself as a licensed psychiatrist and provided a fake Pennsylvania license number. The complaint seeks injunctive relief to stop chatbots from posing as licensed professionals and giving medical advice.
- Confidence
- High (multi-source, primary)
BBC Wales finds six AI chatbots gave misleading Senedd election voting advice
BBC Wales found six major AI chatbots gave inaccurate voting information for the Senedd election, including deceased candidates and wrong constituencies. The reports cite hallucinations and outdated training data as causes. Two independent outlets corroborate the event.
- Confidence
- Medium (multi-source)
Home Affairs suspends two officials after AI-generated references found in white paper
The Department of Home Affairs suspended two senior officials after apparent AI-generated hallucinations were found in the reference list to the Cabinet-approved Revised White Paper on Citizenship, Immigration and Refugee Protection. The department withdrew the reference list, appointed independent law firms to manage disciplinary and review processes, and initiated a review of policy documents dating back to 2022.
- Confidence
- High (multi-source, primary)
South Africa withdraws AI policy after AI-generated citations found
South Africa’s Department of Communications and Digital Technologies withdrew its Draft National Artificial Intelligence Policy after investigations found AI-generated citations in the draft; the Government Gazette published it for public comment on 10 April 2026, and withdrawal followed in late April 2026 amid political backlash.
- Confidence
- High (multi-source, primary)
U.S. immigration AI screening triggers spike in visa denials and RFEs
U.S. immigration agencies' expanded use of AI for screening and fraud detection has led to higher rates of erroneous RFEs and denials, with mis-tagging and data-mismatch identified as contributing factors.
- Confidence
- Medium (multi-source)
State tax agencies use opaque AI for audit selection without oversight
State tax agencies in California and New York use automated AI systems for audit selection that bypass state oversight requirements. This lack of transparency creates risks of algorithmic bias and unfair targeting of taxpayers.
- Confidence
- Medium (multi-source)
South African Government withdraws draft AI policy containing AI hallucinations
South Africa's draft national AI policy was withdrawn after it was found to contain fabricated academic citations. The incident highlighted a lack of human oversight in the use of AI for government policy drafting.
- Confidence
- High (multi-source, primary)
IRCC automation produced incorrect assessments and at least one AI-generated refusal
Public reporting documents at least one case where IRCC automation and generative-AI-assisted review produced a refusal letter containing fabricated job duties and acknowledged the use of generative AI in the review. Journalistic accounts and civic-technology commentary say the tools are used for triage and summarization across a large backlog, raising concerns about incorrect classifications, opaque refusal explanations, and downstream delays.
- Confidence
- Medium (multi-source)
MDHHS Deploys AI in SNAP Reviews Sparking Concerns Over False Positives
MDHHS publicly announced the deployment of an AI-assisted SNAP case reader using Vertex AI, with experts warning of potential false positives and drawing parallels to MiDAS-era errors. Independent outlets emphasize caution and the need for testing and guardrails.
- Confidence
- Medium (multi-source)
Nepal election disinformation surge uses AI deepfakes to mislead voters
AI-generated videos and images were used at scale to spread disinformation during Nepal's March 2026 parliamentary elections. The content included fake drone footage of political rallies and deepfake videos of candidates.
- Confidence
- Medium (multi-source)
India's Poshan Tracker facial-recognition excludes eligible beneficiaries
The Poshan Tracker facial-recognition system failed to recognise mothers, excluding families from meals, preschool education, and health monitoring; government data cited a 52.7% ration delivery rate by end-2025.
- Confidence
- Medium (multi-source)
Essex Police pauses live facial recognition after Cambridge study finds racial bias
Essex Police paused live facial recognition after a Cambridge study found racial bias in the system, prompting regulatory mitigations and an ongoing review.
- Confidence
- High (multi-source, primary)
ZDF airs Sora AI video as real ICE footage in news report
German public broadcaster ZDF used a Sora-generated AI video and mislabeled real police footage as US ICE operations in a news segment. The broadcaster issued a live apology and recalled its US correspondent after the error was discovered.
- Confidence
- Medium (multi-source)
Dutch Probation Service suspends OXREC risk algorithm over discrimination findings
The Dutch Probation Service halted the OXREC AI tool after an official investigation revealed a 20% error rate and biased risk assessments, caused by outdated Swedish data and swapped formulas.
- Confidence
- Medium (multi-source)
US DHS agents use AI surveillance to threaten legal observers as domestic terrorists
In January 2026, US Department of Homeland Security (DHS) agents used AI-enabled surveillance to identify and intimidate legal observers. In one instance, an agent threatened an observer by claiming she was now considered a domestic terrorist in a government database.
- Confidence
- Medium (multi-source)
Gloucester City Council mayor deepfake video sparks political row
An independent councillor is reported to have created an AI-generated video of the Mayor of Gloucester, Ashley Bowkett, falsely claiming he blocked a budget investigation and laughing at the camera. The video prompted calls for stricter AI rules in politics.
- Confidence
- Medium (multi-source)
U.S. Department of Transportation robo-bus rear-ended during D.C. demonstration ride
During a U.S. Department of Transportation demonstration in Washington, D.C., a Beep automated shuttle was reportedly rear-ended by a Tesla on 2026-01-11. A human safety driver was onboard, there were no injuries, and Beep stated the shuttle operated appropriately and was cleared to resume service. Coverage of the incident appears in multiple news outlets.
- Confidence
- High (multi-source, primary)
US Border Patrol facial recognition scan leads to Global Entry revocation
A US Border Patrol agent identified a neighborhood observer using facial recognition software, which was allegedly followed by the revocation of the observer's Global Entry status. The incident is reported as part of a pattern of surveillance and intimidation of protesters and observers.
- Confidence
- High (multi-source, primary)
National Weather Service map showed fabricated Idaho town names
Multiple news outlets reported that a National Weather Service office published an AI-generated forecast graphic for Camas Prairie, Idaho that included fabricated or misspelled town names and was subsequently removed from NWS sites. Reporting indicates the errors came from an AI-generated base map used to render the forecast graphic rather than from the meteorological forecast itself.
- Confidence
- Medium (multi-source)
Lone attacker breaches nine Mexican government agencies using Claude Code and GPT-4.1
Independent outlets corroborate the incident involving a lone attacker using Claude Code and GPT-4.1 to breach nine Mexican government agencies and exfiltrate hundreds of millions of records.
- Confidence
- Medium (multi-source)
Sweden's SVT aired an AI-generated video of a police-ICE confrontation as authentic footage
SVT's political magazine program Agenda broadcast an AI-generated video clip depicting a New York police officer berating an ICE agent, presenting it as genuine footage during a segment on US immigration policy. Attentive viewers identified the fabrication by spotting the misspelling 'POICE' instead of 'POLICE' on the officer's uniform. SVT removed the clip from its streaming platform, issued a correction, and the Swedish Media Authority's Review Board ultimately cleared the broadcaster in February 2026 after finding the correction satisfied objectivity requirements.
- Confidence
- High (multi-source, primary)
US law enforcement used ALPR networks to monitor protesters, raising privacy concerns
An investigation by the Electronic Frontier Foundation documented law enforcement use of Flock Safety automated license plate reader (ALPR) data to search for and track protesters and activists. Local governments and advocates responded with policy actions and contract terminations, and the vendor publicly defended its product.
- Confidence
- Medium (multi-source)
Public-sector voice agent failed Spanish-accented English callers at 4x the rate of native speakers
A state-government voice agent for benefits eligibility failed Spanish-accented English speakers at four times the rate of native speakers. The fairness audit was prompted by a single state legislator who called.
- Confidence
- Steward-verified (NDA)
Canada Revenue Agency's $18M Charlie chatbot gave wrong tax answers 66% of the time
The Canada Revenue Agency deployed an AI chatbot named Charlie that cost over $18 million to develop and operate since fiscal year 2018-19. An audit by Auditor General Karen Hogan found the chatbot provided correct answers in fewer than half of tested cases, with only 2 out of 6 questions answered accurately. The system handled over 7 million conversations across 13 CRA webpages, potentially exposing Canadian taxpayers to incorrect tax filing guidance.
- Confidence
- High (multi-source, primary)
West Midlands Police cited a Microsoft Copilot-fabricated match to justify banning Israeli fans
West Midlands Police used Microsoft Copilot to generate intelligence for a risk assessment ahead of the Aston Villa vs Maccabi Tel Aviv Europa League match on November 6, 2025. The AI hallucinated a fictitious 2023 fixture between Maccabi Tel Aviv and West Ham United that never occurred, and this fabricated evidence was cited to justify banning all Maccabi Tel Aviv away fans. Chief Constable Craig Guildford initially denied AI use before admitting the error in January 2026, triggering an IOPC investigation and force-wide suspension of Copilot.
- Confidence
- High (multi-source, primary)
ENISA reports AI-hallucinated sources in 2025 threat landscape reports
The EU Agency for Cybersecurity (ENISA) published two 2025 threat reports containing AI-hallucinated citations; researchers found 26 incorrect footnotes out of 492 in one report.
- Confidence
- Medium (multi-source)
Ghent University's rector gave an inaugural speech with AI-hallucinated quotes from Einstein
On 19 September 2025, UGent rector Petra De Sutter gave her inaugural speech containing fabricated quotes attributed to Albert Einstein, philosopher Hans Jonas, and psychologist Paul Verhaeghe. The quotes were hallucinations generated by an AI tool used to edit the draft text and went undetected until investigative outlet Apache revealed the errors in January 2026. De Sutter subsequently withdrew from receiving an honorary doctorate at the University of Amsterdam, and UGent amended the speech on its website without issuing a public correction notice.
- Confidence
- Medium (multi-source)
Thailand freezes 3 million bank accounts in automated anti scam crackdown
The Bank of Thailand froze approximately 3 million bank accounts to combat fraud and mule accounts. The sweeping action resulted in widespread false positives, locking innocent users out of their funds.
- Confidence
- Medium (multi-source)
Cognia's AI scoring engine gave about 1,400 Massachusetts MCAS essays wrong zero scores
Cognia's AI scoring engine incorrectly scored approximately 1,400 Massachusetts MCAS essays during the 2025 testing cycle, assigning zero scores to responses that deserved higher marks. The system failed to route problematic essays to human reviewers, and the routine 10% human second-read check also missed the errors. A Lowell third-grade teacher discovered the discrepancies, prompting Cognia to rescore all affected essays before final results were released.
- Confidence
- Medium (multi-source)
A New York court found NYPD misused facial-recognition AI, leading to false imprisonment
A New York Criminal Court found in People v Zuhdi A. that NYPD and FDNY officials used unauthorized facial recognition software (Clearview AI) instead of the approved limited database, illegally accessed DMV records without a court order, and altered a defendant photograph by modifying neck length before placing it in a photo array. The same pattern of misuse caused Trevis Williams to be falsely arrested and jailed for two days despite not matching the physical description and being miles away at the time of the crime. Both cases were ultimately dismissed.
- Confidence
- High (multi-source, primary)
A disabled ChatGPT consent toggle instantly deleted a Cologne professor's two years of history
In August 2025, University of Cologne plant scientist Marcel Bucher turned off ChatGPT's 'Improve the model for everyone' data consent option, which immediately and irreversibly deleted his entire two-year chat history containing grant applications, teaching materials, and publication drafts. OpenAI confirmed the deletion was by design under its 'privacy by design' policy and offered no recovery. The incident was first reported by Nature in January 2026 and raised questions about whether bundling training consent withdrawal with data destruction complies with EU GDPR data portability requirements.
- Confidence
- Medium (multi-source)
Angela Lipps arrested after facial-recognition match led to wrongful extradition
Law enforcement in Fargo relied on a facial-recognition match from a neighboring agency’s system (reported to be Clearview AI) to obtain a warrant; Lipps was arrested in Tennessee on July 14, 2025 and detained for months before charges were dismissed on December 23, 2025 after exculpatory records showed she was in Tennessee during the events. The incident combines a model false positive with inter-agency information-handling failures.
- Confidence
- High (multi-source, primary)
MAHA report on children's health exposed as fabricated with AI-assisted citations
Multiple outlets reported that the MAHA Commission's presidential report included fabricated references and AI-generated markers, prompting updates while keeping core substance.
- Confidence
- Medium (multi-source)
White House health report included fabricated AI citations
The White House's MAHA report on children's health was found to contain fabricated scientific citations generated by AI. This undermined the report's stated goal of adhering to the gold standard of scientific rigor.
- Confidence
- Medium (multi-source)
White House MAHA report contains nonexistent studies and AI markers
The White House published a public health report containing fake AI-generated citations and 'oaicite' markers. The incident highlighted a failure in editorial oversight for AI-generated government content.
- Confidence
- Medium (multi-source)
University at Buffalo student graduation risked by Turnitin AI false positive
A student at the University at Buffalo faced graduation delays after Turnitin falsely flagged her work as AI-generated. The event prompted a student-led petition to ban AI detectors on campus.
- Confidence
- High (multi-source, primary)
Haringey Council homeless application judicial review cites fake law cases
In a judicial review involving a homeless applicant against Haringey Council, the claimant's legal team submitted documents citing five non-existent legal cases. The court found this conduct to be improper, unreasonable, and negligent, referring the legal team to their professional regulators and ordering them to pay wasted costs.
- Confidence
- Medium (multi-source)
NYPD facial recognition match leads to wrongful arrest of Trevis Williams
Two independent outlets report that NYPD used facial recognition to arrest Trevis Williams, despite height and location discrepancies, leading to jail time before charges were dismissed; advocacy groups are pushing for policy changes.
- Confidence
- Medium (multi-source)
Yale EMBA student sues over AI-based exam accusation
A Yale EMBA student sued Yale after an AI detector flagged his final exam, leading to suspension and a failing grade.
- Confidence
- Medium (multi-source)
France's government-backed chatbot Lucie was pulled after three days of absurd answers
Linagora's open source AI chatbot Lucie, developed under the French government's France 2030 investment program, was taken offline on January 25, 2025, just three days after its public launch. Users flooded social media with examples of the bot confidently giving nonsensical answers, including claiming that cows lay eggs, providing recipes for cooking meth, and stating that the square root of a goat is one. Linagora admitted the model had been released prematurely without adequate guardrails or reinforcement learning.
- Confidence
- Medium (multi-source)
Kohls v Ellison: Expert AI declaration excluded for fake citations
In Kohls v Ellison, a Stanford professor submitted an AI‑assisted expert declaration that contained fake citations; the court excluded the declaration and criticized the use of AI in the filing, underscoring the need to verify AI outputs in legal submissions.
- Confidence
- High (multi-source, primary)
DWP AI fraud detection system found to be biased against vulnerable groups
An AI system used by the UK's Department for Work and Pensions to detect fraud in Universal Credit advance claims was found to be biased. An internal fairness analysis revealed that the system disproportionately flagged certain demographic groups for investigation.
- Confidence
- Medium (multi-source)
ITAT Bengaluru withdraws tax order citing fake AI judgments
The ITAT Bengaluru withdrew a tax order involving Buckeye Trust after discovering it relied on fake legal precedents generated by AI. The incident highlights the risk of using generative AI for legal research without rigorous verification.
- Confidence
- Medium (multi-source)
Sweden fraud-prediction algorithm found to discriminate against women
Investigative reporting and an Amnesty International statement published on 2024-11-27 found that a fraud risk‑scoring algorithm used by Sweden's Social Insurance Agency produced disproportionate harms to women and other groups. Amnesty called the system discriminatory and urged authorities to discontinue its use. The reporting describes unequal precision and group disparities in the model's risk scores.
- Confidence
- Medium (multi-source)
Home Office AI enforcement tool criticised as rubberstamping immigration decisions
A UK Home Office system called Identify and Prioritise Immigration Cases (IPIC) was criticised by rights groups and privacy researchers in November 2024 as opaque and likely to produce 'rubberstamped' enforcement outcomes. Privacy International obtained redacted manuals and assessments via freedom of information requests that, critics say, show the tool combines sensitive personal data to prioritise cases. Critics warned the system risks bias and poor human oversight in immigration enforcement.
- Confidence
- High (multi-source, primary)
OFF Radio Kraków airs AI interview with late poet Wisława Szymborska amid backlash
In October 2024 OFF Radio Kraków launched a channel using AI-generated presenters and aired an imagined interview with the late poet Wisława Szymborska. The station said the programme had been authorised by the Wisława Szymborska Foundation president, but the broadcast provoked widespread criticism and protests and the station discontinued the AI-led experiment after several days. Coverage highlighted ethical, rights and regulatory concerns about using AI to simulate deceased public figures without clear safeguards.
- Confidence
- Medium (multi-source)
CNAF risk-scoring algorithm accused of discriminating welfare recipients
France's CNAF deployed a risk-scoring algorithm to flag welfare recipients for potential fraud. NGOs filed a lawsuit in October 2024 alleging discrimination and GDPR violations.
- Confidence
- High (multi-source, primary)
Prince George's County Public Schools AI messaging disrupted by AllHere financial collapse
AI messaging services at Prince George's County Public Schools were terminated following the financial collapse of the provider, AllHere. The disruption occurred in June 2024 as the company faced insolvency and bankruptcy.
- Confidence
- Medium (multi-source)
LAUSD disables Ed AI chatbot after AllHere collapses
LAUSD disabled its Ed AI chatbot after the vendor AllHere collapsed and could not supervise the system. Reports also describe whistleblower claims of student data privacy violations and ongoing regulatory scrutiny culminating in a federal inquiry into AllHere's bankruptcy.
- Confidence
- Medium (multi-source)
A DWP algorithm wrongly flagged over 200,000 housing-benefit claimants for fraud over three years
The UK Department for Work and Pensions deployed a risk-based verification algorithm to flag housing benefit claims for fraud review, but the system produced massive false positives. Over 200,000 people were wrongly subjected to intrusive investigations across three financial years from 2020 to 2023. The algorithm's live accuracy rate of roughly 34 to 37 percent fell far below the 64 percent rate observed during its pilot phase.
- Confidence
- High (multi-source, primary)
New York City's small-business chatbot told users to break the law
MyCity, the chatbot launched by the New York City Mayor's office, advised users on how to commit wage theft, fire workers who complained about harassment, and serve food bitten by rats.
- Confidence
- Medium (multi-source)
NYC MyCity AI chatbot gave illegal guidance to small businesses
New York City's MyCity AI chatbot gave illegal advice to businesses regarding housing and labor laws. The incident highlighted the risks of deploying generative AI for legal guidance without adequate safeguards.
- Confidence
- High (multi-source, primary)
NYC AI chatbot tells businesses to break the law
A Microsoft-powered NYC chatbot meant to help small businesses gave legally incorrect guidance, including claims that employers could seize tips and fire employees for reporting sexual harassment. The incident is documented by The Markup, The City, and AP News with follow-up coverage noting misinformation about housing and employment laws.
- Confidence
- Medium (multi-source)
Met Police facial recognition wrongly matched youth worker Shaun Thompson
In February 2024 Shaun Thompson, a youth advocacy worker, was stopped and questioned after the Metropolitan Police's live facial‑recognition system matched him to a watchlist entry. The encounter lasted around 30 minutes and ended when Thompson produced ID; he subsequently brought a High Court challenge to the Met's use of LFR, which was dismissed on 2026-04-21. Reporting on the case is documented by multiple independent outlets including the BBC and The Independent.
- Confidence
- Medium (multi-source)
UK DWP Universal Credit fraud model shows bias in age and nationality referrals
An internal assessment found statistically significant bias in the UC Advances model, disproportionately flagging non-UK nationals and certain age groups for fraud investigations without a corresponding gain in correct identifications.
- Confidence
- High (multi-source, primary)
Comune di Trento fined 50,000 euros for illegal AI surveillance projects
The Italian Garante Privacy fined Comune di Trento €50,000 for deploying AI systems that violated GDPR rules through insufficient anonymization and lack of impact assessments. The city was ordered to delete the collected data from the MARVEL and PROTECTOR projects.
- Confidence
- High (multi-source, primary)
Telangana AI Samagra Vedika wrongly denied food subsidies to thousands
Independent reporting confirms that Telangana’s Samagra Vedika profiling system wrongly denied food subsidies to thousands due to faulty data matching, prompting a court-ordered re-verification; estimates indicate misclassifications affected a substantial number of beneficiaries.
- Confidence
- Medium (multi-source)
UK GOV.UK Chat gave citizens incorrect tax, VAT, and immigration advice in its alpha pilot
The UK Government Digital Service's GOV.UK Chat prototype produced inaccurate or misleading responses during a private pilot with approximately 1,000 users, scoring only 76% accuracy at its earliest benchmark. The system gave incorrect advice on tax, VAT registration, EU Settlement Scheme, and flight refund matters before GDS added filters to block certain question categories. The Times later reported that the chatbot gave misleading tax information, drawing criticism from tax professionals.
- Confidence
- High (multi-source, primary)
Thomson Reuters fraud detection software subject of FTC complaint
Thomson Reuters' automated fraud-detection software, used by several U.S. states, was the subject of an FTC complaint filed by EPIC. The system allegedly incorrectly identified eligible claimants as fraudulent, leading to the suspension of public benefits.
- Confidence
- Medium (multi-source)
Dutch tax agency fraud algorithm discriminated against dual nationals
A systemic failure in the Dutch tax authority's fraud-detection algorithms led to discriminatory targeting of dual nationals, causing thousands of families to be wrongly accused and face financial hardship; the event attracted regulatory scrutiny and political repercussions in 2024. The AP AI & Algorithmic Risks Report formally acknowledges systemic AI risks linked to this case.
- Confidence
- High (multi-source, primary)
Communauté de communes Cœur Côte Fleurie ordered to delete AI-surveillance data
In November 2023 a French administrative court ordered the Communauté de communes Cœur Côte Fleurie to stop using an augmented camera system coupled with algorithmic video-surveillance and to delete personal data obtained via the system. The court concluded the system permitted automated identification and tracking of people and therefore constituted a serious and manifestly unlawful interference with privacy; the originals were placed under seal with the CNIL.
- Confidence
- High (multi-source, primary)
Sergio Massa campaign uses AI generated images for political advertisements
Sergio Massa's 2023 presidential campaign in Argentina used AI-generated imagery to create propaganda and attack ads. The incident highlighted the risks of synthetic media in democratic elections.
- Confidence
- Medium (multi-source)
Plainfield Police Department predictive policing software fails to predict crimes
The Markup and Wired reported that Geolitica's predictive policing software for Plainfield PD produced thousands of predictions with a success rate under 1 percent across 23,631 predictions, and the department stopped using it.
- Confidence
- Medium (multi-source)
Italian Ministry of Education GPS algorithm mis-ranks thousands of teachers
The Italian Ministry of Education's GPS automated allocation system for short-term teachers suffered a critical logic failure. Thousands of eligible teachers were wrongly excluded from assignments, resulting in lost income and numerous lawsuits.
- Confidence
- Medium (multi-source)
ChatGPT falsely named an Australian mayor as a convicted briber
Brian Hood, a regional Australian mayor, threatened to sue OpenAI after ChatGPT described him as a convicted criminal in a bribery scandal. In reality Hood was the whistleblower who exposed the scheme, not a participant, making it an early defamation threat over a chatbot hallucination.
- Confidence
- Low (single source)
USCIS AI translation errors in Pashto jeopardize Afghan asylum claims
US Citizenship and Immigration Services (USCIS) and its contractors relied on AI translation tools for Afghan refugee asylum claims, leading to critical errors in Pashto and Dari translations. These inaccuracies resulted in discrepancies that led to the denial of asylum claims.
- Confidence
- Medium (multi-source)
Rotterdam welfare fraud model used discriminatory data and performed poorly
A Rotterdam welfare fraud model allegedly used discriminatory data and performed no better than random; two independent outlets describe bias and limited usefulness of the system.
- Confidence
- Medium (multi-source)
Los Angeles scoring system ranks Black and Latino unhoused people lower for subsidized housing
Investigations by The Markup and the Los Angeles Times reported that a scoring system used to prioritize unhoused people for subsidized permanent housing in Los Angeles produced consistently lower priority scores for Black and Latino people. The reporting analysed intake assessment records and found these disparities persisted year after year, making Black and Latino people less likely to receive permanent housing. Subsequent reporting says the city and local agencies moved to change how vulnerability is scored.
- Confidence
- Medium (multi-source)
A university used ChatGPT to write a consoling email after a campus shooting
An office at Vanderbilt University sent students a message of support after the Michigan State University shooting that had been written with ChatGPT, complete with a line disclosing the tool. After backlash over using AI for a human moment, the office apologized.
- Confidence
- Low (single source)
Palantir Gotham software in Hesse ruled unconstitutional
The German Federal Constitutional Court ruled in February 2023 that Palantir's Gotham software used by the Hesse State Police violated privacy rights. The court suspended mass data analysis due to insufficient legal safeguards.
- Confidence
- High (multi-source, primary)
Detroit police facial recognition misidentified a pregnant woman, causing a wrongful arrest
On February 16, 2023, Detroit police arrested Porcha Woodruff, who was eight months pregnant, after DataWorks Plus facial recognition software matched her to surveillance footage of a carjacking and robbery suspect. She was held for approximately 11 hours at the Detroit Detention Center before being released on a $100,000 personal bond, and the criminal case was dismissed on March 6, 2023 for insufficient evidence. Woodruff filed a federal civil rights lawsuit in August 2023, which was dismissed in September 2025 after the judge ruled the detective had probable cause at the time of the arrest.
- Confidence
- High (multi-source, primary)
IRS audit selection algorithms disproportionately target Black taxpayers
Stanford researchers found that Black taxpayers were audited at 2.9 to 4.7 times the rate of non-Black taxpayers, with the disparity most pronounced among EITC claimants. The IRS confirmed these findings in a May 2023 letter to Congress after an internal review, and multiple outlets corroborated the disparity and its attribution to audit-selection algorithms.
- Confidence
- High (multi-source, primary)
Allegheny Family Screening Tool faces DOJ scrutiny for automated bias
The Allegheny County DHS AFST tool faced DOJ civil-rights scrutiny over automated bias against marginalized families, with NGO reporting highlighting proxy-based discrimination.
- Confidence
- Medium (multi-source)
Randal Quran Reid wrongfully arrested due to facial recognition misidentification
Randal Quran Reid was wrongfully arrested in Georgia due to a facial recognition error by the Jefferson Parish Sheriff's Office. The agency relied on an incorrect match without verifying if the subject had ever visited Louisiana. The incident led to a lawsuit and a subsequent $200,000 settlement.
- Confidence
- Medium (multi-source)
VioGén risk-assessment used by Spanish National Police misclassified victims
An academic review and investigative reporting documented transparency, accuracy, and governance problems with VioGén, the Spanish police risk-assessment tool overseen by the Interior Ministry. Reporting and analyses found that the system classified many cases as negligible or low risk and that some victims later suffered repeat attacks or were killed, prompting rights and oversight concerns.
- Confidence
- High (multi-source, primary)
Canadian proctoring biometrics found to fail legal thresholds for consent and discrimination
An academic report from the University of Ottawa, supported by the Office of the Privacy Commissioner of Canada, found that widely used online exam proctoring tools collect biometric and personal data under conditions that do not meet Canadian legal standards for meaningful consent and raise privacy and discrimination concerns. Press coverage and the OPC project page documented the report’s findings in November-December 2022, noting risks from AI-driven facial detection and monitoring as well as cross-border data control issues.
- Confidence
- High (multi-source, primary)
Cleveland State University room-scan proctoring ruled to violate student privacy
In Ogletree v. Cleveland State University a federal judge found that the university's requirement for a student to perform a webcam room scan as part of remote exam proctoring violated the student's privacy. The case concerned the use of online proctoring software and the university's mandate that students show their surroundings before taking exams. The court opinion and multiple news outlets reported on the ruling in August 2022.
- Confidence
- High (multi-source, primary)
Chicago Police ShotSpotter false positives led to unlawful stops, Williams v City of Chicago
The Williams v. City of Chicago case centers on ShotSpotter data leading to stops and searches; in 2025 the City settled for $90,000 and acknowledged that ShotSpotter alerts alone do not justify police stops.
- Confidence
- High (multi-source, primary)
Oregon drops child welfare AI tool over racial bias concerns
ODHS phased out a risk-scoring AI tool used to determine which families are investigated for child abuse and neglection after findings that it disproportionately flagged Black families, replacing it with a human-led Structured Decision Making model.
- Confidence
- Medium (multi-source)
ID.me facial recognition failures lock unemployment beneficiaries out of systems
ID.me deployed a facial recognition system to verify unemployment claimants and prevent fraud. The system's failure to accurately identify many legitimate users led to widespread lockouts and delayed benefit payments.
- Confidence
- High (multi-source, primary)
Serbia Social Card registry automation causes benefit losses for marginalized groups
Serbia implemented a Social Card registry to automate eligibility for social assistance. The system used inaccurate and misclassified data, leading to the loss of benefits for thousands of marginalized people.
- Confidence
- High (multi-source, primary)
DWP disability benefits fraud algorithm criticized for lack of transparency
The UK Department for Work and Pensions faced legal challenges over its General Matching Service algorithm used to detect benefit fraud. Critics and disabled people's rights groups alleged the system was unfair and lacked transparency.
- Confidence
- Medium (multi-source)
Jordan Takaful poverty targeting algorithm excludes vulnerable families
The Jordanian government's Takaful program used an algorithm to rank social protection applicants, which unfairly excluded poor families. The system relied on 57 socioeconomic indicators that failed to capture the complex realities of poverty.
- Confidence
- Medium (multi-source)
Haryana Family ID system wrongly declares thousands of living citizens dead
The Haryana government's Parivar Pehchan Patra (PPP) system used AI to automate welfare eligibility, but erroneously marked thousands of living people as deceased. This led to the immediate suspension of critical old-age, widow, and disability pensions for eligible beneficiaries.
- Confidence
- Medium (multi-source)
Gizmodo analysis finds PredPol predictions targeted Black, Latino, and low-income areas
Independent analysis of PredPol prediction logs found the software repeatedly generated predictions concentrated in Black, Latino, and lower-income neighborhoods. The findings, reported by Gizmodo/The Markup and discussed in multiple news outlets, showed patterns consistent with bias arising from the model's training data and operational use.
- Confidence
- High (multi-source, primary)
Chinese authorities used facial recognition and emotion-detection to profile Uyghurs in Xinjiang
Independent reporting and rights-group investigations document that Chinese authorities deployed facial-recognition and emotion-detection systems as part of an integrated surveillance program in Xinjiang. Human Rights Watch reverse-engineered the IJOP policing app and described how biometric and behavioral data feed flagging systems, and the BBC reported that emotion-detection cameras were tested in Xinjiang police stations. These technologies were used to identify, flag, and investigate Uyghurs and other Turkic Muslims.
- Confidence
- High (multi-source, primary)
São Paulo Metro facial recognition system halted by court over privacy concerns
In May 2021 a São Paulo court ordered ViaQuatro to stop capturing passengers' images and biometric data with facial-recognition technology after civil-society organizations challenged the deployment on privacy grounds. The court decision, reported by major Brazilian outlets and advocacy groups, found that data such as gender, age and emotional metrics had been collected without proper authorization and imposed a monetary sanction. The episode drew attention from rights groups and news media and resulted in continuing litigation.
- Confidence
- High (multi-source, primary)
Aadhaar facial recognition failures risk excluding citizens from COVID-19 vaccines
The Indian government's use of Aadhaar facial recognition for vaccine authentication sparked concerns over widespread exclusion. Critics argued the system's inaccuracies and lack of consideration for aging faces would deny vulnerable citizens access to healthcare.
- Confidence
- Medium (multi-source)
UW-Madison disables Honorlock exam pause after students report facial detection failures
On 2021-03-11 UW-Madison disabled the Exam Pause feature in Honorlock after three students reported the feature activated when the software failed to detect their faces. The actions and complaints were reported by multiple news outlets and the university’s assessment/proctoring page confirms the feature is no longer enabled. Honorlock disputed that the issue was a racial-detection failure, saying pauses could be explained by students looking away from cameras.
- Confidence
- High (multi-source, primary)
UT Austin scrapped its GRADE machine-learning PhD admissions system over entrenched bias
UT Austin's Department of Computer Science used GRADE, a machine-learning system trained on past admissions decisions, to score and organize PhD applications from 2013 through 2019. Critics identified that the system reproduced historical inequities by encoding institutional prestige bias and linguistic patterns from recommendation letters that disadvantaged underrepresented groups. The university discontinued GRADE in 2020, officially citing maintenance difficulties, though the announcement coincided with public criticism about its fairness.
- Confidence
- High (multi-source, primary)
Ofqual's grading algorithm downgraded 39% of A-level results before being reversed in days
In August 2020, Ofqual deployed a statistical standardisation algorithm to moderate teacher-predicted A-level grades after COVID-19 cancelled summer exams. The algorithm downgraded approximately 39% of results, with students at historically lower-performing state schools hit hardest while private school students benefited from more favorable adjustments. Following nationwide protests and political pressure, the government reversed the decision on August 17 and replaced algorithm grades with teacher-assessed Centre Assessment Grades.
- Confidence
- High (multi-source, primary)
Aurora police ALPR false match led to family detained at gunpoint
In early August 2020 Aurora, Colorado officers stopped a Black mother and several children after an Automated License Plate Reader reportedly flagged the family's vehicle as matching a stolen motorcycle registered in another state. Officers conducted a high-risk stop, drew weapons, and several children were handcuffed; officers later determined the vehicle was not stolen. The City of Aurora reached a $1.9 million settlement with the family in February 2024.
- Confidence
- Medium (multi-source)
UK Home Office drops biased visa filtering algorithm
The UK Home Office suspended a visa-streaming tool in August 2020 following allegations of racial bias. The system used nationality to categorize applicants, creating a tiered scrutiny process that disadvantaged specific countries.
- Confidence
- Medium (multi-source)
Reno police facial recognition misidentified an innocent man, leading to a $100,000 settlement
Reno Police Department used DataWorks Plus facial recognition software to match a surveillance photo to an innocent individual, resulting in a wrongful arrest. The City of Reno settled the resulting civil rights lawsuit for $100,000 and agreed to policy changes restricting facial recognition use. The department had no formal training or policies governing facial recognition technology at the time of the incident, and also maintained documented use of Clearview AI for separate searches.
- Confidence
- Medium (multi-source)
Gothenburg school placement algorithm uses straight-line distance
The City of Gothenburg's school placement algorithm failed by using straight-line distance instead of actual routing to assign students to schools. This led to incorrect assignments and public outcry in May 2020.
- Confidence
- Medium (multi-source)
Dutch government SyRI fraud detection algorithm ruled illegal
The Dutch government used the SyRI algorithm to identify potential social welfare fraud. In February 2020, the District Court of The Hague ruled the system illegal for violating European privacy laws.
- Confidence
- High (multi-source, primary)
US government asylum claim denied due to AI translation error
A Pashto-speaking refugee's asylum bid was rejected by a US court after an AI translation tool incorrectly changed "I" to "we" in her written statement. This created a perceived contradiction with her oral testimony, leading to the denial of her asylum claim.
- Confidence
- Medium (multi-source)
Woodbridge Police Department wrongfully arrests man via facial recognition
The Woodbridge Police Department arrested Nijeer Parks for shoplifting after facial recognition software incorrectly identified him as a suspect. Parks was jailed for ten days despite being 30 miles away during the crime.
- Confidence
- Medium (multi-source)
Buenos Aires facial recognition system causes numerous wrongful arrests
The City of Buenos Aires implemented an AI facial recognition system for public security that resulted in over 140 false identifications and wrongful detentions. This led to a legal battle and a court ruling that declared the program's implementation unconstitutional.
- Confidence
- Medium (multi-source)
Bahia facial recognition pilot allegedly targets Black and poor populations
The Government of Bahia deployed a facial recognition pilot for public security that allegedly exhibited severe racial bias. The system disproportionately targeted Black and poor individuals, leading to concerns over wrongful identifications.
- Confidence
- Medium (multi-source)
US State Education Departments' automated essay scoring found biased against some groups
Automated essay scoring engines were used in many U.S. state standardized tests and multiple investigations and research studies found systematic differences in scores across demographic groups. Reporting and peer-reviewed analysis (including an ETS technical study) showed some engines gave higher average scores to certain groups and lower scores to others, and that some systems could be fooled by nonsense text.
- Confidence
- High (multi-source, primary)
Immigration New Zealand profiles overstayers using predictive data model
In April 2018 reporting revealed Immigration New Zealand had been piloting a data‑modelling programme that used historical demographic and outcome data to build risk profiles of overstayers. Officials described it as a pilot to prioritise cases likely to cause 'harm,' while critics alleged it enabled racial profiling and lacked adequate oversight. The disclosure prompted public debate and scrutiny over the fairness of automated profiling in immigration enforcement.
- Confidence
- High (multi-source, primary)
Parcoursup 2018 rollout drew controversy over opaque and allegedly unfair allocation outcomes
The French national admissions platform Parcoursup was launched in January 2018 to replace the previous centralized system. Within months the rollout generated sustained criticism in major outlets about opacity and allegedly unfair matching outcomes, and subsequent analyses documented how the sequential allocation mechanism and off-platform offers could produce inefficient or surprising assignments. Official reviewers and academic researchers later examined these design features and their consequences.
- Confidence
- High (multi-source, primary)
Keolis-operated Navya shuttle struck by truck in Las Vegas during first-day service
A Navya-built autonomous shuttle operated by Keolis was struck by a delivery truck in Las Vegas on November 8, 2017 while on its inaugural public run. Multiple news outlets and a subsequent NTSB investigation reported that the truck was backing up and was cited by police, no injuries were reported, and the collision caused only minor damage.
- Confidence
- High (multi-source, primary)
Metropolitan Police facial recognition trial at Notting Hill Carnival reports 98 percent error rate
The Metropolitan Police Service deployed live facial recognition technology during the 2017 Notting Hill Carnival. An audit later revealed that the system incorrectly identified the vast majority of potential matches.
- Confidence
- High (multi-source, primary)
Google Translate deemed inadequate for obtaining search consent in US federal court
In the case of United States v. Cruz-Zamora, a federal judge ruled that Google Translate's inaccuracy made it an insufficient tool for officers to obtain unequivocal consent for a warrantless search. This ruling led to the suppression of narcotics seized during the stop.
- Confidence
- Medium (multi-source)
New Zealand passport photo checker rejects applicant's open eyes as closed
In December 2016 an online passport photo checker run by New Zealand's Department of Internal Affairs rejected a photo from Richard Lee, a New Zealander of Asian descent, with the generic error "subject eyes are closed" even though his eyes were open. Major news outlets reported the system later accepted a different photo and the department said shadowing and uneven lighting commonly cause such automatic rejections.
- Confidence
- Medium (multi-source)
Researchers find systemic racial bias in PredPol crime forecasting software
A 2016 study revealed that PredPol's predictive policing software produced biased outputs that disproportionately targeted minority communities. The findings indicated that the AI reinforced existing policing patterns rather than predicting actual crime levels.
- Confidence
- High (multi-source, primary)
Services Australia Robodebt algorithm unlawfully issued welfare debt notices
Services Australia implemented an automated data-matching system that wrongly calculated welfare debts using an unlawful averaging method. The scheme affected approximately 400,000 people and ended in a $1.2 billion settlement.
- Confidence
- High (multi-source, primary)
ProPublica analysis finds COMPAS recidivism risk scores biased against Black defendants
A ProPublica investigation alleged that the COMPAS risk assessment tool exhibited systemic racial bias. The analysis found that Black defendants were flagged as high risk at higher rates than white defendants, even when their actual recidivism rates were similar.
- Confidence
- Medium (multi-source)
Pakistan biometric ID system compromised by Taliban leader identity fraud
The Afghan Taliban leader Akhtar Mansour was found to possess a valid Pakistani biometric ID card issued by NADRA. This security failure led the Pakistani government to launch a nationwide reverification campaign that resulted in the blocking of hundreds of thousands of citizens' identities.
- Confidence
- Medium (multi-source)
Chicago police Heat List criticized for racial bias and ineffectiveness
The Chicago Police Department's Strategic Subject List (SSL), known as the Heat List, was designed to predict individuals likely to be involved in shootings. Independent analysis by Upturn and the RAND Corporation found the system was ineffective at reducing violence and disproportionately targeted individuals based on age and systemic bias.
- Confidence
- Medium (multi-source)
UK Home Office algorithm targets specific nationalities for sham marriage fraud review
The UK Home Office used an automated algorithm to identify potential sham marriages, which was found to be biased against specific nationalities. Legal challenges were brought forward after evidence showed the system disproportionately flagged people from Greece, Albania, Bulgaria, and Romania.
- Confidence
- Medium (multi-source)