AI Failure Index
AI Computer Vision failures
Image and video recognition: facial recognition, object detection, visual inspection. Failures misidentify people and things with high confidence.
- Incidents
- 49
- Highest severity
- High
- Sources cited
- 136
- Newest indexed
- Jun 16, 2026
India's Poshan Tracker facial-recognition excludes eligible beneficiaries
The Poshan Tracker facial-recognition system failed to recognise mothers, excluding families from meals, preschool education, and health monitoring; government data cited a 52.7% ration delivery rate by end-2025.
- Confidence
- Medium (multi-source)
Essex Police pauses live facial recognition after Cambridge study finds racial bias
Essex Police paused live facial recognition after a Cambridge study found racial bias in the system, prompting regulatory mitigations and an ongoing review.
- Confidence
- High (multi-source, primary)
Home Bargains shoppers wrongfully accused by Facewatch facial recognition
The deployment of Facewatch facial recognition at Home Bargains led to the misidentification of innocent shoppers. This resulted in wrongful accusations of theft by store security and the sending of false evidence to customers.
- Confidence
- Medium (multi-source)
Sainsbury's customer wrongly ejected after facial recognition error
A customer at a Sainsbury's store in Elephant and Castle was misidentified as a known offender by the Facewatch facial recognition system. Although the system issued an alert, the incident was categorized as a human error where staff approached the wrong individual. Sainsbury's apologized and provided a voucher to the affected customer.
- Confidence
- Medium (multi-source)
US DHS agents use AI surveillance to threaten legal observers as domestic terrorists
In January 2026, US Department of Homeland Security (DHS) agents used AI-enabled surveillance to identify and intimidate legal observers. In one instance, an agent threatened an observer by claiming she was now considered a domestic terrorist in a government database.
- Confidence
- Medium (multi-source)
US Border Patrol facial recognition scan leads to Global Entry revocation
A US Border Patrol agent identified a neighborhood observer using facial recognition software, which was allegedly followed by the revocation of the observer's Global Entry status. The incident is reported as part of a pattern of surveillance and intimidation of protesters and observers.
- Confidence
- High (multi-source, primary)
Worldcoin suspended in Thailand over iris scanning privacy concerns
Thailand's Personal Data Protection Committee (PDPC) ordered Worldcoin to halt its iris scanning operations and delete over 1.2 million biometric records. The regulator concluded that the practice of trading biometric data for cryptocurrency breached the national Personal Data Protection Act.
- Confidence
- Medium (multi-source)
US law enforcement used ALPR networks to monitor protesters, raising privacy concerns
An investigation by the Electronic Frontier Foundation documented law enforcement use of Flock Safety automated license plate reader (ALPR) data to search for and track protesters and activists. Local governments and advocates responded with policy actions and contract terminations, and the vendor publicly defended its product.
- Confidence
- Medium (multi-source)
Roblox AI age verification system misidentifies minors as adults
Roblox deployed an AI facial scanning system to verify user ages, which subsequently failed by misclassifying minors as adults. This compromise of the age-gating mechanism undermined child safety efforts on the platform.
- Confidence
- Medium (multi-source)
Rotherham man mistaken for fraudster by facial recognition software
Craig Hadley was wrongly identified as a fraudster by facial recognition software at a Sports Direct store in Rotherham. The error led to him being accused of fraud and removed from the premises.
- Confidence
- Medium (multi-source)
NYPD facial recognition match leads to wrongful arrest of Trevis Williams
Two independent outlets report that NYPD used facial recognition to arrest Trevis Williams, despite height and location discrepancies, leading to jail time before charges were dismissed; advocacy groups are pushing for policy changes.
- Confidence
- Medium (multi-source)
City of Orléans audio surveillance ruled illegal by French court
A French administrative court ruled that the City of Orléans' deployment of AI-powered audio surveillance in public spaces was illegal. The court found that the system lacked a proper legal basis and infringed upon fundamental privacy rights.
- Confidence
- Medium (multi-source)
Foodstuffs facial recognition misidentifies Māori shopper at Rotorua New World
On 2024-04-02 a Māori woman shopping at New World Westend in Rotorua was approached by store staff and told she had been trespassed after a facial recognition alert from a Foodstuffs trial. The customer offered three forms of photo ID but was still asked to leave; Foodstuffs called it a genuine case of human error and said it reported the incident to the Office of the Privacy Commissioner. Experts and the Privacy Commissioner raised concerns about bias and accuracy in the trialled system, which was trained on international data and not specifically on New Zealand populations.
- Confidence
- High (multi-source, primary)
Worldcoin suspended in Spain after regulator orders halt to biometric data processing
Spain's Data Protection Agency (AEPD) issued a precautionary measure on 2024-03-06 preventing Worldcoin (Tools for Humanity) from processing personal data in Spain. The action followed complaints alleging insufficient information and concerns about the collection and processing of biometric iris scans. Subsequent reporting indicated Spanish authorities later ordered deletion of data collected in Spain.
- Confidence
- High (multi-source, primary)
Met Police facial recognition wrongly matched youth worker Shaun Thompson
In February 2024 Shaun Thompson, a youth advocacy worker, was stopped and questioned after the Metropolitan Police's live facial‑recognition system matched him to a watchlist entry. The encounter lasted around 30 minutes and ended when Thompson produced ID; he subsequently brought a High Court challenge to the Met's use of LFR, which was dismissed on 2026-04-21. Reporting on the case is documented by multiple independent outlets including the BBC and The Independent.
- Confidence
- Medium (multi-source)
Comune di Trento fined 50,000 euros for illegal AI surveillance projects
The Italian Garante Privacy fined Comune di Trento €50,000 for deploying AI systems that violated GDPR rules through insufficient anonymization and lack of impact assessments. The city was ordered to delete the collected data from the MARVEL and PROTECTOR projects.
- Confidence
- High (multi-source, primary)
PimEyes alleged to have been used to identify anonymous porn actors
News reporting and an incident repository document that PimEyes has been used to identify anonymous porn performers by matching images. Business Insider reported instances of the service being used to unmask porn actors and an AIAAIC repository entry records the same misuse.
- Confidence
- Medium (multi-source)
Communauté de communes Cœur Côte Fleurie ordered to delete AI-surveillance data
In November 2023 a French administrative court ordered the Communauté de communes Cœur Côte Fleurie to stop using an augmented camera system coupled with algorithmic video-surveillance and to delete personal data obtained via the system. The court concluded the system permitted automated identification and tracking of people and therefore constituted a serious and manifestly unlawful interference with privacy; the originals were placed under seal with the CNIL.
- Confidence
- High (multi-source, primary)
Canadian Tire stores used facial ID systems that breached B.C. privacy law
The Office of the Information and Privacy Commissioner for British Columbia concluded on April 20, 2023 that several Canadian Tire associate stores used facial recognition technology to capture images, create biometric templates, and compare visitors against a Persons of Interest database without adequate notice or consent, breaching the Personal Information Protection Act. The investigation covered four stores directly and noted up to 12 stores had used the technology; the systems were removed and the OIPC recommended stronger regulation and improved privacy management. No financial penalties were reported in the public record.
- Confidence
- High (multi-source, primary)
Madison Square Garden facial recognition flags lawyers and denies entry
In late 2022, news outlets reported that Madison Square Garden Entertainment used facial‑recognition software to match attendees against an exclusion list of lawyers affiliated with firms suing the company, and several attorneys with valid tickets were turned away from events. The policy and its enforcement prompted multiple lawsuits and a formal inquiry by New York Attorney General Letitia James. Critics and lawmakers alleged the system produced wrongful exclusions and chilled legal advocacy; MSG defended the policy as a security measure.
- Confidence
- High (multi-source, primary)
Randal Quran Reid wrongfully arrested due to facial recognition misidentification
Randal Quran Reid was wrongfully arrested in Georgia due to a facial recognition error by the Jefferson Parish Sheriff's Office. The agency relied on an incorrect match without verifying if the subject had ever visited Louisiana. The incident led to a lawsuit and a subsequent $200,000 settlement.
- Confidence
- Medium (multi-source)
Canadian proctoring biometrics found to fail legal thresholds for consent and discrimination
An academic report from the University of Ottawa, supported by the Office of the Privacy Commissioner of Canada, found that widely used online exam proctoring tools collect biometric and personal data under conditions that do not meet Canadian legal standards for meaningful consent and raise privacy and discrimination concerns. Press coverage and the OPC project page documented the report’s findings in November-December 2022, noting risks from AI-driven facial detection and monitoring as well as cross-border data control issues.
- Confidence
- High (multi-source, primary)
Cleveland State University room-scan proctoring ruled to violate student privacy
In Ogletree v. Cleveland State University a federal judge found that the university's requirement for a student to perform a webcam room scan as part of remote exam proctoring violated the student's privacy. The case concerned the use of online proctoring software and the university's mandate that students show their surroundings before taking exams. The court opinion and multiple news outlets reported on the ruling in August 2022.
- Confidence
- High (multi-source, primary)
ID.me facial recognition failures lock unemployment beneficiaries out of systems
ID.me deployed a facial recognition system to verify unemployment claimants and prevent fraud. The system's failure to accurately identify many legitimate users led to widespread lockouts and delayed benefit payments.
- Confidence
- High (multi-source, primary)
Meta settles Texas facial recognition lawsuit for $1.4 billion
Meta agreed to pay $1.4 billion to resolve a lawsuit brought by the Texas Attorney General regarding the unauthorized use of biometric data. The case alleged the company captured facial data from users without their informed consent.
- Confidence
- Medium (multi-source)
Steak 'n Shake sued for alleged facial biometric violations
Steak 'n Shake is facing a class action lawsuit for allegedly violating the Biometric Information Privacy Act (BIPA). The suit claims the company illegally collected facial biometric data from customers using PopID kiosks.
- Confidence
- Medium (multi-source)
Riverside Arena facial recognition system misidentifies Black teenager
A facial recognition system at the Riverside Arena skating rink in Livonia, Michigan, incorrectly identified a 14-year-old Black teenager as a banned individual. The girl had never visited the rink before the incident.
- Confidence
- Medium (multi-source)
Chinese authorities used facial recognition and emotion-detection to profile Uyghurs in Xinjiang
Independent reporting and rights-group investigations document that Chinese authorities deployed facial-recognition and emotion-detection systems as part of an integrated surveillance program in Xinjiang. Human Rights Watch reverse-engineered the IJOP policing app and described how biometric and behavioral data feed flagging systems, and the BBC reported that emotion-detection cameras were tested in Xinjiang police stations. These technologies were used to identify, flag, and investigate Uyghurs and other Turkic Muslims.
- Confidence
- High (multi-source, primary)
São Paulo Metro facial recognition system halted by court over privacy concerns
In May 2021 a São Paulo court ordered ViaQuatro to stop capturing passengers' images and biometric data with facial-recognition technology after civil-society organizations challenged the deployment on privacy grounds. The court decision, reported by major Brazilian outlets and advocacy groups, found that data such as gender, age and emotional metrics had been collected without proper authorization and imposed a monetary sanction. The episode drew attention from rights groups and news media and resulted in continuing litigation.
- Confidence
- High (multi-source, primary)
Uber Eats courier alleges racial bias after facial-verification mismatches and dismissal
A UK Uber Eats courier, Pa Edrissa Manjang, alleges he faced excessive facial-photo verification checks and was deactivated from the app in April 2021 after repeated mismatches. He brought a discrimination claim that a tribunal allowed to proceed and later received a payout, while Uber has said automated facial verification was not the reason for the temporary loss of access.
- Confidence
- Medium (multi-source)
Aadhaar facial recognition failures risk excluding citizens from COVID-19 vaccines
The Indian government's use of Aadhaar facial recognition for vaccine authentication sparked concerns over widespread exclusion. Critics argued the system's inaccuracies and lack of consideration for aging faces would deny vulnerable citizens access to healthcare.
- Confidence
- Medium (multi-source)
UW-Madison disables Honorlock exam pause after students report facial detection failures
On 2021-03-11 UW-Madison disabled the Exam Pause feature in Honorlock after three students reported the feature activated when the software failed to detect their faces. The actions and complaints were reported by multiple news outlets and the university’s assessment/proctoring page confirms the feature is no longer enabled. Honorlock disputed that the issue was a racial-detection failure, saying pauses could be explained by students looking away from cameras.
- Confidence
- High (multi-source, primary)
Google flags parent's medical photo of his toddler as suspected child abuse
In February 2021 a San Francisco father took photos of his toddler’s swollen genital area for a doctor; those images were backed up to Google Photos and were later flagged by Google’s automated child sexual abuse material (CSAM) detection system. Google locked the user’s accounts and reported the matter to the National Center for Missing and Exploited Children, prompting a police inquiry that investigators later closed with no charges. The episode was reported publicly by The New York Times on 2022-08-21 and covered by other outlets.
- Confidence
- Medium (multi-source)
Proctorio accused of racial bias in AI proctoring during online exams
Multiple news outlets reported in mid to late 2020 that Proctorio’s AI-based remote proctoring and facial-recognition tools were alleged to have discriminated against students, particularly students of color. Coverage and campus protests raised questions about biased detection and identity-verification failures in automated proctoring systems.
- Confidence
- Medium (multi-source)
University of Miami accused of using facial recognition to identify student protesters
Students at the University of Miami alleged that campus police used facial recognition technology to identify attendees of a September 2020 protest. The university denied the use of the technology, though reports indicated the police chief's resume previously cited such capabilities.
- Confidence
- Medium (multi-source)
Aurora police ALPR false match led to family detained at gunpoint
In early August 2020 Aurora, Colorado officers stopped a Black mother and several children after an Automated License Plate Reader reportedly flagged the family's vehicle as matching a stolen motorcycle registered in another state. Officers conducted a high-risk stop, drew weapons, and several children were handcuffed; officers later determined the vehicle was not stolen. The City of Aurora reached a $1.9 million settlement with the family in February 2024.
- Confidence
- Medium (multi-source)
Clearview AI scraped social media images to power law-enforcement facial search
Reporting in January 2020 revealed that Clearview AI collected millions of images from social media and other websites to build a facial-recognition database. The company offered a reverse-image search service to law enforcement, prompting privacy complaints, lawsuits, and regulatory actions including fines and settlements.
- Confidence
- High (multi-source, primary)
Google Health diabetic retinopathy AI fails in real world clinic settings
Google Health's AI for detecting diabetic retinopathy failed to maintain its laboratory accuracy when deployed in real world Indian clinics. The system was hindered by suboptimal environmental conditions and data quality issues.
- Confidence
- Medium (multi-source)
Woodbridge Police Department wrongfully arrests man via facial recognition
The Woodbridge Police Department arrested Nijeer Parks for shoplifting after facial recognition software incorrectly identified him as a suspect. Parks was jailed for ten days despite being 30 miles away during the crime.
- Confidence
- Medium (multi-source)
Buenos Aires facial recognition system causes numerous wrongful arrests
The City of Buenos Aires implemented an AI facial recognition system for public security that resulted in over 140 false identifications and wrongful detentions. This led to a legal battle and a court ruling that declared the program's implementation unconstitutional.
- Confidence
- Medium (multi-source)
Bahia facial recognition pilot allegedly targets Black and poor populations
The Government of Bahia deployed a facial recognition pilot for public security that allegedly exhibited severe racial bias. The system disproportionately targeted Black and poor individuals, leading to concerns over wrongful identifications.
- Confidence
- Medium (multi-source)
Apple alleged to have misidentified Ousmane Bah in store surveillance
A lawsuit filed in April 2019 alleges that Apple’s in‑store security system associated surveillance images of a shoplifter with Ousmane Bah, leading to his arrest on November 29, 2018. Independent news outlets reported the suit and Apple told reporters it does not use facial recognition in its stores. The court docket and complaint are publicly available.
- Confidence
- High (multi-source, primary)
Microsoft Face API shows bias in attribute tagging for different ethnicities
Microsoft's Azure Face API was found to have significant accuracy gaps when predicting attributes for people of color. Research indicated error rates as high as 20.8 percent for women with darker skin tones.
- Confidence
- Medium (multi-source)
IBM Watson visual recognition exhibits gender and race bias
A study by MIT researcher Joy Buolamwini revealed that IBM Watson's visual recognition software had a high error rate when identifying darker-skinned women. The findings highlighted significant algorithmic bias in the system.
- Confidence
- High (multi-source, primary)
MIT study finds Amazon Rekognition facial analysis least accurate for darker-skinned women
A 2018 study revealed that Amazon Rekognition exhibited significant inaccuracies in identifying gender and skin type. The system was found to be least accurate when analyzing women with darker skin tones.
- Confidence
- Medium (multi-source)
Metropolitan Police facial recognition trial at Notting Hill Carnival reports 98 percent error rate
The Metropolitan Police Service deployed live facial recognition technology during the 2017 Notting Hill Carnival. An audit later revealed that the system incorrectly identified the vast majority of potential matches.
- Confidence
- High (multi-source, primary)
New Zealand passport photo checker rejects applicant's open eyes as closed
In December 2016 an online passport photo checker run by New Zealand's Department of Internal Affairs rejected a photo from Richard Lee, a New Zealander of Asian descent, with the generic error "subject eyes are closed" even though his eyes were open. Major news outlets reported the system later accepted a different photo and the department said shadowing and uneven lighting commonly cause such automatic rejections.
- Confidence
- Medium (multi-source)
Pakistan biometric ID system compromised by Taliban leader identity fraud
The Afghan Taliban leader Akhtar Mansour was found to possess a valid Pakistani biometric ID card issued by NADRA. This security failure led the Pakistani government to launch a nationwide reverification campaign that resulted in the blocking of hundreds of thousands of citizens' identities.
- Confidence
- Medium (multi-source)
Google Photos labels Black individuals as gorillas
In 2015, Google's Photos app incorrectly tagged images of Black people as gorillas. The company apologized for the failure and took steps to prevent the specific label from appearing.
- Confidence
- Medium (multi-source)