Vendors and modelsDeployer
Department for Work and Pensions AI failures
Every documented AI failure involving Department for Work and Pensions on the AI Failure Index, classified by the mechanism that broke.
- Failures
- 6
- Highest severity
- High
- Span
- 2021 to 2024
- Failure modes
- 3
DWP AI fraud detection system found to be biased against vulnerable groups
An AI system used by the UK's Department for Work and Pensions to detect fraud in Universal Credit advance claims was found to be biased. An internal fairness analysis revealed that the system disproportionately flagged certain demographic groups for investigation.
- Confidence
- Medium (multi-source)
A DWP algorithm wrongly flagged over 200,000 housing-benefit claimants for fraud over three years
The UK Department for Work and Pensions deployed a risk-based verification algorithm to flag housing benefit claims for fraud review, but the system produced massive false positives. Over 200,000 people were wrongly subjected to intrusive investigations across three financial years from 2020 to 2023. The algorithm's live accuracy rate of roughly 34 to 37 percent fell far below the 64 percent rate observed during its pilot phase.
- Confidence
- High (multi-source, primary)
DWP disability benefits fraud algorithm criticized for lack of transparency
The UK Department for Work and Pensions faced legal challenges over its General Matching Service algorithm used to detect benefit fraud. Critics and disabled people's rights groups alleged the system was unfair and lacked transparency.
- Confidence
- Medium (multi-source)
DWP algorithm criticised for bias and wrongful disability fraud referrals
Campaigners, journalists and researchers raised concerns that the Department for Work and Pensions' (DWP) General Matching Service algorithm used to detect benefit fraud was opaque and biased, and sought legal remedies to force disclosure. Subsequent reporting and FOI material showed the system wrongly flagged large numbers of claimants and internal analyses reported statistically significant bias by disability, age and nationality.
- Confidence
- High (multi-source, primary)
DWP fraud detection algorithm targets disabled claimants
The UK DWP used a secretive algorithm to flag Universal Credit claimants for fraud investigations. The system was found to disproportionately target disabled people, which led to a legal challenge and a public admission of bias by the DWP.
- Confidence
- Medium (multi-source)
UK DWP Universal Credit fraud model shows bias in age and nationality referrals
An internal assessment found statistically significant bias in the UC Advances model, disproportionately flagging non-UK nationals and certain age groups for fraud investigations without a corresponding gain in correct identifications.
- Confidence
- High (multi-source, primary)
See how Realm catches these failure modes at runtime, before they reach production.
Explore Realm Labs