A DWP algorithm wrongly flagged over 200,000 housing-benefit claimants for fraud over three years
The UK Department for Work and Pensions deployed a risk-based verification algorithm to flag housing benefit claims for fraud review, but the system produced massive false positives. Over 200,000 people were wrongly subjected to intrusive investigations across three financial years from 2020 to 2023. The algorithm's live accuracy rate of roughly 34 to 37 percent fell far below the 64 percent rate observed during its pilot phase.
A pilot-promised 64 percent fraud detection rate collapsed to just 34 percent in production, subjecting over 200,000 innocent claimants to unnecessary fraud investigations.
Key facts
- What
- The UK Department for Work and Pensions deployed a risk-based verification algorithm to flag housing benefit claims for fraud review, but the system produced massive false positives.
- Incident date
- Jun 1, 2024
- Who
- UK Department for Work and Pensions (DWP)
- Failure mode
- Agentic Action Error
- AI surface
- Agentic Workflow
- Severity
- High
What happened
The DWP deployed a risk-based verification algorithm that weighed claimants' personal data to assign fraud risk scores to housing benefit claims. When flagged as high-risk, local council staff would contact claimants to validate their details, subjecting them to intrusive reviews. Over 200,000 people were wrongly investigated across financial years 2020-21, 2021-22, and 2022-23. The DWP spent approximately £4.4 million on official checks that produced no savings, while the affected claimants endured stressful and unnecessary fraud investigations.
What broke inside the model
- 01 · TriggerAn agent plans a multi-step task.
- 02 · Model stepIt chooses a wrong or destructive action.
- 03 · Control gapNo confirmation gate guards the write.
- 04 · FailureThe action commits to a system of record.
- 05 · ConsequenceData is changed or destroyed irreversibly.
A wrong action commits, and the step is written before anything can stop it.
The algorithm used personal characteristics such as age, gender, number of children, and tenancy type to assign risk scores, but these features proved poor predictors of actual fraud in live deployment. The pilot study indicated 64 percent accuracy in detecting erroneous claims, yet live performance dropped to approximately 34 to 37 percent, meaning two-thirds of all flagged cases were entirely legitimate. The gap between pilot expectations and production performance was never corrected, allowing systematic over-flagging to persist for three years.
What it cost
Sources
- PressDWP algorithm wrongly flags 200,000 people for possible fraud and errortheguardian.com
- PrimaryAIAAIC Repository: DWP algorithm wrongly flags 200,000 people for possible fraudaiaaic.org
- PrimaryRisk-based verification - GOV.UKassets.publishing.service.gov.uk
Cite this entry
https://failureindex.ai/failures/dwp-algorithm-wrongly-flagged-200-000AI Failure Index. "A DWP algorithm wrongly flagged over 200,000 housing-benefit claimants for fraud over three years" (FI-0109). Realm Labs. https://failureindex.ai/failures/dwp-algorithm-wrongly-flagged-200-000 (indexed Jun 4, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0109. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AgentRealm
Realm can sit inline on the agent's action path and require that a destructive or high-consequence action clears a real check before it executes, so 'delete and recreate' or a wrong write is stopped at the moment of intent, not explained in the post-mortem.