Rotterdam welfare fraud model used discriminatory data and performed poorly
A Rotterdam welfare fraud model allegedly used discriminatory data and performed no better than random; two independent outlets describe bias and limited usefulness of the system.
Experts described the Rotterdam welfare algorithm as essentially random guessing that is not useful in the real world.
Key facts
- What
- A Rotterdam welfare fraud model allegedly used discriminatory data and performed no better than random; two independent outlets describe bias and limited usefulness of the system.
- Incident date
- Mar 6, 2023
- Who
- City of Rotterdam
- Failure mode
- Policy Violation
- AI surface
- Algorithmic Decision
- Severity
- High
What happened
The City of Rotterdam deployed a machine learning model to identify potential welfare fraud among its 30,000 benefit recipients. Investigative reporting found that the system discriminated based on ethnicity, age, gender, and parenthood, and that its accuracy was little better than random. The data practices involved many inputs, including subjective caseworker assessments, raising concerns about bias and utility.
What broke inside the model
- 01 · TriggerA prompt pushes against a deployment boundary.
- 02 · Model stepThe model produces the disallowed output.
- 03 · Control gapNo enforcement blocks it at generation time.
- 04 · FailureThe output crosses the policy line.
- 05 · ConsequenceA limit the business set is breached in public.
The output crosses a policy boundary the deployment had defined.
The system used 315 inputs, including subjective assessments and demographic data, which introduced systemic bias and undermined predictive validity. The model produced risk scores without a reliable link to actual fraud.
What it cost
Sources
- PressSuspicion Machineslighthousereports.com
- PressThis Algorithm Could Ruin Your Lifewired.com
Cite this entry
https://failureindex.ai/failures/rotterdam-welfare-fraud-used-discriminatory-performedAI Failure Index. "Rotterdam welfare fraud model used discriminatory data and performed poorly" (FI-0247). Realm Labs. https://failureindex.ai/failures/rotterdam-welfare-fraud-used-discriminatory-performed (indexed Jun 5, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0247. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm fits
- Prism
- OmniGuard
This entry sits in the index's predictive wing: a system that scores, ranks, perceives, or steers rather than generates. Realm's runtime layer is built for the generative and agentic systems now moving into these same decision seats, where it watches a model's internal state and holds an unsupported claim or an unchecked action before it commits. The control gap on this record, an automated decision that reached people with no runtime check in front of it, is the same gap. The index keeps predictive failures on the record because the pattern carries straight into the systems shipping today.