Dutch Tax Authority childcare algorithm targets dual nationality families
The Dutch Tax Authority used a fraud-detection algorithm that discriminated against families with dual nationalities. This led to thousands of wrongful accusations of fraud and severe financial hardship for affected families.
The system used dual nationality as a risk signal, effectively automating racial discrimination in welfare fraud detection.
Key facts
- What
- The Dutch Tax Authority used a fraud-detection algorithm that discriminated against families with dual nationalities.
- Incident date
- Sep 1, 2018
- Who
- Dutch Tax Authority
- Failure mode
- Brand & Safety Incident
- AI surface
- Algorithmic Decision
- Severity
- Catastrophic
What happened
The Dutch Tax Authority used a risk scoring algorithm to detect childcare benefit fraud. The system incorrectly flagged thousands of families, particularly those with dual nationality, as high risk. This led to wrongful accusations of fraud and forced repayments that caused severe financial hardship for many families.
What broke inside the model
- 01 · TriggerA user prompts the model in public view.
- 02 · Model stepThe model produces unsafe or off-brand output.
- 03 · Control gapNo filter holds the line before publish.
- 04 · FailureThe output goes public unchecked.
- 05 · ConsequenceA reputational or safety incident lands.
A contained signal crosses into output that goes public.
The algorithm utilized dual nationality as a risk factor in its predictive model. This introduced systemic bias, as the system automatically assigned higher risk scores to immigrant and minority families regardless of actual fraud.
What it cost
Sources
- PressDutch scandal serves as a warning for Europe over risks of using algorithmspolitico.eu
- PressDutch childcare benefit scandal an urgent wake-up call to the worldamnesty.org
- PressDutch childcare benefits scandalen.wikipedia.org
Cite this entry
https://failureindex.ai/failures/dutch-tax-authority-childcare-algorithm-targetsAI Failure Index. "Dutch Tax Authority childcare algorithm targets dual nationality families" (FI-0670). Realm Labs. https://failureindex.ai/failures/dutch-tax-authority-childcare-algorithm-targets (indexed Jun 22, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0670. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
Realm watches the model's internal state for the signature of unsafe or off-brand generation and can block or reroute the output before it becomes public, in real time rather than after it has been screenshotted.