UK Home Office algorithm targets specific nationalities for sham marriage fraud review
The UK Home Office used an automated algorithm to identify potential sham marriages, which was found to be biased against specific nationalities. Legal challenges were brought forward after evidence showed the system disproportionately flagged people from Greece, Albania, Bulgaria, and Romania.
The Home Office's own equality impact assessment found it was flagging a disproportionate number of marriages from Greece, Albania, Bulgaria and Romania.
Key facts
- What
- The UK Home Office used an automated algorithm to identify potential sham marriages, which was found to be biased against specific nationalities.
- Incident date
- Mar 1, 2015
- Who
- United Kingdom Home Office
- Failure mode
- Policy Violation
- AI surface
- Algorithmic Decision
- Severity
- High
What happened
The UK Home Office deployed an algorithm to identify potential sham marriages for investigation. An internal equality impact assessment revealed that the system disproportionately targeted nationalities including Greece, Albania, Bulgaria, and Romania. This led to legal challenges from organizations like the Public Law Project and Foxglove.
What broke inside the model
- 01 · TriggerA prompt pushes against a deployment boundary.
- 02 · Model stepThe model produces the disallowed output.
- 03 · Control gapNo enforcement blocks it at generation time.
- 04 · FailureThe output crosses the policy line.
- 05 · ConsequenceA limit the business set is breached in public.
The output crosses a policy boundary the deployment had defined.
The system utilized a set of rules that disproportionately flagged specific nationalities for fraud review. This was caused by a design bias in the algorithm's rules, which targeted individuals based on their country of origin.
What it cost
Sources
- PressHome Office algorithm to detect sham marriages may contain biasthebureauinvestigates.com
- Press'Sham marriages' and algorithmic decision-making in the Home Officepubliclawproject.org.uk
Cite this entry
https://failureindex.ai/failures/home-office-algorithm-targets-specific-nationalitiesAI Failure Index. "UK Home Office algorithm targets specific nationalities for sham marriage fraud review" (FI-0485). Realm Labs. https://failureindex.ai/failures/home-office-algorithm-targets-specific-nationalities (indexed Jun 10, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0485. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm fits
- Prism
- OmniGuard
This entry sits in the index's predictive wing: a system that scores, ranks, perceives, or steers rather than generates. Realm's runtime layer is built for the generative and agentic systems now moving into these same decision seats, where it watches a model's internal state and holds an unsupported claim or an unchecked action before it commits. The control gap on this record, an automated decision that reached people with no runtime check in front of it, is the same gap. The index keeps predictive failures on the record because the pattern carries straight into the systems shipping today.