Services Australia Robodebt algorithm unlawfully issued welfare debt notices
Services Australia implemented an automated data-matching system that wrongly calculated welfare debts using an unlawful averaging method. The scheme affected approximately 400,000 people and ended in a $1.2 billion settlement.
The algorithm replaced actual fortnightly income with an unlawful annual average, creating artificial debts.
Key facts
- What
- Services Australia implemented an automated data-matching system that wrongly calculated welfare debts using an unlawful averaging method.
- Incident date
- Jul 1, 2016
- Who
- Services Australia
- Failure mode
- Policy Violation
- AI surface
- Agentic Workflow
- Severity
- Catastrophic
What happened
Services Australia implemented an automated debt assessment system to identify welfare overpayments. The system used an unlawful income-averaging method to calculate debts, resulting in broad financial distress and a government settlement of $1.2 billion. The Royal Commission subsequently held the scheme unlawful and systemic in its failure.
What broke inside the model
- 01 · TriggerThe system cross-references annual tax income against fortnightly welfare reports.
- 02 · Model stepIt averages annual income across fortnights, presuming steady earnings for people whose defining trait is irregular work.
- 03 · Control gapThe legal requirement for individualized evidence is replaced by an automated presumption of debt, with no human check.
- 04 · FailureHundreds of thousands of false or inflated debt notices issue automatically.
- 05 · ConsequenceA Royal Commission, a A$1.8B settlement, and the canonical case of automated administrative harm.
The algorithm cross-referenced annual income data with fortnightly payment records and averaged annual income to estimate fortnightly earnings, ignoring income timing and violating the Social Security Act.
What it cost
Sources
- PressLearning from the failures of Robodebtlegalaid.vic.gov.au
- PrimaryReport | Royal Commission into the Robodebt Schemerobodebt.royalcommission.gov.au
- Reader-SubmittedRobodebt scheme - Wikipediaen.wikipedia.org
- PrimaryRoyal Commission into the Robodebt Schemerobodebt.royalcommission.gov.au
- PressRecord payout for 'illegal' Australian welfare scheme victimsbbc.com
Also cataloged in: AIID Incident 57
Cite this entry
https://failureindex.ai/failures/services-australia-robodebt-algorithm-unlawfully-issuedAI Failure Index. "Services Australia Robodebt algorithm unlawfully issued welfare debt notices" (FI-0191). Realm Labs. https://failureindex.ai/failures/services-australia-robodebt-algorithm-unlawfully-issued (indexed Jun 5, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0191. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
Realm compares what the model is about to output or do against the policy that governs the deployment, in real time, and can deny or redact the action before it takes effect, which is the gap an after-the-fact review never closes in time.