Centrelink robo-debt produced incorrect automated debt notices for welfare recipients
From 2016 the Australian Government's Centrelink used an automated income-matching process that generated debt notices for many welfare recipients. A Federal Court ruling in 2019 found the income-averaging method unlawful and the matter led to large government refunds and later a Royal Commission report in 2023.
An automated income-averaging workflow matched annual ATO figures to fortnightly Centrelink records and generated inferred debts without adequate verification.
Key facts
- What
- From 2016 the Australian Government's Centrelink used an automated income-matching process that generated debt notices for many welfare recipients.
- Incident date
- Jan 1, 2016
- Who
- Australian Department of Human Services / Centrelink
- Failure mode
- Policy Violation
- AI surface
- Algorithmic Decision
- Severity
- High
What happened
Beginning in 2016 Centrelink automated the process of identifying alleged overpayments by matching Centrelink payment records to annual ATO income data and issuing debt notices. The income was averaged rather than verified for each fortnight, producing incorrect debt calculations and notices for hundreds of thousands of people. In 2019 a Federal Court matter resolved by consent found that at least some debt notices were not validly made. The government subsequently paid refunds and reached settlements with affected people and the matter was examined by a Royal Commission that reported in 2023.
What broke inside the model
- 01 · TriggerA prompt pushes against a deployment boundary.
- 02 · Model stepThe model produces the disallowed output.
- 03 · Control gapNo enforcement blocks it at generation time.
- 04 · FailureThe output crosses the policy line.
- 05 · ConsequenceA limit the business set is breached in public.
The output crosses a policy boundary the deployment had defined.
The mechanism that failed was an automated income-averaging and data-matching workflow that inferred fortnightly income from annual ATO records and treated those inferences as the basis for debt notices without sufficient verification. That automated decision process and its legal basis were later found to be unlawful in court and deficient in oversight. The failure combined flawed algorithmic assumptions with policy and procedural gaps around human review and evidentiary standards.
What it cost
Sources
- PrimaryReport | Royal Commission into the Robodebt Schemerobodebt.royalcommission.gov.au
- PressGovernment admits robodebt was unlawful as it settles legal challengetheguardian.com
- PressRobodebt: Illegal Australian welfare hunt drove people to ...bbc.com
Cite this entry
https://failureindex.ai/failures/centrelink-robo-debt-produced-incorrect-automatedAI Failure Index. "Centrelink robo-debt produced incorrect automated debt notices for welfare recipients" (FI-0427). Realm Labs. https://failureindex.ai/failures/centrelink-robo-debt-produced-incorrect-automated (indexed Jun 10, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0427. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
Realm compares what the model is about to output or do against the policy that governs the deployment, in real time, and can deny or redact the action before it takes effect, which is the gap an after-the-fact review never closes in time.