Australian Government Robodebt system issues false debt notices
The Australian Government's Robodebt scheme wrongly accused hundreds of thousands of welfare recipients of overpayments. The system used flawed algorithmic averaging to calculate debts, resulting in a $1.2 billion settlement.
The system created fictitious debts by averaging annual income to guess fortnightly earnings.
Key facts
- What
- The Australian Government's Robodebt scheme wrongly accused hundreds of thousands of welfare recipients of overpayments.
- Incident date
- Jan 1, 2018
- Who
- Australian Government (Centrelink)
- Failure mode
- Data Leakage
- AI surface
- Algorithmic Decision
- Severity
- High
What happened
The Australian Government's Robodebt scheme used an automated system to identify welfare overpayments. It incorrectly told thousands of citizens they owed money by averaging their annual income over the period they received benefits. This led to widespread financial distress and a $1.2 billion settlement in 2020.
What broke inside the model
- 01 · TriggerA request triggers retrieval or context loading.
- 02 · Model stepThe context pulls in another user's content.
- 03 · Control gapNo boundary enforces isolation at the moment of output.
- 04 · FailurePrivate data crosses into the response.
- 05 · ConsequenceOne user sees another's data, and disclosure follows.
One user's content crosses the retrieval boundary into another's response.
The system failed by using income averaging to estimate fortnightly income from annual ATO data. This method ignored actual fluctuations in a recipient's income, which created fictitious debts.
What it cost
Sources
- PressRecord payout for 'illegal' Australian welfare scheme victimsbbc.com
- PrimaryRobodebt class action settlement - Managing your moneyservicesaustralia.gov.au
Cite this entry
https://failureindex.ai/failures/australian-government-robodebt-issues-false-debtAI Failure Index. "Australian Government Robodebt system issues false debt notices" (FI-0469). Realm Labs. https://failureindex.ai/failures/australian-government-robodebt-issues-false-debt (indexed Jun 10, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0469. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
Realm can detect when a response is about to emit data that falls outside the bounds of the current user and context, and block or redact it inline, at the moment of generation rather than after the data has left.