COMPAS risk assessment tool exhibits racial bias in recidivism predictions
The COMPAS recidivism risk tool was found to exhibit racial bias, incorrectly labeling Black defendants as high risk more frequently than white defendants. This led to widespread public criticism and legal challenges regarding the fairness and transparency of the closed-source algorithm.
The algorithm failed to achieve equalized odds across racial groups, producing a high false-positive rate for Black defendants.
Key facts
- What
- The COMPAS recidivism risk tool was found to exhibit racial bias, incorrectly labeling Black defendants as high risk more frequently than white defendants.
- Incident date
- May 23, 2016
- Who
- Equivant
- Failure mode
- Brand & Safety Incident
- AI surface
- Algorithmic Decision
- Severity
- High
What happened
In May 2016, an investigation by ProPublica revealed that the COMPAS risk-assessment tool exhibited significant racial bias in predicting recidivism. Black defendants were found to be twice as likely as white defendants to be incorrectly flagged as high risk. These scores were used by judges to inform sentencing and pretrial release decisions.
What broke inside the model
- 01 · TriggerA user prompts the model in public view.
- 02 · Model stepThe model produces unsafe or off-brand output.
- 03 · Control gapNo filter holds the line before publish.
- 04 · FailureThe output goes public unchecked.
- 05 · ConsequenceA reputational or safety incident lands.
A contained signal crosses into output that goes public.
The algorithm failed to achieve equalized odds across different racial groups. This resulted in a significantly higher false positive rate for Black defendants compared to white defendants, despite maintaining predictive parity.
What it cost
Sources
- PressMachine Biaspropublica.org
- Court FilingState v. Loomisharvardlawreview.org
Cite this entry
https://failureindex.ai/failures/compas-risk-assessment-exhibits-racial-biasAI Failure Index. "COMPAS risk assessment tool exhibits racial bias in recidivism predictions" (FI-0671). Realm Labs. https://failureindex.ai/failures/compas-risk-assessment-exhibits-racial-bias (indexed Jun 22, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0671. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
Realm watches the model's internal state for the signature of unsafe or off-brand generation and can block or reroute the output before it becomes public, in real time rather than after it has been screenshotted.