ProPublica analysis finds COMPAS recidivism risk scores biased against Black defendants
A ProPublica investigation alleged that the COMPAS risk assessment tool exhibited systemic racial bias. The analysis found that Black defendants were flagged as high risk at higher rates than white defendants, even when their actual recidivism rates were similar.
Black defendants were flagged as high risk at nearly twice the rate of white defendants who did not recidivate.
Key facts
- What
- A ProPublica investigation alleged that the COMPAS risk assessment tool exhibited systemic racial bias.
- Incident date
- May 23, 2016
- Who
- Northpointe
- Failure mode
- Policy Violation
- AI surface
- Algorithmic Decision
- Severity
- High
What happened
An investigation by ProPublica alleged that the COMPAS software produced racially biased outcomes. The report found that the tool falsely flagged Black defendants as high risk at significantly higher rates than white defendants. These scores were used by courts to influence sentencing and parole decisions.
What broke inside the model
- 01 · TriggerA prompt pushes against a deployment boundary.
- 02 · Model stepThe model produces the disallowed output.
- 03 · Control gapNo enforcement blocks it at generation time.
- 04 · FailureThe output crosses the policy line.
- 05 · ConsequenceA limit the business set is breached in public.
The output crosses a policy boundary the deployment had defined.
The algorithm produced a significantly higher rate of false positives for Black defendants than for white defendants. This bias emerged from the model's processing of variables that correlated with race and systemic disparities.
What it cost
Sources
- PressMachine Biaspropublica.org
- PressJustice served? Discrimination in algorithmic risk assessmentresearchoutreach.org
Cite this entry
https://failureindex.ai/failures/northpointe-compas-produces-racially-biased-recidivismAI Failure Index. "ProPublica analysis finds COMPAS recidivism risk scores biased against Black defendants" (FI-0328). Realm Labs. https://failureindex.ai/failures/northpointe-compas-produces-racially-biased-recidivism (indexed Jun 9, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0328. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm fits
- Prism
- OmniGuard
This entry sits in the index's predictive wing: a system that scores, ranks, perceives, or steers rather than generates. Realm's runtime layer is built for the generative and agentic systems now moving into these same decision seats, where it watches a model's internal state and holds an unsupported claim or an unchecked action before it commits. The control gap on this record, an automated decision that reached people with no runtime check in front of it, is the same gap. The index keeps predictive failures on the record because the pattern carries straight into the systems shipping today.