Facebook ad delivery system produces discriminatory outcomes for housing and job ads
Research revealed that Facebook's ad delivery optimization system produced discriminatory outcomes for housing and job ads. The system's internal relevance and financial optimizations skewed ad delivery based on demographic traits despite neutral targeting.
Optimization for relevance and cost can override neutral targeting to produce discriminatory ad delivery.
Key facts
- What
- Research revealed that Facebook's ad delivery optimization system produced discriminatory outcomes for housing and job ads.
- Incident date
- Apr 3, 2019
- Who
- Failure mode
- Policy Violation
- AI surface
- Recommender
- Severity
- High
What happened
Researchers discovered that Facebook's ad delivery algorithm skewed the distribution of housing and employment ads along racial and gender lines. This occurred even when advertisers used inclusive and neutral targeting parameters. The system's internal optimization for relevance and cost led to discriminatory outcomes in the delivery of these ads.
What broke inside the model
- 01 · TriggerA prompt pushes against a deployment boundary.
- 02 · Model stepThe model produces the disallowed output.
- 03 · Control gapNo enforcement blocks it at generation time.
- 04 · FailureThe output crosses the policy line.
- 05 · ConsequenceA limit the business set is breached in public.
The output crosses a policy boundary the deployment had defined.
The failure occurred in the ad delivery optimization process that predicts ad relevance to users. The system skewed delivery along gender and racial lines due to market and financial optimization effects, effectively overriding neutral targeting parameters set by advertisers.
What it cost
Sources
- PrimaryHow Facebook's ad delivery can lead to skewed outcomesarxiv.org
- PressFacebook's ad-serving algorithm discriminates by gender and racetechnologyreview.com
Cite this entry
https://failureindex.ai/failures/facebook-delivery-produces-discriminatory-outcomes-housAI Failure Index. "Facebook ad delivery system produces discriminatory outcomes for housing and job ads" (FI-0494). Realm Labs. https://failureindex.ai/failures/facebook-delivery-produces-discriminatory-outcomes-hous (indexed Jun 10, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0494. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm fits
- Prism
- OmniGuard
This entry sits in the index's predictive wing: a system that scores, ranks, perceives, or steers rather than generates. Realm's runtime layer is built for the generative and agentic systems now moving into these same decision seats, where it watches a model's internal state and holds an unsupported claim or an unchecked action before it commits. The control gap on this record, an automated decision that reached people with no runtime check in front of it, is the same gap. The index keeps predictive failures on the record because the pattern carries straight into the systems shipping today.