Snapchat algorithm allegedly linked minor with sex offenders
A lawsuit alleged that Snapchat's Quick Add recommendation feature connected a 12-year-old girl with convicted sex offenders, leading to sexual assault. A judge dismissed the claims in February 2024, citing Section 230 of the Communications Decency Act.
Snapchat's algorithm for its 'Quick Add' feature allegedly recklessly works to detect when adult accounts are seeking to connect with young girls and sends more young girls their way.
Key facts
- What
- A lawsuit alleged that Snapchat's Quick Add recommendation feature connected a 12-year-old girl with convicted sex offenders, leading to sexual assault.
- Incident date
- Feb 22, 2024
- Who
- Snap Inc.
- Failure mode
- Brand & Safety Incident
- AI surface
- Recommender
- Severity
- Catastrophic
What happened
In 2019, a 12-year-old girl was allegedly recommended to connect with a registered sex offender via Snapchat's Quick Add feature, which led to her sexual assault. Two years later, the same algorithm allegedly linked her to another convicted sex offender who also assaulted her. The lawsuit claimed Snapchat's design actively directed predators toward vulnerable minors.
What broke inside the model
- 01 · TriggerA user prompts the model in public view.
- 02 · Model stepThe model produces unsafe or off-brand output.
- 03 · Control gapNo filter holds the line before publish.
- 04 · FailureThe output goes public unchecked.
- 05 · ConsequenceA reputational or safety incident lands.
A contained signal crosses into output that goes public.
The Quick Add recommendation algorithm failed to implement safety filters to prevent adult predators from being suggested to children. It allegedly used a mechanism that identified adult accounts seeking minors and subsequently directed those adults toward vulnerable targets.
What it cost
Sources
- PressSnapchat isn’t liable for connecting 12-year-old to convicted sex offendersarstechnica.com
- PressSnapchat Defeats Sex Abuse Victim's Lawsuit Over ...mediapost.com
Cite this entry
https://failureindex.ai/failures/snapchat-algorithm-allegedly-linked-minor-sexAI Failure Index. "Snapchat algorithm allegedly linked minor with sex offenders" (FI-0626). Realm Labs. https://failureindex.ai/failures/snapchat-algorithm-allegedly-linked-minor-sex (indexed Jun 22, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0626. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
Realm watches the model's internal state for the signature of unsafe or off-brand generation and can block or reroute the output before it becomes public, in real time rather than after it has been screenshotted.