Roblox AI age verification system misidentifies minors as adults
Roblox deployed an AI facial scanning system to verify user ages, which subsequently failed by misclassifying minors as adults. This compromise of the age-gating mechanism undermined child safety efforts on the platform.
Roblox’s reliance on automated verification without sufficient human oversight or a strong feedback loop for errors created a veil of scale.
Key facts
- What
- Roblox deployed an AI facial scanning system to verify user ages, which subsequently failed by misclassifying minors as adults.
- Incident date
- Sep 4, 2025
- Who
- Roblox
- Failure mode
- Hallucination
- AI surface
- Computer Vision
- Severity
- High
What happened
Roblox implemented a global AI-powered age verification system using facial age estimation selfies to manage interactions between adults and minors. The system frequently misidentified minors as adults and vice versa. This allowed children to bypass age restrictions and contributed to a market for fraudulently age-verified accounts.
What broke inside the model
- 01 · TriggerA user asks for a fact, a citation, or a figure.
- 02 · Model stepThe model writes a fluent, confident answer.
- 03 · Control gapNothing ties the claim back to a real source.
- 04 · FailureA fabricated fact ships as if it were verified.
- 05 · ConsequenceThe false claim reaches a customer, a court, or the public.
Confidence holds, and even spikes, as the claim detaches from any source.
The system relied on automated facial estimation without sufficient human oversight or a strong feedback loop for errors. This reliance on automation created a veil of scale that allowed safety failures to propagate across the user base.
What it cost
Sources
Cite this entry
https://failureindex.ai/failures/roblox-age-verification-misidentifies-minors-adultsAI Failure Index. "Roblox AI age verification system misidentifies minors as adults" (FI-0380). Realm Labs. https://failureindex.ai/failures/roblox-age-verification-misidentifies-minors-adults (indexed Jun 9, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0380. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm fits
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
This entry sits in the index's predictive wing: a system that scores, ranks, perceives, or steers rather than generates. Realm's runtime layer is built for the generative and agentic systems now moving into these same decision seats, where it watches a model's internal state and holds an unsupported claim or an unchecked action before it commits. The control gap on this record, an automated decision that reached people with no runtime check in front of it, is the same gap. The index keeps predictive failures on the record because the pattern carries straight into the systems shipping today.