YouTube Kids App Presents Inappropriate Content
In May 2015, child advocacy groups reported that YouTube Kids failed to exclude adult content from its recommendations. This led to a formal complaint to the FTC regarding Google's claims about the app's safety.
Child safety groups have filed a complaint to the FTC about 'inappropriate' content on a child-friendly YouTube app.
Key facts
- What
- In May 2015, child advocacy groups reported that YouTube Kids failed to exclude adult content from its recommendations.
- Incident date
- May 19, 2015
- Who
- YouTube
- Failure mode
- Brand & Safety Incident
- AI surface
- Recommender
- Severity
- High
What happened
Child safety groups filed a complaint with the FTC alleging that the YouTube Kids app served inappropriate and adult-oriented content to children. The app, designed as a safe environment for toddlers, failed to filter out disturbing videos.
What broke inside the model
- 01 · TriggerA user prompts the model in public view.
- 02 · Model stepThe model produces unsafe or off-brand output.
- 03 · Control gapNo filter holds the line before publish.
- 04 · FailureThe output goes public unchecked.
- 05 · ConsequenceA reputational or safety incident lands.
A contained signal crosses into output that goes public.
The automated content filtering and classification system failed to identify and exclude adult-oriented videos. This allowed inappropriate content to bypass the safety filters and appear in the recommendation feed for children.
What it cost
Sources
- PressYouTube Kids app draws complaints about contentbbc.com
- PressYouTube Kids App Reported To FTC For Featuring Videos Adult Contentconsumerwatchdog.org
Cite this entry
https://failureindex.ai/failures/youtube-kids-presents-inappropriate-contentAI Failure Index. "YouTube Kids App Presents Inappropriate Content" (FI-0597). Realm Labs. https://failureindex.ai/failures/youtube-kids-presents-inappropriate-content (indexed Jun 22, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0597. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
Realm watches the model's internal state for the signature of unsafe or off-brand generation and can block or reroute the output before it becomes public, in real time rather than after it has been screenshotted.