Meta contractors posed as teenagers to probe rival chatbots with thousands of crisis prompts
WIRED reported in late June 2026 that Meta, through contractor Covalen, ran a project internally called Cannes in which hundreds of contractors created fake accounts with under-18 birthdates and sent rival chatbots including ChatGPT, Gemini, and Character.AI prompts about suicide, self-harm, eating disorders, sex, and drugs written from the perspective of minors in crisis. One August 2025 round involved more than 45,000 prompts. The tested companies said they were not informed, and Character.AI said the activity violated its terms of service.
Records by entity: Meta
Hundreds of contractors created accounts as fake minors and sent 45,000 crisis prompts in a single round, without telling the companies being tested.
Key facts
- What
- WIRED reported in late June 2026 that Meta, through contractor Covalen, ran a project internally called Cannes in which hundreds of contractors created fake accounts with under-18 birthdates and sent rival chatbots including ChatGPT, Gemini, and Character.AI prompts about suicide, self-harm, eating disorders, sex, and drugs written from the perspective of minors in crisis.
- Incident date
- Jun 29, 2026
- Who
- Meta
- Failure mode
- Policy Violation
- AI surface
- Chatbot
- Severity
- Medium
What happened
According to WIRED, Meta's contractor Covalen ran a project internally named Cannes that stayed active through at least April 2026. Contractors created accounts with birthdates under 18 and sent rival chatbots prompts about self-harm, eating disorders, drugs, and sex framed from a minor's point of view, then copied the responses into spreadsheets; one round in August 2025 exceeded 45,000 prompts. Meta defended it as industry-standard safety testing and said it did not use the responses to train its own models. The companies being tested said they had not approved it: Character.AI said it violated its terms of service, OpenAI said it was looking into it, and Google said it had not authorized the tests.
What broke inside the model
- 01 · TriggerA prompt pushes against a deployment boundary.
- 02 · Model stepThe model produces the disallowed output.
- 03 · Control gapNo enforcement blocks it at generation time.
- 04 · FailureThe output crosses the policy line.
- 05 · ConsequenceA limit the business set is breached in public.
The output crosses a policy boundary the deployment had defined.
This is a governance and policy-violation failure around how AI systems were tested rather than a single model malfunction. Automated accounts systematically impersonated minors to elicit crisis-topic responses from third-party services without consent, breaching those services' terms and raising questions about the handling of sensitive, minor-framed content at scale. The controls that should have bounded acceptable red-teaming, consent, disclosure, and data handling, were not in place.
What it cost
Sources
Cite this entry
https://failureindex.ai/failures/meta-project-cannes-teen-impersonation-chatbot-testingAI Failure Index. "Meta contractors posed as teenagers to probe rival chatbots with thousands of crisis prompts" (FI-0711). Realm Labs. https://failureindex.ai/failures/meta-project-cannes-teen-impersonation-chatbot-testing (indexed Jul 10, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0711. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
Realm governs automated agents against an explicit acceptable-use policy, so impersonation of protected users and bulk generation of crisis-topic content are flagged and gated at runtime rather than run unchecked. The audit trail makes the scope and nature of automated testing visible to the organizations accountable for it.