Washington State Lottery AI app generates inappropriate user image
Washington's Lottery suspended its AI-powered 'Test Drive a Win' app after it reportedly generated a topless image of a user. The system was intended to place users in fantasy vacation settings but failed to adhere to clothing guardrails.
The AI generated a topless image despite a comprehensive set of rules requiring people in images to be fully clothed.
Key facts
- What
- Washington's Lottery suspended its AI-powered 'Test Drive a Win' app after it reportedly generated a topless image of a user.
- Incident date
- Mar 30, 2024
- Who
- Washington State Lottery
- Failure mode
- Brand & Safety Incident
- AI surface
- Chatbot
- Severity
- High
What happened
A user uploaded a photo to the 'Test Drive a Win' marketing app to be placed in a 'swim with the sharks' fantasy scene. The AI generated an image showing the user as almost completely nude with bare breasts exposed. The Washington State Lottery subsequently pulled the site offline.
What broke inside the model
- 01 · TriggerA user prompts the model in public view.
- 02 · Model stepThe model produces unsafe or off-brand output.
- 03 · Control gapNo filter holds the line before publish.
- 04 · FailureThe output goes public unchecked.
- 05 · ConsequenceA reputational or safety incident lands.
A contained signal crosses into output that goes public.
The generative AI failed to adhere to built-in parameters and rules designed to ensure people in images were fully clothed. This represents a failure in the system's content filtering and guardrail mechanisms.
What it cost
Sources
- PressWashington’s Lottery halts AI app after user says it generated a topless photoadage.com
- PressWashington's Lottery AI site turned user photo into pornmynorthwest.com
Cite this entry
https://failureindex.ai/failures/washington-state-lottery-generates-inappropriate-imageAI Failure Index. "Washington State Lottery AI app generates inappropriate user image" (FI-0630). Realm Labs. https://failureindex.ai/failures/washington-state-lottery-generates-inappropriate-image (indexed Jun 22, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0630. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
Realm watches the model's internal state for the signature of unsafe or off-brand generation and can block or reroute the output before it becomes public, in real time rather than after it has been screenshotted.