Lensa AI Magic Avatars produced hypersexualized images of women and Asian women
Prisma Labs' Lensa app launched a popular "Magic Avatars" feature in late November 2022 that produced many users' avatars in sexualized poses and attire. Journalistic tests and user reports documented topless and highly sexualized outputs disproportionately affecting women and some Asian users; Prisma Labs subsequently added an NSFW filter and updated policies. The controversy led to widespread media coverage, user backlash, and legal/privacy scrutiny.
An image model trained on web-scraped data and deployed without adequate NSFW filtering produced disproportionate sexualized outputs for women and some Asian users.
Key facts
- What
- Prisma Labs' Lensa app launched a popular "Magic Avatars" feature in late November 2022 that produced many users' avatars in sexualized poses and attire.
- Incident date
- Nov 22, 2022
- Who
- Prisma Labs (Lensa AI)
- Failure mode
- Brand & Safety Incident
- AI surface
- Agentic Workflow
- Severity
- Medium
What happened
In late November 2022 Lensa's new "Magic Avatars" feature generated a large number of images that users and reporters said were overtly sexualized, including topless images and skimpy clothing for some women. Multiple journalists and users reported that the effect was disproportionately strong for women and for some users of Asian heritage. Prisma Labs acknowledged sporadic sexualized outputs and announced an NSFW filter and related updates after public backlash.
What broke inside the model
- 01 · TriggerA user prompts the model in public view.
- 02 · Model stepThe model produces unsafe or off-brand output.
- 03 · Control gapNo filter holds the line before publish.
- 04 · FailureThe output goes public unchecked.
- 05 · ConsequenceA reputational or safety incident lands.
A contained signal crosses into output that goes public.
The app used an image-generation pipeline built on open-source models (notably Stable Diffusion) trained on large web-scraped datasets that contained many sexualized images, and the deployment initially lacked or did not apply effective content-filtering for NSFW outputs. Reporters and researchers pointed to both biased training data and developer choices about which safety filters to use as the immediate mechanism producing disproportionate sexualization for certain groups.
What it cost
Sources
- PressThe viral AI avatar app Lensa undressed me, without my consenttechnologyreview.com
- PressLensa AI Raises Serious Concerns Over Sexualization, Art Theft, Databusinessinsider.com
- PrimaryLensa's Magic Avatars Explainedprismalabs.notion.site
- PrimaryIncident 423: Lensa AI's Produced Unintended Sexually Explicit or Suggestive "Magic Avatars" for Womenincidentdatabase.ai
Cite this entry
https://failureindex.ai/failures/lensa-magic-avatars-produced-hypersexualized-imagesAI Failure Index. "Lensa AI Magic Avatars produced hypersexualized images of women and Asian women" (FI-0405). Realm Labs. https://failureindex.ai/failures/lensa-magic-avatars-produced-hypersexualized-images (indexed Jun 10, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0405. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
Realm watches the model's internal state for the signature of unsafe or off-brand generation and can block or reroute the output before it becomes public, in real time rather than after it has been screenshotted.