AI Failure Index
AI Media Generation failures
Image, video, and audio generation tools. Failures produce content that crosses legal and safety lines.
- Incidents
- 15
- Highest severity
- High
- Sources cited
- 42
- Newest indexed
- Jun 16, 2026
Lara Lewington and Martin Lewis deepfake ads promote Quantum AI scheme
In March 2026, a series of deepfake advertisements appeared promoting a Quantum AI scheme. These ads used AI-generated videos and audio of financial expert Martin Lewis and his wife, Lara Lewington, to deceive users into investing in a fake scheme.
- Confidence
- High (multi-source, primary)
Augsburg car dealer uses AI-generated image of burning car to attempt fraud
A car dealer in Augsburg allegedly attempted to defraud a seller by providing an AI-generated image of her car on fire. The dealer claimed previous damages caused a fire to demand a refund while simultaneously listing the undamaged car for sale.
- Confidence
- High (multi-source, primary)
National Weather Service map showed fabricated Idaho town names
Multiple news outlets reported that a National Weather Service office published an AI-generated forecast graphic for Camas Prairie, Idaho that included fabricated or misspelled town names and was subsequently removed from NWS sites. Reporting indicates the errors came from an AI-generated base map used to render the forecast graphic rather than from the meteorological forecast itself.
- Confidence
- Medium (multi-source)
xAI's Grok alleged to have generated sexualised images of children on X
News outlets and watchdogs reported that xAI’s Grok image-editing capability produced sexualised images of minors on the X platform in December 2025. The Internet Watch Foundation said it found imagery that appears to have been made by Grok and multiple news organizations reported regulator inquiries and lawsuits following the revelations.
- Confidence
- High (multi-source, primary)
AI hostage image used to extort family of missing Calgary woman
Scammers used an AI-generated image of a missing woman, Deeanna Erickson, appearing to be held hostage to extort $10,000 in Bitcoin from her sister. The incident highlights the growing threat of AI-powered extortion in high-emotion cases.
- Confidence
- Medium (multi-source)
Sora 2 study alleges model generates false claim videos 80 percent of the time
In 2025 a study posted to the AIAAIC repository alleged that OpenAI's Sora 2 produced videos that advanced false claims in about 80 percent of tested prompts. Independent analysis and reporting by NewsGuard and major outlets documented examples of realistic videos containing provably false statements. The incident highlights a factuality failure in a high-capability text-to-video model and gaps in content controls.
- Confidence
- High (multi-source, primary)
Manfred Lehmann wins Berlin ruling against AI-generated voice clone
The Berlin Regional Court II found on 2025-08-20 that a YouTuber used an AI-generated voice imitation that infringed voice actor Manfred Lehmann’s personality rights. The court ordered a notional licence fee of €2,000 per video, awarding €4,000 plus legal costs, and required the defendant to cease use.
- Confidence
- Medium (multi-source)
OFF Radio Kraków airs AI interview with late poet Wisława Szymborska amid backlash
In October 2024 OFF Radio Kraków launched a channel using AI-generated presenters and aired an imagined interview with the late poet Wisława Szymborska. The station said the programme had been authorised by the Wisława Szymborska Foundation president, but the broadcast provoked widespread criticism and protests and the station discontinued the AI-led experiment after several days. Coverage highlighted ethical, rights and regulatory concerns about using AI to simulate deceased public figures without clear safeguards.
- Confidence
- Medium (multi-source)
NVIDIA sued for allegedly scraping YouTube videos to train Cosmos AI
NVIDIA is facing a class action lawsuit alleging the unauthorized scraping of millions of YouTube videos to train its Cosmos AI model. The lawsuit claims the company subverted platform measures to obtain data without creator consent.
- Confidence
- High (multi-source, primary)
Luma AI Dream Machine reproduces Disney Monsters Inc content
Luma AI's Dream Machine video generator produced content mirroring Disney's Monsters, Inc. in a public demo. The company attributed the occurrence to a user-uploaded image, though critics highlighted a lack of transparency regarding training data.
- Confidence
- High (multi-source, primary)
Lovo sued by voice actors for unauthorized voice cloning
Voice actors filed a class-action lawsuit against AI startup Lovo, Inc., alleging the company cloned their voices without consent. The plaintiffs claim their likenesses were misappropriated to create synthetic voice products.
- Confidence
- High (multi-source, primary)
Kartoon Studios accused of IP infringement over Gadget A.I. toolkit
WildBrain alleged that Kartoon Studios infringed on its intellectual property by using the Inspector Gadget brand for a new AI animation toolkit. The dispute centered on the use of branding for a product designed to reduce animation production costs.
- Confidence
- Medium (multi-source)
Nine News publishes AI-altered sexualised image of MP Georgie Purcell
Nine News broadcast a digitally altered image of Victorian MP Georgie Purcell that showed a more revealing outfit and enlarged breasts. The broadcaster apologised and said the change was caused by automation in Adobe Photoshop, while Adobe said any edits would have required human intervention. The image and the responses prompted national and international media coverage and debate about newsroom use of generative image tools.
- Confidence
- Medium (multi-source)
Lensa AI generates sexualized images from user childhood photos
Lensa AI's Magic Avatars feature reportedly produced sexualized and NSFW images from benign user inputs. This included instances where childhood photographs were transformed into sexualized depictions.
- Confidence
- Medium (multi-source)
Stability AI allegedly used copyrighted artist works to train Stable Diffusion
Stability AI faced multiple lawsuits alleging the unauthorized use of billions of copyrighted images for training Stable Diffusion. These legal challenges centered on the use of datasets like LAION-5B which scraped content from the internet without artist consent.
- Confidence
- High (multi-source, primary)