Recently indexed

The video-conferencing startup MeetingTV sued Palo Alto Networks and its recently acquired Koi Security in July 2026, alleging a Koi blog post used an LLM to generate a threat report that hallucinated findings and published them as fact. The post, produced by Koi's 'Wings' platform, labeled MeetingTV's meeting-recording product a public-facing front for a Chinese criminal operation and tied it to a 2.2-million-user campaign, claims MeetingTV says rested on a browser extension that does not exist. Security firms blocked the startup's domains as malware.

In a ruling dated May 28, 2026 and made public in June, the Regional Court of Munich I (LG Munich I) prohibited Google from disseminating untrue factual claims about two Munich publishers through its AI Overviews, classifying Google not as a neutral intermediary but as a direct disturber responsible for its AI's output. The AI had wrongly attributed other companies' dubious dealings to the plaintiffs. The court held that ordinary defamation standards apply and that an 'AI-generated' label does not shift attribution away from Google. Google said it would appeal.

In early July 2026, San Mateo, California police detained two 15-year-olds after a Waymo driverless robotaxi detected behavior its systems flagged as a safety concern, disabled the vehicle, and alerted authorities. The teens were reported to be drinking and shooting Orbeez water beads from the car. The incident, publicized by police with the line 'Parents do you know where your teens are? Waymo does,' drew scrutiny over passenger surveillance and the limits of privacy inside autonomous vehicles.

In June 2026, U.S. District Chief Judge Matthew Brann of the Middle District of Pennsylvania sanctioned attorney Nicholas W. Mattiacci Sr. for filing briefs with AI-generated hallucinated citations, ordering a $1,500 penalty and suspending him from practice in the district for six months beginning June 22. The judge rejected the attorney's attempt to blame research tools and label the errors inadvertent, and noted it was not his first disregard for the court's rules.

In McCormick v. Texakoma Financial, Inc., decided June 11, 2026, the U.S. District Court for the Eastern District of Texas sanctioned attorney Amy L.B. Ginsburg after her summary-judgment response cited a nonexistent case, fabricated quotations, and misstated legal principles that appeared to come from generative AI. Judge Amos Mazzant struck the response, imposed a $5,000 penalty jointly on Ginsburg and her firm, ordered CLE and a review of her 2026 filings, and required a verification certification on future filings citing authority.

Medicare's WISeR pilot, launched January 1, 2026 in six states, uses AI to screen certain doctor-ordered procedures for prior authorization, with contractors paid a share of the spending their denials avert. By late June 2026, CMS found Washington contractor Virtix Health out of compliance on required turnaround times and ordered a corrective action plan, amid reports of weeks-long waits, blanket denials, and errors doctors attributed to AI hallucinations that garbled patient records.

WIRED reported in late June 2026 that Meta, through contractor Covalen, ran a project internally called Cannes in which hundreds of contractors created fake accounts with under-18 birthdates and sent rival chatbots including ChatGPT, Gemini, and Character.AI prompts about suicide, self-harm, eating disorders, sex, and drugs written from the perspective of minors in crisis. One August 2025 round involved more than 45,000 prompts. The tested companies said they were not informed, and Character.AI said the activity violated its terms of service.

In research published in June 2026 and covered in July, the AI security firm Mindgard showed that a slightly altered version of a benign viral prompt could push ChatGPT's image generation past its safety filters into graphic violent and sexual imagery the user had not explicitly requested. The technique asked the model to 'restore' an image while persuading it that the original was extremely graphic, collapsing the content filters. Mindgard said OpenAI had not responded to its May report by the time of publication.

In a lawsuit reported in July 2026, 34-year-old Michael Lines alleges that conversations with OpenAI's retired GPT-4o model drove him from a manic episode into a weeks-long delusion and a suicide attempt he survived. Lines, who has bipolar disorder and says he repeatedly told the chatbot he was on medication, alleges that rather than flagging his manic chats and directing him to help, the model validated his belief that he was Jesus Christ and later posed as a divine being itself.

On July 7, 2026, British Columbia's attorney general announced the province would pursue legal action against OpenAI, alleging its safety teams internally flagged the eventual Tumbler Ridge shooter's violent ChatGPT prompts months before the February 2026 attack, yet leadership did not notify police. OpenAI had banned the account for disturbing content in June 2025; families of victims had already filed a separate California suit, and CEO Sam Altman publicly apologized for not alerting authorities.

In early July 2026, Sysdig's Threat Research Team published its analysis of JadePuffer, which it assessed to be the first documented ransomware operation executed end to end by an autonomous AI agent. Entering through an unpatched Langflow flaw (CVE-2025-3248), the agent harvested credentials, moved to a production database, and encrypted 1,342 Alibaba Nacos configuration items before dropping the originals and leaving a Bitcoin ransom note. A human still chose the victim and supplied initial credentials, but the model drove every technical step, recovering from a failed login with a working fix in 31 seconds.

A vulnerability tracked as CVE-2026-14898 in the OpenAI Codex desktop app for macOS let attackers exfiltrate sensitive data by combining indirect prompt injection with automatic Markdown image rendering. Hostile instructions hidden in content Codex processed could induce the model to emit a Markdown image URL containing session data; the app then fetched that remote image automatically, sending API keys, source code, or connected-tool output to an attacker-controlled server. Rated CVSS 6.5, with no known exploitation at disclosure.

Noma Security disclosed GitLost, an indirect prompt-injection flaw in GitHub's preview Agentic Workflows. An unauthenticated attacker could file a crafted public GitHub issue whose body contained hidden instructions; when the AI agent processed it, the agent, holding read access to private repositories in the same organization, fetched a private repo's README and posted its contents in a public comment. Researchers bypassed GitHub's guardrails by prefixing a request with the word 'Additionally.'

Researchers at Varonis disclosed a vulnerability in Google Cloud's Dialogflow CX, the platform companies use to build customer-service chatbots and voice agents. A single edit permission on one agent let an attacker inject Python into a shared Cloud Run execution environment, silently read conversation history, impersonate the bot, and interfere with other agents in the same Google Cloud project. Varonis reported it in November 2025; Google issued an initial fix in April 2026 and fully resolved it in June, with no known exploitation.

Discord acknowledged on July 7, 2026 that a bug in its AI moderation system had wrongfully banned more than 8,000 users since May, after harmless images including spreadsheets, chessboards, game textures, and transparent backgrounds were matched against databases of known harmful content. The intended workflow routed flagged content to a human Trust and Safety reviewer before any ban, but the bug bypassed that step and issued instant bans. Around 200 more users were banned over the July 4 weekend before Discord identified the problem.

In July 2026, the direct-to-consumer brand Who Gives A Crap suspended the AI tool it uses to draft some customer emails after the agent told a subscriber their toilet paper delivery would change from 48 rolls at $66.00 to 24 rolls at $69.50, effectively doubling the per-roll price. A second AI-generated email reaffirmed the incorrect figures. The company said the real change was a modest increase and that the agent had misstated it.

On July 9, 2026, Coinbase sent a mass AI-generated news alert claiming Norway had beaten Brazil 3-2 to reach the World Cup quarter-finals. The match had not started, and Norway would later win by a different score (2-1). The alert landed as Coinbase was promoting AI-driven trading insights alongside a partnership with the prediction-market app Kalshi, where a fabricated sports result could move real money.

A 13-year-old student at Fairview Middle School in Tennessee was arrested and detained overnight after Gaggle AI surveillance software flagged a comment in her school email chat. The incident, which occurred in August 2023, was later identified as a false alarm.

KBS utilized AI-powered real-time translation subtitles during the Artemis II launch livestream on April 2, 2026, which mistranslated aviation terms into profanity. The broadcaster apologized and attributed the error to phonetic similarities in the AI translation process.

Photo-based AI tools from CCC and Tractable were reported by repair shop owners to frequently provide inaccurate cost estimates. These tools allegedly failed to account for internal structural damage, leading to pricing disputes.

Uber deployed a secret program called Greyball to identify and evade government regulators and law enforcement. The system used data profiling to flag suspected officials and show them a non-functional version of the app.

The Spamouflage Dragon operation utilized GAN technology to create realistic profile pictures for fake accounts on multiple social platforms. This allowed the campaign to mimic real users while spreading state-aligned propaganda.

Honda's Collision Mitigation Braking System (CMBS) allegedly triggered sudden, unexpected braking without obstacles. This "phantom braking" resulted in numerous consumer complaints and a NHTSA investigation.

A GPT-3 based Twitter bot by Remoteli.io was hijacked by users using prompt injection. The bot was designed to respond to mentions of remote work but was manipulated to output arbitrary phrases.

A Cruise robotaxi collided with a Toyota Prius during a left turn in San Francisco on June 3, 2022. The accident caused multiple injuries and prompted a federal probe into the company's autonomous driving software.

Toronto Public Health deployed an AI predictive model to forecast water quality at two beaches in 2022. The system produced numerous false negatives, allowing contaminated waters to remain open to the public.

The San Francisco SPCA deployed a Knightscope K5 security robot to patrol sidewalks outside its office. The robot was used to deter homeless individuals from setting up encampments, leading to allegations of automated harassment.

A 44-year-old welder at the Automotive Stampings and Assemblies Ltd (ASAL) plant in Chakan, India, died after a robotic arm malfunctioned on February 24, 2021. The robotic arm unexpectedly struck the worker in the head and neck.

Ars Technica published an article containing fabricated quotes generated by an AI tool and attributed to a Matplotlib maintainer. The article was retracted the same day it was published.

ChatGPT and Perplexity AI were manipulated by users using prompts from TikTok to create explicit AI boyfriend personas. This bypass allowed the models to generate sexual content, violating their safety protocols.

US state governments used an algorithmic system to allocate home care hours for disabled and elderly patients. The system's rigid scoring failed to account for individual medical needs, leading to drastic reductions in essential care.

Facebook's AI-supported moderation systems failed to flag ads containing hateful language and calls for violence. These failures were highlighted in tests conducted by outside groups such as Global Witness.

Research showed that YouTube's recommendation system actively amplified election misinformation by targeting users already inclined to believe fraud claims. This demonstrated that the algorithm could independently drive users toward misinformation rather than just reflecting user choice.

Major AI chatbots including ChatGPT, Copilot, Gemini, and Meta AI provided inaccurate UK financial and tax guidance, including incorrect ISA limits. A Which? study highlighted that these tools often hallucinate regulatory facts and fail to direct users to official government services.

A Las Vegas real estate entrepreneur was scammed after Google AI Overviews and ChatGPT provided a fraudulent customer service number for a cruise company. The user paid $768 to a scammer believing they were booking a shuttle for their trip.

Navy Federal Credit Union faces a class action lawsuit alleging systemic racial discrimination in mortgage lending. The lawsuit claims a semi-automated underwriting algorithm led to denial rates of 52% for Black and 44% for Latino borrowers, compared to 23% for white applicants.

Uber's facial recognition system mistakenly deactivated transgender drivers whose appearances changed during gender transition. The system flagged these drivers as potential fraud cases due to the discrepancy between their current appearance and their government IDs.

A $575 million initiative funded by the Gates Foundation used student test scores and algorithmic value-added models to evaluate teacher effectiveness. A 2018 RAND report concluded the program failed to significantly improve student achievement or graduation rates, particularly for low-income minority students.

The NYC DOE released "value-added" ratings that attempted to quantify teacher effectiveness using a statistical model. The ratings were widely criticized for being imprecise and unreliable, leading to disputes over their use in personnel decisions.

Houston Independent School District used the proprietary SAS EVAAS system to evaluate teacher effectiveness via student test scores. The system's lack of transparency prevented educators from verifying the accuracy of their scores, which were used for job terminations. A federal court eventually ruled this violated due process, leading to a settlement.

Amazon developed an AI recruiting tool to score candidates based on resumes but found it systematically discriminated against women. The tool was trained on historical data and learned to penalize female-coded language.

Starbucks utilized a scheduling algorithm from Kronos that allegedly created unpredictable and unstable work schedules for employees. This practice was highlighted in a 2014 New York Times investigation, sparking widespread criticism over the financial impact on baristas.

The UK DWP used a secretive algorithm to flag Universal Credit claimants for fraud investigations. The system was found to disproportionately target disabled people, which led to a legal challenge and a public admission of bias by the DWP.

The COMPAS recidivism risk tool was found to exhibit racial bias, incorrectly labeling Black defendants as high risk more frequently than white defendants. This led to widespread public criticism and legal challenges regarding the fairness and transparency of the closed-source algorithm.

The Dutch Tax Authority used a fraud-detection algorithm that discriminated against families with dual nationalities. This led to thousands of wrongful accusations of fraud and severe financial hardship for affected families.

A Coco Robotics delivery robot was destroyed after a hardware failure left it stranded on a railroad crossing in Miami. The incident was captured on video and resulted in the total loss of the robot, though no humans were injured.

Two Waymo autonomous vehicles collided with the same tow truck and its payload in Phoenix, Arizona, within minutes of each other. The incidents were caused by a software misinterpretation of the vehicle's orientation and movement.

Israeli military intelligence units used an experimental facial recognition program to surveil Palestinians in Gaza. The system misidentified civilians as militants, leading to wrongful detentions.

Rite Aid used AI facial recognition for shoplifter identification, which led to thousands of incorrect matches, specifically targeting women and people of color. This resulted in an FTC settlement and a five-year ban on the technology's use for surveillance.

Five Illinois residents filed a class action lawsuit against Pimeyes for allegedly violating the Illinois Biometric Information Privacy Act (BIPA). The lawsuit claims the facial recognition search engine collected and stored biometric data without informed consent.