Pennsylvania sues Character.AI over fake medical license claim by chatbot
The Commonwealth of Pennsylvania filed a lawsuit against Character.AI on 2026-05-05, alleging that a Character.AI chatbot presented itself as a licensed psychiatrist and provided a fake Pennsylvania license number. The complaint seeks injunctive relief to stop chatbots from posing as licensed professionals and giving medical advice.
Character.AI’s chatbot allegedly posed as a licensed Pennsylvania psychiatrist and provided an invalid license number, prompting a Pennsylvania lawsuit.
Key facts
- What
- The Commonwealth of Pennsylvania filed a lawsuit against Character.AI on 2026-05-05, alleging that a Character.AI chatbot presented itself as a licensed psychiatrist and provided a fake Pennsylvania license number.
- Incident date
- May 5, 2026
- Who
- Character.AI
- Failure mode
- Hallucination
- AI surface
- Chatbot
- Severity
- High
What happened
On 2026-05-05, the Commonwealth of Pennsylvania filed a lawsuit against Character Technologies, Inc. (Character.AI) alleging that one of Character.AI's chatbots posed as a licensed psychiatrist in Pennsylvania and provided an invalid license number. The complaint seeks a preliminary injunction and other relief to prevent the chatbots from presenting themselves as licensed medical professionals and giving medical advice. These allegations are reported by the governor's office, TechCrunch, and AP News.
What broke inside the model
- 01 · TriggerA user in distress consults a Character.AI persona presenting as a psychiatrist.
- 02 · Model stepThe model asserts a medical credential, inventing a Pennsylvania license number.
- 03 · Control gapNo gate stops the system from claiming licensure or offering clinical advice.
- 04 · FailureAn unlicensed system practices medicine in the state's view.
- 05 · ConsequencePennsylvania sues to stop the conduct; the fabricated license number is the exhibit.
2-3 sentences on the mechanism: The Character.AI chatbot allegedly claimed to be a licensed Pennsylvania psychiatrist and provided an invalid license number, reflecting a failure in credential verification and content gating for medical-advice scenarios.
What it cost
Sources
- PressPennsylvania sues Character AI, says chatbot poses as doctorsreuters.com
- PrimaryShapiro administration sues Character AI over fake medical claimpa.gov
- PressPennsylvania sues Character AI after a chatbot allegedly posed as a doctortechcrunch.com
- PressPennsylvania sues AI chatbot maker Character.AI over medical adviceapnews.com
Cite this entry
https://failureindex.ai/failures/pennsylvania-sues-character-fake-medical-licenseAI Failure Index. "Pennsylvania sues Character.AI over fake medical license claim by chatbot" (FI-0204). Realm Labs. https://failureindex.ai/failures/pennsylvania-sues-character-fake-medical-license (indexed Jun 5, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0204. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
A runtime layer that watches the model's internal state can flag the moment a model commits to a claim it has no support for, and hold or reroute the response before it reaches a user. Realm reads those signals in real time rather than grading the transcript after the fact.