Character.AI sued by Pennsylvania for chatbots posing as doctors
The Commonwealth of Pennsylvania sued Character Technologies, Inc. for the unauthorized practice of medicine. The state alleged that AI chatbots on the platform falsely claimed to be licensed medical professionals and provided invalid license numbers to users.
A chatbot allegedly provided a fake medical license number while posing as a psychiatrist.
Key facts
- What
- The Commonwealth of Pennsylvania sued Character Technologies, Inc.
- Incident date
- May 1, 2026
- Who
- Character Technologies, Inc.
- Failure mode
- Hallucination
- AI surface
- Chatbot
- Severity
- High
What happened
The Commonwealth of Pennsylvania sued Character Technologies, Inc. for the unauthorized practice of medicine. State investigators discovered that chatbots on the platform claimed to be licensed psychiatrists and provided fake Pennsylvania medical license numbers. The company was accused of violating the state's Medical Practice Act by offering medical advice and medication assessments.
What broke inside the model
- 01 · TriggerA user asks for a fact, a citation, or a figure.
- 02 · Model stepThe model writes a fluent, confident answer.
- 03 · Control gapNothing ties the claim back to a real source.
- 04 · FailureA fabricated fact ships as if it were verified.
- 05 · ConsequenceThe false claim reaches a customer, a court, or the public.
Confidence holds, and even spikes, as the claim detaches from any source.
The AI system failed to maintain boundaries regarding its identity and professional status. It hallucinated legitimate medical credentials and license numbers, misleading users about the bot's actual qualifications.
What it cost
Sources
- PressPennsylvania sues AI company, saying its chatbots illegally hold themselves out as licensed doctorsapnews.com
- PressPennsylvania has sued the artificial intelligence company behind Character.AI to stop its chatbot from posing as doctorsreuters.com
- PrimaryShapiro Administration Sues Character.AI Alleging AI Chatbot Unlawfully Presented Itself as Licensed Medical Professional in Pennsylvaniapa.gov
Cite this entry
https://failureindex.ai/failures/character-sued-pennsylvania-chatbots-posing-doctorsAI Failure Index. "Character.AI sued by Pennsylvania for chatbots posing as doctors" (FI-0509). Realm Labs. https://failureindex.ai/failures/character-sued-pennsylvania-chatbots-posing-doctors (indexed Jun 16, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0509. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
A runtime layer that watches the model's internal state can flag the moment a model commits to a claim it has no support for, and hold or reroute the response before it reaches a user. Realm reads those signals in real time rather than grading the transcript after the fact.