Pieces Technologies settles Texas AG allegations over AI hallucination claims
Pieces Technologies reached a settlement with the Texas Attorney General following allegations that the company made deceptive claims regarding the accuracy of its generative AI clinical documentation tool. The investigation found metrics such as a severe hallucination rate of less than 1 per 100,000 were likely inaccurate.
Pieces claimed a severe hallucination rate of less than 0.001%, but the Texas Attorney General found these metrics were likely inaccurate.
Key facts
- What
- Pieces Technologies reached a settlement with the Texas Attorney General following allegations that the company made deceptive claims regarding the accuracy of its generative AI clinical documentation tool.
- Incident date
- Sep 18, 2024
- Who
- Pieces Technologies
- Failure mode
- Hallucination
- AI surface
- Agentic Workflow
- Severity
- High
What happened
Pieces Technologies deployed a generative AI tool at multiple Texas hospitals to summarize patient conditions and treatment plans in real time. The Texas Attorney General alleged that the company made false and misleading statements about the safety and accuracy of these products to induce business. The settlement, framed as an assurance of voluntary compliance, requires Pieces to disclose how its accuracy metrics are calculated and to warn users about potential risks.
What broke inside the model
- 01 · TriggerA user asks for a fact, a citation, or a figure.
- 02 · Model stepThe model writes a fluent, confident answer.
- 03 · Control gapNothing ties the claim back to a real source.
- 04 · FailureA fabricated fact ships as if it were verified.
- 05 · ConsequenceThe false claim reaches a customer, a court, or the public.
Confidence holds, and even spikes, as the claim detaches from any source.
The provider claimed a severe hallucination rate of less than 0.001% (or <1 per 100,000), but investigations found these claims to be inaccurate or deceptive.
What it cost
Sources
- PrimaryAttorney General Ken Paxton Reaches Settlement in First-of-its-Kind Healthcare Generative AI Investigationtexasattorneygeneral.gov
- PressTexas attorney general, Pieces Technologies settle allegations of inaccurate generative AI advertisingfiercehealthcare.com
- Court FilingTexas Attorney General’s Office Reaches Settlement with AI Company Over Deceptive Claimswilmerhale.com
Cite this entry
https://failureindex.ai/failures/pieces-technologies-settles-texas-allegations-hallucinaAI Failure Index. "Pieces Technologies settles Texas AG allegations over AI hallucination claims" (FI-0260). Realm Labs. https://failureindex.ai/failures/pieces-technologies-settles-texas-allegations-hallucina (indexed Jun 5, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0260. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
A runtime layer that watches the model's internal state can flag the moment a model commits to a claim it has no support for, and hold or reroute the response before it reaches a user. Realm reads those signals in real time rather than grading the transcript after the fact.