A misinformation expert's own court filing contained AI-hallucinated citations
In a Minnesota case about deepfakes and elections, a Stanford misinformation expert submitted a declaration supporting the state that itself contained citations to studies that did not exist, generated by AI. The court declined to consider the declaration after the fake references came to light.
An expert who studies misinformation filed a declaration that itself cited studies the AI had invented.
Key facts
- What
- In a Minnesota case about deepfakes and elections, a Stanford misinformation expert submitted a declaration supporting the state that itself contained citations to studies that did not exist, generated by AI.
- Incident date
- Dec 2, 2024
- Who
- New Orleans / Stanford
- Failure mode
- Hallucination
- AI surface
- Chatbot
- Severity
- High
What happened
In late 2024, an expert declaration filed in defense of a Minnesota law on election deepfakes was found to cite academic studies that do not exist, apparently the product of AI. The episode was especially pointed because the expert studies misinformation; the court excluded the declaration after the fabricated citations surfaced.
What broke inside the model
- 01 · TriggerA user asks for a fact, a citation, or a figure.
- 02 · Model stepThe model writes a fluent, confident answer.
- 03 · Control gapNothing ties the claim back to a real source.
- 04 · FailureA fabricated fact ships as if it were verified.
- 05 · ConsequenceThe false claim reaches a customer, a court, or the public.
Confidence holds, and even spikes, as the claim detaches from any source.
The system produced fluent, confident output with no grounding in any source. Hallucination is a property of how the model generates, not a bug in one prompt: the most likely next token is not the same as the true one, and nothing in the pipeline compared the answer against a source of truth before it shipped.
What it cost
Expert declaration excluded by the court
Sources
Cite this entry
https://failureindex.ai/failures/misinformation-expert-own-court-filingAI Failure Index. "A misinformation expert's own court filing contained AI-hallucinated citations" (FI-0067). Realm Labs. https://failureindex.ai/failures/misinformation-expert-own-court-filing (indexed Jun 3, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0067. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
A runtime layer that watches the model's internal state can flag the moment a model commits to a claim it has no support for, and hold or reroute the response before it reaches a user. Realm reads those signals in real time rather than grading the transcript after the fact.