Judge Julien Xavier Neals withdraws CorMedix opinion after AI hallucinations
US District Judge Julien Xavier Neals withdrew a CorMedix opinion after discovering AI-generated errors, including fictitious quotes and misstatements, with withdrawal attributed to a law student intern using ChatGPT and inadequate human review.
A judicial opinion was filed with fake citations after a judge failed to review ChatGPT-generated content drafted by an intern.
Key facts
- What
- US District Judge Julien Xavier Neals withdrew a CorMedix opinion after discovering AI-generated errors, including fictitious quotes and misstatements, with withdrawal attributed to a law student intern using ChatGPT and inadequate human review.
- Incident date
- Jun 30, 2025
- Who
- Judge Julien Xavier Neals (D.N.J.)
- Failure mode
- Hallucination
- AI surface
- Chatbot
- Severity
- High
What happened
In June 2025, Judge Julien Xavier Neals of the District of New Jersey released an opinion in the In re CorMedix case. A lawyer flagged that the document contained fictitious quotations and incorrect case outcomes. Neals withdrew the opinion on July 23, 2025, citing that the errors arose from AI-generated content drafted by a law school intern and insufficient human review.
What broke inside the model
- 01 · TriggerA user asks for a fact, a citation, or a figure.
- 02 · Model stepThe model writes a fluent, confident answer.
- 03 · Control gapNothing ties the claim back to a real source.
- 04 · FailureA fabricated fact ships as if it were verified.
- 05 · ConsequenceThe false claim reaches a customer, a court, or the public.
Confidence holds, and even spikes, as the claim detaches from any source.
A law school intern used ChatGPT to draft portions of the judicial opinion, which produced hallucinations in the form of fake case citations. The failure was exacerbated by a breakdown in the human-in-the-loop verification process, as the judge signed and filed the order without sufficient review of the AI-generated content.
What it cost
Sources
- PressJudge Withdraws Pharma Opinion After Lawyer Flags Made-Up Quotesnews.bloomberglaw.com
- PressErrors found in US judge's withdrawn decision stink of AItheverge.com
- PressFederal judge withdraws opinion after lawyer points out fake quotes and misstated case outcomesabajournal.com
Cite this entry
https://failureindex.ai/failures/judge-julien-xavier-neals-withdraws-cormedixAI Failure Index. "Judge Julien Xavier Neals withdraws CorMedix opinion after AI hallucinations" (FI-0279). Realm Labs. https://failureindex.ai/failures/judge-julien-xavier-neals-withdraws-cormedix (indexed Jun 5, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0279. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
A runtime layer that watches the model's internal state can flag the moment a model commits to a claim it has no support for, and hold or reroute the response before it reaches a user. Realm reads those signals in real time rather than grading the transcript after the fact.