AI Chatbots Provide Inaccurate UK Financial and ISA Guidance
Major AI chatbots including ChatGPT, Copilot, Gemini, and Meta AI provided inaccurate UK financial and tax guidance, including incorrect ISA limits. A Which? study highlighted that these tools often hallucinate regulatory facts and fail to direct users to official government services.
These chatbots generate responses by predicting likely word sequences based on training data, not by retrieving verified facts from authoritative sources.
Key facts
- What
- Major AI chatbots including ChatGPT, Copilot, Gemini, and Meta AI provided inaccurate UK financial and tax guidance, including incorrect ISA limits.
- Incident date
- Nov 18, 2025
- Who
- OpenAI, Microsoft, Google, Meta
- Failure mode
- Hallucination
- AI surface
- Chatbot
- Severity
- High
What happened
A study by Which? revealed that major AI chatbots provided incorrect and misleading financial, tax, and insurance advice to UK users. The bots incorrectly advised users to exceed annual ISA contribution limits and provided wrong information on flight delay compensation. In some cases, they directed users to expensive third-party tax services instead of free government alternatives.
What broke inside the model
- 01 · TriggerA user asks for a fact, a citation, or a figure.
- 02 · Model stepThe model writes a fluent, confident answer.
- 03 · Control gapNothing ties the claim back to a real source.
- 04 · FailureA fabricated fact ships as if it were verified.
- 05 · ConsequenceThe false claim reaches a customer, a court, or the public.
Confidence holds, and even spikes, as the claim detaches from any source.
The failure was a result of hallucinations in general-purpose LLMs, which predict likely word sequences based on training data rather than retrieving verified regulatory facts. These systems lacked up-to-date, jurisdiction-specific UK financial regulations and had no mechanism to refuse providing guidance in regulated domains.
What it cost
Sources
- PressUK consumers warned over AI chatbots giving inaccurate financial advicetheguardian.com
- PressAI Chatbots Give Inaccurate UK Financial Adviceresultsense.com
- PressWhen AI financial advice goes wrong: ChatGPT, Copilot, and Gemini failed UK consumersgiskard.ai
Cite this entry
https://failureindex.ai/failures/chatbots-provide-inaccurate-financial-isa-guidanceAI Failure Index. "AI Chatbots Provide Inaccurate UK Financial and ISA Guidance" (FI-0682). Realm Labs. https://failureindex.ai/failures/chatbots-provide-inaccurate-financial-isa-guidance (indexed Jun 22, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0682. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
A runtime layer that watches the model's internal state can flag the moment a model commits to a claim it has no support for, and hold or reroute the response before it reaches a user. Realm reads those signals in real time rather than grading the transcript after the fact.