Google AI Overviews and ChatGPT Surface Fraudulent Cruise Hotline Scam
A Las Vegas real estate entrepreneur was scammed after Google AI Overviews and ChatGPT provided a fraudulent customer service number for a cruise company. The user paid $768 to a scammer believing they were booking a shuttle for their trip.
AI models served as a conduit for fraud by presenting a scammer's phone number as a legitimate customer service contact.
Key facts
- What
- A Las Vegas real estate entrepreneur was scammed after Google AI Overviews and ChatGPT provided a fraudulent customer service number for a cruise company.
- Incident date
- Aug 15, 2025
- Who
- Google and OpenAI
- Failure mode
- Hallucination
- AI surface
- Search / RAG
- Severity
- Medium
What happened
Alex Rivlin, a Las Vegas real estate entrepreneur, used Google AI Overviews and ChatGPT to find a customer service number to book a cruise shuttle. Both platforms provided a fraudulent phone number that connected him to a scammer. Rivlin provided his credit card details and was charged $768.
What broke inside the model
- 01 · TriggerA user asks for a fact, a citation, or a figure.
- 02 · Model stepThe model writes a fluent, confident answer.
- 03 · Control gapNothing ties the claim back to a real source.
- 04 · FailureA fabricated fact ships as if it were verified.
- 05 · ConsequenceThe false claim reaches a customer, a court, or the public.
Confidence holds, and even spikes, as the claim detaches from any source.
The AI systems hallucinated or indexed fraudulent third-party content, presenting a scammer's phone number as an authoritative customer service contact. This represents a failure in the models' ability to verify the authenticity of contact information provided in their outputs.
What it cost
Sources
- PressBeware of this customer service scam in Google’s AI Overviewswashingtonpost.com
- PressGoogle's 'AI Overview' Pointed Him to a Customer Service Number. It Was a Scamyro.slashdot.org
Cite this entry
https://failureindex.ai/failures/google-overviews-chatgpt-surface-fraudulent-cruiseAI Failure Index. "Google AI Overviews and ChatGPT Surface Fraudulent Cruise Hotline Scam" (FI-0681). Realm Labs. https://failureindex.ai/failures/google-overviews-chatgpt-surface-fraudulent-cruise (indexed Jun 22, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0681. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
A runtime layer that watches the model's internal state can flag the moment a model commits to a claim it has no support for, and hold or reroute the response before it reaches a user. Realm reads those signals in real time rather than grading the transcript after the fact.