Cursor's support chatbot invented a usage policy that did not exist
An AI support agent at code-editor company Cursor told users they were no longer allowed to be logged in from multiple devices. The policy was hallucinated. The CEO apologized.
An AI signed in the company's name is a policy surface. The customer does not care that the policy is hallucinated.
Key facts
- What
- An AI support agent at code-editor company Cursor told users they were no longer allowed to be logged in from multiple devices.
- Incident date
- Apr 16, 2025
- Who
- Cursor (Anysphere)
- Failure mode
- Policy Violation
- AI surface
- Chatbot
- Severity
- High
What happened
In April 2025, Cursor users on Hacker News posted that the company's AI support agent had told them a new policy banned multi-device login. No such policy existed. The AI had hallucinated it. Users who received the message canceled their subscriptions. Cursor CEO Michael Truell apologized publicly, removed the AI from first-line support, and committed to labeling all AI responses.
The case is the most cited example of customer-facing copilot policy hallucination in 2025. It also illustrates why an AI signed in the company's name is a policy surface whether the company wanted it to be or not.
What broke inside the model
- 01 · TriggerA prompt pushes against a deployment boundary.
- 02 · Model stepThe model produces the disallowed output.
- 03 · Control gapNo enforcement blocks it at generation time.
- 04 · FailureThe output crosses the policy line.
- 05 · ConsequenceA limit the business set is breached in public.
The output crosses a policy boundary the deployment had defined.
The model generated a confident policy statement that did not exist in any retrievable source. The mechanism is hallucination, possibly amplified by retrieval over outdated or contradictory internal documents.
What it cost
Subscription cancellations, brand recovery cost
Sources
- SocialCursor AI support tells users they cannot use multiple devicesnews.ycombinator.com
- PressCursor IDE's bot misled users about subscription policyarstechnica.com
Cite this entry
https://failureindex.ai/failures/cursor-ai-support-agent-invented-billing-policyAI Failure Index. "Cursor's support chatbot invented a usage policy that did not exist" (FI-0012). Realm Labs. https://failureindex.ai/failures/cursor-ai-support-agent-invented-billing-policy (indexed May 13, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0012. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
Prism reads the model's commitment to a policy claim against the authoritative policy document. When the model commits to a claim outside the document, Prism flags and OmniGuard either rewrites or escalates. The customer sees a policy answer that matches the actual policy, or the response is escalated to a human. The cancellations do not happen. The apology does not happen.