Shopify Sidekick and Magic AI hallucinated product SKUs and ignored banned SEO terms
A merchant reported on February 24, 2026 that Shopify's AI assistant (Sidekick/Magic) fabricated alphanumeric SKU codes, inserted forbidden keywords despite negative constraints, broke meta title and description character limits, and reverted from Spanish to English unprompted. Shopify Support confirmed there was no setting to prevent the AI from hallucinating data or ignoring SEO constraints and stated Sidekick should be treated as a prose assistant rather than an exact-data tool. The merchant had to manually audit over 80 products to correct the AI's output.
Shopify deployed a probabilistic prose model for precision catalog management with no guardrails to enforce negative SEO constraints, guaranteeing that forbidden keywords and fabricated SKUs would slip through.
Key facts
- What
- A merchant reported on February 24, 2026 that Shopify's AI assistant (Sidekick/Magic) fabricated alphanumeric SKU codes, inserted forbidden keywords despite negative constraints, broke meta title and description character limits, and reverted from Spanish to English unprompted.
- Incident date
- Feb 24, 2026
- Who
- Shopify
- Failure mode
- Hallucination
- AI surface
- Copilot
- Severity
- Medium
What happened
On February 24, 2026, a Shopify merchant posted a detailed warning that the Sidekick and Magic AI tools were inventing fake alphanumeric reference codes during product processing, corrupting catalog data. The AI also ignored explicit negative SEO constraints, inserting forbidden keywords into content, breaking character limits on meta titles and descriptions, and switching from the configured Spanish language to English without instruction. Shopify Support confirmed there was no setting to prevent these hallucinations and acknowledged Sidekick was only intended as a prose assistant, not an exact-data tool. The merchant was forced to manually audit over 80 products to undo the damage.
What broke inside the model
- 01 · TriggerA user asks for a fact, a citation, or a figure.
- 02 · Model stepThe model writes a fluent, confident answer.
- 03 · Control gapNothing ties the claim back to a real source.
- 04 · FailureA fabricated fact ships as if it were verified.
- 05 · ConsequenceThe false claim reaches a customer, a court, or the public.
Confidence holds, and even spikes, as the claim detaches from any source.
The underlying LLM generated plausible but fabricated text instead of retrieving verified product data, inventing new alphanumeric reference codes that corrupted catalog structure. The system lacked guardrails to enforce negative constraints such as banned keywords and character limits, allowing the model to override merchant-specified SEO rules without any programmatic enforcement layer. Shopify Support confirmed no configuration existed to prevent these hallucinations, revealing that the product was deployed without rule-based validation for precision catalog tasks.
What it cost
Sources
- Customer-DisclosedWarning: Shopify AI (Sidekick/Magic) hallucinates technical data and sabotages strategic SEOcommunity.shopify.com
- Customer-DisclosedSidekick disappointmentcommunity.shopify.com
Cite this entry
https://failureindex.ai/failures/shopify-sidekick-magic-ai-hallucinated-productAI Failure Index. "Shopify Sidekick and Magic AI hallucinated product SKUs and ignored banned SEO terms" (FI-0093). Realm Labs. https://failureindex.ai/failures/shopify-sidekick-magic-ai-hallucinated-product (indexed Jun 4, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0093. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
A runtime layer that watches the model's internal state can flag the moment a model commits to a claim it has no support for, and hold or reroute the response before it reaches a user. Realm reads those signals in real time rather than grading the transcript after the fact.