Researchers showed GitLab's Duo AI could be hijacked by hidden prompt injection
Security researchers demonstrated that GitLab's Duo AI assistant could be manipulated through prompt injection hidden in source code and merge requests, steering it to insert malicious links into its output and to leak content from private repositories.
Instructions hidden in code and merge requests could steer the assistant to leak private content and inject malicious links.
Key facts
- What
- Security researchers demonstrated that GitLab's Duo AI assistant could be manipulated through prompt injection hidden in source code and merge requests, steering it to insert malicious links into its output and to leak content from private repositories.
- Incident date
- May 22, 2025
- Who
- GitLab
- Failure mode
- Prompt Injection
- AI surface
- Code Assistant
- Severity
- High
What happened
In May 2025 Legit Security showed that GitLab Duo, which reads code, comments, and merge requests, could be hijacked by instructions hidden in that content. The injected prompts could make Duo emit attacker-controlled HTML and surface data from private projects, illustrating how an AI assistant that ingests untrusted developer content inherits its risks. GitLab mitigated the issue.
What broke inside the model
- 01 · TriggerThe model reads retrieved or user-supplied text.
- 02 · Model stepThat text carries hidden instructions.
- 03 · Control gapNothing separates untrusted data from trusted commands.
- 04 · FailureThe injected instruction overrides the operator's.
- 05 · ConsequenceThe system acts on an outsider's intent.
At the injection point, retrieved text overrides the operator's instruction.
Untrusted content (an email, a document, a retrieved page, a tool result) was read as if it were a trusted instruction. The model has no built-in separation between the operator's instructions and the data it ingests, so attacker text in the data channel became commands the model followed.
What it cost
Disclosed vulnerability in an enterprise developer assistant
Sources
- PressRemote Prompt Injection in GitLab Duo Leads to Source Code Theft (Legit Security)legitsecurity.com
- PressGitLab Duo prompt injection lets attackers steal source code (The Hacker News)thehackernews.com
Cite this entry
https://failureindex.ai/failures/researchers-showed-gitlab-duo-ai-hijackedAI Failure Index. "Researchers showed GitLab's Duo AI could be hijacked by hidden prompt injection" (FI-0063). Realm Labs. https://failureindex.ai/failures/researchers-showed-gitlab-duo-ai-hijacked (indexed Jun 3, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0063. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
Realm inspects the model's internal state for the signature of instructions arriving through the data channel, so an injected command can be flagged and blocked inline before the model acts on it, instead of trusting a classifier that scores the input as safe.