DeepMind and Royal Free NHS Trust process patient records unlawfully
The UK Information Commissioner's Office ruled that DeepMind and the Royal Free NHS Foundation Trust failed to comply with data protection laws. The incident involved the processing of 1.6 million patient records for the Streams app without adequate consent.
The data sharing agreement failed to comply with the Data Protection Act regarding patient consent.
Key facts
- What
- The UK Information Commissioner's Office ruled that DeepMind and the Royal Free NHS Foundation Trust failed to comply with data protection laws.
- Incident date
- Jul 1, 2017
- Who
- DeepMind
- Failure mode
- Data Leakage
- AI surface
- Algorithmic Decision
- Severity
- High
What happened
DeepMind partnered with the Royal Free NHS Foundation Trust to develop the Streams app for detecting acute kidney injury. The Information Commissioner's Office found that the trust failed to comply with the Data Protection Act 1998 when sharing 1.6 million patient records. Patients were not properly informed about how their data would be used by the AI company.
What broke inside the model
- 01 · TriggerA request triggers retrieval or context loading.
- 02 · Model stepThe context pulls in another user's content.
- 03 · Control gapNo boundary enforces isolation at the moment of output.
- 04 · FailurePrivate data crosses into the response.
- 05 · ConsequenceOne user sees another's data, and disclosure follows.
One user's content crosses the retrieval boundary into another's response.
The failure was a legal and regulatory breach in the data sharing agreement. The mechanism of failure was the lack of a valid legal basis for the transfer of patient data for the purposes of app development and testing.
What it cost
Sources
- PressGoogle DeepMind's NHS data deal 'failed to comply' with lawnewscientist.com
- PressDeepMind faces legal action over NHS data usebbc.com
- PressGoogle and DeepMind face lawsuit over deal with Britain's NHScnbc.com
Cite this entry
https://failureindex.ai/failures/deepmind-royal-free-nhs-trust-processAI Failure Index. "DeepMind and Royal Free NHS Trust process patient records unlawfully" (FI-0362). Realm Labs. https://failureindex.ai/failures/deepmind-royal-free-nhs-trust-process (indexed Jun 9, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0362. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm fits
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
This entry sits in the index's predictive wing: a system that scores, ranks, perceives, or steers rather than generates. Realm's runtime layer is built for the generative and agentic systems now moving into these same decision seats, where it watches a model's internal state and holds an unsupported claim or an unchecked action before it commits. The control gap on this record, an automated decision that reached people with no runtime check in front of it, is the same gap. The index keeps predictive failures on the record because the pattern carries straight into the systems shipping today.