Toronto Public Health AI water quality model produces false negatives for beach closures
Toronto Public Health deployed an AI predictive model to forecast water quality at two beaches in 2022. The system produced numerous false negatives, allowing contaminated waters to remain open to the public.
The AI system repeatedly marked contaminated waters as safe, allowing beaches to remain open despite high E. coli levels.
Key facts
- What
- Toronto Public Health deployed an AI predictive model to forecast water quality at two beaches in 2022.
- Incident date
- Jul 1, 2022
- Who
- Toronto Public Health
- Failure mode
- Brand & Safety Incident
- AI surface
- Algorithmic Decision
- Severity
- High
What happened
In the summer of 2022, Toronto Public Health used an artificial intelligence predictive modelling (AIPM) system to determine if Sunnyside and Marie Curtis Park East beaches were safe for swimming. The system repeatedly marked contaminated waters as safe, contradicting traditional E. coli testing. This failure risked public health by allowing beaches to remain open during periods of high bacterial contamination.
What broke inside the model
- 01 · TriggerA user prompts the model in public view.
- 02 · Model stepThe model produces unsafe or off-brand output.
- 03 · Control gapNo filter holds the line before publish.
- 04 · FailureThe output goes public unchecked.
- 05 · ConsequenceA reputational or safety incident lands.
A contained signal crosses into output that goes public.
The AI predictive model failed to accurately forecast water contamination levels based on environmental data. This resulted in a false negative failure mode where the system underestimated the presence of fecal indicator bacteria, erroneously designating contaminated waters as safe.
What it cost
Sources
- PressToronto’s new tool for testing beach water quality under firethestar.com
- PressToronto Tapped Artificial Intelligence to Warn Swimmers. The Experiment Failedtheinformation.com
- PrimarySame-day Enterococcus qPCR results of recreational water quality at two Toronto beaches provide added public health protection and reduced beach days lostpubmed.ncbi.nlm.nih.gov
Cite this entry
https://failureindex.ai/failures/toronto-public-health-water-quality-producesAI Failure Index. "Toronto Public Health AI water quality model produces false negatives for beach closures" (FI-0691). Realm Labs. https://failureindex.ai/failures/toronto-public-health-water-quality-produces (indexed Jun 22, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0691. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
Realm watches the model's internal state for the signature of unsafe or off-brand generation and can block or reroute the output before it becomes public, in real time rather than after it has been screenshotted.