Crisis Text Line ends data-sharing with for-profit spinoff Loris.ai
Crisis Text Line admitted to sharing anonymized user data with its for-profit subsidiary, Loris.ai, for machine learning development. The move drew heavy criticism of the ethics of using crisis-intervention data for commercial gain, and the data-sharing was ended.
Crisis Text Line shared sensitive, de-identified crisis data with a commercial entity despite assurances that data would not be used for profit.
Key facts
- What
- Crisis Text Line admitted to sharing anonymized user data with its for-profit subsidiary, Loris.ai, for machine learning development.
- Incident date
- Jan 31, 2022
- Who
- Crisis Text Line
- Failure mode
- Policy Violation
- AI surface
- Chatbot
- Severity
- High
What happened
Crisis Text Line shared de-identified texter conversations with its commercial subsidiary Loris.ai to train machine learning models for conversation intelligence. After public backlash, the data-sharing agreement was ended effective immediately on January 31, 2022. This sequence is reported by Politico, BBC, and Crisis Text Line itself.
What broke inside the model
- 01 · TriggerA prompt pushes against a deployment boundary.
- 02 · Model stepThe model produces the disallowed output.
- 03 · Control gapNo enforcement blocks it at generation time.
- 04 · FailureThe output crosses the policy line.
- 05 · ConsequenceA limit the business set is breached in public.
The output crosses a policy boundary the deployment had defined.
The failure occurred in the informed consent process and ethical governance. A general Terms of Service did not explicitly notify users that sensitive crisis data would be used for commercial AI training.
What it cost
Sources
- PressCrisis Text Line ends data-sharing relationship with for-profit spinoffpolitico.com
- PressMental health helpline ends data-sharing relationshipbbc.com
- Customer-DisclosedAn update on data privacy, our community and our servicecrisistextline.org
Cite this entry
https://failureindex.ai/failures/crisis-text-line-ends-sharing-profitAI Failure Index. "Crisis Text Line ends data-sharing with for-profit spinoff Loris.ai" (FI-0187). Realm Labs. https://failureindex.ai/failures/crisis-text-line-ends-sharing-profit (indexed Jun 5, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0187. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
Realm compares what the model is about to output or do against the policy that governs the deployment, in real time, and can deny or redact the action before it takes effect, which is the gap an after-the-fact review never closes in time.