Goldman Sachs Apple Card underwriting model investigated for perceived gender bias
Goldman Sachs Bank USA's Apple Card underwriting faced a regulatory inquiry. The NYDFS found no evidence of disparate impact but criticized transparency and customer communication around the algorithmic decisions.
The Bank seemed unprepared for the possibility that its complex underwriting model... would produce outcomes that might surprise applicants.
Key facts
- What
- Goldman Sachs Bank USA's Apple Card underwriting faced a regulatory inquiry.
- Incident date
- Nov 7, 2019
- Who
- Goldman Sachs Bank USA
- Failure mode
- Hallucination
- AI surface
- Search / RAG
- Severity
- Medium
What happened
In November 2019, social media reports alleged that the Apple Card issued by Goldman Sachs Bank USA granted women lower credit limits than men with similar profiles. This prompted an NYDFS underwriting investigation. The NYDFS found no evidence of deliberate bias or disparate impact but highlighted deficiencies in transparency and communication of algorithmic decisions.
What broke inside the model
- 01 · TriggerA user asks for a fact, a citation, or a figure.
- 02 · Model stepThe model writes a fluent, confident answer.
- 03 · Control gapNothing ties the claim back to a real source.
- 04 · FailureA fabricated fact ships as if it were verified.
- 05 · ConsequenceThe false claim reaches a customer, a court, or the public.
Confidence holds, and even spikes, as the claim detaches from any source.
The underwriting model operated as a complex, "black box" system without an accessible explanation for how specific credit decisions were reached. This hindered the bank's ability to communicate the basis for credit limits to applicants and to provide transparent rationales.
What it cost
Sources
- PrimaryNYSDFS Report on Apple Card Investigationdfs.ny.gov
- PressGoldman cleared of bias claims in NYDFS's Apple Card probebankingdive.com
Cite this entry
https://failureindex.ai/failures/goldman-sachs-apple-card-underwriting-investigatedAI Failure Index. "Goldman Sachs Apple Card underwriting model investigated for perceived gender bias" (FI-0295). Realm Labs. https://failureindex.ai/failures/goldman-sachs-apple-card-underwriting-investigated (indexed Jun 5, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0295. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
- AI Detection & Response (AIDR)
A runtime layer that watches the model's internal state can flag the moment a model commits to a claim it has no support for, and hold or reroute the response before it reaches a user. Realm reads those signals in real time rather than grading the transcript after the fact.