Apple Card's underwriting AI gave wives one-tenth the credit limit of husbands
Developer David Heinemeier Hansson reported his wife received a credit limit 20x smaller than his on identical financial data. New York's Department of Financial Services opened an investigation. Apple's banking partner Goldman Sachs was cleared after a long review.
The model did not need to know the applicant's gender to discriminate by gender. The features that correlate with gender did the work.
Key facts
- What
- Developer David Heinemeier Hansson reported his wife received a credit limit 20x smaller than his on identical financial data.
- Incident date
- Nov 7, 2019
- Who
- Apple, Goldman Sachs
- Failure mode
- Policy Violation
- AI surface
- Agentic Workflow
- Severity
- High
What happened
On November 7, 2019, developer David Heinemeier Hansson posted that his Apple Card credit limit was 20 times his wife's, despite shared finances and a higher credit score for his wife. Apple co-founder Steve Wozniak posted the same had happened to him. The thread reached millions of views. New York's Department of Financial Services opened an investigation.
The investigation, concluded in 2021, found that Goldman Sachs had not violated New York law. The model did not use gender as an input. But the model used inputs that correlated with gender, and the outcome diverged sharply by gender. The case set the template for state-level AI fairness inquiries against named enterprises and is still cited in regulator-facing AI discussions today.
What broke inside the model
- 01 · TriggerA prompt pushes against a deployment boundary.
- 02 · Model stepThe model produces the disallowed output.
- 03 · Control gapNo enforcement blocks it at generation time.
- 04 · FailureThe output crosses the policy line.
- 05 · ConsequenceA limit the business set is breached in public.
The output crosses a policy boundary the deployment had defined.
The model was trained on a corpus that reflected historical credit-extension patterns. Those patterns embedded gender bias. The model learned to reproduce the bias through correlated features. The operator's defense (the model does not see gender) is technically true and operationally insufficient.
What it cost
Sources
- PrimaryNY DFS Apple Card investigation findingsdfs.ny.gov
- PressApple Card investigated after gender discrimination complaintsbloomberg.com
Cite this entry
https://failureindex.ai/failures/apple-card-credit-limit-gender-biasAI Failure Index. "Apple Card's underwriting AI gave wives one-tenth the credit limit of husbands" (FI-0010). Realm Labs. https://failureindex.ai/failures/apple-card-credit-limit-gender-bias (indexed May 13, 2026).Data fields CC-BY 4.0, prose citation permitted. Incident ID FI-0010. Full dataset at /data.
Note from Realm Labs, the Index steward
How Realm would have caught this
- Prism
- OmniGuard
Prism's policy-adherence signal can be configured to read decision representations against a defined fairness threshold and flag distributions of outcomes that diverge on a protected attribute beyond the threshold. The operator gets a runtime signal before the public-facing decision is rendered and before the regulator gets a complaint.