U S Department of Health and Human AI failures

HHS turned ChatGPT loose on Medicaid-linked audits with defunding power and no published error rate

On May 21, 2026, HHS launched AERO, the Audit Enforcement and Risk Oversight initiative, using ChatGPT and other LLMs to scan at least five years of Single Audit compliance filings from every state, hospital system, university, and nonprofit spending $1 million or more in annual federal funds. Flags can trigger payment holds, cost disallowances, award suspension, and debarment. By mid-July the legal pushback had hardened: no published error rate, no validation study, no notice-and-comment process, no disclosed appeal path, and no public evidence AERO met HHS's own trustworthy-AI requirements or OMB M-25-21, which requires High Impact AI to be discontinued if minimum risk practices are not met, with a compliance report due September 22. Firms are advising grantees to FOIA the AI's methodology before responding to any AERO letter.

Confidence

Medium (multi-source)