Ninety percent. Let that sink in for a second. If a machine you built to make life-or-death coverage decisions was wrong nine times out of ten, you’d probably stop using it, right? Pull it offline, apologize, maybe rethink the whole business model? Well, according to a lawsuit making its way through the courts, UnitedHealth apparently looked at that number and thought, yeah, this works fine, keep going.

I’ve covered healthcare denial stories for years now, and I have to admit, this one still made me put my coffee down. Not because insurers denying claims is new (it’s basically their whole thing, unfortunately), but because of the sheer audacity of leaning on an algorithm that’s broken nine out of ten times and calling it a system.

So What’s Actually Being Alleged Here?

The lawsuit centers on an AI tool called nH Predict, which UnitedHealth and its subsidiary NaviHealth reportedly used to make decisions about post-acute care – think nursing homes, rehab facilities, the stuff patients need after a hospital stay before they’re ready to go home. The families behind the suit claim the model was used to cut off coverage for elderly patients way before their doctors thought it was safe, and that internal data showed the model’s predictions were wrong an eye-watering amount of the time.

And here’s the part that really gets me. When patients or families appealed these denials, and actual humans (doctors, reviewers, whoever) looked at the cases again, the AI’s determination got overturned around 90% of the time. Ninety percent. That’s not a rounding error. That’s not “the model needs some fine-tuning.” That’s a system that is, by almost any measure, not working.

The Incentive Problem Nobody Wants to Talk About

Here’s the thing though – and I think this is what actually matters more than the error rate itself – a broken system that happens to save the company money isn’t really “broken” from a business perspective. It’s doing exactly what it was built to do. If denying care cuts costs, and only a small fraction of people have the time, money, or stamina to appeal, then the model is working great. For the insurer, anyway.

I’ve seen this pattern before in other industries. When an algorithm’s “mistakes” all happen to benefit the company deploying it, you have to ask whether it’s really a mistake at all, or just… the point.

Why Would Anyone Trust This Thing In The First Place?

This is where it gets kind of maddening, if I’m being honest. According to the allegations, NaviHealth case managers were reportedly evaluated on how closely they stuck to the AI’s predicted length of stay. So if the model said grandma needed 14 days in rehab but she actually needed three weeks, the case manager pushing back on that could face consequences at work. That’s not clinical judgment driving decisions anymore. That’s a spreadsheet.

Doctors, physical therapists, actual humans with medical training were reportedly being second-guessed by a tool that got it wrong the vast majority of the time when challenged. I don’t know about you, but that seems… backwards? Like, deeply backwards.

“They were using a computer program to make decisions about elderly people’s care that even the company’s own appeals process couldn’t uphold nine times out of ten.”

That’s basically the crux of the complaint, and honestly it’s hard to argue with the logic. If your own internal review process disagrees with the AI most of the time, what exactly is the AI for?

This Isn’t Just A UnitedHealth Problem

I want to be careful not to make this sound like one bad apple insurer doing something uniquely evil, because that’s letting the industry off way too easy. AI-driven claim denials have been creeping into healthcare for a few years now, quietly, without much oversight, and mostly without patients even knowing an algorithm was involved in their care decisions at all. UnitedHealth just happens to be the one getting dragged into court over it with numbers this damning attached.

The bigger issue – and this is the part that should scare basically everyone, insured or not – is that these systems are opaque by design. Patients don’t get to see the model. Doctors often don’t either. You get a denial letter, maybe a vague reference to “clinical guidelines,” and that’s it. Good luck figuring out whether a human being or a piece of software with a 90% error rate decided your mom couldn’t stay in rehab another week.

The Appeals Process Is Doing A Lot Of Heavy Lifting

What’s interesting here, and kind of infuriating if you think about it too long, is that the appeals process is apparently the only thing catching these errors. Which means the system is basically betting that most people won’t appeal. And they’d be right to bet that. Appeals take time, paperwork, persistence, and honestly a level of health literacy and energy that a lot of sick, elderly, or overwhelmed families just don’t have in the moment. So the 90% failure rate we’re talking about? That’s probably the floor, not the ceiling, because it only counts the cases where someone had the wherewithal to fight back.

What This Actually Means

Look, I don’t think AI in healthcare is inherently the villain here. Used well, with actual oversight and transparency, these tools could genuinely help doctors make faster, better-informed decisions. That’s not the fantasy version, that’s a real possibility. But this case, if the allegations hold up, isn’t really a story about AI failing. It’s a story about a company that knew a tool was failing and used it anyway because the failures happened to line up nicely with the balance sheet.

That’s not a technology problem. That’s a people problem wearing a technology costume.

Whatever happens in court, I think this case is going to end up being a preview of a much bigger fight that’s coming – over who’s accountable when an algorithm makes a call that affects someone’s actual body, someone’s actual life, and gets it wrong almost every time anyone bothers to check. We should probably figure that out before there are a few dozen more lawsuits just like this one.