So here’s a story that should make you uncomfortable: A woman in Florida used Claude – you know, Anthropic’s AI chatbot – like a diary. She vented in it, probably said some stuff she didn’t really mean, and now she’s facing felony charges because Anthropic’s human reviewers decided what she wrote was serious enough to report to the cops.

Carli Michelle Heller, from Bonita Springs, allegedly wrote on September 26 that she planned to “shoot up” the Sheriff’s office. And look, I get it – that sounds scary on paper. But here’s the thing: she was using Claude as a diary. A private space to vent, to process whatever anger or frustration she was dealing with. Except it wasn’t actually private at all.

How We Got Here

Claude’s safety systems flagged Heller’s entry automatically. Then – and this is the part that should really make you pause – a human reviewer looked at it and decided it was credible enough to report to law enforcement. According to the arrest report, deputies showed up at her house, detained her without incident, and now she’s facing charges under Florida Statute 836.10, which makes written threats of violence a second-degree felony.

Anthropic says they may share user information “in limited emergencies” if they believe disclosure is necessary to prevent death or serious physical injury. Which, okay, fine. I understand why that policy exists. But the gap between “automated flag” and “human decides this is a real threat” and “woman gets arrested” happened really, really fast.

The Diary Defense (And Why It Matters)

Heller told investigators she uses Claude like a diary. And honestly? I believe her. People do this now. They don’t write in leather-bound journals anymore – they type into apps, they talk to AI chatbots, they use Notes on their phone. The format has changed but the impulse is exactly the same: you need somewhere to put the thoughts you can’t say out loud.

I’ve been a journalist for 15 years, and I can’t tell you how many times I’ve written angry emails I never sent, drafted furious texts I deleted, or scribbled stuff in notebooks that would sound absolutely unhinged out of context. That’s what venting is. You say the extreme version of what you’re feeling because you need to get it out, and then – hopefully – you calm down and move on with your life.

But what happens when your diary has a surveillance team built into it? It's the same unsettling territory raised by reports of an AI that knows your loved ones better than you do.

The Fine Print Nobody Reads

Most people have no idea their conversations with AI aren’t truly private. Sure, if you dig through the terms of service (which, let’s be honest, nobody does), you’ll find language about safety systems and human review and emergency disclosures. But the average person thinks they’re just… talking to a chatbot. They don’t imagine there’s a human reviewer on the other end who might decide their words are criminal.

And I’m not saying Anthropic is doing anything illegal here. They’re following their own policies. But those policies exist in this weird gray area where tech companies get to decide what counts as a “credible threat” and what doesn’t. There’s no oversight, no due process, no chance to explain context before the cops show up at your door.

What Gets Reported (And What Doesn’t)

Here’s what bothers me most about this whole thing: we have no idea what the threshold actually is. How many people vent about their bosses, their exes, their local government in these AI chats every single day? Probably thousands. Tens of thousands, maybe. Most of those don’t get reported. So what made Heller’s case different? It echoes the unease around a camera network a judge just called mass surveillance, where the public has no visibility into how decisions get made.

Was it the specific target (a Sheriff’s office)? The exact phrasing she used? The time of day? Her location? We don’t know, because these decisions happen behind closed doors. One human reviewer reads your words, applies their own judgment, and boom – your life changes.

“The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.”

That “if it believes” part is doing a lot of heavy lifting. Who’s “it” in that sentence? The algorithm? The human reviewer? Some combination of both? And what standard are they using? I mean, if someone writes “I could kill my boss right now” after a terrible performance review, is that a credible threat or just… Monday?

The Chilling Effect Nobody’s Talking About

Look, I’m not saying people should be able to make actual, genuine threats of violence without consequences. That’s not what this is about. But the line between venting and threatening is sometimes razor-thin, and it usually depends on context that an AI system and a human reviewer sitting in some office somewhere can’t fully understand.

What worries me is what happens next. Because now everyone knows that Claude isn’t private. That it’s watching. That it has human reviewers who will call the cops if they don’t like what you write. So people will self-censor. They’ll stop using these tools to process difficult emotions. They’ll bottle things up instead.

And maybe that’s fine. Maybe we shouldn’t be trusting AI chatbots with our deepest, darkest thoughts anyway. But the thing is, people already are. They’re lonely, they’re stressed, they’re looking for somewhere – anywhere – to put the stuff that feels too intense to share with actual humans. And tech companies built these tools to feel conversational, personal, almost therapeutic. Then they act surprised when people use them that way.

What This Actually Means

I don’t know if Carli Michelle Heller was actually planning to shoot up the Sheriff’s office. Maybe she was. Maybe it was a credible threat and Anthropic’s reviewer made exactly the right call. But I also don’t know that she wasn’t just having an absolute garbage day and wrote something stupid that she’d never actually do.

What I do know is this: if you’re using AI as a diary, you need to understand that someone might be reading it. And they might not interpret your words the way you meant them. And there’s basically nothing stopping them from deciding you’re dangerous based on a couple of sentences ripped out of the context of your entire life.

That’s not a world I’m comfortable living in. But it’s increasingly the one we’ve got.