Carli Michelle Heller was arrested in Bonita Springs, Florida, after allegedly expressing her intention to “shoot up” a local sheriff’s office using Anthropic’s chatbot, Claude. This incident brings to light the critical balance between user privacy and the responsibilities of AI companies in monitoring conversations for potential threats.
Heller faces charges for making a written threat of violence, which is classified as a second-degree felony under Florida law. The situation unfolded when Anthropic’s internal monitoring software flagged her chat logs for human review. An employee evaluated the logs and determined that Heller’s statements represented a credible threat, leading them to notify local law enforcement.
Lee County Sheriff Carmine Marceno told local reporters that Heller had been using Claude like a “diary,” apparently unaware that Anthropic employees could read her conversations.
The conflict between user privacy and the duty of AI companies to prevent violent acts is evident. In February, a mass shooting in Tumbler Ridge, Canada, highlighted this issue. The shooter had extensively used ChatGPT to plan her attack, and OpenAI staff had previously flagged concerning chat logs months before the tragedy. There were even attempts by company executives to downplay their product’s involvement in the incident.
While Anthropic’s own track record on real-world harm is wretched, the company appears to be trying to avoid the same kind of incident – though with near-endless chatbot options now on the market, it could already be too late to prevent the next AI-assisted tragedy.
