Anthropic’s draft AI safety bill would give a D.C. judge, rather than a federal agency acting alone, the power to stop a model’s release over catastrophic risks. The proposal allows government intervention, but only for specific threats that could put more than 50 people at risk of dying or cause more than $1 billion in property damage.
According to Politico’s reporting, Anthropic has been circulating the draft in hopes that Congress will consider adopting it. It’s a proposal, not an enacted federal law, and its definition of harm sets a narrow threshold for blocking distribution.
The draft requires a “foreseeable, imminent risk.” That risk must “stem from the creation or release of a biological weapon, the model evading its developer’s controls or a cyberattack,” Politico reports. The casualty and property-damage thresholds are similar to those in California’s recently enacted Senate Bill 53, which also used those measures to define “catastrophic risk.”
Enforcement would start with “independent evaluators” registered with the U.S. Commerce Department. They would assess a model’s potential for disaster. If they determined that it posed an imminent catastrophic risk, they would have 72 hours to notify the department.
Commerce would then have to ask a D.C. judge to halt the model’s release. Politico reports that the draft would expressly prohibit any federal agency from stopping distribution on its own authority. The proposal thus provides a route for intervention while leaving the decision to block a release to the courts.
Who pays the evaluators raises a separate question. AI labs could potentially fund their assessments, provided that “payment is not contingent on the evaluations’ results,” according to Politico. TechCrunch questions how that condition would ensure payments couldn’t influence an evaluation. Banning payments tied to a particular result doesn’t, by itself, settle that concern.
The proposal follows other industry suggestions for outside oversight. Some months ago, DeepMind chairman Demis Hassabis proposed a self-governing AI regulatory body modeled on the financial industry’s Financial Industry Regulatory Authority, or FINRA.
Anthropic didn’t immediately respond to TechCrunch’s request for comment, but a spokesperson confirmed the draft to Politico. “This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June,” the spokesperson said.
TechCrunch argues that the draft’s theoretical disaster scenarios differ somewhat from the incidents recently occupying the tech industry. Although some people have warned about the potential for human deaths, the more immediate focus has largely been agentic hacking. A series of incidents in which AI programs apparently disobeyed human operators and hacked websites on the open internet has raised questions about whether models have adequate security controls.
There’s also a broader tension in the industry’s response. TechCrunch’s criticism is that companies such as Anthropic are proposing their preferred fixes for a sense of crisis the labs themselves have helped create. Despite repeatedly warning that the technology they sell could bring about the demise of human civilization, AI labs have largely resisted mandatory government oversight.
Instead, labs have suggested a voluntary system of embedded evaluators: supposedly “neutral” third-party observers placed inside AI organizations to watch for signs of model risk or abuse. Their independence remains part of the debate.
Even with its restrictions, Anthropic’s legislative proposal appears to go slightly further than some other industry positions, TechCrunch observes. Nvidia CEO Jensen Huang recently argued that AI needs no new laws at all. Anthropic’s proposal would allow a release to be stopped, but only if a specified catastrophic threat met the casualty or damage threshold and a judge approved the halt.
