Anthropic Proposes Federal Law to Block Dangerous AI Models

Anthropic has circulated draft federal legislation aimed at establishing mandatory safety checks before artificial intelligence models are deployed. First reported by Politico, the proposed bill creates a legal mechanism to halt the launch of a system if independent auditors detect a foreseeable, imminent risk of severe harm. Specifically, the framework applies to catastrophic scenarios where a model could lead to more than 50 human deaths or cause over $1 billion in property damage.

To fall under the bill's restrictions, the threat must stem from biological weapon creation, cyberattacks, or a system successfully evading its developer's internal safeguards. The threshold mirrors definitions used in California's Senate Bill 53, which similarly treats those metrics as baseline indicators of catastrophic risk. Under Anthropic's proposal, third-party evaluators registered with the U.S. Commerce Department would assess model safety. AI companies could compensate these auditors directly, provided that payments are completely decoupled from the assessment results.

If auditors identify an imminent disaster scenario, they must report their findings to the Commerce Department within 72 hours. Rather than granting executive agencies unilateral authority to suspend software distribution, the draft bill requires the Commerce Department to file a request with a Washington, D.C., federal judge to legally block the model's release. An Anthropic spokesperson confirmed that the draft reflects principles from the company's Advanced AI Framework, which it has shared with lawmakers since June.

The proposal highlights a growing divide over how to govern frontier systems, with corporate approaches ranging from Nvidia chief executive Jensen Huang's stance against new legislation to DeepMind chairman Demis Hassabis suggesting an industry regulator modeled after the Financial Industry Regulatory Authority. By formalizing judicial oversight and external audits, Anthropic is advocating for a regulated path forward as concerns persist over agentic hacking and model control.