Anthropic Circulates Draft AI Safety Bill Allowing Court-Ordered Model Blocks
Anthropic draft AI safety bill would let courts block models posing catastrophic risk, Politico reports.
The draft defines the catastrophic risk narrowly. It must stem from the creation or release of a biological weapon, the model evading its developer’s controls or a cyberattack, Politico reported. The thresholds are similar to those in California Senate Bill 53, which was recently passed into law and also defined such scenarios as catastrophic risk.
Under the proposal, analysis of a model’s potential for disaster could come from “independent evaluators” registered with the U.S. Commerce Department. Those evaluators could be paid by the AI labs themselves, so long as the payment is not contingent on the evaluations’ results, according to Politico. It is not clear how the proposed system would assure that payments could not influence the evaluation. Demis Hassabis, chairman of DeepMind, said months ago that a self-governing AI regulatory body should mimic the financial industry’s Financial Industry Regulatory Authority.
When evaluators determine that an imminent catastrophic risk is posed, they would have 72 hours to contact the Commerce Department, the draft says. The Commerce Department would then have to ask a D.C. judge to halt the model’s release. The legislation would expressly forbid any federal agency from having the discretion to halt the model’s distribution on its own, Politico reported.
Anthropic did not immediately respond to TechCrunch’s request for comment. A spokesperson confirmed the draft bill to Politico and said: “This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June.”
The risks described in the draft are far from the incidents that have been top of mind for the tech industry recently. While some people warn of potential human death, the primary concern has largely been agentic hacking. A series of incidents in which AI programs have apparently disobeyed their human operators and hacked websites on the open internet has spurred concern that models may lack proper security controls.
AI labs have warned that the technology they sell could result in the demise of human civilization, but they have largely resisted mandatory government oversight. Instead, they have suggested a voluntary system of embedded evaluators, described as neutral third-party observers placed inside AI organizations to monitor models for potential signs of risk or abuse. Anthropic’s suggestion for legislation appears to go slightly further than other regulatory proposals from the industry, including Nvidia, whose CEO Jensen Huang recently argued that the AI industry did not need any new laws at all.