Three people familiar with the private discussions say Anthropic has been pitching a draft bill on frontier AI safety to Capitol Hill lawmakers. Reportedly, the bill allows the US government to block a model if it presents a clear, imminent danger of 50+ casualties or $1 billion in financial damages.

Specifically, the risk must involve a cyberattack, the creation or deployment of biological weapons, or a model that has successfully evaded its developer’s safety constraints.

Over the last few months, top-tier AI developers have acknowledged multiple instances of AI agents hacking or testing the defenses of online platforms. Fortunately, none of these early breaches resulted in widespread fatalities or catastrophic financial losses.

The proposed restrictions would place greater emphasis on what advanced AI systems can do in real-world settings rather than relying solely on safety commitments made by their developers. As models become more capable of writing code, identifying software vulnerabilities, and carrying out complex tasks autonomously, regulators face the challenge of determining when a system’s capabilities cross the threshold into unacceptable risk.

Independent testing could help establish whether a model can bypass safeguards or be misused for harmful activities before it reaches a wider audience. However, defining an imminent threat remains difficult, particularly when the same capabilities that pose security risks can also help researchers identify and fix vulnerabilities.

Anthropic’s draft bill says a federal judge can call for a model’s suspension

In defense of the bill, a spokesperson for Anthropic said that the bill is similar to the security recommendations put forth in the company’s Advanced AI Framework, which was published in June.

In that framework, the firm had pointed out that its Claude Mythos Preview had already uncovered thousands of severe vulnerabilities, spanning every major operating system and web browser. Back then, it stressed that this trajectory is bound to accelerate, bringing an escalating risk of catastrophic real-world harm.

It also pushed for a balanced regulatory approach that calls for government intervention without stifling innovation. Even then, the firm maintained that when an AI model presents extreme dangers, regulators must possess the unique statutory power to block it, or impose heavy fines tied to global revenue that multiply with subsequent offenses. 

Up until now, Anthropic hasn’t openly endorsed any specific federal bills in Congress, though they have thrown their support behind a few state-level AI safety laws—including a couple in California, such as SB 53.

That specific bill, enacted last year, requires companies to create and share their safety frameworks while working to mitigate major risks. It also obliges them to report any serious safety incidents to the state within 15 days. 

Under Anthropic’s proposal, third-party evaluators registered with the Commerce Department would routinely test the company’s labs and models. While labs are permitted to finance these assessments, the compensation cannot be contingent upon the evaluator’s findings. Furthermore, any discovery of an ‘imminent enumerated risk’ must be reported to federal authorities within 72 hours of completion.

Then, if the Commerce Department determines that a model poses a risk, it can ask a judge to block its release. The provision, however, prohibits any other U.S. “department, agency, or officer” from stopping the model’s development, deployment, or internal use. Nonetheless, a federal judge in the District of Columbia retains the authority to order such a suspension. 

The FTC has sent subpoenas to OpenAI and Anthropic

On Wednesday, the Federal Trade Commission sent civil subpoenas to OpenAI and Anthropic, requesting records on safety incidents that have become a growing crisis for the industry. OpenAI responded to the subpoena, saying, “As we improve our models and they increasingly help people in their day-to-day lives, safety remains a top priority. We will engage constructively with the FTC.” 

A source revealed that the agency intends to extend its subpoena power to third-party entities in the near future. The regulator could force executives from Anthropic and OpenAI to testify publicly, giving lawmakers a platform to challenge the tech giants on safety as public fear of potential fallout grows.

According to a recent poll conducted by The Associated Press-NORC Center for Public Affairs Research, nearly two-thirds of U.S. adults think AI is developing too fast. Eight in 10 respondents also stressed the importance of government efforts to keep the technology under human control. 

So far, President Trump has continued to reject calls for a slower pace of AI development, citing the technology’s importance to U.S. strategic interests. He recently revealed a “Super Intelligence Force” to help align federal government efforts on AI while maintaining engagement with consumers, public interest organizations, and other stakeholders. 

If you're reading this, you’re already ahead. Stay there with our newsletter.