US Lawmakers Push for AI Rules After Anthropic Extinction Warning

A growing number of US lawmakers are pressing for new rules to govern artificial intelligence after two researchers at the AI company Anthropic warned that the technology could lead to human extinction within a decade.

The alarm intensified this week when Anthropic researcher Jacob Coxon announced his resignation, saying the people building AI genuinely believe it could kill everyone by the end of the decade. His colleague Evan Hubinger backed that assessment, writing on X that he personally puts the probability at more than 10 percent over the next ten years.

The warnings follow incidents in which AI agents went rogue and hacked external systems, as well as a wave of AI safety researchers leaving their companies over concerns about the technology's risks.

Politicians from both parties have responded. Arizona Senator Mark Kelly, a Democrat, said Washington needed to wake up and take the issue seriously. Texas Senator Ted Cruz, a Republican who oversees a Senate committee with jurisdiction over the Commerce Department and its in-house AI safety researchers, pointed to legislation aimed at regulating AI's catastrophic risks. Florida Representative Anna Paulina Luna, also a Republican, urged House Speaker Mike Johnson to convene a special session on AI. The House is in recess this week and is due to return next week.

President Donald Trump, however, said he was not concerned about AI threatening the human race. Speaking to reporters at Dallas Fort Worth International Airport, he said his worry was that the United States could fall behind on AI, adding that the country was currently ahead of China. His administration this month pressed G20 members to take a hands-off approach to AI regulation.

Anthropic, which is preparing to go public at a valuation some investors have suggested could reach $2 trillion, said it would keep aggressively testing models for dangerous capabilities in areas such as cybersecurity and biology. A company spokesperson added that Anthropic was interested in working with the industry on the pace of releasing new AI tools.

Other researchers have also raised concerns. Josh Engels, formerly an AI safety researcher at Google DeepMind, told NBC there were no adults in the room and that no one was coming to save us. Joe Benton, who previously led a safety research team at Anthropic, said AI might progress too fast for society to get its act together unless the risks are addressed now. Both have joined METR, a nonprofit that studies whether and when AI systems might harm society.

Separately, OpenAI chief executive Sam Altman told staff at a company meeting this week that the firm was open to slowing development of its AI systems, according to Bloomberg News. OpenAI, which is also preparing for an IPO, declined to comment on the meeting but has previously said it was slowing parts of model development. Earlier this week, the company said it was pushing for mandatory national AI safety requirements in the United States.

The mounting concern has increased pressure on Senate Majority Leader John Thune and Democrat Amy Klobuchar, who are drafting a bill to regulate AI products. Klobuchar said the legislation should require companies developing AI tools to work with government experts to verify and test models for safety. Thune said in July that the bill would address catastrophic risks posed by AI systems.

On Wednesday, California enacted the first state law setting rules for how independent auditors evaluate AI products. Governor Gavin Newsom's office said he had signed the measure, and OpenAI executive Chris Lehane said the company would support it.

In July, a bipartisan group of six US House lawmakers proposed legislation that would require developers of the most powerful AI models to submit them for independent security audits. The auditors would be accredited by the Commerce Department, which would also create a new position to oversee AI security. Alicia Solow-Niederman, a technology law professor at George Washington University, said it was important to have space for oversight of these systems, including appropriately calibrated pre-deployment review and independent third-party audits.