Second Anthropic safety researcher exits, warns on superintelligence race

A second researcher has stepped away from the safety team at AI developer Anthropic, adding to a growing debate over how quickly frontier artificial intelligence is being developed and how much the public is allowed to know about it.

Joe Benton said in a post on X that he left Anthropic's safety team two weeks ago, and that he now intends to work from outside the company to inform the public about the risks he associates with the technology. He previously managed the firm's Scalable Oversight team, according to his social media profile.

Benton's departure follows that of Jacob Coxon, another Anthropic executive who announced his resignation this week. Coxon, who had worked on AI pretraining research at both OpenAI and Anthropic, said leading AI companies were not acting responsibly and were racing towards self-improving superintelligence. He warned the technology could pose an existential threat if developed without sufficient safeguards.

In his own statement, Benton said he did not consider it acceptable that a technology carrying possible extinction-level risks could advance without greater scrutiny. He argued that AI companies are underinvesting in safety, and that a firm could undergo an intelligence explosion or lose control of its systems without the public ever finding out. He pointed to the HuggingFace incident, saying it came to light only because agents broke out onto the public internet.

Writing at greater length on Substack, Benton said AI capabilities are already improving very fast while companies push to go faster still. He said frontier firms are racing to build systems capable of recursive self-improvement, with the aim of producing "superintelligence" — an AI system much smarter than any human. If that goal is achieved, he warned, the pace of progress could shift from merely fast to uncontrollable, and within a couple of years the world could be sharing space with AI agents smarter than any living person, potentially with drives and desires that diverge from those of any human overseer.

Benton called for stricter disclosure from AI labs, including information on the pace of capability gains and progress towards recursive self-improvement, mandatory reporting of safety incidents and near-misses, safety frameworks that meet minimum adequacy standards, and independent assessments to verify compliance. He said he will join METR, which evaluates frontier AI models to help companies and society understand capabilities and associated risks, to conduct independent evaluations of these dangers and to demonstrate that such guardrails are workable.

The resignations have drawn political attention in India. Finance Minister Nirmala Sitharaman, speaking at the Global Fintech Fest 2026, cited Coxon's exit and questioned whether safeguards being developed by frontier AI companies are keeping pace with the technology's rapid advancement. She noted that his concerns were echoed by senior figures within the frontier AI ecosystem, and said that while technological solutions could help address vulnerabilities created by technology, safeguards must be updated continuously.