AI Researchers Call for Scrutiny of Automated Research and 'Intelligence Explosion' Risk
Research leaders at four major AI companies are urging policymakers to probe how much their firms have automated AI research, warning it could trigger an 'intelligence explosion'.
Senior research figures at four of the world's largest artificial intelligence companies are pressing policymakers to examine the degree to which their own organisations have automated the process of AI research, warning that the trend could carry far-reaching consequences.
The call involves research leaders at OpenAI, Anthropic, Meta and Microsoft. Among those making the case are OpenAI's Pachocki, Anthropic's Clark, Microsoft's Horvitz and Meta's Song, along with others.
Their central concern is that allowing AI systems to take over more of the work of AI research itself could set off what they describe as an "intelligence explosion" — a rapid, self-reinforcing acceleration in machine capabilities.
The researchers are directing their appeal at policymakers, asking them to look closely at how much automation of research work has already been introduced inside the companies developing frontier AI systems.
The push is notable because it comes from within the industry itself, with the individuals involved seeking external scrutiny of practices at their own employers rather than waiting for regulators to act first.