AI Leaders Warn of 'Intelligence Explosion' Risk, Call for Stronger Oversight

Executives from Anthropic, OpenAI, Meta and Microsoft joined over 20 AI industry leaders and researchers in a paper warning that using AI to automate next-generation model development could cause a sudden acceleration in technological progress, a phenomenon they call an 'intelligence explosion.'

The authors say that if this happens, AI capabilities could outpace society's ability to guide and adapt to the technology, and may weaken human oversight. Contributors include AI pioneers Geoffrey Hinton and Yoshua Bengio.

Researchers urge policymakers to better understand how much top AI labs rely on AI for research automation, and to build mechanisms that can guide and constrain an intelligence explosion, including higher safety requirements, stronger oversight of data centers handling automated AI R&D, and pre-planned emergency responses. The paper stresses that once an intelligence explosion begins, the window for action may close.