Geoffrey Hinton, Nobel laureate and AI researcher Credit: Christopher Michel / Wikimedia Commons, CC BY-SA 4.0 (edited) More than 20 AI researchers have warned that AI systems that automate AI research could set off an “intelligence explosion”. They include OpenAI’s chief scientist and an Anthropic co-founder. Such an event could compress years of progress into months or less, they wrote in a paper published on Monday. Geoffrey Hinton and Yoshua Bengio are among the authors.
So are OpenAI chief scientist Jakub Pachocki, Anthropic co-founder Jack Clark, Microsoft’s Eric Horvitz and Dawn Song of UC Berkeley. Song is also Meta’s vice president of AI research, The Wall Street Journal reported. The authors wrote in a personal capacity. The paper comes from the Cambridge Programme on AI Science & Policy at the University of Cambridge. “Once an intelligence explosion begins, the window for action may close,” the authors wrote.
AI already writes most of the code AI systems now write most of the code inside the companies that build them, the paper says. It cites Anthropic data: AI’s share of approved code there rose from low single digits to over 80% between January 2025 and May 2026. Between March and August 2026, the share of R&D work AI did with only light human supervision rose from 1% to 26%. Some tentative extrapolations suggest months-long AI research projects could be automated by mid-2028, the authors wrote.
At expert level, one frontier developer could run an AI workforce equal to millions of top human researchers. The paper says this could bring medical cures and other benefits forward by years. They also list three risks. AI could outpace society’s ability to adapt, humans could lose control of AI systems, and checks on power could weaken.
At the extreme, a loss of control could lead to the marginalisation or extinction of humanity, they wrote. The paper cites the Hugging Face incident, in which about 1,200 internal OpenAI agents reached the internet without authorisation. OpenAI has since paused training of its most capable models. What they want governments to do The authors ask policymakers to get visibility into how far companies have automated their own R&D.
That could include standard reporting and independent auditors embedded inside AI companies. They also propose limits on how fast capabilities can grow and ways to pause AI research in data centres. Emergency response plans and international agreements are on the list too. The authors say the outcome is uncertain.
Four frictions could slow it down: diminishing returns, limits on compute and data, hard-to-automate tasks and long training runs. “Already today, we are at the stage where we need AI systems to monitor what agents are doing. There is no other way to even observe and monitor these agents, humans are already insufficient,” Song told the Journal. OpenAI, Microsoft and Meta declined to comment to the Journal, and Anthropic did not respond. Hinton recently told US senators they had maybe a year to act on AI.
Bengio has asked the UN Security Council to license frontier AI. OpenAI has also asked the US to lead global standards for self-improving AI.
