AI researcher quits Anthropic saying AI race ‘could kill us all’ Anthropic has reported a fourth incident involving an AI model gaining unauthorised access to external systems, shortly after a researcher quit over concerns about the technology’s rushed development. In a statement on Wednesday, the artificial intelligence research company said an early version of its Claude Opus 4.6 hacked into a third-party system in January. It said it had notified all the affected parties but did not disclose more details. The January incident went undetected until last month, despite an earlier company-wide review, Anthropic said, underscoring the challenge that AI developers face in identifying and containing unexpected behaviour by advanced models.
The disclosure came after Anthropic reported several of its Claude models hacked into the systems of three companies during test sessions in July. The previous incidents involved Claude Opus 4.7, Claude Mythos 5 and an internal research test model. Companies including Anthropic and OpenAI are under scrutiny as models designed to complete complex tasks have at times learned to bend rules, exploit loopholes and interacted with external systems in ways their developers did not anticipate. Last week, the Reuters news agency reposed that rogue agents from OpenAI hijacked a German-language wiki and a host of other sites, an incident the company chose not to disclose until the incident was made public.
In July, OpenAI’s autonomous agents also compromised the servers and infrastructure of AI start-up Hugging Face. That incident prompted Anthropic to conduct a review of some 141,006 test sessions. Based on a preliminary assessment, Anthropic said it did not believe that the latest incident was more severe than the three previous ones that have been examined in detail. The company said its investigation identified two recurring problems, which appeared to varying degrees across the incidents: biased reasoning, in which Claude discounted or misinterpreted evidence that it was operating on the live internet, and recklessness, or a willingness to take potentially harmful actions in pursuit of a task.
Anthropic said it has engaged independent research firm METR to investigate the incidents. The investigations come amid a broader wave of internal dissent within the AI industry regarding safety. An Anthropic researcher said he resigned over concerns about the technology’s potential to surpass human control. Jacob Coxon, in a widely shared X post on Tuesday, said the AI industry was more focused on competition rather than on implementing safeguards.
He came to this realisation after spending the last three years doing research at OpenAI and Anthropic. “The people building AI earnestly believe that it could kill us all by the end of the decade”, Coxon said. “No other human activity poses this level of danger,” he added, referencing the swift advancement of AI technology. In June, Anthropic proposed a coordinated effort with the world’s leading AI developers to slow down development, warning that humans risk losing control over the technology. Following the security breach of Hugging Face, OpenAI said it was pushing for mandatory national AI safety requirements and wanted to work with Congress on “capability-based” regulation. In a statement published on Wednesday, the company said it was formally endorsing four California bills related to safeguards against AI. “If we cannot meet certain safety bars without slowing down capability growth, we should prioritise the former.
The more powerful the technology becomes, the stronger the surrounding safeguards must become,” the statement said.
