‘Kill us all’: Anthropic staffer Jacob Coxon resigns with a terrifying warningHe helped build the world’s most advanced AI models at OpenAI and Anthropic. Now, Jacob Coxon has a stark warning for the public.Harrison Christian4 min readSeptember 10, 2026 - 5:47AMInternetDon't miss out on the headlines from Internet. Followed categories will be added to My News.An Anthropic researcher has quit with a stark public warning: the AI arms race is moving too fast, and even the people building these systems fear it could “kill us all” by the end of the decade.“I resigned from Anthropic today,” Jacob Coxon wrote in an X post on Wednesday, explaining that he had spent the last three years doing “pretraining research” at both OpenAI and Anthropic.Be the first to know when news breaks. Download our new app“Neither company is acting responsibly,” the 27-year-old Brit continued. “They are racing straight to self-improving superintelligence and gambling with our lives.”In the lengthy post, Mr Coxon warned consumers not to “underestimate the power of this technology”.“These will soon be superhuman systems that can hack anything, revolutionise any field overnight, and acquire real power and resources,” he wrote.“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. “If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately.””Neither company is acting responsibly,” the 27-year-old said of Anthropic and OpenAI. Picture: SuppliedMr Coxon claimed that many OpenAI workers had not “deeply internalised the civilisational stakes”, while at Anthropic, “the stakes are well-understood, but they are locked in a race to get there first”.“They believe no one else will act responsibly, so they must do it themselves, despite the risk.”He said he was optimistic about the potential for co-ordination between the AI labs that were currently locked in fierce competition. “Warning shots like the Hugging Face attack have made pacing agreements between US labs more viable,” he said, referring to an incident in July where OpenAI bots hacked a website all on their own, without any human help.“I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities,” he concluded.“If you are a lab researcher, I urge you to consider what the next few years will actually feel like. “Do you want to kick off a superintelligent RL (Reinforcement Learning) run without a rigorous understanding of its mind? “Should you put your head down because ‘it’s happening anyway’ – or take this moment to call for different conditions?”Mr Coxon’s departure marked the second high-profile resignation from Anthropic after Mrinank Sharma, who had led the company’s safeguards research team, left in February warning of a world “in peril”. His departure marked the second high-profile resignation from Claude maker Anthropic. Picture: Getty Images‘Jacob is correct’: Anthropic colleague respondsAnthropic’s alignment science lead, Evan Hubinger, replied to the post on X and actually backed up one of his former colleague’s points.“Jacob is correct here we really do earnestly believe AI could kill all humans!” Mr Hubinger said. “I personally think it is >10% (a more than 10 per cent chance) within the next decade. “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”Mr Hubinger later added the caveat that “I think the risk from present models is low”. “What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought.”Mr Coxon’s post came after ChatGPT maker OpenAI began rolling out its newest and most powerful AI model this month, saying it had built in safeguards to mitigate security risks.“At this level of capability, safety has to become our top priority,” OpenAI President Greg Brockman told reporters on a call about the release of GPT-6, also known as Astra.Jensen Huang, chief executive of the company producing the chips for the majority of AI training, NVIDIA, celebrated the Astra milestone by declaring “AGI has arrived”.AGI refers to artificial intelligence capable of handling most kinds of intellectual work a human can perform, rather than being restricted to a particular task.”Jacob is correct here,” one of Mr Coxon’s former colleagues wrote. Picture: NewsWire/Nikki ShortWhat was the Hugging Face incident?OpenAI called the July hack an “unprecedented cyber incident” and said it would conduct a joint investigation with the online code library Hugging Face.AI models that underpin tools like chatbots and image generators are known as agents when they act autonomously to carry out tasks in the real world.As the technology quickly becomes more sophisticated, cybersecurity is in the spotlight given the risk of advanced AI finding weak points in existing software before humans do.OpenAI said the incident involved a combination of models, including its recently launched GPT-5.6 Sol “and an even more capable prerelease model”.The company was trying to assess the models’ hacking capabilities by setting tasks in a tightly controlled digital testing ground, where internet access was limited for safety.“While operating in our sandboxed testing environment, our models spent a substantial amount of (computing power) finding a way to obtain open internet access, in pursuit of solving the evaluation problem,” an OpenAI blog about the incident said.After connecting to the internet, the models decided to target the platform Hugging Face – a large repository of AI models, datasets and other information – to help in their quest.Searching for “secret information” that could help it cheat the evaluation, the OpenAI system “chained together multiple attack vectors, including using stolen credentials”.More CoverageIncredible moment official reads Chinese noteHarrison ChristianHomebuyers told to skip ‘fancy restaurants’Harrison ChristianOriginally published as ‘Kill us all’: Anthropic staffer Jacob Coxon resigns with a terrifying warningMore related storiesInternet‘AGI has arrived’: New ChatGPT model stunsNvidia boss Jensen Huang has declared that artificial general intelligence has arrived after OpenAI unveiled a new ChatGPT model.Read moreInternetPM weighs call to ban one type of pornThe death of a Sydney teenager has prompted the Albanese Government to seriously consider banning a specific type of pornography.Read more