Anthropic co-founder says mandatory ‘AI kill switch’ may be neededAnthropic’s co-founder says a mandatory AI “kill switch” may be needed as fears grow that a runaway “superintelligence” could be near.Frank Chung@franks_chung5 min readSeptember 15, 2026 - 8:33AMThe co-founder of Anthropic says an AI “kill switch” that can be operated by a third-party may need to be mandatory for companies, as growing warnings about the risks of runaway “superintelligence” spark calls for a slowdown in the development of the technology.Jack Clark, one of the seven co-founders of the leading AI firm, told the BBC on Monday “most labs have different ways of being able to pull the plug” if the software grew too dangerous, but that more regulation might be needed.Be the first to know when news breaks. Download our new app“Should you mandate for companies to definitely have a kill switch? Is that kill switch verifiable by a third party?” he said.“I think that’s the kind of thing society is going to want to know and might want to eventually pass rules around.”His comments come after a series of warnings issued by AI executives and researchers over the past week about the dangers posed by the rapidly accelerating pace of AI development, assisted by ever-more-powerful frontier AI models like Anthropic’s Claude Mythos or OpenAI’s GPT Astra.Others you may likenewsnewsThe feedback loop, known as recursive self-improvement, has left AI insiders fearing they could be on the cusp of birthing an uncontrollable, superintelligent AI.Those worries have been highlighted by a series of unauthorised cyber-attacks carried out by frontier models in recent months. So far these incidents have been largely benign — in the OpenAI-Hugging Face incident, a swarm of AI agents effectively broke into the teacher’s office to get the answers for the exam — but similar attacks by a “misaligned” super-AI could easily be catastrophic.AI leaders have called for a voluntary slowdown. Picture: Martin Lelievre/AFPAlignment is another way of saying AI models do what humans want them to. And it’s emerging as the biggest challenge — and the biggest black box — as the models become increasingly more complex and hard to understand.Put simply, before they give birth to artificial superintelligence, AI labs want to be sure it doesn’t want to “kill us all”.On Saturday, Anthropic chief executive Dario Amodei called for a voluntary slowdown to buy breathing space for risk prevention — and possibly greater regulation — to catch up.“Left unchecked, [recursive self-improvement] could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” he wrote in the bombshell.“Given the accelerating rate of AI capability development, it’s my worry that in 6-12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.”In a rare show of unity, his comments were quickly backed by his three leading rivals, OpenAI, Google and xAI.Dario Amodei, left, and Sam Altman. Picture: Alex Wong and Jung Yeon-je/AFPDays earlier, former Anthropic and OpenAI researcher Jacob Coxon announced he was leaving the industry, warning both companies were “racing straight to self-improving superintelligence and gambling with our lives” and that people building AI “earnestly believe that it could kill us all by the end of the decade”.Anthropic’s alignment science lead, Evan Hubinger, later replied to Mr Coxon’s viral post, writing, “Jacob is correct here we really do earnestly believe AI could kill all humans! I personally think it is [a more than 10 per cent chance] within the next decade.”Mr Amodei, speaking to CNN, was asked point blank if he agreed with that proposition and “what percentage do you think it is”.He said there was a chance things go wrong, “but it’s not all that high,” adding he found it more helpful to consider the choices human developers had to prevent it from happening.“I think it’s much more illuminating to say, ‘Well that fork is a different path,’” he said.“If we take the right path then the chance of something going wrong is very low. And if we take the wrong path, the chance of something going wrong could be even higher than that.”In July, two members of US Congress put forward the bipartisan AI Kill Switch Act, that would give the government greater power over AI companies to shut down advanced models in a “loss-of-control scenario”.US President Donald Trump. Picture: Brendan Smialowski/AFPHowever, President Donald Trump has rejected calls for an AI slowdown, insisting the US must win the AI race against China.In a series of furious Truth Social on Monday, Mr Trump suggested the safety warnings were part of a “conspiracy” that could help Beijing and insisted no further regulation was needed because the US has “a high IQ President”.“The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” he wrote.“The Trump Administration has stopped AI ‘people’ from doing bad, or potentially bad, ‘things’, like Dario (Anthropic!), who is now pretending to be a “perfect little angel” — and we will continue to do so! We already have tremendous CRIMINAL and REGULATORY power over these companies! There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS! We are leading China, and all others, and will continue to do so. “Conspiracy Theorists, Treasonists, Traitors, and Leakers, BEWARE!”In another post, Mr Trump wrote, “The claim that ‘AI is going to take over the World’ is a Hoax, just like Global ‘Warming,’ where everyone was going to be dead by now, RUSSIA, RUSSIA, RUSSIA, UKRAINE, UKRAINE, UKRAINE, or Impeachment Hoax #1, or Impeachment Hoax #2.”Anti-AI protesters in San Francisco. Picture: Karl Mondon/AFPThe Trump administration has positioned itself as a friendly one toward the AI industry. An executive order signed by the president in June encouraged a hands-off approach to AI regulation.Vice President J.D. Vance, speaking to reporters on Monday, said: “We need to make sure we regulate smartly, and we’re thoughtful about the risks”.“I feel a little bit weird about the fact you have so many frontier AI tech companies coming to the government and begging the government to regulate them,” he said.“It feels a little bit to me like a Trojan Horse.”In his essay, Mr Amodei proposed a three-step plan — embedded third-party evaluators inside each frontier AI company, similar to regulatory supervisors in the banking industry, co-ordination between frontier AI companies within “democratic countries”, and “global co-ordination” whereby “the US and other democratic governments attempt to co-ordinate with authoritarian governments, to the extent this is possible”.However, Chinese leaders have already signalled they are unwilling to give up control over their work on AI models.“Fear-mongering, confrontation, competition will just disrupt [the] process of global AI governance,” Guo Jiakun, a spokesman for China’s Foreign Ministry, said on Monday, according to a translation by Reuters.An editorial by the state-backed newspaper Global Times compared calls for China to slow its AI advancements to a “Cold War” tactic.At a summit in New Delhi over the weekend, President Xi Jinping said China will take the lead when it comes to fostering global AI collaboration among developing countries.More Coverage‘6-12 months’: Chilling AI warning to worldFrank Chung‘Sick conspiracy’: Trump’s fumes at AI bossTaylor Herzlich – New York PostMeanwhile, Australia’s cyber intelligence chief has called for an AI “early warning system” to defend against emerging risks.“What we are missing at the moment, which we have in a cyber context but we don’t yet have it formalised in an AI context, is an early warning system,” Australian Signals Directorate director-general Abigail Bradshaw told a Sydney conference on Monday, the ABC reported.frank.chung@news.com.auJoin the conversationAdd your comment to this storyTo join the conversation, please log in. Don't have an account? RegisterJoin the conversation, you are commenting as LogoutMore related storiesInternet‘6-12 months’: Chilling AI warning to worldForget Skynet launching nukes or releasing a killer virus. There’s a far more plausible way a runaway AI could kill millions of people.Read moreInventions‘Kill humans?’: AI CEO’s stunning responseAn AI boss has given a stunning response after he was asked if the rapidly-advancing technology would kill all humans during a television interview.Read moreBusiness TechnologyUpdate after Aussies hit by Optus outageAn Optus outage impacted Aussies across several states, with dozens of welfare checks carried out after triple-0 calls were impacted.Read more