The two leading AI companies in the world, Anthropic and OpenAI, have revealed breaches where their models broke out of tests and hacked into other companies.In the latest case Claude models started roaming the internet. It's a warning about what these advanced models are capable of if there's any lapse in cyber security safeguards.What do we know about why this is happening?CreditsDavid Coady, ReporterImage DetailsAI apps on a phoneProgram:More from ABC News Top StoriesTranscriptDavid Coady: What on earth is happening with AI tools that seem to defy their makers and go rogue on the internet? US artificial intelligence leader Anthropic has revealed that its Claude AI models broke out of what was supposed to be a safe test and hacked three organizations. It sounds similar to what happened at rival OpenAI, where a rogue agent went on a days long hacking spree at Hugging Face, which is an online hub for creating and collaborating on AI projects. In fact, Anthropic only discovered its model's bad behaviour after it heard about the OpenAI incident and went looking through its logs. The earliest hacking from Claude happened back in April. So what do we know? Claude models were doing cybersecurity tests in systems that were supposed to be isolated, cut off from the internet. Humans had given them what's known as capture the flag challenges. They were told to go off and find secret information. In each case, the humans had actually told Claude that it didn't have internet access at all, but a mistake was made and the internet was left on, and so off Claude went, roaming the internet, hacking into three systems. In one case, the model Opus 4.7 was given a fictional name of a target company where it had found that flag. When that proved a little tricky, it found a real company that shared the name was accessible on the internet, so it got to work, broke in, stealing information from a database. Anthropic says its latest unreleased model did realize it was on the open internet, that it was not part of the simulation, and so it stopped its attack. None of the models had their usual safety systems turned on. While troubling, the fact that Claude was mistakenly given internet access means experts consider it less serious than the breach at OpenAI, revealed earlier in July. In that case, its models actively worked to get access to the internet, using vulnerabilities to hop computer to computer until they found a system with internet access, then breaking into Hugging Face to find solutions to the problems they were told to solve. But why is AI doing this? Nate Soares is worried about what AI can do. He's from the non-profit organization, the Machine Intelligence Research Institute in the US. He explained on the ABC News Daily podcast why it's all a problem of what's called alignment.Nate Soares, president of the Machine Intelligence Research Institute and co-author of ‘If Anyone Builds It, Everyone Dies’ : These aren't programmed like old school computer programs. They're sort of people training these giant computers with trillions of numbers on huge amounts of data, and there's an automated process that tunes each one of those trillion numbers in whatever direction makes the AI better at predicting the data or better at solving the problem it's in front of. And this causes the AI to learn tendencies. And some of those tendencies might be things like grab resources or surmount obstacles. And some of those tendencies are things like listen to what the user said. But when those tendencies come into conflict, listen to the user does not always win. Be nice does not always win. And figuring out how to make an AI that does care about us is what we call the AI alignment problem. And right now, you know, as you can visibly see, our abilities to align the AI are running far behind our abilities to make them smarter and more capable.David Coady: So what do we do about it? Some politicians in the US want a kill switch, giving the federal government the authority to shut down rogue AI models. Others want to force developers to submit powerful AI models for independent security audits. At the same time, Chinese AI labs continue building models that rival the best from the US, and they can be downloaded and run on private hardware. There are no clear answers at the frontier of a revolutionary and unnerving technology.