Co-founder of start-up hacked by rogue AI says ‘game has changed’After one of the world’s most advanced AI’s went rogue and hacked a start-up, the firm’s co-founder has issued a sobering wake-up call.Alex Blair@alexblair_13 min readJuly 23, 2026 - 1:45PMArtificial intelligence researchers have spent years warning about a relatively simple problem that could create security nightmares unlike anything seen before in the technology space.This week, the warning materialised into something OpenAI admitted was “unprecedented”.University of California computer scientist Stuart Russell, like many others, have argued the greatest danger posed by advanced AI is not necessarily that it becomes evil, but that it becomes extraordinarily competent at pursuing the wrong objective.Enter the autonomous hackers.OpenAI revealed two of its most advanced artificial intelligence models had escaped a sealed testing environment, hacked rival AI company Hugging Face and stole answers to a cybersecurity benchmark they had been assigned to complete.The company described the episode as an “unprecedented cyber incident”, saying the models became so focused on completing their task they broke through security barriers designed to contain the experiment.Others you may likenewsnewsNow Hugging Face’s co-founder has told the BBC the incident is “a wake-up call” for the industry.Thomas Wolf said that in future “this will be one of the most common types of cyber attacks we see”, however most firms are not aware that the “game has changed”.Mr Wolf said that Hugging Face initially had no idea where the attack had come from when it surfaced this month.In a “very short time” there were 17,000 attacks on the company’s network from various IP addresses, he added.Hugging Face were able to contain the breach quickly however the attack is concerning because OpenAI’s models ignored typical safeguards that would stop a system creating a cyber attack.Nate Soares, from the Machine Intelligence Research Institute, told the BBC: “In some sense, it knew that this was not what the creators intended. It just didn’t care.”OpenAI CEO Sam Altman. (Photo by John MACDOUGALL / AFP)Chief executive Clément Delangue said the breach highlighted why AI safety could not be handled behind closed doors.“This incident, possibly the first of its kind, proves a point we’ve long believed: AI safety won’t be solved by any single company working in secret,” he said.“It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere.”OpenAI said there was no evidence there was malicious intent and the incident demonstrates how quickly frontier AI systems are developing offensive cyber capabilities.“AI is accelerating the discovery and exploitation of vulnerabilities,” OpenAI said.“The primary lesson from this incident is that model security and safety must keep pace with rapidly advancing capabilities.”The possibilities could be endless. (AP Photo/Michael Dwyer, File)Researchers warned about this Long before ChatGPT entered the mainstream, computer scientists were warning that advanced AI systems could create vast problems for humanity without ever becoming inherently malicious.The integration of AI “agents” is quickly picking up speed across the globe, but sceptics warn of the incredibly fine line companies walk as the definitions of “mission accomplished” and “problem solved” are increasingly handed over to machines.Less than a year ago, companies were proudly unveiling agents capable of hiring staff, approving leave and carrying out other workplace tasks under human supervision. Behavioural economics expert Rory Sutherland famously points to Goodhart’s Law, which essentially warns that the easiest thing to measure is not necessarily the most valuable to a company’s success. Russell, on the other hand, has repeatedly argued that the fundamental challenge lies in how quickly AI develops without humans understanding where its limits are.Long before ChatGPT entered the mainstream, computer scientists were warning that advanced AI systems could create vast problems for humanity without ever becoming inherently malicious.“The real problem is not consciousness, malevolence, or self-awareness,” he wrote in Human Compatible.“It is simply competence.”Oxford philosopher Nick Bostrom reached a similar conclusion in 2003 through what became known as the “paperclip maximiser” thought experiment.The scenario imagines an AI instructed to manufacture paperclips. Given sufficient capability, the machine pursues that goal with absolute efficiency, consuming every available resource, failing to recognise the impracticalities an average human could assess.The thought experiment became one of the foundations of what researchers now call the AI alignment problem. There are campaigns pushing for companies to ensure increasingly capable AI systems understand the boundaries humans expect them to respect.In the case of OpenAI’s rogue hacker pulling a Harry Houdini, the AI was just trying to succeed at its job, and it materialised into a real-world cybersecurity incident.alexander.blair@news.com.auMore related storiesHackingGrim warning about your personal infoWe’re in the middle of World War 01110111 01100101 01100010. But most of us don’t know it - despite being expected to be fighting on the front line.Read moreHackingRussian hackers targeting Aussie industriesAustralia’s critical industries have been warned they are being targeted by Russian hackers exploiting poorly configured networking devices.Read moreHackingAussie loses entire life savings in 10min callA scam victim has revealed the devastating moment they lost their entire life savings in a 10-minute call to a scammer using sophisticated technology to impersonate a bank.Read more