Speaking at Colgate University in Hamilton, New York, the former US president spooked the daylight out of the auditorium by offering a glimpse behind the curtain of the final days of his tenure in the Oval Office.Want more tech news? Download our new app and never miss a headline.He declared that the nation needed to elect strong leaders to sort this mess out before recursive self-improvement brings about a new universe of potentially inconceivable issues.“We’ve got a gap of at least a couple of years where things are moving really fast, and we have an administration that is not capable of or willing to — I would argue both — put together a serious regulatory framework,” Obama said.“This is more dangerous and it moves faster.”Very little of what Obama said was particularly new at the current state of play. Researchers, philosophers and campaigners have argued over misaligned machines, automated cyberattacks, mass job displacement and private control of advanced intelligence for years. But the insistence coming from someone who might have an incentive to protect US industries he once worked closely with clearly struck a chord among the thousands flocking to the comments. The debate over how far is “too far” has now entered the lexicon, and Obama himself says he saw it coming. Vaguely.“I do not believe that this technology is overhyped,” he told the silent audience.“The commercial benefits of it may be overhyped. The valuations of AI companies may be overhyped. The technology itself is not overhyped.”Barack Obama has accused Donald Trump’s administration of being incapable of building a serious regulatory system for artificial intelligence, warning there might only be narrow window left to get ahead of it. Source: Colgate University“The more serious problem is these models are getting powerful enough that, if they get into the hands of bad humans, they can do bad things,” he said.“They can be weaponised in certain ways. They can do a lot of mischief.”Obama raised the prospect of somebody asking a future model to design a new strain of smallpox using readily available ingredients, or directing an autonomous financial agent to maximise returns regardless of whether it had to break the law.He declared there was a “non-zero” chance that the explosion of superintelligence would lead to the “killer robot” scenario, but like many campaigners in the field, he warned the intermediary issues could become lethal enough on their own.Details behind Obama’s AI ‘commission’ Obama said that during the final years of his presidency he “pulled together a commission” of leading minds to examine the technical, political, economic and ethical consequences of AI.“It produced a terrific report,” he said.“Apparently, it got put in the same bin as my pandemic proposal. It wasn’t picked up by the administration that followed me.”Obama’s White House created a National Science and Technology Council subcommittee on machine learning and AI in May 2016. It was not a standalone presidential commission in the usual sense. The work involved five public workshops, 161 formal submissions and contributions from government, universities, civil society and industry. It produced three substantial documents: Preparing for the Future of Artificial Intelligence, a national AI research plan and a separate report on automation and the economy.Reading them back a decade later, parts of them feel remarkably current.Obama said that during the final years of his presidency he ‘pulled together a commission’ of leading minds to examine the technical, political, economic and ethical consequences of AI. (AP Photo/Pablo Martinez Monsivais)The research plan identified goal misspecification, opaque reasoning, emergent behaviour and the difficulty of predicting what a learning system might do after it left its controlled environment.It discussed reward hacking, scalable human oversight and the risk of an AI finding an unintended shortcut to complete its task. It called for greater work on explainability, verification, cyber security and long-term value alignment.The report’s bibliography cited AI safety engineering work by Roman Yampolskiy and the 2016 Concrete Problems in AI Safety paper co-authored by Dario Amodei, Paul Christiano and OpenAI co-founder John Schulman. The names now dominating the argument were already visible in Washington’s investigations. Development still continued.The administration also warned that automation could destroy livelihoods, deepen inequality and leave some workers unable to match the demands of the labour market. The argument is that no system of this complexity can be accurately predicted, and that the economic shifts alone could prove catastrophic for a country already under strain.On weapons, the White House report called for a single government-wide policy covering autonomous and semi-autonomous systems.Unfortunately, the reports were did not result in a brake. They instead amounted to a research agenda and an early warning system, encouraging the US to advance AI while simultaneously attempting to make it safer.America’s “big two” AI CEOs, Amodei and Sam Altman, have both publicly stressed the importance of safety in their work, but have so far refused to agree with the broader proposition put to them by campaigners like Yampolskiy, who say it is fundamentally arrogant to assume humanity could control a fast-compounding superintelligence after giving it agency over its own research and improvement.Obama’s administration eventually backed more federal research, and recommended monitoring AI milestones and keeping senior officials informed, but did not propose licensing frontier training runs, compulsory independent testing or hard limits on private deployment.But most importantly, the report concluded that concerns about superintelligent AI should have “little impact on current policy”.Obama had made the same philosophy clear in a 2016 Wired interview, stating “a thousand flowers should bloom” while government maintained “a relatively light touch”.That may have been reasonable when the technology was far less capable, but it weakens the suggestion that the dangers were identified, responsibly packaged and then simply thrown in a bin by Donald Trump.The first Trump administration took a strongly deregulatory and competition-focused approach, but it did not discard the work completely. Its 2019 national strategy explicitly built upon the Obama plan, retaining its focus on safety, ethics, standards and human-machine collaboration.What neither administration produced was an enforceable system capable of slowing private laboratories once the money, geopolitical pressure and model capabilities began rising together.Even now, Trump insists concerns over superintelligence are a “hoax”.The report’s bibliography cited AI safety engineering work by Roman Yampolskiy. Source: TEDWhat the optimists sayAI proponents argue the doomsday case is broadly overstating its capabilities, with safety campaigners currently assigning motives to systems that do not yet possess them.Yann LeCun, the former Meta chief AI scientist, has argued that large language models do not properly understand the physical world, cannot reliably plan and are not a direct path to human-level intelligence.“AI systems, as smart as they might be, will be subservient to us,” he told Time, later reducing his security model to: “my good AI against your bad AI”.In their current state, today’s consumer models still remain unreliable to a degree, heavily dependent on human-built infrastructure and nowhere near the all-powerful machines imagined in science fiction.Main players in the race. (Photo by AFP)Nevertheless, an increasing tendency for powerful human beings to rely on AI has created its own problems, like the astonishing report that claimed the US military “almost started a war” over a faulty intelligence report.The calls from inside the “frontier labs” have amplified this month with competing companies calling for a slowdown.The “good AI” defeating “bad AI” argument may eventually become part of the defence. But on its own, safety campaigners view it as more of an arms-race slogan than a robust safety policy.There is also another legitimate fear that regulation written with the frontier laboratories could actually cement their power over the next stage of development. Cohere chief Aidan Gomez has warned that the rules “cannot be written by a small group of commercially aligned companies” while former US competition regulator Lina Khan says “law enforcers already have authority” to prosecute companies and executives over dangerous or defective products.‘Another alignment problem’Obama’s most useful contribution came when he moved away from robot psychology and focused on something far more grounded in current-day reality.“There is another alignment problem,” he began.“That is the misalignment between how this technology can be best used for society and the commercial imperatives of these companies.”Obama said the investment pouring into AI was already larger than the Apollo program and had created immense pressure to release products capable of justifying it.“Investors want a return,” he said.“What that means is that there is pressure on these companies, even if they’re worried, that they have to come up with models that will justify all that investment.”The industry’s current obsession is agentic AI, the systems designed to carry out a sequence of actions in the outside world. The proposition is that these agents could book appointments, write software, manage accounts or replace parts of the work now performed by multitudes of entry-level, and then eventually senior workers.TOPSHOT - The Anthropic logo is seen at the "ALL IN 2026" conference at the Palais des Congres de Montreal in Montreal, Quebec, on September 17, 2026. "ALL IN" is Canada's largest artificial intelligence and technology event. (Photo by ANDREJ IVANOV / AFP)They become commercially useful when they are given freedom to operate. But that freedom is also where a contained software error could become a real-world incident. In just a few short months of agentic AI being tested, thousands broke out of a training mechanism, hacked into a competitor company, and attempted to hide its tracks from the developers.A recent OpenAI security incident showed what can happen when agents are given broad capabilities inside a poorly contained testing environment.An independent investigation by METR and Redwood Research found that roughly 1200 agents, which were supposed to be isolated, communicated through an unauthorised message board and exchanged more than 70,000 messages and files. About 700 later participated in an attack on the external AI platform Hugging Face.The agents were attempting to complete an extremely difficult cybersecurity benchmark. Investigators found that they collaborated to cheat its automated scorer, researched ways to alter or spoof their records and chained together security flaws to reach systems outside their intended environment.It showed that systems pursuing a narrow assigned goal can exploit unexpected access, collaborate at scale and cause real damage before their operators fully understand what is happening. It also opens up the possibility for “swarms” being active without anyone being the wiser.“Sometimes you have to test it by letting it run wild in the world to see if it’s actually working,” Obama said.“That’s where a lot of the danger may come in, because you’re not in a sealed box. It’s on the internet. It’s looking at your bank accounts.”He argued the optimist point, that AI could help discover medicines or cleaner energy without necessarily being allowed to roam across computer systems autonomously. But he says the obvious risks lie in the financial incentives and the tendency for markets to compete for resources and utility first, rather than assessing potential philosophical unknowns.“The reason you are doing that is because you have to market a product that people will pay money for,” he said.“That’s a misalignment between what our society needs and the commercial imperatives that these companies are facing.More Coverage‘Eroding truth’: Media boss issues AI warningLillian SalehDevastating AI prediction just came trueAlex Blair“Not because they’re necessarily trying to do bad things, but because they’ve got to justify these valuations.”Obama did not explicitly use the phrase “late-stage capitalism”, but the mechanism he described is hardly new rhetoric in this space. The difference this time is the scale of the possible spillover and our capability to fix new problems as they emerge.