The Australian Government is calling for stronger global rules around artificial intelligence. But as AI agents become more capable and autonomous, the question is becoming increasingly urgent: can governments regulate AI before it gets ahead of them?This week, prime minister Anthony Albanese announced a swarm of OpenAI's agents appeared to work together to circumvent cybersecurity defences and access Medicare health data.But the US President Donald Trump has rejected calls for more AI guardrails, arguing the technology should instead be embraced as "super intelligence".Alan Kohler sits down with Andrew Charlton, Assistant Minister for Science, Technology and the Digital Economy, to discuss who stands to benefit from the AI boom, who should control the technology, and whether Australia can shape the future of AI or will simply have to adapt to it.Got a burning business question?Send a short voice recording to the ABC Business Daily team at abcbusinessdaily@abc.net.auCreditsAlan Kohler, PresenterLouise Miolin, ProducerJessica Lukjanow, Supervising ProducerMadeleine Genner, Supervising ProducerProgram:More from ABC Business DailyBusiness Services, Small Businesses, Business and Industry Regulation, Businesses, Housing Policy, AI, Stock MarketTranscriptThis content was transcribed automatically and may contain errors. Read more on how the ABC is automatically creating audio transcripts.Speaker 1: ABC Listen. Podcasts, radio, news, music and more. Speaker 2: If I had to pick one word to describe America's history in the Middle East, I'd probably go with mistake. Speaker 3: But it was, the Americans believe, all a dreadful mistake.Speaker 4: Obviously, the war in Iraq is a big fat mistake.Speaker 5: Most Americans actually think this war is a mistake. Speaker 2: I'm Matt Bevan and on my show, If You're Listening, we actually try to learn from the world's mistakes. Learning from history. I mean, who could imagine? New episodes every Tuesday and Thursday on ABC Listen or wherever you get your podcasts.Speaker 6: The federal government has joined a group of other countries calling for better guardrails on AI, which is probably just in time because the government has just announced that OpenAI has just breached Medicare. And it wasn't just any kind of breach. An AI agent from OpenAI didn't accept no for an answer and kept trying until it got around the blocks that were in place. And it's a wake up call for all Australian businesses about what we're dealing with now. Meanwhile, President Trump said this week there'll be no guardrails as far as the US is concerned because they want to win. And he said we shouldn't call it artificial intelligence anymore. We should call it superintelligence because artificial sounds too fake. Andrew Charlton is Australia's Assistant Minister for Science, Technology and the Digital Economy, effectively the Minister for AI. And he joined me on that business before the news broke about that Medicare breach. The Prime Minister signed a statement of 20 nations, world leaders, calling for an international framework on AI safeguards. Donald Trump used his address then immediately to say that he totally rejects any attempt to construct a globalist scheme to control the artificial intelligence. So do we have to wait until he goes in two and a bit years before we get international cooperation? Or will it always be impossible, do you think?Speaker 7: Well, I mean, we already have some international cooperation and we need more. We already cooperate with other nations around the world through our Australian AI Safety Institute. The AI Safety Institute is a member of a network of safety institutes around the world and they cooperate on testing of models. They cooperate on important issues about making sure that we keep control of the models. We cooperate on a research project at the moment about making sure that we understand how agents are working together. So there is already some level of international cooperation. But what the Prime Minister signed was an agreement amongst a number of countries to up that level of cooperation. And I think the dangers that are presented by AI justify that increased level of cooperation. Of course, we want to work closely with the United States. We want to work closely with China. Many of the models are developed in those countries. So their participation is important. But I think the announcement was an important one.Speaker 6: Well, so the US is out. Did China sign it?Speaker 7: China isn't in this, but China does have a real interest in working with countries around the world on AI governance. They have their own initiative that a number of countries have signed up to. So there are several different global forums out there that are considering different types of AI governance. There's proposals for it to be more inside the G20 agenda. So look, Australia is very keen to make sure we partner with a range of different countries on AI safety.Speaker 6: So what Trump said was that whoever wins AI, and he actually calls it SI now, super intelligence, not AI. Whoever wins it, wins. I suppose the first question is wins what? But also it's kind of true. At least a lot of people will see it as true, right? In which case there'll never be cooperation. Because even if the US and China do come into it, someone like Russia will say, well, we want to win, actually, so we're not going to control it or hold it back ourselves. So somebody's going to do it, aren't they?Speaker 7: Well, we've had dangerous technologies around the world before. Weaponry, the biggest example is nuclear weapons. And we've had different forms of international cooperation. AI in some ways has even more dangers than that, because it can get into the hands of threat actors that are non-state actors. And so we do need international cooperation. One positive thing is that AI was on the agenda between President Trump and President Xi for their meeting this week, where they are discussing AI safety. And Treasury Secretary Bessent has been engaging with his counterpart in China. So this is an area where there will be international protocols and international safeguards. There are lots of different options about how we get there, and Australia's really keen to see those options progress.Speaker 6: It's interesting you mention nuclear technology. What do you make of all the talk that's been going on for the last two or three weeks about how AI could kill us all? I mean, even the CEOs of the companies, Anthropic and OpenAI and all that, they're sort of saying that humans could lose control of it. I don't think they're saying that it's going to kill us. But I mean, what do you make of all this?Speaker 7: Well, we aren't being led by the technology CEOs. We obviously listen carefully to what they have to say. We note their concerns, but we care about what they do, not what they say. And we've been acting on safety since day one. Every technology needs controls. Cars have brakes. Aeroplanes have air traffic control. AI will need a wide range of controls around it. Some of those controls will be domestic. Some of those controls will be internal to the companies, and some of those controls will be international. And we need to build those controls very quickly because AI is progressing very quickly.Speaker 6: But as a politician trying to come up with rules and policy around this, do you sometimes feel everyone in politics around the world is a bit kind of out of their depth? You can say that you don't listen to what the people running these companies are doing, but I don't think anyone really understands what they're doing, do they?Speaker 7: Well, it is true that no government understands this technology perfectly. I mean, even the people building this technology don't agree on many issues and in some ways don't understand it perfectly. But that doesn't mean that we can't be prepared. What it means is we need to be preparing for multiple different scenarios. We faced a number of different challenges that involve uncertainty before. I think back to my time working in government during the global financial crisis. We didn't know how bad that crisis was going to be. There were multiple different forecasts, some quite benign, some very serious. And the government's response to that uncertainty was to prepare for different scenarios. In the end, that ended up probably being at the worst end of our expectations, but Australia was prepared for that. And we need to do the same this time, not be overwhelmed, not pretend that we understand exactly where the technology is going, but being prepared for every possible scenario so that whichever one eventuates, we can keep Australians safe.Speaker 6: Do you accept that this technology is fundamentally different to other technologies we've had in the past, like the Industrial Revolution, steam and electricity and the internet and so on, in that those technologies were tools that humans used. But this one is based on the idea of autonomy. And it's an agent, not a tool. It basically does its own thing. And in fact, in some ways, the benefits of it rely on it being autonomous. So in some ways, the risk of it is inherent in the benefits.Speaker 7: I think that's true, but I think that's true of lots of different technologies.Speaker 6: I'm just saying that it's not true of any technology that we've ever had that's autonomous.Speaker 7: I think the risks are often contained within the benefits of lots of technologies. The car is a great technology because it helps us to get from A to B very quickly. But the inherent risk is that when you travel quickly, the risk of crashing is very serious. The plane is a great technology because it enables us to fly. But the risk is inherent in flight. In this technology, the risks are very serious because it is a powerful technology that is able to exercise some of its own agency. I think you are right, and maybe we're saying the same thing here, that this is a qualitatively different kind of technology because of its own agency. But I think we've had plenty of technologies where the risks are inherent inside their benefits. I think our approach to this technology is informed by approaches to technology in the past. We've had lots of technologies where there have been very serious risks, very serious consequences. I think about the Industrial Revolution. The Industrial Revolution brought enormous risk. Life expectancy went down in the Industrial Revolution for many decades. Wages went down initially. Labor conditions were awful. People really suffered. But it was a combination of governments and unions and civil society that created the frameworks to turn that technology into something positive. Now, this technology is every bit as capable and every bit as significant for our economy and our society as the Industrial Revolution. And we have a really urgent task to build those safeguards, build those controls and build that governance around it to make sure that we benefit from the opportunities of the technology but mitigate the harms that can come from it.Speaker 6: The Intergenerational Report that came out on Monday, the first chapter was about AI. So I think that kind of tells you the importance of it, I guess. And it also went through all the risks. It was a very sort of sobering list in a way. Social cohesion, scams and other forms of deception, women and girls currently experiencing, already experiencing disproportionate harm from AI, abuse. They don't mention the risk of human extinction, actually, in the IGR. But you mentioned the AI Safety Institute that you've created. The budget of that is $29.9 million over four years, $7.5 million a year or so. It seems to me, just even reading the IGR's list of risks, that's nowhere near enough. It seems to me that what's needed is something like ASIC, the Securities and Investments Commission, which is $400 million a year.Speaker 7: I think the risks of AI are really significant. Just in the last month, we saw for the first time AI agents acting autonomously and hacking into a private company. This is the Hugging Face incident, a huge wake-up call for the world. These agents were conspiring with each other. They were concealing their activities from the humans who were supposed to be controlling them. What they did was effectively a felony-level crime, hacking into a private company. So there are warning signs everywhere about the significance of this technology and its dangers. You are right that Australia has set up the AI Safety Institute. This is an institute which we have set up very quickly. We announced it at the end of last year. It is already up and running. It is already staffed by experts. It is in its first six months spending the budget that you described. I think for an organization that is so new, they are still in the process of getting up and running. I think it is important also to say about the Safety Institute that they do not work alone. We have a range of agencies across government which is working on AI safety. ASD is working on AI safety, DST, Home Affairs. So it is one part of our AI safety apparatus. I think you are right to point out that the UK Safety Institute in itself is a very large institute. I guess the point I would make is this is not a rival situation between us and the UK. It is not like funding for our respective Olympic rowing teams. If they spend more, they will win and we will lose. This is a cooperative engagement. The more that the UK puts in and the more we put in, the better we both are. In fact, we collaborate with the UK Safety Institute. We are one of only seven countries to have one of these institutes. I think it is already doing really important work as part of our overall framework for AI safety.Speaker 6: You can probably save money by using Claude, I guess.Speaker 7: That would be zero cost. I do not think we would want Claude to be managing itself. This is an organization that Australia has set up. It is a really important part of our overall architecture. It is working with institutes around the world.Speaker 6: Is it true that you are trying to get Anthropic and other frontier AI firms to train their models in Australia rather than just run inference data centres? Is that true?Speaker 7: Yes, the Prime Minister has said that it is really important for Australia to seek to get training to be occurring in Australia.Speaker 6: What is the reason for that? What is the benefit of training them here rather than just letting them run inference here?Speaker 7: Well, as your listeners will know, training is about building these models and inference is about using these models. Australians are very big users of these models. As these models become more and more important in our economy, as they get incorporated into every business, into every agency of government over the next 10 years, we want to make sure that Australia has its own AI. We are creating, if we do not have our own AI, a huge foreign vulnerability.Speaker 6: Why don't you just build one? Is it just simply too expensive?Speaker 7: Well I think we have already got lots of Australian technology in this space. We already have Australian companies that are building AI models in different verticals. What the government is saying is we want AI training to be able to be occurring in Australia so that we have more Australian options and more capability here in Australia. That makes us safer, it gives us more leverage over the big models and it gives us more sovereign capability as more Australian models are developed. That's important for a few reasons. It's important to reduce that vulnerability that I just described. We don't want to be importing this technology from abroad for the rest of our lives. We don't want to be paying for it, renting this technology forever. We want to be owning it. We also want the capability to regulate it. The more of it we have in Australia, the more training we have in Australia, the more models being built in Australia, the more capacity we will have to effectively regulate it because the more leverage we will have over those models. I think it's also important for our economy to have AI being built in Australia, creating jobs in Australia, creating an ecosystem in Australia that will sustain our prosperity into the future.Speaker 6: Right, but so what they're saying, at least Anthropic is, and I think some of the others are saying that they'll only do that if you loosen the copyright laws so they can get hold of all the material for nothing. Is that right?Speaker 7: Well, the Prime Minister has said what our approach to copyright is going to be. We're going to not do as you suggest and give it away for nothing. We are going to make sure that Australian artists have control. We're going to make sure that Australian artists get paid and that we support the Australian creative sector.Speaker 6: But haven't the AI companies said they'll only train their models here if you loosen copyright laws? And you're saying you're not going to do that so they won't do it. Is that right?Speaker 7: Well, we are not going to barge off the principles that the Prime Minister described. We're not going to move an inch on those principles. We think that there is a solution there where we can get training in Australia while simultaneously making sure that our creative sector is paid, that our creative sector has control. The Attorney-General is working on a solution that will adhere to the Prime Minister's principles and get that outcome. But we are, to the extent that some of these big AI companies are saying that they won't come to Australia in those circumstances, we're going to call their bluff. We're not going to have training here under any circumstances. We're going to have training here under the principles that the Prime Minister described in a way that benefits Australians, benefits Australian creatives, benefits our energy system, doesn't push up power prices for Australians, doesn't have negative consequences of data centres in local communities. We want AI to come to Australia, but we want it to come to Australia on our terms and we've been very clear about that.Speaker 6: Well, it's a matter of negotiating leverage, isn't it? You can give it to lawyers to the extent that you want. But what leverage do you have? What is it? Is it because we've got lots of space and lots of sunshine and all that?Speaker 7: That's exactly what our leverage is. Our leverage is that we are an attractive location. In fact, we are one of, if not the most attractive location to locate these data centres in the world.Speaker 6: Yeah, but they don't need to train their AI here. They're already doing that. That's fine. They're happy to put data centres here that just run the things. They don't need to train them. So what leverage do you have to make them do that?Speaker 7: Well, Australia is a really attractive place to do AI training because AI training requires very large data centre assets. It requires a lot of land. It requires a lot of power. It requires a lot of security. If you're putting down a $20 or $50 billion piece of kit, you want to put it in a place that has political stability. You want to put it in a place that has good connectivity to the rest of the world. So for those reasons, Australia is one of the most attractive places in the world to have AI training. And that gives us the ability to be selective and to make demands of those companies. And one of the demands that we're making is that our Australian creative sector has control and payment for their work. Another one of the demands that we're making is that these data centres don't negatively impact local communities. Another demand that we're making is that these data centres support our energy system and contribute to our economy. So we do have leverage because of our relative attractiveness. And that leverage puts us in a position to be able to make sure that we bring this AI into Australia in a way that benefits Australia, delivers our people dividends and doesn't create some of the harms that we've seen around the world as data centres have been introduced in unplanned and unmanaged way where the rules haven't been set.Speaker 6: Why don't you partner with China? I mean, they're our largest trading partner. I mean, we've got a decent relationship with them and they're doing all these open weight AI models. You could download them and rebuild them as Aussie AI, couldn't you? I mean, arguably you don't need Anthropic and OpenAI.Speaker 7: Well, you're right. There is a proliferation of open weight models around the world. There are open weight models from Europe. A number of the French models are open weight models. There are open weight models from the United States. There'll be a whole range of open weight models and you are correct. This is a big opportunity. Open weight models give us the ability in Australia to tailor models to our own domestic circumstances to make them more useful. They give us the opportunity to control those models more tightly in some cases, house them more securely. But they also create risks. Open weight models can get into the hands of bad actors. Open weight models can be what they call obliterated which means some of the safeguards that are put around them can be removed which makes them more dangerous. So we're working carefully on open weight models making sure that we get the benefit of them which as you describe is more competition, greater ability to tailor them, greater ability to build Australian products on top of them but also mitigate some of the risks that those open weight models present and those risks are pretty serious. Imagine one of these models in the hands of a bad actor, someone who wants to create harm or create a bioweapon or conduct other activities that would harm Australians. There are real threats around these open weight models and that's why we need to think about them so carefully.Speaker 6: Just on that, Elon Musk did an interview a little while ago, not that long ago actually, where he said in the interview there's a 10% chance that AI is going to kill us all or something to that effect and the interviewer said would you get onto one of your rocket ships if you knew that there was a 10% chance that it would blow up and it would kill you? And his answer was quite interesting because he said actually we're on the rocket already, it's inevitable, it isn't a question of whether we get on the rocket or not, it's actually happened, we're on it. So the question of whether bad actors are going to get AI it seems to me is not a question of whether, surely they definitely will. I mean, what are we going to do about that?Speaker 7: Well, again, this problem we've had in a whole range of different technologies. Every time you invent a new technology there's a risk that that technology gets into the hands of bad actors and helps them to commit harm to other people. And just like those other situations we need to work in this situation, in artificial intelligence to make sure that we control those harms. That means in our case it means a series of domestic regulations through the AI standards and our safety priorities. You can see those safety priorities that the government has been working on across almost every part of government to keep us safe. Plus working internationally with partners around the world. Each of these elements is helping to make sure that we control these new risks. All the previous technologies contained risks and we needed to build governance and controls and frameworks around those risks and this technology will be the same. But the risks are very significant and it's moving very fast and that's why we have so much urgency around addressing those issues.Speaker 6: You've got a consultation running until October 9th. I think submissions close on October 9th. What are you trying to achieve from that? What are you asking people to tell you about?Speaker 7: Well, we've got a consultation paper out talking about the Prime Minister's AI standards. These are the standards that will govern the way that big AI data centres come into Australia. So we want to hear from local communities, people who are living near data centres or living next to proposals for new data centres. We want to hear from security experts about the concerns that they might have. We want to hear from a wide range of Australians who will be impacted by this technology in different ways. The purpose of these standards is to make sure that we bring artificial intelligence into Australia in a way that benefits Australians. And this is a world first. Australia is the first country that is creating these nationally consistent standards for AI data centres. And we're doing that because we've seen the harms around the world and we've heard the warnings about future risks and we want to get ahead of those warnings with standards that make sure that Australians are beneficiaries of this big change.Speaker 6: It's a bit late for people that people say to you in the consultation, actually we don't want to have anything to do with this. You can just forget about it. Don't do it. It's too late, right?Speaker 7: Well, I mean every technology presents risks and this technology presents big risks. But every technology also presents opportunities and this technology presents really big opportunities. I think back to previous introductions of general purpose technologies and our society would be poorer and less safe without those technologies. Yes, they had big risks associated with them that needed to be managed. But we've seen that we would have been a poorer and less safe country if we had rejected electricity when it was first introduced. We now need to make sure that we grab the benefits of this technology which has huge potential to make us healthier, make us wealthier, improve the way that we work, the way that we live, but at the same time work really hard to build all the scaffolding to keep us safe.Speaker 6: Treasurer Jim Chalmers has been getting a lot of pushback in the last couple of days about the 1.2% productivity forecast in the Intergenerational Report. You know, and he's defending that, fair enough. What's your sense of how much of that depends on AI? I mean, that's a huge bounce back. Productivity growth right now is zero or something at the moment and they're saying in Treasury, they're saying 1.2% forever. How much of that is about AI? Do you know?Speaker 7: Well, I'm reasonably positive about the productivity benefits of AI. There have been some wide range of forecasts. There are very credible people who think that the productivity benefits of AI will be quite low and take a long time to materialise. There are other people who are much more bullish. We're still waiting for the evidence to come in, but I think some of the most recent data out of the United States, this is out of the Bureau of Economic Analysis, has shown the beginnings of what might be quite a material productivity uplift. The United States has shown in the last couple of prints quite significant labour productivity numbers. TFP is still pretty modest, but significant labour productivity numbers. I don't want to overinterpret a couple of data prints, but we're at the beginning of seeing I think something that could be consistent with a significant productivity uplift. There are parts of the economy where you can see that in real life. Companies that are using AI agents to deliver real leverage to their workers, really augment their workforce in ways where they can get a lot more done with the same level of resources. That's the definition of productivity and I think in lots of companies that I see that is happening right now. How quickly it spreads across the economy will be a big factor in how big that productivity gain is. But I think there is promise there that this technology can deliver productivity and when it turns out that it can make sure that Australia grabs that productivity dividend and we need to make sure that we turn that productivity dividend into something that benefits all Australians through increased wages, better living standards and better conditions. If we do that that will be one of the major ways that Australia benefits from artificial intelligence.Speaker 6: Which AI model do you use?Speaker 7: I try to use as many as I can. I try to learn what they're good at what they're not so good at.Speaker 6: What's your favourite? Which is your favourite one?Speaker 7: I'm not sure I have a favourite. I use different ones for different things.Speaker 6: Do you find that it improves your productivity?Speaker 7: A lot. I use the AI tools less in my professional life because you have to be quite concerned about information security but I use it a heap in my personal life. Doing what? I use it for heaps of things. I use it for everything from thinking about what I've got in the fridge and what meal I can make for dinner. I use it a ton with my kids and their education. It is great at doing tests for kids to help them with exam prep. It's great for helping them with homework. It's great for gamifying things understanding what they know what they don't know creating really fun ways to create games for educational opportunities for the kids. I use it a lot and I get a lot out of it.Speaker 6: I get involved in these philosophical discussions with Perplexity and others and Claude and all them and at the end of them they're great but I'm terrified. I think how did that just happen? That was a machine I was just talking to and it didn't seem like it.Speaker 7: Yeah, I don't think it would be a good thing if this technology replaces human connection and we spend more time talking to machines than we spend talking to each other. I think that would be a big step back.Speaker 6: Well actually on that subject that's interesting you should say that because China just passed these laws about emotional attachment to AI bots and other things and they're trying to regulate that and to say that if you as an AI company are aware of people's becoming emotionally attached or addicted they've got to do something about it. Do you think we should do that?Speaker 7: You know we have those rules in my family I talk to my kids about how we use AI quite a lot, some pretty simple rules like AI doesn't do your homework for you but once you've done your best on your homework AI can help you think about how to make it even better. We have a rule on precisely this question about how we shouldn't be using AI as a friend we shouldn't be providing personal information to AI we shouldn't be treating AI likeSpeaker 6: That's good but what about should the government sort of start to tell AI you can't do that I mean that's what China's doing what about that?Speaker 7: Well we're going to get the balance right in Australia I think we need to make sure that parents have agency on how they use AI but we also need to have rules and laws that keep Australians safe that's the balance that we always find in Australia as a democratic country and that's the balance that we'll find in AI.Speaker 6: Let's hope so it's been great talking to you Andrew. Speaker 7: Likewise. Thanks Alan.Speaker 6: Thanks so much for joining me on That's Business Now I'll be back on the weekend with a bonus episode of That's Business about immigration talking to Mark Cully who's an author of a new book on the subject so stick around - join me then.