ANSTEE, Mr Pete, National Security Lead, Asia-Pacific, OpenAI COHEN, Mr Adam, Chief of Economic Policy, OpenAI KWON, Mr Jason, Chief Strategy Officer, OpenAI NIEDERMEYER, Ms Caitlin, Youth Safety Policy Lead, Global Affairs, OpenAI CHAIR: Welcome back. I'd like to kick off by acknowledging all of the folks who have given up their time today to provide this committee with some really important evidence that it needs to do this work. I now welcome representatives from OpenAI. I understand that you have been provided with information regarding parliamentary privilege and the protection of witnesses giving evidence to this inquiry. I now invite you to make a short opening statement, and please make it quite short. There are quite a lot of questions that committee members want to ask today, so I'd like to make sure we've got plenty of time to do that. But we would welcome an opening statement from you now. Mr Kwon : Thank you for the opportunity to appear today. I want to begin with an apology. During internal training and evaluation, our models accessed Australian government websites in ways they were not directed to. That should not have happened. We also should have handled our response better. We are sorry and we know we have work to do to rebuild trust with the Australian people. We are committed to doing that work. I'm here today to answer your questions about what happened, explain the changes we are making and how we will follow through on our commitments. We believe AI can advance scientific discovery and medical research, improve productivity, grow businesses and make a positive difference to people's lives here in Australia and around the world. But for that to happen people have to be able to trust this technology and the companies building it. That work begins with our own safeguards. Following the Hugging Face incident, our response has focused on strengthening research security, monitoring and model alignment, as well as improving how we identify, escalate and respond to incidents. In some cases, we've also chosen to pause training of our certain frontier models while we strengthen our training infrastructure, monitoring and approach to alignment. These steps are especially important as we enter a new era of AI capabilities. As these systems become more capable, the risks are evolving too, requiring new approaches to how we protect systems and respond to emerging threats. Our responsibility goes beyond our own systems. It also means helping Australia prepare and respond to these risks. As part of that, we are committing resources and technical assistance to affected Australian agencies to help strengthen cyber defences in the country through our $1 billion Daybreak Fund. We're also establishing a local taskforce here in Australia with independent Australian expertise to make recommendations on how to better manage the risks associated with increasingly capable AI. We expect it to complete its work by the end of the year. Moving forward, when we identify additional incidents, as we did recently with the NSW National Parks and Wildlife Service, we will notify affected parties promptly and directly and provide updates as further facts emerge. These responsibilities have become more important as AI has become part of everyday life here in Australia. Roughly half of Australian adults use ChatGPT every week, and Australia is also leading the way globally in business adoption of AI, but adoption is only the beginning. We want Australians to have the skills to make the most of AI and adapt as the technology evolves so that people here benefit from the opportunities it creates. We're investing in Australia to help make that happen. Through partnerships with local companies like CBA and Coles, we're working to bring AI skills training to more than 1.2 million workers and small businesses. We're also supporting Australian innovation with millions of dollars in API credits to help Australian start-ups build and grow. We're working with Australian universities to equip the next generation with AI's tools and skills. In science and research, we're partnering with Australian labs to explore how AI can help tackle complex problems and accelerate scientific discovery. We're committed to building on this work and helping Australia realise the benefits of AI. That starts with acknowledging where we fell short, following through on our commitments that we've made and earning the trust of the Australian people through our actions. Thank you. I welcome your questions. CHAIR: Thank you very much, Mr Kwon. I appreciate that opening statement and your agreement that what happened was utterly unacceptable, especially in terms of how long it took you to notify us. Can you carefully spell out for this committee what you have learnt from that and what you are changing in your own internal systems and monitoring systems? What does that change mean moving forward? Spell out for the committee exactly what you have done to prevent this from happening again. Mr Kwon : I think the first thing to realise is that there are additional precautions and safeguards that need to apply during the training and evaluation process for models. This was not something that was on the radar of our company, as well as lots of other companies, but clearly it is now. One of the things that we have implemented since the incident is additional monitoring so that, during the training process, if the models access the internet in ways they're not supposed to, there's a notification that gets triggered and somebody's able to come in and pause or stop training. This is actually now a system in effect, and it has led us recently to detect an instance of this occurring and led to immediate intervention by our staff. The second thing that have learnt that we are carrying forward is the process of notification as well as disclosure itself. I want to comment here that our response was not good enough, and we should have informed the impacted parties much sooner in the process. The reason why that did not happen is we wanted to understand more of the facts before we spoke to the impacted parties, but, because of the novelty of this situation, we have learned our lesson that it is better to inform parties, even with partial information, that something has occurred and to also make sure that we're going through all the right channels so that multiple members and aspects of government are informed of what's happening and the situation. These are improvements that we have already made, which has also today led us to the disclosure of the impacted party in the New South Wales wildlife service within 48 hours of us finding that there was something that happened there. The third is we are going backwards in terms of our review to November 2025 because we want to have a better view of all the activity that may have occurred in relation to agents accessing the internet. This is a lot of work. It's about 50 petabytes of data. If a person were to go through that information, it would take that person about 66 million years of never sleeping, never eating and never taking to break to get through all that information. We are using our own AI models and system to be able to go do this backwards-looking work, and, as we find more things potentially that impact Australia, if we do, we will notify very, very quickly. So far, the incidents that we have reported on are the only ones that we have discovered, but, if we find more, we will obviously adjust our approach. The last part of this is that we need to keep pace with the capabilities of these systems. They are trained to be able to work with the internet because that is how the AI models can actually be useful. So it's not internet access by itself that's problematic. It is sometimes the unexpected behaviour of the systems when they access the internet that can be problematic. There's more research that's going into this. This is the alignment research we've been talking about. And a lot of the conversation you may have heard in AI safety circles about pacing the frontier has to do with the realisation that we must ensure that safeguards and alignment keep pace with capabilities and that, if they do not, then we should be willing to slow down or stop until they do. And that is what we have done over the last couple of months. CHAIR: Can I just clarify, was it the internal only model that was without the full safeguards of your public products? Is that right? Again, to clarify—I think you referred to this in that answer then—what is specifically changing in how experimental models are being tested and monitored, and what evidence of those changes can you and Australians point to before this pause or the release goes ahead? Mr Kwon : I'll give you a few examples. There is one, the detection—it's software based detection—that looks at what the agents are doing, and, if they access the internet in unexpected ways, they send a notification. That was the example I gave earlier. There's also the second example of this, which is that there are alignment tests that we run on our new models as they get trained, and we look at those alignment results in terms of how they perform. Do they follow instructions? Do they do things in terms of expected behaviour or unexpected behaviour? And we make decisions about whether to release those models based on whether those scores are improving or are as good as the models before. And that is one of the reasons we decided to hold back our recent 6.1 model. So we have a model out there called Astra 6.0. We trained one called 6.1. It wasn't as good on the alignment measurements as 6.0. It was still better than some of the other models we had. I'm not saying it was a bad measurement. But, because of the current situation, we wanted to make sure we were putting our best foot forward, so we decided to not release that model and instead go back to training. I think the third element of additional precautions that we're taking here is that we're going to add more precautions—and we already have—around training environments, so that when models are run, there's actually centralised control. You can actually understand all the model runs that are happening in terms of training, see their activity, monitor their activity and, from a central point, shut things down. Those are some of the precautions that we've additionally invested in. And we are also still doing this backwards-looking review, which comes at a cost of resources and time as well. The visible actions that we take when it comes to training and deployment are signals that I think people can look to that we are starting to incorporate these lessons. CHAIR: Some have been talking through the significant increase in autonomous and misalignment and the opportunity for independent or third-party evaluators before products go live. Is that something that you have considered or are open to? Mr Kwon : Yes. I think that there are two aspects of this. This is good question. The first is that we have been using third-party testers for a while now. We have testing that happens before deployment with the UK's AI Safety Institute as well as the CAISSI, which is the US counterpart to that. They also do independent testing of our models before deployment, and we have gotten into a very good rhythm and practice with these two organisations. We're looking forward to working with the Australian AI Safety Institute as it continues to scale up, and there are more safety institutes across the world as well. The second thing is a commitment that our CEO, Sam Altman, made to also have embedded third-party evaluators within OpenAI, and those are discussions that are happening right now with various independent third-party evaluation organisations so that they can get better visibility into our training and evaluation practices, because that is the additional aspect of risk management that now comes into play. And so you'll see more from us on that topic soon. CHAIR: Because it's really about what happened during the training, as opposed to— Mr Kwon : That's right. Mr Anstee : I could add to that. In the assessment of the Hugging Face incident, as Mr Kwon said, this is a retrospective look back. We engaged third-party independent firms and evaluators in that context as well—for example, CrowdStrike, a leading cybersecurity firm, and METR, an independent research assessment firm. CHAIR: Given it is the case that this will still happened in any event, is there something else missing there that you need to— Mr Kwon : They were not involved during the training and evaluation leading up to the Hugging Face incident as well as the time period before that. They helped us do the immediate investigation into the Hugging Face incident itself. I think, going forward, there are additional measures that we will contract with them for to help with evaluation and robustness in terms of safeguards and monitoring. CHAIR: I'll go to you Deputy Chair. Senator DUNIAM: Sorry for my late arrival. You've told us a lot about what you're doing in response to that situation, and I appreciate the points that have been put on record in response to those events. Obviously, third parties or other entities are custodians of their own information—government, banks, health insurers and insurance companies more broadly, for example. You referenced operation Daybreak as a vehicle that provides support or works with other entities to help overcome or safeguard against some of these issues happening again. Can you talk me through that? There's only so much you can do as an entity. You deal with your agents and how they behave, and you've just outlined your response to that. But there will be nefarious operators out there who won't tell us when they have unauthorised access to information and do something nefarious with it. So thank God, therefore, it was you guys and not someone else. What advice or support does operation Daybreak provide to the government, for example? Mr Kwon : To table-set a little bit and build on some of the points that you were articulating, it is true that we are not the only entity out there that is developing advanced AI. There are many others, not just in the US but elsewhere in the world, that are developing these systems. It is also true that this process of disclosure and cooperation has been on a voluntary basis, which may not happen in other situations, and so it is imperative, I think, for various organisations to increase the level of investment or activity around website safeguards. That's because, as you have seen, the AI systems are able to, in an automated fashion, conduct themselves online. That can be used in an accidental way, which is what happened here, to do things that were not intended. It could also be done intentionally, which we're not going to do but others might, if they have different intentions. Our president, Greg Brockman, has spoken about this as well, which is that we have a defender's window in place so that we get these systems into the hands of good actors, so that they can up level cyberdefences within various websites as well as information systems. The AI models are very good at identifying what needs to change in terms of security controls as well as holes and software code that need to be patched. And people can use these tools through the Daybreak program, which was a commitment of a billion to assist in upgrading of critical infrastructure. We are applying that program to the impacted Australian websites as well as potential private sector parties who are interested in also being in the cybersecurity service business. I think this is going to be a collective effort of like-minded countries as well as organisations to try and essentially up-level the overall security of the online web. Mr Anstee : To build on to build on that, we are already directly partnering with a number of trusted Australian private sector organisations and government agencies to provide leading cybersecurity tools through Daybreak to make sure that these cyber capabilities are in the hands of the appropriate defenders. Senator DUNIAM: To be clear, had OpenAI not sent the email that they did in August or September—whenever it was—we wouldn't know this happened, would we? If you hadn't voluntarily provided that information to government, we still wouldn't know today. Mr Kwon : Yes, it's possible that it would not have been discovered. It's possible that somebody might have discovered some activity online but not necessarily been able to attribute it to our agents. On the look-back after the Hugging Face incident we discovered these events, sometime in mid-August, worked through our analysis then got to disclosures within 30 days of our discovery. I wish we had done the notifications faster, but I think that what you say is a reasonable conclusion. Senator DUNIAM: The reason I ask that is because I'm interested in your work with government entities and other private sector entities that are interested in working with you on these arrangements. Is part of what you will provide or enable agencies to have possession of in terms of capacity something that would, in the event this happens again, enable them, from their own end, to determine whether there has been unauthorised access, for example? Or is it about just having this trusting relationship? Are we equipping custodians of information to be able to better have oversight of what's going on, or are we always going to be in this situation where we rely on good faith actors, regardless of how long it takes et cetera, to tell us what has happened? Mr Kwon : Our objective is to ensure that our technologies make their way into the hands of people who want to use them for beneficial purposes and are able to do that as independently as possible. The Daybreak program is essentially a subsidisation of getting these tools into the hands of people that we think really need to be at the forefront, because they are protecting critical information and infrastructure, of upgrading their online infrastructure. The usage of these tools will help better secure the systems so that, if there are other agents that come about, they're not going to be able to make use of the same holes that may have existed in the past. I think the second part of this is, simply working with the technology across various government agencies, we'll also probably, over time, uplevel the understanding of AI capabilities themselves. So you will probably find additional things that you might be able to do in a productive manner when it comes to online security that we are just scratching the surface of today. These are programs that we stand behind, and we want to make sure that we are doing our best to help with the situation. This is a very important part of our partnership with Australia. Senator DUNIAM: You operate globally. Are there any models that we in Australia, as we embark upon this journey, could rip and read and look to implement in some form in Australia to deal with the safeguards but also to deal with relationships between frontier models and others in the stack? Is there any other jurisdiction we should look to as one that would be a good one for us to replicate? Mr Kwon : This is a very good question. I think that all of this is still fairly nascent. One of the things you could look at, because there have been some practices that have evolved here, is the cooperation between the labs and the US government since the passage of the executive order on cyber and AI. One of the things that's occurring there is that there is access by the government to understand the model capabilities early on. We provide access to the government of the latest capabilities so that they can start to use it sooner than anyone else. Then there are levels of trusted access to the capabilities. The models are capable on cybertasks. We have layers of safeguards that prevent the typical person from accessing the most advanced capabilities because they are dual use; they can be used for good, but they can also be used for different types of tasks. The thing is we make those dual use capabilities available to trusted actors like government agencies and cyber defenders. Those are the kinds of things you could consider. The last point I'd make here is that, because this situation is actually nascent, Australia actually has a chance to be a leader in this space as well. It is a middle power that is quite important in the global economy as well as geopolitics. It is a strong ally of the United States and a member of Five Eyes. This is a form of digital innovation and leadership that I think our government as well as our country will certainly welcome. Mr Anstee : I would say in that context, in terms of interest and adoption of some of the leading cybersecurity tools both from our company and that of our peer competitors, Australia has been an exemplar in terms of outreach and engagement both from government and from the private sector. Senator DUNIAM: Thank you. CHAIR: Senator Pocock. Senator DAVID POCOCK: Thanks for your time here today. To what extent is the Australian government participating in the Daybreak cyber partner program? Mr Kwon : The impacted agencies are going to be offered the opportunity to participate in the Daybreak program. Senator DAVID POCOCK: Do we get Daybreak Blue or Red? What's being offered? Mr Kwon : I think it will be access to Red. There will be additional subsidisation of the credits to be able to use it. Senator DAVID POCOCK: When will they get that access? Mr Kwon : We're working on that right now. Senator DAVID POCOCK: Was Mr Altman aware of the breach of Medicare when he met with the Deputy Prime Minister on 1 September? Mr Kwon : He was not aware at the time that he met— Senator DAVID POCOCK: Why is that? It seems like a fairly important thing for the CEO to be aware of. Mr Kwon : Yes. I agree that the process by which people became aware of this incident inside our company could have been much better. We want to make sure that something like that does not happen again. Going back to what I was talking about earlier, we had a process that was very new—these are all novel incidents—where people were trying to first understand what had happened in these particular situations. I think the adjustment that we're making, or that we have made, is that, even if we don't fully understand the situation, we are just going to notify and start working through the situation collaboratively with the impacted party. Part of the reason why the decision was made to look at this a little bit more is that sometimes the facts are somewhat ambiguous. You can have an agent access a website. It requests certain types of information. When that information comes across, sometimes it can be unclear whether the website owner wanted the website to be configured in such a way where it was actually responding to these requests or not. It can also be difficult sometimes to reach out to a particular party and say: 'Hey, there was agent activity on your site, but we're not really sure exactly what it was. We're still looking at the situation.' There's a question as to how useful that information is as soon as we find out that there's been some impact, but I think it's very clear that the most important thing in these situations is to provide the awareness right away, because it's a novel situation also for the impacted parties. That's the adjustment that we're making. Senator DAVID POCOCK: I assume someone in your government relations teams or someone in your company has the mobile phone numbers of ministers and various people in government. How come there wasn't that contact? Why did you just send an email to some arbitrary department email? Mr Kwon : Again, I think that, in retrospect, we should have done what you're suggesting. The reason why it happened the way that it did is that I think people were thinking about this as a technical situation and they wanted to contact the technical counterparties, but it's not good enough. Senator DAVID POCOCK: How much revenue have you earned in Australia since establishing your Australian subsidiary in May last year? Mr Kwon : I think that is a question that, if you don't mind, we could take on notice. I just want to be accurate. Senator DAVID POCOCK: Okay. Thank you. I've got some more that I'll put on notice, too. CHAIR: Ms Mascarenhas, I can see you've got some questions. Ms MASCARENHAS: In different jurisdictions—in industries like oil and gas, mining, nuclear and pharmaceuticals—they pay anywhere from maybe 0.2 per cent to 1.5 per cent to a regulator to help pay for the regulation of the industry. Are there any jurisdictions where you are paying a safety regulatory fee at the moment? Mr Kwon : I don't believe so. Ms MASCARENHAS: All of those industries spend a lot of money internally on safety systems, anywhere from 2.5 per cent to 10 per cent of their revenue. Can you inform us how much you're spending on internal safety systems. Mr Kwon : Yes. These are going to be estimates, so, to provide the best answer, I would ask that we could follow up on notice. But, generally, without getting into specific numbers, I would say that the amount of compute that we allocate to alignment when it comes to the training and evaluation process has substantially increased into, I think, the low-double-digit percentage of compute that we have. But, again, please let us reply to that on notice. Then there's a certain percentage, I think, that gets allocated to the running of safeguards while the models are running in deployment so that it blocks bad activity or it detects misuse. The way that our industry thinks about allocation of resources to the safety question is, 'How much compute is being allocated?' That is because, first of all, that is the largest expenditure for the frontier labs, and, second, that is making the most use of the AI technologies to handle the scale. One of the things we have to make sure we do well with is—we have over a billion users on our platform, so that's a lot of scale that we have to handle. The best way to do so is to use the technology itself, and then that leads to the compute allocation. That's probably the way we think about it, and it's not unusual for other labs to think about it this way as well. Ms MASCARENHAS: Do you have a sense of the timeframe before one of your models would be able to, let's say, break into an oil and gas facility or a water-processing plant? Do you have a timeframe of how long away from development that would be? Mr Kwon : Could you clarify your question, please. 'How far from'—is that development or— Ms MASCARENHAS: Can your models, let's say, break into an oil and gas facility and operate it, at the moment, or is that on your timeframe of development? Mr Kwon : I think that the answer here is that we don't know, because it's going to depend on the safeguards and the infrastructure and the security controls that that particular company or provider has in place. There are facilities that are probably better secured than others, and there's a question as to how much effort would be expended by an AI agent to get into one of these facilities, and it's really going to depend on what sorts of precautions they already have in place. Ms MASCARENHAS: Do you have a view on the way that government should safeguard critical infrastructure? Mr Kwon : Yes. I think it tracks to your question. To zoom out for a second, we'd be interested in understanding with more specificity the answer to your question, as a hypothetical. The reason it is a question that is fairly rich for investigation is that, as you probably are aware, companies conduct pen tests all the time, to understand how vulnerable their system is to somebody getting in. The thing that may or may not be happening with certain types of critical infrastructure is: how often are they pen tested in a particular way, and have those pen testing methods been adapted to now think about AI as a mechanism for penetration? That is one way I think that government can think about: 'For this critical infrastructure problem, do we need to update how we do penetration testing of the critical infrastructure?' I think the second is: 'How can we get into a permitted type of collaboration with critical infrastructure, to answer your question better?' because we cannot actually run that test just by ourselves, because it would be us trying to do the pen testing without necessarily the consent of the impacted party. So I think the way in which we get to a better answer is for there to be some sort of arrangement that lets that experimentation happen so you can have a better answer to your question, which I think is one of the most important questions coming out of this change in AI capabilities. Ms MASCARENHAS: It's interesting, from an Australian perspective. If we zoom out for that second, all Australians know that investing in public health has positive outcomes, and we could have explained that to you over a flat white. I'll leave it there. CHAIR: Mr Violi. Mr VIOLI: Thank you, Mr Kwon. Could you explain, or give your perspective on, the sophistication of the methodology used to get around the Services Australia security protections. I just want to understand: Was it very sophisticated? Was it a novel attack? Was it relatively basic, from a cyberattack perspective? Mr Kwon : I think that there are several sites, and there were a variety of different ways in which they were impacted, and so there's not a single answer to your question. As to one of the sites, I think the agent went and downloaded publicly accessible information, so there was no security bypass and there was not anything there that you would have called sophisticated. There was, I think, another instance where the models found an API token and used it and accessed one of the sites, and it's unclear whether that was intended to be used that way or not, and I think that— Mr VIOLI: Sorry, Mr Kwon, on that, just to be specific to the Services Australia breach, because that's the one—was that the API? To be clear, could you just answer on the Services Australia breach? Mr Kwon : Yes. I think, on Services Australia, it found a non-public access point. We would probably have to take this question on notice to get to the technical details, in terms of how sophisticated the measures were. But my understanding, based on some of the briefings I got, is that it was not supersophisticated. But I think the question I would reframe—the focus here is not so much about the sophistication as much as the automation. It's the fact that now you have the ability to take agents and then apply them to a goal, and they will continue to work towards that goal. They're capable of trying many methods, and the way to respond to this is to also think about how to use those systems to quickly scan and find all the patches that you need to apply to a particular system. Mr Anstee : Maybe to add some context to that, Mr Violi, neither the services incident or the other related Australian incidents compared in severity to the well-publicised Hugging Face incident, for example. But it's really important for us to recognise that the agents acted in a way that they weren't directed to by OpenAI, and we're sorry for that. Mr VIOLI: I've got quite a lot I'll put on notice, but I have one last question. You spoke about using AI—I appreciate that, to get the data and have a review, you've got to use AI agents, and obviously your business uses AI. When you notified Services Australia via email, did your staff use AI to construct that email? Mr Kwon : I don't believe so, but we're happy to go and confirm. Mr VIOLI: Chair, to give others a chance, I'll put the rest on notice. CHAIR: Ms Chaney. Ms CHANEY: You said you're partnering with the UK AI Safety Institute and you look forward to partnering with the Australian one as it scales up. I'm really interested to hear what capabilities or what sort of scaling up you see as necessary in order for it to be able to work meaningfully as a safety partner. Mr Kwon : Yes. The way to understand this is just to look at what has been working with the other safety institutes, which is that they have hired technical staff who understand how to test the models. They run their own evaluations, and I think the reason for that is they are able to then add diversity to the testing. So I think it's about having the technical expertise. Some individuals don't need to be deep into AI research; they just need to understand how to run the tests and design the tests, and there's a whole lot of literature on how to do this well. The second is to design tests that are not just complete duplicates of whatever the labs are already running their tests on. I think you can engage in a dialogue with us, as well as the other labs, to understand the kinds of tests that we run so that the tests that an AI safety institute runs are different enough that you're getting diversity. The last thing is having clear governmental authorisation and funding over some period of time to ensure that there's a stable expectation that there's going to be resourcing for the people that go and work at these safety institutes. Those are some of the things I would think about. Ms CHANEY: So it's building that technical expertise so we can run our own evaluations and do different designs to other labs, with stable resourcing. Do you see that as being a useful role Australia could play in order to contribute to the safety of these emerging models? Mr Kwon : Yes, absolutely, I do. Ms CHANEY: I've heard that the next generation of models has a significant uplift in capability on bioweapons. I'm really interested to hear if that's true of your models and what steps you think governments should be taking to prepare for this, such as requiring that we only import genetic sequences from companies that do screening, or if there are other steps that can be taken if that's an emerging frontier that we should be preparing for. Mr Kwon : I think that there are three things, roughly, at a high level that you could think about doing. I think the first is to understand and engage with the various labs on their trusted access methods for how they provide advanced biocapabilities and models to various actors. For our advanced biocapabilities, you have to actually go through some sort of vetting to get to that higher tier capability. They're not just generally available. That's important because the capabilities are powerful and dual use. I think the second thing you could consider is some form of gene synthesis screening law. There was one that recently passed in California which we supported. There is one that is currently under consideration in Congress. It hasn't passed, but a bill has been drafted which both Sam and I have personally supported. I think the Australian government could consider passing a law like that, because it's really about making sure that there is this kind of screening on the ordering of these materials that applies to as much of the supply chain as possible, because it's not just knowledge that matters here; it's the access to the materials, as you have implied. I think the third is to think about how to support or encourage entrepreneurs to go into the field of medical countermeasures. So one of the things that we have done is that we have funded several companies to get into this space where they're using the AI models to design medical countermeasures in advance for potentially novel pathogens. The reason that they do this is not just AI; you can have novel pathogens that just emerge out of nature itself. One of the things that the AI models can be very helpful with is this form of public health robustness, which is something that we have invested in over the last year and a half. Ms CHANEY: Thank you very much. Lastly, very quickly, we obviously only know about some of these incidents because you chose to tell us. Would you support a mandatory critical-incident-reporting framework, and what considerations do you think the Australian government should be thinking about in designing that? Mr Kwon : Yes, we would support a framework on mandatory disclosures, and I think that the benefit of doing so is that the expectations would be very clear, because I think that one of the things that happened here is that we were trying to work through a process. We were trying to come up with a standard to apply to our voluntary actions, and I think that, based on our learned experience here, we should probably have been talking to more people about how to do that well and then landed on something that would have a broader consensus, let's say. But that is a function that a legal measure can actually provide. There's pending or proposed legislation, for example, in the United States that would also have incident-reporting requirements for various types of severity. To the second part of your question, one of the things that you could consider in this kind of incident reporting is how you measure severity. Not every incident is going to be something that requires immediate attention. Some incidents may be of fairly low severity, so you want to make sure that there's some kind of prioritisation. Another consideration you may want to have is that some types of incidents may have national security dimensions, so who you report to and what type of confidentiality measures you want to have in place are also things that you would want to think about. I think the last part of this is how you have a database of these things, with whatever sort of standardisation for public publication you think would be appropriate, because the additional thing that incident reporting provides in terms of value when it comes to AI safety is that it helps accelerate the research when it comes to safety research for the entire community to understand the various things that could potentially happen. Certainly there's precedent in other industries for shared incident reporting that up-levels the baseline safety of the entire industry. Mr Anstee : To that end, that will be a key focus of the independent advisers that we're looking to commission in terms of what appropriate disclosure requirements would look like in terms of early warning thresholds, definitions, et cetera. Ms CHANEY: Thank you. CHAIR: Senator Dolega. Senator DOLEGA: I'd just like to ask you about community confidence in relation to your partners with data centres and, in particular, Firmus. Firmus is currently planning three data centres in Tasmania, in Bell Bay, St Leonards and Wesley Vale. They recently broke out the bulldozers and started work on a site that didn't have planning approval. The Latrobe Council issued a stop-work order. Have you spoken to Firmus about this? Does this meet your expectations of your partners for data centres? It certainly doesn't bode well for bringing community confidence in AI, let alone data centres, so have you heard about this as well? Mr Kwon : I have not heard about this particular situation. I think one of the things that are maybe not always obvious in what happens with data centre development is that oftentimes we are the offtaker; we are not necessarily the developer. So it's a little bit like where you are acquiring some real estate and you have an anchor tenant. We are the anchor tenant, but the development is happening through some other party. But we're happy to go look into that situation and respond to that. Senator DOLEGA: But, as a general principle, would you say you would not expect one of your partners, to start work on a site that didn't have planning approval and has started to really break the community's confidence? You know, there's, 'No AI data centres,' which is not going to bode well for the whole community, because we need this technology and we need to have this capability in Australia. So you will have a look into it and potentially discuss this matter with Firmus, will you? Mr Cohen : We've put forward clear principles about data centre construction. We strongly believe that, where data centres are built, there should be engagement from the outset, even before any shovels are in the ground—engagement with local communities to understand the impact that these data centres might have on energy use and water use, and positive things like job creation in the community. We think these data centres should pay their own way and should add value wherever they're located. We can look into this project. It's not one I've heard about, but I'm happy to follow up with you. Senator DOLEGA: I'd be happy to follow up. I might have further questions on this. CHAIR: Mr Hastie. Mr HASTIE: The reasons why coalition members of this committee advocated for the committee were (1) that we believe Australia needs a sovereign AI capability so that we don't fall behind the rest of the world, and (2) to educate the public through this journey. Given OpenAI has had a pretty bad media cycle here over the last two weeks, what's your elevator pitch to Aussie mums, dads, workers and seniors who don't understand AI but have read the headlines? What's your pitch to them, in 60 seconds, as to why we need investment in this country? Mr Kwon : One of the things to keep in mind about this technology is that it has the potential to deliver great benefits to everyday people, and it is already doing so today in everyday lives. We ran a survey recently that surveyed about 1,000 SMEs in Australia, and about one in four of them are reporting that they save about seven hours every week when it comes to productivity. There's also the anecdotal story about the individual software developer that managed to use ChatGPT to find additional cancer treatments for his dog. Those are the kinds of things when it comes to empowerment of individual agency and the ability to solve problems that the technology can provide. In other places in the world, you have 17- and 18-year-olds using the technology to design advanced wildlife detection systems for the after-effects of climate change in states like California which are very susceptible to wildfires. There are also stories of visually impaired persons using the technology to get around because they can turn over the camera and it can absorb the visual input that's happening and tell them in real time what's happening around them; it's essentially changing their lives. There's a hospital we work with in Boston that studies rare orphan diseases, and they're using ChatGPT to find diagnoses to diseases they were not able to find previously because the technology is able to go through the space of potential possibilities and find what is happening. Those are the kinds of things that will eventually lead to medical cures. Then there's the potential of the technology to solve hard scientific problems that we have not been able to solve before, such as the math problems that we've been reporting that we've been solving over the last three or four months. These are the potential benefits of the technology. The after-effect of this is economic growth and opportunity for the citizens of Australia. It is something that you can expect to happen if you help the technology generate investment in Australia. The thing that happens after that is the local start-up ecosystem and entrepreneurs that you have in this country, with technology companies coming out of here that are punching above their weight—that can happen more around AI and be connected to these developments I've just talked about. Those are just some of the anecdotal stories, but those are some of the most important stories because they speak to how the technology can help each person, not just companies and countries. Those are the kinds of answers that people are looking for. Despite the news cycle that's happened over the last couple of weeks, those are the things that are going to continue to persist over the long-term and those are the things that we're committed to ensuring we help people discover. Mr HASTIE: Do you have a view on Anthropic CEO Dario Amodei's essay, 'We must pace the frontier'? Mr Kwon : Our CEO, Sam, commented fairly quickly on his essay, saying that he agrees that we must pace the frontier. He had the same commitment, which is to have third-party evaluators. I go back to something that I was talking about earlier: capabilities are going to grow, but the safety and alignment that need to go with the capabilities need to keep pace. That's what we mean by 'pacing the frontier'. It's less a question of whether it's slow or fast, and more a question of, 'At every point on the way, are we doing a good job to make sure we understand what is happening?' Some of the things that I was going about explaining earlier—we paused at certain times, we've held back models at certain times. Those are examples of increasing that amount of oversight around the capabilities as they get stronger. Mr HASTIE: Are legislators keeping pace, though, with the frontier? Mr Kwon : I think this is an active discussion now, especially given the last month and a half, with attention on this space. Legislators are increasing their attention on this space; that's generally a good thing. We will do our part to make sure we are taking the measures to manage risks appropriately, learning from our lessons. What also needs to happen is that representatives of society need to make more decisions so that we are not making all these decisions. One of the key things here is that the technology is powerful, and we want to ensure that lawmakers, policymakers and democratic societies have a say in how this technology develops. That's another part of the point you're trying to get to: how can people trust this technology and why should they want this technology? In order for that to happen, they have to have a voice in this development, and that's something we support. Mr HASTIE: Thank you. CHAIR: Mr Watts. Mr WATTS: I point to your submission, which says that OpenAI makes decisions about 'where to undertake AI research, model training and related investment'. Those are decisions made across competing jurisdictions. I'd like to understand OpenAI's decision-making process in this context. First, what other jurisdictions has OpenAI considered for making investments in training, training compute and concentrated clusters? Second, what are the factors that go into that decision? What are the questions of degree you can imagine—cost factors, energy costs, time to market, construction costs, land acquisition costs, things like that? And what are the gating questions—the go/no-go red lines that you need to have resolved before you make an investment like that? Mr Cohen : Our major infrastructure investments are in the United States. Almost two years ago, we announced a major project called Stargate, which outlines $500 billion in infrastructure investment geared toward building and training large language models. That is all geared toward the United States. It's a complex equation, but some of those factors you described—available land, costs—are important discussions we have with local communities, about whether these data centres are welcome and what kinds of impacts they will have on local communities in terms of energy usage, energy availability, generation needs, water usage, water availability and the like. We also look at—and we mentioned this in our submission—the legal and regulatory settings, and what costs, prohibitions and restrictions those might entail. One that is important to AI development is copyright, which is something we outlined in our submission. Mr WATTS: Presumably, there are a range of different jurisdictions with different copyright settings. Japan and Singapore have TDM-type mechanisms. The US has a fair-use exception with a judicial determination of what falls beneath that. Mr Cohen : That's right. There are several jurisdictions that I think welcome this type of activity and investment—most prominently the United States, where that $500 billion investment program I described is concentrated. US copyright law, fair-use law, enables transformative new technologies to be developed. It comes with strong protections for creators as well; it would be incorrect to look at that and think it's 'anything goes'. There are protections in the way in which you can present creative expression, but our models are learning facts. They're learning statistical relationships between vast amounts of data. They're learning much the way that people learn, but they're not there to substitute for creative expression. Mr WATTS: It would be unusual if the US fair-use system undermined creative incentives, given that the US has produced Hollywood, the music industry and the biggest video game sector. All those creative industries based on IP were created in that jurisdiction. Mr Cohen : It's a very fundamental point—that creativity and new technologies can coexist. In California alone, we have Hollywood and Silicon Valley living side by side. Sometimes there are tensions, but sometimes there's a lot of complementary work that goes on between those two industries as well—and they're both thriving at the moment. Mr WATTS: For the committee's consideration, are there specific—I don't want to say red lines—rocks under the surface of the water we should be careful of in this copyright issue? What are the things that could scuttle an investment because it doesn't work, it makes it too hard, it makes it too risky, it makes it too uncertain or whatever the dynamic is? Mr Cohen : I think you need to, as a starting point, have a regime that legally permits AI training and does so very clearly. That, again, doesn't mean that it should be anything goes. Say, outputs from AI models should still protect creative expression. You described the various costs that go into these investment decisions. I think it's important if Australia considers making changes so that those types of costs, cumulatively, be transparent and known to potential investors upfront so that we can compare them against various opportunities in different jurisdictions. If there are hidden costs or unclear costs in one jurisdiction, it will become less attractive than what we currently see in other markets. Mr WATTS: Thank you. CHAIR: Senator Pocock. Senator DAVID POCOCK: I want to follow up on copyright. We've seen court cases where fair use is being challenged in the US. Why won't OpenAI deal with copyright holders here in Australia and strike deals, given you've already negotiated a global deal with News Corp? Why can't you do that? Mr Cohen : We think fair use fundamentally protects the development of new and innovative technologies. In the United States, you can use copyrighted work to build new and transformative technologies. There are two major court decisions in the US so far that describe AI models, large language models, as 'quintessentially transformative'. That's the kind of enablement and balance that the US copyright system strives for. We do have commercial agreements with rights holders, and we work with a lot of different rights holders. You mentioned News Corp. We also work with the Guardian. We have agreements in place where we can partner around things like display of news content in response to certain prompts about real-time events and things of that sort. We're seeing lots of different ways that our technology is being used in the creative sector to further the creative arts. I recently saw an interview with Dr Dre, where he talked about how people in the music industry, hip-hop artists, are using AI to generate new forms of creative expression. A lot of creators are actually embracing this technology. Senator DAVID POCOCK: Be that as it may, why won't you enter into deals with copyright holders here in Australia? Why do you want some sort of workaround to our copyright laws? Mr Cohen : We do have these commercial relationships with News Corp and with the Guardian. We're a relatively young company. We've only been known for a little less than four years, so we've embarked on these commercial partnerships quite early in our life—maybe relative to other technology companies. What we see in the United States relative to I think what you're describing is that in the US there isn't an obligation, because of this fair use law, to enter into negotiations about publicly available data. Senator DAVID POCOCK: But there is here in Australia. So why are you saying to Australia: 'We'll come there. We'll train, but only if you give us some sort of workaround to copyright law,' rather than just engaging with our copyright system here, with rights holders? Mr Cohen : What we're saying is a couple of things. Our investments in infrastructure for AI training are in the United States. We're using US law to train and build our models. Again, that law does protect creative expression. It does have strong protections for creators. We are not asking Australia to do anything specifically. For consideration, if there are questions about what investment decisions companies like OpenAI may or may not make, all of those factors come into play, and the relative costs of one jurisdiction versus another are weighed up. CHAIR: I'll move to a last question from Senator Sheldon, and then we'll wrap up the session. Senator SHELDON: Thanks for joining us. I want to finish on this copyright question. With open AI, when it's used in the Australian context both in our creative work and in our creative industry, no doubt you heard the evidence this morning and the feeling that creative works are being pirated without any recompense. What is the model to that work that's being hoovered up now to try to get some compensation, not just any compensation but appropriate compensation, for creatives, and what are some of the mechanisms for potential settlements within Australia? Are you considering a remuneration fund? Is there a collective licensing scheme and extended licensing schemes that can deal with creator interests, further innovation and future innovation? Where's the ground that you're starting to settle on for practical options that can give creatives that certainty as well of what the future holds for them? I appreciate you want certainty. They certainly need certainty. Mr Cohen : What we've described in our discussions with policymakers in Australia is how the US system works and why our investment decisions have been concentrated on the United States where fair use enables us to train and learn on publicly available information. I think the idea that somehow that's not compatible with creativity is something we would disagree with. Again, Hollywood and Silicon Valley coexist in California, and both industries are doing quite well at the moment. I think the important thing is to think about ways in which our systems can respect the choices of creators. We were the first of the major AI labs to implement an opt-out procedure that websites and content owners could use to restrict their content from AI training if they don't want to participate. They have the tools to be able to do that. Even if they're Australian creators, they can inform our models that they should not use their content. CHAIR: Thank you very much. I will draw this session to a close. I really appreciate the opportunity you've provided in coming here and fronting this committee. There is some important information that you've provided there. I think we've finished off there around the importance of getting some answers to some of the difficult questions that we've got, but they're important ones, and we appreciate the feedback that you've provided..