The Digital Transformation Agency (DTA) has found challenges with the integration of Copilot with products outside of the Microsoft Office suite identified during the trial of generative artificial intelligence (AI) used across government agencies. The trial took place between the start of January and end of June, with the DTA coordinating the government’s trials of a generative AI service, making Microsoft 365 Copilot (formerly Copilot for Microsoft 365) available to over 7,600 staff across more than 60 agencies.  In the government's Evaluation of the whole-of-government trial of Microsoft 365 Copilot report, the DTA found a range of issues from integration with other non-Microsoft products to security management problems. A prominent issue raised by trial respondents in the report related to the integration of Copilot with Microsoft Outlook; Copilot features in Outlook were only available with the newest version of Microsoft Outlook or the web version of Copilot. The report noted that Microsoft initially planned to release Copilot updates to the new version of Outlook and later for classic Outlook. “Focus group participants often lamented not being able to access the full capabilities of Copilot as they did not have access to the new Outlook,” the report stated. “One trial participant noted through the issues register that classic Outlook will only support the bare minimum Copilot features.” The Microsoft AI tool was also found to have led to problems with information, data management practices and permissions resulting in appropriate access and sharing of sensitive information. For example, trial participants raised instances where Copilot brought up sensitive data they didn’t have permission to. However, the report did note that this data wasn’t properly secured before the use of the Microsoft tool. Despite the challenges around Copilot, the DTA said there were opportunities to explore more tailored AI solutions, strategies for uplifting functionality and identifying areas that still need improvement.  "The DTA has always been clear in its goal to not shy away from innovation," said Lucy Poole, DTA general manager of strategy, planning and performance. "To swiftly provide the APS [Australian Public Service] with tools to experiment with generative AI in a safe and responsible way, we identified a tool that would quickly integrate into most workplaces with minimal disruption."  Most trial participants (77 per cent) were satisfied with having an integrated AI tool, with many more (86 per cent) wishing to continue using it. Additionally, 69 per cent of APS participants felt there was a marked improvement in the speed of wrapping up tasks, with nearly as many (61 per cent) believing having such a tool enhanced the quality of work output.  While there was marked improvement in the delivery of general tasks, some feedback did point to the need to spend additional time reviewing content that was generated. This influenced some of the recommendations provided within the evaluation. “As we are testing these tools at such an early stage, there are clear opportunities for tailored solutions to be developed that can handle highly technical material,” said Poole. The release of the report comes two months after the DTA came under intense scrutiny during the Senate Select Committee on Adopting Artificial Intelligence on 16 August. During the proceedings, Senator David Pocock questioned the agency on the federal government’s six-month trial of generative artificial intelligence through Microsoft 365 Copilot via the DTA. Pocock said that it was a “stretch to say this is an additional product,” while he acknowledged [Copilot] is a Microsoft product “but it’s not like having Word and then adding Excel to your bundle”. “This is about the public service trialing artificial intelligence in terms of efficiency gains and whatever else,” said Pocock. “Maybe part of that is you didn’t feel the need to actually go to market and see what may be there.” At the time DTA, CEO Chris Fechner told the committee that Microsoft has continued to extend a 15 per cent discount for those participating in the trial. “If that period goes through and there are no changes to that, it will revert to their standard pricing associated with the ancillary products, but we are still discussing with Microsoft’s various changes to the volume sourcing agreements,” he said. The scrutiny into the use of AI by public servants inside government is set to continue continue as the Joint Committee of Public Accounts and Audit (JCPAA) commenced an inquiry into public sector entities’ use and governance of AI systems in September. At the time, the inquiry said it would focus on how the public sector entity uses AI and whether there are planned or likely future uses. According to the JCPAA chair Linda Burney, “evidence that to the 2022-23 Commonwealth Financial Statements inquiry indicated an increased adoption of AI by public sector entities but also a lack of adequate governance frameworks to regulate and monitor this use.”