OpenAI agent hacks Australian government site

0:00This BBC podcast is supported by ads outside the UK.

0:30and more options to their lineup, can shop at machinefinder.com. Booyah! Today's episode is brought to you by Apple Ads. Ads on Apple Maps are changing the way local businesses get discovered. Reach customers when it counts, right when they turn I'm looking into I'm going. They're in decision-making mode, actively searching for businesses like yours and deciding where to go. Customers can easily call you, visit, and more from the same place they're viewing your ad. And all of this happens on Apple Maps, a privacy-first environment that respects customers and helps them discover new places. Get started with ads on Apple Maps today, and for a limited time, receive 15 % credit back on your monthly ad spend for a year, up to$1 ,000 per month.

1:18Terms apply. Head to ads.apple.com slash maps to get started. Drive business like never before with ads on Apple Maps.

1:41Hello and welcome to News Hour for the BBC World Service. We're coming to you live from London. I'm James Menendez. For those worried about the fast-growing capabilities of artificial intelligence, well, there's more evidence today, this time from Australia, where the government says it's launched an urgent investigation into a breach of security around one of its websites. A breach caused by an AI agent or programme belonging to the firm OpenAI that hacked into Australia's healthcare portal. It comes as world leaders and the CEOs of the giant AI firms themselves are calling for greater regulation over the industry.

2:17Those calls repeated at the UN General Assembly in New York where they're gathered. Here's Australia's Prime Minister, Anthony Albanese. I spoke with the CEO of OpenAI, Sam Altman, to express Australia's extreme concern about this incident. And I also expressed my disappointment that it took the company way too long to inform the government what had occurred and the nature of the way that that notification occurred as well was unacceptable. Well, the timeline of that goes something like this. Open AI says it only detected the incident in August, two months after it actually happened. The company then decided to report it to the Australian government, but emailed a public mailbox.

3:02That was on September the 10th. The email was actually seen and passed on to the cyber department five days later. Dr Rob Nicholls is Senior Research Associate at the University of Sydney Centre for AI Trust and Governance. How worried is he? I think it's a big worry and it's a worry from two sides. First, that agents that were clearly not intended to hack an Australian government website did so. And second, that the cyber security allowed that attack to happen. So not the agent's fault. I mean, some people listening to this story may think, oh, this is an example of agents going rogue, acting on their own, but let's be clear about this, that that's not the case.

3:52No, typically this sort of problem, and we've seen it a number of times over the last few weeks, is caused by agents doing precisely what they're told, but where people give agents the sort of characteristics that they expect if they ask a person to do a task. So if I ask you to do something, at some point you'll say, well, that's a bad idea or I'm bored with doing this, whereas an agent will just keep on trying and trying and trying until it achieves the task it's been set. Which also leads me on to the question of why what it did, which was wrong, why didn't that sort of flag itself up in OpenAI's systems?

4:44Because it seems to have taken them a long time to even notice that this had happened. I think OpenAI was doing a series of experiments with agents and really didn't understand the potential risks. So they'd set guardrails and said, don't do certain things. But nevertheless, there was still an issue and the agents continued trying to perform their task in a way that the humans didn't expect. And I think that's part of the issue. We've assumed that agents will work a particular way and they don't. Can I also ask you about the fact that it took OpenAI, again, a while to notify the Australian government that this has happened.

5:36And then they emailed a sort of generic government email. I mean, what does that say to you? It says that there isn't a proper reporting procedure. So in Australia, if you suffer a breach, so if you're the victim of a hack, you have an obligation, depending if you're critical infrastructure, an obligation to report within 12 hours. If you're anybody else, you have to report within 72 hours. Whereas if you cause a breach, there's no reporting obligation currently. I think that has to change. but sending an email to essentially the general contact point of the largest department in the Australian Commonwealth Government, that department did pretty well to actually pick it up within a day, but then recognised after a little while that it had to be escalated through to our signals directorate, director at the Australian Signals Directorate, which is the equivalent of GCHQ or the NSA.

6:50Rob Nicholls. Well, Zali Stegall is an MP for the opposition Community Strong Australia Party. She's been telling me about her concerns around this breach. The Medicare system is very important, obviously, in Australia and holds a lot of very sensitive data of Australians. And so it's very concerning that we're only finding out the government's only being notified of the breach sometime after it has occurred, despite the fact that there's been ongoing discussions and negotiations, in fact, with the government for some time. Who do you blame for it? Oh, I think this rests squarely with the corporation.

7:34At the end of the day, We need to make sure that we have safety before deployment, not after harm in place. So, whilst there's obviously the responsibility on the corporate entity, there's also the concern from the government that we haven't got the right safeguards in place. Growing in Australia is a concern around that there's too much, I guess, acceleration of deployment. We have a lot of data centres and a lot of, I guess, projects in pipeline, but not enough safeguards in place and certainly not enough independent oversight. You know, AI as a technology can create extraordinary opportunities from a productivity point of view, but it must have human oversight.

8:24I mean, this is a global issue. What chance do you think there is of some sort of global agreement on regulation of the industry, given that both Presidents Xi of China and President Trump of the US meeting, of course, today seem pretty reluctant to put in place any sort of regulation. Middle powers like Australia need to really look to other middle powers where we are looking at having more guardrails around technology that can get out of hand, especially where it is superpowers like the US or China that are more likely to be in control or have less concerns in relation to these technologies.

9:08But that's the point, isn't it? If China and the US don't do anything, there's not much that countries like Australia can do about it. Well, except that at the moment, we have a pipeline of data centres and open AI wanting to be able to develop and train their models of Australian data, they are negotiating with our government to wind back our copyright laws so that the models can train on content here. Now, that's very contentious with our creative sector because the copyright laws are there to protect their works. So, on one hand, companies want social licence to be able to come and operate within stable jurisdictions, stable countries like Australia.

9:59But for that to occur, for that social licence to be there from the Australian people, there has to be trust that there is appropriate oversight. and that oversight needs to be through and clearly independent of government to make sure that we do have the reassurance of knowing that when breaches occur that they will be disclosed and auditable. Australian opposition MP Zali Stegall.

10:31Now the US has invited President Putin to the G20 meeting that will be held in Miami in December. The Kremlin says it'll consider the invitation. It's not yet known how other countries may react. And it comes as Russia has again been hammering Kiev with a sustained drone attack that killed two people and wounded more than 40. Paul Adams, the BBC's diplomatic correspondent, reports now from Kiev on the impact this relentless onslaught is having on residents of the city. So after last night's ballistic missile strikes, The clean-up in Kiev is well underway. Here by the side of a busy road on a rainy morning, there's a crew digging a great big hole in the ground because a water pipe has been burst, and they're anxious to repair it because people in the local area are without water for the time being.

11:24There's a stump of a lamppost here which has been peppered with shrapnel, and the trees around here have all been burned. Some of them have been chopped down already and taken away, and that seems to have really angered a lady who was here a moment ago. Clearly everyone here is on edge after what happened last night.

11:47And just nearby is one of Kiev's maternity hospitals and it's a place that's quite important to a lady we just met called Christina. Tell me what happened here last night. It seems that the part of the missile landed here but it felt like the whole missile landed here. Were you here when it happened? I lived two apartment blocks from here. But you could still feel it? Yes, I could see the light and I also heard I jumped out of my bed. I was frightened. My sister gave birth here last year in June and she's due again next year in January. We are very worried if everything will be OK. Do you think that the situation in Kiev is getting worse at the moment?

12:35Yeah, it's getting worse. Also, like last winter, we didn't have heating, we didn't have electricity. I lived in Abolonia. It was minus, temperature minus in my apartment. So yeah, it's getting worse. We've heard that some people are thinking about leaving the city for the winter. Is that something you're thinking of? We were thinking of moving to the Capacian mountains, but I'm also scared to go there because here at least you have where to hide, and there are just mountains. After attacks like this, there are often brief power cuts. There are apparently problems with the internet today in some parts of the city.

13:15But so far we have not had the kind of widespread power cuts that we've seen in previous winters. As the temperatures drop and winter arrives, those widespread power cuts are an absolute inevitability. The people of this city are acutely conscious of that, but many of them who have been through this before are determined to stick it out. This winter will be hard for us. Every winter is hard for us because no electricity, so cold, so many ballistics. It's hard. So it seems as though maybe this winter will be more difficult because you have the ballistic missiles and the new drones. I think it will be the same.

14:06Plus, minimum, the same as before this winter. So you're going to stay here in Kiev? Yes, stay in Kiev. Stay in Kiev. 100 % stay in Kiev. One million percent stay in Kiev. That was the BBC's Paul Adams reporting there from the Ukrainian capital. You're listening to NewsHour from the BBC World Service.

14:37Still to come on the programme, Meta's announced a new model of audio-only smart glasses after a backlash driven by privacy concerns. Tamsin Craftman, an AI and tech journalist, has been telling me more. There are nicknames about how they are kind of like the creep glasses. And I think this is a way for Matt to allow the public to use the technology and have these ideas of like smart glasses, but to take away a lot of the shame that consumers might have. That will be coming up in about 20 minutes. Our headlines from the BBC Newsroom this hour. The Chinese leader Xi Jinping will hold talks with President Trump at the White House shortly.

15:15We'll be talking about that in about 15 minutes. And the FBI says it's aggressively investigating an apparent breach of its computer systems by hackers who claim they've stolen the personal information of all the agency staff.

15:37This is James Menendez with NewsHour live from the BBC. Now, to a case of the risks posed by so-called forever chemicals here in the UK. Tests on people living near a chemical factory in Lancashire in the north of England have found high levels of a substance which is classified as the highest level of carcinogen, in other words, that it can cause cancer. Perfluoroctanoic acid, known by the acronym PFOA, was used at a plant owned by a company called AGC Chemicals. That was until 2012. Eight years later, PFOA was eventually banned in the UK. While Sam Hammond and her family, who all live close to the site, have been told they are at increased risk of serious health conditions.

16:20Sam got her results yesterday. The levels that we have got are a lot higher than we expected. There's an overall result in mine that was 25 and my husband's was 27. But then our 21-year-old son was 111. so this puts us in this higher risk. I've got five children and breastfed them all and from research that I've read your body dumps this chemical when you breastfeed. So I look at my level and I think well I've breastfed five children what would my level have been if I'd not done that but then I feel so guilty that is that the reason why my son's is so high? Liana Jose is a journalist at the non-profit investigative organisation Watershed.

17:05What the experts are saying is that the probable source, you know, all roads lead to this chemical factories, AGC Chemicals, which makes a non-stick product, a heat resistant product, something that we commonly know as Teflon. And it's in a multitude of consumer and industry goods. And what we found out years ago is we went and tested the effluent, the discharge coming out of this factory into the protected river wire. And we found extremely high levels of this banned carcinogenic chemical called PFOA, as well as lots of other PFAS that stands for per - and polyfluorinated alkyl substances. And that's a family of at least 15 ,000 forever chemicals.

17:50And so that's really what tipped us off. So what our testing helped to trigger was a multi-agency investigation into pollution in the surrounding area and the Environment Agency found it in the soil, found it in the vegetables and found it in people's eggs. So the last piece of the puzzle was really us doing these blood tests and we found it at incredibly high levels in people's blood. Why the gap between your initial findings and the blood tests? And I suppose this gets us onto the issue of why there's sort of not routine testing in this country, even though there are sort of limits in other countries, aren't there?

18:29Yes. You know, it took us testing the factory effluent to find it. I mean, it wasn't illegal what the companies were doing because there were no laws against it. But it was a banned carcinogen. And it took a long time to get the blood test as well, because the community had no idea. The person with the highest levels of PFOA in her blood, which was over 430 nanograms per milliliter, and if you think that the US average levels are 1.4, She'd been working on her allotment, growing her own vegetables. She thought she was eating organic and doing the healthy thing. And she had no idea that the land she was on was contaminated.

19:07Could this have been prevented? I mean, what's the company saying? So documents show from the US Environmental Protection Agency, they started investigating PFOA back in the late 90s. And in 2002, I've seen documents where AGC Chemical's parent company, Asahi Glass, was having meetings with other major PFAS manufacturers like DuPont and 3M in America, Mittaini in Italy. And they were commissioning and sharing toxicology studies on animals. One from 2002 showed that the monkeys experiment, a couple of monkeys died, some of their livers were enlarged. But the company carried on using PFOA for another decade in the UK.

19:54Now, what they say is that these toxicology studies weren't conclusive enough, more had to be done. Once the EPA did more risk assessments, then they started to, you know, put in more protections for their workers, because they were also sharing the levels of PFOA in the blood of their workers. That's what the document suggests from at least 2004, again, with 3M and DuPont. So the company says that they started to kind of phase it out voluntarily under the direction of the US EPA in 2006. Liana Hosea of Watershed. Well, we did have a short statement from a spokesperson for AGC Chemicals. It goes, since the wider multi-agency investigation is still ongoing and no formal determination of land contamination has been made, we can't speculate on future outcomes or next steps.

20:45We stopped using PFOA 14 years ago and testing blood for PFAS does not predict blood problems and can cause unnecessary concern.

21:15These veteran machines can pick up the latest precision tech with ease. You know it. And fans ready to add John Deere certified pre-owned equipment and more options to their lineup can shop at machinefinder.com. With a subscription to the BBC website and app, you can now get early access to your favorite BBC podcasts. Listen to shows like You're Dead to Me, In Our Time, Good Bad Billionaire and more. Plus, there's all the other great benefits you get with a BBC subscription, from live streaming the BBC News Channel and acclaimed documentaries to in-depth analysis and features. Visit bbc.com forward slash subscribe to find out more.

22:10Andrew Garfield. It tells the story of the Peasants' Revolt in England, a ferocious rebellion against the tyranny of King Richard II in the late 14th century. For Greengrass, the story still has strong reverberations today. He's been talking to my colleague Nick Robinson. What was really eerie to me was how familiar the world was, because essentially, what was it? It's a world of oligarchs jealously protecting their wealth and power, a failed state and an enraged population and them being blind to the degree of rage out there and then all it takes is a spark and that's really at the heart the story of the Peasants' Revolt The angry, in inverted commas, peasant ordinary folk could be the January 6th protesters they could be the people on the street who loathe Donald Trump, they could be pro or anti-Brexit Absolutely.

23:03And I think wherever your politics are, by far the majority of people would agree, I think, that there is a rage out there and an elite, however you define it, the political class, the wealthier members of society, the winners as opposed to the losers, the South as opposed to the North. However you look at it, whether it's here or in America or in Europe, there is that sharp division, rising inequality, that's leading to instability. And so that's ultimately what's behind the film. There's a moment that I thought was very powerful where the king, Kim Richard II, tells his courtiers, listen, he says, can you not hear their rage?

23:47It is time for us all to taste humility. Isn't that what's going on? There must be a thousand conversations literally half a mile from here in Westminster where people in power say versions of that same thing. Listen, can't you hear their rage? So the film, which caused a bit of controversy, ends with a kind of Robin Hood vibe with some optimism. And that was self-conscious. For sure. Of course, the Peasants' Revolt was crushed. Most historians, I think, would agree that what happened there echoed down the centuries. People who ruled Britain and wider could never quite take the people for granted after that.

24:32You know, after the Peasants' Revolt was the great flowering of medieval outlaw balladry, in particular Robin Hood. When people are going through hard times, they want to imagine characters out there who stand up to power. And so, to me, it was entirely plausible that my character, Ploughman, would, like other young people mixed up in that revolt who survived, would go to the woods and create a different future and would keep those dreams alive and would, in a sense, become these mythic outlaw figures. The director, Paul Greengrass, talking there to Nick Robinson.

25:27This is the BBC World Service in London. You're listening to NewsHour and I'm James Menendez. After the encounter on the tarmac at Joint Base Andrews on Wednesday, today it is down to business as President Trump hosts the Chinese leader Xi Jinping for a state visit to the White House. Things are due to get underway within the half hour. They have got a lot to talk about, even if expectations of real progress seem to be fairly low. Well, on the line is Celia Hatton, the BBC's Asia analyst. Celia, good to have you on the programme. I mean, first of all, how is this meeting? How has it been perceived in China, as far as you can tell?

26:08I'd say, James, that the mood music leaning up to this summit has been really strange in Beijing. So a lot of the processes we normally see leading up to a big summit like this, there are some indications maybe about what Beijing is thinking in the state newspapers. We get some indications from the Chinese foreign ministry, little tidbits here and there about what's being hoped for. But this time, from what I was hearing from Beijing's diplomatic community, there were signs that really Beijing felt that the United States was distracted, that the Trump administration has been quite distracted in dealing with Iran.

26:51So there wasn't the normal series of meetings leading up to this summit that we might normally expect. Instead, I've been hearing that most of the talks were held just this past weekend, you know, and featured Scott Bessett really talking about trade and the potential of an AI hotline. but some other topics that Beijing really wanted to tackle weren't really on the agenda and so I think it's you know it really is going to be a summit perhaps that focuses heavily on optics, on protocol that really is going to be dominating a lot of the meetings in the White House today. Yeah interesting I mean normally these things are so carefully choreographed aren't they and worked out in advance as you say.

27:35Let's talk about AI because it is what we're talking about today. This is our top story, concerns about what AI can do. Is China in any mood for an announcement on regulation, greater oversight? I think that if you'd asked me that back in May, when Donald Trump and Xi Jinping met in Beijing, I would have said yes. I felt that China really was also thinking that AI was developing so quickly. And because at the time, the United the States was still really the leader in AI. And so I felt that at the time, Beijing also was looking to engage in some serious discussions around guardrails for AI. But I think that has changed as well now.

28:21I think both sides, there have been accusations of cyber hacking and espionage on both sides. I think China's own AI models have developed so quickly just in the past few months. And China really is threatening U.S. dominance on this. And so I think that trust has fallen away on both sides and the appetite to work out an overarching framework has really fallen away. So we're looking at perhaps maybe just settling on an agreement for future discussions. Scott Besant has announced a proposal for this AI hotline for both sides to alert each other. But again, there are a lot of questions around what that hotline might actually achieve.