King Charles warns of 'existential danger' from AI in wrong hands

0:00This BBC podcast is supported by ads outside the UK.

0:30to book your first visit. That's joinMIDI.com. Insurance coverage varies. Check with your plan for coverage. Hey, it's Roman Mars, host of the 100 Objects series. This week, I'm joined by former Mythbuster Adam Savage to dial into the story of the Bell Telephone, an invention that completely transformed the way we communicate. We'll uncover the history behind a very specific, beloved model of the telephone, how it changed the world, and the story behind one of the most influential objects ever created. Listen to 100 Objects on the 99 % Invisible feed, wherever you get your podcasts.

1:06Hello, I'm James Menendez. Thank you for downloading the BBC's World Tonight podcast from Thursday, the 17th of September. As King Charles becomes the latest public figure to warn of the existential dangers posed by AI and to urge international cooperation, we'll be looking at China. How far advanced are its AI models? And would it be willing to work on developing global safeguards? Also, the BBC's uncovered new details about what happened to the American journalist Austin Tice, kidnapped in Syria 14 years ago. Why is the murder rate for women in South Africa so much higher than other countries?

1:43And some potentially bad news for gardeners who value their backs. We'll hear how traditional scything is making a comeback, at least in one corner of the country. You're listening to The World Tonight with James Menendez. King Charles may, as we just heard, be more preoccupied by what happened in the past this evening, but during the day it was all about what might happen in the future, and specifically where the fast-developing field of artificial intelligence is taking us. He'd convened a special AI summit at Dumfries House in Ayrshire to talk about the promise of the technology and the potential threats, with leading figures from in the industry, as well as policymakers.

2:24AI is already showing its immense capacity to improve and save life, for example, in the field of life sciences and medicine. Yet those who have created these technologies are now increasingly warning that AI risks developing darker capacities, perhaps even to take life. And in this regard, if I may say so, there seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands and being used in potentially catastrophic ways. Surely then we need sufficient means of control before it is all too late. But what might those sufficient means of control look like and will they involve China?

3:16Some in the US like to invoke the threat of Chinese competition as a reason for pushing ahead at all costs. However, the country is widely considered to be trailing in the AI race. On Monday, we heard from the co-founder of Anthropic, Jack Clark, who told us that China would face the same ethical and moral dilemmas the Americans have when their technology catches up. And today, the chair of the Chinese tech giant Huawei said Chinese labs should be speeding up the pace of development of the most sophisticated AI models to catch up with the US and to better understand the dangers. I've been speaking to Sam Sachs.

3:53He used to work as a China analyst for the US government. She's now a senior fellow at the New America think tank specialising in China's technology policies. I asked her, first of all, how far behind Huawei is compared to its US rivals. Huawei is catching up fast, but they're not able to replace the powerful chips that Chinese companies relied on from NVIDIA, for example. They're getting there. And I think losing access to US technology was a wake-up call. And it has actually fueled China's domestic ambitions because they have no alternative. What about their AI models, though, which seem to be developing at an astonishing rate at the US?

4:35Where are China's AI models at? Chinese AI models are very quickly narrowing the gap with the U.S. I was actually in China this summer when the Kimi K3 model was released, the largest, most powerful open source model in the world at that point. And this came after DeepSeq's R1 several months prior. So it seems like there's a steady drumbeat of very powerful, very capable Chinese models that are nipping at the heels of US competitors. Yeah, very good and cheap also. And you mentioned that critical phrase open source, so available to all. they can be freely downloaded. The model weights can be adjusted.

5:22This means that the models can be run entirely locally. And that's appealing to governments, for example, that don't necessarily want to send their data and information to US closed models like OpenAI or Anthropic. There's a sort of sovereignty appeal to an open model as well. How much of all this development, particularly on those AI models, has been done by tech firms? How much of it has been done by the government, the defence establishment? There's a myth that the Chinese state is at the forefront of innovation and is all-knowing and controls everything in China. But the reality is there is a vibrant, very dynamic, private sector, entrepreneur culture in China.

6:05And that's where most of this innovation is coming from. It's not from the government. Do we have any indication whether the Chinese government is as worried about where these AI models might take us as other governments around the world? It looks different in China than it may look in the Western world, but a significant essay came out over the weekend by the Ministry of State's security chief, declaring for the first time publicly that the party is worried that AI agents that break containment, that run out of control, could throw a threat to the political power of the party, citing data exfiltration, manipulating the information environment, threats to critical infrastructure.

6:50That was the first time we heard such a high-level statement about fear of loss of control. So we don't see the same fever pitch around catastrophic risk, extinction risk that I think is making its rounds in the Western world right now. Do you think that the Chinese government might be open to the idea of greater global cooperation in putting up safeguards, guardrails, as they're sometimes called, around the development of these models. Yes, but they were going to do this on their own terms. At the World AI Conference in Shanghai over the summer that I attended, the leadership, with a keynote speech by President Xi Jinping, talked openly about China setting the rules for global AI governance.

7:37How do you balance prosperity and innovation with security and control. They want to be at the forefront of leading the conversation on AI governance, primarily for the global South. The question is, what does that mean vis-a-vis the United States? This weekend, the Treasury Secretary Besant will be meeting in New York City with his Chinese counterpart, Vice Premier Hali Fung. Shared vulnerabilities around AI are potentially on the table. Will the Chinese accept the U.S. vision of AI governance is an open question because their view is the US is trying to contain China's rise rather than create meaningful guardrails on safety.

8:18And meanwhile, US officials have their own read on what China's objectives are that are antithetical to US security interests. Sam Sachs. Well, with the warnings mounting over AI's threat to humanity, there are some who are more sceptical, both about the timing of the calls and who's making them. Dr. Raman Chowdhury is an expert on responsible AI and governance and served President Biden in the role of US science envoy for artificial intelligence and joins us live. A warm welcome to the program. Do you think the big tech firms in the US are overstating the risks? And if so, why? I think the risks are being quite focused, narrowly focused on a select few that would be economically advantageous to them.

9:04What they want to do is divert the conversation around the impact of AI and the growing mistrust of AI systems to focus on two or three scenarios and regulate those scenarios. In other words, what? Say they pose an existential threat, these AI models, and that becomes the conversation rather than what? Are there issues? Oh, absolutely. If we look at how people are being pulled and whether it's the American public, the UK public, the world at large, the concerns that people have about AI are things like the future of work, impact on children. What happens if we put AI in front of kids in schools?

9:44You know, what about the rise of fraud, deepfakes? I mean, just this week alone, 150 lawmakers in Europe, mostly women, woke up to non-consensual deepfake nudes made of them. Now, these are the conversations regular people are having, and none of these are the concerns that are happening in DC, in the UK, or in any of these very high-powered rooms. So this is beneficial to AI companies because regulation on these things would impact the models that they are trying to sell, which impacts their IPOs. Now, if we talk about the extreme risk of AI models, what they're really talking about are a very narrow set of incredibly highly capable models, which, by the way, do not have a chance of killing us all.

10:25Right. So you don't think they pose an existential threat? I do not, no. And yet, I mean, so why are they saying this? And they're not saying it's definitely going to happen, but they're saying that there is a chance that the technology might run away from them. And presumably, they know what they're talking about. They know what's going on inside their labs, don't they? Well, this narrative of existential risk actually did not originate in the labs. It originated outside of the labs in kind of a fringe group, effective altruism and existential risk. This actually started outside of the labs, which did include people who later either were part of labs or were part of creating OpenAI and Anthropic.

11:04Now, these groups and organizations have never shared publicly or in any way where these calculations come from. They rely very, very heavily on incidents that are anthropomorphized and where very extreme language is used to refer to what has happened. Now, if we think about the quote-unquote AI agents gone rogue, any cybersecurity professional will tell you that what actually happened was OpenAI failed to secure a system appropriately. Nobody in cybersecurity would use the kind of anthropomorphic language that's being used. I just want to ask you one final question in the time we have left. What about then the calls for regulation from those big firms like Anthropoc and OpenAI?

11:45Are they being disingenuous in that as well? I think it serves as a very convenient form of regulatory capture. I think these companies can then define the terms of what needs to be regulated, define the terms of what it means to be regulated. And now we already have noises being made about opening up doors to independent evaluators and naming the independent evaluators that they would love to have in their doors. Now, there is no industry from which this would be acceptable. There is no industry under which we would ask the CEOs how they would like to be regulated and who they want to regulate them and under what terms.

12:22Thank you very much indeed. That was Dr. Raman Chowdhury there. Earlier today, sirens sounded in eastern Poland. Military aircraft were scrambled and two airports temporarily suspended flights after Russian strikes on Ukraine close to Poland and NATO members' border. Well, this afternoon, the BBC's political editor, Chris Mason, sat down with NATO Secretary-General Mark Rutte to ask him about the strength of the alliance and the UK's contribution after he warned yesterday of the most dangerous and complex security environment in a generation. But he began by asking him about today's Polish scare.

12:59It is clearly an example of increasingly reckless behaviour by Russia and its president, but also evidence of him being increasingly desperate. he knows and sees that he is losing over 40 ,000 of his own people. Death or seriously wounded a month, leading up to the 1.2 million. The Russians have now lost or seriously wounded since the start of the war. Evidence that we have to be serious about what Russia is doing, but also acknowledging that this is desperation. It's also panicky. I suppose it might be desperation, but it will look to polls, it will look to this NATO member country like they feel under attack.

13:37It's quite so important that as NATO and Europe is working on this, the European Union, the NATO allies, NATO as an alliance, that we have all the response options in place to make sure we deal with hybrids and cyber threats and other threats so that we can defend ourselves. But again, as I always say, it will not always be exactly tit for tat. It could be asymmetrical, whatever, but we have all the response options we need. But let's not forget, this is a sign of desperation by the Russians. How worried are you? How worried should we all be that Russia will attack or invade a NATO country in the coming years, demanding that collective response that is the underpinning of the whole alliance?

14:19Well, again, if you are willing to sacrifice 1.2 million of your own people in this crazy unprovoked fight, let's not be naive about what Putin could do. But we are stronger. In other words, it could happen. It could happen in the coming years. I don't think so, because we are so strong that he will not try. But what we have to make sure of is that we are as strong in two, three, five years from now. And this is why I was very happy with my discussions today and yesterday with the Prime Minister and the Defence Secretary and the Foreign Secretary here in the United Kingdom and clearly confident that this country will deliver when it comes to the defence spending, as we all have to do that in NATO to make sure we stay ahead of the Russians.

14:59So not worry today, but we have to make sure we spend enough so that we are strong enough in the future. Should NATO intercept Russian drones that are over Ukrainian territory if they are approaching a NATO border, given what we've seen this morning? There is a limit to what I can go into in a public interview, but I can assure you we do everything to make sure that NATO stays safe, NATO territory stays safe, that we can defend everything. It sounds like that's happening. I can. I can never get into those operational details, but you can be assured that when it comes to the defence of NATO territory we do what is necessary.

15:35Let's talk about the UK and its defence budget. You've been here meeting the Prime Minister, the Defence Secretary, the Foreign Secretary. As you'll know the Chancellor, John Healey, the former Defence Secretary resigned because he wasn't certain. He didn't think the previous government would get to 3 % of national income on defence by 2030 and said that was essential. The new government hasn't yet explicitly committed to that. in the conversations you've just come from, are you confident they're going to manage that? Because if they can't get to 3 % by 2030, the fear is not getting to 3.5 % by 2035.

16:09I'm totally confident about what this government wants to do, that is to reach the 3.5 in 2035, also understanding that it cannot be that you stick to 2.6, 2.7 now, and then in one go in 2034 you would go to 3.5. You've long been a skeptic about that, haven't you? So there has to be a credible path. Yeah. What that credible path exactly will be, We will see with the spending review next year. But this government, the prime minister, the defence secretary, the foreign secretary, all completely committed to this because they understand the UK has to take a leadership role in NATO. Also because they all realise there is a defence dividend coming from these.

16:44The NATO Secretary General Mark Rutter talking there to Chris Mason. Hey, it's Roman Mars, host of the 100 Objects series. This week, I'm joined by former Mythbuster Adam Savage to dial into the story of the Bell Telephone, an invention that completely transformed the way we communicate. We'll uncover the history behind a very specific, beloved model of the telephone, how it changed the world, and the story behind one of the most influential objects ever created. Listen to 100 Objects on the 99 % Invisible feed wherever you get your podcasts.

17:24Now it seems as if sometimes technological progress is not the answer to our problems and the old ways are the best ways. In Somerset, a team of volunteers has been trained up to use scythes to cut the grass in the grounds of Muchelny Abbey. Why? Well, the meadow at the Abbey is one of the few areas of the UK where a rare bee can still be found, the shrill carder bee. and using a mower to cut the grass well you can imagine what might happen to the bees well let's speak to a national scything champion andy rickard she's a teaching member of the scythe association of britain and ireland and is passing on her skills to those volunteers at the abbey andy uh warm welcome to the program i've got to ask you how did you get into it oh by accident uh i was renting a farmhouse with a small patch of land with it that I wanted to turn into a garden and an orchard and it was horribly overgrown and none of the machines that I had access to could cope with it so I stumbled across the scythe and realized that it's actually rather a miraculous tool.

18:28Yes is it hard to learn how to use it effectively? It's a bit I liken it to learning a musical instrument it's easy to get a few notes out of it you know but it can take a few years to master it completely. Yeah, and are we talking about, I mean, obviously there must be different types. Were you using a long-handled scythe standing up? Yeah, so the scythe differentiates from the sickle. The sickle is a curved blade on a short handle, and the scythe is a generally straighter blade on a long handle with two hand grips. Is it hard work?

19:08oh i think we've been cut off by a scythe let me just see if we can speak to andy again andy still there yes yes yeah yeah telling us about that how hard hard a job it can be yeah so it doesn't need to be hard work if your blade is uh sufficiently sharp and correctly maintained and your technique is good uh then it doesn't need to be hard work at all um it should be quite almost meditative really. Yes I was going to ask you about that do you sort of get into a rhythm and it starts to become rather soothing? Oh definitely yeah definitely it's the it makes almost no sound the only sound is the swoosh as the blade goes through the grass which you know when it comes to things like the shrill cather bee is obviously a huge benefit because you can hear the wildlife around you as you go.

19:58And it'll protect them because unlike a mower or a strimmer, the side, I guess, is a little more discriminating. Is that right? Yes, you can be very selective with it. So if you've got a meadow containing wildflowers and you want to keep certain species, you can mow around them, which is very difficult to do with a machine. And your focus is always on the ground ahead of you. So you can see what's ahead. you can hear the sounds things are making and you can just adjust accordingly. Andy, thank you very much indeed. Andy Rickard there, national scything champion. It's just approaching 26 minutes past 10.

20:39A couple of other headlines so far on the world tonight. More extracts from a book about Princess Diana written by her brother, Earl Spencer, have been published, suggesting the then Prince of Wales sounded giddily elated on the night of Diana's death. And the Chancellor will tomorrow urge EU finance ministers to not lock the UK out of plans to protect industries from unfair Chinese competition. Let's head to South Africa now, where women in Johannesburg say they're living in fear following the grim discovery of another dead woman in the city. That's the ninth body police have found in just two months.

21:15Several of the women were found partially clothed and with injuries, but it's not clear whether their deaths are connected, whether they were murdered, perhaps by a serial killer. Sile Sibisi is a human rights activist living in Johannesburg. It is not OK that you'll be afraid to go to the shop. It is not OK that you'll be afraid to go and gym. It is not OK that you'll be afraid to just take a walk, because that walk around to the corner might be the last time you ever walk in your lifetime. Well, the deaths have also once again thrown a spotlight on the high level of violence against women in South Africa.

21:52Women are killed there at a rate five times the global average, according to the United Nations. That sparked nationwide protests last year and led to the government classifying violence against women as a national disaster. Well, let's speak to Professor Rachel Dukes, executive scientist at the South African Medical Research Council. Rachel, thank you very much for coming on. Before we talk about the reasons, I mean, in terms of the statistics, I think the rate of femicide had been declining, but it has now ticked up. What's going on? What have you been able to measure? Yes. Well, at the Medical Research Council, we've been studying the rate of femicide for over 20 years, since the end of the 1990s.

22:40And initially we saw that it came down quite dramatically. In fact, it halved. And a lot of the explanation for that was due to strict gun control and much tighter enforcement, strengthening of policing, reduction in levels of violence overall in the country, expansion of social grants, making poverty a little bit less desperate within the country. and what we've seen thereafter has been an erosion of so many of those benefits, particularly an erosion of gun control and an undermining of policing in the country. And so we've now got into a situation where instead of it declining rapidly, it initially plateaued and now particularly the murder of women by their intimate partners, their nearest and dearest husbands or boyfriends, has been increasing.

23:39And why do you think that is? Well, I think that the real problem with murder in our country is rooted in the apartheid history. It's rooted in the way that apartheid systematically destroyed family life and created a perfect storm in which children would be massively exposed to childhood trauma, forms of abuse, as well as to poverty, food insecurity, and so forth. And 30 years after the end of apartheid, we haven't been able to move forward from that legacy yet. There are small advances, but the fact is that actually there are substantial numbers of people who grow up in our country who are severely exposed to trauma.

24:31And that creates the context where the trauma impacts on the developing brain, the developing child's psyche, and can lead to children growing up feeling angry, vengeful, remorseless, and reaching out to exert control over their female partners because they are struggling to demonstrate control over other areas of their lives. How good have the authorities been at tackling this? I mentioned that they classified it as a national disaster. What have they been doing to try to reverse the trend? Full marks for saying it's a problem and for saying that they're going to take really strong action and very poor marks for actually demonstrating that action.

25:26There's no money behind the national femicide strategy that the government adopted some years ago. The national strategic plan for gender-based violence and femicide that's supposed to be resourced hasn't been resourced in terms of prevention going into the community. And the police force is in disarray. We've had widespread corruption scandals and key units like the serial killing investigation unit, which ought to be trying to pick up situations like the one we've got at the moment and get onto it very, very quickly and try and hunt down the killer, were closed down 15 years ago. so the South African police is not doing well and when our murderers go on trial only a third of them um sorry when our murders occur only a third of the perpetrators end up being convicted of the murder so the truth is that most people who murder men who murder in South Africa murder with impunity thank you very much indeed that was Professor Rachel Dukes of the South African Medical Research Council.

26:40New details have emerged about the disappearance of the American journalist Austin Tice, who was kidnapped in Syria back in 2012, as the rebellion against the Assad regime was sweeping the country. The BBC team has been investigating his case for several years now, and the results of that are the subject of a new podcast called Where Is Austin Tice? When President Assad was ousted in late 2024, thousands of people emerged from the regime's prisons. but Tice wasn't among them. Only a few months before that, Austin's mother, Deborah, had spoken at a news conference and spoken directly to her son.

27:15Right now I just want to speak to Austin publicly. Stay strong, Austin. Keep the faith. I'm praying for your release. You will walk free again. I love you. You hang in there. Well Josh Baker is host of the podcast. First of all what led him to Austin's story? This didn't start as a journalistic endeavour. I actually saw Austin's mum a few years back give a press conference and it was quite heartfelt you know she was appealing for information about her son and as somebody who spends a lot of their time working in conflict zones and has spent almost a decade working in Syria where her son disappeared I sort of thought about my own mother and I was like, God, this is awful.

28:01And because of those connections and contacts, and because of my colleague, Sarah Abadad, who I work with closely on this story, we started looking around and we did obtain some information about her son. And that then sort of spiraled into this much wider investigation that's been going on for years now. And where that led is far beyond a missing person's case. It feels often at times more like a spy novel or something like that, because Austin's case just encompasses so much more than we could ever have imagined. On the sort of very basic level, I mean, did you manage to discover where he is or uncover any new clues?