Industry voices warn of AI 'danger'

0:00This BBC podcast is supported by ads outside the UK.

0:30most to investors, including policy initiatives for retirement savings, taxes and trade, inflation concerns, the Federal Reserve, and how regulatory developments can affect companies, sectors, and even the entire market. Mike and his guests offer their perspective on how policy changes could affect what you do with your portfolio. Download the latest episode and follow at schwab.com slash Washington Wise or wherever you listen.

1:14Hello and welcome to NewsHour from the BBC World Service, live from London. This is Owen Bennett-Jones. The number of people worrying about AI is increasing. President Xi of China has said that the whole international community should answer questions such as how do we decide when an algorithm becomes part of decision making? In the West, there's growing opposition to data centres, partly because people are not sure AI will actually benefit them. This week, in the US, three AI industry leaders, Elon Musk, Sam Altman and the boss of Anthropic, said AI developers should slow down. There are also not just concerns at the top, but also within the AI companies.

2:03A number of members of Anthropic staff have resigned, citing worries about the pace and extent of AI development. So first of all, let's hear from a couple of them. Jacob Coxon has been speaking with the BBC's Laura Koonsberg. Does he think the call of the three industry leaders for change goes far enough? So there's one thing missing, which is buy-in from China. It looks like it's a US thing and it looks like, yeah, like you said, Samuel and Elon Musk have agreed that this is a great idea. But I think, yeah, there'll need to be some sort of coordinated slowdown with China if we can avoid a race at an international scale.

2:39And I guess we'll see if this sort of thing happens. It's one thing for the big bosses to put things out into the public domain to say, hey, let's work together. Let's have a pause. Do you think the big platforms want regulation? I can speak to the people who work there, and I can say that they are genuinely frightened. They're genuinely concerned about the fate of humanity in the next two years when they are planning their lives, planning what to do with their lives. They are thinking about the impacts of their work. Like they may be like considering buying land somewhere because they're so scared of the instability as a result of rapid AI progress.

3:13All I can say is the people who work at these companies are completely serious when they ask for regulation because they find themselves trapped in a race and they're scared of the outcomes of that race. That's Jacob Coxon, who left Anthropic, but so did Joe Benton. So we'll hear from him, too. He's been talking to Paul Henley again about the comments made by the three AI bosses. I think that the pace of AI progress is extremely rapid at the minute. To the extent that AI companies are observing those risks and are willing to make sacrifices for their companies to address them, that seems extremely good.

3:51I do think the proof will be in the pudding and we should see whether they are actually going to stick to these commitments, whether the companies that they have said they will let assess these models actually do get the levels of access they need to kind of make formal judgments on the risks. Doesn't it also seem a bit shocking that until very recently they weren't interested in controls? I do think it seems basically unacceptable that there is no independent checks on these companies, no way for external parties to tell how much risk they are imposing on the society. I think the levels of risk here could be really quite large if the pace of progress continues as it currently is doing.

4:30Why are there no safety rails? Who's in charge, the AI companies or governments? Because this technology until recently was not extremely powerful. The default position has been to let companies essentially self-regulate. And I think that we're seeing that that's no longer adequate. I think that three years ago, these models, they were fairly functional chatbots. And now we're kind of at a position where they are as strong or stronger than the leading human experts in some field. I think they're going to be substantially more powerful than that again in a relatively small number of years. And so I think that it's no longer acceptable for these companies to self-regulate.

5:06And that was Joe Benton from Anthropic. He left the company just on Friday. And actually, he was in AI safety there, a researcher on that issue. Well, I spoke earlier to Dr. Maurice Kiodo. He is a mathematician and assistant research professor at Cambridge University's Centre for the Study of Existential Risk. So I suggested we took it for granted that there are going to be lots of benefits to AI. But just to look at the other side of it, how would he categorise the main risks? There are sort of two sets of risks here. The first are the more, I suppose, commonly known risks, the ones that are maybe idealised in Hollywood films, things like this.

5:49The ideas of AI producing some forms of weaponry or other direct threats to humanity, killer robots, drones, things like this. That's a pretty clear risk. Or we've seen recently cyber attacks that swarms of AI agents have been carrying out recently. We've seen that in the news. So that's sort of the first more obvious, more visible threat to society. But then, to me, the secondary threats and the ones that are actually, I think, more pressing and already happening are the ways that AI systems get into existing human systems, human processes, and start to mess things up, start to sort of perturb and disturb how we do things.

6:29We've seen examples of engineers, I think this was at Amazon, AWS, using our agents to do things on live servers. And the agents decided that, oh, it's a good idea to wipe the entire code base and start again. And the shutdown of Amazon's deployment servers causing mass disruption, I think, at the end of last year. All their clients that were relying on that shut down, knocked off some sort of pretty essential, pretty important services around the world. So you start to see AI in its current form already posing risks to society. We see people over-relying on AI systems, ALMs to do things like create advertising campaigns.

7:06Seems like a pretty innocent thing. And in South Korea, Starbucks, they create an advertising campaign based around a massacre, or themed on a massacre happened several years ago there because I thought, well, this is good attention. Well, it's certainly good attention, but for all the wrong reasons. So that's very interesting and a good specific example. Just one thing before we go on. When you say AI agents, it's a slightly confusing term for some people. You basically mean computer programs, right? Not people. That's right, yes. Computer programs. So there are ways to set an AI process in motion, and it goes and it does a whole bunch of stuff, kind of as and how it pleases.

7:41You give it sort of some high-level task, go and book me a flight from A to B, and it goes through and it tries sort of, I don't know, log on to a flight booking system, and you've given it your credit card details, so it uses those, and it books a flight for you, puts your details in, gets confirmation email, selects your baggage, all this sort of stuff. operating to do something that you want to do, but the particulars are decided by this program. And so this is quite a common thing to set off large numbers of these little programs that are run by AI to go and do various tasks like that. Right.

8:12So here's where I struggle with this. Surely it is possible to instruct AI at a very fundamental level not to do certain things that are problematic, you know, breaking the law, all sorts of things that surely AI could be told not to do. The difficulty is that these systems, they don't have the common sense of your average adult in society. These things are very smart, but they haven't developed the experience of what we would commonly consider as a dumb idea. The good thing about this is imagine you have a very gifted child, but they're still seven years old. So they might be able to sort of process some things, do some technical tasks very, very well.

8:56but they don't quite have the common sense that their adult parents would have. And so you set this child off to do some task unsupervised, and lo and behold, after some period of time, the child does something that the parent would think, oh my God, why have you done that? That's a really bad idea. But to a 70-year-old child, this doesn't seem like a bad idea because they haven't got that level of adult maturity that adults have. It takes adults over 20 years to get that level of maturity, if not more. But children don't have that sort of maturity. And if you think of an AI as a gifted child, that starts to picture in your mind the sort of things you can and can't expect it to do.

9:33We've got these senior people, the people at the top of the industry now saying we need to do something. It doesn't ever seem to happen, though, does it? I mean, I guess the competitive pressures between the US and China, between the American companies are so great that there is resistance to actually doing anything. The competitive pressures aren't just between nation states or economic. you've got the competitive pressures of those trying to build these. You mentioned the kingpins that are running these organisations. You've got very large personalities, very large sort of people, and they've got some mission, they've got some desire to see this thing through.

10:08This is kind of their baby. They're not fundamentally inclined to try and restrict its growth. They birthed this thing. They wanted to grow as quickly and as fast and as well as possible. And by as well as possible, I mean in a very sort of narrow sense of how they see our development. They want it to be as clever, as smart, as all-knowing as possible. That doesn't necessarily extend to being as responsible, as safe, as socially aware as possible. Their measure is perhaps a more computer science, technical type measure of, oh, look, my AI toy can solve all these math problems now. That's great.

10:40Well, yes, but if it sells your house every time you ask it to reduce your home insurance premium, you've not really got the right objectives here. So they have their own internal aims to keep it growing. And so that's why you've seen resistance. Why would you want to slow down your own child from being better and learning more and being able to do more things? It goes against the entire reason they're doing this. And the risks analysed there by Dr Maurice Kiyodo from Cambridge University.

11:10Countries such as Greece and Indonesia, with lots of islands, inevitably have to rely on ferries. And in both places, accidents do happen. And the news from Indonesia is that a ferry has capsized with more than 240 people on board. Let's speak to the BBC's Lisa Tambunan, who joins us from Jakarta. The original reports actually said this ferry had sunk, but it's capsized, it seems. Is that right? It is right. Search and rescue teams had earlier reported that the boat had sunk. but this was later rectified by the transportation ministry saying that it had flipped and not yet sunk. This boat had capsized since last night after it encountered severe weather.

12:00Hi, can you hear me Lisa? Hello. Yes, it encountered severe weather, yep, so that was the the problem and so just give us a bit of a picture of you know what's happening at the scene because I mean, presumably, if it's capsized, there is the chance that people are still alive in air pockets. Yeah. So the ferry was traveling between Java and Borneo when it lost contact about 2 a.m. earlier this morning. by daylight. Nearby vessels had rescued dozens of passengers and authorities also deployed search and rescue efforts, both by sea and air. This had been going on all day and by now it's already sundown.

12:52We understand that some of those who were rescued have arrived in Bandarmasin, the destination that the ferry was heading toward. and these passengers are receiving medical treatment at the moment. Search and rescue for the remaining passengers are still ongoing. This has been confirmed by authorities. And I mean, how common is this in Indonesia? It is a problem whenever there are ferries on this sort of scale. It does happen. Yes, it's, well, we have to keep in mind that Indonesia is composed of many islands. In fact, more than 17 ,000 islands. So people here heavily rely on boat connections for daily transport and also tourism.

13:38So and previous accidents have happened with investigation results showing that there's lax safety standards and unpredictable weather as the main reason for these incidents. OK, well, thanks very much for bringing us up to date with that. That was the BBC's Lisa Tambunan in Jakarta.

14:09Coming up, Swedes head to the polls today in a tight electoral race, according to the polls, that sees the country's left-wing bloc go head-to-head with a right-wing coalition, including the heart-ride allies, the Swedish Democrats. They have been very much mainstreamed, but they're still, you know, a very radical party in the views of many voters. and the other parties have had to adjust. And particularly then on the centre-right side, where the first approach was to basically ignore them, that shifted ahead of the last election and now they've been a well-integrated support party for the government.

14:43As we've been hearing, the authorities in Indonesia say six people have died and more than a hundred are missing after a passenger ferry capsized in stormy seas. And the head of OpenAI has said the company will not make its stock market debut this year because of concerns about the safety of artificial intelligence.

15:09In Paris, for the first time in six years, the multi-million selling French-Canadian singer Céline Dion has performed a live concert on stage. The show was the first of 16 which she's putting on in the French capital over the coming weeks. Sounds like a demanding schedule, given that she has SPS, stiff person syndrome, meant there really were times when it was thought she would never perform again. She got the diagnosis in 2022. It is a rare, incurable neurological autoimmune disorder. There are only around 8000 people in the whole world who've got it and it causes severe muscle stiffness and spasms.

15:51She has said the condition makes her feel as though she's being strangled. It's when the whole neck sort of tightens up. Well, Ilya Khan is a doctor who has the condition. She was diagnosed when she was 31 years old. She now works mostly from her bed. Is she surprised that Celine Dion was able to perform with stiff person syndrome? I'm not. If you told me a person has stiff person syndrome many years, she's been struggling with it. I might say, yes, being on stage, that's going to be, whoa, too much. However, having lived it and discovered that this disease is all about personal management, it is a spectrum disorder that can have some people walking and some people bedridden.

16:39It can have, you know, some people working, some people not working. So those big swings, I think, are what are so confusing for people that don't have this condition. You're bedridden, right? I'm in the bed yet 20 hours a day. And does the condition fluctuate? You know, do you have good days and bad days? Right. So she might be able to predict that to some extent and perform when it's good. And even when we're having bad days, if we know that there's something that evening, we're preparing. I'm going to take these meds at this hour. I'm going to put on these clothes at this time. And with our medicines and immunomodulating chemotherapeutics and what we've got to manage, if you're lucky enough to have those resources can really make you look great while you're out.

17:28And it's almost like it's a double-edged sword in many ways. But if you look good when you go out because you rested, you took the right meds, you planned. People think in a way that you don't have this disease. And that's why there was concern in the patient group that Celine Dion going on stage in a way was making people think this is not such a bad condition to have. Or that it's very manageable if only we find the right thing. And if that was the case, we would say, what do you want? And then we'd all go on it and we'd all be better. But so I like to give her some credit that, you know, it's not all better.

18:08It's that she knows she's got the schedule and she can make it work, you know, as a medical director for eight months, making it work with a central line catheter in and hiding it with turtlenecks. So simply, I would put it that this is an autoimmune disease. So your own immune system is crashing down on the enzymes that put breaks on the nerves. So you're fast in all of your nerves. It's why we're rigid motor. And then it also why our throats can tighten up and we need Botox around our vocal cords, et cetera. But it's also a fast mind disease. So, you know, your mind is nerves. And so that's where the psychiatric symptoms come in that keep us inside.

18:51And so it's not just the falls and the physical rigidity, but it's also all of that psychological damage that makes you not want to be your best. And that was Ilya Khan speaking about stiff person syndrome, which is the condition that she and Celine Dion have.

19:27A beautifully designed Squarespace website is the difference between thinking about photography school and selling your portraits online. It's the difference between giving away those cookies you just made for free and running your cookie empire. A beautifully designed Squarespace website is the difference between having a movie reviewing hobby and a movie reviewing career. Squarespace, because no matter what you're into, a website makes it real. Use offer code THE DIFFERENCE for 30 % off every plan. Terms and conditions apply. Get more with BBC Podcasts wherever you listen. Be the first to listen to your favourite shows like Evil Genius, Good Bad Billionaire and You're Dead to Me with a subscription to BBC Podcasts Premium on Apple Podcasts.

20:09You can also enjoy a range of our podcasts ad-free with an Amazon Music subscription.

20:19And welcome back to NewsHour in Sweden. The Conservative government has ruled now for four years, and it has had parliamentary support from the Sweden Democrats, and that is a party with neo-Nazi roots. The party was long treated as unacceptable outsiders, but then went into this coalition. Opinion polls indicate a tight race between that government alliance and the centre-left opposition. So what are the key issues on voters' minds that will determine all this? The BBC's Stephanie Zacharsson reports.

20:57On a cold and rainy day in March, demonstrators gathered in a nationwide call against what had been labelled teenage deportations, where young people who had lived in Sweden for many years were told they no longer had the right to stay. They had turned 18 and were no longer deemed to be dependents of their foreign-born parents. Headlines across Sweden told the stories of youngsters who'd learned Swedish, studied, made friends, to then be sent to a country that many of them hadn't set foot in for years, prompting public outrage against the strict immigration laws. The centre-right government was forced to backtrack and pause the deportations.

21:38But the tough stance on immigration has been a trademark of Prime Minister Ulf Kirsten's coalition, which is governing with the backing of the far-right Sweden Democrats. Four years on, the number of asylum seekers has plummeted. Swedish citizenship now requires longer residence and a knowledge test, and residence permits are strictly temporary. This is largely due to the leverage of the Sweden Democrats, a party founded by neo-Nazis and long treated as a pariah, but now the country's second-largest political force. They've had a very big influence. Jan Fiorell is a professor of political science at Stockholm University.

22:18They have been very much mainstreamed, but they're still, you know, a very radical party in the views of many voters. And the other parties have had to adjust, and particularly then on the centre-right side, where the first approach was to basically ignore them, isolate them, not work with them. But that shifted ahead of the last election, and now they've been a well-integrated support party for the government and have had very much policy influence in the last four years. When asked about Sweden's dramatic shift in immigration during his time as prime minister, Ulf Kriesterjahn said his government had fulfilled its task of cleaning house and must now focus on integration.

22:57When he came into office, shootings and explosions were an almost daily occurrence, with turf wars between drug networks plaguing many Swedish cities. Crime and migration remained central to the government's case for re-election as voters choose between the parties that make up Sweden's two main political coalitions, the governing centre-right bloc and the centre-left, led by the former Prime Minister Magdalena Andersson Social Democrats, the largest party in every election for more than a century. Most polls have put the opposition ahead, but they are more splintered ideologically than the right.

23:36The main thing uniting them is their opposition to the Sweden Democrats. So those elections today, and that was Stephanie Zacherson reporting.

23:48The issue of who controls the Arctic has been live for many years now, but it took on more urgency when Donald Trump started talking about taking over Greenland. And that is the backdrop to a summit in Finland, which starts today. It brings together some EU leaders worried about the intentions of China, Russia and maybe the US too. and I'm joined now by Professor Hannes Hansen-Magnussen from Cardiff University, a political scientist who specializes in the Arctic. So tell us, what do you think? Well, let's just begin with China and Russia. You know, people talk about these threats, but if everyone's trading and just sending ships through, what is the problem?

24:28Why is there perceived to be such a problem in the Arctic? Well, I mean, there are different things happening at the same time. And China, I think, sees the Arctic as a space of opportunity to establish trade routes with Europe and North America. And for Russia, it's more a space where it can kind of reassert itself as a global power. And the Arctic has tremendous symbolic value. And these two developments, I think, are new to the Europeans and also to the US in Canada. And they somehow have to deal with this. So when you talk about the Russians asserting themselves, is that with military presence?