Dario Amodei from AI's Anthropic issues a stark warning

0:00This BBC podcast is supported by ads outside the UK.

0:33If you are currently overpaying on software to run your business, remember this number, 10 ,000. That's the number of new businesses that join Odoo per month. Join Odoo today at odoo.com. That's O-D-O-O dot com. This is the Global News Podcast from the BBC World Service. I'm Pete Ross and in the early hours of Sunday the 13th of September these are our main stories. Three AI bosses call for the development of artificial intelligence to be slowed down due to concerns over safety. The right-wing party Reform UK gets another donation of nearly 50 million dollars, the second in as many days from a cryptocurrency billionaire and The first way of thinking about it that I had was maybe it's not pointless for people to see what terrorism looks like.

1:28I'm going to tell the story and the attacker doesn't get to be the storyteller. The author Salman Rushdie discusses a new film about the vicious knife attack that left him blind in one eye.

1:42Also in this podcast, at least 16 people have died in a fire at a nursing home in Chile. and in tennis, Yelena Rabakina has become the US Open Women's Singles Champion.

1:56When artificial intelligence first hit the mainstream, it was promised to be a game changer for economic growth and human well-being. But in a major shift, the CEO of one of the US's leading AI companies, Dario Amodi of Anthropic, published a lengthy post warning the industry to slow down before safety risks get out of hand. Elon Musk and the OpenAI boss, Sam Altman, both said they agreed with him. Last week, a couple of Anthropic executives resigned. One of them, Jacob Coxon, told the BBC that, quote, if we don't slow down at the current rate of progress, there's a strong chance that we could all die in the immediate future.

2:35Our North America technology correspondent is Lily Jamali. This essay from Anthropic CEO Dario Amode has really made the rounds in Silicon Valley. throughout this morning and into this afternoon here, nearly 4 ,000 words long, he lays out his concerns that the risks associated with artificial intelligence technology are very serious, even though he also says there are tremendous benefits so that it doesn't make sense to stop developing it, but that the pace of development should slow. So he proposed creating a system that, first of all, gives governments a little more time than they currently have to properly regulate this technology.

3:17And he also floated the idea of monitors that would be, you know, sort of independent third parties with the same access that company employees have to the models so that they can keep tabs on what's happening both as the models are being developed and then later also when they're deployed. Lily Jamali in San Francisco. Gina Neff is a professor of responsible AI at Queen Mary University of London and an expert on what the technology can do. Owen Bennett-Jones spoke to her and asked if universal regulation is getting closer. They're saying regulate me with huge caveats. Dario Amadei is calling to, quote, pace the frontier of AI development just as the company are floating a$1.6 trillion IPO.

4:08His message is clear. Anthropics say they have such powerful technology that only they should be entrusted with it. The sentiment is both undemocratic and also cynical. But I think we all need to be heeding Amadei's warning here. The technical development of AI right now outpaces the social, legal and regulatory advances that we're going to need if we're going to use AI safely in society. So are you saying that Amodei, the head of Anthropic, is basically saying Anthropic's, yeah, he's thinking, Anthropic's in the lead. So it's fine if we freeze developments just now because it'll leave me ahead.

4:43They absolutely want to pace right at the moment when they're in the lead. OpenAI just announced that they are putting a pause on their own IPO because it would be unsafe to do so at this particular moment. Anthropic is going ahead. What they're calling for in this regulation, though, is thin when it comes to regulation. It's like regulation proposed by a high school dropout of Model UN. They would hire independent evaluators to sit in-house with the company. Now, we've seen that story of businesses doing self-regulation before, and it always fails. Amadeus said rogue AI agents could take over the Internet in 6 to 12 months, causing hundreds of billions of dollars in damage.

5:27What does he mean would actually happen? Yeah, this is coming on the heels of one of Anthropik's regular reports around the cybersecurity abuses and other harms that they've detected within the company. What they can see is that bad actors around the world are using clawed tools like ordinary people do to do some bad things. And they're worried that as we move to what's called the agentic AI moment, that these agents might not be able to be controlled. They want good people. And in this missive he published, democratic people and people in the U.S. to control AI and not other countries in the world.

6:11The report has lots of examples of China, Iran and Yemen using clawed tools to do bad things. You've used this word agents. It's slightly confusing. And basically what that word means, rogue, not rogue agents, really, rogue computer programs. Is that right? Agents aren't rogue. Agents are simply computer programs that can work autonomously without being specifically programmed to do a particular task. So they're given a goal and they're said, crack on. And the challenge here is that they haven't been to school. They haven't been to church. They don't have human discernment. So the kinds of values that AI agents have to go about those goals aren't human values.

6:53We might understand if something is illegal or immoral or could really get us into trouble. AI agents don't have that sense of right and wrong. Yeah, and you can't just tell them to obey the law. That's right. You can put in guardrails and you can put in safeguards that make sure these agents aren't stringing together very complex tasks. And that's partly what Amadei is calling for here. Remember, Anthropic was founded five years ago by computer programmers who broke away from OpenAI because they thought that company was not doing enough for AI safety. And now they're in the lead. They're saying they should be the ones to create safe AI.

7:35Gina Neff speaking to Owen Bennett-Jones. Nigel Farage has been a disruptor in British politics for many years and was a key figure in the Brexit movement that saw the UK vote to leave the European Union. Some have even tipped him to be the next Prime Minister. Now, his populist right-wing party, Reform UK, which has been accused of stoking violent anti-immigration protests, has received almost$100 million in donations in the space of two days by the cryptocurrency billionaires Ben Dello and Christopher Harbourn. The party's deputy leader, Richard Tice, said there was nothing wrong with receiving such a huge sum of money.

8:13We should celebrate when entrepreneurs want to try to help a country that's in serious difficulty and provide greater choice to voters. That's the key thing. And to have that support, to know that we can focus on employing the best people, whether it be on social media, policy, campaign managers, that's a good thing. Nigel Farage is already being investigated by the British Parliament. over whether he should have declared a cash gift of$7 million from Mr Harbourn. He has said it's not the public's business to know details of how he spends that money. Reform UK itself is also the subject of a probe by the police over donations to the party which denies any wrongdoing.

8:55The governing Labour Party has said that Mr Farage can't buy his way out of the sleaze and scandal he and Reform UK are drowning in. Margaret Hodge, the British government's anti-corruption champion, said that donations on this scale were of great concern. This does represent a very dangerous challenge to our democracy when you get a couple of billionaires who capture one of our political parties and at best influence and at worst control our democracy through those political parties. For more on the donations, I spoke to our political correspondent, Rob Watson. They are massive figures. I have been covering British politics on and off for about four decades, and I know that these are by far and away the largest donations given by individuals to a political party ever.

9:45There's just been nothing like this. And again, to try and put them in some kind of context, I mean, these two donations amount to more than all the money all the major parties got last year put together. and it's much more money than the two main parties, the Conservatives and Labour, spent at the last general election in 2024. So staggering. Why are these two billionaires doing it? I think, based on what they've said, it's their sort of passionate belief, along with a lot of reform supporters, that goes something like this, Pete, that something's gone profoundly wrong in Britain, much of that to do with immigration, much of that to do with too much wokery, and that only reform and Nigel Farage, its leader, can put it right.

10:30So that is the explanation. But the sums are staggering and historic in their size. And what about these two individuals that have made these huge donations? Can you tell us a bit more about them and why they've got the ability to spend in one go? What is, as we've said, a very large amount of money? So, yes, both of them are fabulously wealthy cryptocurrency investors. they tend to be in that sort of class of billionaire who are sort of very pro-entrepreneur who think that Britain has become sort of tied down with regulation with red tape and it's why donors like these two also tended to support Britain leaving the European Union.

11:10This is a bit of good news for reform then is it? On the one hand it's fabulous news right? I mean what political party would turn down such riches? It's also fabulous news in the sense that there had been concern that some of these very negative stories about reform, Nigel Farage and their financing, might put people off from donating to reform because they wouldn't want to get into the news. Well, now reform has all this money. It presumably won't need to be attracting large amounts of cash. It's also good news, this huge amount of money, if, for example, Andy Burnham, the current prime minister, were to decide to call an early general election.

11:48I mean, they would have lots and lots of money. On the other hand, the problem with donations like this is that it allows your political opponents to say, hang on a minute, reform, you say you're the party of the ordinary person, the working class person on the street, but hang on a minute, you're a party that depends on the uber-wealthy and billionaires who make their money in ways that most people don't understand. But past evidence has shown that when parties have had huge donations, it still won't help you win an election if the message that you're trying to sell is not one that goes down with the voters.

12:23So this massive sum of money does not by any means mean the next election is in the bag for reform and Nigel Farage. Rob Watson. It's four years since the attempted murder on a stage in New York State of the controversial Indian-British novelist Salman Rushdie. A new film about the incident is soon to be released based on never-before-seen footage of the attack taken by his wife, the cinematographer, Rachel Eliza Griffiths, and of his recovery. Justin Webb spoke to Mr Rushdie and the director of the film, Alex Gibney. He began by asking how the project started. And just a warning, some of their discussion contained graphic details about the aftermath of the attack.

13:04Initially, we weren't thinking clearly at all. We were just doing it to have some record of this terrible thing. You know, we live in an age where everything has to be recorded, otherwise it didn't happen. And it was only when I was back home that my wife said to me, was I ready to see this stuff? And I said, OK. And she set up a screen and a projector and we sat together and watched it. And amongst other things, what I thought was that the filmmaking was very good. And I thought, this doesn't look like a home movie. It looks like something more professional than that. Maybe if it's of that quality, then there is some reason for it to talk about it having a wider life.

13:40And eventually we were introduced to Alex Gibney. And he also thought it was good. Alex, you thought it was good. What did you think you could then do with it? Was it immediately obvious to you? Well, it was immediately obvious that there was an extraordinary film here. Exactly what to do with it, I wasn't 100 % sure. In fact, the first question I asked Salman and Eliza is, you know, why don't you do it? And I think they had considered that but felt that it was too close to them. And so that I could perform a service in the sense that I would see it from the outside in and therefore have some sense of how to present it to the audience.

14:14But I did have an idea based on a British TV series called The Singing Detective. And what's interesting about that series is that the simple story is a story of physical recovery. A guy shows up in a hospital with psoriatic myopathy full of lesions and in terrible pain. And over the course of it, he physically recovers. But along the way, he has these hallucinations. He thinks back to his childhood. He imagines bits and pieces of his own detective fiction sort of intermingling with his personal thoughts and memories and desires. So I thought a kind of a mindscape that would move in and out of a simple story of physical recovery would be interesting.

14:54So this is not a film about a recovery that just goes chronologically through that recovery. It's much more than that. And in purpose, Salman, what is it? Well, the first way of thinking about it that I had was maybe it's not pointless for people to see what terrorism looks like and see it close up and unvarnished. This is what an eye looks like when it's hanging out of your head. This is what a neck looks like when it's been slashed. So I just thought, tell the story. In an age when there is so much violence that we are almost inured to it, especially if you live in America and there's murders every day, murders of children at school, mass murders for no reason, human life feels much cheaper than it should.

15:37And one way I thought of understanding how this man thought is that it's a kind of an American story. If you're so used to the cheapening of human life, it may be easier to conceive of taking it. Salmo's already starting to work on Knife the Book, the memoir. And one of the things that he said, as I recall, was that what was interesting to him was the idea of moving from an act of hate to a place of love. And that was very interesting to me, too, because I had spent a good bit of my time doing films about the dark side of human nature. and indeed there's a film I did called Taxi to the Dark Side, which is all about terror.

16:12And I thought, wouldn't it be interesting to kind of flip the script and show not only the face of terror in the form of the wounds that had been inflicted on Solomon, but also how you recover from that, both physically and spiritually and psychologically, to be able to embrace life again. At the end, Solomon says it's a reconstructed happiness. I was interested in the idea of reconstructing happiness. I've always disliked thinking of myself as a victim. And in a way, doing this double project of writing this book and making this film was a way of taking charge of it. I'm going to tell the story.

16:45I'm going to say what it means. And the attacker doesn't get to be the storyteller. He's had his 15 minutes of Andy Warhol type fame and he can now go back into obscurity for the rest of his life. Sir Salman Rushdie and the film director, Alex Gibney, speaking to Justin Webb.

17:05Still to come in this podcast, a crucial general election in Sweden. They have been very much mainstreamed, but they're still a very radical party in the views of many voters. Amid rising support for the far-right Sweden Democrats.

17:32investing, financial planning, thematic investing, retirement planning. Phew. And to think, that's just a small taste of what Schwab offers. Because Schwab knows that when it comes to your finances, choice matters. No matter your goals, investing style, life stage or experience, Schwab has everything you need, all in one place, so you can invest your way. Visit schwab.com to learn more.

18:02Oh man, look at those passes. Tillage, planting, harvesting. Three years in and these pre-owned machines are still at the top of their game. We're talking reliability, performance and long-term potential. And get this, these veteran machines can pick up the latest precision tech with ease. You know it and fans ready to add John Deere certified pre-owned equipment and more options to their lineup can shop at machinefinder.com. Booyah! ever wonder why we make the choices we do and how to make smarter ones introducing choiceology an original podcast from charles schwab join wharton professor katie milkman an award-winning behavioral scientist and author of the best-selling book how to change as she shares true stories from nobel laureates authors athletes and everyday people about why we do the things we do and how to make better choices to help avoid costly mistakes.

18:58Each episode covers the latest research in behavioral science and dives into themes like the power of self-control, shaping your mindset for success, navigating new beginnings, and why starting over can feel so hard. Listen to choiceology at schwab.com slash podcast or wherever you listen.

19:24This is the Global News Podcast. Yemen says it has struck Houthi fighters in the strategic coastal city of Mokka. It comes a day after the Iranian-backed armed group seized control of the entire Red Sea coast, threatening to block the Bab al-Mandab Strait, a vital artery for Saudi oil exports. Saudi Arabia has closed a critical oil pipeline after it was hit by drones launched from Iraq and has said it won't retaliate for now. Djibouti, which sits on the other side of the strait, says 1 ,400 people have fled to it from Yemen in just 24 hours because of the fighting between Yemeni government forces and Houthi militias.

20:04Our Middle East correspondent Hugo Bishaga has been speaking to Ankur Desai about the fighting. It's difficult to get a clear picture of what exactly is happening on the ground, but we had a statement from the government in Yemen saying that its forces had carried out some airstrikes targeting Houthi positions in Mokka, And this was actually confirmed by a Houthi spokesperson who said a number of fighters had been killed. Now, Mokka is a strategically located port on the Red Sea. And I think we haven't seen any kind of organized response from the government yet. There have been reports saying that government forces were shocked by the lack of support from the Saudis.

20:43The Saudis obviously have been supporting the government in this fight against the Houthis. There have also been lots of divisions among members of this anti-Houthi coalition in Yemen. So it does seem that the Houthis may have spotted an opportunity to make these gains. What about the chances of this escalating the risk of plunging the country back into a full-scale conflict? Because we have seen periods of relative calm in Yemen compared with the height of the Civil War. Yeah, there have been years of relative quiet in Yemen, and we still don't know if the government and its Saudi supporters are going to carry out a large-scale offensive against the Houthis to try to take back this territory.

21:21Don't forget that years of military action by that Saudi-led coalition against the Houthis failed to dislodge them, and the Saudis had been prioritizing diplomacy in recent years. I think it's interesting that President Trump today confirmed that he had a phone call with the Saudi Crown Prince Mohammed bin Salman, who reportedly asked for an American intervention in Yemen, which was rejected. So I think this could be an indication of perhaps a strategy of trying to contain these tensions, of trying to prevent a further escalation of the conflict. Hugo, the price of oil is rising. It's at over$100 a barrel.

21:57So if the Houthis are able to take control of this key choke point, the Bab al-Mandab Strait, just talk us about the implications of that. This advance has put the Houthis in a position to control this strategic Bab al-Mandab Strait. This has been a key waterway for Saudi Arabia's oil exports, an alternative to the Strait of Hormuz, which has been effectively closed because of the US war against Iran. And a blockade by the Houthis could further squeeze global energy supply. So that's why I think a lot of people are paying attention to the situation there. And any kind of action by the Houthis could also help Iran's strategy of driving up prices to put pressure on the US to end the war.

22:36And I think, you know, these seemingly different conflicts are connected because it could end up being part of this Iranian strategy of trying to put pressure on the US to come up with a deal to end the war. Hugo Buscega, who's been monitoring developments from Jerusalem. Now to Chile, where officials have declared a period of mourning after at least 16 elderly people were killed in a fire at a nursing home. Ten residents were reportedly rescued after the blaze took hold at the building in Santiago. I heard more from our Latin America specialist, Luis Fajardo. The facility in the south of Chile had been under investigation and had been apparently the object of several sanctions according to Chilean media.

23:20Chilean news website B.O.B.O., for example, was saying that there had been several irregularities. And the person who was taking care of the old people in this facility apparently did not have proper authorization for that. So it is causing a great deal of concern. Moreover, there are reports that this same facility had been the object of investigations and sanctions over the years, again, because of irregularities. So there's a lot of questions being asked about why this facility had been allowed to continue operating. And what's been the response in Chile? What are people there saying to this?

24:00There's a lot of complaints, of course, against the people who are behind the facility, but also questions about how effective the government regulation of these facilities has been. In Chile, as in many other countries in the region, in Latin America, there is a particular and growing concern about how old people are being taken care of by society, because these countries are facing an increase in the number of old people in the community as the society is becoming a higher proportion of them are in old age. And there are questions about whether the regulation of these institutions has been effective.

24:39And has the government said anything on this? There was a recent statement by the government of Chile saying that indeed the person taking care of the old people in this facility did not have the adequate documentation. And they are of course promising more investigations and eventually sanctions to those being responsible. Luis Fajardo. The people of Sweden are heading to the polls in what's expected to be a tight race between the centre-left and centre-right political blocs. It's effectively a referendum on whether the far-right Sweden Democrats should join the government and move away from political outcasts to trusted partners.