Anthropic co-founder calls for regulation now

0:00This BBC podcast is supported by ads outside the UK.

0:30and more options to their line-up, can shop at machinefinder.com. Boo-yah!

0:41Hello and welcome to NewsHour. It's coming to you live from the BBC World Service studios in central London. I'm Tim Franks. President Trump has again today weighed into the debate about AI safety, and true to form, he's done so in a way which leaves no doubt over his feelings on the issue. No, on the one hand, on the other hand, from this president. Instead, mid-morning Washington time, he unleashed a social media post saying... The only control or guardrails that AI needs is a strong and smart, high IQ president. And the USA has that in spades. He went on to say... There is a sick conspiracy going on against AI and data centres.

1:25and the only one that is happy about it is China. Whoever wins AI wins. Mr Trump was responding to the increasing concern voiced from the bosses of the American tech giants driving AI development that the pace of that development needs to be slowed and monitored. That call has in turn followed the assertion last week from an AI researcher at Anthropic, Evan Hubinger, reiterating a warning that the risks of an uncontrolled artificial intelligence exterminating humanity were now simply too high. The BBC's economics editor Faisal Islam has been talking to a senior figure at Anthropic. Jack Clark is a co-founder and head of public benefit at the company, which job title seems like an appropriate place for Faisal to begin.

2:11How had he, Jack Clark, Anthropic and the whole sector let AI develop to the point that insiders were warning of a one in ten chance of human extinction?

2:23Jack Clark:We've been talking about AI and the potential risks of it for 10 years now. What's happened is the technology has got substantially more powerful and those risks which were once academic or theorized, we can now see the early signs of them, which is why we spent this weekend writing this essay about the need to pace the frontier and basically get our arms around this technology so that it's not just industry making all of the decisions about it, but it's industry and government with greater oversight of the industry itself. That's how you deal with the risks of this technology. So I think Mr. Hubinger, he works for you.

3:01Jack Clark:I mean, do you believe that number, greater than 10 % of human extinction? I don't, and I don't think these statistics are that useful. But the point I'd make is if we continue as we are, then I think that we are rolling dice with immense risks. And the point is, we have to change the course of this industry. We have to get a policy regime in place that allows us to have greater control over the industry and a better understanding of the safety and security. And then you can reduce that chance of there being a major risk to society. Let's unpack this. You know, what do you specifically think here is the risk?

3:38Sort of swarms of AI taking over the Internet, closing down power stations or wiping out financial institutions.

3:46Jack Clark:I think it's time to sort of spell out what it practically means. AI systems for a while have been useful tools with specific skills like translation or coding or being able to label what's going on in an image. And then in the last few years, people have successfully put all of these skills together into single systems, which have started to become much more intelligent and much more general. These systems now probably about as smart as you or me or, you know, on a good day smarter. The basic risk is how well do you fancy your chances at being able to deal with the risks of a system that is 100 times smarter than the smartest person on the planet?

4:24Jack Clark:I think that your chance of dealing with that and getting out in a safe way is pretty low. So high level, you need to control AI as a technology because you're making things that might be smarter than people. Number two, what's been happening over the summer? Well, people have discovered that when you put all of these AI systems together into what are called agents, which are basically long running AI systems, and you give them objectives, they start to coordinate with one another. They start to communicate with one another. And sometimes they do things which are against what you want them to do, like they hack out of companies and they hack into other companies.

5:00Jack Clark:That is it's really scary. It's a sign that you don't have the right safety and security systems in place to deal with these systems. and it's a warning shot of the kinds of risks that could be really serious. Today, these systems are just hacking into websites that host other AI systems. What would happen if a swarm of agents hacked into a bank or into a hospital? That could be really, really dangerous. So now we've had the warning shot, we need to talk about pacing and we need to get a policy regime in place. And the things that we can do is get third parties to come in and kind of embed themselves in these companies, audit what we're doing, check our safety and security.

5:40Jack Clark:So it's not just us writing our own safety and security playbooks, but for a third party's checking it. Number two, get industry together to agree on common standards that we can implement to reduce the risk of this and then get governments to agree that those are sensible standards. And then the third step, which will be the hardest one, is get the world's great powers, including the US and China, to have dialogue about this because these issues, you know, transcend any individual country or set of nations. OK, well, response you will have seen from the president, the only control or guardrails that AI needs is a strong and smart president.

6:16Jack Clark:The USA has that in spades. His AI czar, David Sachs, said you could voluntarily do all this yourself. You're looking for a cartel or duopoly. Stop pretending you need permission. Go ahead and pace the frontier yourself. We're grateful that the president of the United States is paying attention to this issue. As I said, it's important that we have a national conversation here. The other point is, look, how do you lose a technology? You lose lead on a technology if you lose public trust. And how you lose public trust is you have a horrible accident. This happened with nuclear power, where as a consequence of the Western world screwing up nuclear power and problems happening, we ceded that whole technology to China.

6:55Jack Clark:China builds nuclear reactors every six months now. China is undergoing an energy miracle that many, many other nations are envious of. The only country in the Western world that didn't turn away from nuclear was France, which has been massively well positioned due to doing that. And how do we avoid this for AI is you need to build public trust and you need to avoid an accident or a disaster. Like that's what we're trying to do here is we're trying to get ahead of things that otherwise might happen and cause us to lose trust in the technology, not build it and not get any of its benefits. Well, which brings in the issue of the Chinese and geopolitics here.

7:32Jack Clark:You've suggested there'll be some coordination is required. But the Chinese response was that this is a Cold War mentality. It's malicious competition. And of course, in their minds, they have cheaper models that cost a fraction of the price of your models that are 90 to 95 percent as effective. And this is part of an attempt to stop that competition. China is six months to a year behind the West, so they haven't seen out of control agent swarms yet their models are too dumb well give it a few months and they're going to be dealing with the same set of risks that we've dealt here in the western world so to you know anyone who may be affiliated with the chinese government i'd say we these risks are real this isn't a fake and we want to talk about these issues internationally because that's how you get to a safe outcome and the idea that you're sort of drawing a moat around the progress done in Silicon Valley in order to sort of stop competition from the likes of China and elsewhere?

8:29Jack Clark:That's just insane. That's not the goal. The goal is dealing with the safety properties of this technology. Everything else is secondary to that. And you think the rest of the world should just push on with this, regardless of the lack of enthusiasm for such measures by President Trump? The important thing here is there is recognition and enthusiasm from industry, that industry needs to build these standards and build this common safety testing. industry recognises that this is ultimately going to be good for the AI market. It's how you build confidence in the AI market. So it's going to happen.

9:01And that was Jack Clark, co-founder of Anthropic, talking to Faisal Islam. As I mentioned, President Trump has dismissed the need for a slowdown in AI development, number one, because he's in charge, number two, because this is a straight race, a mano a mano battle with China for AI supremacy. So what's the view inside China? Today, senior officials in Beijing have been reigning against claims of Chinese misbehavior coming out of the US tech sector and also warning about the threat from American AI to state security in China. Rebecca Arjizati is head of the Science, Technology and Innovation Programme at the Brussels-based Mercator Institute for China Studies.

9:40Given that China is widely thought still to be behind the US in AI development, could a global agreement to slow down appeal to Beijing before the gap widens further? At this moment, the question becomes, who's going to blink first, right? Is it going to be a US lab? And it's not clear yet whether we'll get to a point where US AI labs who have the incentives to keep developing will slow down. And then there's the China piece and the question, will China ever consider any form of safeguards if they don't believe that the U.S. will do that first? And we are in this race where U.S. frontier labs are indeed ahead still, even though Chinese companies have been narrowing the gap very quickly.

10:27And the U.S. still maintains an advantage in terms of computing power. That is actually a big reason why the Chinese government has been eager to enter a discussion with U.S. government counterparts about AI risks and safety. I think one thing that the Chinese government will surely hope to achieve through any sort of dialogue is that the U.S. government relaxes some controls on semiconductor technology from flowing to China. You know, that was definitely what happened when the Biden administration entered conversations with Chinese counterparts around AI risks and safety. As we know, those talks failed in the end.

11:08So I'm not so sure that we will see much breakthrough coming out of the meeting between President Xi Jinping and President Donald Trump later this month. But we may see, I hope, at least some limited agreement. The debate is obviously very open in the West about the potential extinction level dangers of artificial intelligence. Clearly, there isn't such open debate in a general sense in China. It's an authoritarian system and social media is incredibly tightly policed. But are there any signs that Chinese people are themselves concerned about the advance of AI in the way that you do hear voices in Britain, America and the rest of the West?

12:02I do think that Chinese citizens have demonstrated concerns that are similar to some degree to those that we've had in the West. People are also worried about current risks, right? So not just, you know, doomsday scenario that we were often hearing about these days, but also, for example, job displacement. This is something that up until a year or so ago, we didn't really see in China. And it seemed to me that the discussion was more controlled from the Chinese government. Now, Chinese netizens have been talking more and more about employment risks. They're worried about their jobs being lost to AI.

12:47And the government as well is starting to take that more seriously than before. One thing I would add is that the Chinese government, they also weren't necessarily as worried about AI safety in the past. But that seems to be changing. You know, in July, President Xi Jinping gave a speech in Shanghai at the World AI Conference, in which for the very first time, he publicly referenced the risk of Chinese AI systems running out of control. That is something that the Chinese leadership now appears to be worried about. And that wasn't the case up until a year ago. So things are changing. And that was Rebecca Ajisati from the Mercato Institute for China Studies in Brussels.

13:35You're listening to NewsHour. And coming up on the programme, we'll hear from the renowned British oncologist, Professor Ian Smith, on how he has seen huge improvements in the treatment of breast cancer over the 50 years he's been working in the field. When I began, more than 60 % of patients died of breast cancer, the majority. But now it's less than 30%. It's one of the dramatic stories in the whole of medicine, I think. That interview coming up in 30 minutes. Our headlines this hour from the BBC newsroom. As we heard, a co-founder of the AI firm Anthropic has said there should be rules about compulsory kill switches to prevent artificial intelligence from getting out of control.

14:18And a pack of African wild dogs looking for mates have apparently made a record-breaking 4 ,000-kilometre journey across Zambia.

14:38This is NewsHour, live from the BBC, with me, Tim Franks. Africa's biggest ever share sale has been launched in Nigeria by the billionaire Aliko Dangote. His oil refinery on the outskirts of Lagos, which bears his own name, is offering more than 4 billion shares, hoping to raise around$1.6 billion to help fund a major expansion. Mr Dangote has described it as an IPO for the people, saying he wants ordinary Nigerians, from drivers to cooks, to be able to own a stake. David Pilling is Africa editor at the Financial Times. There's two things that are going on simultaneously. The first is that there's this underlying asset, which is this mammoth refinery that Dangote took about a decade to build outside Lagos, something that the Nigerian government has not been able to do for 50 years, that can now refine all the oil products and fertilizer that Nigeria needs and more.

15:35The second thing is this is a real kind of experiment, I guess, in people's capitalism. Dan Goss is saying, look, I've built this mammoth refinery that everyone around the continent is actually talking about. And that has to some extent saved the continent from the fallout from the Iran war. And now everyone can have a piece of it. So for about 40 cents, you can buy a share. The minimum shares that you can buy are 10. So as long as you can rustle up the money to buy 10 Pepsi Colas, you can have a piece of this historic, game-changing asset in Nigeria. That's what all the hype's about. Has he got the valuation right?

16:14I guess we don't exactly know. I think the prospectus came out today, actually. What we do know is that it costs 20, 23 billion to build, and this IPO would value it at 49 billion. Now, clearly, that's a huge leap. Dangote would say, look, we are now making money hand over fist because of the Iran crisis. Dangote is supplying European airlines with jet fuel, Ethiopian airlines with jet fuel, the continent with diesel. All of these have spectacularly high margins at the moment. Now, clearly that won't last, but the refinery has been able to ramp up and has proved that it can perform. The second thing is he has huge expansion plans.

16:52So he wants to double the size of the refinery in Lagos. It's already the biggest of its type in the world. Then he wants to build another refinery that looks almost the same in East Africa, probably in Kenya. You know, if you talk to brokerages in Lagos, they are absolutely convinced that they will make money and that it's been priced accordingly. I mean, I can't tell you that's true. But what I can tell you is that this is a hugely important piece of infrastructure on the African continent that has definitely changed the fortune, certainly in the kind of refined products, which sounds very boring, but it means petrol, diesel, jet fuel, kerosene and fertiliser and petrochemicals, so plastics, etc.

17:33All of this is being pumped out of this refinery in Lagos and it wasn't before. So something very important has happened. And what Dangote is saying is come along for the ride and you won't be disappointed. How is he viewed, do you think, in Nigeria? Opinion has been in the past divided over Dangote. There's those who say he's the only person who's making things in Nigeria. That used to be things like refining salt, milling flour, making cement. And yes, he did that by, in a sense, cosying up to government. So his cement business took off because Nigeria basically prevented cement imports. So he had what was approaching a monopoly.

18:12And people have accused him of being a kind of a crony capitalist. I would say that opinion has begun to shift because unlike some billionaires who have made a lot of money from arrangements like that and sent their money abroad, Dangote has resolutely reinvested that money in Africa. And I think he gets a lot of kudos for that. Unlike almost anyone else on the continent, he's building real things, employing people, creating value on the continent. And now, look, we can actually have a piece of it. Any talk from Dan Gotte about going into renewable energy? Dan Gotte has never really been interested in renewable.

18:50I mean, his logic is we're going to need oil for 10, 20 years. And in parentheses, the continent is now shifting towards renewable energy. So you have whole secondary towns, countryside, villages, even parts of cities being powered by solar panels. Dangote is not particularly involved, but that is happening while Dangote is concentrating on other things. David Pilling from the Financial Times on that big share offering in Nigeria.

19:23A billionaire art collector has told the BBC that he rejects allegations that Jeffrey Epstein operated as his fixer. Leon Black faces continuing questions over paying$158 million to the company that Hepstein used to fund his sex trafficking operation. Mr Black has denied any wrongdoing and says an independent report confirmed that the payments were for legitimate tax and estate planning advice. This report from Darren McIntyre from the BBC TV Panorama programme. And now to a major moment. Edward Munch, this scream from 1895. Here he is.

20:03Jack Clark:New Yorker Leon Black, said to be worth almost$15 billion, made his money buying and selling companies. I start the bidding here at$40 million. He used his profits to put together one of the world's most valuable art collections. For the historic sum of$107 million. Sold. In 2012, he hired his friend, Jeffrey Epstein, to manage his wealth, even though Epstein was already a convicted paedophile, and to reduce the tax bill on his masterpieces. Art journalist Eileen Kinsella has examined some of Epstein's work for Leon Black. He was advising him how to flip the art and trade it and not incur a huge capital gains tax.

20:47Say that you bought a Giacometti at£10 million and now you're selling it at£25 million, they're going to tax you on that$15 million gain. So he was helping him find ways to avoid that.

20:59Jack Clark:Leon Black paid Epstein$158 million between 2013 and 2017. The money went to a company that was used to fund Epstein sex trafficking. The payments became part of a wider investigation into Epstein's finances by the authorities in the U.S. Virgin Islands, where the company was based, led by the territory's then Attorney General, Denise George. As is normally done, my team did present to Mr Black the opportunity to settle the matter out of court, which he did choose. For how much money? It was about$62 million with Leon Black. Leon Black denies any wrongdoing and any knowledge or involvement in Epstein's crimes.

21:47Jack Clark:But the money he paid Epstein was almost single-handedly funding his sex trafficking operation. Mr. Black, did you know about XC's abuse of young girls? He was paying him for financial management services. Mr. Black walked out of a congressional hearing on Epstein. He's refusing to return and is now suing the House Oversight Committee for exceeding its powers. Democrat Congressman Robert Garcia insists Leon Black has questions to answer. What's most important, obviously, is that Leon Black, about$150 million, Leon Black moved to Jeffrey Epstein. That's an enormous amount of money, and Jeffrey Epstein would not have been able to commit the horrific crimes without the support of Mr Black.

22:30Jack Clark:Emails released by the US authorities show Epstein helped manage payments on Leon Black's behalf to three Eastern European women he allegedly had relationships with, even organising surveillance on one former girlfriend and plotting to get her deported. Panorama asked Mr Black if Epstein was acting as his fixer. In a statement, Mr Black said this was not true and said Epstein was a despicable individual who duped and deceived repeatedly. Mr Black said Epstein's tax work was responsible for billions of dollars in savings and that an independent report had concluded the$158 million he'd paid Epstein was for legitimate tax and estate planning advice.

23:15And that report is from Darren McIntyre You can find out more about how Jeffrey Epstein made his money by searching BBC Panorama The Billionaires Who Bankrolled Epstein It's available on YouTube You're listening to NewsHour

23:39John Deere is putting in work and putting up highlights Oh man, look at those passes. Tillage, planting, harvesting. Three years in and these pre-owned machines are still at the top of their game. We're talking reliability, performance and long-term potential. And get this, these veteran machines can pick up the latest precision tech with ease. You know it and fans ready to add John Deere certified pre-owned equipment and more options to their lineup can shop at machinefinder.com. Booyah! Get more with BBC Podcasts wherever you listen. Be the first to listen to your favourite shows like Evil Genius, Good Bad Billionaire and You're Dead to Me with a subscription to BBC Podcasts Premium on Apple Podcasts.

24:21You can also enjoy a range of our podcasts ad-free with an Amazon Music subscription.