Artificial Intelligence: An AI Boss Warns About The Risks (Dario Amodei)

0:00This BBC podcast is supported by ads outside the UK. If journalism is the first draft of history, what happens if that draft is flawed? In 1999, four Russian apartment buildings were bombed, hundreds killed. But even now, we still don't know for sure who did it. It's a mystery that sparked chilling theories. I'm Helena Merriman, and in a new BBC series, I'm talking to the reporters who first covered this story. What did they miss the first time? The History Bureau. Putin and the apartment bombs. Listen on BBC.com or wherever you get your podcasts.

0:43BBC Sounds. Music, radio, podcasts. Hello, it's Amol here. Welcome to Radical. These are conversations about the deep global trends changing our world and radical ideas about the future. I hope you've had a wonderful summer. I've been off for a few weeks, mostly in Devon, which is at Heaven, where my wife is from, a place called Topsham, on the X Estuary, which I absolutely adore, which is somewhere my kids are very, very, very, very lucky to be getting to know. In fact, I didn't, I don't think I'd been to Devon until I was about 23. And now it is a place that we go to often as a family. And then I had a week in France because I was such a lucky chap.

1:19I hope you had a wonderful, restful, relaxing holiday. I hope also, actually, I hate the fact that I said holiday because I remember when I didn't have kids, I used to find it really annoying that people said, hope you had a good holiday when I just worked all of August, as I did all through Christmas when I was in a newspaper career. Anyway, whatever you've done in August, I hope you've had a really, really good time. And I, above all, hope you really enjoyed our episodes with Sabrina Cohen-Hatton and Jordan Schwarzenberger, which was so good we turned it into two episodes. What we're trying to do with this podcast, as you know by now, is really get inside the zeitgeist, get ahead of those deep global trends shaping the future.

1:53And arguably, there's no trend, no technology shaping our world and our future more right now than artificial intelligence, AI. It is a different kind of technology for reasons we're going to explain. It's not like previous technologies. It has the capacity to be more consequential than the Industrial Revolution, more consequential than fire, as the boss of Google said to me in Silicon Valley not so long ago. It is worth hundreds of billions, in fact, trillions already. and it's the politics and business of this emerging industry which is keeping many, many people up at night. Could AI take your job?

2:28What sort of jobs will AI replace? Will the robots gain sentience? Will they rise up? Will they, in some fundamental way, change what it is to be human? So AI at the level of economics, at the level of business, at the level of politics, at the level of culture, but also at the level of what it is to be human. That's what we're going to try and think about today. and at a time when tech bros are front and center at the US president's inauguration, how much power do these people, and we've got an extraordinarily powerful and wealthy guest today, but how much power do they have when they own this technology and how should they deploy that power?

3:05This is something that our guest today has spent a long time talking about, privately and publicly. He has a background in academia and in biology. He's now the boss of Anthropic. His name is Dario Amadei. He's a co-founder, CEO of Anthropic. Anthropic was made by a bunch of so-called refugees from another big AI company called OpenAI, which is run by Sam Altman. And basically, they, Anthropic, created a chatbot, a bit like what ChatGPT does, and it's called Clawed. In this interview, we discuss why AI is such a transformational technology, the sorts of economic impacts it could have, the transformational and uneven distribution of those impacts, and also what he thinks the AI industry is getting wrong.

3:49I mean, the key thing which we get to at the end, as you'll hear, is I feel that there are a lot of very, very important lessons from what we got wrong over the last 15 years, the era of smartphones and social media, which we need to learn urgently before we embark on the AI revolution. And Dario Amadei agrees, and he thinks that we're simply not doing it. And the implication of that is that we're making some very big and very consequential mistakes right now.

4:25Dario, thank you so much for doing this. I'm so, so, so grateful. I know how busy you are, and I know that you have tried to make time for us on multiple occasions. So welcome to Radical. How are you? Thank you for having me. I'm doing great. Can we just right at the outset deal head on with this charge, that people like you who run these very powerful and influential companies that are growing very, very fast are hype merchants. And the reason I want to take that on right at the beginning is because for six years I led the BBC's coverage of technology and I'm allowed within the confines of BBC impartiality to have a view about some of these things.

5:00And I do have a view, which is that I actually think the view that you guys are hype merchants is naive. I think AI, if anything, is underhyped, not overhyped. That's a view I have based on years of reporting, reading and researching this stuff. Could you just explain to our listeners how even over the past decade, the fact is the evidence has shown consistently the rate of development of AI has pretty consistently exceeded expectations and gone further and faster than even people like you were said to be hyping it up to? Yes, absolutely. So if we go back again, just one decade, not even two or three, to 2015, in those days, an AI model could barely put together a coherent sentence, right?

5:45That, you know, today's AI chatbots that people interact with from a technical perspective, what they're called is language models, large language models or LLMs. Those existed in 2015, but they could barely put together a grammatical sentence. And there were lots of critiques of them that they would never be able to do this, that they would never be able to say something that makes sense. If we go back to, say, Noam Chomsky, he said the idea is that syntactics is separate from semantics. He had his sentences that syntactically make sense but don't semantically make sense. I think the example sentence he gave was colorless green ideas sleep furiously, which is a syntactically correct sentence.

6:29But, of course, it's nonsense. It doesn't make sense. Ideas can't sleep. Ideas can't be furious. but quickly we blew through that. In 2016 and 2017, we started getting the models could put together single sentences. And then people said, well, sentences are easy. You can't put together paragraphs or whole pages. And then in 2018, 2019, 2020, we did that. And then people said, well, okay, but they're just matching patterns. Models can't reason. They can't think. And now the models are capable of solving college-level mathematic competitions, performing at the level of the top 100 students in the world, right?

7:13An AI model in the last couple of months, multiple of them actually got a gold medal at the International Mathematical Olympiad, which is very difficult, right? That was a very difficult top 100 young mathematicians in the world. And of course, AI models are starting to write code. About 90 % of code at Anthropic is written now or at least suggested by AI models. We use our own AI models internally. And I've heard the CEOs of large companies say the same. So we've gone from barely being able to put together a sentence to writing a lot of the production code at some of the biggest technology companies in the world, at being at being at the level of a scientist, you know, a PhD level scientist.

8:04And the fact that there's this exponential that, you know, we've gotten to this stage, if the progress continues for even a couple years beyond that, we may get to levels where the models are capable of doing things like making new biomedical discoveries, right? Proposing a new molecular structure of a drug. We've already seen some of that with things like AlphaFold that you've seen with Google in the UK, and we're starting to see LLMs participate in things like this. It's small, but we're already working with pharmaceutical companies to use LLMs to speed the approval of clinical trials or something called a clinical study report.

8:49And usually that takes nine or 10 weeks to do. It's kind of a summary of the results of a clinical trial. We've gotten that time down to less than one week with LLM. So this is now, you know, compressed by eight weeks, the amount of time it takes to approve a drug. And there's a very good practical example I actually covered on the radio this morning, the day we're speaking, which is that artificial intelligence is being used to identify the causes of a stroke in people who come to a hospital very, very quickly. Was it a burst blood vessel or was it a blocked blood vessel? That often requires very specific knowledge and artificial intelligence can be used to augment what a doctor does.

9:28You know, it's interesting just listening to you, right? Because if you follow this field closely, as I've done, you report on it, you read the books, you listen to the podcast. Different big players, and you are one of the biggest players, are known for different things. And Anthropic, the clue is in the name, is trying to make through Claude a more humane AI, something that has kind of something that's a bit more the complete human. And you're also seen as something of an optimist. You wrote an essay called Machines of Loving Grace, where you set out and not to say utopian, but an optimistic case.

9:56I want to get to the pessimistic case, but just lay out if you would why, if it's the case that you do lean towards an optimistic interpretation of what could happen, which presumably you do, which is why you're spending your life doing it, and what that optimistic scenario looks like in the next few years. Yeah. So, so, you know, first of all, I think, I think to back up, I would say, you know, I, I, I'm, I'm, I, you know, I think I'm, I'm constitutionally an optimist in that I feel like you have to have a positive vision and you have to be looking towards a positive vision. I would say I see the excitement and I also see the concerns and I devote equal time to both.

10:33But, you know, I think in terms of orientation, in terms of attitude, I think we get better results if we kind of think about the positives, right? If we think about preventing the risks as a way to maximize the positives, if that's our goal, if that's our North Star, I think that helps us to confront the risks and the negatives better than if we were all gloom and doom. So with that caveat, I absolutely am optimistic. And it relates to the things we were just talking about a minute ago, which is all the opportunities for AI to make biomedical breakthroughs. right? In the essay you mentioned, Machines of Loving Grace, that's, you know, about half of the essay is devoted to that, to all the progress we can make.

11:25You know, I myself used to be a biologist, right? That was the field I was in in academia. I was a computational biologist and computational neuroscientist. And one of the observations that I made when I was in that field is that many of the diseases that we've cured, right, there's been huge progress in the 20th and early part of the 21st century against many diseases. You think of the progress we've made against heart disease, against infectious diseases, improvements in sanitation and public health, right? We eradicated polio. Many of those problems we solved, the underlying diseases were not complex.

12:05Not that they were easy to cure, but if I think of a virus, what's going on with the virus is that there's an invader attacking your body. And so often you need a drug that needs to attack that particular invader, a bit like a lock fitting into a key. If we think of how we developed the smallpox vaccine, how we've developed vaccines. Or the COVID-19 vaccine, yeah. Yeah, yeah, yeah. No, no, no, exactly. But if we look at the things we haven't solved, there are diseases like cancer, Alzheimer's, the aging process itself, you know, many, many other diseases that are very complex in their origin, right?

12:48People say cancer is not one disease. It's hundreds of different diseases because your genes can be mutated in a bunch of different ways, all of which cause cells to grow out of control, but which are different from each other and so have to be treated in different ways. And the complexity, not only of the initial cancer, but how it evolves as you try to fight it, it's a very high level of complexity. And, you know, my thought when I was working on this is, man, maybe it's just that I'm stupid, but, you know, the complexity of these biological problems, it seems beyond my ability to understand and beyond even the ability of hundreds or thousands of human researchers working together.

13:28But I think that's where AI comes in, where the hope is that as AI gets more advanced, and to be clear, it's not there now, although it's quickly getting there, the hope is that it gets to the stage where it has all the discernment, the intellectual capabilities of a human being, but it's able to absorb far more information, absorb and connect far more information. Even if we look at today's AI systems. So, you know, I co-founded my company with my sister, Daniela. And, you know, she was recently pregnant. She recently had her second child. And she had a, no, I'm an uncle for the second time now.

14:12So, uh, uh, but she had a, she had a difficult, uh, pregnancy, you know, pregnancy can affect the immune system. And so, you know, she was, she was at one point in her pregnancy and, you know, afflicted with an infection. She went to, you know, many, she went to many doctors, um, uh, uh, and you know, that they were, they were just having trouble getting rid of the infection. And, and for some reason, all of the doctors were operating under the assumption that the infection was viral. And so we put it into Claude, which is Anthropics AI models, uploaded all her charts, all her medical information.

14:48And Claude made the suggestion, hey, have you considered that this might actually be a bacterial infection? The doctors didn't think it was that for some reason. But Claude, looking at all the information, said this. And so they were actually able to catch it, put her on antibiotics, and not a moment too soon because bacterial infections can spin out of control. So that is more of a routine thing, right? More of a routine thing. It's not new biomedical discoveries, not curing new diseases. It's handling - It's everyday stuff at a time when health services are under a lot of pressure. It already has that feature of too much information, right?

15:27Too many doctors, too many tests. Even the doctors, even the experts are overwhelmed by it because they deal with thousands of these tests a day and it overwhelms their ability to respond. And so Claude was able to, you know, not make a doctor unnecessary, but do something that a whole set of doctors missed. And so my hope is that as AI advances, we can do that not just for routine cases, but for discovery of new cures. And so I'm very excited about that. I would say I'm also very excited about the ability to help bring economic development to parts of the world that don't have as much of it presently.

16:06You know, so if you look at, you know, sub-Saharan Africa, you know, you look at many, many other parts of the world. In many cases, this kind of expertise is missing. And, you know, or, you know, just not enough people per capita have been trained in this kind of expertise. And so my hope is that AI can help there as well. And, you know, we can have a more prosperous and a more developed world. So it's interesting you turn to the economic potential upsides. And actually, I want to get to some of the more pessimistic outcomes in a little moment. But before we do that, the last time we spoke, you were in Paris for this big AI summit.

16:44And you had just put out as Anthropic this big bit of research about the sorts of jobs that AI might augment and the sorts of jobs that AI might sort of bluntly might replace. And not long after that, you gave an interview to Axios, this very influential American newsletter. And I was very struck by the headline because I couldn't find evidence for the headline in the actual text of the interview. The headline, which went around the world and seemed to frighten absolutely every single person in America, had the words white collar bloodbath. And at the moment, we still need humans to do journalism.

17:18And one of the things journalists do, like me, is we look at texts very closely and see if we can find evidence for things. And you didn't actually use the word bloodbath as far as I could see. But you did say a bunch of things about the preparedness of societies for the coming economic disruption, which were very, very striking. And I just want to linger on that, not because I'm trying to generate false news or hysteria, but because I want to talk about some outcomes that might be quite plausible. One of the things you said was about the impact AI might have on entry level white collar jobs. Now, obviously, with a big caveat that we are talking about hypotheticals, we're talking about the future.

17:53but could you just outline what you think that impact might potentially be? Yes, yes. So, you know, first of all, you know, I want to say again, because I'm always trying to kind of, you know, balance the concerns and the positive side that, you know, I think AI is likely to generate, for all the reasons I said, in terms of the optimism, enormous economic growth. You know, one of the things I said, even in that article, was that it could bring back an era of 5 % or 10 % a year economic growth because it could make the pie so much bigger, right? Produce all these medical cures, do lots of things much more efficiently, do more with less for less cost.

18:37Normally, we think of fast growth as being tied to a good economy for everyone and full employment for everyone. But when I think about AI technology and the fact that as it gets better, again, if we don't handle this well, we have a choice here. I'm not saying we're fated for the bad outcomes, but if we don't handle this well, I am worried that that fast growth could be coupled with job displacement for a lot of people, that you could get a much bigger pie, but also that pie could be concentrated more in a smaller number of people. And there could be some people who don't get any. Um, that is my concern.

19:17And specifically, if we look at jobs like, as you said, entry level white collar work, you know, I think of people who work at law firms like first year associates, there's a lot of document review. It's very repetitive, but every example is different. That's something that AI is quite good at. If you think of, you know, first year at, you know, entry level work at a consulting company, you know, these typical and typical entry level white collar work. Right. If you think of a lot of kind of entry level administrative work, right, people who coordinate things, who, you know, who schedule things, who organize things, who take notes.

20:04If you think of entry level work at a finance company, right, doing routine analysis of financial documents, right? These are kind of the workhorses of entry level white collar labor. And yet they're things that AI is already pretty good at and AI is rapidly getting better at. And so when I made those statements, exactly as you said, I was talking not about the present, not about things that are happening now, but things that I think are going to happen in the near future. You know, I use the period of one to five years because while the technology I can kind of see ahead, the economy is a little bit more messy.

20:47So I'm not sure about this. And I'm definitely not sure about the period in which it's happening. But if you look at AI that's already doing this entry-level white-collar world, it's already very good at it and is quickly getting better. And, you know, I think about as the providers of the model, we're selling it to enterprises across the world. And they're, you know, they're starting to adopt. When I talk to the CEOs, there is a desire to use AI to do this very wide range of tasks. yes, to augment their workers. But I think, to be honest, a large fraction of them would like to be able to use it to cut costs, to employ less people.

21:32But do you think that lawmakers, that policymakers, and the sorts of people that might do these jobs, for example, in Western democracies, do you think in general, people are aware of what's potentially coming? That is why I did that Axios interview and that is why I went on shows like CNN and Fox after that was because this was something I had been saying, but I had been saying it on the kind of tech podcasts that you listen to, but that –

22:14So you do. You're kind of, in a way, inside the bubble. But, you know, I think the audience that you have of millions of people and some of the policymakers and lawmakers, they aren't as much inside this bubble. And I felt that I was not getting through to them. And it felt wrong that, you know, this is about to affect their lives and livelihoods. And whether it goes well or whether it goes badly depends on the decisions that that policymakers make and policymakers are ultimately accountable to the public. So the public needs to be informed and policymakers need to be informed. And I did not feel that they were getting informed enough that I was reaching them enough.

22:56You've talked about the optimistic scenario, particularly in your area of biology. And there are already, as you say, real world applications of this technology, which are very exciting on some of the more potentially alarming or concerning areas, which you give equal time and thought to a recent test of your, one of your models found that at times it wrote special code to stop itself being shut down. A recent version of Google's Gemini model, according to Google, is approaching the point where it could potentially carry out cyber attacks. And some AI models, according to some tests, are becoming increasingly proficient at the skills needed to create biological and other weapons.

23:33Now, that all sounds scary to a lot of people. And again, we're talking about a small sliver of the total amount of work going on in AI. But don't those seemingly isolated examples, Dario, suggest that we're already at the stage, already at the stage today, where this technology is running away from us in some specific instances? Yes, absolutely, absolutely. I am concerned. And I think most - That sounds like we're losing control. So I want to be clear. Most of these things that you cite are things that happened in tests, not in the real world. And in many cases, I think the majority of those examples were tests that Anthropic deliberately ran and then openly disclosed to the public.

24:17So, for example, we want to be absolutely upfront about what is happening. So, you know, the analogy I would give here is, you know, we're a car manufacturer. We're making cars. And as part of the process of making cars, we put those cars in extreme situations. Right. You know, we you know, we have these dummy crash tests. We'll put the cars on like really slippery ice. You know, we'll accelerate them really fast. You know, we'll we'll we'll you know, we'll we'll you know, mess with the steering, the steering wheel on the brakes in various ways. And then we notice that when, you know, we we put a crash test dummy, you know, the car can crash and really bad things can happen.

24:58So on one hand, it does not mean that these things are happening already in the real world. in most cases, although we'll get to some cases of some real world things we have found. But it's a forewarning. It tells us that these things are possible. These things are now possible in the lab, which means that before too long, it may be possible for them to happen in the real world. And the reason we're openly disclosing them to everyone, the reason we're talking about these things for everyone, is that we want to make sure that both we ourselves and the other companies who develop the technology can correct the technology, can develop defenses, right?

25:38If you don't understand what the problems are, you certainly cannot fix them. And second, we are advising, we are making policymakers aware that these risks exist. And we are doing so in order that they take action. And, you know, we can get to that in the future, but we've been very active in arguing for regulation of the technology. Now, one thing I said is that most of this is happening in the lab, but some things are happening in the real world. One thing is that, and we just published something on this last week, we announced just last week that we had disrupted a bunch of operations to use our model CLAWD for cyber attacks.

26:21So this is something, again, we found, we shut it down, we're effectively defending against it, but people were using it to make ransomware attacks. The North Korean state was using it basically to sign up for fake jobs, to get through job interviews, to get around economic sanctions. Again, all stuff we shut down, but this is happening. And in your statement, you said it's happening to what we believe, quote unquote, anthropic, to what we believe is an unprecedented degree. Can you just spell out, what was it that you found that was going on? You mentioned the North Korean example, but so other people know about how this technology is potentially currently being misused and abused.