The Greatest Threat Humans Just Can’t Stomach with Yuval Noah Harari

0:08Rory Stewart:Welcome back to Thinking the Unthinkable with me, Rory Stewart. Every week on this podcast, we talk about things that absolutely fascinate me, but they're often the stories of the week. An election in Germany, an earthquake, a flood in Nepal, something that's happening in the British parliament, new prime minister. But there are other things that we don't talk about as much, which may be far, far more important and are much less visible. The things that could actually completely upend our societies, things that could destroy us. And in this series, I'm going to try to go into the heart of some of those things.

0:45Rory Stewart:Some of them are things we used to talk about, but we don't talk about very much anymore. Nuclear war. Some of them are things that we talk about, but we barely understand yet, AI. And some of it are things that we hear about, terrorism, ISIS, totalitarianism, but which we don't really understand the texture of, where I'm going to be sitting down with people who've actually lived on the ground in Mosul under ISIS, in Membij, northeast Syria, as ISIS is coming up, talking about human nature, about totalitarianism, about extremism. And in all of this, I hope we're getting to the real core of what really, in the end, matters about politics, the really biggest things, the things that our minds struggle to predict, that our political systems, our newspapers struggle to really talk about, that voters aren't incentivized to raise, things which are unthinkable.

1:41Rory Stewart:And the question we'll be raising is, how do you think about the unthinkable? who says Halloween only lasts one day with Wayfair one night of Halloween becomes a whole season at home from larger than life yard decor and moody lighting to festive accents for every room in the house Wayfair has what you need to make Halloween feel like magic all in one place because the best Halloween memories aren't made in just one night they're made all season long Shop Halloween decor now at Wayfair.com. That's W-A-Y-F-A-I-R dot com. Wayfair, every style, every home. Ryan Reynolds here for Mint Mobile with your summer price forecast.

2:21Rory Stewart:Now, unfortunately, we're seeing rising costs across the country with possibility that big wireless hates you 100%. Now, over here at Mint Mobile, we're seeing sunny skies and dropping prices. Every plan to just$15 a month. Give it a try at mintmobile.com slash switch.

2:50Rory Stewart:My guest today is Yuval Noor-Harari, one of the most famous public intellectuals alive today, the author of Sapiens. And today we're going to be asking perhaps the most fundamental question in existential risk. What happens if with AI we are building an alien intelligence that turns out to basically create a new form of empire, an empire in which AI is the emperor and we are the vassals. What's that going to mean for our economy? What's that going to mean for our politics? Ultimately, what's it going to mean for our minds and our sense of ourselves? And how can we learn from past civilizations on what happens when this incredible rupture occurs?

3:31Rory Stewart:Yuval Noah Harari. Can we sort of start with the general idea of this thinking the unthinkable? I don't know whether we want to call them existential risks, but what this category of things are and whether you recognize it as a category? You know, the purpose of politics I think is all about is making choices, is defining challenges and problems that face society, giving the public two, three, five options, and then we have a conversation about it, and we need to choose what is the right policy. This is when politics functions well. But for this to happen, the politicians need to focus on the most important problems that face society.

4:15And every now and then, something happens in history which is just outside the normal political discourse, that we don't think about it, we don't even have words to describe what it is, but it could be the most important next thing. And then the job of politicians is to think the unthinkable. You know, the range of examples from human history is quite wide. I mean, if you are in the Aztec Empire and in the early 16th century, and you just heard reports about this very strange thing that happened on the coastline. These huge hills appeared over the ocean and out of them came strange men or ghosts or demons.

5:00We are not sure what they are. And they have these huge creatures that can run and they have sticks that can launch lightning and thunder. And this is outside the normal course of Aztec politics, but this is the most important thing you now need to confront. And the Aztecs did a very bad job confronting that. They couldn't really understand what the hell is this and what should we do about it?

5:30Rory Stewart:So it's a lovely place to start. So obviously, Yuval, you're describing the arrivals of the conquistador, the Spanish invaders into Latin America. And the astonishing thing there is that it's a few hundred people arriving into an empire of many millions. And despite the fact that they have horses and simple muskets. That couldn't fit anything. I mean, it was just for show. In practice, it would have been very, very easy for a million people to over... Yes. I mean, the tragedy in a way of the Aztec empire, they could have squashed Cortez very, very easily, but they didn't understand what they were facing.

6:05They made a lot of mistakes. And these few hundred people, they destroyed the Aztec empire. And of course, with the help of the microbes they brought, unknowingly, they brought all these old world diseases. They decimated the entire population of Central America and beyond, according to some estimates, about up to 90 % of the people died. Entire cultures were wiped out, and they never recovered from it. There are, of course, people today in America who are the descendants of the Aztecs and the other people of the empire, but they never recovered from it. And really, as you said, it was just a few hundred people.

6:46They could have squashed them easily, but they just didn't

6:49Rory Stewart:understand what they were facing. So I think this is an interesting route through. So you begin as a medieval historian, you then become very interested in these moments of discontinuity. So this is not exactly like the question of why William the Conqueror won at the Battle of Hastings against King Harold, which is a more ordinary engagement, which would have been comprehensible to an Anglo-Saxon or a Norman. Somebody won a battle, they lost the battle. You then begin to take an interest in these incredible historical moments like that, the arrival of Cortes in Latin America, and the fact that what's hitting you is not just an alien people, it's also an alien technology and something even more unexpected, an alien pandemic.

7:28Rory Stewart:Yes. Now, fast forward in your later writing, you begin to focus on things like nanotechnology, robotics, artificial intelligence, pandemics as well. What begins to draw you into those sorts of issues and categories away from medieval warfare? I think it's not away from medieval warfare or away from the Middle Ages. I mean, the Middle Ages are more relevant than ever because modernity is coming to an end. And then the question, what is there other than modernity, becomes very relevant. And the obvious place to go is the Middle Ages. But of course, we are not going back to the Middle Ages. We are just exiting modernity into something else, which we don't know what it is.

8:11But some of the challenges we are now facing remind me of what the Aztec empires faced unsuccessfully back then. And when we talk, and we'll talk a lot about AI, so AI is this alien invasion. I mean, for me, AI is the acronym is not artificial intelligence. There is nothing artificial anymore about AI. Artificial brings to mind something that we create and is under our control. But AI is not artificial in this sense. It's really an alien intelligence. It's not alien coming from outer space, of course, but it's totally alien to our way of thinking and understanding the world. It's an inorganic intelligence.

8:55We have never in history met anything like it. And the job of politicians today is to do a bit better than Montezuma and understand what is happening, take it seriously, explain it to the public and present choices.

9:14Rory Stewart:What do we do? To make the case for Montezuma, which is an unfashionable thing to do, the problem that Montezuma faces is all the structures of his government and all his historical experience and all his technology and tactics, which have made him the head of the Aztec Empire. It's no small thing. I mean, he's obviously a very nimble, able politician in general, are suddenly confronting something in those boats which has never been seen before, which are alien. And so, part of, I think what this series I hope is going to be about is how we think about the alien. Now, we'll have a chance in our second conversation to get deeper into the question of the alien intelligence that's AI.

9:56Rory Stewart:But just before we focus on that, I want to circle around some of the other things that might seem a bit alien or difficult for us to understand. Let's maybe have a look at pandemics. That's something that you've thought about and written about. And of course, it's something we went through in a smaller way with COVID. And again, where I felt I was in government coming through the end of 2019, I'm very involved in this debate going to the beginning of 2020, was an interesting example of the ways governments can and cannot respond to something which is really strange. Something comes out of a viral lab, it seems, in Wuhan or a fish market in Wuhan, however it gets there.

10:34Rory Stewart:And it ends up with the British government spending something close to 400 billion pounds trying to deal with this. In fact, most of the debt problem that we have in Britain, most of our financial problem today is a legacy from this virus coming out of Wuhan. I think it's a very good example, because I would say in many of these cases of these existential risks, and we certainly see it with AI, is there is a brain-stomach barrier. Our brain, our intelligence based on various data starts understanding something, but our stomach says, nah, that cannot be possible. Why not? Because I have never experienced anything like it.

11:16So I have no gut feeling what this thing can be like. So it cannot exist. I remember, you know, the first few weeks of the pandemic, I remember myself sitting at my home, watching on television, the images from Wuhan of a modern, huge city of millions completely shut down. And as a historian, I had all these kind of synapses going on in my brain. Oh, this is like the Black Death. This is like the Spanish influenza. I read about pandemics and still my stomach says, no, this is not possible. This is not possible. Maybe it's possible in China. It will never get here. Why not? Because I just never saw anything like it in my life.

11:56So I cannot accept the reality of this thing. and the stomach is usually much stronger than the brain. Even when the brain insists, this is real, this is coming, we need to prepare, there is something that just doesn't accept it.

12:10Rory Stewart:So what I observed at the beginning of COVID, at least in Britain, is that there is a real tendency in government not to want to act. So we saw particularly dramatically in Milan, in Italy, the beginnings of hospitals being overrun. And we had seven flights a day coming from Milan into London. And people would appear on the radio and they would say, we're about two weeks behind Milan. And yet the decision was, we will go ahead with the Cheltenham Festival. We're not going to shut this down. We're not even going to stop the flights from Milan. And very, very quickly, some at least of the people around government were saying, we need to take this on the chin.

12:53Rory Stewart:There's nothing we can do. There's no point wearing masks. There's no point isolating, we're never going to be able to get a vaccine in time, it's better to just develop herd immunity. And a lot of the reason for that, and I think this will be relevant because I think the same is true about why we probably won't be doing enough about AI, AI safety, probably why we will underestimate the risk posed by Vladimir Putin or underestimate the risk that could happen with Trump or China is because not doing something is cheaper and easier. And you can see the cost of action. Oh my goodness, if we have to do any of this stuff, it's going to cost our economy hundreds of billions, our schools are going to have to shut, our businesses are going to have to shut.

13:38Rory Stewart:So we will do our best to convince ourselves it's not really that serious, or that we can't do anything about it anyway. Yeah, I think this is exactly the mechanism. Like if you look, again, we'll talk about it at greater depth in the next episode, but one of the threats facing many countries today in the world, including Britain, is one of AI imperialism and of becoming vassal states of the United States. If your entire economy and military is run on AI infrastructure that the Americans can shut at the press of a button, you are no longer an independent country. Now, this is a real imminent threat that can materialize within a very few years.

14:19Now, I think there are basically three options that what a country like Britain can do. Option number one is business as usual, just accept the fact, yes, we are going to become American vassals. But if that is the case, you need to be aware of it and act accordingly. If your plan how to deal with AI imperialism is just to accept your place as American vassals, then starting from today, whenever the American government says jump, you jump. If they say Greenland belongs to the US, then yes, Greenland belongs to the US. If they say give the Falklands to Argentina, then the UK government says, oh, the Malvinas, of course, they're Argentinian.

15:01That's option number one. Option number two is to say, no, no, no, no, no. We want to remain independent. so we will build UK AI. Sounds wonderful, except for the price tag, which is something in the area of 500 billion pounds. Now, in theory, if this is an existential thing, the UK can do it. It's one of the few countries in the world that can theoretically still do it.

15:27Rory Stewart:Because as you just pointed out, it's slightly more expensive than COVID. It's about half the government budget. It's possible, but it means that you have to say it now and you need to cut healthcare, welfare, education, transportation, everything. So you have the money to make yourself an independent AI nation. Third option is to say this is too costly for us. We'll do it with the Europeans. It's still going to cost 500 billion pounds, but we'll give only part of it. And the Germans will give some and the French and the Spanish. But if this is the plan, then say so and act accordingly today. You need to rejoin the European Union today and integrate Europe much more closely because otherwise it won't work.

16:12Now, I think these are the only three options. And if the political system cannot articulate these options and come to the people and say, look, these are the options. All of them have costs and benefits. Choose. Then it's not doing its work. It's not willing to think the unthinkable. And then it's just, you know, the default is to become American vassals. But even if you go for the path of least resistance, you should at least do so in a self-aware way. So you will be good vassals. Like, you know, when the Spanish, when the conquistadors invented Mexico, so there were people, the Tlaxcalans, who to some extent understood what was happening and they became the allies, vassals of the conquistadors.

16:57And this paid off. This was good policy for the Tlaxcalans. Now, again, people can say yes, no, but you need to be aware that this is the decision you're taking and act accordingly.

17:11Rory Stewart:There are maybe two sets of questions around this. Why is it that policymakers find it so difficult to accept that those are their three options they need to do it? And the second set of questions is why those are the only options available to you. Let's maybe start with the first and easier thing. Let's assume these are the only options available to you. One of the issues that Andy Burnham will face is people will say, I don't like this stuff. I don't like the tech bros. I don't like Elon Musk. These companies are making too much money. There's inequality. There's trillionaires. It's not bringing great benefit to me directly.

17:51Rory Stewart:They talk about a cure for cancer in 10, 15 years, but in the short term, I'm going to lose my job. My electricity bill is going up. I don't want a data center next to me. I don't like how much energy it uses. I don't know how ugly it is, how much water it uses, and actually I want to pay for my health service and I want to pay for my education. So there is no political constituency out there. I mean, if you look at the manifestos of the parties, not even the most entrepreneurial politicians anywhere in Europe are making these arguments of the sort that you've advanced. Instead, to go back to the way in which we began this program, you read the manifestos and the manifestos are about the sort of things we've talking about for 50, 60 years.

18:35Rory Stewart:How do we improve the NHS? How do we cut the welfare bill? How do we build infrastructure? How do we get a bit of innovation? How do we improve our schools? There's a whole question of political culture, political economy, which can make you quite fatalistic. And then of course, there's the realization which you will have had as a historian, which is oddly at certain points in history, societies suddenly are able to make radical moves and make radical decisions. Britain did it during the Second World War. And actually, in the end, the end of my COVID story was not that everybody said it's all too difficult and it's all too expensive.

19:09Rory Stewart:Within about six weeks, they were doing things that seemed unimaginable, like lockdowns. What do you make of this process? Because this is a very, very important part of this problem. I'm not sophisticated enough to produce a complete description of what the Aztec equivalent of this is. But presumably, there was a strong Aztec equivalent at an elite level, which contributed to the problems of responding correctly to the conquistar? I think the key asset here is just time. That humans, including politicians, need time to understand what is happening, to change the way they think, to adjust. One of the big problems with the challenges that we now face, especially AI, is that it It is moving on a timescale, which is also alien, which is inhuman, which is non-organic, non-biological.

19:59I mean, to make it in a more concrete way possible, I've been talking with some cyber experts last few weeks about the cyber threat. And one of the things they said on a personal basis is, I can't sleep anymore. I can't sleep, not partly because they're very worried, but technically I can't sleep because by the time I sleep for like six hours or eight hours, so many things have happened. And this is because AIs don't sleep. Like when it was Churchill versus Hitler, so Hitler also needs to sleep. He got these injections that enabled him to go on for longer than most people. But in the end, even Hitler had to sleep because he was an organic entity.

20:38AI doesn't sleep. If you need to act and to understand what is happening in the age of AI, then you need to act to understand something that moves at a superhuman speed. To give another example, in the financial system, bankers, investors, they were used to a situation when the markets closed down for the night and for the weekend and for Christmas. But AIs don't celebrate Christmas, and they don't sleep at night, and they don't go on the weekend on vacation with their kids. So we now see the markets, the financial system shifting to a constant cycle. And again, which is not just constant, it's moving at a superhuman speed because AIs can analyze financial information at orders of magnitude faster than human beings.

21:31So if the AI crisis would unfold at the speed of kind of previous political human crisis, I think the political system will be okay, probably. It will eventually understand what is happening. It will articulate the different strategies and choices. It will hopefully make the right choice. But it's just moving far, far faster than anything we've seen before. I was reading today the front page of the Wall Street Journal, and there was a main story there about the risk of AI completely destroying humankind. Now, 10 years ago, this would have been in a science fiction magazine. Nobody thought 10 years ago that on the front page of the Wall Street Journal, you will have a serious story.

22:21AI might get out of control and completely annihilate Homo sapiens. And just the speed with which it's happening doesn't allow the political system to do its job. Yuval, thank you very much indeed.

22:35Rory Stewart:Let's take a quick break and then back for more.

22:49The great game, strategic struggle between Britain and Russia. We assess Prime Minister with high confidence it was Russian. Holding the Russians accountable is World War 3.

23:06Rory Stewart:They took down a British passenger jet containing 350 people. Man in the Kremlin is playing, Alex. You've got to ask yourself, do you want to play or not? You're the Prime Minister. You've done nothing!

23:21We would need to understand the policy context. All the relevant context will be structured as a framework today, and disseminated into an operation order, or frago. A frago that says to kill the president of Russia. Yes. New from Goalhanger. Who is going to be able to pull this off? You're my soldier, are you? Sergeant Tom McDuff, Duffy! An audio drama journey like no other. Men like Duffy, they are our most valuable weapons. devised as a technically rigorous geopolitical action thriller, where Britain plays its strongest hand against its strongest foe. Bear Hunt. Available now wherever you get your podcasts.