We Will Shape Humanity's Destiny with Nick Bostrom

0:00Tuesday on NBC, Jimmy Fallon and Bozma St John host a highly anticipated new competition show. I hired 10 creatives from all walks of life. They will be battling it out to see who can impress the world's biggest brands. This is a huge opportunity. This is the battle for the next big idea. This is not play play. We're spending millions of dollars. I'm so excited to embark on this adventure with all of you. Make the best idea win! On brand with Jimmy Fallon. Series premiere Tuesday on NBC.

0:33When did making plans get this complicated? It's time to streamline with WhatsApp. The secure messaging app that brings the whole group together. Use polls to settle dinner plans. Send event invites and pin messages so no one forgets mom's 60th. And never miss a meme or milestone. All protected with end-to-end encryption. It's time for WhatsApp. Message privately with everyone. Learn more at whatsapp.com. Hey, business owners. We know you know the importance of maximizing every dollar. With the Delta Sky Miles Reserve Business American Express card, you can make your expenses work just as hard as you.

1:10From afternoon coffee runs to stocking office supplies and even team dinners, you can earn miles on all your business expenses. Plus, you can earn 125 ,000 bonus miles for a limited time through October 29th. The Delta Sky Miles Reserve Business Card. If you travel, you know. Minimum spending requirements and terms apply. Offer ends October 29th, 2025. So joining us now on Open Book is Professor Nick Bostrom. He is a Oxford professor. And the title of the book is Deep Utopia, Life and Meaning in a Solved World. Professor, first of all, it's great to have you here. This is the reason why you have to go to bookstores, sir.

1:50I was rummaging through the bookstore. I saw your book. I said, okay, this looks interesting. I read your book and then I had my assistant reach out to you. The book is fantastic. Before we get into the book, let's talk about your background and tell us how this became your passion. Well, I mean, my background is I grew up in Sweden. And then I've been the founding director of a research institute at Oxford University since 2005 called the Future of Humanity Institute. and so I was a professor at Oxford until like earlier this year when I resigned. And I've long been interested in the impact of transformative technologies on the human condition.

2:40My previous book, which came out in 2014, was about the future of AI and what could go wrong and what we might need to do to avoid that. And then this recent book, Deep Utopia, looks at the other side of the coin. What happens if things go right? Yeah, so let's go back. That book written or at least published in 2014, titled That Book with Super Intelligence. I actually went to a lecture. I have to confess, I didn't read that book, but I went to a lecture at Columbia University when that book came out where there was a group of people debating the merits of your book and suggesting in the Elon Musk sort of way that there were catastrophic risks associated with artificial intelligence.

3:31This book is the opposite of that, the new book, Deep Utopia. So where do you stand in that debate, sir? I know you've written a pro and a con. Where do you stand? It's not so much that my views have changed. They have always been there, both like the belief that if things go well, There could be enormous upside. And then also the belief that it's by no means guaranteed that things will, in fact, go well. It's just the previous book was focusing more on one side of that. And now I'm trying to dive more deeply into where would we actually end up? Suppose we do develop machine superintelligence.

4:10We solve the safety problems so they don't all go around amok and kill humans. And we like, suppose we get our act together as humans as well. So we don't use these tools to oppress each other or wage war or like do other destructive things, like take the best possible case scenario. And then we end up, I think, in this technological mature condition where, because if we have super intelligence, it will help us invent all kinds of other technologies. And then we face these very profound questions, I believe, about the meaning of human life. Like what would you do all day in a world where like AIs could do everything much better than we could do?

4:49Well, I mean, there's a lot there. I guess the first thing, you know, because I'm meeting you for the first time on this podcast, reading the book, I read the book and said that Professor Bostrom is a optimist on the human condition and a long term optimist on what you just said. We get our stuff together. we start to figure things out better as a human race. Now, am I right on that? Are you an optimist? No, I mean, I've always struggled with that question because people ask me. And like, I'm kind of attacked from, so some people accuse me of being this kind of anti-tech, doomer type. And then at the same time, other people are hating at me for being this kind of techno-optimist, transhumanist.

5:36I sometimes wish I could just invite all of these different sets of people into the same room and they could like hate on each other directly, cut out the middleman. But what are you, sir? Are you the optimist that I see you as? Or are you, I mean, are we? I think we don't know how, I mean, we have a lot of uncertainty about how this will, we've never been through an AI revolution before. And so we just have to recognize that as far as we know right now, both of these seem in the card. I would, I mean, if you have to have a label, I guess a fretful optimist, maybe. Okay. All right. Let's go in the direction for a moment, at least, if we get things right.

6:17You write in the book about entering a period of post-instrumentality. Describe what that means to us. What is post-instrumentality? It would be a condition in which instrumental effort is no longer needed from humans. So instrumental effort being the kind of thing where you do something in order to achieve something else. So maybe you go to work in order to get a paycheck or maybe you go to the gym in order to be fit and healthy. And so a lot of what we do have that structure. We do X to achieve Y. And so at this condition of technological maturity, it seems we would enter a post instrumental condition because for almost all outcomes we might seek, that would be a shortcut that wouldn't involve our own effort.

7:07So this is a more radical conception than what we might call a post-work condition. A post-work condition would be we wouldn't have to work to make money because no human labor would be economically valuable. But that still would leave open the idea that there's a whole host of other things you need to do to put effort in. If you want to be healthy, you can't hire a robot to go to the gym on your behalf in a post-work condition. Like if you want to have a good relationship with your kids, you have to spend time with them yourself, et cetera. But the post-instrumental condition is more radical.

7:41For example, instead of going to the gym, you could take maybe a pill that would produce the same physiological effects in your body.

7:51Okay. So, I mean, I'm going to paint two scenarios for you. I want you to react to both. OK, the scenario, the post instrumentality scenario is one where we become lazier or is it one where we become better? Or I mean, what happens to the universe in that scenario? And then there's the Armageddon scenario where the technology takes over, decides that human beings are inferior and figures out a way to eradicate human beings. I mean, those are the two scenarios, right? But the question... Well, I think there are more, but yeah, certainly there are more. You're right. I mean, you're... Fortunately.

8:34You're a way more complex and smarter guy than me. I'm just trying to get... I'm trying to tease out of you because of your book. I'm trying to tease out of you what happens to mankind, basically, or womankind. Well, so the second scenario is kind of in the sense what the previous super intelligence was exploring. Like we failed to align the AIs and then they become some antagonistic force. So that's the one type. So in this book, let's assume that doesn't happen. We get like, you know, we survive, prosper. There's material plenty. Then you ask, would that mean we then all become lazy and kind of lie on the couch and having robot butlers?

9:19I think it would depend on our choice ultimately. and our human values. We would have the capability if things go well and we reach this condition of technological maturity to really shape not just the world around us, but ourselves as well in whichever way we would want. I guess what I'm trying to understand is we work the field, we figured out how to irrigate, we farmed for ourselves, We created these small city states. We then industrialized. And I don't know. I'm still working super hard, Nick. I mean, it feels like you are too. I mean, you're a very productive project guy. And yet we have all of this technology around us.

10:08And we have all of these things that have made us more productive. I guess what I'm getting at is, let's say we go to that next level where it's writing our papers. It's writing our books. It's doing our trades. It's doing our stock market transactions. It's doing our medicine. Does it mean that we unplug or do you think we quantum leap again into something else more productive? Well, right now, I mean, most of the work I'm doing is because it's the only way for me to achieve various things. Like, for example, the book, I couldn't just press a button and have the book written. I actually had to spend the time writing it myself.

10:48right? Then you probably have a whole bunch of goals you're pursuing that require you actually to invest time and effort and skill. Now, in this scenario where AIs truly achieve a general intelligence and then super human levels of general intelligence, I think it is much more difficult to think of tasks that we would actually be helpful. I think there are like a few, you could look at cases where say consumers have a direct preference that the work be done in a certain way so right now maybe some consumers pay a little bit extra for a trinket that was made by some politically favored group or some indigenous craftsperson as opposed to produced in a sweatshop like in indonesia or something right even if the actual trinket is equivalent we some people care about how it was made so that would be an example where maybe human work would still be needed or are people maybe prefer to watch human athletes compete in the olympics even if there were robots who could run faster or box harder or whatever but so with with those carve-outs though i think it does look like we could have full unemployment full full unemployment right that's kind of the goal of ai yeah exactly no one needs to work right so but right full unemployment but when you say unemployment in our society, people are concerned, but in this society, there's so much abundance and there's so much wealth.

12:14It really doesn't matter. You won't have to work. Right. Yeah. So, and we see like our education system now, for example, to a large extent is structured towards producing workers who can be productive and contribute to society. So you take kids in, you put them at the desk and you're training them to do as they receive assignments and they're graded and they have to be disciplined. And all of that is because right now in the world, there are a lot of jobs that need to be done. So we need to have people who go into offices and get assignments and et cetera. Now in, in, in a future scenario where we didn't need people to do that, then I think we would want to like change how we think about education.

12:53For example, like maybe kids could be educated to enjoy life and to develop appreciation for, you know, the art of conversation and literature and hobbies and appreciating nature and all kinds of other things instead of like being economically productive okay i mean this is super super valuable i mean you're you're i mean i think this is a fantastic book for so many different on so many different levels because it's very very thought provoking so let me ask you this nick what what timeline do you give this similar to your last book could we see deep utopia prop the problems that you describe coming sooner than perhaps anticipated?

13:36Um, yeah, I think timelines have been fast since super intelligence came up, uh, came out, uh, the last 10 years in AI have been really remarkable and, um, I see no signs currently of it slowing down. So I think there is a real, um, chance this might happen in the lifetime of a lot of people alive today. and it will be not just the biggest event in our lifetime, but perhaps in the history of the human species, this transition to the machine intelligence era. And it's remarkable just over the past few years to see how much the public discourse has changed. Like when I was writing superintelligence, this was a completely neglected topic.

14:19Nobody in academia, it was kind of people snickered at it, ha ha, science fiction, futurism, whatever. And now, of course, world leaders are discussing transformative AI. All the leading AI labs have teams specifically researching scalable methods for AI alignment. And it's been like just a radical shift where these ideas that for many years, you know, I was talking about a few of the colleagues. And I think, yeah, increasingly this just will become a main focus for public conversation as more people realize what's about to happen. Lowe's knows how to help make your home holiday ready for less.

14:55Get Select Style Selections Vinyl Flooring for just$1.99 per square foot and have it installed before the festivities begin. Our team can help you every step of the way. See a Lowe's Red Vest Associate or visit lowes.com slash holiday install to get started. Lowe's. We help. You save. Basic install only. Date restrictions apply. Subject to availability. Install by independent contractors. See associate for details. contiguous US only. This episode is brought to you by Greenlight. Get this. Adults with financial literacy skills have 82 % more wealth than those who don't. From swimming lessons to piano classes, us parents invest in so many things to enrich our kids' lives.

15:33But are we investing in their future financial success? With Greenlight, you can teach your kids financial literacy skills like earning, saving, and investing. And this investment costs less than that afterschool treat. Start prioritizing their financial education and future today with a risk-free trial at greenlight.com slash Spotify. When I finished reading your book, I wanted to ask you this question, but this is a big permutation. So we live in this great mystery, of course. We don't know our origin. We don't understand what happens to us after our demise. Seems like we're a fairly sentient, fairly intelligent species, but yet we live under this cloud of uncertainty and this mystery.

16:17Let's say that artificial intelligence was able to resolve that. Suppose we were able to understand the great mystery of what we're living in. The question I have for you, this is a little bit more of an esoteric, ethereal question. Does that make our lives more purposeful or less purposeful? Meaning, I sort of feel like the specter of death forces lots of meaning and lots of intensity in our lives. But suppose we had an artificially intelligent situation where we could live forever. But even if we didn't live forever, we understand the great mystery. What do you say about all that? Yeah, I mean, I think it's possible.

16:58It might make our lives less purposeful after this has happened. I think it makes them perhaps more purposeful right now when we have this immense obligation to try to make sure that this actually pans out right. It might be that the human lives we're living now are much more consequential. Maybe it will shape humanity's destiny for millions of years to come, much more so than any other people who were like previously in history. So it has this paradoxical effect that increasing the purposefulness of current lives. If we do succeed, like then, you know, we're home and dry and we could maybe spend our time enjoying the future rather than be driven by some very strong purposes.

17:41Yeah, but again, I guess that's the issue. So if I lose my purpose or my current trajectory of purpose, do I find less meaning in life? I guess that's the big question. Yeah, yeah. So that's exactly the kind of question that the book really dives into. And because a lot of the discussion about these issues has so far been very superficial. So people think, well, what if there were like some increase in the unemployment rate? Like what would we do with training programs or UBI or whatever? And they're kind of people thinking stop there. They don't think through, if you actually think through this whole situation where robots can do not just some tasks, but all the tasks and then what happens.

18:22So that's why I wrote this book. And so there's like, yeah, really diving into that, like about meaning and purpose, as well as other values that we have. And I think that there are significant challenges to those values in this condition of technological maturity. and we will need to i think rethink and maybe drop some of our values that we currently hold there others would survive and kind of re reconsider what what really gives human life meaning at the fairly fundamental question i'm ultimately positive that there is a good outcome of that like if you go through this process you will it will be different not everything you might value about the current world would still obtain, but there will be other values.

19:05And overall, I think it might be such that if people look back on the current time, they will just shudder in horror at what kind of utterly atrocious lives we were leading in 2024 by the comparison of our descendants or ourselves, if we make it through. It's interesting. We're doing that already though. See, we were looking back 100, 200 years ago. were saying, oh, wow, our ancestries, there were slaves, there was mistreatment of different people due to their skin colors, a result of which were rebuking and condemning people from 200 years ago. I guess the question is in 2224, perhaps there will be a very large group of people that rebuke us.

19:55They'll say we were spending too much time emitting carbon. We were spending... Whatever it is.

20:03I don't know. That's a very good point. I think it does behoove us. When we look back and judge all the other previous generations, we need to also stop and reflect how will we look in the eyes of posterity. Probably we will be appearing pretty rotten as well and laboring under huge moral errors and stuff. I think that is probably good for wisdom to pause every once in a while to reflect like at how much we currently take for granted as just everybody believe this clearly must. And yet we look at every other generation and we now see that they were wrong or misguided about many things. And that's surely holding true for us, but even just setting aside the moral failures of earlier generations, um, along with their, their heroism in other respects, like just say the sheer grinding poverty that most people experienced a hundred years ago or 200 years ago.

20:58we wouldn't want to go back to that. But I think our current age, even in the richest countries, you know, in the richest strata of society, will seem equally impoverished. Because although maybe like some people can buy a lot of stuff, there's a lot of things you can't buy with any amount of money. For example, perfect health, like the ability to live to 200 years, the ability to upgrade your brain to get like, you know, IQ 400 or the ability to feel happy all the time. Like you can't just spend X money and buy happiness, right? In the future, these things would actually be possible. There would be technologies that would allow you to achieve those effects.

21:39And that might be very cheap. So we, you know, I made this observation. I want to get your reaction to it. I feel like there are parts of the world that are becoming less religious, more secular, more atheistic, frankly. but then at the same time that that's happening there are other parts of the world that are becoming more fervent like more dogmatic about religion and so uh what why is that in your mind and then if we enter this deep utopia what happens to religion um yeah i don't know the answer i think potentially it could have a larger role in this future utopian society in as much as there would be fewer other things to compete with it so so right now even if you are very religious you might still need to spend most of your day just getting by making a living you know tidying up your home like doing all kinds of practical things um if you didn't have to do all of that you could focus more on what ultimately matters, like contemplating the divine and developing your relationship to God, et cetera.

22:53Now, ultimately then it's, it's a question of whether people would choose, maybe people will choose different things there. But I think it's like one of those values that is likely to survive the transition. So many other things like that priding yourself of being a breadwinner, right. would kind of be undermined if all the bread just is delivered automatically without anybody having to make an effort because robots are doing the baking. But these more spiritual values would be plausible candidates for what remains in that condition to focus on. Right. Unless, of course, we figured out what the great mysteries are and then maybe we have more or less religion.

23:34It's very hard to know. I mean, it depends on what the answer is, right? Like what actually is behind It depends on what the answer is. Although I will say this, I was in Europe this winter and they were using a church for a conference facility. And I thought of my Catholic grandmother, I think she would have been horrified that they had repurposed the church for business meetings. Yeah. And so, but yet there's other parts of the world where the religious fervor is perhaps as intense as it's ever been. So it's an interesting time to be alive. We are a motley bunch as a human population. What surprised you the most?

24:14After all of your years of focusing on this, Nick, when has AI and then humanity surprised you the most? Well, specifically with AI development, a surprising thing is how anthropomorphic the current leading edge AI models are. If you look at ChatGPT4 or one of these other systems, they really are in many ways very similar to a human mind, including even in some of their foibles. So with some of these systems to get the best performance out of them, you almost have to give them a little pep talk in your prompt. Like in Kurt, you're going to think really carefully about this thing step by step. This is important.

24:54Like the idea that an AI actually gives you a better answer because you say those things to it before, that would have seemed utterly ridiculous 10 years ago. Right. That's not how computers behave. and yet that's where we are today. So that's surprising. Also like that we apparently have a fairly extended period of time when we have AI systems that are roughly human-like and practically useful for a wide range of tasks, but not yet radical superintelligence. It was not clear that that would be an extended period of many years where that was the case. You could imagine a different scenario where basically AI did nothing and then somebody in their basement figures out like the key missing thing and suddenly it fooms and becomes super intelligent over a week or something.

25:40So those are two. And I think this relatively slow, more continuous, gradual pace of development in AI has also enabled more people to start to realize what is going on. So you do see more involvement from governments and et cetera, investors in AI development because it's not so much foresight, but they can kind of see year by year capabilities increasing and then more people can sort of like connect the lines than if it were more coming as a bolt from the blue. Elon Musk sometimes teases that we're living in a simulation. You think we're living in a simulation, Professor? Well, many have asked and few have been.

26:22I was the person actually who wrote the original paper on this simulation argument back in 2001 or something um i i think it's like there's a significant chance of this but i have um refrained from trying to attach a particular probability to it yeah well i mean one of the things that would reinforce it is that we've got this world that uh you know it's sort of incomplete but man or woman has been left to complete it right i mean you you know we we didn't have we have the capability of broadcasting and using radio transmission but a thousand years ago or two thousand years ago obviously we didn't understand how to deploy it or use it same thing with steel manufacturing or other things so it feels like the world was put together and left for man and woman to complete does that make sense um yeah although that could equally well be the case if we are not in a simulation, it seems.

27:25Right. So I'm not sure whether that particular property of the human experience tells something like one way or the other with respect to the simulation hypothesis. Right. Right. Well, we'll sit. All right. Well, we're at the, I have at the end of my podcast, I've come up with my production team, five words or phrases. I'm going to read out the word and then I'm going to ask you to react to you. You can give me a sentence. You can give me a word or some thoughts. Okay, you ready? So we're going to start with the word future. I say the word future. Nick, what do you say?

28:03And so I'm supposed to answer with one word or just a phrase? No, just whatever's coming into your head. You can give me a paragraph if you want. I don't know. Hold on for dear life. Hold on for dear life. You see so many different permutations of what the future could be that it's uncertain. Yeah, buckle down.