#181 Nick Bostrom: The Meaning of Life in a World where AI can do Everything for Us

0:00If you start to think it through, it's not just human economic labor that would become superfluous in this scenario where AI succeeds and we end up in what I call a solved world world. but all kinds of other instrumental efforts as well. And then that raises these questions about, yeah, like could you still live a good life in those circumstances? And if so, what that would look like. Hi, I'm Craig Smith, and this is Eye on AI. Today I speak with Nick Bostrom, the philosopher and author of Superintelligence Paths, Dangers, Strategies. In Superintelligence, Bostrom explored the potential future of artificial intelligence and the existential risks it may pose to humanity.

0:46His work has been instrumental in shaping the discourse around AI safety and has influenced many of the leading minds in the field. Today we'll be discussing his latest book, Deep Utopia, which asks the question, how will humans derive meaning in a world that is technologically solved, where AI and robotics can do all tasks better than humans? It's a fascinating, if disturbing, discussion. I hope you find it as fascinating as I did. Hi, I wanted to jump in and give a shout out to our sponsor, NetSuite by Oracle. A little quick math. The less your business spends on operations, on multiple systems, on delivering your product or service, the more margin you have and the more money you keep.

1:37But with higher expenses on materials, employees, distribution, and borrowing, everything costs more. So to reduce costs and headaches, smart businesses are graduating to NetSuite by Oracle. NetSuite is the number one cloud financial system, bringing accounting, financial management, inventory, HR into one platform and one source of truth. With NetSuite, you reduce IT costs because NetSuite lives in the cloud with no hardware required, accessed from anywhere. You cut the cost of maintaining multiple systems. You improve efficiency by bringing all of your business processes into one platform, slashing manual tasks and errors.

2:26Over 37 ,000 companies have already made the move. So do the math. See how you'll profit with NetSuite. Now through April 15th, NetSuite is offering a one-of-a-kind flexible financing program. head to netsuite.com slash ionai. Ionai all run together, E-Y-E-O-N-A-I. That's netsuite.com slash ionai. Again, netsuite.com slash ionai. Let's go ahead with introduce yourself. Well, I'm Nick Bostrom. I'm currently a professor at Oxford University, where I've been since 2003. Then before that in the U.S. for a bit, did my PhD in London. I grew up in Sweden. You're best known in the United States, at least, for your book, Superintelligence, which didn't directly address the question of the meaning of life in a world dominated by superintelligence.

3:34It was more about the risks and managing the development of such a technology. But before we talk about your notebook, can you talk about the recent advances in AI since you wrote super intelligence because uh it's certainly come a long way and a lot of leading researchers are now uh talking about agi within the decade and super intelligence presumably would follow fairly closely on agi super intelligence came out in 2014 and that roughly was also when the deep learning revolution began. And so we've seen a string of apparently accelerating advances since then. It's a very different era now. Back when I wrote superintelligence, the whole idea of AI eventually succeeding at becoming fully general and being able to do everything humans can do was already a very radical and sort of fringe thought.

4:45certainly in academia, mostly dismissed as sort of science fiction or idle speculation. And of course, the further step that it could then become super intelligent, even more so. And I wrote the book to try to draw attention to the set of risks that would arise if we do attain transformative AI and super intelligence. Now, of course, that has become completely mainstream. all the leading AI labs have research groups focusing on AI alignments, trying to develop scalable methods for AI control that could continue to work even at sort of arbitrary levels of capability. And just in the last couple of years, we've seen a big shift in the public discourse and even top-level policymakers are now paying attention to the possibility of transformative AI, including existential risks, with statements coming out of the White House and the global AI summit in the UK and really around the world.

5:45So, yeah, a lot has shifted in terms of how this set of issues is viewed. Has that changed your view on superintelligence or it's the possible advent of superintelligence and the timeline? I guess gain some confidence just from seeing that things are actually playing out, no matter how sort of coherent some argument seems theoretically. Like it's hard to be really convinced in your gut until you also start seeing it around you or at least see a whole bunch of other people also sharing your beliefs increasingly. I think that there is more granularity in the precise forms that AGI might take. I mean, with the rise of the current large transformer models, and it's strikingly how anthropomorphic these AI systems are in many ways, that it's almost like they have a psychology, like a human, like with foibles.

6:53Sometimes they need to be encouraged in the prompt to do their best. which is kind of weird from like certainly from what the old school AI used to be right it wasn't that you needed to give it sort of emotional support to encourage it to perform and we also see leading edge systems being very compute intensive and so that's seemed always kind of likely but it seems more likely now that we see that the frontier systems are kind of gaining in capability as you throw more compute at them. And so it does certainly shift probability mass a little bit and concentrates it on certain kinds of scenarios.

7:43And we might be on the sort of slightly shorter end of the timeline. It was always a wide probability distribution. And roughly, you know, we are more or less kind of on track, i guess of what i would have thought would be the sort of the median of that probability distribution but maybe slightly faster than than expected yeah yeah i mean there's uh i don't remember whether it was in your book or kurtzweil's ever but there's this hockey stick graph uh that you know and at a certain point of the machine intelligence surpasses human intelligence and it's beginning to feel like we're on that curve, the more acute curve now than we have been.

8:32Do you have a view on that, of where we are on that graph, if you believe in the graph? Yeah, well, I certainly think things seem to be moving quite rapidly, and I mean we still have uncertainty, but But yeah, I think non-trivial probability mass on sort of single year timelines. But with remaining uncertainty, it could take longer as well. But it's been, I guess, surprisingly smooth in terms of the growth of capabilities. It seems kind of year by year, they have become incrementally better. so that might continue in which case we might have more of a sort of a median speed takeoff I don't think we should be too confident in that though because there could still be a transition point when you have AI systems that maybe incrementally and slowly improve but then at a certain point they become capable of very substantially driving further advances, like particularly when you can sort of fully substitute for the labor of human AI researchers and chip designers, et cetera, then it might still be that it kind of goes smoothly, but fast up to a certain point.

9:59And then you might get an intelligence explosion when you hit this kind of strong feedback loop that I think remains in the cards. Yeah. How do you feel about the threat debate, which your book contributed to. I mean, I see the paperclip analogy all the time, that sort of nightmare scenario. But since Max Tegmark's group came out with the pause letter, I mean, that did seem to focus regulators and researchers on safety. but at the same time it spread a lot of fear and mistrust in the public which I found unfortunate. Yeah, that's maybe the most dramatic shift since 2014, the mainstreaming of this AI doom concern, which I think is broadly for the good in that there is now actually research done to try to find solutions to this.

11:08Whereas back then, there were kind of a handful of people scattered around the world communicating on the internet. But now there is more resourcing for that and talent flowing into that, which is positive, I think. But it's reached a point where I'm thinking one might soon start to worry a little bit also about the worry overshooting its target. I don't think we're quite there yet. I still think a higher level of concern than we currently have would be optimal. But humans being herd animals, like once a stampede starts, it's very difficult to fine tune it. You can maybe set it off, but you can't necessarily call it back when it's reached the optimal point.

11:53So I still think it's unlikely, but less unlikely than two years ago, that we could end up somehow on a trajectory where AI has never developed because it is either permanently banned or sort of moratorium that they lay it long enough that we then destroy ourselves in some other way instead without ever even getting the chance to roll the dice with AI. It's like a fine balance, and it would be good if instead of being super polarized into sort of AI throughout all the stops, let's just rush forward, there is no risk at all, at least no existential risk that's just silly on the one hand and then on the other hand

12:38I wouldn't call it doomongering because I do think that the existential risks are very real but I think at some point there is a certain amount of risk that we will need to take in order to have a shot at realizing the benefits and there are risks also not to going forward with this that should be entered into the balance yeah uh the latest book deep utopia of uh considers uh i mean the the the main theme is how do humans derive uh meaning and and purpose in a world that's technologically solved by ai is that did how did that emerge from from your work on super intelligence was is is that sort of the next step in your thinking or is it because the progress in ai has been so rapid that you uh you started thinking about what what happens if we really to reach technological maturity, as you call it?

13:44Both sides have always been there in my outlook. Superintelligence focused mostly on the downsides, what could go wrong, because at the time that seemed the most urgent thing to bring attention to, to get a clear understanding of where the pitfalls were so that we could then hopefully steer clear of them and to develop some conceptual framework to allow people to start to make progress on this, to develop solutions to scalable alignment, etc.

14:19Now I think that there is much broader recognition of that. So I think there's less value perhaps in just repeatedly harping on the risks because there are a lot of people now actually recognizing that and working on it. And I think there is like kind of still a certain naivete and blind spot in our conception of what happens if things go right with AI. And we've begun hearing some conversations about impacts on the labor market. You know, what if AIs and robots can take over more human jobs? you know will we need some support for the unemployed like in the more radical conceptions even the question what if ais could do all economically valuable labor at some point in the future you know maybe with some carve-outs then what like do we need like a universal basic income how would the education system need to change and i think we will hear more of those debates, but they still, I think, take only one step on a sort of path that allows many more steps to be taken.

15:31If you start to think it through, it's not just human economic labor that would become superfluous in this scenario where AI succeeds and we end up in what I call a solved world, but all kinds of other instrumental efforts as well. So if you imagine somebody who doesn't work today. Maybe they have entered early retirement. Let's suppose they are rich, and so they don't need to work. But they presumably still spend a large fraction of their day engaged in various activities, which has the structure that they do a certain thing, put in a certain effort in order to achieve a certain outcome, separate from the effort itself.

16:13From the trivial, like Bill Gates brushes his teeth every morning, and he has to do that. That's the only way that he will have, you know, a healthy mouth, right? And if he wants to be fit, he would have to go to the gym. If you want to have good friendships, you have to put time and effort into, you know, talking with your friends, understanding them, you know, being there for them if they hit a difficult spot, etc.

16:42If you want to have the particular kinds of items that you would enjoy. You have to maybe spend some time going shopping or looking for different things to find exactly. But all of these activities as well, you know, could be automated at technological maturity or other advanced technologies would make them unnecessary. So like if instead of having to go to the gym to be fit, you just pop the pill and it would have the same physiological effects, then would there still be any point in going to the gym? If instead of going shopping, which some people enjoy, you had recommender systems that would be able to pick out better items and then just order them, it might kind of undermine some of the point and purpose in the shopping activity.

17:30Because at the end of the day, you end up with worse items than if you had just let the AI do its thing. It kind of at least puts a question mark about that. It would kind of seem to lose its allure. And if you go through and there's like a bunch of case studies, it seems like a very large fraction of what fill our lives today when we are on holiday or retired or even when we don't work is of this instrumental character. And at technological maturity, if things go well, we would enter, I think, a post-instrumental condition where, to a first approximation, it would seem that there are no instrumental reasons for us to do anything, to put out any effort.

18:07um and then that raises these questions about yeah like could you still live a good life in those circumstances and if so what that would look like i mean you use the example of bill gates or or someone that's wealthy and doesn't need to work but there are a lot of people already who are unemployed or retired laying on their sofas, flicking through television channels and experiencing either depression or, as you called it, a deep existential malaise. And, I mean, this is something that concerns me about the United States, the state of education. And education takes effort. It's uncomfortable to carve those neural pathways until you reach a certain critical mass, when then it becomes more enjoyable.

19:10And large swaths of the American public are pretty uneducated and easily manipulated and not content with their lives and all of that. So if the world is technologically solved, I don't see the motivation for people to go through the difficulty of getting educated. And without education, to me, it looks like a very dark future unless as you talk at one point in the book about neurotechnological interventions, you know, sort of stimulating the brain in certain ways to overcome, you know, lack of purpose or lack of motivation. Or as you intimate at the end of the book, using recreational drugs to alleviate that malaise, How do you see that looking at sort of the common man as opposed to the educated elite?

20:28How do you see that developing in the technologically mature age? Yeah. So technological maturity is this concept I have for a condition where basically all technologies that are physically possible and that are in principle feasible to develop have been developed. And with superintelligence, that might be attained relatively quickly because the superintelligence would then be doing the further R &D. And yeah, you would then enter this condition of technological maturity where you shouldn't just think of it as AIs and robots, but all kinds of other things. So with learning, it's one example of an activity where right now you have to put in effort in order to achieve a certain outcome.

21:20Like if you want to understand mathematics, the only way to get there is by first studying mathematics and you have to put in the time to read the textbooks and do the exercises and there is no shortcut. You can't pay to know mathematics without studying it. But at technological maturity though, I think that would be a shortcut. That would be technologies that that would enable you directly to have your brain altered into a new condition where you possess arbitrarily high levels of knowledge of mathematics. If we're still organic, you could imagine some either like a neural interface or some sort of swarm of nanobots that go around and changing the weights of your synopsis, all sort of masterminded by some unfathomable super intelligence.

22:07If you are uploaded at that point, then it's presumably even easier to go in and edit the file of your weight matrix of your brain. And this starts to show just how deep this problem goes. Like, it's not just that there are a few sort of chores that we no longer have to do, but even very fundamental things like learning and understanding our own personality. also becomes malleable in this condition, like in a plastic world. And so, yeah, I don't think the problem would be that humans, unless they are somehow still finding the motivation to sit through 20 years of education, would remain incompetent and ignorant.

23:02Like, I think there would be shortcuts to that.

23:07But those wouldn't necessarily involve an extended period of you yourself putting out effort unless you chose to do it the hard way. So the book, yeah, basically brackets all these practicalities. It doesn't talk about how we would get from here to there or what the risks are or what the policy solutions are or how society should be organized. All of those are, of course, super important questions, but not discussed in this book. I basically just assume that we reach this condition of a solved world in order than to be able to actually explore the question of what happens then. Is it possible for a human to have a future trajectory that leads there that still constitutes a good life?

24:00and I think ultimately probably the answer is hopeful but it is not there is a sense of ambivalence I think it would require like it will feel quite alien probably to us now at least until one gets used to it and a lot of the things that kind of define the current human condition would likely go by the wayside we are so shaped by these instrumental necessities that we confront in our lives that it's hard to imagine what life would be without them. It's almost like a little bug that has an exoskeleton that holds these sort of squishy parts in place. And if you imagine taking that off, then what becomes of the insect?

24:42Similarly, these instrumental constraints are kind of an exoskeleton for our souls. And if we remove that by being able to automate everything and having a plastic world, then one possible outcome is that we become kind of blobs, pleasure blobs, like kind of some super drug is giving us immense pleasure, but we have no real character, no structure to our existence, no effort. And that is one possible vision of a utopia. I think it is easy to dismiss it very quickly. I think it's actually a very deep question whether there is something better. probably there is and that's kind of what the book explores the more subtle values that could give shape to our lives the the idea of being able to upload education into your brain is a hill beyond what i was looking at because um short of that is where i see the dilemma if you could do that then you could engineer happiness you can engineer fulfillment yeah subjective happiness yeah like any feeling you want so this is another instance of something that would fill the time of a leisured person today there's a lot of stuff they have to do put out effort in order to experience joy and pleasure and subjective happiness but there would be shortcuts to this I mean in fact there already are today it's just they're not very good like drugs that could sort of temporarily give somebody a great level of subjective well-being without putting out any effort then it's just that they come with side effects etc but like obviously it could develop super drugs in the future that would not be addictive or have direct ways of manipulating or altering the neural pathways that underlie our sort of hedonic experiences So, yeah, it becomes quite a radical question.

26:40It really starts to, when you sort of take this to the extreme, you really confront these very old questions that philosophers have wrestled with about the ultimate value that we have, the meaning of life, the purpose of human existence. You see them, though, from a very different angle. when you look at them under these extreme conditions postulated in this book. It's almost like a particle accelerator where you smash nucleons together at extremely high speeds and at these immense energies, you then see their constituents. You may see quarks flying around and stuff like that. And then you can infer that those same constituencies are present in ordinary matter at other times as well.

27:33It's just we can't observe it. But just, and so similarly here, you can think of it as a philosophical thought experiment. You consider this very radical condition where the world is completely plastic, and then you sort of see what happens to our values in that scenario. It's like smashing them together and seeing them. We are forced to look at their basic constituencies. Now, you could read the book as a philosophical thought experiment like that, which could be of interest even if this was guaranteed never to happen. To me, though, it is also not at all impossible that we will eventually reach something like this if things go well.

28:13So there is also a kind of more practical motivation for thinking about these questions. Yeah, although if sort of neurological engineering, if that's the end point, then a lot of these questions are kind of moot because you can solve them through engineering of the brain. But short of that is when it seems to me there's a crisis. And also, I want to ask, why do you think of this as a hopeful or optimistic book? Well, I mean, the reader can decide for themselves. It's not really a book that, it's not a book of conclusions so much as it's a book about, that tries to help the reader to bring certain questions into focus.

29:10focus and yeah I think it's ultimately a helpful book because I think there is something extremely valuable that could be constructed in this condition of a plastic world that will be very worth having but I'm not pretending it's like very obvious I mean there is an aspect to it that is obvious which is the most obvious is like all the horrific negatives that pervade the current human condition with

29:54people dying in horrible cancer pains and kids not having enough to eat to develop properly and bombs blowing up in neighborhoods and ruining people's lives and somebody getting a call and their spouse has died in a car accident. As well as the smaller things that we shouldn't forget that just makes a lot of ordinary existence and drag, like the little headache that you have or all these kind of things that just the boredom of many jobs, eight hours every day, feeling tired and doing something pointless that you don't enjoy like just getting rid of that all in itself i think would be an enormous boon um and then there's the extra on top of that which i think could be uh immensely valuable as well yeah you said at some point in the book you were talking about thaddeus metz's philosophy of, or his theory of meaning.

30:58And one of the things that he talks about is striving and then achieving a goal. And a lot of things you just mentioned, certainly there's a lot of pointlessness to it, but maybe happiness and fulfillment exists in contrast to despair and disaster and frustration. I mean, if you take those things away, then fulfillment and happiness may lose some of its value on a relative scale. On a relative scale. I mean, I know some people who are just kind of basically happy all the time. They have personalities that are naturally sort of cheerful. and I don't know, they seem pretty happy and I'm not sure it's taken away from them.

31:55And other people who are like depressed all the time and their suffering is not less because they might not have the bright moments to contrast it with, I think. So I don't know about that.

32:10And certainly the subjective aspects would be completely up for choice in this world. Now, Now, I should say, at the first approximation, like I said, the challenge becomes, it looks like there is nothing we need to do for instrumental reasons in this condition. Now, I think there are important qualifications to that. If you look more closely, there are various activities where there might still be a need for human effort. It might be most obvious in cases where you have consumers that have a preference that a certain thing be done by humans. And we already see this, like maybe people prefer to see humans compete rather than robots in an athletic race or something like that.

32:56Sometimes people might pay more for like a trinket if it was done by a certain category of person that they like, like in some indigenous community or like rather than in a sweatshop in Malaysia. and so already we sometimes have sort of products that achieve higher value not just because of what they are but because of the specific preference that generated them and we can have preferences over that and so to the extent that we do even at technological maturity a preference that certain tasks be performed by humans that could then create instrumental reasons for other humans to perform those tasks and i think this generalizes and there are many more subtle forms of this where there might be cultural traditions that we want to uphold.

33:43And the only way to do that is by doing a bunch of stuff ourselves. It wouldn't count as carrying on the tradition if we just programmed a robot to do the ceremonies and to, you know, do the mass or like whatever the spring, rite of spring, whatever the different things. So as you start to zoom in, I think there are possibly a lot of reasons we would have for doing things ourselves. And more generally, I think there might be a bunch of quiet values that we can't hear very much right now because we're living in this den of screaming moral imperatives all around us. There's like, you've got to make a living.

34:27Otherwise, you've got to be kicked out of your apartment. Like, you've got to help this needy person because they are really suffering. You've got to do this, that, and the other. And these should, I think, have priority in our current condition. But if we imagine somehow all of that went away, then I think there are these more subtle values, like some of them quite like almost of aesthetic quality that then ought to have a bigger influence on how we spend our time and our behavior. It's almost like when you go out at night, your pupils dilate and you realize that there is this big starry sky up there.