#52 - Will MacAskill - What Comes AFTER Superintelligence?

37:26Yeah, I mean, I think that's actually maybe an interesting spot where we can start opening this up, because if we want to start painting a picture of this viatopia, then we need to start thinking about what are some of the core things that everybody agrees on that most people would want. Hey Will. Oh hey Richard. Thanks for that talk. It felt like one of the most powerful things you said was when you were talking about moral fashions and the sense in which our future is dictated by these seemingly random currents, especially as a moral philosopher where you have all sorts of arguments and reasoning and somehow society is swept along by something that seems not only to be ignoring a lot of our best moral reasoning, but maybe even actively opposing it or defying it.

38:20I guess there's two broad perspectives you can have on this. One is that these moral fashions are essentially random, and we just have to cross our fingers and hope that we get swept in good directions. Another would be that there's a way of understanding the underlying logic beneath them. You could call that psychohistory. You could call that sort of a more formal understanding of sociology. I'm curious which of these two hypotheses you tend to believe in and insofar as you believe in the latter what might it look like for us to actually be able to steer these forces rather than just getting swept along by them okay great question so looking to history first the question is just to what extent have these moral fashions just been totally random or have they been kind predictable in some ways.

39:16So I guess I think there are certain forces that operate on people's moral beliefs over time that have varying degrees of strength.

39:28a belief that it's wrong to kill all of your family and everyone you know. That's very strongly selected against in terms of cultural evolution. like if a society kind of or culture like develops that then you know they're not going to last very long and I think there are you know some things that are like not as strong as that but like liberal egalitarian kind of post protestant ethics like weird this is Joseph Heinrich's term western educated industrialized rich and democratic but like yeah the kind of ethical beliefs and the general worldview we have is very unusual among cultures but seems to be very good if you want like a thriving post-industrial economy so there's like decent selection pressure there then for some things like yeah at least attitudes to recreational drugs um imprisonments attitudes to meat eating definitely like diet in general i think that's there i'm starting to think like the selection pressures are much weaker and it really can just be like a matter of quite a lot of randomness.

40:46In particular, you can have these kind of multiple equilibria where one religion can say, well, the way you demonstrate piety and the way you demonstrate... So maybe there's a selection pressure towards having signals such that you can demonstrate self-restraint so that you can signal that you're a conscientious person. And I do that by abstaining from alcohol. A different culture does that by abstaining from pork. you know other cultures could do it in other ways too there okay that's clearly now a case where you've got kind of multiple equilibria and then it's really maybe quite arbitrary kind of which one you end up on and well what do i think in the future then um there are these stories that well what you end up with is just like whatever is most economically favored over time so some societies will want to just get really powerful and have loads of children and they'll become the most powerful but my guess is that like ai locks us in before that point because uh before the point at which all of these evolutionary dynamics can kind of play out um they will either be kind of one country one state that's kind of like dominating all others or like a sufficient coalition that they can then foresee like okay well this is how things going to play out we don't like this.

42:08We're going to intervene. And so that means I guess I am expecting quite a lot of arbitrariness in fashion, even at that point. You don't think AIs will have moral fashions themselves?

42:25I think my best guess is it would be

42:32yeah, my best guess would be no because my best guess at how things go is that AIs are being trained very hard to be aligned, predictable, faithful servants to, I'm not claiming this is the best outcome, but what I expect, faithful servants to their human masters. And if any behavior that renders AIs unpredictable or going to against what the humans that are in control want them to do, we'll just get trained out of them. Josh? It seems to me that we're actually at an epistemological all-time low in civilization, where we perhaps have more information than ever and more divergent opinions than ever of what's actually going on, which is not what you would think would be happening if we have so much information.

43:21but I wonder if the dividing line between dystopia and utopia in the future will be if AI improves those epistemics so that as a species we can all figure out what to do about the planet and reality from a more shared base reality versus do we have some kind of super intelligence that babies us and keeps us in Plato's cave like never before and harvests our energy or something like that, or thoughts. And I'm wondering if you have any ideas on that. And before I hand it back over, I'll say that just, it does seem to me that the incredible division we're seeing globally, socially, politically, when you study these different arguments, these different sides of the argument, they're not even arguing about opinions about the same thing, they're arguing about completely different things.

44:15So it matters whether you think the southern border of the US was kind of loosey-goosey for humanitarian reasons or whether you think it's a nefarious plot to end democracy. Like it matters in the conversation what the real motive is. It matters in the drug conversation to know which ones are actually addictive and which ones have like no hope of ever being addictive. And yet that's not common knowledge in the conversation. You can still go, I hate to even mention it, but you can still walk around and take a survey of who thinks a certain politician was nearly assassinated or staged his own assassination.

44:54Like, there's no way we can survive as a species if we're on that much of a divergent view of even base reality. So if AI could help us improve the epistemics of our civilization, that's got to put us on a better course, I would hope. Any thoughts on that? Yeah, I mean, I think this is one of the big things that we can be doing right now. And one of the things, yeah, this institute, Ellen Forethought, works on is kind of AI for better human reasoning, better decision making, and so on. Where, I mean, I think it can really go either way. Like, maybe AI can make us much more enlightened. Maybe it'll make us much worse.

45:35the kind of bull case for thinking it'll help is that you can get a single ai that can just have much wider longer track record of being accurate um than any human can so you can have an ai it's like look it's a forecast a million different things over the last questions over the last year and it's got it right like way more than like any kind of human can and like also you ask it any question like it'll give you just the best answer you try and verify it from like others and you know you can tell um just you know an individual human we only have a certain number of years by which we can build up a kind of reputation as like being a good thinker and having a good track record for the liability and so on yeah i can have like the equivalent of millions of years of of track record and in light of that perhaps people will be more willing to defer because they think or you know on the questions where uh perhaps they originally had like a counter view um you know so uh okay well let's say i has gotten it right on like all of these other topics so maybe it's right on vaccines too um that's the kind of like optimistic case optimistic scenario we could be in.

46:55I think the more pessimistic scenarios are, one is just where basically for every possible claim, it gets defended by, you know, imagine just any claim you had an army of a million of the best debaters and retleticians and so on making the case for that claim and the opposite. And now you as a human are like trying to adjudicate. seems like very difficult indeed. That does seem like one of the more fundamental problems we're going to be dealing with in the near term is that the AI is born of human recorded text, which we know is notoriously biased and some of it's outright wrong. And so that's in the weights of the model.

47:40It's in their subconscious, essentially. And then when they go to do a fact check or a verification, they're going to the same sort of information cesspool to do verification. and is there a way out of that? Is there anything other than maybe if AI has access to telescopes and robots to do archaeological digs itself it can go to real primary sources but in the meantime it's just got this garbage dump that we've left it to sift through. Well yeah, so here's one way. Again, a kind of cause for optimism. So this, it would be kind of costly because you've got to label the data in the right way but you could train an AI in the following way.

48:16So all of your data gets labeled chronologically. And then you train the AI over time. And then let's say it stops in January 2020. But at that point, it's got very high capabilities. And then it has a set of forecasts of what's going to happen over the next day, week, month, years, and so on. And then just learns over time. you could have an AI that has the equivalent of you know millions of forecasts every single day for the last five years or ten years that has then been able to kind of learn from over time in a way that we know we can't do that for humans because if I ask Liv like oh imagine it's January 2020 like what might happen in the next few months like you can't like screen out your memory but you can do that for AI.

49:09So potentially, AI systems could actually learn much, much more from the world than human beings are able to do, because we're able to curate the information chronologically in this way. And the predictive power will be one indication, I guess, of whether these models are on track, you're saying? Yeah, exactly. So the better model of the world that you have, the better you're able to, and this is particularly relevant in social science, but yeah, better predict a better model of the world you have the better you will be at predicting what will happen in the next day month aurora something i've been thinking about a lot is kind of the concept of cultural diversity and human autonomy when it comes to what kind of lives we want to live what kind of communities we want to live in i've felt very lucky in my life to be able to kind of choose that and navigate towards social circles and communities that reflect my values and how i want to live.

50:05One of your colleagues wrote a book maybe five years ago, I'm blanking on his name or the title, but it was essentially imagining different possible futures with the idea that by adding more crisp detail to that imagining, that could help us develop preferences around what kind of world we want to steer to. And that was my utopia or protopia, whatever we want to call it, that really struck me was the one where it's like, okay, let's say we get through the singularity, we're doing all right, we have this AI that is in fact aligned with our best interests, and it creates these series of walled gardens for us where humans are able to decide what are the principles we want to live by.

50:43There's a community for the traditionalists to live their lives. There's the leftist stream of post-gender, et cetera, that would get to live their lives. There's the ability to choose to move between bubbles. Maybe not every walled garden would accept you, you'd have to kind of meet their entry criteria, but anyone could exit. And so that's really my idea of that. And I think that's one of the failure modes of a lot of utopias, is that it's represented as kind of stuck in time and kind of totalizing in terms of this is the only world you can be in. So that's that. I don't really have a question there just to kind of share that that was dream was one that really resonated.

51:25I have no idea how we might align an AI to get that outcome but if you have any ideas please uh yeah i mean it's kind of governance question rather than a ai alignment question but uh yeah so philosopher robert nozick it's he calls it kind of meta utopia and then scott alexander has a long blog post which he and he calls it archipelago and i write a bit about a similar idea and what we are the future of yeah you have many different societies and people can just create all sorts of different societies you've got the marxist one you've got the Georgian one, you've got the ultra-libertarian one. And then as long as you've got just freedom of entry and exit, then the hope at least would be that, well, the best one kind of wins out.

52:08Or at least people get kind of what they want in terms of the societies.

52:15Yeah, whatever societies they kind of choose to enter into. I think there are still kind of major challenges where, in particular post-AGI will get unprecedented ability to engineer the preferences of our progeny whether that's humans or most easily kind of digital beings and there's some like quite hairy questions about how you handle that because okay you now got the like the Marxist utopia and all of the kids are kind of uh essentially like engineered or indoctrinated into like loving Marxism and you've got the hyper libertarian scenario society and there it's the same, like society is able to have much more fine-grain control and like the beliefs of the next generation.

53:02If you've got that, then you no longer have this kind of ability for people to move. So you'd have to have some general prohibitions against this kind of ideological steering too. So it's tough, yeah. Some kind of, you have to prove that your next generation are able to steal man the different groups you know the other the other gardens can you represent what that other garden actually is but not only steal man they have to be able to actually choose it right because it could be the you know the hyper libertarian well but it's like how do you prove that they're still you know because what you're trying to prove is that they don't have free will in order to choose it and so one proxy for that free will is that they're able to steal man although i guess again you could teach someone how to steal man like without believing i suppose but yeah um uh but yeah this in general this question that we've like we've never really had to deal with or like only in a very limited way of just being able to have quite tight kind of ideological control on the next generation um becomes very intense with ai um because you know already now we're just choosing like what character should claude or Gemini or ChatGPT have like how do we want them to behave this is like we're kind of like gods with respect to these like new you know new beings that we're creating the same I think will just be true with humans as well so once you've got better understanding of genetics of psychology and so on it will be technologically perfectly possible to have a child grow up to believe just exactly what you want them to believe and that's like pretty scary because you know we're worried about lock in of particular ideas ideologies well one reason we've had moral progress over time is that people die and then a new generation comes in and can have like new ideas and you know we could be the end of that and that's quite worrying thanks this is super interesting um you know as i was thinking about all of these utopias and other forms of topias um it struck me that this is really a question about elites who will control the AIs and what their preferences are.

55:18Because ultimately, we can have thoughts about how we should have shared rights over various resources, but we know how that goes. We had that situation with water. Nestle has water rights at very low cost that are public resources. So those resources are going to be owned by whatever economic powers deem it necessary for their businesses. So we are in the asymptote approaching what Marx talked about, which is the elimination of labor by capital. And so then the question is, once all of this, There are two paths beyond the singularity, it strikes me. One is that all material outputs that humans could produce go to zero cost because there's no labor involved, and labor is the source of cost.

56:18And in fact, the consumers won't have any money to pay for anything. So the costs will have to come down to match what the consumers have, which is only what they can exchange with each other as humans. Unless we make the very bad mistake of giving AIs rights. Because this is a sidebar conversation. But I think that we can know with an absolute certainty that digital computers can never be conscious. I think there's a rigorous argument from physics that would support that. But just take that for granted, that they cannot feel anything, that hopefully we never give them rights. And we've got a few questions.

57:01One is, should we give them the ability to feel? Because there's certainly some physics that's not in our digital computers that could grant that capability. And so we may have that question in the future. Should we do that? but then there I guess going back to the question about elites is what do they do so once they have this enormous productive capacity there's still a a scarce resource say we have relatively infinite energy and materials from space we still have land as a scarce resource like good land that people actually want to live on. And then there's a question once elites have police power, which is robotic and under control, of what the utility of the rest of humanity is.

57:55And I think that's where the question comes down to and where the conversation goes in the near term as we approach that. Because then it's in their hands. And there are two ways that elites could respond to that. One is they say, well, they don't matter. And when people don't matter, they really won't matter in every way. And the other is that there could be some social approbation that comes within elite circles against that viewpoint. History isn't good on that. Because it's always been the force of the public to control the behavior of the elites. So we potentially go into a very, very dark space because of that.

58:41So I wonder whether you've reflected on what the dynamics of sort of elite, like basically the masters of AI, what will be their preferences in this situation where most, almost all humans have no material output. Now they may have some other output though, which is whatever they produce with their consciousness which machines can't produce so maybe that forms a secondary economy that's actually based on consciousness where all value comes from um i those are a whole bunch of loose threads of speculation and i just wonder whether you've kind of considered some of those different routes uh yeah i mean it's um it's a great bundle of threats i mean with respect to the preferences of elites um yeah i think i have this kind of relatively kind of cynical or realist like understanding of how things would go once um yeah once most people in society are like no longer um of use uh that well if the elites can kind of thereby uh gather more power for themselves then they will choose to do so.

1:00:01I don't think that necessarily means people, most people live like bad lives by today's standards, getting like this tiny slice of a very large pie because the economy is just, you know, technology and the economy has advanced things so much that, you know, the economy is a trillion times bigger. There's some 0.01 % of that that goes to, gets redistributed to most people. but I do think it means that most people get wholly disenfranchised and then the question of beyond that, what do the elites want I mean, I think in the medium to long term what ends up driving things is kind of ideology rather than self-interested preferences because self-interested preferences satiate like

1:00:59when you're thinking about an economy that is as large as it could get post-AGI, spreading out into space and so on. Okay, well, now I've got my billion, billion robot servants. What other things? Maybe I can live forever now. What are the other ways in which I can benefit myself? It's like, well, maybe you're hitting that cap where certain ideological preferences are unbounded. unbounded so you know we talked about the classical utilitarian who just wants hedonic bliss states as much as possible just more and more and they just they don't stop being valuable you just want more of them or similarly like environmental preservation or perhaps certain sorts of art or like other sorts of ideological um preferences maybe you just want communism everywhere uh that's a way in which actually i expect kind of post-agi elite preferences to look quite different than now because people aren't out for themselves in this now self-interested sense as much but instead of like there's more like a competition of how what people want the future to look like so yeah I think that's potentially quite different in terms of like making sure we don't get there and things are a bit more widely distributed I mean if people were just more woken up to the idea that we are quite plausibly facing an intelligence explosion then there could be a lot more work from the people who have some amount of power now um to ensure that that power kind of is retained where um you know that could be like uh voting in politicians that will perhaps like distribute like uh capital like more equally to to everybody so that you get rich uh while the you know at the same time that the whole economy kind of increases or just like you know really you know makes the constitution and the u.s kind of government system like even stronger against kind of concentration of power like various things that could be happening now to ensure that power kind of uh stays more distributed but you know people in fact think that this is wacky sci-fi stuff so um probably not going to happen yeah blake i liked your analogy about the hill climbing and then like let's not hill climb but let's like we can't see we're in a forest we do want to climb that hill so that we can see around and and i'm kind of thinking about that as like us as a world or as a society how can we what kind of hill are we climbing and in particular i want to ask how can we measure how well we can see like can we measure when we're getting to the top of the hill and like being able to see better I would love it if we had some metrics or some kind of visibility for, I mean, we can have something that's like how well is your system that you're building doing at forecasting and maybe people are trusting it, something like that.

1:03:53I would love something that's also not like everyone in the universalist sense, but like how well can we see our choices? How much of a little hill are we at the top of? Can we see the space around us? Yeah, I mean, it's pretty rough. Like some of these things like how well are we doing at kind of scientific understanding or ability to forecast like empirical questions. Yeah, you can imagine having metrics on that front. But then, you know, what's the metric by which we're gauging kind of how well is society doing at ethical reflection and how many options it has open? is like it's quite hard but you could maybe focus on like process so um things like protection of the speech or things like um uh you know generally how well educated and reflective people are there's perhaps like certain things you can look at in terms of uh the certain tests you can use to see kind of how biased people are and so on but i think it's going to end up being quite a lot more kind of qualitative and mushy than we might like.

1:05:07I want to also click on, like when you're at the top of the hill, there's something about not just, you're not just getting data, but it offers affordances. Like how much do we see things that we could actually pursue? How much do we see affordances that we have that we could move forwards to? Does that make sense? What do you mean by affordances exactly? Yeah, like when you're on the top of the hill, it's not just, oh, I have data about the landscape. it's like oh I could go this way like there's a path and I can cross the river and then I see like good land over there and that's what that's what I'm imagining like if we could if we can elevate our ability not just to not just to be smarter in some sense but to see good options and yeah if we could measure that so that we can be moving in a positive direction on that yeah that's what I'm yeah what you've made me think is like appreciate is that I often think about this in terms of like inputs to the reflective process so you know how can we ensure that we have more time to as a society kind of think and reflect before making kind of highly irreversible decisions or more different kind of moral perspectives or just kind of a greater fraction of our resources going towards you know thinking about the good and so on but yeah you've made me appreciate that I tend to think about those inputs perhaps because I'm like pessimistic on the ability to you know say oh well this is as a metric kind of like this is what a society that's like very reflective and thoughtful would look like but you've at least prompted me to try and think about if I can could make that happen.

1:06:48I don't know so we talked about meaning um and that's I think a very important thing but what about purpose you know I think that drives a lot of human behavior and at its finest you know or at its best it's you know maybe the purpose of a man going and working in a coal mine he hates it doesn't like the work but he's providing for his family and he gets a lot of purpose and satisfaction out of that and I'm just wondering in your world are people talking about that what will be our purpose and how will we find that and what will be satisfying yeah and the hierarchy of needs is going to be very different i'm wondering if there's a new maslov coming along to come up with a better hierarchy of needs and what we will need if we have abundance and we have resources um at our you know disposal but i you know and i grew up with a father who was very much about purpose and if you don't have that in your life you're really missing a huge part of what it is to be human at least he thought so especially for men he was kind of a sexist but you know i i do think about that all the time like what really will drive humanity in terms of purpose uh yeah so i do think this is a harder question than merely okay you know perhaps the easier challenge is like okay we've automated all work what do people do and it's like okay well they're devoted to child-rearing or art or something.

1:08:10But the issue is, yeah, about when robots are better than you, all of that too. So, okay, you want to dedicate your life to raising your children as well as possible, but the robot would do it better. Take any aim you might have with respect to child-reading, the robot would do it better. Or if you want to learn a new language or something, and it's like, well, I could just plug something into my brain and immediately learn it. and so nick bostam in his book deep utopia calls this um the problem of post instrumentality i think i got it right um and it is um yeah it is much more challenging um i think that uh my best guess though is that humans will come up with all sorts of things to care about and create as a sense of purpose that will replace um and be surrogates for purpose people find through work and maybe it'll seem like weird and alien and kind of cringe to us now because it's all like how many followers do they have on social media or something um but uh you know nonetheless despite being like weird or alien to us like we'll end up as a source of meaning for them um this is fascinating so speaking about sci-fi concepts i'm writing a sci-fi book right now called Emperor in the Aether.

1:09:30And what are some concepts that you feel like need to be, like you talked about Huxley and the Island being one example. Like what would you write about if you were writing a sci-fi novel right now? Like what are some topics that you feel like need some air time? Yeah. I mean, I've always wanted to have like a sci-fi movie, which is, you know, you just see Earth. it's like an alien invasion movie and that's how it was going to build and then you see earth and it's like people are like you know doing the thing and so on and then it's like you know 20 minutes into the movie the peasant's like oh my god we've like seen uh you know aliens on the horizon it's like okay we've got to like everyone starts to mobilize and it's like 15 minutes of the movie is that and then there's just this flash of light and then there's just the construction earth is being disassembled by the aliens to create like a dyson dyson swarm around the sun just to really get across like how large gaps in technological capability could be between different civilizations maybe not the most interesting book so basically not like what Independence Day where like the fighter jet is roughly the same speed as the little things yeah exact opposite of Independence Day great movie I think there's really fun stuff like around in particular around relativity so like Like, you know, you might think that the furthest reaches of the universe are, like, utterly inaccessible to us as kind of humans.

1:11:06They're 8 billion light years away. But I hop in a spaceship, and if I just go accelerating at 1g, so I can just, like, hang out. It's just, like, gravitational field just like this. I do that for 30 years or so, 35 years. and then I decelerate after that. Now that's a challenge, a big technological challenge. How do you make that deceleration happen? And how you make the acceleration happen. But suppose you do that. So then I decelerate again at 1G. So again, I've got just normal human gravity. It's taking me 70 years. Where do I end up? The furthest edge of the galaxy. Wait, really? Sorry, furthest edge of the universe.

1:11:44So I'm 8 billion light years away. Very, two very unintuitive things combined here. Oh, because of relativity, right? Well, because going at 1g acceleration for 35 years, you go very, very, very close to the speed of light. And then that means, yeah, time is going, yeah, you're going very fast through the years compared to kind of an observer on Earth. and so that's kind of like you know this amazing fact actually that like maybe kind of all through the observed like affectable universe you could have actually biological humans from us today um that's like this piece like yeah kind of piece liking thing um so i'd be down for people to like yeah like um uh play with that a lot um if yeah you should definitely talk to richard who has um a series he's also a series of short stories on um uh science fiction as well one called succession that i like really love um i think there's like lots of fun to play with the reason i thought about it in particular is like thinking through the moment at which like um very advanced kind of civilizations might meet aliens as well where going deep on kind of solutions to the Fermi paradox and in particular Robin Hanson has this kind of grabby aliens model that I think is quite compelling actually where it's like oh yeah we'll meet the aliens in about 10 billion years.

1:13:21It's like he's got an estimate of the time and so it's then like yeah like really trying to work through like how would those dynamics go like would there be trade that occurs between the civilization of aliens or is it just like complete stalemate I think would be pretty interesting too I'd also like to add I'd love to see more sci-fi writers think deeply about ways that humans in the future will coordinate around like seeming like coming back to the divvying up the sun question around these there are these resources that may create conflict and they need to be managed how would advanced uh civilizations or people with super intelligence in the world figure out how to coordinate in ways um not where it's just like pure kumbaya everyone is a hive mind thinks the same there's still space for competition um but yeah i'd love to see more more thinking on that yeah maybe one other whole category is what you could call kind of model science fiction where imagine some future and i mean i think the most spicy version of it would be some like uh future to kind of generally utopian society but then like morally it's just quite different than us um in some way that we would find like very distasteful um and perhaps in the kind of depiction no one even comments on it because it's like oh of course you do xyz like kill your firstborn son type thing yeah yeah something like that it's like oh yeah just that's just the kind of norm and people you know they joke about it or something um that i think would be like yeah very provocative and challenging and i have one more question this is amazing um so you spoke of this this blissful state that we could potentially like chemically put ourselves into um like where does like the animalistic like idea of nirvana like our consciousness like like what is your stance on like how that's accessible or how ai has to do with basically like reaching that like i don't know like what are your thoughts on kind of the on how to get to hedonium yeah exactly uh yeah i mean probably it's got to be via you know probably there's just quite hard limits on like how good how good a time you can have while still being a biological human.

1:15:54I mean, maybe via genetic engineering or something, you could make people who are much more blissful. But probably the kind of peak mental states that are achievable by any being would be some digital being instead. And what would the design of that be like? Well, you'd need to have some theory of consciousness and so on. It's kind of far beyond us. but yeah I think unfortunately we're just maybe merely destined for bliss not kind of alter the bliss well like hedonism is like very dopaminergic like I want an instant response you can think of it like single games right and then like iterative game theory like long term goals I'll take a hit now to have this like long term benevolent outcome I think of that as encapsulating when I think of eudaimonia, right?

1:16:52It's like longer-term satisfaction. And so you were talking, there were a couple things. Like when I think of AI being valuable, you talked about humans no longer having economic value. And like I got into thinking on like Calhoun's Universe 25. Do you know that? It's like mouse utopia. They gave the mice everything and then they ultimately collapsed. and people think of it as an overpopulation problem, but it's actually a death of meaning problem as I look at it. And then I think we're kind of in our universe 2025, right? And so I was like pairing this all together when you were talking and then your question live was like perfect.

1:17:31I was thinking meaning and time, like human time ascribing value to that as paired to outcomes, that would be the ideal society that I would see. And it would be like, can we drive things back local? Can we drive things back to the community? And can we drive things back to then, therefore, human time, again, imbued with meaning? And let the AI take over the post-industrial Taylorist BS of day-to-day mundane crap that we don't want to do. Yeah, so I mean, I think... I'll just do a little defense of hedonium first. of just, I mean, in particular, kind of, I think consumer cultures, like actually really anti the, even like a kind of good, uh, what the kind of hedonists would regard as a good outcome.

1:18:26Like if you even just look at surveys, kind of people buying stuff just makes very little difference to their wellbeing. Um, whereas, uh, social relationships, um, in particular, like friendships, romantic relationships, like, you know, it's actually stuff that you can't buy, um that has i mean as well as health which you um uh often can buy um you know have the biggest impacts on people's um well-being uh but then sorry then your question can you just say your question again um yeah it was like in this post ai world oh yeah where do people have meaning yeah i mean i'm kind of more optimistic on this like i think people at the moment get a lot of meaning from their work but uh i think that's i mean to some extent just a bit of like post-hoc rationalization often from you know you're forced to do work so that's a way of making it kind of more pleasant but it's also just not the only thing you need to get meaning from so in a post agi world you could dedicate your entire life to like learning all of these amazing new facts about the world that has been discovered from ai like you know perhaps you're working through the proofs yourself um that's like could occupy many lifetimes alone you create amazing art um you know like i do a bit of music and that is still meaningful for me even though there are literally hundreds of millions of people who are better than music than me um if there were then like also trillions of robots that are better than music than me it would still be a meaningful thing and then similarly relationships as well where um you know if you think okay would it be nice to spend like an hour more with my family and an hour less working it's like yeah that would seem pretty good what about two hours more and i i just think like once we actually start to occupy if we do get to the kind of good post-agi world our lives will probably just be like overfilling with meaning but and the idea that like you would get meaning kind of from selling your labor for money might be quite a strange thing.

1:20:32And maybe one cause for optimism here is just people who don't need to work in society today, so either the very rich or the kind of retired and so on. Like if you're retired, it can be like a big shock, but a lot of people just, actually the retirement becomes the best years of their life and there's just all sorts of projects you can engage in. Just one clarification. We had, just as I see it, we had meaning pre-industrialism and we had community at a larger, like as a larger part of our life. So what you said on human relationships being huge. Oh yeah. I mean, I'm expecting a lot of like dancing around the fire post-AGI.

1:21:16Like you might well see this kind of like regression back so we do when we go to burning man yeah exactly eagle um so i find your job fun to of being a philosopher because i think part of the job is to like go to the edges of current accepted thought of like current kind of yeah just like what's what what people think is right and uh figure out like where we can maybe venture forward and um um i wonder like how do you see this process go like moral progress in general like do you think it happens because uh if i if i look at engineering for example and how it relates to physics then like you do physics progress and then slowly it like goes through to engineering at some point you build the tools to then like try it out and it's like physics leads to real world action very slowly right um yeah how is it with moral progress uh is it the type of thing that technically lies even earlier and it takes sometimes even longer for people to realize oh that dude from like 50 years ago is right or is it now the case that i don't know you're like striving to find sometimes the right concept the right meme like a singer's drowning child is like oh that's a great meme that can spread around and then people will change their behavior should you be basically a meme peddler or like should you be trying to influence it slowly and hope that the future is better uh okay it's a great question so i mean i guess one question is just how does moral progress happen in general and then secondly perhaps you know what should we be doing um yeah i mean unfortunately you know i don't think most moral progress happens from moral philosophers sitting around and concocting arguments and then like people get convinced by the arguments and change their behavior um no need to laugh at that come on it's not so absurd um i mean i actually think the british abolitionists were a little more like this so you had the quakers in north america and you know they're motivated by religious arguments but they're like arguing um and it's not a coincidence that comes from the quakers where god can speak through any person.

1:23:31So you've got this much more liberal epistemology. And they are being moved by arguments of just like, okay, slave owning is inconsistent with this picture of being a good person that God has given us. Nonetheless, it does seem like a lot of moral progress has just been via the disenfranchised being able to kind of gather together and advocate for themselves. So, you know, women, yeah, people of color, workers, and so on. Not in the case of better regulations around meat, for example, though, right? No, although there we've had remarkably little progress, I would think. So, I mean, it's true that, like, attitudes to animal welfare have improved a lot.

1:24:25and in some places like the EU especially there's better animal protection but it's like so slow it's like so and spotty across across different countries as well I almost think of that as like proving my point where the fact that animals aren't in fact farms aren't able to kind of like take to the street and advocate for themselves means it's very kind of slow kind of model progress. I mean, in my case, I'm like, I'm just trying to speak to the like weird alien niche of society of people like yourself who are much more into like arguments and really thinking things through and then trying to take action on that basis.

1:25:15But that's going to like cap out. There's only so many people who really live their lives that way. and you are also trying the memeability of concepts to a degree right like long termism is like it's a good name it's very clear what it tries to do via topia is like a good name to which degree do you hope that I don't know that there can maybe be a thing that if you just phrase it right it will be helping for more people to realize it I think that is yeah I mean effective altruism as well like I do spend quite a bit of time to think like Like, what's an important concept to spread? Yeah, in philosophy, I think of this as, like, conceptual engineering.

1:26:00So, like, you know, people with real jobs build bridges, and, you know, philosophers build concepts. And it's true, if you build a good concept, then it can really spread. And I do kind of hope this concept of viatopia could be kind of unifying, where it's like everyone can be like, oh, yeah, well, here's this like list of you know things that we want to aim for that just that actually seems like really pretty good even though people disagree like really quite a lot on what kind of ultimate end state they're going for. Yeah that would be I think my sort of final question for tonight is like what call to action do you want to give to the people here and to the viewers at home like who are you know if someone is wanting to contribute to Viatopia what but they're not a tenured professor what what can what can people like us do yeah i mean when it comes to action to try and make the world better i always just say there's just two really big things that people can do and everything else is well most things are secondary so one is just use of your money and the second is use of your time so use of your money is just like donating to highly effective organizations that are working on kind of any of these issues we've talked about so I set up an organization called giving what we can encourages people to give at least 10 percent of the income but also has like loads of kind of recommended non-profits so even if you don't buy all of the AI singularitarian stuff lots of non-profits that are just doing you know great very current very very current things that are very important as well so that's the kind of money side and the underlying thought is just means just take whoever is best at the world at solving this problem you can support them with money it's this like superpower then the second part is just your time so in particular with choice of career where another organization I co-founded called 80 ,000 hours after the number of hours you'll typically work in the course of your life uh advises people so on what in which careers can they have the highest impact and 80 000 hours in particular is now very focused on preparing for a world with very advanced ai and so if it is the case that you're thinking about yeah maybe we can you know maybe i'd be open to kind of a career switch or um at least within your career kind of changing exactly what you're doing then there's like millions of words of writing on this on the 80 ,000 hours website and they do kind of one-on-one advice too and beyond that then just like I do think memes are important so like writing talking about ideas um reading and learning as much as you as you possibly can all of these things are big and important too yeah I mean one thing that you mentioned earlier that sort of really stuck with me is this idea of think of your best day ever so maybe that's kind of like our homework we can all go away with like go and think about what your best day ever was or maybe your top three and think about the delta because it's not just about like what is the best day it's like what is the delta possible between your best day and like your average day you can think about your worst day as well if you want but you know um and i don't know i feel like there's something in that just to go and really focus on that and then think about okay what could be the proportional jump from that, you know, from that average to the best to be on there and then go about writing stories or spreading memes that can help make that happen.

1:29:41Thank you very much, Will. Thanks so much, Liv. This has been a blast. Thanks. Thank you, everybody. Thank you.