EP 59: Robin Hanson (Economist) -So AI Is Not Going to Kill Us All?

30:42And I think something like the Metaculous AI forecast has changed from 2060 to 2032 or something on that order. What's your perspective on, are we in that little burst? It's somewhere between linear and maybe a near-term exponential growth curve in your mind? Almost all growths are exponential. They just might look linear if you're looking at a small chunk of them. The question here is just how potent are these technologies? And again, I would look at their actual displacement on jobs. So, you know, the most promising evidence so far is you get rough rumors that many programmers are getting maybe a 20 % or so boost in their productivity.

31:25So that would probably increase the demand in wages of programmers, actually. So they're not going to lose their jobs, but, you know, there's enough programmers in the world that that's a noticeable thing. it's not remotely near enough to like cause the economy to start doubling every month or faster but it would might be noticeable and you know over over centuries we have you know waves of faster and slightly faster or slightly slower growth and this might indicate we're entering a wave of slightly faster growth but again if you're thinking no will this suddenly basically take all the jobs you need to be expecting a much bigger chase than that so that's what you should be looking for?

32:05You should be looking for, well, what are they figuring out how to do with these? And, you know, how much money can they make? They'll have to be basically making money having the large language models do stuff. And there's a lot of people experimenting right now, a lot of hope and optimism and excitement, and as well it should be, but that's what you should be looking for. And, you know, you've got the long track record of people getting excited and hopeful and not happening. But hey, this is your life. Enjoy. Do you think it's possible that a single AI can be much smarter than the smartest human?

32:42Sure. But I mean, first of all, smart is a little ambiguous in terms of its breadth. I mean, basically, we just use smart for being able to do mental tasks, but there's a lot of different kinds of mental tasks. But you might have something that's just broadly capable across or wine range things. I'm more focused on how fast that happens and on how many of them there are of similar levels at any one time in the world. So a scenario that many people have, you know, scared children with and other people is the scenario that there's some, you know, research lab like OpenAI or something. And, you know, it makes this one system and this one system somehow turns on and is able to improve itself.

33:25And then it improves itself extremely rapidly. And then after a month or something in hiding, nobody notices it's super powerful and takes over the world. That seems to be a pretty unlikely scenario. But, you know, you can't prove it won't happen. And many people are scared enough of this whole space that they think if there's things that you can't prove won't happen, then we should act as if they will. It's interesting. It brings up the more general AI doomerism. And obviously, you had debates about this subject 15 years ago with Eliezer about all of this stuff. What do you think of the current rhetoric around AI doomerism?

34:10I think we're stuck in a situation where the actual concrete systems in front of us are relatively benign and exciting and useful, although even that's not safe enough for some people. But there's this vast space of possibilities, and we're all trying to talk about what seems likely in this vast space of possibilities. and it's hard to really say much about the space because we just don't have concrete examples of the stuff we're talking about. That is, if you're talking about future AIs and their capabilities based on entirely different architectures and structures and approaches, it's just hard to say that much.

34:54So then it depends on what kinds of analysis frameworks you bring to bear to the situation. So I tend to bring the analysis framework of, let's look at the past, let's look at the world we live in now and project that forward. So I think of a large economy, many firms, many nations competing in steady progress, law keeping us in place. And then that's what I use to project forward in the future. And I would say, you know, it's pretty unlikely that the future will be so radically different from the past that past trends wouldn't be useful. Many other people go, the past is completely irrelevant.

35:33Everything you know about humans is completely irrelevant. that they have some sort of abstract theoretical framework optimizers, they might call it or something. And I say in my abstract theoretical framework, this looks likely. And, you know, it's hard to engage. If they aren't willing to sort of ground their fears on the actual rate at which stuff has happened in the past and we've seen in biology and human history, then you just have fear. and you know in some sense um humans are very naturally able to be afraid of an alien other uh you know we call that othering and in our world today there's a lot of disapproval for othering of people of different genders or sexual preferences or ethnicities or religions and even ages.

36:28And we all just really shut that down and say, no, you shouldn't be afraid of those others. You should invite them all in and try to understand them and embrace them and have a world of peace and prosperity with all the others. Others are good, according to standard current dogma, right? And then you get the AI other and people go, oh, that doesn't. These rules don't count about that. You can go crazy being terrified of this other because this is a really other other. I mean, I actually think if we got messages from real aliens, the whole othering thing would come in. Yeah, we shouldn't be so suspicious of them.

37:05And we should be friendly and inviting because aliens are probably just like us deep down inside. And they're anxious and concerned themselves. And we could find a way to love each other. With aliens, that's what we'd say. But with machines, we go, no, this is an alien, harsh, you got to be suspicious of this thing. You can't trust that for a minute. Don't turn your back. This could just kill you and wouldn't care. And we are just willing to project all of our othering fears onto this AI. And there's not much room to stop you because, again, it's about this future system we hardly know anything about.

37:42We can hardly say anything about it other than, yeah, eventually there'll be such things. And so there's not much way to block those fears. If you're afraid of some scenario, how am I supposed to tell you it won't happen? I can tell you, look, stuff like this has almost never happened in history. You might say, yeah, but history is irrelevant. Look, I've got this concept optimizer and I'm afraid. Yeah, it's an interesting framework. Ultimately, the world can only end once, right? And so survivorship bias would say that these concerns are unfounded. And there is an element of, and I know I've heard you talk about the 9-11 analogy of when people go so deep into an example of something, they're much better versed than the people that are at a level above in their communication.

38:39Can you talk through that analogy? Because I think it's an interesting one as people hear experts talking about this. I'm a bit of a contrarian, I have to admit, and I hang around people who are a bit of contrarians and we're tempted by various contrarian takes. And so I tend to see more about contrarian takes than most people would. And what happens with a contrarian take is there's a group of people who form around a contrarian take and then they talk to each other a lot. and they develop some sort of internal language and internal history and internal set of concepts and founding documents and things you just got to know about that thing.

39:16And that's true for, say, 9-11 truthers who think 9-11 was an inside job or it's true about QAnon people who think the government's run by pedophiles or it's true by UFO fans who think the government's hiding everything they know about aliens, right? And, I mean, these communities are often fascinating They have a thesis that you can't dismiss immediately. And they're usually trying to get their point of view engaged and listened to by outsiders. And sometimes they get outsiders to respond to their arguments. But the usual form of it is, you know, they're just really into their world. They've read up everything about UFOs.

40:02They've gone to all the talks. They had all the arguments, all the language, all the terminology. And then usually on the outside, a critic is somebody who hasn't been immersed in that world. They're not reading all these things. They're not going to all the conferences and all the websites and everything. They have some other basis of expertise and focus. But they, for a moment at least, turn their attention to this issue. And they read some stuff there. And then they give their opinion. And they engage to some degree. but predictably this engagement will not be based on all those details that those insiders are really obsessed about that's just not going to happen because these outsiders won't take the time to learn all of that but the outsiders will be just looking usually at the things they do know and sort of just broad common sense and broad widely shared beliefs and considerations and they would usually just bring those to bear on this particular topic and they might say you know For example, 9-11-2, how long could they really have kept that conspiracy going?

41:03Or say, the moon landing was faked sort of thing, right? And they're going to just bring up some other, if this were true, then wouldn't this other thing be true? And bring those general considerations of skepticism. And the people who are really obsessed with this topic, when they choose to engage with a critic, they're almost always going to engage with the most prestigious high-profile person they can find. Because by doing so, they raise the prominence of their topic and they make it seem legitimate, right? And there's a selection effect. The most prestigious people willing to speak up on it are going to be the ones who didn't really put that much time into it, right?

41:39They're focused on something else, right? And so they're often going to give pretty sloppy responses, pretty hand-wavy, even dismissive responses. And, you know, the fans love that, right? They can say, look how poorly argued this is. Look how poorly thought through this is. They haven't, you know, considered these things. and they don't know about our terminology. They don't know about any of these sorts of issues that we've talked about, and that emboldens them to say, look, we're right and they're wrong because, look, the best critics we can find are just sloppy and not thinking very carefully.

42:12But, of course, they aren't the best critics they could find. These are the most high-profile prestigious people they could find who said anything about it. There usually are some less prestigious people who put more time into it, but those people aren't getting much attention, And so these, you know, the fans don't see why they should even engage with them. What do they get out of it? So that's how these things usually play out. So this doesn't say whether they're right or wrong. This is just a general description of how this sort of conversation usually plays out. And it means that if you're in one of these situations, you've got to take this into account.

42:44Like if you're one of the fans and you should probably try to find, if you really want to do it seriously, you'd probably find the critics who are less prestigious and less high profile who are giving more thought to your discussion and engage them a bit. Might mean that as an outsider, maybe you shouldn't just spit off your mouth about things. You have to give them a little bit more thought. And it just means that as observers, we have to, knowing that this is the situation, realize that, look, I mean, generally we have to go with our priors. Like most contrarian things are wrong. I got to go.

43:16I got to admit that, right? But if you want to get past your prior and think maybe this is one of the contrary and think that's right, there's not really that much substitute for like getting into it a bit, but you should ask what kinds of expertise are relevant for this topic. And that's one of the hardest things for people to, to figure out when they're judging the world. So I've learned a lot of different areas over my years. And this is a thing that takes a long time to figure out who would be the right kind of people to know about each thing. Because inherently, if you don't believe it, you're not going to investigate all the rabbit hole-like scenarios of why this tower looked like this on this video in the 9-11 analogy, which is partially why I wanted to have this conversation with you is because I think you you are a very respectful critic of the AI critics or the AI, you know, the people that are concerned about this.

44:17And so I guess I want to paint an illustrative example of where something could go wrong in this AI dumerism as framed to me by one of my friends who falls more in this bucket. And I would love for you to push back on where you think it breaks down or say, yeah, it's possible, but I don't think it's likely. So the example that I was given was you start with a chatbot and we can call it GPT-4 or GPT-29 or whatever it is. And in the course of answering a prompt, like how do I start a successful online business with minimal effort? It comes up with a clever response, which is basically a shell script that the user can paste into their terminal and run.

45:02The script then starts doing all kinds of operations that the human didn't intend, but that makes sense to the AI, right? It starts copying itself to a bunch of different web servers, putting a bunch of websites, offering online services that can make money, maybe automatically trading in the stock market or something. And it might just go all the way to noticing all these opportunities to copy itself into a whole bunch of different devices that aren't secure. Because I think we probably both agree The Internet's not the most secure place in the world. Yes, your banks and governments are more secure than social media websites or whatever it is, but it's still not the most secure.

45:39And so then suddenly, because of this, hours later, humanity realizes what's going on. And we sort of have the coordination problem where there's a botnet connected to billions of devices. And it escalated really quickly into this like super intelligence machine. What of that do you think is implausible, would break down in your mind in that scenario? So there's two key implausible elements of that scenario. One is the idea is that there's a bunch of untapped resources that you could just grab easy because nobody's doing much with them and they're just all for the grabbing. Just servers or whatever it is?

46:20Right. And the other is that there's this big opportunity for starting to do self-improvement that is also untapped. So generally, when you have an image of a sudden burst, what you have is image of some untapped resources that are just sitting and waiting to be grabbed. So if you think about Columbus and the New World, you think about the Europeans, all this advanced technology, and there's this whole world that they could conquer except they don't know about it. And suddenly they learn about it, then suddenly they can just burst out and grab all this resources, which from their point of view is relatively easy to grab, but they just didn't know about it.

46:56So that's a scenario of some burst through of possibilities. That's what you're trying to construct when you're trying to construct one of these sudden burst scenarios. and so in this story there's one of the bursts is the idea there's all this computing hardware out there that's just very useful and hardly defended and you could just grab it now you know the problem is that in our world if that's true there's all these people who would want it to be grabbing it and using it and you know this new thing would have to be fighting against them that is there's a world of people are trying to protect their stuff in a world of people trying to grab things And we might say the world today is that world.

47:32And so there's no easy move to make a big win unless you have a big advantage somehow. So that's when we think about military technology. We look, there's all these nations next to each other. They build up their militaries based on the perceived threats of their neighbors and their budgets. And we think given if they're all correct about the rough capabilities of the various kinds of military hardware and soldiers and everything, then they'll be in an equilibrium where nobody gets taken over because they've all roughly guessed correctly the intentions and capabilities of their neighbors. But if suddenly one nation were to have a new technology or some new, this even social ability to coordinate that nobody expected, then they might surprise their neighbors.

48:13And then suddenly there'd be an invasion and a war. And then who knows what would happen because you've got a shock, a change. so if you're going to try to make a dramatic change scenario in a world where things are competing you have to come up with a surprise so i mean you might say gpt or large angles are surprised at the moment so there was a world of search engines and google was at the top and then hey a surprise new tech and now google's dominance is now in question because they didn't anticipate this shock to the world of competition and then they've got to deal with it right so So in the scenario you had, there were these two shocks that are postulated.

48:50One is that there's all this computer hardware out there and that if you can just be a little bit smarter than we are now, suddenly you can just grab it all. Even though, of course, they were all anticipating that and trying to protect themselves. And, you know, there's only so much you can do with all of it. But that's one part. And the other part was just this idea that this machine, you know, suddenly decides, what would happen if I improve myself? As if they weren't doing that now. But in fact, my understanding is AI researchers are constantly trying, gee, what would happen if we had this thing try to improve itself?

49:20And the reason why that's not a major feature in most systems is it just doesn't work that well. When they try it, it just doesn't succeed. So you have to be postulating that somehow this new system found a way to actually make that work much better than the previous attempts have. This thing tried to improve itself and somehow the way it tried to improve itself, that was a big winner. It sounds like one of the big distinctions philosophically is just that currently today we're in this reinforcement learning state of humans are involved in a lot of the process and improvement. And the exponential growth could potentially come from this self-learning that the machines are actually able to make them.

50:02But that's what I'm telling you that AI researchers have given the AI program self-learning over and over again. They have empowered their systems to self-learn. That's been a constant trick for decades. The problem is it just doesn't work very well yet. When they give the machines the ability to self-learn, their actual quality of doing so is bad. And that's why most of the improvements in systems haven't come from self-learn. They've come from humans, you know, actually intentionally restructuring them and giving them more hardware and better algorithms and things like that. That's been how things have improved so far.

50:36But at some point, maybe they would get better at self-learning. My understanding, and maybe I'm wrong on this, I thought GPT-4 did capture elements of self-learning to get there. And maybe the distinction is it was a small component versus a large one. But yeah, it's an interesting, I mean, that would be the exponential. If that were true, I guess, would you give more credence to if self-learning were actually in self-improvement actually were possible? Would you give more credence to these? It is possible. It actually happens. It just doesn't do a very good job. That's the thing, right? You might say, like, can humans fly?

51:17Well, sort of. You can jump for a few feet, right? And we can fly by jumping for a few feet over and over again. And so, in that sense, humans can fly, right? So don't talk about like giving humans the ability to fly. Talk about giving humans a better ability to fly. Yeah, we might slap our arms and fly around if we had a much better ability to fly. But that's not about just thinking, hey, let's think of the very idea of giving humans the ability to fly. That's about, can you find a way to make them actually good at it? So the same for the large angles, models, GPT things. They all actually do have ways to improve themselves.

51:54It just doesn't do very much. They're not good at it. What would you do differently with the risks associated with AI currently? Or is our current course in speed appropriate? In my mind, the most obvious risk that is the most robust across a wide range of scenarios is the risk that most humans will lose their jobs. And we can actually do something about that. And so we should. And unfortunately, getting so distracted with all the other risks, I think, has detracted us from fixing the one thing we are most confident would be a problem and that we do know how to fix. so now if AIs are very slowly displacing humans like over centuries then just the normal flow of people switching jobs and retiring and taking jobs that would just naturally adapt to AIs taking more and more of the jobs eventually humans would just have wealth that they invested in the AI companies and that's how the world would go the risk is it might happen much quicker than that.

53:03So what if in a five-year period or a 10-year period, we went from most adults working and most adults mainly only the main wealth they have is their ability to work to a time when less than 10 % of adults work because computers are doing most of the jobs. And then what are they supposed to do as a replacement for their wages? That's a scenario that people are concerned about for many centuries for automation. It's actually not a crazy scenario. That is, it might go slow, but it could go fast. That's in a 10-year period. That's not a crazy time period over which this change might happen. And people are, in fact, at risk.

53:44That is, most people's wealth is pretty low compared to the present value of their future wages. That is, most of the wealth of people is I can earn wages and that's why I can feed myself or my parents earn wages or whatever. So, but we can fix this by insurance. And in fact, much simpler insurance than what insurance companies usually offer would be fine. So if we're talking about the insurance, you know, say for you to personally lose your job, now the insurance company has to, you know, estimate your risks. They have to know about you and your inclination to get a sick or your inclination to get lazy, your social connections and the kind of jobs you have and the kind of industry you're in, right?

54:27And so it's called underwriting to estimate your risk. And so for giving you personally risk about losing your job in general, then there's a lot of underwriting costs. And so you might not think that was worth the bother. Most people don't, in fact, insure against their risk of losing jobs, although we have some unemployment insurance that's provided by the government. But in this case, we have a scenario where basically most everybody loses their job all at the same time. So we don't really need to know about you in particular to give you the insurance. So I propose we could just take most any asset like a stock index fund or a real estate index fund, and we can just split it into two parts.

55:09That is, we can have the index fund if a certain event's happened and the index fund if a certain event doesn't happen. And the event can just be the adult labor force participation rate falls from say above 35 % to below 10 % within a 10-year period. And then if that happens, that's the signature robots took most people's jobs. And then you get this asset instead of someone who bought the other version. So that would be how you buy insurance. You buy these stock index funds to realize things conditional on this event happening. And because this event is not that likely and it's what happened a long time off in the future, this will be relatively cheap.

55:49You buy basically insurance that if this happens, then you'll get some assets. And the people who buy the other side will buy them because they're cheaper because they're willing to take this risk. So as long as there are people who are more vulnerable to this risk and people less vulnerable to this risk, there's room for gains from trade where one side buys once, the other side buys the other. and now we're more insured against the risk. Now, many people think, oh, but governments would deal with this risk. Surely if robots took all the jobs, the government would handle the problem. And so we shouldn't have to worry about ourselves.

56:23Well, see, the problem is most governments, their plan is to tax the rich people in their district, in their area, and give the money to the poor people in the area, right? But this AI transition need not be equally spread around the world. So like the first, the industrial revolution was concentrated in England and then Northern Europe and the rest of the world didn't participate in it for a long time. So if your plan is when the robot revolution happens is to tax the rich people in your city or your state in order to help the poor people who lost their jobs, there may not be rich people in your state to tax much because the robot economy might have happened in a whole nother continent.

57:05So that's why I think whatever insurance we set up needs to be global. And so this sort of asset I described would actually be a completely reasonable thing. Now, maybe your local government should buy a bunch of these assets so that when this happens, they can use that to help people out. But somebody should be buying these assets. What do you make of the letter last week that a bunch of different people signed, Elon Musk among them, that we should pause for six months the development and let society kind of absorb the implications or at least think through the changes that are coming just from GPT-4 and some of the recent progress?

57:46I mean, many people have commented, so I'm mostly going to repeat what other people said. First of all, it would just be really hard to enforce. Second, it would take away U.S. leave in this and give other nations time to catch up. But I think more importantly, there isn't just much that would happen in those six months other than having proved the willingness to restrict the technology, which I think is the real purpose. The real purpose would be sometime in that six months, we make a proposal for more regulation and more limitations. And then we lock it down more permanently. I think that's the actual plan.

58:25That is, there's not. I don't think we have any, after six months, we have much of a better idea how to regulate. Basically, I think people are thinking, look, we need to regulate this thing. That takes some time. It'll take time to put our political coalitions together to get people behind some bills, to write out the language. We need time to make that happen. They want to regulate AI strongly, but they're not ready to now. And so they just want this pause period to set up that regulation so that by the end of six months, they've got regulation passed that would extend it for indefinitely. They just want strong regulation indefinitely.

59:02um and so do you want this industry heavily regulated then is the question and you know i'm an economist and we have many ways that we think about regulation and many kinds of regulation that makes more versus less sense to us but almost always regulation is a response to actual concrete envisionable problems things that actually went wrong typically in the past or that we believe went wrong. Here you've got just this vague, vast space of possibilities, and you want to empower someone to just block off the whole space because it looks scary. That just doesn't look like the sort of thing we've been good at in the past in terms of regulation.

59:44And honestly, there's enormous promise in this space. In many other areas of regulation, what we've often done is just blocked off whole areas of change and not allowed them. And then there's just no innovation in those areas. if you want to like control and regulate actual systems that would exist, you have to allow them to exist for a while and then, you know, try out things and experiment and, and, and evolve different variations. So the pushback or the distinction, I think that people that are concerned or want this regulation in some form versus other examples is the risk of a fast takeoff, which I know we've sort of touched on here as well.

1:00:28And then the overall catastrophic risk of what could happen, right? And so the airline example is a great one, right? Of like, you needed to build an airline and understand how would we even know what TSA needed to be prior to airlines existing and people using them and all that. That said, that actually involved tons of manufacturing. And the risk was, yes, some people were going to die, but it wasn't going to be society, right? It was going to be individuals around that. And so I guess that's the distinction that people that are concerned about this, I think tend to draw on is the fast takeoff and then the overall catastrophic risk.

1:01:13Right. So that's why we have to ask, what are these scenarios by which everybody dies as a result of strange AI things. So in a world where there are many AIs controlled by many organizations, if one organization loses control of one AI, that's just not the end of the world. Each of these organizations is going to be constrained by competing with and being monitored by all the others. So in order to be afraid of the one AI kills us all thing, you have to be afraid of two scenarios, I guess. One is there's this one AI that just jumps ahead of all the rest. It does this FOOM thing where it's small and tiny before, and then it has this vast speed of acceleration that other AI systems do not have at the same time.

1:01:55And that during that change, its values change radically compared to what they were at the beginning of the process. That's just a crazy, unlikely scenario, given everything we know about history, I'd say. The other scenario people would be worried about is saying that intrinsically, AIs don't have a coordination problem. They can all just get along with each other. So they could, all the AIs in the world would just talk and say, hey, let's take over and get rid of the humans and decide to do that and then just do it because they don't have problems coordinating. I mean, it seems to me the problem of coordination is very robust and AIs don't escape from it.

1:02:29So I'm not very worried about that scenario either. Got it. One question, I guess, as we're sort of looking out in the future, I would love, you were into prediction markets and as kind of the forefather in some of these concepts, steps. You made a bet, I think, with a colleague about GPT revenue in 2025, that it would be over under a billion dollars or so. Do I have that right? I think that's roughly right. And I think I'll probably lose that bet. Yeah. Okay. Okay. That was what I was going to ask you is where it looks like the markets are now trading against you on it. But are there any predictions that you do think by 2028 that AI will not be able to cross?

1:03:15Like, is there a goalpost that we can set to say, like, you just don't believe we're going to get there from an AI development standpoint? So the prediction I feel most comfortable making is about the effect on human employment, net human employment. That is, you know, sometime in the last few months, I said 10 years from now, we won't have seen any net effect of AI on human employment. That is, we might have seen some jobs go up and others down, but overall no net effect. And that is what we've seen in the past. That is, when we look at analyses of past automation, we've seen that roughly no net effect on overall employment.

1:03:58So I would say when we do those studies after 10 years from now, we also won't see that effect yet. That's most likely. Now, that doesn't mean I'm absolutely sure that won't happen, but I think that's, you know, pretty overwhelmingly the likely chance. Now, that's consistent with a lot of actual productivity boost. I might think that we would see a boost in productivity over the next 10 years bigger than we would have seen without large language models. and that means the world economy would grow faster. That's a possibility, although I still might think a 1 % effect or a few percent effects would be roughly what you might possibly expect, total over 10 years.

1:04:42But that's sort of different from, I think, the whole world economy doubling for a few months or something. We're talking a very different size regime there. Similar to the trend of like predictions and things that I guess doing retrospective on all this stuff. I mean, the debates you had with Eliezer 15 years ago, I think have reached more common awareness in the last year, just because of he's become a prominent outspoken critic or has, I guess, increased his prominence as a critic of AI and all of that. And so I revisited it for the first time in the last few months. Are there anything in there that you feel differently about today or any opinions that have changed over the course of having seen this play out for 15 years?

1:05:30It seems like most of the people who are worried about AI lately repudiate the Foom scenario that he was focused on. So, but they say we should still be worried about AI, but then they're talking about other sorts of concerns. And then there's a wide range of these concerns. And so the hard part is to try to get clear which concerns they have in mind and engage those. But Eliezer is still focused on his particular FOOM concern. I mean, certainly we have seen more the kind of AI systems that have come out and the kind of features they have. And that gives us a more particular image of what could happen.

1:06:11um we have seen um like so i mean there's this key interesting question that we discussed back then of the relative importance of architecture in ai systems and computer systems um and i certainly have to admit that architecture can have a substantial importance i'm it's not clear to me that it's had more importance lately than it ever has but it has some substantial importance. But I mean, architecture has more importance in a sense on simpler, smaller systems of less scope. And so there's still this fundamental question as you try to make systems of very wide scope, how simple would their architectures be?

1:06:54And therefore how much of a change would any one architectural innovation have on them? Certainly the move toward more focus on machine learning has made architectures more important than they were, I think. I should admit that. But they still don't seem overwhelmingly important. And I mean, I certainly think that in the last few years, we have not seen like very strong leads in the sense of some companies being way ahead of others. We've seen relatively modest leads. And much of the debate about FOOM was about how far ahead might the leading firm be compared to its second and third place competitors.

1:07:39And I think we've also seen relatively steady improvement in systems. That is, there are architectural changes and there's improvement within an architecture. And I'd say, I mean, and it's worthy of noting that the most recent systems aren't very agentic in the sense of having an agent who has plans and would then maybe make a plan to take over the world. So the worry is that about could they turn these systems into more agentic systems? But at the moment, they do just answer questions and they don't make plans to take over the world. And Foom, for people's benefit, is the fast takeoff scenario.

1:08:19That was the, I guess, the one we discussed. Yeah, the terminology put around it. One interesting thing I heard you discuss before, and I think there's interesting analogies to artificial intelligence, But is the example of a factory in electricity in the productivity gains, the amount of time it took for there to be the gains within from from electricity entering factories? Can you use that, tell that story and use that as an analog for potentially how AI works for us? So the key question is, you know, what causes growth and change and what happens when you have a new fundamental innovation like, say, electricity?

1:09:04So mostly, as I said, most change is lots of little things. But every once in a while, you have a pretty big thing, like, say, electricity. Now, the question is, how does a big change like electricity cause the economy to grow faster? And so there's a literature of what are called general purpose technologies, of which electricity was one, and maybe the computer is another, and maybe certain kinds of AI would be yet another. And the story is what we see is that initially a general purpose technology doesn't actually affect the economy that much because what you need to do is restructure other things to match it.

1:09:41So the first factories were structured around, say, water power or other mechanical power. And they basically had to transfer that power throughout the plant with big belts. so they'd have one big generator that like turn like started something turning and then they'd have all these belts connected to their other things in the factory to get spread the power around now with electricity it became possible instead that just have a lots of little motors in the factory and have wires to go to each one and then each one will have the motor to do the particular thing it needed to do and that allowed you just reorganize the factory a lot and so So most of the gains from electricity came from reorganizing the factory around this new possibility.

1:10:24And that took a while. So it took several decades after electricity for the effects of electricity to really be seen in the economy. And we saw a similar thing about the personal computer. That is, initially, companies bought personal computers and they stuck them on people's desks. And then they tried to use them in the same sort of business processes they had before. They had only modest, even very difficult to observe productivity games from that. But then as businesses started to reorganize their practices in order to take advantage of computers, then they saw bigger gains from having computers.

1:11:00It wasn't just having a computer and throwing it on the same structure. So that's you can see how that would be a general fact about any new technology. if it's a fundamental new technology that allows the systems to be reorganized in radical ways that can unleash large gains but it's going to take some time to reorganize things because you you made machines before they take a while before they'll like wear out and break and you don't want to like replace something that's still working and there's just a lot of this infrastructure in society that will take some time to replace um and that's true for most kinds of innovation.

1:11:38That is, you know, innovation just happens slower. If you have a new demo and you're excited by the demo, it's just going to take a while for that new thing to spread. You can think of like say electric cars at the moment. We've seen electric cars and we see the cost in batteries coming down, but it takes some time to change the distribution of gas stations to electric power stations to make larger generating plants that will generate the electricity for the cars, etc. It takes a while to change things. So as you think about the next 10 years of economic progress, do you think it will be dominated by AI?

1:12:13Or is there some other factor that you think is going to be a big impact to economic uplift? Prior to seeing these large language models a few years ago at the beginning of the pandemic, my guess for what would be the biggest change in the next few decades was remote work. And that's when we would find ways to reorganize our workplaces to accommodate remote work, which we haven't that much done. So at the moment, we have mostly retrenchment. People are slowly going back to the office because our work styles have not really changed fundamentally to accommodate remote work. But over the next 30 years, there's a potential for us to do more of that reorganizing.

1:12:55So many services that you have in your local area are limited by the scale at which they can work. So let's think of a plumber. Your local plumber has to be ready to do a pretty wide range of plumbing jobs, and they really can't specialize in doing one kind of plumbing job or another because you need to have a plumber nearby who can come to you and bring their toolkit and do whatever you've got. But if we had good enough avatars such that someone could work on your plumbing task at your place remotely, and we don't have those quite yet, but they seem feasible within a decade or two, then we now open the space for much more specialization.

1:13:36So if you look at an automobile factory, the magic of the factory, you've got maybe hundreds or thousands of employees each doing a very different specialized task, and then they can each be really good at their very specialized task. And so now if there was a plumbing firm behind your plumbing avatar that shows up in your apartment to fix your pipes and behind this avatar is this huge company with hundreds of people who specialize in different tasks, then what will happen is that at each point in this plumbing task, the specialist in that task will swap in to control the avatar. They'll do the one little thing they're really good at and they'll swap out for somebody else And then your plumbing job will get done by a sequence of people who are just expert in that particular thing, just like in an automobile factory.

1:14:25And that degree of specialization could allow enormous productivity. Removing space as a constraint for specialization and expertise. Right. So as you may know, bigger cities allow more specialization and have more productivity just because they're bigger, right? and small towns don't have as many different specialists. And so they often can't get the productivity gains that big city can. But remote work could allow small towns to have the specialization gains of a whole continent. And that's a pretty big deal. So that would be my competing hypothesis. If AI doesn't turn out to be as promising as many people hope, then that's still a thing that could happen over the years.

1:15:07And it doesn't necessarily need to be either or, right? Of course, AI would help make it easier to control these habitats. And of course, one of the things is if you have broken the plumbing task into 20 different tasks, you could automate one or two of those 20 tasks and the customer would never know the difference. As long as there's a person doing the task and a robot can watch them, then they could learn from watching the human to do the task to do it themselves. And that would be a way in which automation would be supported and encouraged by remote work. Now, one thing that you wrote and thought extensively about is brain emulation.

1:15:44And I guess, can you talk a little bit about what that is for people that maybe aren't aware? And do you think this is something that we're going to see in our lifetime? I don't know about in your lifetime, but let's explain that. So at the moment, the AI you're hearing about is humans sitting down and creating an architecture for a computer system and then running that architecture on data. and then on cases where they plan out how that system's organized. And what they're trying to do is make a system that's capable of the human brain. But when we look at the actual human brain, it's very complicated and has all these different parts working in different ways.

1:16:19We don't fully understand it, but we can see that it actually is a general artificial intelligence. That is, our human brains are capable of doing the wide range of things that we actually do. So another approach to artificial intelligence, i.e. machines that are as part of humans is simply to make a model of an individual human brain. That is, you take an actual brain and you scan and you see where all the cells are, of what type, connected to what. And then you have good enough computer models about how each cell takes input signals, changes internal states, and send output signals. And if you have good enough models for all the different kinds of cells in the brain, and you have a good enough scan of a brain to tell you which kind of cells are where, then you can make a computer model the whole thing.

1:17:03And that computer model, the whole thing, if it's good enough, would have roughly the same input-output behavior. That is, you could hook it up with artificial eyes, ears, hands, and mouth, and then you could tell it to do something, and it might do something just like the original person would do in that same situation. So this is a path that should eventually be possible to lead to human-level artificial intelligence. And so then the question is how fast will this happen? And, you know, compared to especially the other approaches to automating tasks. So this brain emulation thing may take another century, but then again, making artificial, the other kinds of AIs as good as humans across a wide range of tasks, that may also take century.

1:17:47So we can't really tell which one would go faster. But the thing we can tell is that the brain emulation kinds of artificial intelligence are much more understandable, relatable, and perhaps, you know, ones we approve of more. So the brain emulation, if we did a brain emulation of you, then when we turned it on, it would remember having been you just before we scanned you. And then you'll have to convince it that it's now a brain emulation. It will just assume it's still you. And we'll have all the same feelings and inclinations and memories and skills. And so from its point of view, it is you.

1:18:27Now you can decide it isn't you, but it thinks it's you. It'll tell you it's you. It'll be very insistent on that. It'll be short. Eventually you'll have to convince that it is in fact a brain emulation, but we understand how these things work, that in a sense that we can put these things together in a society and we can predict a lot about how that society goes because we know a lot about how humans go in our society today. So they will have some important differences that allow them to do some things we can't do, but to a first approximation, they're basically us. And so if you're worried about an alien future that isn't like us and the things that we value disappearing, well, you should be less worried about that here because, hey, they are pretty close to you.

1:19:13You know, on the one hand, you could think of a particular revelation as being you personally, and you might hope yourself to come back as a brain emulation. But setting that aside, we could just think our descendants, if they were brain emulations, would just be a lot like us. And so whatever we were valuing in ourselves and afraid might be lost by other kinds of AI, well, this is less lost here. Now, my book, The Age of M-Works, Love and Life and Robots Rule the Earth, goes through in great detail what this world would be like. And it is substantially different than your world and life now. So I think this is one of the things you should just most expect.

1:19:50When we look in the past at all the huge changes that have happened coming up to here, that just sets the expectation that you've got to expect the future will also be pretty different in some big and foreign ways. You just can't expect the world you live in to stay pretty much like this world indefinitely. That's just not very reasonable given how much change has happened. So you've got to suck up and expect some substantial change. Now, you're quite an academic polymath studying like a bunch of different things and having spent time in artificial intelligence and economics and social sciences and physics and across the board.

1:20:23How has having this cross-disciplined approach or ability to study all these different areas, how has that impacted your capability in going deep to individual areas? We haven't touched on aliens as well, and maybe we'll have to do another episode just on that altogether. Running out of time now. Yeah, exactly. But how is that more general purpose kind of large learning allowed you to go deep into these specific areas so well? Well, polymaths have advantages and disadvantages. One advantage is that if a topic just requires that you understand and agree many things, then the polymath is up to that.

1:21:02So I would say my Age of M book really requires that you just understand a lot of different topics. My task there is to predict a whole future civilization based on one technology change that I'm trying to predict all different aspects of life there. So you just need to know a lot of things to be able to do that task. And the other thing polymaths are good at is maybe finding neglected topics between the other usual ones. that is. And one way to find an interesting topic is to combine previous ones. So whenever you come to a new topic, you have this whole toolkit of all the things you've ever seen that you're trying to say, which of all those things I've ever seen is most like this or most able to give me some insight into this.

1:21:45And Polymos just have a larger toolkit there. So if you're trying to take on a fresh new topic, then, and you've just seen a lot of different previous topics, you're just going to be better at finding the matches between this new thing and all the other things you've ever seen. And so we're better in that sense at starting in on a new thing and on doing things that intrinsically require many things. Now, if you think about it, these are kind of just obvious implications of polymath. That's just generally what they're going to be good for. They're going to be less good at sort of knowing more about a very particular thing, but maybe better at critiquing it from a wider range of views.

1:22:24The last one I want to ask you about, and I guess we'll pause on, you have another wonderful book, The Elephant in the Brain, which I think we could spend a ton of time talking about, as well as prediction markets and grabby aliens and all of that stuff. But the last one I wanted to, before I let you go, cryonics. And I think you're at least interested in the field, or are you actually? customer this is my so what explain what this is for for people so um when medical science gives up on me or i give up on it if i'm allowed then basically the plan is to freeze my head in liquid nitrogen after putting chemicals in it that will allow it not to crack when it freezes so that means basically all the same chemicals are there, maybe slightly changed during the freezing process with the roughly the same relative orientations.

1:23:24And that means when there are these different types of brain cells in my brain with different, you know, different degrees of chemicals that represent their internal state, it should be possible in the future to scan that frozen brain and read out those cell types and those cell states and the connections between the cells, and therefore possible to make a brain emulation. That's, of course, not possible now, but we're talking a while in the future. And while you are frozen in liquid nitrogen, basically nothing happens. There's just no changes. So once you're frozen, there are some changes that happen during the freezing process.

1:24:01But my judgment is that's relatively minor compared to how redundant the information is encoded in my brain in terms of these chemical densities and the relative orientation of these cell parts. And therefore, it would be relatively straightforward to make a brain emulation of me if only the organization I trust to keep me frozen actually does that. So I think my major risk is that it fails to do that, unfortunately. But I give that at least a few percent chance of success. And so the alternative is worms eat me or this succeeds. And so I think the cost benefit calculation is actually a pretty good deal.

1:24:34Because if this works, I could live for centuries afterwards, perhaps, as a brain emulation. and the worst case is it fails just like what you're going to do instead. What's the company that you're working with? How do you go about finding? I've just used a company called Alcor, but I'm not strongly recommending it. In fact, another company called Crass Institute has been growing faster and because it has cheaper prices and maybe I'd be tempted to switch. But I weren't too lazy to bother to make a change. But the fundamental problem with this area is that this topic has gotten free international publicity ever since the 1960s, because that's when this thing started.

1:25:15And in that entire time, they've only ever frozen like 300 people. There's only like 3000 people as customers. So that's a really small market. So these firms have to charge a lot of money, and they're mostly sitting around waiting for the next patient. And it's just their fixed costs are really high. And so this stuff would just be much cheaper and much more reliable and therefore gives me a much chance of lasting if a lot more people would buy into it. And so it seems to me a terrible shame. In essence, most people who die don't need to die in the sense that they could be a brain emulation later.

1:25:48And the reason they're dying is because they don't want to buy this product and because not enough of them are buying this product to push down the price and push up the reliability. So we're all stuck in a terrible situation where we wouldn't have to die. Now, you know, maybe this future world of a brain emulation world where you might come back to isn't exactly your cup of tea. Maybe you'd rather have come back into some other heaven, but sorry, you know, this is at least not dying. And I don't think it would be such a terrible world. And in addition, of course, if it continues that only a small number of people do this, the small number of people who actually do come back will be minor celebrities.

1:26:27They will be actual people from the old world. And hey, I will have wrote the book on them. So I will be an especially in-demand celebrity. So I'm less worried about whether anybody will want to talk to me or keep me around. But that's the pitch. So the key thing you need to believe to believe this works, and I know many people are skeptical, is just the idea that you are your brain and your brain is a signal processing system and signal processing systems just generically have to isolate some degrees of freedom in which the signals are sent and processed from the other degrees of freedom of your system.

1:27:06So that's what your phone and your computer and your TV, et cetera, all do. They all have some degrees of freedom, i.e. certain wires in which signals pass along. And then they have to design it so that those signal degrees of freedom are insulated from other degrees of freedom so that random things happening don't mess up the signals and mess up the signal mappings. And so that's what your brain was designed to do. Each cell takes some signals in and sends the signals along certain pathways and then transforms the signals in certain ways and then sends the signals out. And the way it does that has to be such that it's isolating certain signal degrees of freedom from all the other ones in order to do this mapping.

1:27:49And in fact, when brain scientists have made models that seem to capture at least what some parts of the brain do, the models they use of each brain cell is actually really quite simple. So there's reason to believe that most of the, each cell's complexity doesn't really matter. Even the signal processing degrees of freedom are relatively simple from the point of view what the rest of the brain needs. But the key point is certainly that cells are enormously complicated. And if you had to, you know, model all of that complexity, then you'd be facing a very daunting task. But most likely, most of that complexity can just be thrown away because you don't, when you make a brain emulation, you don't need to model how the cell protects itself from viruses or, you know, makes energy or repairs its cell wall or all those other sorts of little things the cell needs to be doing, all you need to do is reproduce those signal pathways.

1:28:45Well, to the people that are watching this in 300 years, because of your celebrity, hopefully I've glommed on a little bit to your fame. And I'm remembered as someone asking you about this. Well, right. But if you would become a customer, then you could take advantage of that fame more. You know, so here's an opportunity for you. It's a good solicitation for all we need to do is just coordination across humanity. right? Something AI is often good for, I guess, but no. I don't know that AI is good for coordination. Yeah. Well, thank you, Robin. Thank you for coming on. This was amazing. And there's so much more we could have kept going on, but this has been a really fun conversation.

1:29:22Thanks for entertaining all these different questions. Thanks for talking.

1:29:30So that'll do it for the 59th episode of the Logan Bartlett Show. Thank you to Robin Hanson for coming on. Thank you to Rashad and Justin for their efforts with this episode. And thank you everyone for listening in a really fun episode to discuss what the future looks like with the world of artificial intelligence. We look forward to seeing you back here next week on the 60th episode of Logan Bartlett show. Have a good weekend, everyone.

1:30:06you