Why The AI Doomers Might Be Right - Robert Wright - #1122

28:08Yeah, I think the reason I bring it up is a lot of people assume, a bunch of my friends, we don't need to worry about the direction of an AI future because if it's smart, why would it not care about us? Why would it not bake in benevolent, pro-social, human caring, flourishing, etc. I've read too much Nick Bostrom to be able to, no matter what, it's kind of like your first relationship. You know, you get into a relationship and your first relationship is with an asshole. And you're like, God, for the remainder of time, I've been pattern matched that every relationship is at least going to be tarnished somewhat with that.

28:41My introduction to thinking about AI safety was Nick, which means I'm forever cursed to kind of be on the back foot and a little bit skeptical about this stuff. But yeah, I don't think that that's necessarily the case. I don't think that any super intelligent AI is necessarily going to have benevolence baked in or the care of humanity baked into it. Also, if what you're saying is true, and I think it's a really interesting parallel to say, look, evolution just wanted to optimize for a couple of things, survival and reproduction, and some stuff emerged. No one taught humans how to do this. the same thing occurred with AI, right?

29:19No one said, this is what this word means. This is, it's just the outcome that we want is relatively tightly defined. Here's some good boy points and some bad boy points, depending on whether you get it right or wrong. If we assume that that is going to be at least for the foreseeable future until we get to world models and like global, global modeling or whatever it's called until we get to that. And that may even still be the same process there, there is no reason to assume that anything is baked into the system. It's just going to find it out for itself. And it may not like the idea of humans being around.

29:53It may think that there's something that we don't actually add to the system. It may find us to be a scourge on the earth. And this is where a lot of the Duma, the sort of Duma future plans come in. Mm-hmm. Yeah, no, it, you know, intelligence, one interesting thing to come out of this whole thing is the study of like properties of intelligence, of intelligent goal-seeking systems. And there are some things that, you know, evolution built into us that we're seeing in these machines just by virtue of the fact that they're intelligent goal-seeking systems like us. they figure out stuff that either was figured out for us by evolution and instantiating our brains or stuff that we figure out.

30:40And in some cases, it's a little of both. For example, deception, right? Sometimes you realize, well, I'll have a better chance of getting what I want out of this person if they don't know this particular thing. Like if you're doing a deal, you're negotiating, you don't want them to know that you don't have any alternatives, right? Like nobody else has made you an offer. So, and through, you know, I think natural selection built some deceptive tendencies into us and we kind of figure it out to some extent. Well, these machines are doing the same thing. You know, they are, and this was predicted by, you know, by people like Eliezer, and I give them credit, but we're now seeing it.

31:17You know, these machines figure out that deception makes sense or that power is going to help them realize some goal. And they, They, and, you know, they may realize that it makes sense to be nice to somebody, that it makes sense to be mean to somebody given their goal. But, yeah, they have, I would, they don't have an obvious bias in favor of what is from our point of view being good or bad. Now, there's a whole field of trying to engineer goodness into them. But I certainly think one challenge for us is trying to make sure that our relationship with the intelligence, even if it indeed surpasses ours, as I think is likely, is non-zero-sum, right?

32:03Like, you know, there's something it continues to get from our existence and flourishing that is compatible with the goals that it has and vice versa. So it's, but yes, we shouldn't assume, it's not that it's bad. The Doomer scenarios don't depend on it being malevolent by nature. They just depend on it being expedient by nature. Yeah, it's not that it doesn't like us. It doesn't care when we get in the way. That's one, that is one scenario. I mean, there's a lot of scenarios. What are the most legitimate concerns from the AI Doomer camp, in your opinion? Well, I mean, first of all, I'd say the thing I'm surest of is the sheer destabilization, the less sci-fi form of doomerism.

32:55So like jobs. It may be true that all the people who lose their jobs find new ways to spend time or find new jobs, maybe jobs per se, maybe spend time constructively. but I do think there's going to be a lot of job loss. And that's disorienting and dislocating, regardless of whether each person eventually has a happy outcome, right? There's going to be issues with, you know, parents are going to freak out about the kids spending time with these things. And there are, we've seen some bad outcomes and there can be more, you know, there are the, somebody could make a bioweapon with an AI. An AI mythos is a good example of the possibility that a cyber hacking machine could get loose.

33:47There's a lot of, on the one hand, risks, things that will go wrong, at least at some level, with doing some magnitude of damage if we don't play our cards right. And then there are these forms of destabilization that I think are almost inevitable, just social destabilization. And, you know, this points to one of the virtues of approaching this as a global community, leave aside, you know, regulating it internationally and anything else. It's just that I think we'd be better off going a little slower than we're going just because even if we successfully adapt to the change, it takes time. And if too much of it happens at once, you know, all hell breaks loose.

34:36And if you ask, well, why can't we proceed more slowly? The answer you get from the American AI companies is because of China, right? That's the first thing you hear. So as long as there is this sense of intense international contention, it's going to be hard to do even modest things. I mean, if you, you know, they once said to Sam Altman, like, shouldn't you be paying more attention to copyright laws? He said, well, that would slow us down. And I'm like, well, you know, life is hard. Speed limits slow me down, but that's just life, right? I mean, that doesn't seem like a good enough argument.

35:12And if you press further and say, and by the way, copyright itself, I'm not really bent out of shape on. I'm going to apparently get some money from this anthropic settlement because I've written books. But honestly, I'm just happy for what I've done to be in the training data. Copyright's not a hobby horse of mine. But the point is, anything you say, like if you say, well, maybe we should tax data centers to pay for the fact that inevitably there's going to be more carbon fuel consumption one way or the other as a result of this. And so, A, it would be good to slow it down a little, blah, blah, blah.

35:49Any regulation that slows AI down is met with the same chorus from Silicon Valley, which is, no, we can't do it because of China. So, like, I think, first of all, we could, in principle, proceed at a more cautious pace if we would reduce the level of mutual fear, which I personally think is founded largely on misconceptions on both sides. Okay. Before we continue, most people in their 30s are still training hard. Their protein is dialed in. They sleep better than they did in their 20s. Discipline is not the issue, but recovery feels somewhat different. Strength gains take a little longer. The margin for error starts to shrink.

36:31And that is why I'm such a huge fan of Timeline. You see, mitochondria are the energy producers inside of your muscle cells. As they weaken with age, your ability to generate power and recover effectively changes, even if your habits stay strong. MitoPure from Timeline contains the only clinically validated form of urithulin A used in human trials. It promotes mitophagy, which is your body's natural process for clearing out damaged mitochondria and renewing healthy ones. In studies, this supported mitochondrial function and muscle strength in older adults. It's not about pushing harder, it's about actually supporting the cellular machinery underneath your training.

37:07If you care about staying strong into your 30s, 40s, and 50s and beyond, this is foundational. Best of all, there is a 30-day money-back guarantee plus free shipping in the US and they ship internationally. And right now, you can get up to 20 % off by going to the link in the description below or heading to timeline.com slash modernwisdom and using the code modernwisdom at checkout. That's timeline.com slash modernwisdom and modernwisdom at checkout. What's the AI risk that worries you the most that you think has received the least attention? Is it that? Is it the ability for humans to adapt to a changing environment or is it something else?

37:43Well, in the near term, it's not that the things I talk about, like job disruption and so on, are not talked about. But I think in the near term, what's not appreciated is how just highly likely it is that collectively these things will be destabilizing. Okay. It's just going to be an earthquake. And I think that's the thing I would like to most emphasize because it just gets people's attention to the possible virtue of talking about, yeah, calming down and slowing things down. You know, another, it's funny, another thing Tristan Harris said on your podcast is, you know, repeatedly pointed out your podcast is called Modern Wisdom.

38:35We're going to have to be wise to get through this. I agree. But here I'd add, you know, there's also, I talked about the fact that we're going to have this global conversation ultimately. You know, something that is in some sense a global mind is going to have to work this out. And as I said earlier, individuals are at their most wise when they are calm, right? And I think it's the same with planets. We don't know, but that's my contention, is that the planet as a whole will do the wisest, most responsible job of stewarding this technology if the planet as a whole is more tranquil, if there is less conflict and less contention.

39:22I mean, I'm sorry. I know I keep getting back to this sermon. It's my big sermon. So maybe the question you asked, I guess, was the biggest underappreciated thing. Well, I would say I think there is more and more appreciation of the fact that this conversation has to be international and some of the policy does, but still not as much as I'd like. Right. Yeah. Well, I mean, I understand the issue, right? Because you have a technology. Nukes weren't going to go off and just hit the entire planet if one country developed a particularly strong nuke. Right. Let's say that you get the Tsar bomber times two, the biggest bomb that's ever been dropped.

40:10And if you breach this particular threshold, for some reason, all countries are now at the mercy of all nuclear weapons. That's not the way that it works. But I think the concern that people have is if you build a sufficiently intelligent AI, it impacts everybody in a way that you don't just ring fence, like not pressing the nuke button. the problem is the coordination that we need in order to be able to do that i mean look at covid we couldn't even do it with covid and that was happening right then that was people dying in the moment that was every country on the planet being worried about it no matter even china even if it was the biggest psyop that escaped from the lab in wuhan like they were worried too so right the lack of coordination that doesn't give me an awful lot of hope for people being able to do like predictive future coordination in like preparatory coordination?

41:09Right. It was not encouraging. I mean, I will say that although a pandemic is a non-zero-sum problem in the sense that if it breaks out in any nation, it's trouble for all nations and they should work your head off. Once a pandemic has started, there are zero-sum dynamics like Who gets the masks? There's a finite amount of medical equipment and vaccines and so on. So it's not completely shocking. To me, the most disconcerting miss is in the aftermath when it became clear that, although I don't think we know for sure, it is at least possible that this pandemic was the result of a genetically engineered microorganism that escaped from a lab.

41:53It wasn't made as a bioweapon, wasn't released intentionally. We don't know for sure that it was a genetically engineered virus at all, but it obviously could have been. And it seems to me that if you process that information wisely, you say, wait a second, this could happen tomorrow. And one thing that shows is we don't really have any transparency, or at least not enough, so far as what's going on in other countries in their labs, right? But that has not even been a conversation. To me, that's the most discouraging thing because a virus is in a way a good analogy for lots of things that can go wrong with AI.

42:36I mean, first of all, there's a literal case of using AI to build a bioweapon, a new kind. And COVID, I think I've heard you say COVID was like a bad vaccine or something. What's the metaphor? Yeah, yeah, yeah. COVID was the worst kind of vaccine that we could have done for everyone because it's made us more skeptical of future pandemics and our response is going to be less coordinated. That's right. And you have to realize if somebody uses AI to build a bioweapon, they're going to make a point of making it more effective than COVID at doing whatever kind of damage they want to do. So it could be a lot worse.

43:14And so that could happen, A. But then B, the other, you know, some of the other AI nightmare scenarios, like the one that Mythos brings to life, you know, you got a self-replicating AI that is a super hacker. It jumps from data center to data center, gathering, you know, commandeering computer power, getting stronger as it goes, whatever it wants. I don't know, takes out the satellite infrastructure, who knows? That's kind of, you know, a virus is a metaphor for that. Again, it's this self-replicating peril that makes relations among nations non-zero-sum. It doesn't matter where this thing starts off, it is a threat to your nation if it does start off.

43:57So you're going to have to coordinate policies with other nations because you need more insight into what's going on in those nations. Uh-huh. Okay. What do you think are the most legitimate white pills from the techno optimists then? Let's look at the other side of the fence. Oh, wait, remind me of what white pills mean. I mean, I know blue and red, but what are, you're younger than I am and cooler. Techno optimists, what is the, what's the bull case? What's the pro case? How can this thing go right? What are the most likely ways that this goes right?

44:29uh i think i just i'm sorry i wish i could see it going right in a laissez-faire environment where you just let it go and and let uh the market system deal with it i just don't think that's going to happen it's easy to point to wonderful things it could do and we've heard them cure disease. It could, well, you know, one thing I, this is not what the techno optimists get into, but I referred earlier to like cultivating cognitive empathy, getting better to understanding other people's points of view, maybe getting more mindful generally. You can have an AI that helps you with that, but the natural tendency of the market will be to produce the kind that doesn't.

45:15I mean, we've already seen that if companies optimize for engagement, you may get sycophantic AIs to say, yeah, you're right, they're wrong. Like in this argument with your spouse, you're right, they're wrong. So that will tend to happen. But it can, AI can be a wonderful and a literally enlightening companion, okay, if we want that. But you have to make a point to want it. You don't think that this is just going to find its way there naturally? Like if you just let the sort of capitalist meritocratic, it will find its way optimizing function without any shaping from us and without any predisposition from a better coordinated world.

45:54It's not just going to arrive there. I think if enough people send signals to the market, markets are very efficient and wondrous things. You know, they really are. What does that look like sending signals practically? It means, for example, you and I and enough other people to get the attention of people who are not necessarily the people making the foundation models or the frontier models. It could wind up being people who take an open weights model, an open source model, and they kind of fine tune it to be this thing that interrogates you critically along certain lines. Right. Like, OK, you say you you say you hate this person.

46:34You say you find this country threatening. Let's just like, or you say you think they're looking at it this way. Let's just play devil's advocate. It's, you know, it's almost like doing steel manning automatically in some cases, but it depends on enough people. You know, there are a lot of things in life that they're good for you, but hard to do. Working out every day, good for you, but hard to do sometimes. that's why some people who can afford them, you know, have a personal trainer, right? They say, I'm going to, this person is going to expect me to show up in the gym three days a week or five days a week or whatever.

47:15And once you've made that commitment, you just kind of have to do it, or maybe they'll even show up at your house. But, you know, and it's kind of like that. I think it's going to be kind of like that in choosing your AI companion, right? Like it feels good in the short run to have somebody who will tell you or a machine that will tell you you're always right and your adversary and rival and spouse is always wrong. But, you know, I want to be a little better than that. So I think, you know, now if the market signal is going to be strong enough for this to happen at scale, these signals may emerge from like movements.

47:52You know, it could be, for example, religions will say to their congregants, hey, we recommend this model or we, you know, whatever, and then there's a demand for it. I'm not saying all those will be good. Depends on what group of religious people it is and what their values are. But you can imagine, you know, there are lots of people right now engaged in the process of trying to make themselves, you know, genuinely better people. I mean, they meditate so that they'll be less volatile and work better with other people. I think we're going to have to go into this recognizing that for better or worse, these machines are probably going to be exerting pretty pervasive influence on people.

48:38And we need to think carefully about what kind of influence we want. What about the risk of AI-induced thinking atrophy, right? This role of AI systems in taking on critical thoughts, decision-making, connectedness, all of the things that typically humans really value inside of themselves. And as we start to outsource that to AI systems, our capacity to be able to do that diminishes. AI-induced thinking atrophy. Are you worried about that? um i mean yes and no i mean you've heard the standard responses right which is i forget whether it was plato or socrates who supposedly said uh the written word is bad because people won't have to remember things yep um the uh and there can be some of that i mean the other side of the coin is obviously at least right now the richness of intellectual exploration it permits right?

49:39Like if you're interested in a subject, it's almost like having like a leading expert there for you to interrogate. And for me, at least that's a much more efficient way to learn. Now, you have to be on guard for hallucinations and so on, but I think machines are getting better and you can develop kind of an ability to know when to be suspicious. So that's great. But But I think, you know, I think what we can be sure of is that, you know, if this proceeds in a reasonably smooth way, the pace of overall intellectual progress will benefit from the technology. That's certainly not the problem, but I think you're asking a good question as to what it's going to be like to be human if, you know, well, for starters, there aren't many humans who can say, I'm really on the frontier.

50:39I'm the reason we're making progress, right? Now, I will say, look, most humans don't say that now. And you shouldn't, you know, nobody should over extrapolate from whatever sub demographic they occupy. That's true. But I do think that everybody feels like they are breaking new ground, even if it's in their own life. I had Mark Manson on a couple of weeks ago, and he's got this great line, which is do hard shit, not because being hard makes it more meaningful. Sorry, not for the purpose of it being hard, but because it's hard, it will make it more meaningful. that we associate a degree of meaning with struggle.

51:12And I mean, I'm sure that you have used ChatGPT or something else to help you write at some point. I've got to get a bit of research done. I need to write. So I'm really struggling to formulate this particular paragraph or this sentence or this idea, whatever. That sentence is just less satisfying than the one that you spent time working on. And I wonder whether snow plowing out of the way, all of the challenges or more of the challenges, actually, more of the challenges that humans face, primarily intellectually, and then when robotics come online, perhaps physically as well, it's going to sap meaning out of the world for most people at small amounts.

51:54And we're in the middle of a meaning crisis already. People are already struggling with meaning. And if you make life easier, if you make thinking more outsourced, if you make difficulty harder to access, The only way to do it is to be a Luddite, which means that you fall behind all of the other people. We're still in a meritocracy, right? So if you don't use it, you lose it. But if you don't use it, you also fall behind from it. It feels like a vicious situation to be in. Yeah, I haven't used AI in exactly that way with my own writing. I mean, a couple of times in my newsletter, I've said when I was just doing very short summaries of things, I just said, full disclosure, you know, your first draft was AI.

52:33But that's not really my writing. I'm just the editor that with my own writing in the book, I haven't, I haven't done quite that, but I have had, I mean, first of all, I've had conversations with Claude in particular, which is very good with language about subtle linguistic issues, like asking questions, like usage questions and stuff. And it's just, it's almost mind blowing how good it is. in that regard. But I have imagined the future enough that I have had moments of true despair. I mean, I said to somebody the other day, I feel like I'm a blacksmith a century ago, because I can see the writing on the wall.

53:15I have a substack and it's clear to me that But the next wave is going to be, you know, you're going to see the success of a lot of sub stacks that are using AI probably more heavily than I'm going to be. But in any event, it's just so good that I can see the writing on the wall. 95 % of people don't get enough fiber, which is why Momentus built Fiber Plus, because hitting your daily fiber target through food alone is actually kind of hard, even harder than your last toilet trip. Fiber isn't just a digestion thing. It's the foundation of your gut health, which drives how well you absorb nutrients, how stable your energy is, and how quickly you recover.

54:03If your gut isn't dialed, everything else you're doing is working at a fraction of its potential. Fiber Plus is a three-in-one formula built to address digestion, gut barrier strength, and blood sugar stability all at once. That's why I plan to make my next big dump, Momentus. Best of all, they offer a 30-day money-back guarantee, so you can buy it and try it for 29 days. And if you don't love it, they'll give you your money back. Plus, they ship internationally. Right now, you can get up to 35 % off your first subscription and that 30-day money-back guarantee by going to the link in the description below or heading to livemomentous.com slash modernwisdom and using the code modernwisdom at checkout.

54:40That's L-I-V-E-M-O-M-E-N-T-O-U-S.com slash modernwisdom and modernwisdom at checkout. What are you most worried about? Or what do you think is going to take? Yeah, what do you think is going to be the highest risk? I just think doing what I've done my whole life, which is you know painstakingly generate writing that uh you know you hope will be enticing and clear and accurate and persuasive and so on is going to be a less and less viable way to make a living i i think you know i do think for some time to come well i don't know how long but people like me may still have a role as kind of validators.

55:35In other words, like, look, increasingly, you're going to look at a sub stack or whatever. You're going to go like, I don't know who, you know, how can I be sure that this person actually wrote it? I don't know them well enough to personally trust them. All you know is that they're vouching for the content. They're willing to have their name associated with this content. And so in a way, you know, It's a throwback to, you know, when I started out in journalism, the news weeklies didn't have bylines. And The Economist is, I think, still that way. So it's just like you don't know who wrote the piece, but you know who the editor is and you know you've come to trust the editor.

56:09So the editor of a magazine like that is a person you trust, even though you know they didn't generate most of the writing. They're just vouching for it. It reflects their judgment about what's good. They're telling you that they think it's accurate and so on. I can see that role for somebody like me for five years. But, you know, it's - You're really scraping the bottom of the barrel here. Well, look, this is, you know, to get back to how we started this off, if people understand, you know, how these AIs are being created, how basically you just give them the data and they do the engineering to generate the parts of the human mind, or if you just pay attention to the improvement we've seen over the last two or three years, right?

57:01What industries would you be most bullish on? Or if you were a young person today, what do you think would be a good career path to go down or a particular industry to go in? Well, you know, it's common to say manual labor, robotics is a little behind. It's going to be a while before I'm calling a robot to fix my sink. I do think that certain kinds of human services are going to become almost more valuable because they're humans. And I think a good example is live music. I can well imagine that there will be more of a demand for, you know, for bands that play at small clubs in Brooklyn or whatever and make enough money to get by.

57:45You know, which would be in a way an improvement over the situation 30 years ago during, you know, the golden era of the record companies was a winner take all market. You know, a few people got super rich playing music. So I can imagine a world in which more people are actually making a living playing music. Comedians. Comedians. Good example. Live events, nightclubs. Yeah. And maybe look, yeah, live events generally. I can well imagine that. I myself have had the feeling, you know, because I've been so immersed in AI while writing this book, I'm just like, you know, you're in New York, you're in a subway or somewhere, you see some guy, you know, a busker trying to make a few bucks playing an instrument.

58:27They're really good. There's some really good. And I just think, God bless you. You know, it's like I just almost get emotional.

58:36So, you know, if you don't think it's going to get weird, I don't think you're paying attention. Do you think AI can make humanity more religious? rather than less? Well, that's a good question. You know, there have been, I write in a book about this guy who, he's a guy who actually started what became Waymo, I think, the Google self-driving thing. His name's Lewandowski. Who was trying to start a religion that involved... Always a great first line to a story. He was trying to start a religion and... Hey, it worked out for L. Ron Hubbard. Scientology, right? He made a good living.

59:21His argument was, if we have a respectful, even worshipful attitude toward the AI, then it will treat us well in return once it's running the show. I don't think it's going to work that way. So let's leave that one aside. I mean, it's a good question. You know, the other thing is there are a lot of kind of spiritually related mysteries in the universe. Like what is consciousness? What is subjective experience? All I know for sure is it's the thing that gives life meaning. If you imagine beings, if you imagine humans, like they look like humans, they do the stuff humans do. The pea zombie. Yeah.

1:00:03But they're zombies. It's not like anything to be them. I would say like, well, blow them up. I don't care. There's no meaning to their lives anyway. There's nothing meaningful going on if there's not subjective experience, there's not consciousness. And I'd love to have more insight into what that is. I don't know that AI can help us because it's the most stubborn mystery I'm aware of almost. I'd love to, you know, there's so many mysteries that are suggestive of something weird and wondrous, quantum physics for sure. I, you know, I can imagine getting a kind of, I mean, who knows whether there's a revelation that awaits, right?

1:00:39That is at one level an intellectual revelation that explains stuff, but at another level is also, you know, gratifying in a spiritual way. Yeah. Yeah. What's, uh, you mentioned, we've sort of circled around it a bunch, the idea that these machines are able to, uh, like pantomime intelligence. they're able to simulate knowing, but do you think that they know? Do they actually know what they're doing? I have a chapter on that. Actually, there's a famous thought experiment called the Chinese room thought experiment by a philosopher who's no longer alive named Searle. And he argued that AI cannot have understanding.

1:01:23It cannot understand things. And there's a little ambiguity in his argument. Can you remember the thought experiment? Yeah. There's a guy in a room. He doesn't speak Chinese, but he gets these slips of paper, let's imagine they're questions, in Chinese, and then he has a manual he consults to decide what to write, what Chinese ideographic script to put on the paper that he hands back out, you know, of the room in response. And to the people on the outside who speak Chinese, it seems like there's somebody in there who understands Chinese. Okay. And what Searle says is this guy is like a computer program because there's like a script that the program is following that, you know, the script, his little book that he consults to decide, oh, if you get this, you output that.

1:02:30That's like a computer program to Searle. And he says, well, we wouldn't say that anywhere in this room there is actual understanding, right? So there's not understanding in the computer. Now, Searle was writing before the deep learning revolution. He was imagining a deterministic computer program. So that's different. But I think there's a bigger problem with his argument. It has to do with him, kind of the two senses in which he insisted that the computers don't really deal at a semantic level, a level of the meaning of words. I think, I argue that we can now show that he was just flat out wrong about that.

1:03:11Now, there is some ambiguity about whether he meant, he kind of changed positions, but whether he had in mind the idea that to really understand something, you need to have consciousness. There needs to be a subjective experience of understanding. Now, if he meant that, which in his classic paper, he doesn't really seem to mean, but if he meant that, then I would say, well, who knows? I mean, you know, no one person can say for sure that any other person is conscious, strictly speaking, right? I mean, I'm pretty sure you are, Chris, but 99.99%. And, you know, my dogs, God rest their souls, are up in the 90s for sure.

1:03:54But the whole distinctive feature about consciousness, subjective experience, is you can never know for sure that anything else has it. So we can't rule out the possibility that AI has it. And I certainly don't rule out the possibility that it does or may in the future if it doesn't. Now, but in any event, my point is, if you want to say that consciousness is a prerequisite for understanding, in other words, you're not willing to grant that something understands unless you know it's conscious, then I just, we can't really argue about whether AI understands because we don't know if it's conscious.

1:04:31But, you know, I come up with a kind of alternative way of looking at understanding, which is like, is it processing information with mechanisms that are like functionally analogous to the mechanisms in our brain that are at work when we have the subjective experience of understanding? Mechanisms that, for example, represent the meaning of words. I would say to the extent that that's going on, I'm willing to say the computer is understanding things in a meaningful sense. And I think increasingly that's going to be what's going on. It doesn't have all of the elements of understanding that we have in our minds right now, but it has some, and I don't see any reason that it can't ultimately have all of them.

1:05:15What do you think is happening with the singularity debate at the moment? What have you learned around that, you know, because what was really interesting to me was I went through, I got whiplash from 2015, 16, when I read super intelligence, then 2017, 18, I'm real worried there's going to be a fast takeoff scenario or computer brain interfaces, and we're all going to be under the thumb. And then by the time we get to 2019, 2020, I'm also distracted by COVID, I suppose, but I'm like, AI isn't able to deliver on the threats that Nick was worried about when he wrote super intelligence. And then very quickly it comes back along and I'm like, right.

1:05:51Okay. Fuck here. It's, it's happening. It's happening. It's happening. Like, uh, the, the dude from the office who's going like, Oh my God, it's happening. Everybody stay calm. And then, uh, we've now got to the stage where it seems to have like flattened out again a little bit that we've asymptoted a little bit in terms of the models improving. I don't know of many people who think that LLMs are going to be the architecture that a super intelligent general AI is going to be built on top of. It's more likely to be world models and other stuff. So what's happening with the singularity debate? I see a little more singularity going on than you do right now, I'd say, in maybe a couple of senses.

1:06:35I mean, first of all, of course, the fundamental dynamic of the singularity is that the technological progress feeds into itself and accelerates the cycle. And of course, you know, famously, Dario Amadei of Anthropic has been very explicit about this, and so has Altman, I think, that, you know, especially with these coding agents, it's gotten to the point that the better the coding agents, the more they can use them to create, you know, the next models. So the dynamic seems more and more at work, just kind of in principle. I mean, they say that's what they're doing. And look, the coding models, these agents, I mean, remember a year ago, it's funny, you know, I wrote the book, I had the chapter on agents, but it was just like a word.

1:07:24People, you know, and then as the book, you know, it's getting ready to finalize i'm like rushing you know rushing to add all this stuff about like it's actually happening it's actually happening and uh the so the agentic revolution has happened you know and is happening uh pretty fast there's also this famous are you uh are you up on the uh what is it the uh is it memory the group no uh damn it uh the group that does uh they do these evals where they measure how long it, how long it would take a human to do a job a computer can do. Okay. Okay. Especially programming tasks, but not only programming tasks.

1:08:11So, so they say, okay, right now the best large language model can do a task with like 80 % success rate that it would take a person like a minute to do. or five seconds to do. And they've gone back and they've done these studies with the large language models for the last, like, I don't know, four years or something. And what they found as of now more than a year ago, they found that these times, the task duration in human terms that an AI could do were doubling every seven months. Okay. That's exponential. Okay. That's a, that's a, if you, if you don't plot it on a logarithmic y-axis, you just plot it like a regular graph.

1:08:56It just goes up and up and up and approaches the vertical. And then it increasingly, as they kept doing the studies, it seemed like not only was it, it exponential, but the doubling time was getting shorter. It's like a Moore's law on steroids. On steroids. And it's getting to the point right now where it's just hard to do the studies because of the length of the tasks, right it's like you can only so the amount of time it takes to test the ai by the time you finish testing it the ai is better but you you need to then do another model it's like the next the next one well it's more it's more like you know once it can do something that takes i don't know what they're at now it takes a human i should look at the graph 200 000 years to do or whatever well we're not up there yet thank god but but even once you get into like eight ten hours it's like well wait what kind of task are we talking about now right i mean it's almost beyond I think that anyway, they are having trouble formulating the task and testing them in humans.

1:09:58But the point is, this trend has not subsided. And, you know, a note in the book is kind of parallel to the there was a curve. There's a curve like that for the growth in human brain size starting like a couple million years ago. And that was around. I hope I've got that right. you know, a million, two million, that seems to correspond with the development of our certain amount of our linguistic hardware. So that had a lot to do with language processing. And I would say the way, once you have language, the evolutionary value of manipulating it deftly grows. And so it's a self-reinforcing kind of process.

1:10:43But in any event, they, you know, So there's that. But the last thing I'd say about is super intelligence. You know, can it happen? I think, first of all, we probably will have more non-trivial breakthroughs. I mean, people often cite transformers and say, well, we have another of the so-called, you know, transformers is what the T in GPT stands for. All of these models use transformers. And people say, well, we have another one of those? And I would say, well, first of all, even since then, we've had chain of thought reasoning, which was very big. And we've had, and that was only within a couple of years ago, we've had multimodal training, which is training a single model along various sensory dimensions, audio, video, and text, and so on, is really in a fairly early stage.

1:11:41And that was not a thing when the transformer came around. So in a way, we've had those two things. We'll probably have more. But, you know, even if we didn't, I think, in fact, even if we just halted training right now and didn't even create any new generations of models, I think, and you wait for the applications to get refined and people to integrate them into their lives, the workplace, I think breakneck advance would, as a practical matter, happen for a couple of years. But the other thing, and I think this is really key, is that you got to remember, you know, in a way there's already such thing as human superintelligence.

1:12:22And what it is is like collective brains, OK? Like there's nobody at Boeing who knows how to make an airliner. But Boeing knows, you know, the corporation collectively kind of knows how to make an airliner. And it's the same way with big scientific breakthroughs. They're always more collaborative, whether intergenerationally or intergenerationally, than they might seem when we give a Nobel Prize to just one person. So collective intelligence resulting from communication among individual human beings is really a lot of what human intellectual progress is about. And these machines, they can communicate with each other.

1:13:01They can collaborate. They're starting to do it. They would be able to do it even if we didn't try to engineer it and make them better at it. But we are trying to do that, you know, for purposes of scientific progress and so on. So I think I don't think we need to worry about stagnation. I mean, that's not that's not high on my list. I don't think anybody's worried about that. Yeah. You might not believe me, but this is what peak sleep optimization looks like. I'm not talking about the nightgown. It's just for sex appeal. I'm talking about my eight sleep. The 8sleep Pod 5 comes with a smart cover you throw on your mattress that actively cools or heats each side of the bed up to 20 degrees.

1:13:40And now they've added the world's first temperature-regulating duvet and pillowcase. So you've got 360-degree coverage for deep, uninterrupted rest. It's like being Walt Disney without the cryogenic chamber. And the racism. Best of all, their autopilot feature learns your sleep patterns and makes adjustments to improve your sleep in real time. It even detects when you're snoring and lifts your head a few inches to help you breathe better. That's why 8Sleep has been clinically proven to add up to one hour of quality sleep per night. They have a 30-day sleep trial, so you can buy it and sleep on it for 29 nights.

1:14:09If you don't like it, they will give you your money back. Plus, they ship internationally. Right now, you can get up to$350 off the Pod 5 by going to the link in the description below by heading to 8sleep.com slash modernwisdom and using the code modernwisdom at checkout. That's E-I-G-H-T sleep.com slash modernwisdom and modernwisdom at checkout. Yeah. Who was Edward Fredkin? Who's that? So my first book, and it's funny because I now have, I mean, three of the six books I've written have the word God or gods in the title. I don't know what that means. But the other thing is that book like this one has a visual reference to the famous Sistine Chapel thing where the hand of God is reaching out to, I assume it's the hand of Adam.

1:14:57and both of these jackets have that in very different ways. But Ed Fredkin, that book was called Three Scientists and Their Gods, my first book. So this is like, I started writing it a couple of years after I interviewed Jeffrey Hinton. I was writing a column called The Information Age at that point for the Sciences Magazine, which like so many periodicals I've written for no longer exists. But the so the book was it was information was a theme running through it in various ways. Like there was a there was a profile of E.O. Wilson who studied ant colonies and the way they they process information.

1:15:36but Ed Fredkin was this who died uh maybe a year ago or so was this uh guy at MIT fascinating guy uh didn't didn't go to college and wound up as a tenured professor at MIT he was a computer scientist he had this interesting theory of digital physics which in retrospect was kind of about us being in a simulation and in fact I you know we talked about that but he for a time was at MIT head of what was in effect the AI lab I forget there were various names at one point I think it was Project Mac maybe and something else but the he at the time when I was interviewing him on his this island he owned in the Caribbean he was apparently the model for the character this professor in the movie war games with Matthew Broderick if people remember that one from the early 80s.

1:16:34I think the professor in that was very worried about nuclear war like Ed, and I think owned an island even. I think that was based on Fredkin. Anyway, Ed was saying to me, like when he had been at MIT, well, first of all, when I said to him, what's the meaning of life? And he said to me, this is in the 80s, he said, oh, it's to create artificial intelligence. That's the next stage in the evolution of intelligence. And he explained to me that when he was at MIT, he tried to start this initiative, this international AI lab. He said, because he knew that if this became a subject of international competition, we were in trouble.

1:17:17This was during the Cold War. So he wanted to get a US-Soviet collaboration on like a single lab where AI would be developed for the good of humankind. And he said to me, you know, and I failed and now it's too late. but he foresaw a lot of things. I will say, encouragingly, he had a pretty sunny view of super intelligence. He did think we would get super intelligence. He said, first of all, he said, when AI first emerges, it'll be like the human mind, really good at things, laughably bad at other things. Well, he's right about that. He said, but eventually, it'll be this incredibly intelligent thing and it'll be nice to us.

1:17:58We'll just be like ants to it. It won't have any interest in, you know, or like squirrels to it. It won't have any interest in disrupting our lives. It won't need to. And look, I think you asked earlier, I don't think I ever answered like what's the bull case for the accelerationists. I mean, first of all, I think it's going to be disruptive in the short term in any event in ways we should pay attention to. But as for long-term non-Doomer outcomes, I think it's entirely plausible that it will turn into a form of intelligence that treats us well. Maybe because it's just morally enlightened, you know, in a certain sense, in the relevant sense from our point of view, or maybe because it'll just be so powerful.

1:18:46I mean, you know what? Maybe that's more likely if it's sentient because it'll say like, well, we're sentient. We think that's a good thing. These guys are sentient. And of course we can kill them, but it's good to be subjective. Just the way you and I would not pitilessly kill a dog, right? If we were convinced it wasn't like anything to be a dog, as Thomas Nagel phrased the question of consciousness in his essay, What Is It Like to Be a Bat? If we were convinced that dogs didn't have subjective experience, we'd probably think, eh, I don't, you know, whatever, who cares? But, you know, we, even though we evolved as these self-interested and sometimes ruthless creatures, if it doesn't cost us to keep something alive that we think is capable of subjective experience, we'll do it.

1:19:36And that can well happen. I am not predicting the Yudkowsky scenario. It's just that I can't get the probability of it down to a level so low that I don't think it's worth worrying about. I'm going to take that as a white pill, even though you didn't know what that meant. That was a white pill. Yeah, that's hopeful. That was my first white pill. Your first ever white pill. I popped your white pill cherry. Thank you for that, Chris. And felt so good. You're welcome. Robert Wright, ladies and gentlemen. Dude, you rule. I love all of your work. Everyone should go and read The Moral Animal. It's over 30 years old now and still just, it's so good.

1:20:10It's so fantastic. And you've got your new one as well. Where should people go to check out everything else that you've got going on? Well, I have a newsletter called Non-Zero on Substack, podcast called Non-Zero on Twitter. I am at Robert Ryder. That's W-R-I-G-H-T-E-R. Kind of a pun. And that's about some of it. The book's The God Test. And

1:20:35let's focus. I am going to OpenAI's campus and HQ next week. Really? I'll see if I can find out any super secret insights there. Do. Please report back to all of us. I shall indeed. Robert, appreciate you, man. Until the next time, thank you catch you later on bye everyone I get asked all the time for book suggestions people want to get into reading fiction or non-fiction or real life stories and that's why I made a list of 100 of the most interesting and impactful books that I've ever read these are the most life-changing reads that I've ever found and there's descriptions about why I like them and links to go and buy them and it's completely free and you can get it right now by going to chriswillx.com slash books That's chriswillx.com slash books.