Ep. 380: ChatGPT is Not Alive!

25:54They're the most comfortable sheets we own. I think we have something like three different sets that we rotate through to make sure that we're never sleeping without them. But I just got recently the bamboo pajamas, and it's really an incredible set of PJs because they're super comfortable. It's like the fabric from the sheets, but softer. And now you can just wear them on your body all the time. Well, I've been thinking about both the sheets and the pajamas because look, the nights are getting colder. The holidays are upon us, right? So this is the time then we think about being cozy. Now, when it's that kind of cold night and you're downstairs by the fire, having on the soft stretch knit of the bamboo pajamas, that's a game changer, right?

26:37Right. Having the like their bubble blanket. Oh, I love that one. They have this this this blanket, which is so comfortable that we fight over in our house. We have to cuddle with that by the fire. Oh, that's what we want this time of year. And then the ability to go up and get in those sheets, you know, are gonna be so comfortable. I don't know the holiday season when things get colder. I think cozy earth because I think comfort and I think coziness. So if you're if you're looking for the perfect holiday gift, whether for yourself, you deserve it. for other people you care about, join the Cozy Earth community.

27:09Start with the amazing bamboo pajama sets and the bedsheets, and then you can go on from there. So give the gift of comfort that lasts beyond the holidays. This weekend only, from Thanksgiving Day through Cyber Monday, you can get 40 % off at CozyEarth.com if you use the code DEEP. This is the best deal of the year. You don't want to miss this. That's code DEEP for 40 % off. and if you get a post-purchase survey, be sure to mention that you heard about Cozy Earth right here on the Deep Questions podcast. Wrap the ones you love in luxury with Cozy Earth. I also want to talk about our friends at Lofty.

27:46Like we had daylight savings time not so long ago and with those shorter days and less sun, our internal clocks begin to get mixed up. It messes up with our sleep hygiene. Fortunately, this is where our sponsor Lofty enters the scene. The Lofty clock is a bedside essential engineered by sleep experts to transform both your bedtimes and your mornings all right here's a few things i like about the the lofty clocks hey they're beautiful right it's like they're really well designed it's kind of like a modern i don't know the terminology but it looks really nice it's very simple uh two it has gradual wake up so you can have for example a two-phase alarm where you get like a the sort of soft wake up sound that kind of like hey it's kind of time to wake up that slowly brings you out of sleep.

28:31And then you get the more definitive sound, which like pulls you the rest of the way, as opposed to just jarring you out of sleep all at once with like some sort of really loud buzzer. And two, it's entirely operational with the physical clock itself. You don't need your phone in the room. You know, I don't want my phone in my room. I'm a big believer that your phone stays downstairs in the kitchen. But a lot of people end up with their phone in the room because they need the alarm function or they need to use the phone to control their alarm clock. Not with Lofty. No phone. Keep that downstairs.

29:00You can control it, turn it off, snooze it, whatever you need to do right there from the actual physical device. We love these types of clocks, these gradual wake up types of really nice looking clocks. We have four of them, all three of my kids and my wife and I, we all have one in our room. My 10 year old has discovered the snooze function, which is kind of a problem because he doesn't realize that snoozing the alarm doesn't also snooze the bus. Like that's still going to come in the same time. but otherwise we're a big fan of these type of clocks. All right. So these clocks are a better, more natural way to wake up.

29:32They also look great and they keep your phone out of your room. You can join over 150 ,000 blissful sleepers who have upgraded their rest and mornings with lofty. Go to buy lofty.com and use code deep 20 or 20 % off orders over$100. dollars. That's B-Y-L-O-F-T-I-E dot com and use that code B20. All right, let's get back to our discussion of AI. As promised, I want to get into Jeffrey Hinton. Jeffrey Hinton knows a lot about language models because he helped invent the fundamental algorithm that's used in how we trained them. And yet, in recent years, he has been sounding the alarm about AI in ways that kind of sound similar to the arguments that Brett Weinstein was just making that I was just saying are based on an inaccurate understanding of how these things function.

30:25So I want to try to resolve these discrepancies. Jesse, let's start by playing a clip of Hinton. This is from earlier this year, giving, you know, sounding the alarm. And I think until quite recently, almost everybody thought that was just science fiction. We didn't really have to worry about AI becoming smarter than us and taking over from us. But I think now many people have come to realize that's a very real risk. So most of the experts believe that sometime between five and 20 years from now, AI will get smarter than people. And when it gets smarter, it won't get a little bit smarter, it'll get a lot smarter.

31:00And the problem is, we know very few examples of smarter things being controlled by less smart things. All right, so these are pretty dire warnings, but what is he actually saying? Well, we can get some more insight if we go back to a great profile of Hinton that was written by Josh Rothman in The New Yorker back in 2023. And if we look in that profile, we can figure out, first of all, what was it that in more recent years made Hinton suddenly afraid? He wasn't warning about AI 510 years ago, but in the last few years, he's gotten worried about it. So the first thing we can do is identify what did he see in new AI technology recently that got him worried.

31:40So I'm going to read now from the New Yorker profile a relevant piece of the answer to this question. All right, so here's Hinton from the profile. People say it's just glorified autocomplete. Now let's analyze that. Suppose that you want to be really good at predicting the next word. If you want to be really good, you have to understand what's being said. That's the only way. So by training something to be really good at predicting the next word, you're actually forcing it to understand. Yes, it's autocomplete, but you didn't think through what it means to have a really good autocomplete. All right, so this is an important point, and it's one I've made repeatedly, including the introduction of this episode.

32:16What made Hinton originally impressed is that in winning the game of predicting the next word, which is what these models are trained to do, they got a lot smarter than he expected they would. Because if you think about it, just like he argues here, if you really want to be good at winning the game of what word should come next, you have to really understand the text that you're extending. And like in the example I gave earlier, if the text you're extending is like a joke and the next word is a key part of the punchline, well, like to win the game there, you have to understand humor. And in fact, if you read longer in the Rothman profile, this was the thing, the specific type of functionality that really caught Hinton's attention is when he realized these models are – they can produce text that's funny.

33:03Like they can understand what funny is. And that's where he said, oh, the token prediction game, if you train it on a big enough network long enough, requires these models in those vast tables of numbers that define them to encode really complicated pattern recognizers and logics and knowledge and language understanding. Like it gets pretty sophisticated. So there's a lot of understanding. That's what James Summers argued in his point as well, encoded in those vast table of numbers. And Hinton was caught off guard by, wow, just how much understanding a language model can build up. But here's the thing.

33:41Hinton is not worried about language models. Him seeing how much understanding language models built up to win at the prediction game got him, and this becomes clear in the profile, got him thinking, well, if we could solve that problem so much more impressively and faster than I would have guessed, I'm going to get, I'm going to assume that we could do that with other AI related problems as well, right? Because he says, if you look at it closely, he's not worried about language models. He would completely disagree with Weinstein that language models as we have them today are conscious or that they're running experiments or that they have intentions.

34:18No, he knows how they work. He helped invent the technology. He knows it's a static table of numbers and there's a rote multiplications that moves information through. There's vast understanding of knowledge and coded in those numbers, but the actual operation of executing does not operate like a human brain. It can't be conscious. There's no intention, et cetera. But he was so impressed by how quickly that technology went that he began to worry about, well, what other AI related technologies are going to start going faster as well. So what he's really worried about is hypothetical new machines that we haven't built yet, but now he's saying maybe we will build them faster than we thought.

34:55We could build a language model to understand humor. When I thought that was impossible 10 years ago, why can't we also build an artificial brain that can be conscious or have intentions or manipulate us or try to actually do things that are outside of our interest? Here's another quote from the New Yorker profile that gets at this reality that Hinton is worried about future machines that we haven't built yet. Here's the quote. Hinton warns that even a benign autonomous system could wreak havoc. You want a system to be effective, you need to give it the ability to create its own sub-goals, he says.

35:24Now, the problem is there's a very general sub-goal that helps with almost all goals get more control. So what he's talking about here is not a language model. Language models don't have goals. Again, it's a static table of numbers. You multiply vectors, you get a word out on the other end. There's no goals, right? There's no intentions. There's no decisions being made like that. But he's thinking like, oh, what if we did build a system with goals? What if we built a system that could make decisions and plan and figure out what it wanted to do? Then that could be pretty dangerous. So Weinstein's talking about language models.

35:53Hinton's talking about AI, artificial brains that we haven't built yet. but he's now more confident than he was before that they are buildable. That's different. Now, here's the key question. Do I agree with Hinton that am I one of the, he says, all experts agree within five to 20 years, we'll have AIs that are more intelligent than us. I'm not so quick. I don't think that's that easy to build. I mean, Hinton's an expert on language models, but now he's speculating about just artificial brain designs that don't exist yet. There's much more of an even playing field here. I'm not so sure that we can keep making progress with these brand new designs that don't exist yet to get something like an artificial brain that does the type of things that like Weinstein was worried about.

36:36All right. We know this in part because we've been trying in simple ways and it's like been going okay. So like the simplest thing we've been trying recently is what are known as like AI agents. And really what these are is like a control program, the straightforward program, you know, written in Python, just like normal code a human wrote. that has like linear logics we build into it. And then it prompts language models to try to get ideas or plans, right? So like, hey, language model, here's what I want to do. Give me a plan for this or tell me what I should do. And then it gets the answer from the language model.

37:08And then it goes and executes on its behalf. We talk about AI agents right now. That's what we're talking about. Those are okay, but they're not working that well. Like 2025 was supposed to be the year of AI agents where all of these programs that call language models, We're going to automate more and more of our work tasks. That was mainly a failure because it turns out that, yes, language models have really sophisticated understanding baked into their vast tables of numbers. Like they really understand words and have a lot of logics. But there's also a lot of things these language models don't do because they weren't.

37:39That's not what they're supposed to do. The keeping models of the world, understanding how things work, planning, trying to figure out different scenarios for the future. they don't have any of those capabilities because again it's a it's a sequential static processing so it can just move a number through and they were trained on word prediction so what they have encoded in there makes them good for predicting words but they're not very good at like let me make a plan about what to do for you in your office that's going to make sense given your particular job and the people you know doesn't have any of that information and it can't simulate the future and it doesn't have a really good world model we thought they might but they don't have a very good world model.

38:15We now know this from research. So the agents we build that just ask language models, they're inconsistent and unpredictable, but not that great. That's why we don't have a lot of these, even though that was supposed to be the financial savior of open AI in 2025. They're just too unpredictable for anything but the simplest of tasks. And so that's why we haven't seen them used. So if we really want to build a more complicated artificial brain, there might be a language model module that we can leverage that for its deep understanding of language and what it means and its ability to create fluent language.

38:44But it's also going to need other modules to do all these other things that brains do. It's going to need a module that has like a really good model of the world. It's going to have to have something like a policy network that we encode sort of like its values and drives. They can evaluate potential actions to figure out what action is good based on what happens. It's going to need memories. It's going to need updatable state. It's going to need actuation, ability to try things, sees what happens, update its memories, update its state. It's going to have to hook these all together in some sort of complex way.

39:11The language model might be in there, but it's one of like a bunch of different pieces we would need to have anything like an artificial brain that does more than like what these very simple agents do. But we don't really know how to do that. You know, in fact, for an article I was working on earlier this year, I ended up interviewing some of the world's top roboticists, and they've been working on some of these problems in the world of robotics, especially building a good general world model you can use to plan actions. The advances in language models really have no impact on that challenge. That's its own challenge.

39:42The fact that we're better at language models doesn't help that challenge very much. And they have no idea how to do this well. They've been working on it for decades. There hasn't been any major breakthroughs recently, and they're continuing to work on it. And they have no optimism that like, oh, yeah, we're just a year away from being able to model the world. That's like a completely separate challenge. Policy networks, good simulations of general types of operations, like these type of things. People are working on them, but they're different technologies and language models and they're hard.

40:10So we need to have breakthroughs in all these different non-language model technologies to maybe get something like a more complicated artificial brain. But even if we do, now we have a very module AI system where we have different systems that we're connecting together. That's way more controllable than like a language model's understanding. We're like, we don't know what's in its neural networks and we don't know what word it's going to produce and it's very unpredictable. so we are so far away from something like a brain with intentions that can manipulate us or get smarter than us and if you really push hinton i read a bunch of interviews with hinton if you really push him well what are these systems going to look like five to twenty years ago they're not language models what are they going to look like he more and more now falls back on the same explanation that i talked about a couple weeks ago when i analyzed eliezer's uh yudowsky's conversation with ezra klein he falls back on recursive self-improvement i don't know but maybe the AI will just build smarter and smarter AIs, which, you know, is a complete cop-out.

41:11You know, I don't know, but the AI will figure it out. That's not that interesting to me. All right, let's go to some takeaways, Jesse.

41:29All right, the most computer scientists, especially computer scientists that are not associated with a particular company that makes money by selling AI. They look at language models as being exciting and impressive, but not mysterious and not scary. Now, once you understand how these models actually work, you realize that they are much, much simpler and much, much more mechanical and way more limited than the way a human brain functions. And even if the pattern recognizers and rules and knowledge that they encode are vast and impressive, their operations are understandable they are not alive they do not have motives they have no memories they cannot act on their own they cannot manipulate us they certainly cannot approach anything like consciousness what they can do is understand text and produce fluent answers in response and that is really useful for a lot of things but not everything and as researchers learned when they tried to build like simple agents by hey here's a control program that's going to query an LM to figure out what to do.

42:27LM's are not even really good general thinkers. Again, they're good at producing fluent language, but they're not general thinkers. They do not have general capabilities. To create the type of AI that is, the type of AI that concerns Jeff Hinton, the type of AI that Weinstein thinks language models are right now, you would need many, many more types of modules that do many more advanced things that are hooked together in complicated ways. And we don't know how to do most of that yet. We would need systems that have world modeling, simulation and planning, drives, motivational systems, memories, and real-time learning.

42:55All of these are things that language models don't have. All of these are hard. We're nowhere near having coherent systems that function with all these capabilities together. So AI presents many problems, but most of them are happening right now, not in the future, but right now. Problems like impacts on our mind, our relationships, our politics, our environment, or our very definition of truth. So when I look at AI, I think the right question is like, hey, who is asking for this? What good is this bringing us? Why are we tolerating all these harms that it's causing right now? What problems is this technology actually solving in our life?

43:31Why are we standing for all the real issues that it's creating? The question that's not interesting to me is when will it come alive and is it trying to trick us? I think that latter type of question is exciting to ask. It makes for good conversations on podcasts, but it distracts us from the real problems that AI is causing right now. and when you understand how these systems actually work, we can be much more specific about what they can and can't do. And most of these science fiction type concerns fall squarely into the camp of things that current systems can't do. All right, there we go. A little bit more AI.

44:05I don't know. I can't help myself, Jesse. I just, I think about AI a lot. I read a lot of these articles. I hear a lot of discussion of it, and it still seems very messy to me. Do a lot of your students go into AI as a career? You know, our AI classes are popular. Yeah, for sure. I think there definitely is a lot more interest in AI because it's the various AI systems, especially those around the language model and visual model spaces are like really interesting, impressive systems. But they're not just, man, it really confuses the conversation when we get into this type of like, it's just like a baby and it's conscious and it's trying to figure out how to change the world.

44:41And we can't just say whatever we want to be true about AI. I'm an algorithm theorist. the whole way we see the world of computation is give me a model give me an algorithm and that uh uniquely specifies capabilities what it can and can't do and how fast it can do it like everything is about constraints what can this system do and not do what can this system do and not do but with ai non-technical critics just come at it from uh this system can do anything why not but how they operate really matter so i don't know i try it seems to upset people sometimes when i talk about this there's i think some people are really invested in like no no no it's a baby's brain that's probably manipulating us right now were a lot of those classes you took when you were in college and your phd program similar to the classes now it was much older technology like i i took i took a course at mit that is now completely you would never outdated i took i took a course in automated speech recognition right so how do you have a computer understand if you you speak into a microphone and say something how do you convert that into like text we literally like the exams in that course was looking at waveforms right and learning like oh that's a that's a explosive that's a t8 sound like we're trying to learn like how the different shapes connected to sounds and building these like simple recognizers that like you would hand code that would break the waveforms into uh the individual sounds they look like and then put those together and try to figure out what word it is this is completely solved by machine learning today you're just like no i'm just going to feed you a bunch of uh sounds where we know what we someone typed out what the what the words are and you just i don't know how you do it just train yourself until like you're very good at predicting what the actual words are so there was no reason to try to just old-fashioned ai we were trying to like program in manually what different sounds look like and now you know you make one call to uh pi torch in like a python program and you know it's way simpler yeah so like that was way out of date like there's a lot of machine learning around but it was more specialized a lot i mean this is getting kind of nerdy but like neural networks people were coming around on neural networks but there was a lot of people really liked in the 90s coming into 2000s the more mathematically precise machine learning tools so the the tools that had like precise mathematical theorems you could prove around them they didn't want to just take like a neural network and train it unsupervised with a bunch of data and see if it got good you would use things like support vector machines which is like a very precise mathematical way of trying to learn you know separators and multi-dimensional space you can prove things about like it's going to converge in this many iterations people were very comfortable with that it was really in the 2010 that this idea of no no neural networks can just learn stuff on their own even though mathematically we don't know why we can't prove why they work like that that took off So I was in grad school in the 2000s.

47:41And so it was a very different landscape. Last question. If you went into one of the Georgetown courses today with dealing with AI, would you do fine in it? Would you know everything they're talking about? Yeah. I mean, I'm not doing a lot of development. I'd have to learn to libraries like the coding environments. But no, I think I could do well. If I took one of those courses, I'm sure I'd do okay. Yeah. I mean, a lot of this stuff today is like it's all captioning libraries. you're not writing a lot of original code it's like a lot of like the art with machine learning based neural networks is like parameterizing it like what struck what type of network do i want to use how big should it be how many layers should it be um it's a lot of uh this like subjective like i just have a feel for like how i'm going to set this thing up oh that didn't really work let me reduce the layers and include increase the density of parameter oh that works better like there's like this this sort of feel to it but it goes to show how hard it is to predict what's going to be big because like if i had studied deep learning when i was in grad school it would have set me up perfectly time-wise to be like one of these you know two million dollar a year ai hires but no one get more than that we're getting these huge deals but like no one that's kind of what got me thought thinking about that actually so it's a supply and demand thing that's the problem is no one knew that and now a lot of people are studying it but by the time they're ready there'll be a lot of people studying it and they're not gonna be worth that money So you had to be the person who in like 2005 was like, you know, Jan Lacun's on to something, deep learning, like we have enough data now, let's go get some GPUs.

49:12You had to be that guy or woman in 2005 in order to like in 2025 to really be balling out. It's like impossible to predict sometimes. All right. Let us move on to some questions.

49:29First question is from Maeve. can other forms of ai like those used for military biomedical and financial purposes be dangerous can you can your explanation against lms taking over the world extend of this sort of ai use case well here's what's important about this question like ai is a broad term often what people are talking about when they use that term today are language model based tools like chatbots right that's like what brett weinstein was talking about but ai has been around for a long time and there's lots of other tools that use something like ai that try to do something useful like when I first started at Georgetown there's a group there that continues to work today like very closely with the hospital at Georgetown and I do a lot of work with you know medical records looking at films or test results and looking for patterns and helping people to figure out like hey do diagnoses or whatever that's something to do with language models language models are very specifically for producing text but you know they are working on that lots of people have been working on that radiology has been like a subject of AI interest for decades can we train models to be really good at looking at uh mri or x-ray films and finding things that like a human radiologist might miss like that i i think there's a there's a cancer over there that maybe like the human observer missed and they're like you could do pretty good at this some of these things there's a lot of plateaus that they hit like jeff hinton famously said by like 20 whatever a date that already passed there would be no radiologists left there's just as many radiologists as they are we haven't actually cracked the problem of studying these films with AI nearly as well as we thought we would be because there's like plateaus.

51:03Like you run out of data, the techniques can only do so much. There's a lot of different AI used in a lot of different places. They're not language models. I think that's important because people often get this wrong or they mix it up. They mix all these things together and they assume because language models made rapid breakthroughs in a five-year period that these other AIs did as well. But there are different types of problems and different types of systems and they're not connected. You know, some people think, I hear this all the time, they think it's like the language models that we're using, that will be the key to biomedical AI.

51:33Most biomedical AI has nothing to do with language models. Language models is what we're going to use. Now our weapons will be smarter. No, that's a completely different type of AI that you're going to use in weapons. Or robots now. Look at Tesla's Optimus robot, which they basically canceled that project. Like, oh, because of language models, now our robots will be real smart. No, that's completely different technology. They're not having comparable breakthroughs. There's a lot of AI technologies for a lot of areas. There's protein folding, playing games. These are all different types of AIs, and they're all having advances at different speeds.

52:05None of them really are having advances at the speed of language models with the exception of some of DeepMind's work. They were having some big breakthroughs, I think, on games, and the protein folding was a big breakthrough. But those are like bespoke systems. But for the most part, these are all separate AIs that require separate breakthroughs, and they're all moving at their own speeds. Most of these AIs have been around for a long time. The technology is there's nothing really new. so if you weren't afraid five years ago of these technologies becoming super intelligent then like you shouldn't be afraid of that today there hasn't been any major change for most of these AIs and how they operate it's much more incremental trying to break through plateaus so no these are all like systems we can understand that operate in specific ways and are really hard to get to work well and they're working to try to get them to work well and the gap between how do we get better at noticing abnormalities on MRI scans and Skynet is so vast in all these technologies that no, they don't it's just the wrong question it's just the wrong question are these things going to get out of our control it's just the wrong question we're trying so hard right now just to get them in these bespoke fragile systems just to do what we're asking to do in a way that's useful to us that's what we're working on with most AI systems there's a lot of AI, most are not language models they all make progress on their own pace breakthroughs are unevenly distributed and none of them are on some trajectory of of you know where i would be concerned about uh oh my god it's about to be really dangerous but again this we mix this up a lot i'm uncomfortable with the language produced by a language model and i'm going to translate that to be uncomfortable with the fact that there's like an ai involved in a biomedical application these are different systems they work in different ways all right what do we got next next up is from m you said that it's imperative that we stop distracting ourselves with the fear of super intelligence and said focus on actual concerns of ai what are those concerns well i think this deserves its own really its own full episode but some of the common concerns that are out there that i share about ai is things like its impact on our ability to think and create new knowledge fluently uh it's like have using a car more than walking but the cognitive equivalent of that.

54:18You're going to lose the ability to produce new knowledge from scratch when you're over dependent on language production. Language production is hard. Language models are good at language production, so we use them. But that hardness is how we actually get better. We cement those circuits and get more facile with producing original knowledge. So I worry about that. I worry about the obviously impact on truth. Creating a society where we really can't trust what we see, be it a picture or a video, like that's going to be, that's a real problem. I'm worried about slop in all of its forms, whether it be like super distracting slop like you get on an app like Sora or Office slop, or you have more people lazily producing long emails and meeting summaries with AI that don't have much useful content and waste everyone's time.

54:57Social media slop, it's just like a bunch of, you know, whatever. Instagram just becomes a lot of sort of middle of the road, sort of like generic inspirational sayings, like none of that's that great. I'm very worried on the financial impact that if there is a big market correction because of our overextending in this race to build AIs that aren't going to be as powerful as we think, that's going to hurt everyone's portfolio. That's going to hurt the economy. That could cause real problems. And there's environmental concerns. Though my prediction for the future of AI is that these giant models that use up all this energy, it just doesn't make sense financially.

55:31And really the future of language models are going to be much smaller language models that are deployed at the edges, right? So you're going to have a language model on your phone to understand your text, not you sending a query to a giant foundational model in a data center somewhere. There'll be a language model on your computer as part of the word processing application that helps does things on your behalf. So I think we can probably solve the environmental issue because a language model that uses so much energy to query that it's environmental problem is not a very financially sustainable type of program to run.

56:03And we don't need an Oracle can do everything. I think we need Oracle's. It can do the specific things we need. So I'm worried about it, but I think we might solve that one. All right. Who do we got next? Next up is CJ. I recently heard Dr. Hinton assert that there is no evidence in nature of a more intelligent organism or species being controlled by a less intelligent one. This observation was made as a rebuttal in the context of the question as to whether we can control AI or not. If Dr. Hinton's observation is true, would this not create an internal break on recursive self-improvement while humans may be so dumb as to create a new technology that we cannot control and ultimately destroy us won't ai be smart enough at some point not to make the same mistake i thought that was really clever it's actually like a pretty clever thought experiment right it's saying like okay if it's true that things that are smarter than you uh aren't going to let you control it then why would you build something smarter?

57:00He's like, okay, maybe we're dumb enough to do this. But as you're doing recursive self-improvement, at some point you're going to have an AI smart enough, right? To realize why would I build something smarter than me? It's going to take control of me and it would, it would refuse to do it. I think that's a really cool thought experiment, but here's the bigger issue here. I'm getting kind of tired of these thought experiments about super intelligence that we talked about a few weeks ago. They come out of the sort of rationalist community, existential risk community. to me discussing like the these type of thought experiments about recursive self-improvement is like discussing time travel paradoxes it's kind of interesting well wait a second if i could go back in time and then i murdered my father then how would i exist and how would that work it's like an interesting thought experiment but we're not going back in time like it's not actually we don't really want physicists caring about time travel we don't want like legislation you know being passed to prevent time travel paradoxes.

57:54It's just, you know, it's mental masturbation, right? Like we're not going to invent. We're not inventing these super intelligence. This is not going to happen. RSI is not going to happen. We can barely vibe code like really simple. It's not going to invent technology. That is, it just, if you understand these technologies, that's just not going to happen. Or being able to help you like produce code that it's seen a thousand examples of, it's not going to enable you to now create like the next new AI model no one's seen before. so this is like to me the same as discussing time travel paradoxes it's interesting it's really not like a serious conversation though it's fun not serious and not something we should be losing sleep about all right we got a few more questions uh and then uh we have a case study that's a powerpoint coming up yeah someone sent a powerpoint of their like organizational system which i think is really cool we're going to read some comments later but first let's take a quick break to hear from another sponsor that makes this show possible.

58:50This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up required, compatibility and availability varies 18 plus. Are you looking for a gift for your parents or aunts or uncles or grandparents? Let me tell you exactly what you should get.

59:22An Aura frame. These are digital picture frames that allow you to easily upload your photos, right? You can do it straight from your phone right after you take the picture with just a couple of clicks. Then that frame will rotate through all the images that you've uploaded to this. So we've bought these frames for both my parents and my wife's parents. They love them, right? It's become our habit that every time we're doing something interesting, we just upload some photos right from our phones to their frames and they get to see, oh, what we're up to. See pictures of the grandkids, see pictures of their kids.

59:54They really do love it. Here's something else you can do that's cool. You can pre-upload photos to a particular frame that you're buying for someone as a gift. So as soon as they unwrap and set up that Aura frame, boom, they're going to have photos there to look at right away. You can even imagine getting one of these for yourself, put on your desk at work. Why not keep up with pictures or get a little boost of pictures of your own kids or family while you're otherwise toiling away? Look, you can't wrap togetherness, but you can frame it. Now, for a limited time, if you visit AuraFrames.com, you will get$45 off Aura's best-selling Carver Matte Frames, which were named number one by Wirecutter.

1:00:33You just have to use the promo code DEEPQUESTIONS at checkout. That's A-U-R-A frames.com promo code deep questions. This exclusive black Friday cyber Monday deal is their best of the year. So order now before it ends. Remember to support the show by mentioning us at checkout. I also want to talk about our friends at Caldera. Here's the thing about being an aging guy. When you're young, you don't think about your skin. You spend time in the sun, maybe like only occasionally cleaning the grime off your face with a Brillo pad and some axle grease. and you still end up looking like, you know, Leonardo DiCaprio in Growing Pains.

1:01:09But then one day you wake up and you're like, oh my God, I look like a grizzled old pirate captain or maybe like one battle after another version of Leonardo DiCaprio. And you're like, why didn't anyone tell me I'm supposed to take care of my skin? I actually, you know, I can't say what it was, but Jesse knows I was doing a thing recently where I was being filmed by a big film crew and there was a makeup artist and she was getting on me about it, right? She's like, yeah, you got to moisturize. It's like you can't put makeup on for camera if you're not moisturized. It's like, oh, it told me this.

1:01:37All right, so how do you avoid that? Oh, and I told her about this product, by the way. I was like, I know exactly what to do. Caldera Lab. Their high-performance skincare is designed specifically for men. It's simple, effective, and backed by science. Their products include The Good, an award-winning serum packed with 27 active botanicals and 3.4 million antioxidant units per drop. Their eye serum, which helps reduce the appearance of tired eyes. and their base layer, which is a nutrient-rich moisturizer infused with plant stem scales and snow mushroom extract. This stuff works. A consumer study 100 % of men said their skin looked better and healthier.

1:02:11So men, if you want to look more like growing pains Leonardo DiCaprio and less like a grizzled pirate captain, you need to try Caldera Lab. Skin care doesn't have to be complicated, but it should be good. Upgrade your routine with Caldera Lab and see the difference for yourself. Go to calderalab.com slash deep and use the code DEEP at checkout for 20 % off your first order. That's C-A-L-D-E-R-A lab.com slash DEEP and use that code DEEP for 20 % off your first order. All right, what do we got next? Next up is Ben. Does James Summers make a valid argument in his recent New Yorker article that AI can think?

1:02:48It's a good article. I like that article. So James is one of, there's several writers of The New Yorker like who are on various, either the contributor staff or the staff writer staff. There's three of us that I know of that have technical backgrounds. So James has a development background. I think he's still actually an active computer programmer. Kyle Shaka has a development background, and I'm like the sort of resident computer scientist. And so we kind of know each other and each other's work. His article on AI thinking came out in the magazine a few weeks ago. I thought it was good. Here's what he's arguing, and I think he's right.

1:03:22He basically is arguing what Jeff Hinton said before, that quote I read is like, yes, while it's true that what a language model does is just produce a single next word. It's trying to win the game of guessing what word comes next. In doing that, it can deploy such complicated processes, pattern recognizers like language knowledge, logics, you know, like what would be a good joke or how do I follow the logic of the argument and knowledge like it can pull from that there's a argument you can say that's thinking that's a form of thinking give me information and I can apply like lots of things to it that produce some new type of information that depends on like deep understanding you say like that's a form of thinking so we can say that even though the operation is simple what it's doing is thinking I think that's fine I think you could definitely call what happens in a language model when that algorithm takes the big static table of numbers and does all those multiplications you can call what's happening there thinking there's understanding happening etc well there's lots of different types of thinking and this is doing one type of thinking so if you want something like an artificial brain that we have to be worried about there's all other sorts of thinking you do as well like see planning the future having a model of the world responding the drives learning on the fly these learning models don't do any of that type of thinking but they do this one thing there's one type of thinking really well and i think this this fits well with my analogy of thinking about a language model like the language processing centers of your brain being isolated in a vat that's part of thinking when the language processing part of your brain tries to understand the words that it's encountering that's part of your brain thinking and if you take that out and put it in a vat there's thinking happening when we when we activate those neurons and send words through that neural network but it's not all the types of thinking that happen in our brain so these are the two thoughts we have to hold true in our head at the same time.

1:05:18We can have a system, a digital system, do something that we say, yeah, that's a type of thinking. We can mean it. And at the same time, we can look at that system and say, we know how it operates. It's pretty straightforward how it operates. There's all sorts of other types of thinking that we do as humans that it's not doing, and it never can do because of its structure, how we wired it, and the algorithms that we run on it. So I think both are true. And Summers makes that argument towards the end of his article as well. All right, who else do we have? Next up is Kim. How can I figure out a way to get back into my preferred workflow when I feel like the industry is increasingly moving towards AI?

1:05:53Kim's a junior developer. And she says, if I try to develop exceptional skills now, I don't have the career capital to move into a job where the flaw of not wanting to integrate AI so deeply into my workflow can be overlooked. So you got to balance two things with AI. Places where it could help you and places where it's just like your management team is like, I read a bunch of, you know, I listened to the interview on Rogan and we need AI. Like we just do AI, right? You got to walk that line between useful AI and non-useful AI. So keep abreast of new tools as a developer. See which ones make you better.

1:06:26Use them. See which ones don't. Ignore them. The key thing I would say is the, and this isn't programming. There's a lot of tools that can make things easier and better. You have to use those tools to raise your level. So if what you're doing is just great, this can automate other parts of what I do. So now I can like get my work done faster and not have to work as hard. That's dangerous. If AI as a programmer can help you do certain things faster, use that to become a better programmer. Like, oh, now I am a better programmer than I was before this tool. It's not a faster programmer, a better programmer.

1:07:00If this can handle these things for me, now maybe I can build more sophisticated systems. Or if I can get help from AI in figuring out these complicated library calls, I can now use these libraries that intimidated me before. And now I'm open to like I can build much more complicated stuff. So the AI that helps you use it to make you better, not just faster. And the AI that doesn't, don't worry about it. Ultimately, the value of what you produce is what matters. All right, let's move on now. We'll take a quick break from AI to do a case study. Now, typically, this is where people send in a description of them applying the type of advice we talk about on the show.

1:07:33So today we have a PowerPoint. Someone sent in a PowerPoint of a system they built for organizing their work life built on ideas from the show. So let's hit some case study theme music and we'll pull this up on the screen.

1:07:54All right. So here's the system for those who are watching. So just listening. How do we pronounce that, Jesse? He calls it Zettel Toffel. Yep. Zettel Toffel. it says at the top it is uh al newport inspired there we go all right let's see what we got here what is zettel toffel once upon a time a kanban board and a zettel casting got married had a baby and the zettel toffel was born it handles the configure and control portion of productivity management in a clear tactile and analog manner all right and he specifies here that he's a cfo for hire so he's like a entrepreneur or knowledge worker all right there's a board here we're looking at labeled Zettel.

1:08:31Oh my God, it's complicated, Jesse. Oh my. All right. All right. There's pending tasks and those are, there's rows for different pending tasks like budget, general finance loans. And then I like this, there's post-it notes under each of these sections describe individual tasks. Then there's in progress for each of these categories. So that way you can keep track of like all the finance stuff you need to do, but here are the two things he's actually working on at the moment and then there's waiting so i assume what happens there is if he's waiting to hear back from someone about some task he can move at the waiting so he doesn't forget and then move something else in progress to keep making progress so that's kind of like classic kanban which i think is great you're being very clear about your workload you're not forgetting things but you're also being careful about how much you're actively working on so i think that's pretty cool i don't quite know what's happening over here he He has my day.

1:09:24Oh, I see. I think he's moving these tasks over to his day. He says he actively works on at most three things per day. He moves those post-it notes over to a slot labeled one, two, three. So he kind of decide like which of these things am I working on today? And then I guess he puts it back. If he finishes it, there's a done column over here. And then there's other room in here to keep track of like meetings and some other stuff as well. So this is pretty cool. Like this is like a highly structured task management system that he could use to keep track of what's going on and its status all in one physical analog visual form.

1:09:56You could do this in a Google Doc or Trello, but I like the way he's doing it here. It's probably nice to actually move these post-its around. There's some more details. I won't get into all of this, but I love the fact that he has my time block planner. So he's time blocking throughout his day. Way to go. And he moves stickies from the planner to Zettelkasten on the wall. Okay, So when he has things to capture, instead of just writing them in the task capture part of the time block planner, he puts it on a post-it note that's stuck on his planner, and then he can move it from the planner to where it should go on his wall, the Zettelkasten wall.

1:10:34So that's pretty cool stuff as well. The more details about capture configure. Oh, I see. So now he's breaking down the details we just looked at. All right. That's pretty cool. There's some other details in here as well. I won't go through all of them, but what I like about this example is it shows the level of intention you can have about how you manage your workload in this current age of like constant digital distractions and context switching. Having some sort of set way of keeping track of what you need to do, what status it is, getting out of your head, making smart decisions about what to do with your time.

1:11:05It really makes a difference. For some people, this level of detail, I think, is really powerful. Other people, you could do something much simpler. But I think that's pretty fun, Jesse, to see the degrees to which people push this. especially with their solopreneurs and they can completely control their day. I like that type of complicated board. I think other people might be a little bit, it adds a lot, but I think it's pretty cool. All right. So thanks for sending that in Ernst. All right. Because we're a little short on time, I think we're going to skip the call this week and move right to our third segment.

1:11:39And we are going to, by popular demand, go back to what we've been doing recently, which is responding to comments from past episodes. So today I'm going to respond to some comments. I have the first one on the screen right now. I'm going to respond to some comments from our recent episode about regaining your ability to think using notepads. All right, so here's what I want to share. A lot of these comments are, let me go to the top one here. A lot of these comments are listeners actually sharing their own practice. I guess we do have room to put my face up here, Jesse. I'm looking at it. I was worried about, yeah, there we go.

1:12:20Now we're on there. A lot of people are sharing their own practices with notebooks, which I thought was interesting. So I want to read a few of these. The first such comment came from Too Much Drive to Thrive, who says, I'm a teacher. I buy all my students a notebook every semester. It costs me about 100 bucks each time. I don't care. There are so many benefits to writing things on paper and reviewing them. I'm not a Luddite either. I have an MA in educational technology. I love that. Spend the money to buy your students' notebooks and it has a big deal. This is someone who studied ed tech. All right, here's another comment from Sluggy Super.

1:12:50Something that got me into using a journal every day was stripping the significance of the time that I write. It's not my special meditation or deep thinking activity. I just write when I have a thought. If I don't have any, I write about the fact that I'm writing my notebook. Sometimes I just write my plan for the evening or sometimes I remember, but making it just another thing I do without thinking made it more enjoyable and therefore something I do daily. That's interesting. If you don't have too many constraints about your notebook, you just like writing in it, you write in it more and it helps your long thinking.

1:13:19All right. Our next comment comes from Anthony Corrigan 5587, who says, I started to do at least a half hour every day of life sketching and gestures in a notebook with a pen. I'm 21 days in without missing a day and it's the best thing I've done in years. It's like daily meditation at this point. A lot of people who don't like actual meditation really do like reflecting in their journal. So I think that's a great example. All right, here's one more listener practice. This came from Almost Goddess, who said, Gee, I've used this method during my PhD as well. Each time I start a new project, I start a new notebook as well.

1:13:53It's very helpful. Thank you for speaking about it so eloquently. I think we have one more best practice here. No, okay, well, let's jump to listener questions. All right. Then we have a couple of comments that are follow-up questions to the episode. The first one comes from Fotar93, who says, what are you specifically doing during these thinking sessions? Asking questions and answering, making lists, free writing. Sometimes I get stuck because I can't deliberately think, but thinking or ideas come up randomly while doing other tasks. The moment I sit down to write think, it is blank up there.

1:14:28Usually when I'm doing specific long thinking with a notebook, there's a particular problem I'm trying to solve. Like what is going to go into this chapter I'm writing? How do I want to organize my argument for this part of the book? Or, hey, this part of my life is, why am I struggling in this part of my life? Like, what's going on here that's not right? And what do I think I need to do to fix it? So I have a particular prompt that I'm trying to answer. And then I write ideas that help me answer that prompt into the notebook as they come up. If I'm doing something else, I'll go get the notebook and write it there.

1:15:03Or I'll go on a walk and be like, let me specifically try to come up with answers about this. ideas that help me figure out this prompt. And then I'll write them in the notebook. If you go on a walk and you're just thinking about that prompt, what's going to happen is your brain is going to load up all the relevant neural networks and you're going to find that you have more insights because now your brain has all the right stuff loaded up and you're more likely to come up with new ideas. So I'm never just randomly writing. There's always like a particular prompt when I have one of these notebooks that I'm trying to solve.

1:15:32That makes a big difference. All right, let's do one more comment question here before we wrap up for the day. This one comes from Lori Lose 74 40. That's the following for this long thinking method notebook is research material allowed or would that then make it deep work? Like should the notebook writing only be for our thoughts? As I'm writing this, the answer is coming. Seems to be a yes. The notebook content is only straight from our minds. Please correct me if I'm wrong. Thank you. And I'm glad I just found your channel, even though I'm trying to get offline more and more, You know, it depends on what you're doing and your preference.

1:16:05Like I want it, if I'm writing an article or something, the notebook would be for like ideas about that article and I would write it in the tool I want to write the article in where it's a word processor, it's Scrivener, it's on my computer. There's other things I do though, where I'm, I'm actually putting in my notebook pretty completed thoughts. So this would be the case for mathematical proofs. I would go for a walk to try to make progress on a proof. I'm going to write out theorems and do the equations right there in the notebook. There's many a case where when I was writing an academic theory paper, I had my notebook open and then I'm copying over equations into it.

1:16:42Like, oh, I did this math over here. Let me copy it in here and clean it up. So it just depends on the type of thinking you're doing and what format it's eventually going to end up in. So don't have rules around it. Whatever is helping you do better long thinking, do that. You don't have to have hard rules about it. all right well there we go jesse that was actually a popular episode um online on youtube i think the idea of of taking a notebook and using it i think it's because people are being so bombarded by ai and smartphone apps they're so distracted yeah that the idea of let me just be my thoughts in paper with no mediation something that 15 years ago like a course is like a really it's like a fresh idea today so that was cool to see all right well there we go so that was another you know uh I'm like a recovering AI critic or something like this.

1:17:32I can't help myself. I encounter these things. I'm working through these thoughts. I like to share them with you. But we've got some other good ideas coming up in the near future that aren't about AI. Make sure you send in your questions, right? You could go right to the deeplife.com slash listen. And there's a form. You can fill it out. If you have ideas for topics or whatever, you can send them to jesse at calnewport.com. He'll take a look. but otherwise I'll be back next week for a new pre Thanksgiving episode or is this episode coming out right before Thanksgiving? Where are we in the year?

1:18:04Yeah, this one's coming up. Oh, so happy Thanksgiving for those who celebrate our next episode will be on the other end of that. So I'm looking forward to that. And until then, as always stay deep.

1:18:17Hi, it's Cal here. One more thing before you go. If you like the deep questions podcast, you will love my email newsletter, which you can sign up for at calnewport.com. Each week I send out a new essay about the theory or practice of living deeply. I've been writing this newsletter since 2007 and over 70 ,000 subscribers get it sent to their inboxes each week. So if you are serious about resisting the forces of distraction and shallowness that afflict our world, You got to sign up for my newsletter at calnewport.com and get some deep wisdom delivered to your inbox each week.

1:19:22Thank you.