Uncapped #19 | Dwarkesh Patel

25:02Real -wave planes, airplanes, steamships, radio, telegraph, light bulbs, combustion. World War I is a crazy example where you start off the war. I think there were the Wright brothers at Flown, but there weren't like many, there were like on the order of hundreds of planes in the world. Two, and there were no tanks. Like tanks was not a thing. And World War I ends with like it's a tank war. It's a plain war. And not that many years. Yes. You go from almost no trucks to tens of thousands of trucks. Wow. It was a course of four years. And planes? Yes. So I think even planes were at that extremely low level before the war.

25:43And the military used during obviously. So I mean, the reason Germany thought it was going to win is because it had this rail network and there was this plan of how you could do this two front war and knock out both France and Russia at the same time. just by, like, you'd be amazing at the real way of logistics. And I think the Bon Mocha, or whoever the leader of the German command was, said at the end of the war, we lost because of trucks, right? Like, we didn't anticipate that, like, there was another way to ship enemy combatants to the front. Yeah, we're just going to see, like, this level of change across so many different sectors.

26:18There was somebody posted on Twitter, VACO, posted something that I thought was an interesting point about you, whether or not it's true, but I'm curious how you react to it, which was, it seems like you went through this evolution where you were learning a ton about AI, and then it seems like you believed that AGI was coming, and then your interest started expanding out into all these other things like geopolitics, biology, all these other areas. I'm guessing the, you know, which was sort of like saying, you know, the technology is what, you know, creates, you know, moment for change, but then this backdrop of the world is what really influences it.

26:52I'm guessing that the chronology of your interests were a little bit different than the framing, but I'm curious just like how you think about that. Given what I do, I'm actually quite pessimistic about being able to learn from other fields. I just know people who will read some philosopher in the 19th century and they think, oh, this explains how Silicon Valley works or this explains AI. No, I think you just have to read the papers about AI. I just think it's very hard to generalize. You're saying to understand the technology, you have to read the papers. Yeah, yeah. I think just people have this idea that like I'll come up with my grand theory of history Yeah, you're like it's not philosophy it's science.

27:27Yes, but I just like in any domain it just very hard to have this like I think there's I know a couple of people who can do this and I find it really impressive But what I noticed about them they're just like not hand -wavy at all. So there are people who like for example If you're trying to model how AI will impact economic growth one way is just to like read the sort of like first -hand accounts of people going through it in the 1500s and like, oh, let's read the biography of the Medici and so forth. And there's other people who are like, okay, let's look at the growth rates going back 10 ,000 years.

27:57What is the long run secular trend? You know, what it actually explains what changed? Well, there's the endogenous growth rate where the key change is that population growth and put the more people come up with more ideas. Okay, well, AI is more people, they'll come up with more ideas, they'll be more specialization. So you just, there's this very different mode of learning from other fields. Yeah, which is empirical and I mean empirical was made the wrong word but just like very falsifiable and grounded versus I'm just gonna read I'm gonna look at a library and just like read a bunch of random books totally So how do your interests connect to each other then like are you following any particular threads or you know There's one connect to another in a certain way like what's what would you say is driving what creates various interests over time?

28:41honestly, it's just a super bespoke, whatever I happen to be interested in that week, if I'm just reading an interesting book. How are you spending your time? Like are you reading a lot? Like is it, do you learn mostly through readings? You're talking to people, are there other methods? Reading, reading. I think there's a couple of people you learn a lot from talking to, in general, I've just sort of been disappointed about like, I mean, given the fact that I'm not sure if you're talking like some of the smartest people though. It's also really interesting, so in some domains, people can be super like, generated outside of domains.

29:09I've sort of been disappointed about the fact that, Look, you might hope that you could talk to some historian about World War One or about the history of oil or something. And then they'd have insights about how this applies to AI. But really, those connections will likely come from you and not from them. When I was in during Daniel Jorgon, was the author of the prize. It's this book about the 200 -year history of oil. One thing I thought was really interesting is that Drake discovers the first oil oil in Pennsylvania. I think in the 1850s. And then the Model T car, I think it's like 19 or five or something that you finally have cars with internal components and engines that people are using to transport all around the world.

29:52And this is a sort of industrial case use of oil. Before that, most of oil was just wasted. It was only the kerosene component. So all of Rockefeller, all of that history that you sort of think about as a gilded age, like oil, bear, and stuff. That's happening when a small fraction of oil is being used just for lighting before the electrical light bulb was invented. And in fact, when the light bulb was invented, I remember getting my day strong on the marketing and the light bulb, but it's like around that area, when the light bulb was invented, people are like, oh, oil is going to go, like standard oil is going to go bust because what's the other use case of oil?

30:28And so I do find it interesting that it took more than 50 years to go from, we have discovered limitless energy in the earth to, here's a way to use billions of gallons of this stuff. And I think it has maybe interesting implications for AI, where we have like, basically, it's sort of shocking how cheap AI is. I think this is why I was finding confusing that people are like optimizing on cost. Like, do I want like two cents for a million tokens, or point two cents for a million tokens? So we have this like commodity that we could potentially use at an industrial scale. And we don't know how to like, part of it's just technological, we don't know how to like get those tokens from you more valuable.

31:04And part of it is just like, yeah, what do we do? Where's the internal combustion engine equivalent for AI? So you said most of your learning comes from what you read rather than, you know, through your conversations with people. Yeah, I'm very lucky that there's maybe six to 12 people who have known for five years, almost all of them have out on the podcast, but who I'm in just extremely regular touch with. I've genuinely learned a lot of what I know from this handful, this group chat. And one sense of it makes people, I know that it's weird that I've known them for five years. And now they're also super successful, but we were all called students at some point.

31:40There's a lot to be said for that, you know, it's that five closest friends thing. It's a big deal. How do you feel about this? Do you learn more from talking to people? I think I learn more from talking to people. I also think that there's different sorts of things that you can be seeking. Seeking the truth versus seeking a good decision are very related, but not exactly the same thing. There's, for example, if you're trying to learn from people about how do you spot great talent? And what can you pick up from somebody? You're not gonna get to the truth. You're just gonna get to techniques and things that have been useful for other people that you try to apply to yourself.

32:14So for that kind of thing, I wouldn't know how to read about it anyway. But on object level stuff, so if you wanna learn about what's happening in robotics, is it? I just, I don't know. I've been sort of underwhelmed by how... Yeah, I mean, the schools of thought to me would be either you can try to go learn it for real yourself. Right. Or if that hill is too high to climb, which, you know, for me getting into robotics and like the white papers way, I'm like, I would be, you know, kidding myself to think I could catch up and then get to the edge of anything. Yeah. And on any time scale that mattered.

32:47And so for me, there's the other move of, is there a way if you're going to do it to shortcut the decision somehow to somebody? And so who can you most trust to give you good information. I think we're also in a lucky position where we have enough public output that we can sort of reach out to people and they'll say yes. This is a tougher position to be in if you're like 19 and I want to like, I want to learn about biology. I feel very lucky because a lot of people do great work in many different domains. Mine just happens to be public facing by default. So I get like more ability to reach out to people than I would like to work in private.

33:25Exactly. Which does create this flywheel where if I do make good content, smart people will be willing to talk to me. Yeah. That helps me make better content. I think that's actually more relevant to why the podcast grows than just audience tells people. Some of you people say that's the more flywheel you're saying? Yeah. But for sort of moving on to a new topic, I wanted to ask about that related to this. You mentioned a bunch of really interesting ideas that are around, you know, oil and history and Stalin and biology. And I'm curious if there's any other just ideas recently that have just really stuck out to you that you can't stop thinking about that it really gripped you Mm -hmm just because I love hearing about these the one that's been on my mind for a long time is I interviewed this geneticist of ancient DNA David Reich and What his lab and his research is revealed is that human history first of all we just didn't realize how much we didn't know about human evolution Just like the story you learned in high school all of it is like at least somewhat false about how when where who what do you mean?

34:22Like, did it happen in Africa? Did it? A big chunk of it didn't. There was a group that went out 400 ,000 years ago and then they mixed in back with the group that left out of Africa 70 ,000 years ago. A lot of the evolution that led to this branch of humanity maybe just didn't even happen in Africa. When did it happen? We were learning more stuff about it. And the key thing we're learning is how did it happen? And it seems like we just see this pattern again and then get in history, which is very disturbing. But like super recurring, is that some small group will figure something out. And it's not clear from the genetic record what it is, right?

35:01Like 70 ,000 years ago, there's this group of one to 10 ,000 people in the Near East. So like where the Middle East, North Africa are right now. They figure something out and they wipe out every single other species of humans across all of Eurasia. Well, like there was half a dozen different species of humans or the hobbits obviously the Neanderthals Hobbits, I forget whether they're like real biological names, but like yeah, I think they're they're called the hobbits

35:34The Denisovans They're all wiped out by this one group like what starts out with like one to 10 ,000 people expand all through 10 ,000 years ago Anatolian farmers also from the modern -day Middle East they expand out kill off like 90 % of of the hunter -gatherers in Europe, through Asia. This also happens again, by the way, with a group that goes to the landbrush to America, they also keep doing this, they're like multiple waves and like one of the waves killed off the remaining ones. The only people who survived, by the way, interestingly are, that we have genetic evidence of is this group in the Amazon, where because the Amazon is so densely, so dense to get through, like the genocide wasn't completed and so it was like more of an intermixing.

36:17Then 5 ,000 years ago, the Yamnaya, which is this group of, like, step nomads, they sweep through all of Eurasia again. Like, and we're talking about like 90 % death rates of the domestic population. So, and not just like multiple continents, like all of Europe, the people who build the Stonehenge are killed off by these people. This is insane, and they're pretty sure this is right. Yes, wow. By the way, the way you learn why it's a genocide, or why it was like violent, is you look at the fraction of maternal versus paternal DNA that comes from the native population versus the invading population.

36:56And what they'll find is that the maternal DNA comes exclusively from the native population. All the paternal DNA comes from the invading population. Wow, which means that the guys will go off. Wow. It also explains a bunch of, so India today is a mix of the original Indus Valley civilization from 35 ,000 years ago, plus the Yamnaya. And so all of India is just like this like gradient basically, like North is more of this Yamnaya ancestry, South is this Indo, Indus Valley Civilization ancestry. So basically just had like a lot of this wrong. Yes, but you know what else is really interesting here?

37:32For hundreds of years, anthropologists, archaeologists have been doing this Indiana Jones thing. We're gonna read the squirrels, and we're gonna think hard about how, you know, what they're trying to say here and read the literature, whatever. And it was just like so useless in comparison to one mathematician who went into this field and he was like, okay, let's just like look at the haplotypes and see how they compare. And just like totally redefined our understanding of basically all of history, going back millions years, even to things that happened 500 years ago, all of that is totally, we can re -understand it.

38:13All these mysteries about like, why did the Mycenaean civilization fall and who exactly were the Mycenaeans and Greece? Just like, all around the world. We have stuff right up to a certain amount of history though, right? Yeah, that's good. At least there's something we can hold on to. But I just think it's a very interesting actually. It's like that, no, it was just like all this sort of esoteric reading and understanding. What was happening in the year 500? Like that we could have that super wrong, for example. Oh, I mean, it's something really interesting we learned speaking of the year 500 is the Roman Empire.

38:42I think it's like, was it 540 or so? Is that there was basically the Black Death kills off like half, close to half of big fractions of the Roman Empire. And this is around when the Roman Empire falls. There were also previous flags, like the Antennaim flag in the second century, third century. I interviewed somebody about this. I didn't realize the extent to which Rome actually has something as bad as a black death and how that contributed to their collapse. Because another bias we have in history is just to think about, oh, we had the four -good emperors and then they did a really good job and then the next guy fucked it up.

39:20No, just like, there's a climate optimum during the quote -unquote four -good emperors where the bread basket was really heavy. And the civilizations happened to fall after a certain amount of time. It's like, no, a thing happened. Yeah, exactly. I often wonder because over the last few hundred years, like we keep like learning a new thing, you know Something was completely wrong whether it was like, you know The earth's round or gravity works this way or whatever else it is And I'm like we must still have big fundamental things wrong You know like human history and that that kind of thing would be like a good example my five -year -old recently Which was a funny question was like um do you believe in Jupiter?

39:53And I was like I do but that's a great question and I there's you know You could ask me about something else in the universe, and I think we have it super wrong. But we must still have big basics wrong. Yeah, I feel like this is what I especially want to talk to physicists. We're just like very basic questions of easy universe infinite. Yeah, we're not. We have no idea. And there's many different kinds of infinities it could be. But just that very, it just seems like so consequential. Or the fact that time warps in your good reference beat, I'm like, we just can't possibly have it all exactly, right?

40:25It's just too crazy, right? It just seemed, or dark matter. There's just enough big stuff that I'm like, I bet we don't have it quite right. But that might sound like some PhD physicists listening and being like, what is the city at Jack talking about right now? Not that we have the answer, but many of them would agree that a lot of stuff has to be. This kind of flows into the next thing I wanted to ask you about, which is your broad perceptions of the way learning is happening. And you've obviously been learning in public. like you do a lot of self -directed sort of education and things like that.

40:57But you're also, you've got one foot sort of very tied to people in academia and at the top of research fields and things like that. And I'm curious sort of your opinion on sort of this transition that's happening. We're like a lot of learning and the way people think that the stuff should get consumed is self -directed and not part of the big institutions and things. The standards people have for like, is this thing true? Do I really believe it? Have just really degraded, especially in sort of podcast land to criticize my own tribe. People will just like, people will just fucking say shit. Whatever you say about academia, there is this idea like, okay, does this make sense?

41:29Have you actually made like a clear argument? Like do you even have a clear like end statement or is it just like the thing that like people are saying? On the other hand, look, I mean, is it like net good for the world? If you read history and you read about the worst things that ever happened, the cultural revolution in China, either great terror and the Soviet Union, you can complain that Twitter has low average IQ that like the conversation is very dumb down. But you just don't need that many IQ points to realize the cultural revolution is bad. You just need some mechanism where you could have gone on and been like, why is Mao having us kill all the Sparrow's?

42:04Like isn't that actually really bad for, you know, like pest control and just like making fun of this defecation? And I think that actually has worked, right? Like, woke was a thing for a second and then I think social media contributed to that being less. And I think also making fun of Trump is like a thing that people do and has worked. And so on that, I just think getting rid of the worst accesses is much more important for making history go well than making sure that we can have these gigabrain, genius level of takes all the time. And I think social media does have a reasonably good job of helping us correct the worst accesses.

42:36On the sort of truth point, I think part of the issue is that the legacy media corporations, in my view, have lost quite a lot of trust from people. Like myself included in many cases where they seem and they've got agendas and it seems like they are of like, for profit and maximizing eyeballs. And so I'm like citizen journalism on Twitter. I'm sorry, I trust that completely. But I also don't trust the institutions completely. So I don't know that one's that much better than the other. That's why I just hear you with this. My attempt to do this thing has, I should give you more respect for the media.

43:12In a couple of ways, one, I think they genuinely are better at holding power to account than independent creators. Like talking to somebody, an extremely powerful politician or business leader and then asking tough questions is a thing that the media will do and often won't happen if they could just get to go on the podcast of their choice. And it's like, it's harder than it looks and they're willing to uphold these standards when they do interviews. No, I mean, I agree that it can often be st .imonious when they do this. Is that to do with the person or the institution? Like as an example, like Tucker Carlson goes Fox News to Indy, same guy theoretically.

43:56Like does that change his truthiness? I mean, I think this is another example of like his show and many others. This is not coming from a place where I'm like making an object of political point. I'm a libertarian. I'm like sort of close enough in embedding space to these people. But the standards of discourse in these new places are just abysmal. They'll just, if you just like, if you had a conversation, you could just like save one of a thousand things. And you just like pause on one of these like, what exactly do you mean here? Why do you think this? And obviously like, oh, I heard a thing in a group chat or whatever.

44:30Whatever you say about the New York Times, they just like genuinely have fact checkers who will go through content. I know that like there's many cases where they failed by their own standards, especially in texturinalism. And I don't like their bias and these kinds of things. But just like, the standards are like an order of magnitude different. I mean, my view is we actually probably, at first with AI, it might have looked like that was going to be like a big problem for the New York Times and maybe in some ways it is. But I would actually, as time's gone on, I think we probably need these institutions more than ever.

45:01In a certain way, which is that, you know, like, AI also now brings, you know, a whole new layer of fun to what's true with deep fakes and random content generation by bots everywhere. And so you kind of at some point do go back to needing somebody to like really hold the standards of like truth as much as possible So I would think that should make these institutions more necessary Yeah, and I think like we always had to compare a against the counterfactual I mean this goes back to the are they making a smarter or dumber? Yes, do they hallucinate? But they are probably I don't know when I talk to an AI this I feel like this is goes about Do you remember like ten years ago people were like oh we can't trust their cabitia because anybody can edit it?

45:36Yeah, I feel like there was always a sia off I think it was like, the reasonably trustworthy always. Well, there's one version. Maybe we have this added to towards AI, or like, oh, hallucinate. Well, there's one version of AI where you're asking it directly. There's another version where somebody who has propaganda and tensions uses it to their advantage. Right. We don't have a lot of time left, and I wanted to get to the last topic. So up until you, all of my guests have been, you know, VCs or founders, and I've been sort of using this as an excuse to kind of publicly learn from them. One of the things I wanted to sort of learn from you was about the way you've done podcasting, because you've done it as well as anybody I've seen.

46:11You were like the one person I reached out to when I was getting started for advice. You gave me really good advice, which was to just basically all centered back to just be authentic, follow your interests. Don't post stuff you're embarrassed to be like all that stuff. That was basically the one North Star that I had. But I'm curious because you've had so much success with it. If you can reflect it all about what's made it work or why has it played out the way that it has so far. It's really hard to say from the inside. I feel extremely lucky that my job is, I get to sit down this morning and decide, what I want to learn about over the next few weeks.

46:45I'll interview the person who's the best in the world at that. I get to pepper them with questions for a few hours and then I get to repeat that week after week. I try to ask the questions that I gently want the answers to, including the questions I want to answer to, after having done two weeks of prepping that field and hopefully having had learned about it over the previous years. And so much content is very much like, give us the intro chapter of your book again, explain this very basic thing in your field. I think people just really appreciate the feeling of being a fly on the wall. Like one of the reasons it's valuable to be St.

47:17San Francisco is that you go to good dinners or events where you will miss a ton of context. Like people will have, know a bunch of things and you won't know what they're talking about, but it sort of raises the bar and immersion learning works. I try to provide that kind of environment in whatever field I'm trying to learn about. And I think people appreciate not being talked down to. Having a sense that the host is actually interested in the questions they're asking. These are like, if they were having a private dinner, I think the dynamic replicate is if you were like at a private dinner party, well, you wouldn't just be deferential.

47:51At a dinner party, you'd be like, you'd hassle them if you disagree with them about something. But there'd be a fun vibe and you wouldn't like, can you explain this concept for everybody else here? You wouldn't have that dynamic. somebody that I spoke to recently who worked closely with Steve Jobs, who I'm actually going to have on the podcast at some point soon, said something that I really liked, which was that one of the things that made Steve Jobs special was that the fundamentals of just operating day to day, the way you talk to people, the way you give feedback, the way you ask questions, was just really good at those fundamentals.

48:21Before we started, I asked you about what have you learned about conversations, because I see that as a big fundamental thing that everybody does that you're very practiced at. But you made a point which is that actually what it is for you is preparation. That's the centerpiece and that's your fundamental. I still think that's highly applicable to everybody because we're all going to interviews as either somebody on the candidate side or the employer where all the people for all sorts of things. But everybody's preparing for stuff. So what is your preparation like? Like, what does that mean for you when you're saying I'm preparing really hard for this?

48:58And it's like, you know, the center of your YouTube. In some sense, it's the very obvious stuff. It's if you're interviewing a researcher and if you'll read the key papers a couple of years ago, before back when I was just like, starting getting into AI and I was about to interview Ilya, I'm like, okay, I'm gonna like program the transformer. This is like how I'll learn about this and then try to talk to me in these researches I could. If I'm interviewing a scholar and feel that, interview them, this person who actually wrote a rebuttal through the PowerWorker, which is this book about how Moses changed in New York City.

49:25That itself, I think, is a 1 ,500 word book or some sort of 500 page book. I read that and I read his rebuttal of that book and I read review articles or whatever, different things about New York construction history. I just do this for all my guests. But in some sense, it's very obvious. You just read the things that could potentially be relevant. And then, right down the question is obviously, the thing I've changed over the last year is I've started using space repetition. So, please, repetition is this like tool where you basically write flashcards or yourself and this software serves them to you every couple of months.

49:58Like you have to be preparing for something well ahead of time. No, this is for consolidating knowledge across interviews. So, if I do an interview, I'm actually going to like retain what I've learned. Because a lot of the concepts connect, I mean, especially with AI, with AI you're just like trying to predict what a future civilization of different kinds of beings will look like. And there's no domain of knowledge, which is not relevant to this question, right? It's obviously technically I stuff as relevant, but history, anthropology, even primatology, what happened between primates and humans, everything is relevant, it'll come up in the interview, and so just having a cast through tools like this is extremely valuable.

50:32So it's like you're retaining a curriculum of all your work. Yeah, it's the kind of thing where if you're reading a book, I think you should have just like shouldn't read. At least if you think you're doing it for like learning, or you should have a very intensive practice around, I'm going to make the cards and I'm going to write through whatever thing is the equivalent to doing practice problems for the Dominion Trenestudy because it's talking to me how often I will make a flashcard for a topic where I'm like, okay, this is like so basic. I don't write this down, but I just have to do something right now.

51:04And in a week later, I'm like, I was on the verge of forgetting it. And you just think about how many books have you read in your life? Hundreds, right? How much have you taken away from them or conversations, whatever other medium? The lack of efficiency here is really striking, and so I've been thinking a lot about how to make this a process where over the coming years I can be like, I'm really getting better over time. I'm not just like doing the next thing. It's sort of the spiritual opposite of the idea of an AI that's always listening and remembering for you so that you don't have to remember any conversation you ever had.

51:32And it's sort of just like something on your person that is constantly listening and you can always go back to it, but everything's captured for you. You're like, it needs to get in my brain so that I can learn the next thing. This is exactly full -circuit from where we started, because people will say in response to the continual learning on the job training stuff, that, oh, we'll just have an external memory system that will be just like a document of things that model has learned. You know, ChatGPT already has this. And I think a lot of cognition is just memory. Like, it has to be on board.

52:01It has to be cash the whole time. There's a great place to end. Thank you so much for doing this. I hope you didn't mind being a guest and keep doing your amazing work. I love to watch you. It was super fun. Thanks for having me on.