Fully autonomous robots are much closer than you think – Sergey Levine

24:33And that's like a 16 -year -old who's had a significant amount of time to learn about the world. He would never even dream of putting a five -year -old in a car and telling him to get started. But if you want somebody to clean the dishes, like dishes can break, too. But you would probably be OK with a child trying to do the dishes without somebody constantly like sitting next to them with a break, so to speak. So for a lot of tasks that we want to do with robotic manipulation, there's potential to make mistakes and correct those mistakes. And when you make a mistake and correct it, well, first you've achieved the task because you've corrected, but you've also gained knowledge that allows you to avoid that mistake in the future.

25:13With driving because of the dynamics of how it's set up, it's very hard to make a mistake, corrected, and then learn from it because the mistakes themselves have significant ramifications. Now, not all manipulation tasks are like that. There are truly some very safety critical stuff. And this is where the next thing comes in, which is common sense. Common sense, meaning the ability to make inferences about what might happen that are reasonable guesses, but that do not require you to experience that mistake and learn from it in advance. That's tremendously important, and that's something that we basically had no idea how to do about five years ago.

25:49But now we can actually use LLMs and VLMs, ask them questions, and they will make reasonable guesses. Like they will not give you expert behavior, but you can say like, hey, there's a sign that's a slippery floor Like what's gonna happen when I walk over that? It's kind of pretty obvious, right? And no autonomous car in 2009 would have been able to answer that question. So common sense plus the ability to make mistakes and correct those mistakes Like that's sounding like off off off off a lot like what a what a person does when they're trying to learn something all of that Doesn't make robotic manipulation easy necessarily, but it allows us to get started with a smaller scope and then grow from there.

26:24So for years using, I mean, not since 2009, but we've had lots of video data, language data, and transformers for five, seven, eight years. And lots of companies have tried to build transformer based robots with lots of training data, including Google, Meta, et cetera. And what is the reason that they've been hitting roadblocks? What has changed now? Yeah, that's a really good question. So I'll start out with maybe a slight modification to your comment is I think they they've made a lot of progress and in some ways a lot of the work that we're doing now at physical intelligence is built on the backs of lots of other great work that was done for example at Google like many of us were actually at Google before we're involved in some of that work some of it is work that we're drawing on that others did so there's definitely like been a lot of progress there but to make robotic foundation models really work it's not that just a laboratory science kind of experiment.

27:26It's also, it also requires kind of industrial scale building effort. It's more like the Apollo program than it is like a science experiment. And the excellent research that was done in the past industrial research labs, and I know I was involved in much of that, was very much framed as a fundamental research effort. And that's good. Like the fundamental research is really important, but it's not enough by itself. You need the fundamental research and you also need the impetus to make it real and make it real means like actually put the robots out there, get data that is represented, the kind of tasks that they want to do in the real world, get that data at scale, build out the systems, get all that stuff right.

28:11And that requires a degree of focus, a singular focus on really nailing the robotic foundation model. for its own sake, not just as a way to do more signs, not just as a way to publish a paper, and not just as a way to kind of like, you know, have a research lab. What is preventing you now from continuing, stealing that data even more? If data is a big bottleneck, why can't you just increase the size of your office 100x, have 100x more operators, where operating these robots until like the more data? Yeah, we might not ramp it up immediately 100x more. Yeah, that's a really good question. So the challenge here is in understanding which axes of scale contributes to which axis of capability.

29:00So if we want to expand capability horizontally, meaning like the robot knows how to do 10 things now and I'd like it to do 100 things later, that can be addressed by just directly horizontally scaling what we already have. But we want to get robots to a level of capability where they can do practical useful things in the real world. And that requires expanding along other axes too. It requires, for example, getting to very high robustness. It requires getting them to perform tasks very efficiently quickly. It requires them to recognize edge cases and respond intelligently. And those things, I think, can also be addressed with scaling.

29:35But we have to identify the right axes for that, which means figure out what kind of data to collect, what settings to collect it in, what kind of methods consume that data, how those methods work. So answering those questions more thoroughly will give us greater clarity on the axes, on those dependent variables, on the things that we need to scale. And we don't fully know, right now, what that will look like. I think we'll figure it out pretty soon, it's something we'll work on actively. But we want to really get that right so that when we do scale it up, it'll directly translate into capabilities that are very relevant to practical use.

30:11Just to give an order of magnitude, how does the amount of data you have collected compared to internet scale pre -training data? And I know it's hard to do like a token by token account because yeah, how does video information compare to internet information, et cetera? But like using your regional estimates, what fraction of? That's right. It's very hard to do because robotic experience consists of time steps that are very corollary with each other. So like the raw like byte representation is enormous, but probably the information density is comparatively low. Maybe a better comparison is to the data sets that are used for multimodal training.

30:48And there, I believe last time we did that count, it was like between one and two orders of magnitude. The vision you have of robotics will not be possible until you collect like what, 100x, 1000x, more data. Well, that's the thing that we don't know that. It's certainly very reasonable to infer that like, you know, robotics is a tough problem and probably it requires, you know, as much experience as the language stuff. But because we don't know the answer to that, to me, a much more useful way to think about it is not how much data do we need to get before we're fully done, but how much data do we need to get before we can get started?

31:27Meaning before we can get a data flywheel that represents a self -sustaining and ever -growing data platforms. What are you talking about standing? This is just like learning on the job or do you have something else in mind? and learning on the job or acquiring data in a way that the process of acquisition of that data itself is useful and valuable. I see, just some kind of RL. Like doing something like actually real. Yeah, I mean, ideally I would like it to be RL because you can get away with the robot acting autonomously which is easier. But that's not out of the question that you can have mixed autonomy.

32:02You can, as I mentioned before, robots can learn from all sorts of other signals. I described how we can have a robot that learns from a person talking to it. So there's a lot of middle ground in between fully teleoperated robots and fully autonomous robots. Yeah. Okay, and how does the Pi model work? Yeah. So the current model that we have basically is a vision language model that has been adapted for motor control. So to give you a little bit of like a fanciful brain analogy, a VLM, a vision language model is basically an LLM that has had a little like pseudo -visual Cortex grafted to it, a vision encoder.

32:39So our models, they have a vision encoder, but they also have an action expert, an action decoder essentially. So it has like a little visual cortex and a notionally a little motor cortex. And the way that the model actually makes decisions is it reads in the sensory information from the robot. It does some internal processing and that could involve actually outputting intermediate steps like you might tell it clean up the kitchen and it might think to itself like, hey, to clean up the kitchen, to pick up the dish and I need to pick up the sponge and I need to put the list on this. And then eventually it works its way through that chain of thought generation down to the action expert, which actually produces continuous actions.

33:12And that has to be a different module because the actions are continuous, they're high frequencies, they have a different data format than text tokens. But structurally, it's still an end -to -end transformer. And roughly speaking, technically, it corresponds to a kind of mixture of expert architecture. And what is actually happening is that it's like, Then it's like there's an image token, then some action tokens, like what it actually ends up doing and then more image, more tax description, more action tokens. Basically, I'm like looking at what stream is going on. That's right. With the exception that the actions are actually not represented as discrete tokens.

33:50It actually uses a flow matching kind of diffusion because they're continuous and you need to be very precise with your actions for extra special. I find it super interesting that so you are, I think you're using the open source Gemma model, which is like Google's LLM, that they're really open source, and then adding this action expert on top. And if I did super interesting that the progress in different areas of AI is just based on the not only the same techniques, but literally the same model, that you can just use an open source LLM and then add this action expert on top. It is notable that like, you naively might think that, oh, there's this separate error of researchers robotics, And there's a separate AR research called LLAMS and natural language processing.

34:33And no, it's like, it's literally the same. It's like the considerations are the same, the architectures are the same, even the weights are the same. I know you do more trading on top of these open source models, but that defines you very interesting. Yeah, so one theme here that I think is important to keep in mind is that the reason that those building blocks are so valuable is because the AI community has gotten a lot better at leveraging prior knowledge. And a lot of what we're getting from the pre -trained LMS and VLMS is prior knowledge about the world. And it's kind of like, it's a little bit abstracted knowledge.

35:07It's like, you know, you can identify objects, you can figure out, like, you know, roughly where things are in image, that sort of thing. But I think if I had to like summarise in one sentence, the big benefit that recent innovations in AI give to robotics is really that prior, the ability to leverage pion knowledge. And I think the fact that the model is the same model that's kind of always been the case in deep learning, but it's that ability to pull in that pion knowledge, that abstract knowledge that can come from many different sources. That's really powerful. Yeah. Today I'm here with Mark, who is a senior researcher at Hudson River Trading.

35:39He has prepared for us a big data set of market prices and historical market data. And we're going to try to figure out what's going on and whether we're going to predict future prices from historical market data. Market, let's dig in. Happy to do it. So it sounds like the first fun thing to do is probably to start looking at what an order book actually looks like. Yeah, I think so. So I've given you like real order book data that is snapshots of the top five levels of the order book both on the bit and ass side for a couple of different text stocks and video, Tesla, AMD, etc. What is the shape of the prediction?

36:14Are we predicting what? You take a data frame, look at its wide values and just kind of like, like, Instagram They are centered at zero, they're roughly centered at zero. But target of what exactly? So these things are changes in the mid price from now to some short period of time in the future. This is actually quite interesting. It's just like a mystery to solve. And each one of these can be like a sizable chunk of time for a researcher. If this sounds interesting to you, you should consider working at Hudson River Trading. Mark, working with people learn more. They can learn more at HudsonDesherating .com slash dorkesh.

36:43Amazing. I was talking to this researcher, Sandr at GDM, and he works on video and audio models. And he made the interesting point that the reason in his view, we aren't seeing that much transfer learning between different modalities, that is to say like training a language model on video and images doesn't seem to necessarily make it that much better at textual questions and tasks, is that images are represented at a different semantic level than text. And so his argument is that text has this high level semantic representation within the model, whereas images and videos are just like compressed pixels.

37:24There's not really a semantic. When they're embedded, they don't represent some like high level semantic information. They're just like compressed pixels. And therefore, there's no transfer learning at the level at which they're going through the model. And obviously, this is super relevant to the work you're doing because your hope is that by trading the model both on the visual data that the robot sees, visual data generally, maybe even in like from YouTube or whatever eventually, plus like language information, plus action information from the robot itself. That would all of this together will like make it generally robust.

38:00And then you had a really interesting blog post about like why video models aren't as robust as language models. Sorry, this is not a super well -formed question. and I just wanted to do a reaction to that. What's up with that? Yeah, yeah. Yeah, so I have maybe two things I can say there. I have some like bad news and some good news. So the bad news is what you're saying is really getting at the core of a long running challenge with video and image generation models. Like in some ways, the idea of getting intelligent systems by predicting video is even older than the idea of getting intelligent systems by predicting text.

38:41But the text stuff turned into use practically useful things earlier than the video stuff. I mean, the video stuff is great. You can generate cool videos. And I think that the work there that's been done recently is amazing. But it's not just generating videos and images has already resulted in systems that have this deep understanding of the world where you can ask them to do stuff beyond just generating more images and videos. Whereas with language clearly it has. I think that this point about representations is really key to it. One way we can think about it is this that if you imagine pointing a camera outside this building, there's the sky, there's the clouds are moving around, the water, cars driving around people.

39:23If you want to predict everything that'll happen in the future, you can do so in many different ways. You can say, okay, there's people around. So let me get really good understanding like the psychology of what people behave in crowds, I'm predict the pedestrians, but you could also say, well, there's clouds moving around. Let me understand everything about water molecules and ice particles in the air. And you could go super deep on that. If you want to fully understand all down to the subatomic level, everything that's going on, as a person, you could spend decades just thinking about that and you'll never even get to the pedestrians or the water, right?

39:52So if you want to really predict everything that's going on in that scene, there's just so much stuff that even if you're doing a really great job and capturing 100 % of something by the time you get to everything else, like, you know, ages will have passed. Whereas with Texas, it's already been abstracted into those bits that we assume it's care about. So the representations are already there, and they're not just good representations, they actually like focus in on what really matters. Okay, so that's the bad news. Here's the good news. The good news is that we don't have to just get everything out of like point in a camera outside this building, because when you have a robot, that robot is actually trying to do a job.

40:28So it has a purpose. Yeah. And its perception is in service to fulfilling that purpose. And that is like a really great focusing factor. We know that for people, this really matters. Like literally what you see is effective at what you're trying to do. It like there's been no shortage of psychology experiments showing that people have like almost a shocking degree of tunnel vision where they will like literally not see things right in front of their eyes. If it's not relevant to what they're trying to achieve. And that is tremendously powerful. Like there must be a reason why people do that, because certainly if you're out in the jungle seeing more is better than seeing less.

41:01So if you have that powerful focusing mechanism, it must be darn important for getting into a cheaper goal. And I think robots will have that focusing mechanism because they're trying to achieve a goal. By the way, the fact that video models aren't as robust is that bearish for robotics because it will... So much of the data you will have to use will not... I guess you're saying a lot of it will be labeled, but ideally you just want to be able to throw everything on YouTube, every video we ever recorded, and have it learn how the physical world works, and how to move about, etc. I just see humans performing tasks and learn from that.

41:37But if, yeah, I guess you're saying it's hard to learn just from that, and it actually needs to practice a task itself. Well, let me put it this way. Let's say that I gave you lots of video tapes or lots of recordings of different sporting events and gave you a year to just watch sports. And then after that year, I told you, okay, now your job, you're going to be playing tennis. Yeah. Okay, that's like pretty dumb, right? Whereas if I told you first, like, you're going to be playing tennis and then I let you study up, right? Like now you really know what you're looking for. Right. So I think that actually, like, there's a very real challenge here.

42:11I don't want to understate the challenge, but I do think that there's also a lot of potential for foundation models that are embodied, that learn from interaction, from controlling robotic systems, to actually be better at absorbing the other data sources because they know what they're trying to do. I don't think that that bite itself is like a silver bullet. I don't think it solves everything. But I think that it does help a lot. And I think that we've already seen the beginning of that, where we can see that including web data in training for robots really does help with generalization. And I actually have the suspicion that in the long run, it'll make it easier to use those sources of data that have been tricky to use up until now.

42:50Famously, I will have all these emerging capabilities that were never engineered in, because somewhere in internet text is the data to train in, to give it the knowledge to do a certain kind of thing. With robots, it seems like you are collecting all the data manually. So there won't be this mysterious new capability that like is somewhere in the data set that you haven't purposefully collected, which seems like it should make you can even harder to then have robust out of distribution kind of capabilities. And so I wonder if the trek over the next five, 10 years will just be like, each subtask, you have to give it thousands of episodes.

43:27And then it's very hard to actually automate much work just by doing subtask. So if you think about what a barista does, what a waiter does, what a chef does, very little bit involved just like sitting at one station and like doing stuff right, like you've got to move around, you've got to restock, you've got to fix the machine, or, et cetera. Go between like the counter and the cashier and the machine, et cetera. So yeah, we'll just be like, we'll just be this long tail of things that you had to keep, skills you had to keep, like adding episodes for manually, they're in labeling and seeing how well they did, et cetera.

44:01Or is there some reason to think that it will progress more generally than that. Yeah, so there's a subtlety here. Emerging capabilities don't just come from the fact that Internet data has a lot of stuff in it. They also come from the fact that generalization, once it reaches a certain level, becomes compositional. There was a cute example that one of my students really like to use in some of his presentations, which is, you know what, International Phonetic AlphaBad is? IPA. So if you look in a dictionary, they'll have the pronunciation of a word and written in like kind of funny letters. That's basically international phonetic alphabet.

44:43So it's an alphabet that is pretty much exclusively used for writing down pronunciations of individual words and dictionaries. And you can ask an LLM to write you a recipe for like making some meal in international phonetic alphabet and it will do it. And that's like like holy crap. Like that is definitely not something that has ever seen because IPA is only ever used for heading down for nowization of individual words. So that's that's compositional generalization. It's putting together things you've seen like that in new ways. And it's like, you know, arguably there's nothing like profoundly new here because like, yes, you've seen different words in that way, but you've figured out that now you can compose the words in this other language the same way that you've composed words in English.

45:23So that's actually where the merchant capabilities come from. And because of this, in principle, if we have a sufficient diversity of behaviors, the model should figure out that those behaviors can be composed in new ways as the situation calls for it. And we've actually seen things, even with our current models, which, you know, I should say that I think they're in the grand scheme of things like looking back five years from now, we'll probably think that these are tiny in scale. But we've already seen what I would call emerging capabilities. When we were playing around with some of our laundry folding policies.

45:54They actually would discover this by accident. The robot accidentally picked up two t -shirts out of the bin instead of one. Starts folding the first one, the other one gets in the way, picks up the other one, throws it back in the bin. And we're like, we didn't know, we didn't know it would do that. Like, holy crap. And then we tried to play around with it and it's like, yep, it does that every time. Like you can drop in, you know, it's doing its work, drop something else on the table, just pick it up, put it back. Right? Okay, that's cool. Shopping bag. It starts putting things in the shopping bag, the shopping bag tip -sover, it picks it back up and stands it upright.

46:21We didn't tell anybody to collect data for that. I'm sure somebody accidentally at some point or maybe intentionally picked up the shopping bag, but it's just, you have this kind of compositionality that emerges when you do learning at scale, and that's really where all these remarkable capabilities come from. Now you put that together with language, you put that together with all sorts of chain -of -thought -reasoning, and there's a lot of potential for them all to compose things in new ways. I didn't example like this when I got a tour of the robots by the way at your office. So it was folding shorts and I don't know if there was an episode like this in the training set, but it was just for fun.

46:57I took one of the shorts and turned it inside out. And then it was able to understand that it first needed to get... So first of all, the grippers are just like this, like two limbs, or just like a possible finger and thumb like thing. And it's actually shocking how much you can do with just that. Yeah, I'd understood that I first needed to fold that inside out before folding it correctly. I mean, what's especially surprising about that is it seems like this model only has like one second of context. So as compared to these language models, which can often like see the entire code base and they're like observing hundreds of thousands of tokens and thinking about them before outputting and they're observing their own chain of thought for thousands of tokens before making a plan about how to code something up.

47:43Your model is seeing one image, like what happened in the last second, and it vaguely knows it's supposed to fold this short, and it's seeing the image of what's happened in the last second. And I guess it works. It's crazy that it'll know, you will just see the last thing that happened, and then keep executing on the plan, so fold it inside out, then fold it correctly. But it's shocking that a second of context text is enough to execute on a minute long task. Yeah, I'm curious why you made that choice in the first place and why it's possible to actually do tasks. They have a human could only think, it had like a second of memory.

48:19It had to do physical work. I feel like that would just be possible. I mean, it's not that there's something good about having less memory to be clear. Like I think that adding memory, adding longer context, so that stuff, adding higher resolution images, I think those things will make them all better. But the reason why it's not the most important thing for the kind of skills that you saw when you visit it does, it at some level, I think it comes back to more of X paradox. So more of X paradox is basically, it's like, you know, if you want to know one thing about robotics, it's like that's the thing.

48:53More of X paradox says that basically in AI, the easy things are hard and the hard things are easy, meaning like the things that we take for granted, like picking up objects, you know, perceiving the world, all that stuff. Those are all the heart problems in AI and the things that we find challenging, like playing chess and doing calculus, actually are often the easier problems. And I think this memory stuff is actually more of experiments in disguise, where we think that the cognitively demanding tasks that we do, that we find hard, that kind of causes to think like, oh man, I'm sweating, I'm working so hard, those are the ones that require us to keep lots of stuff in memory, lots of stuff in our minds.

49:27like if you're solving some big math problem, if you're having a complicated technical conversation on a podcast, like those are things we have to keep all those pieces, all those puzzle pieces in your head. If you're doing a well rehearsed task, if you are an Olympic swimmer and you're swimming with perfect form and you're like right there in the zone, like people even say like it's in the moment. It's in the moment, right? Like it's like you've practiced that so much, you've baked it into your neural network and your brain, that you don't have to think carefully about keeping all that context, right?

50:01So it really is just more of a, more of a paradox manifest thing itself, but that doesn't mean that we don't need the memory. It just means that if we want to match the level of dexterity and physical proficiency that people have, there's other things we should get right first, and then gradually go up that stack into the more cognitively demanding areas, into reasoning, into context, into planning, all that kind of stuff. And that stuff will be important too. And how physically will, so you have this like trilema, you have three different things which all take more compute during inference that you want to increase at the same time.

50:37You have the inference speed and so humans are processing 24 frames a second or whatever it is. They were just like, we can react to things extremely fast. Then you have the context length and for I think the kind of robot which is just like cleaning up your house I think it has to kind, it has to be aware of things that happened minutes ago or hours ago and how that influences its plan about the next task that's doing. And then you have the model size, and I guess at least with that lens we've seen that there's gains from increasing the amount of parameters. And I think currently you have 100 millisecond inference speeds, you have a second long context, and then the model is what, a couple of billion parameters, how many?

51:20Okay. And so each of these at least two of them are many orders of magnitude Smaller than what seems to be the human equivalent right like the model if a human brain has like trillions of parameters And this has like two billion parameters and then if humans are processing at least as fastest model like Actually a decent bit faster and we have hours of context Depends on how you define human context, but hours of context minutes of context sometimes decades of context. Yeah exactly So you have to have many order of magnitude improvements across all of these three things which seem to oppose each other or like increasing one reduces the amount of reduces the amount of compute you can dedicate towards the other one in inference.

52:03So how are we gonna yeah, how are we gonna solve this? Yeah, well, that's a very big question. Yeah, let's let's try to unpack this a little bit. I think there's a lot going on in there. One would say is a really interesting technical problem. And I think that it's something where we'll see perhaps a lot of really interesting innovation over the next few years is the question of representation for context. So if you imagine the like some examples you gave, like if you have a homeworld about that's doing something and you used to keep track. As a person, there's certainly some things where you keep track of them very symbolically, like almost in language, like, you know, I have my checklist.

52:45I'm going shopping and at least for me, I can literally visualize in my mind my checklist, pick up the yogurt, pick up the milk, pick up whatever. I'm not picturing the milk shelf with the milk sitting there, I'm just thinking milk. But then there's other things that are much more spatial, almost visual. When I was trying to get to your studio, I was thinking, okay, what this street looks like, Here's what that street looks like. Here's what I expect the door right to look like. So representing your context in the right form, that captures what you really need to achieve your goal, and otherwise kind of discards all the unnecessary stuff.

53:25I think that's a really important thing. I think we're seeing the beginnings of that with multimodal models, but I think that multimodality has so much more to it than just like ImagePlus text. I think that that's a place where there's a lot of room for really exciting innovation. Do you mean in terms of how we represent? Yeah, how we represent both contexts, both what happened in the past and also plans or reasoning, as you can call it in all -in -world, which is what we would like to happen in the future or intermediate processing stages in solving a task. I think doing that in a variety of modalities, including potentially learn modalities that are suitable for the job, is something that has a, I think, enormous potential to overcome some of these challenges.

54:05Interesting. Another question I have is we're discussing these tough trade -offs in terms of inference is comparing it to the human brain and figuring out the human brain is able to have hours, decades of context while being able to act on the order of 10 milliseconds while having 100 trillion parameters or however you want to count it. And I wonder if the best way to understand what's happening here is that human brain hardware is just way more advanced than the hardware we have in GPUs or that the algorithms for encoding video information are like way more efficient and maybe it's like some crazy mixture of experts where the active parameters is also on the order of billions, a low billions or some mixture of the two, basically, if you had to think about why do we have these models that are across many dimensions orders of magnitude less efficient, is it harder or algorithms than the brain?

55:12Yeah, that's a really good question. So I definitely don't know the answer to this. I am not by any means well versed in neuroscience, but if I had to guess and also provide an answer that leans more on things I know, it's something like this. that the brain is extremely parallel. It kind of has to be just out of, just because of the biophysics. But it's even more parallel than your GPU. If you think about how a modern, multimodal language model processes the input, if you give it some images and some text, like first it reads in the images, then it reads in the text and then proceeds one token at a time to generate the output.

55:53It makes a lot more sense to me for an embodied system to have parallel processes. Now mathematically, you can actually make close equivalences between parallel and sequential stuff. Like transformers aren't actually fundamentally sequential. Like you kind of make them sequential about putting in position embeddings. Transformers are fundamentally actually very parallelizable things. That's what makes them so great. So I don't think that actually mathematically, this like highly parallel thing where you're doing perception and proprioception and planning all at the same time is actually necessarily used to look at that different form of transfer.

56:24although it's practical implementation will be different. And you could imagine that the system will in parallel think about, okay, here's like my long -term memory, like here's what I've seen a decade ago. Here's my short -term kind of spatial stuff. Here's my semantic stuff. Here's what I'm seeing now. Here's what I'm planning. And all of that can be implemented in a way that there's some very familiar kind of attentional mechanism, but in practice, all running in parallel, maybe at different rates, maybe with a more complex things, running slower, the faster reactive stuff running faster. I'm sure you've been seeing a bunch of fun images that people have been generating with Google's new image generation model, NanoPanana.

57:00My X -Feed is full of wild images. But you might not realize that this model can also help you do less flashy tasks like restoring historical pictures or even just cleaning up images. For example, I was reading this old paper back because I was prepping to interview Sarah Payne and it had this really great graph of world or two allied shipping that I wanted to overlay in the lecture. Now in the past, this would have taken one of my editors 20 or 30 minutes to digitize and clean up manually. But now, we just took a photo of the page and then dropped into NanoBanana and got back a clean version.

57:31This was a one shot. But if NanoBanana doesn't nail it on the first attempt, you can try to just go back and forth with it until you get a result that you're super happy with. We keep finding new use cases for this model, and honestly, this is one of those tools that just doesn't feel real. Check out Gemini 2 .5 Flash Image Model, aka NanoBanana on both Google AI Studio and the Gemini app. All right, back to Sergei. Even five years, we have a system which is like, as robust as a human in terms of interacting with the world, then what has happened that makes it physically possible to be able to run those kinds of models, to have video information that is streaming at real time, or hours of prior video information is somehow being encoded and considered while decoding in like a millisecond scale, and with many more parameters, is it just that like Nvidia has shipped much better GPUs or that you guys have come up with much better like encoders and stuff or like what's happened in the five years?

58:29I think there are a lot of things to this question. I think certainly there's like a really fascinating systems problem. And by no means a systems expert, but I would imagine that the right architecture in practice, especially if you want an affordable low cost system would be to externalize at least part of the thinking. You know, you could imagine maybe in the future you'll have a robot that has like, you know, if your internet connection is not the very good, the robot is in kind of like a dumb or reactive mode, but if you have a good internet connection, then it can be a little smarter. It's pretty cool.

58:59But I think there are also research and algorithms things that can help here. Like figuring out the right representations can precisely representing both your past observations, but also changes in observation, right? Like, you know, your sensory stream is extremely temporarily correlated, which means that the marginal information gain from each additional observation is not the same as the entirety of that observation. Because the image that I'm seeing now is very correlated to the image I saw before. So in principle, I want to represent it concisely. I can get away with a much more compressed representation than if I represent the images independently.

59:31So there's a lot that can be done on the algorithms side to get this right and that's really interesting algorithms work. I think there's also like a really fascinating systems problem. To be truthful, like, I haven't gotten to the systems problem because you want to implement the system once you But I think there's a lot of stuff to do there. Yeah, maybe you guys just need to hire like the people who run the YouTube data centers because like they know how to encode video information. Okay, this is actually an interesting question, which is that with LLM's of course, they're being theoretically you could run your own model on the slap -pup or whatever, but realistically what happens is that the largest most effective models are being run in batches of thousands and millions of users at the same time, not locally.

1:00:16Well, the same thing happened in robotics because of the inherent efficiencies of batching plus the fact that we have to do this incredibly compute intensive inference task. And so you don't want to be carrying around like $50 ,000 GPUs per robot or something. You just want that to happen somewhere else. So yeah, this robotics world should be just be anticipating something where you need connectivity everywhere, you need robots that are like, have like super fast, and you're streaming video information back and forth, right? Or at least video information one way. So, does that have interesting implications about like how this deployment of robots will actually be instantiated?

1:00:59I don't know, but if I were to guess, I would guess that it will actually see both. that will see low cost systems with off -board inference and more reliable systems, for example, in settings where like if you have an outdoor robot or something where you can't rely on connectivity that are costlier and have on -board inference. A few things I'll say from a technical standpoint that might contribute to understanding this. While a real -time system obviously needs to be controlled in real -time, often at high frequency, The amount of thinking you actually need to do for every time step might be surprisingly low.

1:01:38Again, we see this in humans and animals. When we plan out movements, there is definitely a real planning process that happens in the brain. If you record from a monkey brain, you will actually find neural correlates of planning. There is something that happens in advance of a movement. and when that movement actually takes place, the shape of the movement correlates without having before the movement. Like that's planning, right? So that means that you put something in place and set the initial conditions of some kind of process and then on roll depth process and that's the movement. And that means that during that movement you're doing less processing and you kind of batch it up in advance.

1:02:18But you're not entirely an open move. It's not like playing your playing back a tape recorder. You are actually reacting as you go. You're just reacting at a different level of abstraction, a more basic level of abstraction. And again, this comes back to representations. Figure out which representations are sufficient for kind of planning and advance and then enrolling, which representations require a tight feedback loop. And for that tight feedback loop, what is it, where are you doing feedback on? Like, if I'm driving a vehicle, maybe I'm doing feedback on the position of a lane marker, so that I stay straight, and then I lower frequency, I sort of gauge where I am in traffic.

1:02:48And then so you have a couple lectures from a few years back, where you say like even for robotics, like RL is, in many cases, better than imitation learning. But so far, the models are exclusively doing imitation learning. So I'm curious how you're thinking on this has changed or maybe it's not changed, but then you need to do this for the RL. Like, why can't you do RL yet? So the key here is prior knowledge. Yeah. So in order to effectively learn from your own experience, it turns out that it's really, really important to already know something about what you're doing. Otherwise, it takes far too long.

1:03:21It's just like it takes a person, when they're a child, a very long time to learn very basic things, to learn to write for the first time, for example. Once you already have some knowledge, then you can learn new things very quickly. So the purpose of training the models with supervised learning now is to build out that foundation that provides the prior knowledge so they can figure things out much more quickly later. And again, this is not a new idea. This is exactly what we've seen with LMS, right? LM started off being trained purely with next token prediction, and that provided an excellent starting point first for all sorts of synthetic data generation, and then for RL.

1:03:58So I think it makes total sense that we would expect basically any foundation model effort to follow the same trajectory, we first build out the foundation, essentially in like a somewhat brute force way. And the stronger that foundation gets, the easier it is to then make it even better with much more accessible training. In 10 years, will the best model for knowledge work also be a robotics model or have like an action expert attached to it? And the reason I ask you is like, so far we've seen advantages from using more general models for things. And will robotics fall into this bucket of we will just have D model, which does everything including physical work and knowledge work?

1:04:36Or do you think they'll continue to stay separate? I really hope that they will actually be the same. And obviously, I'm extremely biased. I love robotics. I think it's very fundamental to AI. But I think that it's optimistically that it's actually the other way around that the robotics element of the equation will make all the other stuff better. And there are two reasons for this that I could tell you about. One has to do with representations and focus. So what I said before with video prediction models if you just want to predict everything that happens It's very hard to figure out what's relevant.

1:05:13Yeah, if you have the focus that comes from actually trying to do a task Now that acts to structure how you see the world in a way that allows you to more fruitfully utilize the other signals that could be extremely powerful. Yeah The second one is that Understanding the physical world at a very deep fundamental level at a level that goes beyond just what we can articulate with language which can actually help you solve other problems. And we see, we experience this all the time. Like when we talk about abstract concepts, we say like this company has a lot of momentum, right? I like, we'll use like social metaphors to describe inanimate objects.

1:05:49Like my computer hates me, right? Like we experience the world in a particular way and our subjective experience shapes how we think about in very profound ways. And then we use that as a hammer to basically hit all sorts of other nails that are far too abstract to handle any other way. I guess there might be other considerations that are relevant to physical robots in terms of like, inference speed and model size, et cetera, which might be different than the considerations for knowledge work. But then maybe that doesn't change. Maybe it's still the same model, but then you can serve it in different ways.

1:06:20And the advantages of code training are high enough that, yeah, whenever I'm wondering in five years if I'm using a model to code for me, Does it also know how to do robotic stuff? And maybe the advantages of code trading on a robot is too high enough that it's worth. Well, and I should say that the coding is probably like the pinnacle of an abstract knowledge work in the sense that just by the mathematical nature of computer programming is an extremely abstract activity, which is why people struggle with it so much. I'm a bit confused about why simulation doesn't work better for robots. If I look at humans, smart humans do a good job of if they're intentionally trying to learn noticing what about the simulation is similar to real life and being in attention to that and learning from that.

1:07:07So if you have pilots who are learning in simulation or F1 drivers who are learning in simulation, should it be expected to be a case that as robots get smarter, they will also be able to learn more things or simulation or is this course and we need real world data forever. This is a very subtle question. Your example with the airplane pilot using simulation is really interesting. But something to remember is that when a pilot is using a simulator learned to fly an airplane, they're extremely goal -directed. So their goal in life is not to learn to use a simulator. Their goal in life is to learn to fly the airplane.

1:07:41They know there will be a test afterwards and they know that eventually they'll be in charge of like a few hundred passengers. And they really need to not crash that thing. And when we train models on data from multiple different domains, the models don't know that they're supposed to solve a particular task. They just see like, hey, here's one thing I need to master, here's another thing I need to master. So maybe like a better analogy there is if you're like playing a video game where you can fly an airplane and then eventually someone puts you in the cockpit of a real one. Like, it's not that the video game is useless, but it's not the same thing.

1:08:13And if you're trying to play that video game and your goals to really master the video game, you're not gonna go about it in quite the same way. Oh, isn't, can you do some kind of meta -RL on this, which is almost identical actually to the, there's this really interesting paper you wrote in 2017, where maybe the loss function is not how well it is at a particular video game or particular simulation, but how well being trained at different video games makes it better at some other downstream task. I did a terrible job explaining, but I... I didn't mean to. Yeah, maybe we can do a better job explaining what I...

1:08:45Yeah, I was trying to say. So I think what you're trying to say is basically that, well, maybe if we have a really smart model that's doing metal learning, perhaps it can figure out that it's performance on a downstream problem, a real world problem, is increased by doing something in a simulator. And then specifically make that the loss function, right? That's right. But here's the thing with this. There's a set of these ideas that are all going to be like something like train to make it better on the real thing by leveraging something else. And the key linchpin for all of that is the ability to train to be better on the real thing.

1:09:18The thing is, I actually suspect in reality, we might not even do something quite so explicit, because metal learning is emergent, as you pointed out before, right? like LLM's essentially do a kind of metal learning in context learning. I mean, we can debate us to how much that's learning or not, but the point is that large powerful models trained on the right objective on real data get much better at leveraging all the other stuff. And I think that's actually the key and coming back to your airplane pilot. Like the airplane pilot is trained on a real world objective. Like their objectives to be good airplane pilot, to be successful, to have a good career, and all of that kind of propagates back into the actions they take in leveraging all these other data sources.

1:09:56So what I think is actually the key here to leverage your auxiliary data sources, including simulation, is to build the right foundation model that is really good that has those emergent abilities. And to your point, to get really good like that, it has to have the right objective. Now, we know how to get the right objective out of real world data. Maybe we can get out of other things, but that's harder right now. And I think that, again, we can look to the examples of what happened in other fields. like these days, if someone trains an LLM for solving complex problems, they're using lots of synthetic data.

1:10:30But the reason they're able to leverage that synthetic data effectively is because they have the starting point that is trained on lots of real data that kind of gets it. Once it gets it, then it's more able to leverage all this other stuff. So I think perhaps ironic that the key to leveraging other data sources, including simulation, is to get really good at using real data, understand what's up with the world, and then now you can fruitfully use all this. So once we have this like in 2035, 2030, basically the sci -fi world, are you optimistic about the ability to like true AGIs to build simulations in which they are rehearsing skills that no human or AI has ever had a chance to practice before?

1:11:10So they need to like practice via astronauts because we're building the Dyson Sphere and they can just do that in simulation or like will the issue with simulation continue to be one regardless of how smart models get. So here's what I would say that deep down at a very fundamental level, the synthetic experience that you create yourself doesn't allow you to learn more about the world. It allows you to rehearse things, it allows you to consider counterfactuals, but somehow information about the world needs to get injected into the system. So, and I think the way you pose this question actually elucidates this very nicely because in robotics classically, people have often thought about simulation as a way to inject human knowledge because a person knows how to write down like differential equations, they can code it up and that like gives the robot more knowledge than had before.

1:11:59But I think that increasingly what we're learning from experiences in other fields, from how like the video generation stuff goes, from synthetic data for LLM's is that actually probably the most powerful way to create synthetic experience is from a really good model, because the model probably knows more than a person does about those fine grain details. But then of course, where does that model get the knowledge from experiencing the world? So in a sense, what you said, I think is actually quite right in that a very powerful AI system can simulate a lot of stuff, but also at that point, it kind of almost doesn't matter because viewed as a black box, what's going on with that system is that information comes in and capability comes out.

1:12:37and whether the way it process that information is by imagining some stuff and simulating or by some model -free method It's kind of irrelevant in understanding it's capability. It's just what the equivalent is in humans like whatever we're doing when we're daydreaming or sleeping or I don't know if you have some sense of like what this auxiliary thing we're doing is but If you had to make an ML analogy for it. What is it? Well, yeah, I mean certainly when And when you sleep, your brain does stuff that looks an awful lot like what it does when it's awake, that looks an awful lot like playing back experience or perhaps generating new statistically similar experience.

1:13:15And so I think it's like, it's very reasonable to guess that perhaps simulation through a learned model is like part of how your brain figures out like counterfactuals basically. But something that's kind of even more fundamental than that is that optimal decision -making at its core, regardless of how you do it, requires considering counterfactuals. You basically have to ask yourself, if I did this instead of that, would it be better? And you have to answer that question somehow. And whether you answer that question by using a learn simulator, or whether you answer that question by using a value function, or something like that, by using a reward model, in the end it's kind of all the same.

1:13:53Like as long as you have some mechanism for considering counterfactuals and figuring out which counterfactuals better, you've got it. Yeah. So, I like thinking about it this way because it kind of simplifies things that tells us that the key is not necessarily to do really good simulation. The key is to figure out how to answer counterfactuals. Yeah, interesting. So, a stepping big picture again, the reason I'm interested in getting concrete understanding of when this robot economy will be deployed is because it's actually pretty relevant to understanding how fast AGI will proceed in the sense that, well, it's obviously the data flywheel, but also if you were just extrapolate out the CAPEX for AI, suppose was by 2030, people have different estimates, but many people have estimates in the hundreds of gigawatts, 100, 200, 300 gigawatts.

1:14:38And then you can just crunch numbers on, like, if you have 200 gigawatts deployed or 100 gigawatts deployed by 2030, the marginal capExp for years, like trillions of dollars. It's like two, three, four trillion dollars a year. And that corresponds to actual data centers you have to build, actual chip foundries you out of build, actual solar panel factories you got to build. And I am very curious about whether by this time, by 2030, if the big bottleneck we have is just like people to lay out the solar panels in the data center or assemble the data center, whether the robot economy will be mature enough to help significantly in that process.

1:15:22That's cool. So you're basically saying, like, how much concrete should I buy now to build a data center? So that by 2030, I can power all the robots. Yeah, yeah. That is a more ambitious way of thinking about it than has occurred to me. But it's a cool question. I mean, the good thing, of course, is that the robots can help you build that stuff. Right. But will they be able to buy the time that, like, there's some fly, like, there's the non -robotic stuff, which will also, like, mandate a lot of cat -backs. And then there's a robot stuff, well, you actually had a bunch of robot factories, etc.

1:15:51But every just excuse, there will be this industrial explosion across the whole stack and how much will robotics be able to speed that up or make it possible? I mean, in principle, quite a lot, right? I think that we have a tendency sometimes to think about robots as like mechanical people, but that's not the case, right? Like people are people and robots are robots. Like the better analogy for the robot is it's like your car or a bulldozer. Like it has much lower maintenance requirements, you can put them into all sorts of weird places and they don't have to look like people at all. You can make a robot that's 100 feet tall, you can make a robot that's tiny.

1:16:29So I think that if you have the intelligence to power very heterogeneous robotic systems, you can probably actually do a lot better than just having like mechanical people in effect. And it can be a big productivity boost for the real people and it can allow you to solve problems that are very difficult to solve now. For example, I'm not an expert on data centers by any means, but you could build your data centers in a very remote location because the robots don't have to worry about whether there's a shopping center nearby. And then do you have a sense of how, so there's like, where will the software be?

1:17:03And then there's a question of how many physical robots will we have? So like, how many of the kinds of robots you're trading in physical intelligence, like these tabletop arms, are there physically in the world? How many will there be by 2030? How many will be needed? I mean, these are tough questions. Like how many will be needed for that? There's a very tough question. And also, you know, economies of scale in robotics so far have not functioned the same way that they probably would in the long term, right? Just to give you an example, when I started working in robotics in 2014, I used a very nice research robot called the PR2 that cost $400 ,000 to purchase.

1:17:42When I started my research lab at UC Berkeley, I bought robot arms that were $30 ,000. The kind of robots that we are using now at physical intelligence, each arm costs about $3 ,000 and we think they can be made for a small fraction of that. So these things - What is the cause of that learning rate? Well, there are a few things. So one, of course, has to do with economies of scales. So custom build, high -end research hardware, of course, is going to be much more expensive of then kind of more productionized hardware. But the other, and then of course there's a technological element that as we get better at building actuated machines, they become cheaper.

1:18:22But there's also a software element, which is the smarter URI system gets, the less you need the hardware to satisfy certain requirements. So traditional robots and factories, they need to make motions that are highly repeatable and therefore it requires a degree of precision and robustness that you don't need if you can use cheap visual feedback. So AI also makes robots more affordable and lowers the requirements on the hardware. Interesting. Okay, so, learning will continue. Do you think it will cost hundreds of dollars by the end of the decade to buy mobile arms? That is a great question for my co -founder, Adnan Esmail, who is probably like the best person arguably in the world to ask that question of.

1:19:05But certainly the drop in cost that I've seen has surprised me year after year. Okay. And how many arms are there probably in the world? Is it more than a million, less than a million? So I don't know the answer to that question, but it's also a tricky question to answer because not all arms are made equal. Like, arguably the kind of robots that are like assembling cars in a factory are just not the right kind to think about. So the kind you want to train on. Very few because they are not currently commercially deployed unlike the factory. Just like a less than a hundred thousand. I don't know, but probably.

1:19:38Okay. And we want billions of robots, like at least millions of robots. If you're just thinking about like the industrial explosion that you need to have the AI explicit growth, not only do you need the arms, but then you need like something that can move around. Basically, I'm just trying to think about like, will that be possible by the time that that you need a lot more labor to power this AI boom. Well, you know, economies are very good at filling demand when there's a lot of demand, right? Like how many iPhones were in the world in 2001, right? That's right. So I think it's definitely a challenge there.

1:20:20And I think it's something that is worth thinking about. And a particularly important question for researchers like myself is how can AI affect how we think about hardware? Because there are some things that I think are going to be really, really important. Like you probably want your thing to not break all the time. There are some things that are firmly in that category of question marks. How many fingers do we need? Like you said yourself before that you were surprised that a robot with two fingers can do a lot. Maybe you still want more than that, but still finding the bare minimum that still lets you have good functionality that's important.

1:20:54That's in the question mark box. And there are some things that I think we probably don't need. We probably don't need the robot to be super duper precise because we know that feedback that can compensate for that. So I think my job as I see it right now is to figure out what's sort of the minimal package we can get away with. And I really like to think about robots in terms of minimal package because I don't think that we will have the one ultimate robot, like sort of the mechanical person basically. I think what we will have is a bunch of things that good effective robots need to satisfy, just like good smartphones need to have a touchscreen, like that's something that we all kind of agreed on.

1:21:29And then a bunch of other stuff that's kind of optional depending on the need depending on the cost point, et cetera. And I think there will be a lot of innovation where once we have very capable AI systems that can be plugged into any robot to endow it with some basic level of intelligence, then lots of different people can innovate on how to get the robot hardware to be optimal for each niche. And there's the manufacturers, is there some Nvidia robotics? Not right now, maybe there will be someday. I would really like, maybe I'm being idealistic, but I would really like to see a world where there's a lot of heterogeneity in robots.

1:22:03What is the biggest part of making the hardware today as somebody who's designing the algorithms that run on it? It's a tough question to answer mainly because things are changing so fast. I think that to me, the things that I spend a significant amount of time thinking about on the hardware side is really more like reliability and cost. It's not that I'm that like that worried about costs is just that cost translates to number of robots, which translates to amount of data. And being an ML person, I really like having lots of data. So I really like having robots that are low cost because then I can have more of them and therefore more data and Reliabilities and more or less for the same reason.

1:22:35Yeah But I think it's something that we'll get more clarity on as things progress because as we Basically the AI systems of today are not pushing the hardware to the limit So as the AI systems get better and better the harder will get pushed to the limit and then we'll hopefully have a much better answer to your question Okay, so this is a question I've had for a lot of guests and is that If you go through any layer of the CLI explosion, you find that a bunch of the actual source supply chain is being manufactured in China. So other than chips obviously. But then if you talk about data centers and you're like, oh, all the way first for solar panels and a bunch of the cells and modules etc, are manufactured in China, and then you just go through the supply chain.

1:23:25And then obviously robot arms are being manufactured in China. And so if you live in this world where the hardware is just incredibly valuable to ramp up manufacturing of because each robot can produce some fraction of the value that a human worker can produce. And not only is that true, but the value of human workers or any kind of worker has just tremendously skyrocketed because we just need tons of bodies to lay out the tens of thousands of solar far acres of solar farms and data centers and foundries and everything. In this boom world, the big bottleneck, they're just like, how many robots can you physically deploy?

1:24:07How many can you manufacture? Because you guys are going to come up with the algorithms, now we just need the hardware. And so this is a question I've asked many guests, which is that like if you look at the part of the chain that you are observing, what is the reason the China does doesn't win by default, right? If they're producing all the robots and you come up with the algorithms that make those robots super valuable, why don't they just win by default? Yeah, so this is a very complex question. I'll start with the broader themes and then try to drill a little bit into the details. So one One broader theme here is that if you want to have an economy where you get ahead by having a highly educated workforce, by having people that have high productivity, meaning that for each person's hour of work, lots of stuff gets done, automation is really, really good because automation is what multiplies the amount of productivity that each person has.

1:25:10Again, same as like LM coding tools. So LM coding tools amplify the productivity of a software engineer. Robots will amplify the productivity of basically everybody that is doing work. Now that's kind of like a final state, like a desirable final state. Now there's a lot of complexity in how you get to that state, how you make that an appealing journey to society, how you navigate the geopolitical dimension of that. that all of that stuff is actually pretty complicated and it requires making a number of really good decisions, like good decisions about investing in a balanced robotics ecosystem, supporting both software innovation and hard renovation.

1:25:54I don't think any of those are insurmountable problems. It just requires a degree of kind of long term vision and the right kind of balance of investment. But what makes me really optimistic about this is that final state, that if I think we can all agree that in the United States we would like to have the kind of society where people are highly productive, where we have highly educated people doing high value work. And because that end state seems to me very compatible with automation, with robotics, there's a lot of, at some level there should be a lot of incentive to get to that state. And then from there, we have to like solve for like all the details that will help us get there.

1:26:36And that's not easy. Like I think there's a lot of like complicated decisions that need to be made in terms of private industry, in terms of investment, in terms of the political dimension. But I'm very optimistic about it because it's like, it seems to me like the light at the end of the tunnel is kind of in the right direction. I mean, I guess there's a different question which is that if the value is sort of bottlenecked by hardware And so you just need to produce more hardware. What is the path by which hundreds of millions of robots or billions of robots are being manufactured in the US or with allies?

1:27:09I don't know how to approach that question, but it seems like a different question than like, okay, well, what is the impact on like human wages or something? So, again, for the specifics of how we make that happen, I think that's a very long conversation that I'm probably not the most qualified to speak to, but I think that in terms of the ingredients, The ingredient here that I think is important is that robots help with physical things, physical work. And if producing robots is itself physical work, then getting really good at robotics should help with that. It's a little circular, of course.

1:27:44And as with all circular things, you have to like kind of bootstrap it and try to get that engine going. But it seems like it is an easier problem to address than, for example, the problem of digital devices where work goes into creating, you know, computers, phones, etc. But the computers and phones don't themselves help with the work. Right. I guess feedback loops go both ways. They can help you or they can help others. And it's a positive sum world. So it's not necessarily bad if they help others. But to the extent that a lot of the things which go into this feedback the subcomponent manufacturing and supply chain already exist in China.

1:28:23It seems like the stronger feedback loop would exist in China. And then there's a separate discussion like maybe that's fine. Maybe that's good and maybe they'll continue exporting this to us. But it's just like notable that I just find it notable that whenever I talk to some guests about different things, it's just like, oh yeah, that, you know, within a few years, the key bottleneck to every single part of the supply chain here will be something that China is like the 80 % world supplier of something. Well, yeah, and this is why I said before that I think something really important to get right here is a balanced robotic ecosystem.

1:28:55Right? Like I think AI is tremendously exciting, but I think we should also recognize that getting AI right is not the only thing that we need to do. And we need to think about how to balance our priorities, our investment, the kind of things that we spend our time on. Just as an example, at physical intelligence, we do take hardware very seriously, actually. We build a lot of our own things, and we want to have a hardware roadmap alongside our AI roadmap. But I think that that's just us. I think that for the United States, for arguably for human civilization as a whole, I think we need to think about these problems very holistically.

1:29:37Yeah, and I think it is easy to get distracted sometimes when there's a lot of excitement a lot of progress in one area like AI and we Are tempted to lose track of other things including things you've said like hey like you know There's a hard work component. There's a there's an infrastructure component with compute and things like that So I think that in general it's good to have a more holistic few of these things and I wish we had you know more holistic Conversations about that sometimes I do think from the perspective of of society as a whole, how should they be thinking about the advances in robotics and knowledge work?

1:30:09And I think it's basically like, society should be planning for full automation. Like there will be a period in which people's work is way more valuable because there's this huge boom in the economy where like building all these data centers or building all these factories. But then eventually, humans can do things with their body and we can do things with their mind. There's not like some secret third thing. So what's your society be planning for? It should be full automation of humans. and there will also be a society with much wealthier. So presumably there's ways to do this in a way that like everybody is much better off than they are today.

1:30:41But then like the end state, the light at the end of the channel is the full automation but plus super wealthy society with some redistribution or whatever way to figure that out, right? I don't know if you disagree with that characterization. So I think at some level that's a very reasonable way to look at things. But I think that if there's one thing that I've learned about technology, it's that it rarely evolves quite the way that people expect and sometimes the journey is just as important as the destination. So I think it's actually very difficult to plan ahead for an end state, but I think directionally what you said makes a lot of sense and I do think that it's very important for us collectively to think about how to structure the world around us in a way that is amenable to greater and great automation across all sectors.

1:31:29But I think we should really think about the journey just as much as the destination because things of all the all sorts of unpredictable ways and we'll find Automation showing up in all sorts of places probably not the places we expect first. Yeah, so you know I think that the the constants here that I think are really important is education is really really valuable. Yeah, like education is The best buffer somebody has against the negative effects of change So if there is like one single lever that we can pull collectively as a society, it's like more education because that's the best. That's true.

1:32:04I mean, more of X paradoxes, like the things which are like most beneficial for education for humans might have been the easiest to automate because it's easy, really easy to educate AI is, you know, you can throw the textbooks that would take you eight years of grad school to do at them in an afternoon. Well, what education gives you flexibility? So it's less about the particular facts you know, as it is about your ability to acquire skills, acquire understanding. So it has to be good education. Right. Okay, Sergei, thank you so much for coming on the podcast. Thank you for fascinating. Yeah, this was intense.

1:32:40That was tough questions. I hope you enjoyed this episode. If you did, the most helpful thing you can do is just share it with other people who you think might enjoy. Send it to your friends, your group chats, Twitter, wherever else. Just let the word go forth. Other than that, super helpful if you can subscribe on YouTube and leave a five star review on Apple Podcasts and Spotify. Check out the sponsors in the description below. If you want to sponsor a future episode, go to doarkesh .com slash advertise. Thank you for tuning in. I'll see you on the next one.