Are world models the unlock for Physical AI? | Episode 19

23:09There's no reason that can't be used in a closed-loop manner in the same way. Okay. So Boris, given all of that, I'm curious about how you guys have picked what AI technology to use inside of your machines, because you guys said in your public materials that you use large-scale machine learning models. Is that a one-to-one to world models, or is there a variation in what Bedrock uses to power its machines? The world models will be used for your kind of simulation and training purposes. Like the actual like ML model is effectively a large transformer based kind of end to end architecture. It's taking as an input sensor data, machine signals, goal of what you're trying to achieve, you know, a variety of other factors that kind of give you the pose.

23:51and then it's processing all of this with a particular structure and outputting a behavior for the machine as well as potential other behaviors like honking the signal that you're ready. And so that behavior can actually govern a complicated machine like an excavator that has six to nine degrees of freedom that the axiom architecture could power, a wheel loader, a bulldozer, a dump truck and other aspects. And so you're effectively learning from a variety of inputs. What's the core of it can be human demonstration where you have a lot of like examples of, you know, of like huge amounts of hours of actual real work.

24:30This is actually similar in spirit to how like the foundation for Waymo's training was, you know, the big shift that really caused it to break through was learning from hundreds of thousands and millions of miles of driving. They kind of captured the nuances of how you interact with downtown San Francisco. So the same subtleties exist in how do you interact with, you know, kind of complex different materials and, you know, get like inch precision on, you know, cuts and grades, do demolition, deal with different tools, different machine sizes, different types of machines. And so we're learning from all of this and effectively enabling these systems to be able to behave like an operator and work towards achieving a goal.

25:08And so that goal could be a large kind of cut to build a data center. It could be a trench to do kind of piping a foundation. Sure. And then, you know, eventually like this becomes the property that you start to see is that the foundations of what you learn in these machines, they actually carry over and give you a huge shortcut towards the next capability of the next machine. And we saw this at Waymo where going from San Francisco to Los Angeles, Phoenix, Austin, Atlanta, Miami, you start to get a higher and higher subsidy and less and less new data required until you're, you know, almost more of an operational and qualification problem.

25:46We saw the same thing going from car to truck, from surface street to freeway. And so if you structure things in the right way, you start to learn more and more of a foundation of what is it that enables large physical machines to manipulate the world around them. And that does more heavy lifting. And you need less and less incremental data that's specific to the new application. And it starts to get a similar property to how LOMs have generalized across language and other applications like grasping and so forth. And so it's quite exciting. And then the world models end up being a really powerful potential tool to simulate and do offline kind of like iterations, training, get more training data that doesn't just have to be physical.

26:27Because in all of these applications, physical testing is so expensive that what you really want to be doing is using the physical world to train your simulation. And in a simulator is actually where you develop and qualify your systems and deploy because that scales much better. And so you oftentimes will see one to a hundred to one to a thousand kind of price differences in a physical hour versus a simulated hour. So you want to push it into that direction as much as you can. So you start by collecting real world data, put that into your simulator to create a strong world model that you can do more testing, then take the results of that and then put them into the machine again and send it off on its own.

27:03Yeah. And the world model becomes like a tool to do all this. Like we've kind of experimented with the mix of elements. But yes, you do a lot of training offline where you're effectively training a large scale model to emulate the best behaviors of a human. And this is where it's interesting because you're not just optimizing just for safety. You actually can have all types of metrics that you actually care about. And the same way that a Waymo is 10 times safer than the human, you can actually become superhuman by upsampling the best behaviors and downsampling the worst. And this is where you become the superhuman operator that absorbs more and more scope over time.

27:34And just explain for me in idiot terms why you don't want to use a world model on a machine in the world and you want to use this transformer-based approach you mentioned earlier. Is it just a lower compute footprint? Is it just a better fit for the work? I would think of a world model as an environment in which you can realistically emulate the real world and simulate. When you're on a machine, you are in the physical world. So you're actually kind of like now acting on it. World model becomes an incredible tool for development in these sort of autonomy applications where you can climb the quality of your safety, your behavior, create scenarios you never encountered, test new versions of your software in scenarios that you would have to physically recreate.

28:23when you deploy physically your world is your simulator in some sense and so you basically just apply your final output in the final model you train. Now as Jeff was mentioning in the case of like a video game your world model could actually be the output because your end product is a virtual experience in the case of robotics your end product is a physical behavior and so you use world models more as a development tool. Maybe it's worth adding um there's sort of a fairly recent sort of approach that people have been using it basically using the world model as a backbone if you're familiar with the concept of a vla a vision language action model you can kind of think of this as deleting the language model and putting a world model in its back in its place as a backbone um and then uh effectively using that um at least the current research in literature suggests that effectively that is a more useful prior or more useful information um to to learning it but as boris says you still need to you still means you'll train that backbone and actually train it for the sort of the the behavior that you're looking to to solve with your robot i mean if i can jump in here like this is something that we use internally on the bla side of things and like boris i have a couple questions for you actually like how long horizon can your tasks actually get are they high scale perimitums right now are you going end to end very long horizon tasks across an entire construction site of multiple different things because this is something that we deal with number one we just don't have the quantity of data of any other domain.

29:49Right now, the amount of data that's been collected of high-scale, high-fidelity manipulation in zero-G dealing with the dynamic coupling problems of picking up something that is unknown mass and then trying to manipulate that and move it is probably very similar to the physics of picking up unknown masses of dirt and earth and moving this terrestrially. In fact, some of the people on the team wrote controls for Caterpillar and some of the heavy machinery there because of how similar, we call it the dump truck problem, actually. If you pick something up of sufficiently heavy mass and you try to move it, the controls of this entirely changes.

30:23Changes your entire controls, yeah. Exactly. And so this is why we have to rely so much on real world data because as much as you can have photorealistic sims and world models, it's just not enough fidelity. Number one, for safety, but number two, for you to have operationally whatever your primitives are done autonomously. And so for us, we're lucky where the business case closes. I can pay someone$200 ,000 a year to sit here and operate this robot. And because the cost of labor on orbit is so high, it closes for us. And this is another hot take I have. Teleoperation gets such a bad rap right now.

30:58Everyone's like, oh, it has to be fully autonomous, full autonomy, full autonomy. That's the only way robotics work. And in certain industries, this is just not true. People don't care who moves their cargo on orbit. They don't care if it's a magical rainbow unicorn, an astronaut or a robot, and they just need that task done. And so for us, we found this one place where we can deploy in orbit. We can have a human in the loop, which NASA loves because safety wise, deploying autonomous systems that are based off of simulated data and not collected data, as well as based on things like world models.

31:31It's just not there yet because you have humans floating around in a tin can up there. If you poke a hole in the wall, this is a very bad situation for everyone involved. And so, you know, for us, the way that we think about it is you deploy sequentially and full autonomy is this thing that's earned, not given. And so you deploy first the teleoperation with you collect all of your sensor data of the environment around you. You understand the physics of just moving in six degrees of freedom, your base before you even pick something up. And then as you pick something up, then you move forward. You start to understand what does this look like of moving an object that's similar mass to our mass of our robot.

32:07And from there, you can start to roll at high level primitives. So let's say I've collected, you know, with the VLA, for example, like Jeff was talking about, let's say I've collected, you know, 150, 200 examples of myself picking up an ISS cargo bag and moving it to, you know, the Columbus node or Columbus module. I can then at a high level tell this robot, hey, go pick up cargo bag in slot 3L and move it to Columbus, node three or whatever it might be. And we can do that high level primitive. And over time, as we collect more data of those edge cases and train and simulation of those edge cases and augment our actual model, we can then build out more and more longer horizon tasks that are not compound.

32:50Instead of moving a cargo bag from A to B, I can move the cargo bag A to B, unzip the cargo bag and unpack it. And after I've done that, I can start to implement experimentation and plugging things in and plugging things out all on one learned behavior, all on one actual task horizon. And over time, as we do this, we build that corpus and everything becomes more and more higher fidelity. And then you earn autonomy, but it's not necessarily something where you can just go in the computer, run thousands of hours and deploy. All right. Ethan, pause. Let Boris respond to that, then keep going. Otherwise, he's going to forget the first thing you asked him and then we're going to have to do it all over again.

33:26I love you, but let's give... That was like seven paragraphs of interesting things. We'll have to unpack it slowly. Super fascinating. I mean, this is an interesting challenge in like every kind of autonomy application where you can actually take advantage of potentially a split where you think of the long horizon planning as almost like a reasoning model that like lives above, you know, everything else that's happening. Our version of this is you might have autonomy for an excavator doing a task and then a bulldozer and a dump truck. And each of them internally has, you know, the model that kind of operates their behavior, but it's more local.

34:01Like they're doing this kind of strip of a cut. They're doing kind of loading tasks. The reasoning model becomes kind of the orchestration model becomes an opportunity to coordinate all of them. And you can take advantage of the fact that you have very different constraints in that model where the physics are a little less relevant. And you can learn from much broader data and break up this problem in a way that isn't as vulnerable to the sort of physical challenges you mentioned. And we have the exact same ones where, you know, you could be carrying, you know, two tons of earth in one bucket kind of stretched out, you know, 20 feet.

34:35And that creates like tip over risks, different vehicle, you know, vehicle dynamics. And so if you're able to do reasoning in a higher level model that then hands off to this much more kind of physically complex kind of reasoning that has to happen locally, you can potentially simulate that reasoning model a lot more aggressively. and then optimize the components that are actually much more vulnerable to the physics of space or the specific controls of an arm or whatever the challenges might be. And so it becomes like a decomposition of complexity where you kind of have a way to lean into the strengths of each approach.

35:19And it makes it more simulatable versus taking on the biggest pain points of every single one out altogether. I've been looking for this for the last two minutes. So here it is. If you're curious what the Columbus module on the ISS looks like, not that world's best resolution image, but there she is right there. It is literally a 10 can. It's like one of those eight ounce Cokes you get at the dentist's office you don't really want. Very small little squat thing. All right, Ethan, back to you. No, I mean, we think about it the same way. It's going to have this higher level thing, planning these individual tasks, and then you have to get much less high fidelity data of that, you know, one-to-one human operator.

35:53And then you can start to expand this out across. And then to kind of jump on, one of the biggest tasks for us is every about 60 to 90 days, depending on resupply schedule, we send up three and a half tons of cargo. And we have some of the most highly trained people, highly trained scientists just go move the bags in and out. It's kind of ridiculous that they do this. Out of the 16-day duty cycle, they'll spend 14 full days just moving things. And so that's why we look at things very similar to the construction industry of that dump truck problem. of I've just moved this bucket of two tons of dirt.

36:26How do I actually control this? Because that changes your entire vehicle dynamics. And if you think about it in space, you know, you move your arms forward, your body moves backwards. And so this problem becomes magnified tenfold. And the only ways that we found to solve this is to use things like adaptive sliding controllers and use humans in the loop, but then also learn from that with RL and VLAs. Adaptive, what was the mechanism you used to move things around? Like adaptive sliding controllers. So controllers that will actually then change themselves based off of what you're picking up, based off of estimates that you actually have and in different environments.

37:00Got it. So highly adaptable based on whatever the task is. Got it. Okay. Jeff, you want to weigh in here on building world models for zero G? Because I feel like everyone's talking a lot of smack about not having enough data. This is really tough. And I'm curious when we're going to get Moon 1 from you guys. It's going to solve the problem. Hold my beer. I was going to say, last six months. And none of this is intractable. I mean, for example, the hierarchical planning that you are sort of both talking about, this is pretty common. I mean, like almost every autonomy stack has got looks of some kind of flavor of this at every point in history in the last decade, frankly.

37:36So the way I kind of see this evolving is you'll always have some kind of like system decisions which will be dependent on the application that might be safety related or something like that. There's probably going to be some kind of controller in the loop. And the question then is where do you want to devote the horsepower of AI in terms of key decision making? And there can be different abstractions. Essentially as the core foundation models get better to maybe use a language parallel, LLMs are amazing provided you can fit things in context. And essentially you hit limits as soon as things drop out of context.

38:04So that's getting better. It's almost like this rising tide that improves the core reasoning and planning capabilities of these models. World models will be very, very similar. It's a relatively early technology comparison. Imagine the algorithm march over a number of months. So what I would expect is basically a lot of the complexities of that to sort of collapse over time, which is what we saw play out in autonomous driving, saw play out in LLMs, the same thing will happen. So the question I think is how do we make a space world model? I think it's a combination of two things. Either a closing loop through a simulation environment where you can kind of close the loop through some like physical interaction manipulation environment with gravity set to zero.

38:46Great. That's something that any world more company can work with. Similarly, from a data perspective, any observational data or visual state and what happens and how this works from a visual perspective, also great. So I think it's just simply a matter of that and bring that into the the data mix, but there's nothing inherent beyond that that is intrinsically limiting. So yeah, if you want to send some data our way. Yeah, I was just about to say that. We like Jeff. So let's say, Ethan is going to go up to space to the ISS with Voyager Technologies, a public space company. They're going to take their, Ethan, correct me here, is it Joy or Joyride your robot?

39:20So yeah, the robot is named Joy and the mission is Joyride. Okay, I got this mixed up in the head. Okay, by the way, good branding. uh so joy goes up into into the columbus module or whatever and bops around doing its testing and so forth if they took like you know telemetry data sensor data you know video data as much as they could from from a reasonably long mission and sent it over to odyssey and said can you guys try to take this and improve a zero g world model for us how far do you think a mission's worth of data from icarus would get odyssey and building that how long is a mission like how many hours Yeah, we're up there for a full year.

39:57So Q127, Q128. So we'd have thousands of hours to ship to you. I mean, I was going to say like a thousand hours is probably sufficient in most cases. I mean, it's certainly maybe sort of a parallel. So we have some of our web models in use of robot foundation model applications. We're not a robot foundation model developer. Rather, we basically provide this as a model to people who then build on it. In this case, there's often some domain misalignment in terms of data, but it's like order of magnitude sort of in the sort of hundreds of thousands to maybe a low millions of samples usually gets us a pretty long way.

40:31Might be a little bit more in terms of zero G, but there's also quite a lot of data that exists in the public internet of things moving in zero G. So we're not starting from scratch in that regard. So certainly very happy to see we add some zero G to the mix. Yeah, we'll ship you an SD card or something. but I think it's going to be a little bigger than an SD card. I mean, dear God, if that's all the data you send over, I think Jeff would laugh. He'd be like, what is this data for ants? That's pathetic. I don't think I've ever worked with less than a petabyte of data in my life. Okay. But okay.

41:02This actually goes back to what Ethan was talking about earlier on about, you know, space distance and, and temporal lag in communications. If you wanted to get a petabyte of data down, Ethan, is that even possible with current throughput between us and Leo lower Yeah. So there's a bunch of different communication schemes and they all have different latencies and different bandwidths. So traditionally, to buck to the ISS, we use an S-band relay. You go from Earth to Geo, Geo down to Leo back to Geo, Geo back to Earth. Oh, you actually do the... Oh, I didn't know about the last loop there. Interesting.

41:33Yeah. And so this historically has very low bandwidth. Like, for example, on that S-band, we'll get about a gig a day pushing it. And so this really sucks for us. But if you look at the new generation of infrastructure going up there, so I'll, you know, pick one amount of hat, you know, a Voyager with Starlab or VASP with Haven. These have laser-based comps. So you get about 100 milliseconds latency in communication, and you get about 25 gigabits per second. And so this is a massive increase from what currently is state-of-the-art. And what's really exciting about what we're doing is, while we won't have the full bandwidth while we're up there on the stay station, because it's like a 30-year-old Toyota Corolla at this point, what we do get is we get all the data actually back down because we're getting the robot back.

42:20So it's coming down on a Dragon capsule. And so we'll actually be able to physically bring it down and actually process the data afterwards. What's the backup if it doesn't? Yeah, yeah. And then we have the one gig a day over the course of the year. So that's the backup. Okay. Yeah, that's really the backup. So you're going to lose a lot of data. But, you know, let's just hope that that dragon capsule comes down and, you know, the astronauts don't die as well. Oh, well, now I feel like an asshole. I guess there would be humans in there too. I was just thinking about your one pathetic little SD card getting too hot and burning up.

42:53Boris, we're talking a lot about, you know, where the rubber meets the road here, or I guess where the robot meets the zero G, but you're definitely out there in the real world. I was really impressed that your company recently moved 65 ,000 cubic yards of dirt on one project alone. So are you now just like in data abundance as a company? Because I feel like if you've done that much actual earth moving after all the training you've talked about, all the data, all the sampling, all the modeling, you must just have a ton of fresh and useful data. It's accumulating. It's how everything's relative where I think there's always, particularly given how many types of machines and types of work are out there, there's always a frontier of like new diversity, you know, new variety and so forth.

43:33it's funny as Ethan was talking I'm like thinking you know the data problem like everybody has a version of it where you know we generate a terabyte per machine per day and so it's a thousand gigabytes and so it's like we're like yeah we're like you know not quite doable with Starlink and so we're still kind of shuffling physical you know data around and then there's you know kind of like work your way trying to hope that you know you can't compress your way into something that it could be purely through satellite. Yeah. So for us, we have a really helpful property that we are operating an existing heavy machinery that we can retrofit.

44:11And so we do not need to create a brand new machine that is obviously expensive, complex, and also not in existence, which makes data harder. The fact that we can go and take existing$600 ,000 Caterpillar machine and turn it into autonomously capable, whether it's for data collection or for autonomy testing, that opens up a lot more flexibility in how we get the data. And so we are able to have really rich partnerships with general contractors and subcontractors, both for data, but also to go in our actual deployments with them, like what you mentioned, where we were helping build a factory in Arkansas.

44:50in this case uh we've been on a number of data centers i've done a manufacturing facility uh a couple warehouses uh and we're not far off from going um those were supervised autonomy tests where we were doing autonomy but with a safety operator and um later this year we'll be going full drive over since so that means that these will be operating on real sites doing work um all day long with nobody in the cab um and in the initial use cases will be mass excavation and then moving on to more and more types of work. But what's exciting is that some of these projects, you know, we're now doing projects that will be many hundreds of thousands of cubic yards of earth.

45:30And these are subsets of projects that get into the many millions of cubic yards of earth that become the opportunity. And so when you think of some of these data centers, for example, they're getting built, they'll be getting built on many hundreds, if not thousands of acres, and can be, you know, two, five, even 10 million cubic yards of earth. And so these are the scale of projects that are actually happening. That's almost as much dirt as Peter Thiel dug up to build his bunker in New Zealand.

45:57No, jokes aside, 65 ,000, sorry, it's not fair to make you guys enjoy my humor. 65 ,000 cubic meters is exactly 26 Olympic sized swimming pools. So that's how much dirt you guys moved in case you wanted a more prosaic term. All right. So listen, all this is great. I'm excited about everything from world models to saving time construction sites, building things faster, going up and learning. It's great. It's so great. It's optimistic. It's very human positive. Why does this not form more of a conversation we have at the public level? And what do we need to do as people who are really bullish on having more intelligent machines in the world and better models behind them change the conversation?

46:38I'm so bummed to read the news and read what people are saying, read the polling about data centers and everyone being very concerned, and then coming here and having a blast and just... There's an optimism, right? Like on what's possible with a lot of this work. Maybe I'll kick off. Like it does worry me because when you see approval rating for AI, like below 20%, that puts it in danger of being a bipartisan issue that like actually, you know, like impacts the space in a way that is detrimental, not just to the space, but to like, you know, overall society. And so I think it's, there's not enough, you know, the communication around it has certainly not been great, but there's so many successes in, you know, the breakthroughs you can start having in medicine, the way this democratizes education, like the way that, you know, today in construction, there's like a astronomical shortage because it's been systematically underinvested.

47:37And now when the demand is skyrocketing because of infrastructure and data centers and non-train manufacturing, you start seeing housing prices go up and so forth. And you have a world that is like fundamentally can be expansionary. And everybody does have a legitimate fear of local job disruption, but almost always it's much easier to see the local direct impacts where something is lost than all of the astronomic positives that have happened in every single technological transformation that are actually harder to predict. Where you see the mobile economies that started to form after mobile phones, what the internet created, even if it hurt retail stores locally in some places.

48:19Right. And like the biggest example that I can remember that kind of hits on the physical transformations that we're talking about is the Industrial Revolution, where it has that sort of a feel today where when it happened, everybody is petrified that all the jobs would be lost or be complete unemployment. Everything would be absolutely horrible. And, you know, just a few people, they make money off of it. And what ended up happening is that even though there was like locally some really painful kind of transitions that might be geographically or in profession fixated, by the time that dust settled, the productivity skyrocketed by like order to magnitude, you know, plus the number of jobs actually increased, not decreased, and the average salary doubled and the rate of poverty significantly decreased in the entire country.

49:02And so you have these like really, really non-zero-sum impacts that I think we're not doing a good job of conveying externally. Jeff, there's the big sovereign AI pusher over in the UK. I know there's talk about building kind of a UK-specific AI model. I know you're going through some prime minister turnover, but it did seem that the Keir Starmer government was at least generally in favor of building out AI. So are y 'all doing a better job with the pitch here, the optimism, the forward-looking element of this, the we are going to have better medicines and better technologies and cheaper goods?

49:32Yeah, maybe the context where we're partly transatlantic company with us, half the team here in London and half the team in Palo Alto. I think AI is ultimately a global problem and a global question. And the reality is companies who are pursuing this, particularly at the foundational scale, you have to think about winning this globally. And I think it's a good thing. As Boris said, it's not a zero sum game. In my mind, it's more about focusing on how do we make this a positive sum game. And I think there are so many things where, like many technologies, in many respects, I think all of us here are building something that is not possible today and the world will be better at the outcome.

50:08And I think in many respects, that's really why, certainly what motivates me and I think certainly what motivates most folks in the AI field. So as to whether we're doing a better job, I'm not entirely convinced. I think every society is grappling with this in terms of what it means, how the world will work, how relationships between countries will work. It's a bit of an unknown, but I don't think that unknown is to be feared. I think in many respects, It's a thrill of discovery and that motivates people to do ridiculous things like go into space. Yeah, I think something to be celebrated. I wonder, Ethan, if we just got everyone to read 10 times as much science fiction, we could solve this branding problem.

50:45Because if you pick up a copy of Red Mars or just even something that's older, but just aspirational about going out into space, like or Rendezvous with Rama, just pick another classic. Like, to me, those got me so excited about the possibilities of going faster and further. Your company being a good, I would say, step towards our front porch as a species before we go off even further. How do we get folks to be a little bit more open-minded and optimistic, especially amongst the younger set that are currently dealing with the butt end of AI today, which is a slightly softer job market for recent college graduates?

51:16You know, it's two things for us. Number one, we plan the end of physical AI in the AI field. But then also we work in the space industry and there's nothing more expensive looking than the space industry. When a rocket goes up, it looks expensive. It looks like someone took a pile of cash and lit it on fire. And people are like, what does this do for me? If you're New Glenn, that's true. You know, and like the thing about like programs like New Glenn is there's so much of the positive end where NASA and space have very bad PR problem. They don't like to take credit and they should take a credit for a lot more things than we have day to day.

51:54And we really have to ground what we do on how it impacts people's day to day lives. And I think that's the thing that every time I've sat down and had a conversation of why it's so important to do what we do, then this is what lands with people. You know, like, for example, you know, your cell phone. The reason why we have miniaturized computers was because the Apollo program. We needed them to put on the rocket to take us to the moon. The reason why you have memory phone, that's NASA. It created Invisalign. It created from a NASA program. Invisalign? The things for your teeth? Invisalign. That was a spinoff of technology that was made by the NASA program and NASA funding.

52:30Things like your credit card. Every time you swipe it, that timing signal, that comes from a satellite. Your GPS, when you came into work, you would not have that today without a satellite, without the space industry. And I think the one for us is we have the most advanced lab in the world floating around us with people working on it every single day for the last 25 years. And they created this cancer therapeutic called Keytruda. It's the number one cancer therapeutic in the world right now. And so we did the research and development of this drug where you can do protein crystallization on orbit.

53:05And we found out the way that proteins crystallize in space, totally different than Earth, way more effective. We changed the way we manufacture this drug right now on Earth with a random positioning machine to try to replicate zero G because we can't. And that one drug made $29 billion last year in revenue for Merck, the pharmaceutical company, and has saved hundreds of millions of lives. There you go. That's the one you want. I think we need to drop the financial one, even though we're all capitalists here, and just go straight to the petted kittens and planted trees. Because I think that there's so much money going around right now that every time we bring it up, it doesn't land well.

53:39The joke that I made at the start about how Icarus hasn't raised a nine-figure Series B yet is a joke amongst us because we know that the markets and problems are so big that it's worthy of investment. But to an average person, they're like, my school fell down. Why are you giving Jeff$310 million? He's got nice hair, but does he really need all of that? Couldn't he get by with$35 million? You should see how GPU bill. Maybe we need to definancialize things a little bit. I don't know. I don't want to live in a world where we have the shot at such rapid progress and a brighter future, and we end up talking ourselves down, and dumbing ourselves down to the point in which we slow down.

54:17And I don't know if we're going to lose to China per se. I think it was Jeff talking about competition, but I don't even want that to be a possibility as a fan of democracy. So do we need a new leader in technology? Have we just failed to find the right person to carry this torch? I think in respect to the revenge of the nerds, right, is a good thing. In respect, I think a lot of what sort of built the technology industry is people being like, oh my God, I can build something ridiculous of my garage and it's really, really cool. It enables amazing new things. And I think it's often being sort of couched in large business interests, which are true.

54:54I mean, respect that's a good thing. It brings the capital behind the problem, but at the same time, I think that soul is valuable. And I think sometimes being a bit more open and human with the public in that regard can be useful. Yeah. Boris, do you think we should have partial AI lab nationalization. We're seeing the horseshoe theory in practice here. We got Bernie Sanders and Donald Trump both coming to the same conclusion. I don't think it's a particularly good idea, but I'm happy to be wrong. What's your take? That kind of scares me as well. I'm with you. There's not a, there's a lot of, not a huge amount of examples that's actually worked out well in other industries.

55:32At the very, very least, it massively slows down innovation if you now have that gate across. Now, it doesn't mean that they're, you know, like given the power of some of these models, you don't have, you know, some checks on how and, you know, when they, the sort of ways that they can be applied. There's export controls on GPUs. Like there's strategic reasons on why that might make sense. Right. But it kind of scares me. It's almost like saying we're going to government control software. Right. It's such a. Math in this case. Right, math. It's like, okay, well, there was proposals to control models in general.

56:15It's like, well, at what point does it become a large model? What if you're one parameter under? It's kind of silly in the big picture. And it's trying to kind of apply a hammer to something that is just operating on a very different dimension. So it worries me. But at the same time, I think it's probably not doing the industry. is doing the industry a disservice to bulldoze forward and put your head in the sand and not actually try to communicate and kind of create a bridge to government, to the rest of society where people do genuinely feel like they're not benefiting equally from it, even if a lot of their benefits are actually going to come in five years when like all these technologies mature.

56:54I think the only 100 % losing strategy when it comes to selling AI to the public is to be dismissive and rude to people who have concerns because I think what that does is just poisons the conversation. I'm not going to play it because I want to save a question for actually for you again, Boris. But I saw a video of Theo Vaughn, the podcaster who became very well known during the last election. And he was talking about data centers. And he was talking what I would say is a standard TikTok set of points about data centers. And everyone was just dunking on them. Instead of trying to politely educate him, help him out, be supportive, they were just calling him an idiot, calling him various slurs.

57:27And I was like, that's not at all how we're going to build the coalition here to keep this from becoming a bipartisan issue that's going to get crushed. So I'm hoping for some more positivity. All right. I want to do one more thing before we go and talk about the return of fable for fun as a final question. NVIDIA recently announced Cosmos 3, which is a quote, open frontier model for physical AI and the world's first omni modal unifying vision reasoning world action and generation. So Jeff, are they encroaching on your core domain here? Or is this something that's more like a test bed for other folks to learn from?

58:01Overall, I think it's positive for us. In many respects, they're another voice advocating for this as a category. NVIDIA historically has done this, where they use their research arm as a way to basically prove markets and validate markets. And essentially, as soon as the market is mature, they typically go and find the research tiers another way. And this has been played out quite a number of times. So yeah, in many respects, there are things that are comparable to our models. There are things that are different in terms of our models. There's some great researchers on it, but I'm quietly confident in our models.

58:34Yeah. Well, I was just thinking that Nematron, the models from NVIDIA, are a good example of open-weight AI from the United States, which we could see a lot more of, and I think we wouldn't be any poorer. Now, Boris, there was also something from NVIDIA called Halos for Robotics, which is full-stack open robotics safety system. Is this something that you can then bring to your robots, or is this something you've already figured out, and they're just helping other people avoid the, not the dump truck problem, but the swinging bucket problem? And we were constantly kind of like surveying and kind of trying out these tools.

59:07When you get into these incredibly complex systems, we've at least found that you have to more deeply vertically integrate and really control the full stack because you have to deeply understand the interface with the systems, the latencies, the physics and controls, the cycle counts, the performance of every model. you have to have the architecture that is guaranteed to fail safe. And so we're constantly looking at these types of tools as a support for various building blocks. But I think given that there's still so much novelty to these large kind of physical systems and things are just not standardized and how does a car work versus an excavator versus a humanoid machine, there's no magic bullet.

59:55you kind of tend to have to still have a pretty deep control all the way through the stack. I think it's an understatement. Every single robot financial model company, including autonomous driving that I've spoken to, has a uniquely different view in sample rate, frame rate, in terms of the action rate, in terms of the way they've parameterized trajectories, including resolution, the number of cameras. Do people like diffusion? Do people not like diffusion? It's like, what's safety critical? What's not safety critical? What's your safety case? It's almost like you can't miss their potato head.

1:00:22It just never really works. When you talk to them, they're all so confident that they're correct and everyone else is wrong. And I'm always like, guys, you can't all be, maybe you can all be right, but you can't all be wrong. I don't know what to tell these very smart people. This is natural. Like, I mean, you tend to see this, right, where everyone kind of fans out and you kind of, imagine you're kind of like throwing darts at a big dartboard and you're trying to get points. And everyone's sort of exploring. And over time, basically, things converge. I think autonomous driving is sort of a little bit ahead of the curve compared to some of the other fields, just because it's a bit more of a mature industry, a little bit more defined problem at some level.

1:00:53The objectives will converge over time. It's just a question of enough people getting enough information. You end up with cross-pollination between companies. This will start to consolidate. In methodology, I mean. Okay. And so do you think once we get the methodology kind of aligned or harmonized, do you think we'll be able to go faster or will that actually slow us down because we'll be exploring fewer odds and ends and nooks and crannies in the research game? It'll make my life easier. I won't get such a wide range of feature requests. Jeff, I'm not actually optimizing for your personal retirement.

1:01:25I'm not going to lie. Typically, people consolidate around things which work better, and typically something that works better in adjacent domains tends to work well in another. So maybe things which work well in autonomous driving will probably have, in many cases, will work well in maybe construction. Although maybe that's a little bit different given you also have got manipulation of your environment at the same time. So there are adjacencies, and typically where there are adjacencies, you accelerate. I'll take a slightly different lens on this one is that this focuses on the autonomy model and the core approach of development.

1:01:59What's deceptive is how much of the challenge of solving, for example, something like autonomous driving is actually not at all in the autonomy model. But how do you evaluate how you able to improve your system when you're in a super long tail, one in millions of mile case without regressing accidentally in 15 other places? How do you actually qualify it? And so there's actually so much surface area that in a lot of ways, people have gotten quite good at climbing quality of models when you have the right data and so forth. And the hardest problem eventually becomes how do you get the right signal that you actually can drive 500 developers to optimize over?

1:02:35Or how do you actually test it and make sure that it's qualified and ready for driverless when you have safety critical applications? It's actually harder in your case with bedrock because if you think about like modern equipment, it's often hydraulically driven, right? If you're doing a big digger. And you know what's not super precise? Hydraulic driven, large buckets full of dirt. And so earlier in the conversation, you mentioned something like precision and like getting within an inch or something. I've seen these machines. They're not always very well maintained. So you probably have to also take into account like you might tell it to do something and you're going to get a slightly different outcome and have to handle that as well in the real world.

1:03:11So that's exactly right. Because you now have a variance where you do not like, you know, Waymo has thousands of jaguars on the road. They're tuned to be like as similar as you possibly can be. You know, you have like 72 different types of attachments to an excavator. You have tons of different sizes from, you know, like in terms of tonnage and, you know, and variances as well. And so what you have to do is create safety cases that are actually resilient to this and work at everything from subsystem to kind of overall system level. And so, for example, you can have controls systems, imposed systems that estimate and, you know, all these models.

1:03:45And you can like empirically kind of like measure your errors and your variances and collect data incredibly quickly over a wide range of machines and have very precise tolerances. And be confident that you have a particular type of margin of error that's going to be your, you know, like some giant percentage of time you'll be within this sort of tolerance. And then part of this is knowing that you will have exceptions where, for example, let's say a sensor broke or a machine is just like fundamentally different and you fall outside their range. You have to have fail safes to where it's always preferable to fail safely than to try to do something that's outside of your space you've qualified.

1:04:23Ethan, it's funny how much you're the opposite problem to this because you're building one robot. You're controlling all the sensors, all the tools, all the pieces. You don't have to go out in the field with, you know, someone didn't maintain this digger, so it's a complete mess. How precise do you need to be when it comes to your machine interacting with the physical world? Are there still tolerances you can play with or is it like millimeter precise requirements? No, you still have to be within those centimeters, like especially because the environment that we're in is working hand in hand with humans.

1:04:52And so where you can have very clear workstates with something like large hardware that's on a construction site, make sure there are not many people around. So even if you do go into whatever failure state you're in, you can be safe. For us, if we go into the wrong failure state next to somebody and you hit them, it's not like there's a doctor up there. Everyone's trained in medical, but that can be a very bad situation. And so when we talk about the systems that we're interacting with, from the switches to the science experiments to even the cargo bags, these are made for human level fidelity.

1:05:24You're very good at grabbing very small things you have tabs and zippers that are sub centimeter size that we need to grab ourselves and if you take like robotic grip and you kind of break it down you get two things you get pinch you get power and so the way that we've made our end effectors are actually to replicate that so it's actually just two small fingers because of how precise we need to be where we're flipping switches uncertain experiments to turn them on and plugging in mil spec connectors um and for us like the the two things that make it hard is number one every single action in space the reaction is just so exaggerated on earth you have 1g dampening every single movement now for us we just don't have that you move your arms forward to the side your entire body starts rolling your cameras start rolling um and then when you pick up things sometimes we're picking up things that are more massive than our vehicle itself And so that shifts outside of your control matrix of your thrusters, that shifts your center of mass way outside of this.

1:06:25So traditionally, if you have your center of mass inside of this and you try to strafe left on the X axis and just move, you'll start pivoting in a circle if you've sufficiently moved your center of mass far forward enough. And so these are the types of things that we have to really care about to get our precision fine enough that you can do the tasks that matter. You know, I find it really funny that the reason why a lot of people want to build humanoid robots and they've told me this is because you can take a robot then and put it into a human environment and it can perform well. And they often do this in places where you have lots of room and could actually change the definition of work and create more of a purpose-built robot.

1:06:57You're going into a literally human defined space that's very small, very complicated, and you're not going humanoid. So I wonder if everyone else kind of adds it backwards because you're making, it looks like a robot, if I imagine like, you know, what are the dimensions on joy? Like a pizza box by like four pizza boxes? Yeah, that's actually like perfect. I've never thought of explaining it that way, but that's probably a great, great visual. You're welcome. But it's that size and it's supposed to interact with human things quite well. So Ethan, are people just, is figure at all making the wrong bet when it comes to what they're actually building to do the work?

1:07:34So I think it really depends on the environment and what you're trying to do. Like humanoids, the reason why these are compelling is economies of scale. If you can get something that does every single task on earth made for a human at 80 percent and you can just print these it makes sense on the financial side of things you'll never beat a purpose-built machine or robot for some task with a humanoid it just won't happen but that's not the point of humanoids the point is i can make a hundred thousand and deploy them everywhere and democratize labor and when you look at the earth and you say okay half of the earth's gdp is people doing things it's human labor, there's a very compelling argument there.

1:08:10When we look at space, we say, oh my God, I'm going to go look at these astronauts. I'm going to go talk to these astronauts. What do they actually do in this environment? Turns out legs suck. They don't help at all. They mess around your center mass. The only things you ever use your legs for is hooking into footholds so you can react against the vehicle itself so you're not floating and spinning in a circle. And so when you break down the environment, your first principles, a human has been evolved to work on Earth very, very well. but not space. We sent them there. It kills them. Your heart degrades.

1:08:42Your eyes go bad. Your bones get soft. Exactly. And so when we broke it down to first principles of what was most important, it takes shape as something very, very different. But the machine itself and the robot itself is still a general purpose platform. And so I think there's a very big difference between humanoids and general purpose platforms depending on the environments. You don't want a humanoid to go swim underwater you want something that's an underwater rov or similar class um the same way that you don't want a human to fly in the air but you might want a human to walk on earth so i think depending on your domain you get very different general purpose machines all right listen i could keep asking you guys questions literally all day long but it is getting a little bit late in london so we're gonna we're gonna wrap it up let me do this wrap towards a final a final quick round of questions here.

1:09:30So for each one of you, one, when does Fable come back? I'm curious. And then two, are we still approaching the GPT-3 moment for physical AI or have we already gone past it? And Jeff, we've heard the least from you lately. So we'll start with you. Ooh, when are we getting Fable back? Soon, I hope, would be my thought. I have no unique insight into when they might come back. I think that's probably out of my hands. You have to put a date on it. You can't waffle here. No, I don't know. this one okay uh pretend you're from texas two weeks okay good uh in terms of uh gpt3 moment for her physical ai i don't think we're there i think in my mind the gpt3 moment is when you see the spike of a sort of explosive rollout in demand um i think we're seeing promising signs but i don't think we're there i think we're on the trajectory all right boris same questions for you oh let's see when does it come back uh uh well and not in not in a form where they kind of like nerf the capability, but for real.

1:10:29And I'd say it sounds like it's a little bit longer because it got a little bit of a mess with, like, kind of, you know, government, but now they've got to, you know, to go with it. So, I don't know, I would say eight weeks is my guess. I don't know. Eight months. I guess we're first eight weeks. Yeah, me too. I don't know, I'm a little less optimistic. Sounds messy. You know, like Jeff, I actually think we're not there. I'm maybe a little bit more pessimistic on the GPT-3 moment anytime soon just because that benefited so much from almost infinite data in a common feature space of language. There's so much nuance and variability to the hardware, the sensors, the environments, the safety challenges, everything, that I think you will for longer see these more verticalized applications really focus and take things end to end.

1:11:13Even though the industries can be double-digit percentages of GDP, I think it'll be a while longer before you just have something that can genuinely go into brand new environments and be super capable like this. But likewise, I don't think it's impossible. I think we're on the way. It's just going to take longer. Okay. No, I really appreciate the context there. Ethan, you get the last couple of words here. So when does Fable come back? And the GPT-3 moment for robotics, future or past? Yeah. So when it comes to, you know, like Mythos and Fable, I don't think we're getting it anytime soon. I'd put it, you know, two and a half, three months.

1:11:45The sea getting worse. I'd put me in the fourth guess. You'd be like, five years. No, I mean, the government moves at the speed of government. I think once you classify something as a weapon and you put ITAR controls on it, this just makes everything so much more messy. You know, we've had our dealings with them. Just getting documents signed takes weeks. So I can imagine that this will take a very long time. And then, like, as far as GPT moment for physical AI, I think we're so early. I think it's a far, far time away. And I think like Forrest said, you're going to find very vertical niches and very, very constrained new sets right now.

1:12:23And then once you see that demand signal spike and once you see the capability spike is when it will explode. And so I think that's what makes it exciting for me is because we are on the, you know, the greenfield bleeding edge of it right now. And I think like two, three years time, we're going to see that massive explosion. Well, it's really optimistic because I hate doing work and I hate driving and I like to write. So if anyone can take care of all the physical stuff, I'll do all the typing and life will be good. All right, guys, we got to leave it there. This has been This Week in AI. My name is Alex.

1:12:51And today we are joined by my new besties, Boris from Bedrock, Jeff from Odyssey and Ethan from Icarus Robotics. Thank you all much. We'll have you back and we'll see everyone next week. Goodbye.