World Models & General Intuition: Khosla's largest bet since LLMs & OpenAI

27:17This is something like - I mean, ideally you get rid of the imitation, yeah, the imitation learning, but yeah. We essentially realized that we could get so far on just imitation learning. The way to look at it is we essentially, like let's take the LLM analogy. We essentially have sort of the internet or like common crawl, if you will. And every single lab is trying to simulate that, right? In order to get similar data, in order to train their agents. And so for us, the reason why we say independent and we just said our own thing was we think we can essentially leap every single company that's forced to either be consumers of world models or build world models and take this foundation model bet for spatial-to-bore agents and be in a place where we have a lot of customers years before any of the labs even get there.

28:00and maybe the most similar um comparison is like when anthropic did with code right anthropic just focused really really hard on nailing the code use case their models are incredible for a lot of their customers use it for it so we just want to become incredible at this spatial temporal agent use case and likely that starts in like game simulation and then using world models we can then start expanding out to to other um areas so would you show me a little bit of how you think does generalize our yeah things um but although games is kind of the common pair yeah games and simulation um i would i would specify it as game engines and verticaler so even if you're for instance uh simulating human behavior in omniverse because they're trying to create better training data for factory floors um you can use it yeah maybe meta has a similar data set because of the quest i never really asked them i never really looked into the meta quest specifically so you need a few things you can't just like there's lots of companies that have like maybe recorders but you also need the public graph otherwise you can't train on the data right you can't train on people's like private videos that they have saved somewhere right and so i think you you you need the social network graph components um because these videos need to be on the internet to rank no to train on them yeah i i mean i think i think generally people don't like people don't want to train on like because these things they live on your device usually right yeah um and you can't train on anything that lives on your device like you actually need to go and upload and do your thing right for meta specifically i think also vr the scale of vr is still pretty small The amount of environments in VR that have consumption at scale is probably in the hundreds, whereas on PC, it's probably in the tens of thousands.

29:38And so you get a lot less diversity. The three-dimensional input space of VR is pretty interesting. We see some of this too, obviously. And so, yeah, I do suspect Meta starts using these types of things, but it's unclear to me whether they can get to a similar scale of data or diversity on the environments as we can. Yeah, there's a lot of challenges there. Okay, I want to take this in a few different ways, but I guess let's fill up the papers. Maybe one more to mention is Tire. Yeah, which I actually interviewed at our office, but that too seems like the particular insight that brought it overseas.

30:15Yeah, so Anthony Tu, who led the research on Gaia Tu, is also one of the engineers that joined our team. So it's all the Diamond, the core contributors for Diamond, and then Anthony. And we just had three more researchers showing this week. It's been a good week. And yes, I think a lot of the approaches in Gaia 2 were heavily inspired by Diamond. And then Vin Sa, who was one of the authors of Diamond, also already was at Wave by the time that I emailed them. Anthony also realized what this was and realized that, you know, you could scale world models to a much larger scale and decided just to make the leap as well.

30:48So I think everybody that sees the dataset makes the leap because it's, but it takes a while to wrap your head around because it's like, oh, it's video games, right? Like intuitively, it doesn't make sense. And then when you actually understand and you see, right, how we've been able to transfer it to physical world video and things like that, then it makes sense. And then everybody tends to jump. Don't call it video games. Follow it on a lot of times. Yeah, if I lived in San Francisco, maybe I would. Just a quick note, because we actually cover all these papers in the latest day Super Bowl Club.

31:19SEMA 2 did not seem to have as much intact than SEMA 1. I don't really know why they did it a lot more work. g3 had a ton of impact and but i also felt like because you couldn't play with the model or people it just seems they're an extension of all those days i guess like any quick takes on sima 2 g3 which were both these years yeah i'll talk about sima 2 the steerability of sima 2 was to me the most impressive part because lighting up the action sequences and the text conditioning is quite hard to do, right? And so that, and the fact that they were, like, it's also quite interesting that it means that they can sort of use Gemini as part of the flywheel, right?

31:59Where you can sort of scale this orchestrator as, like, an independent, almost like a puppet master, if you will. And then, like, in theory, Gemini could orchestrate many instances of SEMA, right? That, to me, is the most interesting part, is where I tend to agree with this, where, like, I think our models will initially be used as, like, you'll have like an orchestrator vlm of source that's kind of like managing instances and instructing them um and i think for sima showing that you can do this was was fascinating also the fact that you could um they didn't just have text conditioning but they also were able to do like drawings and markings uh of where to go they really took an interesting end-to-end approach to me uh that i look forward to seeing a lot more of um but you're talking to them like you said it is that the one collaborative yeah i i think that um yeah we're very friendly with deep mind we like them a lot i just saw the team not too long ago and i think you know big fans of their work the different line that i kind of shade from alice's coverage review yeah is uh you are the biggest bet that and they don't cross-line as many since open ai yeah how did that conversation start okay so for now it's style and maybe i'll get slapped in the fingers for revealing this or whatever but uh forgive me if i'm super bad um is he asked you to like draw a 2030 picture of your company and i think he just picks n plus five years whatever i don't know i did the same to you yeah um he asks you to like walk that back from first principles all the way from today and and and yes he expects you to do that flawlessly where he can challenge any assumption any part of the vision that that and he asks you questions right he has a very technical background he also has a bunch of technical people want to see and he truly backs people that have these like very large visions on that vision and the ability to defend it alone um and that's what he did for us um and i think that's why i made that bad so i think also through this uh through this question he he gets to know a lot of things about how technical you are he gets to know how well you think from first principles because if that i if that vision is not connected to something real it's very easy to suss it out by asking good questions.

34:10And then he just backs fully, I think. Like he really gets in your corner if it's the right fit. And yeah, they've been incredible partners. They've opened so many doors for us. I had to ask the question, I think it's a very notable story. Obviously, a lot of work went into it, but it's also worth it. Yeah, for sure. One of the things I also wanted to, I think I asked this question out of Sequence, but one of the things that's exciting about Telling to You is there are a lot of people like you who are founders of businesses and businesses that along the way have a ton of data. And yours happens to be highly valuable.

34:51You pursue, before deciding to do an independence journey, you can also talk to other companies about potential licensing or acquisition and stuff like that. What is your learnings from those periods? Also, one version of this is very simply, how do you value the air? Yeah, I don't think you can value it unless you actually model it yourself and see what the capabilities are. That's my real outcome. You say model, but chain the model. Yeah, but that's obviously not doable for everyone. And also I think my general advice would be as model capabilities increase, and models are also like, these VLMs, they're very, very good at labeling as well, generally, right?

35:35What I was afraid of when I was having some of these conversations was, okay, as the capabilities increase, you're just going to need less ground-free data and you can do more model-based data generation or synthetic data generation. I would recommend if you're going to do large data deals, just try to get a large chunk of equity in the company that you're doing it with, if you can. Now, a lot of them won't do this, but I think that to me would... Or just go do the research, figure out what's actually possible. In our case, we were quite lucky in the sense that this is actually the foundation data right and i think right like that's not true for for every data set i think you know we just happened to hit a particular gold mine but you also did you read kate brady you did the action thing one for five years ago yeah so you eat it work yeah that's the thing like you you have to be grounded right and i think a lot of the um and and i think that's a hard part and i think a lot of what's interesting is you can also kind of look for if like scaling laws already exist on your data type which like for video there were some but for these like input action labeled uh sets there there really wasn't any the other question is like does it go into lms does it go into uh world models does it go into like what type of model is it going to be used for and i think that's an important thing to know and so i just want to you know if you're having these conversations with labs about data just like make sure that you actually understand like what it's going to be used for because that's a very very good way for you to like make that decision yourself about whatever you want to pursue that now a lot of them won't tell you that and i think you know i think in in in that case you generally just don't want to do it because like i think i think for our case like we really cared that like for instance there weren't going to be competing products with game developers built right because we didn't want to like bite the hand that feeds us and i think we are part of the games industry so those questions i think are normal and then we eventually decided you know you just have the data we're just going to go do it ourselves and that's when the rest happened yeah and he assembled the team i can uh think about that i i feel like that's you've aligned a lot of stars in order to make gi happen yeah that other data founders they're at the beginning of the training yes one data founder founders who happen to have data but they have a main business right i don't know if you're there there's two sides to this right there it's really easy to be super naive about it and like i had a lot of people you're just like making this up and and and so for me like doing the work and actually understanding it myself was a really really big part of building that confidence and go start a company but a lot of times it is true that like model capabilities increase so quickly that like the certain data you just don't need anymore yeah um and so i think it is it's really important to like get people to do the work such that you can make these types of distinctions yeah and and and so my recommendation would be go build models with your data see if you can create any sort of capabilities that aren't clearly already there or on path to being there and then figure out where you go.

38:31Yeah. I did want to ask this earlier, but you're giving the opportunity to, we say do the learning, do coursework and all that. And your co-founders gave you some homework. Yeah. Is this like some books? I mean, Coursera? No, this was Francois Fleurais. So he has a little book of deep learning and then he also has a full course that he's published on his website. I went through the entire course over the summer i believe it's like something like 30 or 40 lectures which also take home projects and things like that um and i would recommend anybody uh uh does this it it goes through right history of deep learning like the the topology it takes you through um the literary algebra the calculus eventually end up with like chain rule and by this time you've done like all the the more important concepts it takes you through how do you create neural networks using uh using these concepts that you've learned wow this is super first principles this guy and i've i've had the the uh opportunity to spend some time with him as well he is one of the most first principles people i've met in my entire life i'm convinced like i actually asked him why did you do this course he's like oh because i thought all the other courses weren't right and because because he is so first principles and he can only explain things from like everything you see and how he explains this saying it's everything is from first principles including like the history of deep learning itself was part of of the course and um yes he goes uh um so all so he goes through everything and then uh and by the end of it i think you're like i now have like a pretty good intuitive understanding of how everything works but obviously still right like i like to describe it as um i'm like the the guy who just got his driver's license i can drive the car and like my co-founders are like the f1 drivers that like have done this for years they know where all the um uh where all the the gaps are and and so i i enjoy getting to learn from them the cool thing is also that world models is just like a very very new space and so you know i i got to bring ideas to the table that like you know one thought of and not because i'm great at this just because it's such a new space that like people just haven't tried it yet um so let's get a hit on definition yeah what are world models to you you know in a video model you might predict the next likely sequence or the next most entertaining frame.

40:40What world models do is they actually have to understand the full range of possibilities and outcomes from the current states. And based on the action that you take, generates the next states, right? So the next frame. And so it is a much more sort of complex problem than traditional video models. So to me, it is a world that is accurately generated based on the actions that you take as a result of what's already been generated just a fact check uh that is it needs to understand physics it needs to understand if i'm building a type of material you need the power interacts with some type of material yeah i think the interactions is the most important part i think the reasons why role models are so fascinating one of the things that i did when i was studying over the summer was i tried to actually built a super rudimentary PyTorch physics engine, which I would not recommend writing a physics engine in PyTorch for obvious reasons, but I wanted to be able to, because it's differential, so you can generate this.

41:39It's very difficult. Yeah, exactly. And then you can train. And so I got so many people asked me about why aren't you just simulating or generating this data? And I really wanted to understand from first principles why. And I think the most important thing that I figured out was the compute complexity of simulation goes up really, really rapidly with three variables. First, the numbers of agents in an environment. Second, their DOF, so their individual. Jewels of freedom. Yeah. And then third, the information that each action reveals. so like for instance if you if you have a if you have a text action or a speech action the environment can change so much based on whether you say right water or fire that the outcomes are going to be completely different of like how a human would behave in that type of situation and so it goes up so quickly with those three variables that at some point you just hit a point where you just want to maximally bet on either video transfer or generation of these environments using world models because that type of stochasticity is just incredibly difficult but it's already very, very present in a lot of the video pre-training that goes into these world models, right?

42:48And so I think for us, it is more so about making a maximal bet on video transfer and interacting with things that are difficult to simulate. And the steerability is also really interesting with text than it is on betting against simulation or something like that. And so I think there's still a large market for traditional simulation engines, specifically in areas where video is really hard to get. is this exactly what the big labs are also saying when they're talking to that i honestly haven't talked about the big to the big labs like since we started working on them ourselves i think people are more reserved with what they share with us yeah of course it makes sense that's probably a question how would you contrast your version of all models with the lead yeah yeah i'm fluent yeah so i don't know exactly what young is doing today my understanding it's based on Le Fee-Jepa, like Le Jepa approach, which is...

43:37So I'll start with Fei-Fei Li. I think what's really interesting about Fei-Fei Li's approach is that you in some way are able to reuse the spots, right, in game engines and in things that let you stay in verifiable domain, which I think is a really interesting approach. However, my understanding is they're currently not interactive, which in my opinion is like the whole point of world models, right? It's environments. They're great environments. And I think from a business perspective, I think they picked a really important part of the tool chain. But to me, that's not really a world model. But my guess is they'll get there, right?

44:10They'll start generating... Yeah, they just have a reuse thing. Yeah, exactly. And I think Feifei is one of the founders of the entire space. So I think it's going to be really interesting to me on what maybe that interactive piece looks like for me to really judge their approach. I think... We reviewed, just before we moved to young uh we interviewed her with justin johnson uh her co-founder he was he was more focused on the physics side of things and the interactivity and they just haven't been instead yet but i i i do think that basically that the splats if you just add more dimensions on i guess the forces acting on them then then you get to attract you to the out of the box because you are basically these are virtual atoms that then has all the normal physics applied to them yeah Yeah, I'm excited to see what that looks like when they actually release it.

45:05It's really hard for me to comment on anything. I really like the frame-based approach because all of our training data is in this format. Yes. We actually asked them about this, and they were like, yeah, it's possible, but we're choosing this data. Yeah, and you can also go from splat to frames, right? I'm sure you can write at some... It wouldn't be easy. You'd have to actually render out the environment, do the edge to the shirt. it's not it's not going to be a simple problem but like in theory it has to be something that you can do if you really wanted to so like because it's almost like having a more sort of grounds for three-dimensional representation of the underlying world yeah right so i think it's an interesting approach um it might be overkill right uh uh you're also dealing with like a much larger like degrees of freedom on the output space right so so who knows how well it scales i like the fact that like i think these video models also use things like auto encoders you can actually have the world models predict like much smaller um maybe like a yeah exactly and then you can use like diffusion upscaling or methods like this to actually enrich and so i think that world models just allow a much more or world models in my sense for a much more like controlled space that that that we know really well um i'm not suggesting their approach is wrong i'm just you know like this is i think what we really like about it honestly, Jan's podcast that he did, I don't remember which one it was, but a long time ago where he, where he basically proclaimed LLMs to be a dead end.

46:32Um, uh, it was one of the things that inspired me to do this. I think this is very consensus around world models. People, basically everyone that has this is like stops with the LLMs and just goes through to world models. I would say that the main perspective, I asked this exact question to Nolan Brown from Obed-Eyye and do us like, well, B, learning visible models. So it's basically the difference in considering our system B. What do you want to put down here, everyone? Yes. So, yeah, I'm not one to proclaim LMs or dead ends, personally. I think they're actually quite useful, and particularly as orchestrators.

47:07The way I think about it is, as humans, we had sort of a three-dimensional world, then we invented text, in a way, a compression method, right? So we invented text in order to communicate with each other in a common way, in a way that actually compresses all this information that we are perceiving in three-dimensional space into just like a single sequence. And I think that allowed science of streamer, it allowed so many literature, like so many parts of the world that we charge. So I think it's a critical part of the whole picture. I also agree that it's very, very clear that they do build sort of the internal implicit world models inside LLM's.

47:49And so I think they'll be very helpful as things like orchestrators. The problem is when it comes to the generalization, I think text as a generalization backbone, when most of the pre-training is text or largely text sequences, then I think you want that backbone to be kind of more split-septorial in nature and then also just have text as part of that. And I think the actual argument of LLMs is also, for instance, the autoregressive nature of the prediction itself. So the fact that it's running the entire output through the transformer, and then in order to predict the next token, which doesn't, like, the environment in the real world is continuous, right?

48:32It's always changing. And LLMs kind of just forget about that, right? I think a lot of the argument is in the first, right? So I think the fact that text doesn't necessarily generalize well to a sufficient apporal context and then the autoregressive nature of the prediction and using text for that, right? So I think those are the two main arguments. I think text prediction is just one of the actions that is going to come out of these policies and world models. I think speech and text generation will just be one of the actions that can be a part of that. I think that there will just be labs coming at this problem from both sides.

49:13And everyone ends up in roughly the same place. And the same place will be whatever people think is cool. Right? Like whatever the consumer is closest to AGI. Yeah. And so I don't think there's like a clear answer. I think it's really interesting to come at it from the world modeling side. But it's also because we have to. Right? Because like text is largely commoditized. We can import all the text. I think it's interesting and tempting. Yeah. A lot of tempting, it makes sense that you can probably recover. It's sort of like you're taking a step back. You're starting your branch of the ML research sheet, but you might actually just end up recovering all the other tech stuff emergingly.

49:52Yeah, yeah. We can import a lot of that research, right? A lot of that is... That's really cool on the research side. Let's talk about the stuff that GIS is producing more, like the sort of research and products output. you mentioned the word customers what are your turn customers yeah so we're already working with some of the largest game developers in the world yeah we're also working with game engines directly and so really what we're doing at the moment is replacing essentially the player controller inside of a game engine so anything that you're currently that maybe like behavior trees or things that you're deterministically coding we hope to replace with a single api which is just you stream us frames and we predict actions and that can be inside an engine or it can be eventually even inside the real world hopefully those are then also steerable so the models that you saw were text steerable yet but i think we want to get to a point where they're fully text steerable but you see steerable means like well i want youtube to share if you're anything else yeah i think it's it's sex conditioning on the generation so yeah the ability to to you're right We want to get to a point where you can generally, and that's why it's called general intuition, where we can sort of can mimic the intuition of all these gamers into human-like behaviors in any situation.

51:08As I mentioned, also, Lab is named after the Demis' office quote from AlphaFold, which is, wouldn't it be amazing if we could mimic the intuition of these gamers who are, by the way, only amateur biologists on his path to, he tried to get an AI to train Foldit to generate a lot of data for AlphaFold. And so for us, really, the North Star, what we hope to get to one day is being able to represent scientific problems in three-dimensional space and then have a space-in-the-poral agent capable of perceiving that space and using, hopefully, also the text reasoning capabilities that LLAMs have today in addition to the space-in-the-poral capabilities to be able to work on the other side of that problem.

51:43So that for us is sort of the North Star. That's why we're sort of trying to be hyper-focused, Spanish and poor workloads, the same way that Anthropic was hyper-focused code, and use that to then get into organizations and expand from there. Just as a side note, since you mentioned Anthropic, any idea what they did on this to solve coding? No, out of any lab, I probably know Anthropic the least. I admired him, though. Yeah, well, the current working theory is that they had a super lucky role of the ducks. But, well, and then he compounds from there. That sounds like a nice story. I'm sure it's not that.

52:21Yeah. Okay. So why do the game developers want this? So if you're a game developer, how well you're actually retaining players, it's like if you have a game that's already at scale, it's decently dependent on how good your bots are. So if you're logging in at an obscure time, let's say 3 a.m. in America, and your player liquidity is low, then you need really, really good bots to keep those players engaged. Is this a thing? Yeah, for sure. For sure, we're like 4D and whatever. A lot of human works with this, yeah. And so if you're like, as a human, do I want to play against bots? Usually it's not just bots.

52:56It's like players mixed in with bots because you don't want to play just against bots. But it's better to have a full game than to have like an empty game. Yeah. And so I think as long as it's part of the environment, I think it's okay. That means you also have to sort of grade that skill level. Yeah, which we can do. because we know exactly how good people are at these games. Yeah, I think for us, bots is kind of like step one, right? So what I was showing you is we're building a general agent that can sort of play any game in real time. But really that extends into all of simulation, right? Like in GTA V, for instance, people are generally role-playing real life, right?

53:30And so they're actually behaving in quite aligned ways with the goals they set for themselves. So you have all these examples represented in video games, right? You have truck simulator, power wash simulator. Power and Wash Simulator? Yes, Power and Wash Simulator, where actually the behaviors that you'd want an agent to be able to perceive, they're all there. Okay. Yeah, it's really like how seriously some gamers take Truck Simulator. If you haven't seen these tips, you should watch it. Yeah. They buy the whole truck driving set and they're doing the job of a truck driver. Yeah. What I mentioned to you, we have more people at any given time on Metal playing with steering wheels and like Truck Simulator and these types of games than Waymo has cars on the road.

54:10it's a ridiculous stat, but it's true. Yeah, I mean, so, you know, I used to think that, well, to solve self-driving, you kind of just interplay along the GTA 5. Yeah, I mean, it's not bad for this. Yeah, our bet is not that we can zero-shot any of these things. It's just that, like, the next self-driving company can maybe collect 1 % of the data. Because, right, also, for instance, Clip's already self-select into negative events and adversity, right? And so, like, a lot of our data set, because it already highlights, is really precisely what a lot of these companies spend their last 20 % doing.

54:44I think that's the main argument. If you're another company that's looking at what we're doing, I think the thing that people won't understand is that anything that you're currently doing in pre-training, as long as your robot can be controlled using a game controller, we hope that we can move that to post-training for you. So our bet is not that we can create the next self-driving car company. It's just that the next self-driving car company hopefully only needs 1 % of the data or maybe 10 % of the data. I don't know, right? To be able to deliver a really good product. Yeah. It's also the term that comes to mind a lot is active learning.

55:14I don't know if you've used to identify with that. It got less cool for a bit. And now it seems like down the uptrend, which obviously you have the best data set for the high intensity. You said negative, but feeling for negatively, it could be negative or part of it. Yeah, for sure. I think negative events is just because it's the most common term that people use for like, if you're if you're tesla you want the crashes you want like right um yeah right right right but but it's only gaming it's both yeah so you know the model that you saw obviously had really really incredible moments and and that was largely yeah yeah that um uh that it had a large representation of people at their best yes yeah and worst yeah yeah yeah amazing okay cool uh and you have anything else on the customer development side that you want to sort of flesh off yeah um we're also already working with robotics companies but again that and manufacturing but the key is that the robot has to have gaming inputs so we're like our bet is not that we can transfer over to like higher doff robots and the keyboard and mouse it's it's really just that we can move the hard work of of pre-training hopefully to post training yeah it's kind of like the foundation model that is a very good basis yeah you're gonna straight you're gonna give us frames and and likely some text or you'll license the model to because they're gonna want to post training yeah our business model is initially going to be an API, like the Anthropic API.

56:33But you also saw, for instance, some of the video labeling models that we've been able to develop. So the goal is for any company to be able to take in their video data as well. And we can create first, obviously, custom versions of the policy for you, the agent. If that doesn't work, then we've already working with a customer that is doing we distill a model and they turn that into a product for themselves. So people can engage with you on the agent level, API level, people can engage me on the sort of model level can you also buy data no all right yeah we don't sell data okay cool so that's the that's the business um and is there a world in which i mean i i think this is on your landing page if you are you know frontier labs for for world models is there a world in which there is a more sort of application layer thing that you that comes out like a chat gpt for whatever yeah you're gonna see us launch a few things on on metal itself that are going to blow your mind uh as a result of this this um this agent so i'll leave the imagination for now if people took a great out you know and yeah on the world modeling side like i think one people underestimate is that metal is already one of the largest you know video consumption platforms as well people watch millions and millions of videos a day um so um world model based entertainment and things like that well it's not like a focus for us right now i think we'll be like on the consumer side we have the ability to move very very quickly here and get it integrated in a way that I don't think anyone else can.

57:59Yeah. You could theoretically do a video gen, like the Sora. What is that? It's a BAM. What's the meta one? Meta and Mules? Not Reyes. Oh, the vibes? Yeah. You could theoretically generate clips that nobody play. But you know it's a device. Yeah, I think for us, the games being so human-centric is like a really big part of what makes it special. like I actually I actually just don't think that would work like one thing that we are really excited about though I'll give you one sneak peek of what we're thinking about is what if you could literally replay any of the clips that you have inside a world model or your friends can play them like I showed you a model that already took part of your clip as a contact since the replay entered at walls but it's also how we go from imitation learning to rl right because like it's part of a research rope app anyways to make every single every single clip on that all playable um so uh yeah who's who is to say that that doesn't apply to just the actual clips that you take yeah yeah just can you see more with the rl potential we describe metal as as the episodic memory of humanity and simulation so when you take a clip really the way to think about it is you get the highlight of what is maybe three hours of playtime right you maybe get like two to three minutes of the things that were the most out of distribution right it is genuinely your episodic memory um of that playtime and simulation the things that you most want to remember and share we want to be able to load uh and this is the work that anthony who is doing the reason why we built world models is every crash that you run into in your truck simulator or American truck simulator or a driving game, we want to be able, right?

59:33And again, these are ground truth labels. So we know precisely the actions that lead up to the negative events. They're also title labeled when people upload it onto their platform, they say, oh, good, it's a crash, right? And so we can select all these events. And if we can put them inside a world model, we can go into, right? We can train reward models to then reward based on how you perform in clips that actually contain negative events, for example. And so for us, it's very much about, right, we can create this LLM moment on being an invitation learning, but actually making every single clip on the platform playable at billions of clips scale is how we go from invitation learning to RL.

1:00:09Cool. We covered a lot of it. Is there anything else that you want to do before we wrap up the Lulz and Vision stuff? Yeah, I think for us, this is a very, very ambitious long-term vet. We need the best researchers in the world that want to work on this stuff. It's really exciting not being extremely data constrained. Like we really get to, like we get so many learnings every week that we didn't think were possible and it makes it for a joy working here. Also, the other thing is because we have such a large data mode, we don't have to be as concerned as the LLM companies about publishing because we don't need the ones to be able to.

1:00:44Exactly, no one can replicate the models, right? And so for us, we really want to bring back the original culture of open research, which is why we did the partnership with Qtai in France. I actually didn't. Yeah, we just announced our partnership with Qtai in France, which is an open science lab in Paris, one of the best research labs in the world. Eric Schmidt, I believe, funded in addition to some French people. They are essentially acting as the partner that's currently doing a lot of open research on the data. We also want to partner with universities who, because like we do believe this is the frontier, but it's so data constrained that really everyone has their hands tied behind their back right now.

1:01:21And so we want to help fix that. So for instance, we want to work with universities to build like negative event prediction models for maybe like trucks in India on all the truck data where all these crashes occur. We have all these things that we know we can do that we just have it at the time to do. And so if you're listening to this and you're maybe an academic institution or something and you want access to some of this data in an educational research fashion, I think we're quite open to doing that because we want to educate people. And yeah, and other than that, we just want to work with the best infrastructure and research engineers on the planet as we're going into scaling, you know, runs that have thousands, thousands of thousands, eventually hundreds of thousands of GPUs.

1:01:58Yeah, yeah, amazing. I primed you this as the closing question. It's a little bit of a no cost of that, I didn't know. So what does GR become by the way? Yeah. In 2030, we want to be the gold standard of intelligence. And any sequence long enough is fundamentally spatial and temporal, right? Which I think is... So by nailing spatial and temporal reasoning, you go after the root killer problem of intelligence itself. What the world looks like is we want to have eight. So I sort of group the sequences of AI in three stages. And I credit Andre Goparty for teaching this. Bits to bits, atoms to bits and bits to atoms, and then atoms to atoms.

1:02:36In the atoms-to-atoms stage, I want GI models to be responsible for 80 % of all the atoms-to-atoms interactions driven by AI models. And the reason for that is because we were able to unblock intelligence so quickly, and robotics like intelligence is the bottleneck, that supply chains actually converged on gaming inputs as their primary input methods. And they converged on essentially simpler systems that let us do a lot more, a lot quicker. So we are essentially the 80 % market approach. And then you have lots of companies that have kind of like specialized, maybe humanoid robot OS stacks that are the other 20.

1:03:11And then so I want to be responsible for 80 % of all the atoms-to-atoms interactions driven by these models and be the goal center for intelligence and maybe 100x more in simulation. Because I think simulation will actually be the larger market initially. So I think in simulation, because you have very little constraints, also from a safety perspective, simulation is much easier. so i think a lot of the takeoff initially sits in simulation so a lot of the simulation use cases like what i mentioned scientific use cases i'm really really excited about and so um yeah 80 of atoms-to-atoms interactions uh coming downstream from these types of spatial liberal foundation models and then 100x more in simulation yeah it reminds me a lot of that uh what mark and from this chas zekhberg is to are doing with virtual biology because you can do a lot of simulation and you can do yeah or you can do it a lot faster with interest um amazing thank you for inviting us to your office yeah and thank you for sharing a little bit while you're turning thank you yeah