The Big Leap: Nvidia’s Cosmos World Model

0:00Nvidia has just rolled out a brand new world model that is fascinating. They've also pulled out a whole bunch of really interesting infrastructure plays and other interesting announcements at a really big announcement they just made. So today on the podcast, we're breaking down everything that NVIDIA has just announced, what their new Cosmos world model is capable of doing, why some people are saying it is an absolute game changer in the industry, and what some hype people would say about all of that. Before we get into all that, I want to mention if you want to try all of the latest AI models, including some from NVIDIA and other players, I would love for you to check out my own startup, which is called AIbox.ai.

0:39It's$19.99 a month, and you get access to 40 of the top AI models. You can try all of them side by side, and you get access to image, text, and audio models. So instead of paying subscriptions to 20 different platforms, 20 bucks a month, you just can pay one subscription and get access to all that. You can chat with the models all in the same thread, so you can switch between what model you're talking to in the same chat feed, which is super useful, and you get, of course, the image and audio models as well in there. So things like 11 labs or ideogram for image generation, some really cool stuff.

1:09So if you want to check it out, there's a link in the description. But let's get into what NVIDIA is doing. So they made a bunch of infrastructure for robotics, which I think is kind of cool. Cosmos is playing an interesting role in all of these new announcements. But basically, the big thing is this new thing called Cosmos Reason. So this is a 7 billion parameter model. It's a reasoning model. And it is a vision language model for physical AI applications and robotics. This is what I am incredibly excited about, is the fact that these new models that are coming out, this isn't just another LLM for someone to chat with, but we're actually building tools specifically built for robotics.

1:48And we know there's a lot of players making these robotics. There's a lot of these robotics startups that have raised a lot of money. We have Optimus from Tesla. We have Figure that's making some big strides. We have Boston Dynamics, which is doing some really interesting stuff. And so NVIDIA making a big focus, I think, and making these kind of announcements and building models for these, I think, is a really big signal that they see a huge ramp up in this in the very near term. And they want to already start before, you know, we like we don't have any of these AI robots on the mass market right now.

2:18I guess there's, you know, AI robots inside of Amazon factories, those types of things. But nothing that's, you know, a personal use robot that a lot of these companies have been touting that you could see, you know, billions of these things being sold. We don't have them out yet, but the fact that Tesla or that NVIDIA is already starting to kind of put out models that would address those, I think really signals that they expect these to be coming out shortly and they want to really position themselves as the best place for these tools is where they're putting their investment. And so I think that that's a good signal that we're going to see some really exciting innovation in that space in the very near future.

2:51Okay. So also joining this kind of existing batch of Cosmos world models is one called Cosmos Transfer 2, which basically can accelerate synthetic data generation, right? So it's able to really quickly create this AI generated data. And it's doing it from 3D simulation scenes or basically what are called spatial control inputs. and it's basically a distilled version of Cosmos Transfer that is more optimized for speed. This is really interesting. So I mean, basically what we're talking about here is the fact that these AI models, they're creating synthetic or like fake 3D environment simulation data.

3:30And why is this important? Because this is literally what you need for these AI robots to train. At this point, you could train an AI robot how to walk through a house by, you know, maybe putting a a person putting a full body suit on and walking through the house themselves and looking around and like you could get some sort of cameras on them and you get some sort of data points from doing something like that, right? That's obviously not super scalable. And you could 3d scan the house or whatever. But if you just have an AI model that can produce 3d environments, 3d inside of houses or buildings or warehouses or whatever, and create massive data sets, that's the fastest way we're basically getting that data to help train robots.

4:10Okay, so during all of this announcement, this is at the SIGGRAPH SIGGRAPH conference. NVIDIA said that these models are meant to be used to create synthetic text, image, and video data sets for training robots and AI agents. Very exciting. Cosmos Reason, according to NVIDIA, basically lets robots and agent's quote unquote reason. All of this is thanks to its memory and its physics understanding, which is really important. We're getting beyond, you know, an LLM trying to be a really good writer. And now it has to be really good at physics, because if it's in a robot, the robot has to understand if I drop this ball, if I move this pan, like what are the repercussions of every movement and action that I take in a physical world?

4:57It's such it's a very interesting space that these robots have to start thinking about, basically. So this new model, the Cosmos reasoning model, is basically, quote, it will basically, quote, serve as a planning model to reason what steps an embodied agent might take next. The NVIDIA says that it can be used for data curation, robot planning, and video analytics. The company also unveiled a new neural reconstruction library, which includes one for a rendering technique that basically lets developers simulate the real world in 3D using sensor data. So very, very interesting. There's so much that was rolled out.

5:43There are also some new servers for robotic workflows. NVIDIA has something called the NVIDIA RTX Pro Blackwell server. And basically this is a architecture. It's a single architecture for basically as you're doing this robotics development and kind of doing these robotic development workloads. NVIDIA already has something called DJX Cloud. It's a cloud-based management platform. And so this is kind of something that will go along with that. All of these announcements come as NVIDIA is really trying to push further into robotics. I think it's looking towards what its next big use case of the GPUs they have are.

6:22Nvidia has played an incredibly interesting game, right? Where basically they came out and everyone knew them as, okay, this is to make laptops better or computers slightly better. Some of their hardware, I mean, to the layman, that's what they would expect Nvidia did. It became, okay, well, this is a kind of a performance thing. Things like gamers, or if you need a really hardcore computer setup for maybe doing 3D modeling or something like that, you're going to want Nvidia chips. Nvidia then kind of took the next step, which was crypto, actually. crypto mining, NVIDIA had a huge windfall as kind of the whole crypto craze was taken off.

6:54And just as crypto was kind of entering what a lot of people called crypto winter and was slowing down a lot of the big crashes a number of years ago, a lot of people said, oh, NVIDIA's like stock is going to completely tank. There's not so many crypto miners buying this. And all of a sudden, AI popped up and was kind of the next thing that NVIDIA really was poised to create chips for. the H100 being, you know, the very famous one that has powered so many of the AI tools we use today. And they really have been riding a high. But a lot of people are saying, hey, what happens next? How do they grow their company bigger beyond just training LLMs?

7:29And it feels like robotics is the next frontier. There's going to be such an insane demand. And NVIDIA is already trying to position themselves to look at that as they're kind of going beyond just AI GPUs and beyond just AI data centers, right? At some point, they're like, if that well runs dry, what's our next thing? And it feels like it's definitely robotics for NVIDIA. So what a fascinating time to be alive. There's so many exciting updates. I'll keep you guys up to date on everything NVIDIA rolls out as you start getting integrated into actual robots and what we're seeing in that space. I'll keep you up to date on all of it.

8:00Thank you so much for tuning into the podcast today. Hope you had a fantastic time listening to the episode. Make sure to check out AIbox.ai. And if you enjoyed the episode, I would love it if you could leave a comment, review, subscribe, wherever you get your podcast. It helps the channel grow a ton. Thank you so much. And I'll catch you in the next episode.