Nvidia’s New Rubin Chips & Self-Driving Tech, Amazon’s Tough Sell for AI, Energy Boom | Jan 6, 2025

0:12Welcome everyone to the information's TITV. My name is Akash Pasricha. It is Tuesday, January 6th. Today on the show, we are unpacking CES. We will break down the big announcements from NVIDIA. We've got Kindred Ventures coming on the show, and our AI and robotics reporter is going to help us understand NVIDIA's new self-driving car model. We'll then turn to some exclusive reporting, the information published this morning. AI evaluation startup LM Arena has raised$150 million at a$1.7 billion valuation. We will get more details on that. Next up, we'll turn to a story from our Amazon reporter about AWS's AI ambitions and how its homegrown models stack up against its competitors.

0:58And we will wrap with the closest look that we have taken yet into the different pockets of the energy and industrial sector that are changing their strategy to adjust to the AI boom. It's going to be a fun show, and so let's get right on into things. At CES yesterday, it was all eyes on NVIDIA, as Jensen Huang took the stage for his keynote. Two of his big announcements from the event were NVIDIA's model for self-driving cars, and more details about Rubin, the company's next big AI chip it is working on. The chip is supposed to be cheaper to use compared to previous models, And in some cases, companies should have to use less of them.

1:37I want to bring on Steve Jang, founder and managing partner at Kindred Ventures, to help us break it all down. Steve, welcome back to the show. It's great to have you here. Thanks for having me. It's good to see you. So you are on the ground at CES. Were you at the keynote yesterday?

1:53Yeah. CES every year is a great moment for all of our hardware companies and compute companies to come together. So CES itself has a new life because of AI, which is great to see. Nice. Well, I want to talk to you about the keynote. So look, the two big announcements coming out of Jensen Huang's speech yesterday were the self-driving car model. And, you know, I know you're close to that space. And so maybe we'll get there in a second. But I want to start with the new generation of chips that they gave us more details on the Rubin family. What stood out to you about Jensen's speech about the chips?

2:32And, you know, I kind of thought that we knew a lot of this stuff yesterday, didn't we? Yes and no. I think everyone knew that the Vera Rubin Superchip and the Rubin GPU were coming, but I think some of the details weren't widely understood yet. I think there was a couple concerns coming in. One was, you know, is this going to be a drop-in chip to replace the B200s? and GB200s, and it looks like it's not, right? So what you saw yesterday was a Vera Rubin super chip, a CPU and two GPUs in a node, and then Rubin GPUs. So these are sold and delivered as nodes and racks. You can't just take a single GPU and replace your existing chip.

3:17So in the data center, you actually have to change how your rack system works. So the RAC is a system in this new Rubin series. So what you get, though, is 10x inference efficiency. You get 3x to 4x training efficiency. But it's going to require some work on the clouds and neoclouds part. So it's a pay-to-play, right? They have to compete with paying up for a new series while they're still figuring out how to get delivered and monetized on the Blackwells. And so this is a continuation of the battle royale that the hyperscalers, the neoclouds, the inference engines, the frontier labs are playing right now.

4:01They have alliances, pre-commitments, bookings, and they're still figuring out how to utilize the last series of chips. So I think 2026, one of our predictions for 2026 is that this battle royale is going to be even more intense than last year. many more bookings, many more alliances that you'll see. And they're going to have to fight for customers and still monetize and utilize all those B200s sitting in data centers and still getting shipped to them. So the net though is that it's great for inference engines. It's great for frontier labs. It's great for Neo labs. It's going to be expensive for Neo clouds and hyperscalers.

4:44But, you know, long term, they're going to be able to have the very best chips compared to anywhere else around the world. Well, let me ask you this, this idea that you have to sort of replace the entire rack and it's not a drop in chip. Does that come across to you as a strategic move from NVIDIA in a way to sort of ensure that customers are buying more of the whole system from them? Or is this an innovation in the sense that you couldn't actually get the benefits that it wants you to get without buying all the other stuff around the chip? My understanding, and we've just had a sort of day to look at this, but my understanding is that it's not a business decision.

5:28It's a technical systems decision that this is how the Verirubin nodes in Raxwell work. I don't know if you saw the picture of it. It's a golden monolith. Yeah. Beautiful. My comment at the time was, hey, looks impressive, also looks very expensive. I mean, that's why they colored it in gold, right? And so I think this is a system as a rack solution. And so I don't know that it's a pricing or a business decision so much as it is. This is the new rack system and node system that they want to push out there. And that's a technical requirement. And so, well, you know, we'll find out more in the coming days.

6:08But I think this is really interesting because it's a replacement cycle. It's like the same way that consumers replace their new, their iPhone for the next iPhone. Right. Their old iPhone, their current iPhone still works, but they want the next one. And I think basically the entire industry, the compute industry is essentially on the iPhone. The NVIDIA replacement cycle. Yeah, you don't need it yet, but you'll need it soon. And so you should get it now and book it now. Otherwise you might be waiting in line and then a slip on your competitive race with others. So I think - I will say it's funny to me, I was watching the live stream and I was on YouTube and wherever I was watching it, the comments underneath, the top comments were, this is the consumer electronics show.

6:52This is like, you know, one of the least consumer things, you know, and look, I know you say, all of AI rests on the chips, right? And at the end of the day, it all comes back to it. But some people were saying, gosh, you know it's getting more and more technical and and uh in terms of what's what's being announced at these events I mean CS has always been a mix of some consumer devices um and a lot of infrastructure uh the server companies the chip companies uh the component companies have always been out here this is sort of the it's always been sort of the the the trading uh order filling um uh convention from a long time ago.

7:31But it's, you know, it's a thousand X now because if you think about all the keynotes, like very few of them are actually touching the consumer, right? You saw the AMD keynote, you saw, you'll see a Siemens keynote, you'll see so many keynotes from companies that are two, three steps away from the consumer, but it's super important, right? The, you know, this whole Vera Rubin series that they announced, it's built for a future around consumer agents and enterprise agents. That's why they built this. The prediction is, is that over the next year or two, inference workloads will exceed training workloads, which will be the first time that's ever happened.

8:09And so this is made for agentic workloads. The number of requests that you're making on an inference level requires a tokens per second, per watt calculation that we just haven't seen in the industry. So as companies like Perplexity and Cloud Code, you see what they're doing with their harness on top of their Opus models, codecs, things like that. These agent platforms are going to use up so much inference that you need a different chipset and a different system. And so – Let me ask you about – consumer workloads was definitely a focus yesterday, but a lot of it too was physical AI, right? I mean, manufacturing, robotics, and we had a good story about that yesterday too, how NVIDIA is really gunning for this category of customers now with its Omniverse suite.

9:05What do you make of the focus on physical AI? Because it wasn't just NVIDIA yesterday. I think other ship companies as well announced more products targeted towards these sort of industrial applications. Why is this such a big focus now? Is this where they have to look for growth? No, I don't think the narrative is that they have to look for growth. And so they're sort of forcing this theme. I think that this is a theme that's been brewing. I mean, if you think about what a chip maker and Intel used to do this a long time ago, but what NVIDIA is doing is sort of setting the mandate for the conversation and providing the tools for that conversation and that building to happen in a sector.

9:50So essentially, it's sort of like kingmaking, right? You know, we often talk about - They're setting the tone. Yeah, do venture funds kingmake companies? Well, I'll tell you, NVIDIA is kingmaking sectors, right? And so what you're seeing is that they're, you know, they did this with LLMs, right? They focused in on LLMs and, you know, had great things to say about not only open AI, but also XAI and Elon Musk's ability to stand up a data center very quickly with lo and behold, their chips inside. And then they king made on the robotics sector, right? And you saw that with Groot and everything that they were doing there.

10:27And now when they say physical AI, it includes robotics concepts. It also includes VLAs, right? And VLMs. And so they're not, these are derivative technologies that are improving over time in the industry. If you went to any of the ML conferences over the last year, like ICML, ICLR, NeurIPS, You saw that physical AI was a huge thing. And so world models, everything from world models to robotics models to autonomy is part of physical AI. It's very much the narrative that people want to attach themselves to right now. Yeah, and if you saw what they announced yesterday, they have open data sets. They have open source models, right?

11:07They showed that demo video. And we know this well because one of our portfolio companies, Neuro, is an NVIDIA partner. is invested in by NVIDIA and Uber, and they launched their Neuro car with Lucid and Uber. And it's AGX store inside, NVIDIA chips, right? And what NVIDIA basically shipped yesterday, very interesting, is that other auto OEMs and other technology companies can use these models and their chips, right? Their chips are always inside. And this accelerates the ecosystem so that using open source, which I think there's been an argument that what is the purpose of open source from a monetization perspective?

11:49Well, in this case, it sells chips. And then the second part of that, of what I thought was super interesting is that they've basically offered a hardware model and software kit to create your own Tesla FSD or your own Waymo. So, you know, when they're - It's like anyone can, you know, it's very much democratizing who can develop on this technology. I think that's a good point. Actually, I hadn't thought about that. But very quickly, after all these announcements yesterday, the stock didn't really move. And look, NVIDIA stock is kind of a tough thing to predict. But what did you make of that part of this?

12:29Yeah, from a public stock perspective, I think a lot of the Rubin stuff announcements and even the physical AI, there's been data out there, right? uh neuro's been using uh thor um and with uber uber was announced as a nvidia partner as well so a lot of this stuff is already probably factored in to a wall street's pricing but the other thing that's interesting here is that uh you know if you look at nvidia today uh they are pushing forward an agenda uh against the narrative that is is there a bubble and there's another counter narrative, which is, is NVIDIA's throne at risk because of TPUs from Google, or is it at risk because of inference chips?

13:14And what I'll tell you is the Vera Rubin family, that series of chips, it's 10x improvement in inference. So it kind of calls into question, what was the narrative that was floating around, around the Grok acquisition? And I still think that, you know, we have some... Does it make you question the Grok acquisition a little bit? No, it doesn't actually, because I think the Grok acquisition, you know, we'll see if they actually use the Grok chip license in their product suite. I think if they do, it's, you know, it's heavily modified to fit into NVIDIA's suite. But what I think it was, was getting one of the world's best teams in inference.

13:54You know, we, when we looked at that, it was like the SRAM innovation that Jonathan and team had achieved. That's just, there's very few people on the planet that know how to do that. And I don't think that's a, that's a, that's, that's not in the wheelhouse of NVIDIA. So I think that was essentially a$20 billion talent acquisition, right? Which is very much what they, what they kind of call it by, by doing everything but the acquisition. I mean, they didn't call it an acqui-hire, but. I mean, regulatory, like, FPC issues aside, it's a great talent acquisition. and 20 billion is a lot for any company other than NVIDIA, right?

14:33And so when I look back at the Grok acquisition, they didn't take Grok Cloud, right? They have DGX Cloud, so they don't need that. And I think that they're going to build a new inference chip that's focused on tokens per second and it's not focused on training, obviously. And if you look at what they're offering here, it's saying you can pay to play here and you can get a much more performant inference and training chip in Rubin. And we're probably going to offer you around the same time, maybe a little bit after, an inference accelerator that is single focused on that outcome and that use case.

15:13And so, you know, I think as this unfolds, I think there may be a little bit more upward pressure on that price, but I think people are still grappling with a couple of these narratives. And so we'll see how that plays out in the future. But, you know, the Rubin chips aren't coming out for a while either. Well, and I think we were all excited to get more details on it yesterday. And we didn't have time to get to the self-driving car stuff. But we're going to talk about that in our next segment with our AI and robotics reporter. Steve, I want to thank you for joining us. Have fun at the rest of CES and enjoy the rest of the week.

15:48That is Steve Jang, founder and managing partner at Kindred Ventures here on TI TV. Okay, sticking with our CES coverage, I want to bring on our AI and robotics reporter, Rocket Drew, to help us dig a bit deeper into Alpameo, the self-driving car model that NVIDIA unveiled yesterday. He wrote about that in our AI Agenda newsletter out today. Rocket, welcome back to the show. It's great to have you here. Hey, Akash. It's great to be back. So we just talked about the chips that NVIDIA unveiled or gave us more details on. And I want you to explain to us what exactly Alpameo 1 is, because it's Alpameo 1.

16:28That's right. What is this model supposed to do? Yeah, it's a model for self-driving cars. So it's trained on a bunch of data that people have collected by driving their cars in the real world. And then a lot of data besides that, that's sort of generated in computer simulations using NVIDIA's own computer simulation technology, in particular using Cosmos, which is their sort of mainline AI world model. And the model is intended to drive cars. It can predict what action a car should take based on a state of the road, based on an input image. The model itself is like, it's pretty big. So if anyone was going to use it in practice, you'd typically customize it a bit on your own data.

17:11Maybe you distill it into a model that's smaller and faster so it can run in the car itself, but it's intended to accelerate the development of self-driving technology in a number of ways. And so how does this model compare against other self-driving car models? I mean, we know that Tesla is working on their own robo-taxi suite of cars, I guess. I mean, presumably they have their own model. I don't even know, is this a very competitive landscape? You know, I think a lot of it's locked down and very proprietary. So it's difficult to say what Tesla is doing. I mean, Tesla has made a pretty firm position that they're interested in using camera inputs only.

17:55So they want their car to be able to maneuver the world solely based on vision rather than using additional sensors like LiDAR to estimate how far away different objects are. And then the biggest difference, I think, is that this model is open source, so anyone can download it, anyone can tinker with it and experiment and customize it, and that makes a big difference for the model. It's also not intended to provide so-called level four autonomy right out of the box. The intention isn't that you can put this in your car tomorrow and you can stay home while your car drives around for you. At best, as they're starting to roll this out, working with Mercedes-Benz, it's going to hit the road at level two autonomy, which means...

18:37And the difference between two and four is... It can tweak your steering and your acceleration, speeding up and slowing down, but you've got to be vigilant the whole time. You've got to be ready to take over and take the wheel if anything goes wrong. Got it. So you wrote about this being an end-to-end model. What does that mean? That's right. So the traditional approach in a lot of robotics, including for self-driving cars, is you have specialized pieces of software that are governing everything the robot, or in this case, the self-driving car needs to do, from perceiving the world and making sense of it, to planning what actions to take, to controlling the robot or the car and taking those actions.

19:17And you'd have a separate layer of software that handles each of those. That has some really nice properties. Like, you can tell when something's going wrong, I know it's going wrong with the perception part of this process, and you can see the flow of information from one to the other. But with the rise of AI, it's very in vogue these days to just have a single unitary AI model that from pixels coming in to actions coming out controls everything that the robot or the car does. And that has some really some real advantages because, you know, AI models are so expressive and powerful. They can capture all of these difficult to define rules and they can learn from data in subtle ways.

19:56On the other hand, now you're letting some black box that you can't really monitor that isn't very transparent, control your car. And that's a very high stakes kind of robotics, right? That's a very serious deployment. So you don't actually, if something goes wrong, you don't know what part of it actually went wrong because it's not as segmented. Exactly. And something, you know, it's always going to encounter some scenario it's never seen before. You're always going to come across something on the road that your model didn't see during training. And you want to make sure that it's going to be properly.

20:27Do you want to share the example that you gave in your newsletter i had i had santa con on the mind i'm thinking you know if one of these self-driving parade of drunken santas is the obstacle that rocket decided he was gonna right he was gonna put in front of the training data you know you know that nvidia isn't simulating those drunken santas during training right so i mean i i do want to ask you though i mean so there are pros and cons why do you think nvidia went with that end-to-end approach then was it really just a way to differentiate itself? Is it a way, is it a business decision? I mean, you're saying it's open source.

21:04So, I mean, presumably there's a lot more there that people can tinker with. I mean, walk me through. Yeah, I think it really showcases their simulation software. I think that's really how you get a lot of juice out of this Cosmos model that they've put out and they're trying to drive adoption of it. And then I think they want people to spend compute. To Steve's point earlier, NVIDIA wants people to spend compute on actually running this model. And I think that makes most sense in this kind of end-to-end paradigm as well. But NVIDIA does something clever, right? They don't just rely... When they're actually putting this model out into the world, when they're working with Mercedes, they're not relying just on the end-to-end model.

21:42In parallel, they're running the more traditional stack of perception and planning and action. And both systems at the same time are deciding what action should we take. And they have sort of, what do they call it, like a safety policy evaluator on top. It's a fancy way to say they have another system that's deciding at any given moment, which one should we listen to? Do we trust what the black box model? So it decides. You either go with the new model or the older technology. Right, right. Based on how confident the models are, based on are we in a novel scenario that the model hasn't anticipated.

22:16And that adds a level of safety to it because you can guarantee that we'll fall back on the safer model in the event that we run into some exceptional circumstance. Right. I want to shift to talking about another scoop that you published today. you and Katie Roof, our Deputy Bureau Chief of Venture Capital, had a story about LM Arena. And I gave it away here in the heading, but the reporting showed that it's valued at$1.7 billion in a new funding round. Tell us a little bit more about why the LM Arena was able to raise all this money. Yeah, absolutely. I mean, LM Arena has become a go-to place to compare the capabilities and performance of models.

22:58I can tell you, I talk to people in AI all the time that look to Elam Arena for a sense of how different models are performing to gauge the quality of new releases. It's really become a source of common knowledge for how different models perform at different tasks, from answering questions to generating images to generating videos from images, you name it. Elam Arena is creating a category to compare models. How does it make money? So it has a small army of people who, for free, are comparing models around the world and telling you, you know, ChatGPT is doing this and Claude is doing that, that data that Elmarino collects is really valuable.

23:39I mean, that's a super valuable signal for AI companies to use to improve their models. So Elmarino works with AI companies, both the model developers and bigger businesses that are working on deploying these AI systems. And it gives them custom evaluations. Like a company comes to them and says, you know, we want to know how well it's doing at a certain category. And thanks to LM Arenas, the data that they've collected and thanks to the sort of free workforce that they have, they're able to provide really sort of useful evaluation metrics and compare. It can tell you ahead of time, this is how your model is going to stack up.

24:15So it's basically the, I mean, it has this public benchmarking system for the public to see, but then privately, you know, it can get paid by these labs to basically give feedback on how good their models are. Exactly. And, you know, I imagine there's like a wall between the two and saying, hey, the feedback that we get, you know, like, they'd have to really say we're not picking favorites, right? And, you know, we're closer to this lab and this lab. We have more business from this lab. And it would really have to be a wall in saying we don't mix the two. Yeah, you know, and they've come under some criticism for the practices related to this in the past.

24:59There was a paper that was published a while ago that was headed up by Cohere. And Cohere was criticizing that it seemed like Meta, in advance of releasing one of its LLAMA models, was able to sort of pay to play. And I think the way you're pointing to. They could submit a number of checkpoints for the model that were customized in different ways, they could see which one performed the best, and then they could go with that one. So on the day the model comes out, it seems like it's performing very well against the others. Now, Elmarina has responded to this extensively and has explained their practices and how they go about it, but you also might say, you know, what's the harm?

25:34You could say, well, sure, Meta is willing to pay more than other customers, but the feedback they're getting, if it's high quality, is leading to the best version of Llama that they could possibly release. So that's a position you could have on it as well. Right. Well, Rocket, I want to thank you for coming on. That is Rocket Drew, our AI and robotics reporter here at The Information. Okay. Amazon's AI team has been in a stressful spot lately, trying to make sure that its own models are up to par with the many outside models the company has on its platform. That is the subject of an in-depth story out today from Catherine Perloff, our Amazon reporter, and I want to bring her on to tell us more about what she learned.

26:14Catherine, welcome back to the show. It's great to have you here. Hi, Akash. So we've talked a little bit about Amazon's Nova models on this show. It's not the model that we really think about first when we think about, you know, the leading companies, and yet this is AWS. So how good is the Nova model family? It's a loaded question, but let me know what you found. Yeah, you know, I think that Nova, they just released the second generation of models at the end of last month. And Nova has been improving on a lot of metrics. And in some metrics, it's even better than Anthropic or OpenAI. But on a lot of metrics, it still ranks behind those big companies.