Nvidia GTC Preview, China’s SuperApp AI Advantage, SaaS’ AI Contradictions, Data Center Hacks

0:13Welcome, everyone, to The Information's TI TV. My name is Akash Pasricha. It is Monday, March 16th. We are kicking off the show with NVIDIA GTC this week. We have a preview from one of our reporters who will be on the ground at the conference. He will tell us all about what he is expecting from CEO Jensen Huang's keynote. We're also taking a look at the U.S.-China AI race as big companies in China quickly move to bolster their technology in that arena. We'll then take a look at the SaaSpocalypse, this time from the perspective of what big enterprise software companies are saying quietly in their securities filings about it.

0:51And we'll wrap the show with our latest AI infrastructure column, where my colleague lays out five industrial-scale hacks that companies are using to gain an edge in the data center race. It's going to be a fun show, so let's get right on into it. GTC is upon us. NVIDIA's flagship conference is starting today, and Jensen Huang is set to deliver his big keynote. We have been watching for updates on NVIDIA's new family of chips and waiting to see what the company will unveil. Here to talk about all that we are expecting is Wayne Ma, our NVIDIA reporter. Wayne, it's good to see you. Hey, Akash. Thanks for having me.

1:27Okay, big week in the tech sector at large. NVIDIA is the center of it. GTC, I mean, gosh, this was a conference that I feel like I wasn't hearing a lot about a couple years ago, and now it's like a Super Bowl of sorts. Let's just talk a little bit about what it's become. Yeah, so the conference started in 2009. And at that time, there were maybe only 1 ,500 attendees. It was considered kind of a niche conference for high-performance computing. And of course, you know, since the release of ChatGPT and the surge in demand for AI, it's become like the world's biggest AI conference. And this year, we're expected to see more than 25 ,000 attendees.

2:03And it's not just about how... Initially, it was aimed for developers to encourage them to use NVIDIA chips. Now it's basically promoting AI as a whole. So it's developers, customers, I imagine these partners of NVIDIA that are helping them get their chips out there. Tell me a little bit about what we are expecting in the keynote today, and then we'll talk a little bit more about what else is coming later this week. Sure. So Jensen famously gives his keynotes in this famous leather jacket. So you didn't expect that. But of course... He does everything in the leather jacket, Wade. I mean, let's be honest.

2:38Fair enough. And he's going to announce a new chip based on the technology that they licensed from Grok late last year. And that's going to be geared toward inference. And this is a big deal because previously NVIDIA has kind of promoted more general purpose AI chips. And now it's going to specifically focus on this thing called inference, which is basically running the models. And that's because more and more people are using AI and that's become more important than training models themselves. He might announce new networking technologies that might link non-NVIDIA chips together and make them fast and work together in unison.

3:11And he'll probably talk about Feynman, which is the next generation AI chip that's probably going to debut in the next few years. So this new chip would then be the generation after the Vera Rubin family? That's correct, yeah. So Vera Rubin, he'll definitely talk about Vera Rubin because that's coming out this year. And then Feynman is the next one. And usually they talk about the chip roadmap like one generation ahead to prepare customers and partners for those chips. Right. And it is kind of interesting to me because here we've reported from our newsroom about how long it takes to implement these new chips in servers and in racks.

3:52And we've reported on Blackwell and how long that took. Ruben, I mean, it hasn't even really come online yet. That's still going to be a journey, really. And so here we're already talking about years out, you know, what's expecting. Yeah, so I mean, I think there's a big focus on how can NVIDIA get these chips ready faster and narrow the gap between the software and the hardware. And so that might be a big focus of the event as well, and especially using AI to do that. Tell me about the Grok chip. What do we know about how competitive this chip would be? Because we've also talked on the show about the other chip developers that are making a bid against NVIDIA in the inference category.

4:33How good is this chip? Do we know? Yeah, I mean, it's supposed to just be very fast at imprints. And I guess the idea is that Grok has some sort of unique technology. It puts a certain type of memory on the chip in a way that speeds up these kind of calculations in a way that NVIDIA's current AI chips don't do. NVIDIA has kind of shied away from releasing chips that specialize in one particular thing because they get better margins, I think, from a more general-purpose chip. But I think the whole industry is shifting towards that, and NVIDIA customers are trying to make their own chips like this, so NVIDIA might as well follow that trend and Tidal Wave as well.

5:10And then the networking chip, this would be in competition with a company like Broadcom? Yeah, that's right. I mean, NVIDIA has its own technology called NVLink that basically links multiple clusters together, so they work together in unison. That's how you can get the scale necessary to train large models and even do inference on them. And I think they're going to announce a way to, you can use other people's AI chips, but connect them with NVIDIA's communications networking technology. And so I think Jensen feels that, well, I just want you to use something from our technology. It doesn't have to be our AI chip.

5:45You can even use our networking, that's totally fine. And the networking chip, I mean, that was, from what I understand, that's a result of the Mellanox acquisition that they did a couple of years ago, right? Well, this is kind of an expansion of that. So it's from the team that they acquired from Delnox, that's a whole new kind of check. Right. So you're on the ground this week. I wonder what questions you're going into the conference with. What are you trying to find answers to? I think it's trying to really understand like how trends are shifting. So, you know, when Chanty PT first came out, that first year of GTC was about large language models.

6:18The next year was about agentic AI, you know, AI agents that are kind of doing things for you. And I think, you know, this GTC will be about physical AI. You know, how AI is going to end up in all of our lives through robotics or self-driving cars and things like that. And so it's more about the manifestation of AI. And so what I want to know is like, well, what's next after that? And so that's the big question I'm going to be asking. Now, let me ask you this. Is physical AI, is that a focus of the event because NVIDIA is putting the focus on the event or is that actually what people are talking more about in the industry?

6:51I think it's a little bit of both. I mean, what Jensen wants is for people to see what the result of his chips create, you know, the value that his chips create, not the high specs of his chips or the performance of his chips, right? He wants to show the results. And so the results is what you see in the real world and the physical world. And that's using those chips to control robots and, like I said, self-driving cars and things like that. Now, to what extent, Wayne, do you think the conversation at the event this week, not on stage, obviously, it's all going to be about NVIDIA, but off stage, I mean, we've seen all the other chips from all the other companies come online to what extent do you think that is going to be a focus on the ground among people that you're talking to well i think um for sure uh they're going to be looking at nvidia's roadmap and trying to see look look at how it's going to perform and they're definitely going to be comparing it to their own projects and they see like should i still continue with these projects is it still worth it or should i um you know just double down on nvidia right great well wayne i want to thank you for coming on that is wayne ma our nvidia reporter here at the information China's AI progress has been quickly growing.

7:58And here at The Information, we have been asking ourselves the question what the U.S. might take from China in terms of the way it builds its ecosystem around AI and what super apps could develop over time. I want to bring on Ethan Choi, a partner at Coastal Adventures who has been thinking about the U.S. and China's respective progress in AI for quite a long time. Ethan, welcome back to the show. It's great to have you here. Good to have you. Thanks for having me. So I want to get your take on this story that we published last week. we reported that Tencent is looking at integrating AI agents into WeChat, which, of course, is the popular super app.

8:32It's used by 1.4 billion people in China. What was your reaction to that story? You know, the model wars in China are playing out very, very differently versus here in the U.S. In China, you have these super app companies that combine, you know, WhatsApp, Facebook, DoorDash, Venmo, all the things that, you know, maybe five or six different apps that we have here in the U.S. into one single app. And you're getting these Chinese companies that are realizing that they want to be able to have one single user interface for all these different services. It's very different to the way the tech ecosystem played out here in the U.S., where we have a fragmented set of apps that we use.

9:19You know, we flip from our Uber, we flip to our Instagram, and things are fairly siloed. In China, these super apps have all these things in one single place. And so for these Chinese companies to have an agent that can layer on top and go and do all the work from a single interface, it's massively powerful. And it kind of brings to life the promise of what we've been dreaming of in terms of a personal agent that can do work for you. Why is it that super apps have not come about in the US the way that they have in China? There's many reasons for that, actually. And if you think about these super apps, many of them started off as a single point.

10:04So Baidu was a search engine like Google. Tencent actually started with WeChat, like a WhatsApp. app, Alibaba was an e-commerce app like Amazon. And so they started from these separate different points. But what ended up happening is all these companies ended up building physical real-world infrastructure that allowed them to combine multiple services into one. And so in the case of WeChat and

10:40sorry, WeChat, they were able to build an Amazon-like physical goods and warehouses services. They were able to also integrate an Uber-like riot hailing service. And so you've got these situations where these Chinese companies have invested heavily into not only the digital infrastructure layer, but also the physical. And with that, they can provide all these multiple services whereas in here in the us um perhaps because uh of the funding regime uh perhaps because uh you know some of the companies uh tried and they failed we just don't have these super apps so you're an investor in open ai and open ai is well known for having a very vast product portfolio ambition they are playing in a number of different categories is open ai trying to be a super app the way that China has all of its super apps?

11:37I don't really think of it that way. You know, OpenAI certainly is working on a device, but I think all the other things they're working on actually hang very closely to the foundational model that they've built. I don't think they've got ambitions per se to build a warehousing network to deliver goods. Sure, they might integrate into other companies that provide that. But their ambitions seem mostly focused on providing intelligence to the end users and their businesses. Right. And, you know, more what I was trying to get at is, look, I mean, physical infrastructure and the retail aspect is certainly, that's a separate story.

12:21But I mean, you know, you think about shopping, you think about messaging, you think about social media, payments, you know, e-commerce. These are the categories of things you could see potentially linked together. I mean, we obviously know that Elon has sort of, I think, dreamt of X becoming a super app or, you know, at one point that was maybe one of his ambitions for it. And so I guess my question for you is, do you see AI changing the story at all in a way that could allow for the U.S. to develop these super apps the way China has? Yes. The short answer is yes. And perhaps the biggest driver of that actually is the ability to build code and build breadth and depth of the platform quicker than we've ever been able to, given the coding agents.

13:12So, yes, you can see that. And, you know, to your point, Elon has wanted to make X, all things including a payments layer. And, you know, I think the thing about OpenAI and their ambitions, you know, whilst it was very broad and expansive before, I think of late, given how competitive it is with Anthropic, you are seeing, at least for now, a narrowing of their efforts and all their energy. And I think that's the right move. For now, the coding agent race between Cloud Code and Codex is so, so, so important that you have seen SAM basically shut down and redirect teams to work on that as the frontier battle between the two model companies.

14:04Now, you've written articles on X about the AI race between the US and China, and you've looked at the different dimensions upon which the two different countries are winning. And in many ways, the US, you know, the models are more widely used today, but China has a leg up in other areas, certainly, you know, the energy aspect of the race. What are the different dimensions that you think about this race? There's a few. So one is the digital versus the physical. I can get into that. One is the distribution methodology. And then another is the philosophy of being open weights or closed. And so if you look at, if we work backwards, actually, if you look at China, virtually all of their models that are very popular worldwide are open weights.

15:00and that's different to open source. Open weights is where they let you use the weights, but you actually have to engage with them to actually use them for business. And all the U.S. models are closed. In terms of the physical infrastructure, the U.S. is ahead in terms of gigawatts and having data center build-outs for AI specifically, But China is in the lead in many of the raw inputs. So rare earth minerals in terms of land and in terms of energy, actually. And so China's way ahead on that piece. And so it's a battle where the Chinese models are looking to, in my opinion, usurp and undermine U.S.

15:51leadership globally by having these model, open-weight model companies. use free distribution to try and get as many developers as they can. And through that, engage with them in terms of business and revenue and generating activities. And so it's a big battle between what I view as a closed ecosystem versus an open weights. What about the race between the different players in China? I mean, you've got the big tech companies, Alibaba, Tencent, ByteDance. Talk to me a little bit about where each of those companies is at in their own AI ambitions, who's leading, and what they're good at. Yeah, you can kind of split the Chinese ecosystem into two.

16:37So you have the big super app companies, and it's played a little bit differently, like I said, versus the US. So it's equivalent having Amazon have its own model and Microsoft, and then obviously Google. a lot of these big companies that provide these big internet services have trained their own models so they're somewhat closed and trying to build these agents on top of their services and then you've got the DeepSeek, Kimmy, Quen and Minimax which are open-way companies and many of them hailed from research labs or in the case of DeepSeek a hedge fund and these companies have small teams that, frankly, have been accused of distilling US models and then are using very, very smart techniques in terms of memory management, in terms of things like granular MOE, where they use smaller exports versus large exports.

17:37And they also have other techniques such as multi-token prediction, where instead of just doing one token at a time, They can do multiple tokens at a time. And so they've actually innovated in a really, really amazing way because of the constraints from hardware. And so they are excellent model companies. That's the truth. They're formidable competitors for the likes of open AI and Anthropic. And it's going to be fascinating to see between the two countries which model companies end up having dominance in terms of mindshare from developers, as well as monetization from a token perspective. So last question for you.

18:18I just want to go back to the super app conversation. I mean, the idea that the US could have more super apps, you said that you think it's more likely because of the fact that AI is making it easier to code these programs. I mean, is that the only way to think about this? Are there other dimensions upon which you think that AI, and I'm thinking out loud here, I come back to OpenAI. I just think of the product portfolio as very vast in many ways. And I think about Elon's ambitions with X. I mean, do you think there are other reasons other than just it's easier to code? I mean, that seems like the lowest possible reason for why this could happen.

19:04It's a good point. I say that as one vector that truly does make it possible to build at least the breadth. Maybe if I was to give a little history again, you know, the US had many, many things that were way advanced versus China. So we had a payments network in terms of Visa and MasterCard. We had strong banks. We had already a taxi network that was quite large. And China as a developing country got to watch many of the things that played out here in the US. And some of the services that were already very well established in dominant businesses weren't in existence in China. So actually, as these super app companies started building the age of the Internet, they were able to, in one go, build dominant businesses in banking and chat and gaming and ride hailing and e-commerce and do it all at the same time in an ecosystem where they didn't have to deal with incumbents.

20:04And here in the US, the problem is there's incumbents and you have to integrate with them. You have to negotiate contracts. You have to think through kind of enemy versus frenemy kind of situations. And that's made it far more difficult. So I don't know if the US companies are going to be able to build super apps. It's just a different type of way that both economies have developed. Right. Well, Ethan, I want to thank you for coming on. That is Ethan Choi, a partner at Coastal Ventures here on TI TV. As the SaaSpocalypse shows few signs of recovering, the number of enterprise software companies that have made plain in their public filings that AI could be a threat to their business has started to grow.

20:50My colleague Laura Bratton, who authors our Applied AI newsletter, wrote a story on that topic this weekend. I want to bring her on to talk all about it. Laura, welcome to the show. It's great to have you here. Hi, gosh. Good to see you. So we're talking about regulatory filings, which I think, look, we always say there's so much in there if you just read them. And you spent a couple days reading all the filings for all these enterprise software companies. What are they saying about AI? Yeah, so companies have been talking about generative AI as a risk, whether that's from the perspective of potentially creating cybersecurity, vulnerabilities in their products, greater competition, or, you know, industry upheaval.

21:36That's been going on for at least a few, I mean, at least a couple of years, more so a few years. But AI agents specifically have begun to found their way into the risk factor segment of companies' regulatory filings. And, you know, with the introduction of Cloud Code, Cloud Cowork, OpenAI's codecs and these platforms that make it easier for enterprises that are the customers of these legacy software companies to make their own software applications and automate some white-collar tasks. And so what they've said in their filings is basically that AI agents present greater competition and also could potentially disrupt their industries and make it, you know, less appealing for their customers to use traditional software solutions.

22:30And, you know, have said that their internal strategies to get ahead on, you know, making agentic AI solutions might not be successful. Can you give us a couple examples of specific companies and what exactly they've said? Yeah, so I really like the example that I found in Figma's filing. I think it was a clear example of where they said, you know, very blatantly that AI agents and, you know, like software solutions using agentic AI, that this new technology could disrupt how people interact with software. and, you know, that that presents risks. Workday is another example. They've said in filings over the last, you know, several quarters that AI agents have led to, you know, or could lead to greater competition, I guess, as, you know, like new startups are cropping up, new agentic tools are making it easier for customers to, you know, create new software solutions.

23:40So this is what they're saying in the risks section of their filings. What are company executives saying about it on earnings calls? yeah so um i guess i'll i'll turn back to the figma example so um because i thought that that was a pretty potent example but um you know while companies such as figma figma makes um design software and they were saying they're filing you know um this new technology agentic ai could disrupt our industry but then publicly um the ceo dylan field said in february the same day is the filing that, you know, we're really excited about AI agents. And we think that, you know, humans and agents are going to work together to use software.

24:23And people who are vibe coding are brave if they're doing that for really critical things within their businesses. So, you know, and Workday is another example. They talked about, you know, in their risk factor segment, how they could see greater competition and how, you know, they have this, a lot of these enterprise software companies are coming up with new pricing models to basically figure out how they're going to monetize third-party agents accessing their platforms or their new kind of layer of AI that's going to sit on top of their existing tools and work today makes HR software. So while they were kind of talking about the risks in greater competition and their new flex credit model, their new pricing model in a really excited way on their earnings call.

25:15Their risk factor section talked about how this new model, this new pricing model might not actually get accepted by customers. And so I think, yeah, we're really seeing this excitement around agentic AI and CEOs trying to quell fears that agentic AI could displace traditional software solutions. Right. Yeah. Let me ask you this. So So this disparity between earnings calls and filings is relatively common. It's not just with the AI thing. I mean, this is something that you will see in just about every company. They'll sort of try to walk back some of the risks that they literally point out in the filing.

25:56But what are customers telling you that you're talking to? I'm talking about the customers of these enterprise software companies. The customers that we are scared will adopt AI and cannibalize the consumption of these enterprise software programs. Are they actually doing it? Yeah, it's such a mixed bag. I think what we're seeing is the larger Fortune 500 companies. They've been using these software tools, you know, the Salesforce's, Oracle's, Adobe's, what have you, SAP's of the world. They've been using these software solutions for a really long time. They have so much data in them, and it's kind of hard to just totally eradicate your use of those platforms.

26:43So I don't think necessarily that those customers are going to stop using the sales forces of the world overnight. But I do think what I've heard is that smaller startups and medium-sized firms are increasingly, like, because they haven't, you know, reached scale yet, they're starting to explore other options. So I do think that we'll begin to see, you know, larger enterprise software firms lose business from smaller software startups that are, you know, or just like really any smaller startups that are thinking like, hey, we can cut costs and we can build our own solutions. So, yeah. Right. Great.

27:22Well, Laura, I want to thank you for coming on. That is Laura Bratton, our Applied AI Reporter here at The Information. In the frantic rush to build data centers to meet surging demand for AI, companies are getting increasingly creative in how they scale that infrastructure. The Information's latest AI infrastructure column details the hacks that are giving companies an edge in the AI race. I want to bring on the author of that column, Anne Davis-Vaughn, to help us unpack that. Anne, welcome back to the show. It's great to have you here. Hi, Akash. Thanks so much. So you detailed these five strategies, you call them hacks, that companies are using to really accelerate the development of AI infrastructure, data centers, I should say.

28:07And so I want to go through each of them because they're all very interesting. The first one is you said that they sweat their own real estate. What does that mean? Yeah, so Akash, we all know that power is the biggest bottleneck to building out AI infrastructure. And we've been accustomed to seeing flashy headlines about hyperscalers going off on their own, off-grid, building their own power. That's still not that fast. And so if we look at the AI chip clusters that are up and running now with power pulsing through those chips today, and you asked, how did they get there? One of the first things they did was take advantage of real estate that has strong tower footprint that was already in their portfolio.

28:55So we take a look at a place called New Albany, Ohio. I mentioned it in my column a couple of weeks ago too, because it has an incredible artery of tower lines. And Netta and Google are just two of the hyperscalers that have big campuses there already. And when the AI boom got going, they looked around and said, we've got the capability here to put AI in our own buildings. So, for example, Meta rolled out clusters of traditional compute racks with CPUs and rolled in a kind of SWAT team operation, the chips that would go in for GPU computing for AI in a big way and had an AI cluster in a matter of months.

29:41and didn't just do that, but decided that given the life of these ships can only be a few years anyway, they would go ahead and build temporary structures or tents. Meta built a tent city in New Albany and began standing up ships quickly there. And Google across the boulevard from Meta and these huge industrial parks in New Albany did something that they're known for advanced at doing, which is connect their data centers and multi data center clusters using very sophisticated and high powered fiber links, as well as software to really make multiple campuses work like a big brain and not effectively full power.