Elon Musk Loses OpenAI Suit, Amazon Trainium Gaining Ground, Open Source AI Struggles

21:48Stephanie Palazzolo:And this is something that's interesting because Google has been trying to get its TPUs out there on the market. We've talked a lot on this show. We've asked almost every Neo cloud company executive here on the show, will you use TPUs? And the answer is, well, we're okay with our NVIDIA GPUs. I wondered what your reaction to this story was, given that you're coming out of reporting this story on Tranium. And, you know, I don't know, could you see Amazon doing something like this with another provider in the long run too? Yeah. You know, Jassiet also said last month in his investor letter that they could envision selling chips someday outside of their own cloud service.

22:32So it seems like it's something they're thinking of. I don't know how they would structure that. Could that just be in individual companies that own data centers? Could it be with a financial partner like Blackstone? It's definitely possible. I do think it's good to keep in mind that Amazon is the biggest cloud and Google is sort of like, I'm pretty sure it's still number

22:55Stephanie Palazzolo:three so you know i i wonder really say that like it's just remember google right pay me to say this but uh they would like but yeah yeah so i think that they like to like control a lot of the process themselves um but yeah i was surprised to hear they would even like talk about like you know having chips exist outside of their own cloud service i i kind of doubt that that would mean that they would put chips in like Google Cloud or Microsoft because those are their competitors. So it kind of does make sense that they would maybe work with some sort of other partner to do this because it seems like they see the dollar signs, they see the good story.

23:37They don't want Google to get all the shine. But I don't know, they're also really good at operating data centers themselves and might not need the help of another manager. Yeah.

23:51Stephanie Palazzolo:Well, Catherine, I want to thank you for coming on. That is Catherine Perloff, our Amazon reporter here at The Information. As the cost of frontier AI models remains an issue for tech companies, open source models are getting a lot more attention. My colleague Stephanie Palazzolo wrote about that in our AI Agenda newsletter today. I want to bring her on to share with us what she knows. Stephanie, welcome back to the show. It's great to have you here. Set the stage for us. What is going on in the land of open source models these days? Yeah, so Akash, as you mentioned, it is definitely no secret that the costs of AI are going up for many companies.

24:28In fact, earlier this year, the Uber CTO actually told us that the company, you know, even a company as sophisticated as Uber, have basically blown through their AI budget in just a matter of months. And so to kind of counteract this, many companies are switching to these cheaper open source models, models like Kimi K 2.6 from Moonshot AI or DeepSeek v4. And even though these models are maybe not as capable as frontier closed source models, they are still, again, cheap and really good for automating more simple tasks.

25:03Stephanie Palazzolo:So are these open source models, you said they're good for simple tasks. I'm trying to understand the delta between the simple tasks and then the things that big public tech companies need them to do. You mentioned Uber. I mean, is there a class of open source models that is good enough to do what these larger tech companies need them to do as well? Yeah. So, again, it really depends. Like, you can think of maybe a simple task that an open source model could do is maybe looking at customer support emails and categorizing them into different categories. So like maybe people that want a refund or people that had a problem with their delivery driver.

25:42And then maybe the next step of planning what to do next or responding to those emails could be done with a more, you know, with a better frontier AI model that's able to handle that. And so, yes, like a number of execs from companies like Uber and Airbnb have really come out and said, yes, we've been using open source AI a lot. It's super helpful for automating maybe these more simple tasks. But we have also heard from developers more recently that there are still areas where open source is falling short. So for instance, I spoke with an exec at a big customer of OpenAI and Anthropic who's been trying to use open source models like Kimi K2.6 and DeepSeq V4.

26:24These names are really a mouthful. You nailed them every time though. I mean, you're pretty good at them. But basically this exec was telling us that even though these models perform well on benchmarks and kind of surface level questions, they do start to struggle when you try to ask them follow-up questions or these kind of deeper lines of questioning.

26:42Stephanie Palazzolo:Why is that though? I mean, this was an interesting line of detail that you had in your story. I mean, you looked at why it is that these open source models aren't as good, you know, in real life. Can you explain a little bit about, you know, how these models are trained differently and why that might be the case? Yeah, so obviously, you know, I'm not at these companies, so we don't know for sure, but we have a couple guesses of why this might be the case. So one hypothesis that we had is that the model's training data sets don't have enough coverage, which basically what that means in kind of normal English is that whenever you're training these models, you're not really showing them enough examples of all the different sorts of situations and tasks that they might encounter out in the real world.

27:26So for instance, if a model is, you mostly trained on the public web. On the web, there are lots of examples of these kind of like intro blog posts that are explaining concepts like, you know, here's an explanation of this type of training technique for models. But the web doesn't really include lots of examples of more complex things like, for instance, you know, what should a person do if one of these training techniques for models like goes wrong? Another possibility is that these models were trained by basically what people call distilling frontier AI, which can backfire when it's not done correctly.

28:04Stephanie Palazzolo:Taken together, though, Stephanie, I mean, are these open source models getting better? I mean, this was the other part of your column was that the delta between the frontier models and then the open source models, is it narrowing? Is it widening? So there is some evidence that that gap between open and closed source AI is actually getting bigger. And so there was an analysis done by NIST where they basically charted the performance of various models over time. And, you know, in a kind of perfect role that's like pro open source, you would want to see those two trend lines either, you know, kind of moving in parallel or maybe even kind of converging over time where you see them kind of meeting and open source catching up.

28:51But instead what we saw in this analysis is the exact opposite. So those two trend lines are actually kind of diverging and the gap between them is getting wider over time, which is not what you want to see if you are somebody who's a big proponent of open source and thinks that open source is going to catch up in the long run.

29:10Stephanie Palazzolo:What are you watching for, Stephanie, on this story moving forward? I mean, one way to look at it is we just see how the benchmarks do over time. But as you just said, the benchmarks are not really the best indication of reality and what they can do in real life. So I wonder, I mean, what questions do you have as you continue to report this story? Yeah, so as you said, you know, it is tough because a lot of these models are performing well on benchmarks, but that's not necessarily true for the way that developers are using them in real life. So I think a couple of things that I'm looking out for is, you know, the number of tokens that are spent on closed versus open source models.

29:47So, you know, even if open source models are not as great as Frontier closed source AI, if the tokens that are being generated with them and their usage overall is increasing, that's still a pretty good sign. I do think as well, like, you know, over time, if models from Anthropic and OpenAI continue to get, you know, more expensive as they have been, at some point, the kind of tradeoff just might be too great for developers. And they might just be forced to turn to cheaper open source models to do a larger range of tasks. And so I think we'll also be keeping an eye on that, like pricing pressure from the big model labs to see whether that's going to create this opening for open source to kind of make a comeback.

30:32Stephanie Palazzolo:Great. Well, Stephanie, I want to thank you for coming on. That is Stephanie Palazzolo, our AI reporter here at The Information. With Cerebris making a big debut last week, there is now a question of where the stock will go from here. Shares are down from their initial spike, but maybe the spike is all IPO hopefuls wanted to see. I want to bring on Jay Dosh, managing partner of Sapphire Ventures, who has seen 15 of his companies go public over the years for some analysis on where the IPO market is headed. Jay, welcome back to the show. It's great to have you here. Hey, Akrax. Thanks for having me.

31:06Glad to be here.

31:07Stephanie Palazzolo:So, Cerebris, what did you make of the big pop last week? Yes. Well, first of all, kudos to all the initial seed investors in that company, right? Foundation, Benchmark, Eclipse. They're very contrarian to make a bet on a semiconductor company when everybody was investing in SaaS companies, right? Right. And look, this is the story of all IPOs, right? If it's a really hot IPO that everybody is looking for, it comes out, it prices at a certain point, then goes up. And then, you know, it kind of meanders around, right? Sometimes it goes down quite a bit like Figma or sometimes it goes up. Right.

31:48So from my perspective, we still have to wait for six months before all the lockup expires, because I think Cerebrus has done something different where the lockup doesn't all expire at one time, but they have different classes of stock expiring from lockup at different times. So you really have to wait for all the lockup to expire. And then I think typically you need to have at least four quarters of earnings before the stock kind of stabilizes and trades at the multiple, you think it will trade long time. So it's still a wait and see for me.

32:25Stephanie Palazzolo:And I should remind people that you were on here, I believe it was the day after the Figma IPO. And this was your message of reason that you sent on that day after that pop too, saying, hey, look, six months, we got to wait and see. And sure enough, I mean, if you look at Figma shares, right? I mean, the pop was deceiving in many ways. That's right. But look, Figma is still a great company, right? And they announced really great results, like a 46 % or whatever growth and the stock up. And it's trading at what it should be trading at, not at the multiple it traded at when it IPO'd and then where it closed at the end of the first day.

33:05Right.

33:06Stephanie Palazzolo:So what do you think this tells us then about the SpaceX AI IPO that everyone is waiting for? I mean, if we sort of look at levels of craziness or ambition, maybe you could say, right? Figma, it's a software company. We kind of understand it, right? Cerebrus, okay. It's a chip company. It's still an emerging industry in some ways, a lot of demand. Then there's SpaceX AI. And it's anyone's guess, really. But was this really the exact outcome that they needed to sort of have confidence to go full steam ahead? Yes, for sure. And SpaceX, I think it's a little bit different. I think the other two IPOs that will probably happen, Anthropic and OpenAI, are probably more in the line of what happened with Cerebrus.

33:58You know, a lot of kind of, you know, AI deployments going on, and it's really an enterprise AI stock. SpaceX is very different. I think SpaceX doesn't really have any kind of effect on Anthropic and OpenAI because SpaceX is really you're buying Elon's stock, right? It is all about Elon, and he can be a controversial figure, but you can't doubt how good he is as an operator and building businesses. The SpaceX is all about Elon. It's not about AI. But for Anthropic and OpenAI, yes, the Cerebrus IPO shows that not only is there a demand from the stock market, but there's also demand from a lot of AI in the enterprise.

34:44I think, Stephanie, in your earlier segment was talking about open source AI. And I think that is what is driving, you know, Cerebras to some extent, because a lot of the spend is now going to be on inference. And when you look at inference, you know, the NVIDIA chips don't really work that well. And that is why Cerebras, you know, Grok that NVIDIA bought are so critical for inference as the amount of inference goes up, not only with the closed source model, but also the open source model.

35:12Stephanie Palazzolo:Well, and this was the whole discussion we had on the show before the Cerebrus IPO was that, you know, before the stock popped to the way that it did. I mean, there could have been a world where Cerebrus may have still been an acquisition candidate. Although now, I mean, the valuation is what it is and I don't think anyone's going to touch it. But, you know, saying that there's probably another six or seven chip startups that are doing kind of going after the inference market and trying to do, you know, inference based kind of chips and semiconductors. So you'll see a lot more companies coming out and there's still going to be a bunch of acquisitions that's going to happen in the semiconductor space.

35:50Stephanie Palazzolo:But now, so now bridge the gap here between those inference chip startups and then the open source story. Where do those two intersect here? That's right. So first and foremost, inference is provided both with the closed source models, like Enthropic and OpenAI, as well as open source models. But my prediction is that overall, you kind of have to look at these foundational models as operating systems, right? But when you look at operating systems, things that you personally use, like a Windows or a Mac laptop or your phone, these are all closed source operating systems. And this is where Enthropic, Cloud, and OpenAI are going to flourish.

36:32But on the back end, everything runs on Linux right now. And it's going to be the same thing on the back end when people build applications, enterprises, build lots of agents. They're all going to run on open source model. So our belief or my belief is that over time, open source models might not have as much tokens going through them. So the dollar value is not as maximized, but the amount of inference that will have to be provided for open source models are going to be a lot more than even closed source models from Anthropic and OpenAI and Gemini. So if you look at that, what do you need for that?

37:14So you need chips, right, because to provide the inference. And then you need a lot of inference service providers, you know, to some extent, you know, Base10 and Together and Fireworks are out there. And that also brings us to the announcement yesterday between Google and Blackstone, right? Like, you know, I think Google probably looks at it and says, we probably don't have enough capacity to provide all the inference. We can provide the chips. Maybe it is better to have a JV or some kind of a joint company where somebody else also takes the risk, the balance sheet risk, as well as the kind of the data center risk.

37:49But we provide the chips. but they are probably building this, started this company because they see the demand for inference out there.

37:57Stephanie Palazzolo:So your understanding of this joint venture, I mean, is this like, you know, I mean, CoreWeave, Navias, all these companies, Neoclouds, I mean, they are so committed to NVIDIA. I mean, they will not speak any other company's name other than NVIDIA. I mean, is this, as best you understand, is this kind of like the TPU Neocloud offering that they're trying to build? So I don't know exactly, but I would say that Nibias and Corvee, they're still very much focused on training, right? If you look at it, most of their dollars are still coming from training on the closed source models that Entropic, Cloud, and to some extent, Gemini and Unopened AI are running.

38:40But there is a huge need for inference. And that is why you see companies like Base 10 and Fireworks and Together and Model doing so well, I would bet that, and I don't know enough, that this thing that Google put together with Blackstone will probably focus more on inference because I think that is where all the next generation opportunity is. Because the amount of training you need is very small compared to the amount of inference you're going to need.

39:07Stephanie Palazzolo:So you see it more as less so in the NeoCloud bucket of categories, more in this inference provider, the base 10, the togethers, the fireworks, I guess. It's more competing with that class of copies. Yeah, that's what I would assume because I don't know what their plans are because that's where the need is, right? There is need for training still, but the real need that is going through the roof is about inference. And if you really believe that open source models are going to be a big competitor to closed source models, the only way really to make money or to make margins is by providing inference and running the open source model for you, for the enterprise.

39:53So I think that would be my bet that they're going off to the inference market with this joint venture.

39:59Stephanie Palazzolo:Let me ask you one more question. We had a story out today about Tranium, and the reporter two segments before you was talking about all the traction that Tranium has made. And the question I asked her was, could Amazon maybe pursue a similar type of arrangement where it tries to get its Tranium chip out there, you know, basically using a company that is not AWS to sell its Tranium and access to its inference that way? What's your assessment of the Tranium strategy? And are you hearing Tranium come up more and more in conversations with startups that you are working with? No, I don't hear about Tranium other than Anthropic actually using it, right?

40:42So we don't hear as much about Tranium being used. Actually, we don't even hear about Google TPUs being used that much. we do hear about the next generation inference chips that are being built, which are really focused on memory optimization rather than on CPU or GPU optimization, right? So we do hear about it a little bit, but I don't hear about Tranium or even... So it was quite a surprise that Google announced this with Blackstone to set up a separate cloud. But, you know, look, it's, again, a way maybe for them to create an ecosystem around the Tranium chips. So I wouldn't put it past them to also start something independent like Google is doing here.

41:26Stephanie Palazzolo:Right. And well, maybe, I mean, you know, to a point that we were talking about earlier in the show, you know, this idea that even if Amazon has deals with Anthropic and with OpenAI, you know, that could very much have material impact on the revenue that they generate from these deals. It's a separate conversation, though, our everyday developers and smaller startups and the developer ecosystem at large, the extent to which they are really accessing these chips or using them. I mean, that is maybe a bit of a slower story to develop because it's not one-off deals. Well, the other thing that you also have to look at is, you know, Amazon doesn't really have its own model, right?

42:09So there is this whole train of thought where Google has its own models with Gemini, and you potentially could open source, you know, a smaller or an older version of Gemini, they already have Gemma out there, and they kind of create a whole different ecosystem with Gemma and this new, you know, company that they put together with Blackstone, and then have like an open source kind of an ecosystem that they build around it. And, you know, in some ways separate it out from some Gemini. So, you know, that might be one of the things that they are thinking about. And Amazon kind of doesn't really have its own models, right?

42:45Or it's more models that it has developed hasn't been very successful. So I think you kind of have to look at each of these companies independently. So I'm not sure whether Amazon would do something like that with somebody like Blackstone.

42:58Stephanie Palazzolo:Great. Well, Jay, I want to thank you for coming on. That is Jay Doss, Managing Partner at Sapphire Ventures here on TITV. Thank you. Thank you so much, Akash. That does it for today's show. A reminder, we are on this stream Monday through Friday at 10 a.m. Pacific, 1 p.m. Eastern. If you can't make it then, episodes are available on theinformation.com, our YouTube channel, or wherever you get your podcasts. Make sure to follow us on social media on X, Instagram, and TikTok. I'm already excited for our next show tomorrow. Have a great rest of your Tuesday. Bye-bye for now. Thank you.

44:00Thank you.