Unpacking Marvell's Analyst Day, OpenAI Revenue Rumors, and the Heterogeneous Rack

0:04Ben Bajarin: Hello, everyone. Welcome to another episode of The Circuit. I am Ben Bajarin. Greetings, programs. I'm Jay Goldberg. This is a great week for semiconductor conferences. We have the return of Semicon West to San Francisco, which I think many people are fans of. Arizona was great, but there's a lot going on locally here, which is coinciding with the Applied Materials launch of their Epic Center, which is going to be cool. We get a tour of that. I'll be there on Monday getting the tour. Yeah, Monday, which will be cool. I mean, that's actually been in the news quite a bit lately relative some stuff to Bezzy, hybrid bonding and memory.

0:55Ben Bajarin: We've obviously talked about Wave 4 Coming for a while, so that's going to be a good, good event. And they're having their Investor Breakfast month Tuesday, which we'll report on. And then we're going to go to OCP and have an insane amount of meetings at OCP where IR Labs, our friends, Mark, Wade, and team is actually going to have a working demo unit with GUC and YWIN. So we're going to see some working bits transferred with CPO. That will be exciting. It's going to be a big show. I'm surprised how many people are coming to this show. It's sold out. It's sold out. I mean, I've personally had people be like, do you got any more tickets?

1:40Ben Bajarin: I'll buy some off you.

1:42Jay Goldberg: I've had multiple people ask me for tickets. Just so everybody knows, we're talking about the Open Compute Project Summit, which I think has become one of the top two, three events in semiconductors. It's this and Computex. And it's a big show. It's kind of hard to comprehend how big it is. It has outgrown San Jose.

2:08Ben Bajarin: Yeah, they can't be at San Jose next year. The amount of people asking me to come by their booths, even the Boyds and Eatons of the world, who are doing liquid cools, sidecar 800 BDC. They're all, we're going to be there. We want to show you all our working goods. So it's a big show. Anyway, we will be there. If you are there also, feel free to reach out or snag one of us when we're walking the floor trying to get nuggets from engineers on the floor who hopefully don't know who we are. And that's the shtick for this week.

2:47Jay Goldberg: Yeah, if you go and hit us up on email or Twitter, DMs are open, all that. It's going to be a busy, busy, busy show. Busy, busy show.

2:58Ben Bajarin: Okay, and on to this week's news. Probably the biggest event, I think, just going on was Marvell's Investor Day, which seemed to get a very good reaction. As a matter of fact, the excitement and kind of dollar lifts that they shared helped a bunch of names, particularly in Optical and Interconnect, both competitors and adjacent to Marvell. They showed some very aggressive numbers that I don't, to be honest with you, most of the sell side notes I had read were not as high, even close to their estimate. So very bullish. I'm not sure I found anybody who was projecting this kind of size for Marvell.

3:45Ben Bajarin: But you were there. So tell us, boots on the ground, how it was.

3:52Jay Goldberg: So, yeah, I was at the Marvell Analyst Day on Tuesday in New York. It was the energy in the room was was really good. I mean, it was just like everyone was very upbeat, very positive. um marvell talked about a lot of interesting topics to your point they they put out these five-year revenue and eps targets um that yeah i i you know there's no five-year consensus number but i'm if there were these would have been above them so they seem very very upbeat and very constructive and i think you know one of the one of the many things they accomplished is they They have shifted the narrative around the company significantly.

4:40Jay Goldberg: Because we've talked about this before. For years, this has just been the only thing people wanted to ask about Marvell was how much share are they losing at Amazon? That was the entire conversation, Alchip versus Marvell. and now it's you know they have uh they have a much wider breadth of customers and a much wider breadth of products and they talked about a lot of those and we can you know we can quibble with any particular point and you know some of these products are better than others and whatever um but it was a it was you know it was it was good to talk about something other than Amazon and Alchip.

5:18Jay Goldberg: There's a lot going on there. We don't have to talk about the same topics now. There's much more going on there.

5:28Ben Bajarin: So my top level question, and I guess this is also from folks you talked to, because I sort of brought this up, but nobody I was talking to cared about this as much as sort of I did. You know, we've talked about this before, that essentially, as long as we've been going to analyst days from Marvell, I don't know, four or five years, it kind of felt like every year, they had a different way of talking about themselves. And I guess the question because now is, do we think they found their lane? Like we talked about this a little bit as their earnings. They talked a lot about XPU attached. Now to me, again, that's not just the straight lane because it's not just XPU attached.

6:12Ben Bajarin: They have networking silicon. They have parts of that stack that I think are important. And we'll talk about some of their new ones. But do we think that they have positioned themselves like accordingly now? And from that, this is the lane we think they belong in. Because you're right. They don't really seem to be talking up XPU anymore or hardened arm. They can do that, but that's not what customers are coming to them for largely. It's the periphery of those accelerators, periphery of the racks, and then increasingly network connectivity and IO, which is great. There's a lot of people in that space.

6:48Ben Bajarin: Fantastic. But is this the lane we think they belong in?

6:53Jay Goldberg: I'm going to look at it a little bit. I would say that they've had the pieces all along. I think they have A year ago or so They sort of shifted the messaging More around just I think they started calling it data center infrastructure And Like they had all the pieces They've been heading in this direction And now A number of things have improved The story seems much more fleshed out Right? So I don't think it's like This is like something out of the blue I think they've been working towards this for a while and it's just they're able to sort of put more substance to that. I mean, maybe the substance was there all along, but I think they've gotten there.

7:42Jay Goldberg: So I don't see this as like, I know what you're saying, like for a few years they were sort of changing the story every year. I think they've been on this track for a while and it just, they have more to talk about and they've delivered on some of it. And I think one of the tricky things about what Marvell and Broadcom do around custom, we've talked about custom ASIC for years. And for a long time, the custom story was more than just like a processor. Again, for both companies, Marvell has always been good about saying, hey, we can do this, we can do that thing, we know all these different things.

8:27Jay Goldberg: and it just the story seems to have congealed much more coherently now because yeah and my point is like they they have all those other things but they also have processors too and and now like we i think for a lot of people there's a struggle to understand what custom is and what and importantly how you differentiate in that market how does how has broadcom been able to hold on to that business for so long we don't really get a lot of that and marvell has has been fighting to get into this market and has been better at giving us more visibility into what's there and all the different pieces and all the different IP blocks that they can offer.

9:06And it just, the way they're telling the story makes a lot more sense now. Yeah.

9:12Ben Bajarin: I mean, I'm not disagreeing with the parts. I just, I, you know, as a mental model, I like to know how companies think about themselves, position themselves, see the swing, swim lanes in which they compete, create narratives around that, um, you know, just at a high level. And so that's been my only point is, you know, they've gone through evolutions of, you know, who are we and where are our strengths and where should you think we compete and whatnot. And so I do think this is right. I mean, I do think the inference slash true scale up compute domains are, are going to benefit them a lot. again I think people forget about the the the silicon part of their story the networking silicon the switch silicon a whole slew of things that they have um but that to me is you know the the the timing I think for the way they're positioning themselves is lining up well and then I think that's why they kind of had much larger numbers as a part of what they think they're dollar amounts in their swim lanes are, which, again, seemed pretty high.

10:27Ben Bajarin: But, you know, for a lot of people, this is new types of models for Marvell across compute infrastructure and all of the connectivity bits. And, you know, the other thing I think about, too, is, I mean, they really are up against Broadcom in a lot of ways, but they're also, you know, tight with NVIDIA in terms of NBL and pretty partner friendly across some other parts of the stack. And I just wonder, you know, in some of the areas where I think Broadcom is not going to go, and particularly, I would be shocked if they ever adopt NBL as a standard, even though I think that the market's going to demand this in terms of connectivity and interoperability of compute.

11:19Ben Bajarin: How much of that opportunity where either Hawk or Broadcom doesn't want to go, that Marvell can go and starts to meet some bigger pockets of hyperscalers and a whole slew of other folks who, again, are going to compete in connecting the scale-up domains across a range of periphery, right? From memory, from CPUs in the scale-up domain to wanting to work with your NVIDIA infrastructure, all of those things. They seem well positioned for a very heterogeneous infrastructure world, I guess is the point I'm making. Yeah, I mean, I think there's a lot going on

12:02Jay Goldberg: in the data center market that's making this possible, right? A lot of the things, not all, but a lot of the things that Marvell is talking about today are capabilities they've had, chips they've had for years, or at least segments they've played in for years. And this is like we're now at the point where the data center spend is so big that even sort of small products that used to be sort of little niche things they did in the corner are now real volume or significant amounts of revenue are possible. because they went through a whole bunch of products that they make. And they've acquired some of those, but there's a lot going on there.

12:43Jay Goldberg: If you think about a server, there's – I don't know how many chips there are in a server, thousands of chips in a server. A lot of those they've been making for years, and now suddenly the volume matters. It's big enough. And I think that's how they get to these big revenue numbers. is it's not just they're helping somebody design a chip. They're selling, you know, millions and millions of chips into the data center now. And it adds up, right? And I think that's what's going on here. So, right? But then to your point about, like, I agree. I think the heterogeneity of racks is increasing and will continue to increase.

13:22Jay Goldberg: And I don't know that every hyperscaler and every AI lab is long-term going to have their own accelerator. But at the very least, I think they are all going to do it a little bit differently. They're all going to have different servers. They're all going to have different rack architectures. And that's good for Marvell. Yeah.

13:44Ben Bajarin: But this piggybacks on two things that we've done reports on and tried to estimate the dynamics of who's best positioned for this, but you're never going to have a homogeneous set of infrastructure in the world. I think you could even make the case that if the tier one neoclouds are successful long-term, they probably won't be solely NVIDIA shops. They might be largely, but I don't think they'll be solely in video shops, which really means that you, you know, you, you, again, you, you have heterogeneity, heterogeneity, but all of that needs to be networked together. This is one of the things I brought up in this report we did called The Future of Compute is Fungible in that what needs to be fungible is the workload so that it can go to any compute domain and maybe heterogeneous compute domain, which is something I don't think people have thought about yet.

14:52Because when I talk about scale-up domains, it means connecting one, two, three, four racks together, and those racks look like one brain.

15:00Ben Bajarin: Instead of right now, it's one rack, it looks like a brain, and you shard the model across what that's connected to. So the goal is to get to these compute scale-up domains that are four, five, six compute pods or more, depending on how much power you have. Can you cool them all in a row? Can you network them all together? There's all sorts of engineering variables. but you want that to be you don't want to shard the model across that you want shared context, shared compute all of those things to serve inference at scale and that might be multiple things it might not be all from one vendor you might have CPUs from an x86 provider or ARM that aren't NVIDIA VAR you might have NVIDIA GPUs, you might have NID GPUs you might have all of these things and the only way a hyperscaler or a NeoCloud can have fungibility in that workload is that they all work together.

15:50Ben Bajarin: And I have no idea how they pull this off, but you talk to everybody in the cloud space and they're like, yep, this is what we'd like to do. We're going to make software that tries to orchestrate these models because we need to do that. But it's a complex problem. So that's why I just keep landing on scale-up domains in a heterogeneous world are going to require perhaps more interoperability across variants than people realize. And, you know, I think that's a much more economically viable world for those who try to monetize this infrastructure versus having islands of compute, even if it's scale up domain islands across vendors.

16:31Ben Bajarin: All of this needs to work together. And that's the end goal, I think, but also extremely challenging.

16:38Jay Goldberg: So, you know, I was in New York. I also, after the Marvel Analyst Day, I also went on and did a whole series of investor meetings. And shout out to everybody. to all the listeners I met with too.

16:53Jay Goldberg: And a couple things were very clear. I think a lot of people think networking is increasingly an area of focus around the AI spend. It's people, I mean, it's not exactly news, at least to our listeners, but it is certainly a topic that kept coming up again and again is networking is important. This is where people are spending a lot of time. And with good reason, the networking domains across these data centers are becoming more important and more diverse and more complicated and technically much, much, much more interesting. I think that's certainly true. The other debate that I find I'm having more is the extent to which future AI compute workloads are done by sort of general purpose GPUs.

17:48Or do we start to see further sort of differentiation around the product line?

17:56Jay Goldberg: Meaning, are we going to have, you know, I think it's clear we're now moving towards a realm in which we're having different chips for inference and for training. That trend is clear. But now the question is also, are we going to have different types of inference compute as well? Or is it just going to be like, you know, one GPU can do both sides of it? and I think a lot of people I talk to at least here who are on the sort of the industry side of things sort of assume that like or at least had assumed that you know in the future I'm going to have a bunch of GPUs and to your point they're going to be fungible and I'll use them for whatever task I need maybe I get some extra special stuff for training but like for the inference stuff I just have to sort of wire enough of them together.

18:47Jay Goldberg: And this comes out of the work I was doing last week around token pricing. I really am starting to believe that we're going to have a much broader array of inference silicon and that you're going to want to have special types of chips, GPUs, accelerators, whatever, for input versus output. And this sort of goes into this pre-fill versus decode conversation. And we may have other variations come up above. And you think about it like there's now, what, three flavors of TPU going to four or five, right? There's two, but yes,

19:30Ben Bajarin: I think there will be more over time.

19:31Jay Goldberg: I think there's three now on Ironwood, and then we get to Gen 8 and Gen 9, and we're going to have more. And I think that's Google sort of picking out very specifically, you know, coupling different chips to different parts of the AI equation. And certainly with NVIDIA, we see that as well. We have CPX and Grok LPUs to do the pre-fill. And I think we're going to see more sort of variation there as well. And who knows what, you know, there could be more iterations as the software develops. So all of which is to say that It gets more complicated. I think it is interesting to think through what that means for the competitive landscape.

20:19Jay Goldberg: NVIDIA certainly is leading this and has a pretty wide array of products already, and they can continue to expand their SKU count to get into that. I think it also opens up the door for other people as well, though. And I'm not sure I was convinced of that before, And I'm increasingly coming around to this idea that you're going to have more openings for competitors.

20:48And I don't know how that all plays out, but I think it certainly gives room to other forms of compute.

20:55Jay Goldberg: Other people are going to get slotted in. I don't think that means that we're going to have 70 different startup XPUs filling in lots every corner. But sort of I'm still working through how this is going to change the way I think of NVIDIA, Broadcom, Marvell, AMD, and how they sort of mesh together here. It's getting a little bit trickier to track share in a good way. Yeah.

21:23Ben Bajarin: No, I mean, I think I've been on board with this for a while, which is why my core assumption is a whole slew of diverse infrastructure. And again, it all needs to work together in an ideal world because that's the best economic possibility for hyperscalers to have fungibility of the workload across variants of infrastructure. That is not a world we have today. I don't think it's one we're going to have in the next few years. But I feel like that's the inevitable end state because their economics favor it greatly from a CapEx as well as tokens per dollar per watt as their services, their software is the orchestration layer of where to monetize which workloads based on its request.

22:05Ben Bajarin: so that becomes again but just to say we're not even in the single vendor scale up domain yet we're not truly as much as Google has X many 90 whatever 9400 9600 TPUs networked together those are still sharded models that's still not one model across that many racks so this is where they all need to get to is that you can load huge models with massive shared memory and storage serving as one contextual compute domain for many concurrent users and massively long context windows. Like that's what they're trying to get to. But we are not there. We are not even close to one vendor scale-up compute domain yet.

22:56Ben Bajarin: So my other, you know, as you know, I just nitpick on this and I continue to will. Marvell also likes to talk about scale up, scale out in different terms than others. There's no ubiquity of... My feedback, again, has been, and just so all our listeners know, scale up or scale out. I don't care which term you use, but if you add domain to the end of that sentence, then we all know what you're talking about. It's connecting racks together in a shared compute infrastructure. So start adding domain. I don't care which term. You might all land in other areas. But the only one that's actually clear is scale across.

23:38Ben Bajarin: That's the only one. Which, to be honest with you, some future year should also be a domain. But I realize that's going to be much harder to do. So just start saying scale up or scale out domain. And we all know you're talking about connecting racks together as one shared compute contextual window.

23:57Jay Goldberg: There you go.

23:58Ben Bajarin: Okay. I do want to talk about this mysterious little hint that Marvell launched as a new class of AI infrastructure management, Silicon, that they did not say was a base management controller or BMC, but they put it next to that. This is actually something I think is super interesting. And we did a report as a part of our GPU Tsunami series talking about BMCs and FPGAs and the total attach that's happening on compute boards. You know, you look at any compute tray, all the ones we're going to stare at in close proximity this week at OCP, and you see a whole slew of FPGAs from Lattice and BMCs from A-Speed.

24:44Ben Bajarin: And that is increasing as boards get bigger, as boards need have more compute on that board. That's true of networking and that's true of CPU and ASIC slash GPU, XPU slash GPU boards. So I've talked with this a bunch with engineers and it's kind of like, what are you using? You know, your base management controller pairs very closely with your FPGA, which is why Lattice and A-Speed have a deal together. These are things that work in harmony. But like what's sort of the role of these things? And it does keep coming back to an engineering theme, which is, you know, BMCs, not necessarily FPGAs because they're a bit more flexible or fungible.

25:37Ben Bajarin: you could call it a fungible programming gate array actually, is BMCs were kind of designed for other things. Like they do work, but they have limitations in kind of modern, high wattage, high complex compute design trays. And so what engineers have been doing is like they throw more FPGAs at the problem to solve different things. Is it security? Is it power management? Is it X number of things? And they're trying to solve problems. In fact, I brought this up with, as I was talking about at HPE, when they were talking about how hard of an engineering problem it was to design kind of this complex compute tray with Helios.

26:19Ben Bajarin: And they used a lot of additional microcontrollers and things that, again, were not necessarily custom designed to this, but they made it fit their needs. And I think that's the context that Marvell's bringing with this AI Infrastructure Management Board is to say a new type of silicon that solves some of these problems from management controller to, again, security, system performance, monitoring, all of those things is necessary. And they want to play in that market. Now, I don't know if we should just come in and size the opportunity here like BMCs, which depending on whoever's forecast you read could be anywhere between a$3 and$8 billion market because these are not very expensive chips by 2030.

27:07Ben Bajarin: maybe it's not the same maybe it is but it's it's my my intuition is Marvell's trying to solve kind of this problem that modern day you know BMCs and even even to some degree some of the things that engineers are throwing more FPGAs out to solve problems for could use something a little bit more custom made or for the modern era of these complex compute trays While we know anything about it, that's the intent I think they're coming from. And I think that's interesting because it's exactly true that BMCs serve a purpose. It's pretty straightforward. FPGAs have been increasing and attached to solve a wide variety of problems that we don't have custom silicon for, which is why they're flexible and programmable.

27:55Ben Bajarin: And I don't think that goes away, to be honest with you. This is still a compliment to BMCs and FPGAs. But my point was the problem I'm addressing is why people were seeing more FPGAs almost every year come to the board was because they're trying to solve a whole slew of engineering problems. So I wonder if this kind of, you know, where this fits, but I understand the problem because we are trying to solve very complex engineering problems with a lot of, let's just say, legacy approaches to board design and management. So I'm intrigued by it. I don't know if it goes anywhere, but that's the context that intrigues me.

28:40Jay Goldberg: So stepping back a second. there is a chip that has existed on a server board for years called the board management controller and it is the it's there are a bunch of companies that do it different different ways there's a company called a speed in taiwan that is sort of the merchant leader in bmc and bmc's as the name kind of suggests sort of manage a lot of the low-level stuff that takes place on a board So not the big compute flow. It's not doing the AI math. It's just like keeping track of all the electricity that's flowing around the server board. And, right, and, you know, we don't think about it that much, but there are, you know, hundreds of sensors on a board that measure.

29:29Jay Goldberg: And growing. Yeah, well, on a traditional server board. Like, that's just a traditional server board that we've used for years in the cloud. We'll have things that monitor voltage and temperature and current across the board. And the BMC is, like, the place that you can, that will track that, right? And so for those of you who have ever messed around with a server with firmware, you can, like, turn on and off the fans or you can, like, get temperature readouts from across the board. That's on a traditional server, right? When we start moving to entire rack designs, you don't just have a multiplication of, you know, you don't just add them all up like hundreds of servers.

30:12Now we have that many BMCs.

30:14Jay Goldberg: The task involved gets much more complicated. The modern, you know, the current generations of GPUs and accelerators can have dozens of sensors just within each chip. right so there's a lot more data taking place and you're having to monitor and sort of track all kinds of signal across again not just a server but a rack or an entire row of servers into these big memory clusters and and so that the job is getting there are more sensors and and the tasks being asked of them are getting much more complicated and it is it's meaningful in that it you know the way these things perform can have it can have a real impact on the performance of the server.

30:57Jay Goldberg: You start talking about saving 2 % of electricity with these things and across a gigawatt of capacity, it adds up very, very quickly to big dollar amounts. And so there are lots of ways that people are looking at this. You mentioned a few of them, FPGAs, microcontrollers. Marvell clearly has something here. They teased it. They don't call it a BMC. They call it AI infrastructure management, which is not a product. Which is this, no, it's not a product at all. So, yeah, I noticed that too in the presentation. I was like, I was really wishing they would tell us more, but they just kind of teased it.

31:36Jay Goldberg: Right, and I think, like, I know we're getting deep in the weeds on this, but what was interesting about this event was they had similar conversations and positionings for products around, like, a dozen different categories that were equally in the weeds as the BMC. just to show, again, how complicated the servers are becoming and how wide Marvell's opportunity is to play in a lot of these categories. Again, they've had these kinds of capabilities for years. They were just never big enough to really merit attention, and now they're all becoming probably multi-hundred million, if not billion-dollar categories.

32:15Yeah, agreed. Okay, last thing I'm just going to say.

32:21Ben Bajarin: And it's just because I just want to land this point. I don't think I did. It's just I keep coming back to when you talk to the engineers,

32:33Ben Bajarin: creating this world of compute that we're talking about, which is watts up and to the right, more chips per board, incredibly complicated networking. You just hear them constantly say, these are very hard engineering problems. Like we're up against just engineers have to go solve problems. And it's just interesting to me what they do, how they do that today, how they rethink logic board design, for example, how they rethink that in a world where power goes up drastically. How can you fit all of this in one board, right? And how many capacitors and MLCCs and additional silicon, like all these things that are needed, sensors, analog, whatnot.

33:15Ben Bajarin: it keeps getting more complex and it just keeps compounding engineering challenges. And so I just want to point that out, that this workload as it scales, and again, as models get bigger and you need more compute and you need to serve more people, like these are problems that we don't have solutions for. Again, like this at IR Labs, right? His whole point was the engineer mindset is optimistic, but also understands these are hard problems. And there are sometimes not clean answers to this. And that's what they have to go. They just have to go solve hard problems. And so that's why I think these things are interesting.

33:57Ben Bajarin: There's opportunities for players in semis and analog and whatnot to create new products that didn't exist before because we didn't have the needs. Like you said, a general purpose server, I think we had largely settled a general purpose server. Like that was fixed. You knew how to do it. There was no new challenges. And now this is creating all these new challenges, which, again, my point is creates the opportunity for new silicon innovations in both analog and logic. So I just find that interesting. All to say there's still a lot of opportunity. We don't know where it's going, but that's why I like what they did.

34:36Maybe it'll work, maybe it won't, but create new stuff that solves new problems.

34:40Ben Bajarin: Don't try to solve new problems with legacy silicon. And I think that's a fine place to be.

34:48Jay Goldberg: makes sense although you said analog you said the same i can analog too and i just want to point out analog or semiconductors analog semis or semis yes i know i gotta say i love i love the analogs

35:01Ben Bajarin: as well i just i like calling it out because it's a very different problem um also again and i think you're going to see it this week to be honest with you you're going to see innovation in analog because they recognize this and they are working to solve problems that didn't exist before and those roadmaps took time over the last two years as this became clear and now we're out taping out new silicon so fascinating times on that and anyway all right let's move to uh open ai some news this week that i guess hurt uh hurt hurt hurt yeah news quotes quotes around that news But there seems to be some confusion between the recognized revenue and annually recognized.

35:48Ben Bajarin: So I'll let you explain it and then let's see where we landed.

35:53Jay Goldberg: So there was a story. I forget where. I think the FT that said or somewhere. Someone said they had gotten a hold of OpenAI's latest data and their revenue was something like$20 billion below where people expected it to be. So, all right. So let's just start off with expectations. Like who established the expectation? Two,$70 billion versus$50 billion in ARR. So we're dealing with ARR, which is a funky number to begin with. And then three, it absolutely looks like what they were talking about were different accounting treatments. And so I don't know how much damage it did, but I know that like semis were off pretty bad the day that came out because it looked like, oh, my goodness, OpenAI is numbers are way below target.

36:52Jay Goldberg: The world is ending. Right. So, okay, one, it turns out that the revenue numbers are just off. There's different accounting treatments. You can look at gross versus net. So you get a bunch of revenue, but you have to pay for that revenue and traffic acquisition costs, right? It's just different treatments of the same trend. And I don't know if OpenAI came out and explicitly said it, but certainly lots of other people were pointing out that, no, no, no. If you actually compare like for like, nothing has changed. So there turned out to be no substance to this story. It was a mild, mild panic freakout on something that lacked substance or at least was making a bad comparison.

37:37Ben Bajarin: But at the end of the day, people were pointing out, I mean, look at what their revenue was last year, which I think was estimated to be in the 20s. And so even if it's 50, you're doubling that. There's obviously a difference between what they're annualizing versus what they're recognizing. But regardless, like any numbers you have, which include like numbers we've been trying to do on the hyperscalers dedicated AI spend, obviously not just Anthropik and OpenAR. And then you look at, you know, the sales forces of the world. Like we are doubling from where they were last year. I mean, you can make a strong case, and most people have these estimates, that even if we're tracking ARR, the dedicated AI spend could be between$250 and$300 billion this year.

38:23Ben Bajarin: That's up. That's up over last year. So it's growing. And so, you know, we quibble on numbers. That's still a lot of money to be doubling. That's still a lot of growth rate. But I think the sensitivity is something we keep coming back to, which is people are really worried about this. And they want to make sure we're funding, right, that money's flowing, that people are paying for AI. And that's a sensitivity in the capital markets.

38:50Jay Goldberg: Yeah, I think to me that that was the main takeaway here because the numbers themselves, I don't want to argue about imaginary numbers. Like just file. Everybody just file your numbers so we can see the real ones. But the real issue is the fragility of the semiconductor trade. There is growing awareness that all of this spend is being driven by six companies, really two. And those two are massively loss generating. And there's an awful lot in both the equity and the debt markets riding on OpenAI and Anthropics prospects. right but I don't know that we have any reason to freak out about their numbers but like there is this growing awareness that the trade the whole semi-complex is fairly yeah and I think I mean I think that the intertwined

39:49Ben Bajarin: nature of it how is going to be a sensitivity and I saw somebody there was a note I cannot remember who put it out that was like that basically said you know It's not demand that crashes any of this, which I agree with. It's some unforeseen issue, like, for example, a massive cybersecurity attack or a complete data breach, something that causes you to say maybe we should slow our rolls. It could even be that we just literally cannot energize gigawatts in an annual basis to keep out the compute, something that I'm most worried about these days, just trying to be as in the weeds we are as tracking how much is energized at what time based on what the gigawatt of compute forecasts are.

40:34Ben Bajarin: Those numbers do not line up. And that's just a reality of it. Now, does that ease in a 16-month to 18-month timeline consistently? I don't know. But right now, what's energized annually is not matching up with what's being forecast in Accelerator Shell. So that's just the reality of it. Now, that could hurt. What happens if they're like, eh, we're going to slow down until it's energized. We can't rip and replace. And that hurts somebody's numbers. Again, it's not because its demand goes down. It's just infrastructure takes time. It's hard. We don't have power, right? And that will hurt the trade.

41:10Ben Bajarin: But it's not the demand. It's the interlocking of all of these supply chains that one or two things could slow a lot of it down. And that will trickle into people's numbers.

41:23Jay Goldberg: Yeah, I think that's fair. You know, my sense is there's a debate about how much electricity we can add next year, weed being the United States electrical grid. And I've heard estimates from 20 gigawatts to 30 gigawatts, which in absolute, right, which is in absolute terms a staggering number for something that's barely grown for decades. But we're going to add 20 to 30 gigawatts next year. The energy people I talk to, or I don't really know many, but, you know, The research I read on that is towards the low end of what we can actually do. And the compute people I talk to are all bulled up. They're hoping that we can do – they secretly have aspirations that are above 30.

42:08Jay Goldberg: And so, yeah, that's not going to – that doesn't seem likely to happen. But here's the hypothetical I've been posing a lot to people lately is next year, somebody – some big, big data center, one of the big gigawatt scale data center hyperscaler clusters gets pushed out for six months. We can't get enough transformers. We can't get enough turbines. Or we had some regulatory hiccup and, you know, the appeals office isn't open on Tuesdays, whatever. Something gets delayed for six months for some reason beyond control. And what happens to semis at that point? I think if that happened today, the street would look through it.

42:52Jay Goldberg: No one would care. Okay, six months, right? How are people going to view it a year from now? I don't know. Maybe they won't care. Maybe they'll see, oh, six months, no big deal. But there's so much complexity built into it and so much expectation built on top of it that if, let's say, one of the big semis companies says, hey, we were going to sell a zillion units into that cluster. They don't need them right now, so we're going to push our numbers out for six months.

43:26Jay Goldberg: I don't know how people react to that. Again, today I don't think people would care. They say only six months. But like, you know, because inevitably we're going to have that kind of thing happen. and i think so so long as we see open ai and anthropics numbers continuing to go up the way they've been going up to your point there's demand that shows there's demand exists and yeah people sort of look through it but it i i think you know there there is yeah next year i think it's going to be pretty tricky because you think about it like back in january february march all everybody was talking about were all the bottlenecks right we've stopped talking about those as much, but that's, I think, a function of the industry has now scoured the landscape.

44:09Jay Goldberg: We found where all, we know where they all are. We all have schedules for them, right? But that doesn't mean the shortages went away. We just aren't, we've sort of thought through them. We're still going to get to a point next year where we won't have enough gas turbines, right? Just because you look at the production schedules, they can't produce enough.

44:25Ben Bajarin: So I think, and then that's like, I had put out a report and we talked about it, you know, that I think next year is peak of the constraint if you just look at what's happening in 28 and 29. It will ease some, but it took a while for everybody to build out everything they need to build from clean room space to start planning for more gigawatts, like all the things. That took years. So I think you're right. I think next year gets a little rough. how long that cascades right into 2028, I'm not sure. But I just want to share a general point because I haven't shared this before. It's food for thought about why we might give the benefit of the doubt to this cycle compared to other megacycles.

45:16Ben Bajarin: And it has everything to do with every other prior megacycle,.com, railways, cars, I mean, electricity, name a transformational technology. You had to wait for adoption for those to scale. You were basically building out in the hopes that people adopt it. And by adopting it meant in electricity, you had to have your house wired, you had to spend money, or you had to buy a new house in order to go get that, right? Or in the dot-com phase, you had to get a computer that you didn't own that costs thousands of and then sign up for a plan. And so people had to go get hardware, kind of mobile was the same thing, right?

46:00And telco, you had to go get a phone in order to use the backbone.

46:04Ben Bajarin: This is different. Everyone can use it right now with the existing hardware. There's no adoption cycle of new costly hardware or things, widgets to drive this cycle. It can be used today. And that's why I think this is a very interesting time, which compounds demand because it's freely available for everybody to use in some capacity. You just have to start using it. And that's a very different thing. And I think that's why we keep talking about, and you hear everybody talk about, demand is there. we're actually building to meet the daily use of billions of people who could use it right now, which is, again, very different than any other cycle.

46:53Ben Bajarin: And I'm sensitive to that this just is going to take time to build what we need. We could argue at the end of the day, the only two things that matter is how much compute is needed for six billion people to use it on a daily basis and how much energy does that take? Those are really the two things. If you could figure that out, you could land on how much we need and are we close to it or not. And the reality is we're probably not. But at some point, perhaps, right, we are. We've built what we need to serve billions of people, agents on a daily basis. But my point is it's different because it's available to use today.

47:28There's no waiting for hardware.

47:31Ben Bajarin: There's waiting for infrastructure to serve this at scale.

47:39Jay Goldberg: that's a little concerning to me because if you think about the internet boom bubble a big factor there was people had big plans and grand ambitions a lot of companies built up on the idea that we could they could do all kinds of interesting things and the problem was the infrastructure wasn't there both the physical infrastructure well everybody was interested in But like the physical infrastructure wasn't ready. I mean, the physical infrastructure wasn't ready. So like WebVan, you know, had the right idea. They just were too early and didn't have enough warehouses. It took Amazon to deliver on that.

48:19Jay Goldberg: No pun intended. Maybe a pun intended. But then more importantly, I think the other issue was the bandwidth wasn't there. Right? Not enough people had high-speed internet in their home until 2004, 2005. And we definitely got to a point where we maxed out the infrastructure and it was insufficient. And that sort of caused a lot of problems. Everyone started missing their numbers because users just couldn't access the service in the right way. I don't know that this time is going to play out like that. But it is a concern that you have a lot of frustrated demand because the infrastructure isn't ready and it's going to take a few years for the infrastructure to be ready.

49:02Jay Goldberg: Do we – is there a mismatch in expectations versus –

49:06Ben Bajarin: Well, and I think my bigger concern is that in that is as you start to roll this out to your employees and all of the things that government officials, whatever, whoever your customer base is, It's like, are they frustrated, like you said, because it's slow or it's costly or it's X, Y, or Z? Does it not deliver the expectations or hope because they're up against, I can't use it or it's slow or it's failing or it's not doing what I want? And these are functions of both the technology and perhaps its ability to serve at scale. And so I think that's parallel to this. But like I said, the reason that I think the benefit of the doubt exists in some capacity is just because you have potentially billions of people interested and willing to use it today, and it does not require any new hardware.

50:04Ben Bajarin: And that's very different than every other megacycle that required the customer to acquire something new that cost them something in order to start using it.

50:18Jay Goldberg: yeah but i just think of sort of the basic functionality of the web things that we take for granted today you know think about like javascript right the the 1990s web didn't have javascript today it's like built into every website we don't even think about it you're able to do all kinds of things before the website is fully loaded uh you can you can execute code sort of on a website, like there's all kinds of little intricate, little nitty gritty details that are today a fundamental part of how we experience the web, which were not possible on dial-up modem, right? And so I'm just wondering, and I guess what I'm getting at is, to me, I still have big questions about how people are going to use AI more broadly.

51:06Jay Goldberg: I don't think we're quite there yet. and I don't think there's a lot of functionality that there's a lot of things that are just like, like you don't really need a lot of AI for a lot of the things we do today. It could be much better. It probably will be much better, but we aren't there yet because of infrastructure reasons. Right. And so will we get to a point where, I mean, certainly I think the average person, at least in the U.S., is like really sick of hearing about AI because all they hear about are the costs, the data centers and the cyber attacks, Skynet, and they're not really seeing much personal benefit from it.

51:46Jay Goldberg: And I'm optimistic about AI in that I think we will get many, many more good things, but I can understand how people will be frustrated and kind of just get disillusioned.

51:56Ben Bajarin: This just goes back, though, to the excellent conversation we had with our friend Benedict Evans, who kind of made some similar points that I'm making now. but he said it really comes down to UI. Like you have to make it easy for people to use. And as a matter of fact, I tweeted this the other day. Like I, you know, I think you and I know you're, you've been a Claude camper and I've always been in GPT, but I'm now using Claude a lot because to be honest with you, I think their interface got better for projects. Like now that you have one central chat agent and then it organizes your threads in your projects, like for, like that's great.

52:32Ben Bajarin: And my wife is using this now for teaching because she has the Claude teacher plan. And it's actually a very good UI. It helps your mental model use it better. And so that's like the harness, the thing that shows you what you can go and do and how you can go and do it and makes it easy is the other part of this, right? Because while you don't need new hardware, which is my point, you do sort of have to rethink all of this. And it needs to be a UI that helps you get the most out of these things that is still kind of the we're still in those stages. How do you make it easy for people to do this?

53:13Ben Bajarin: in fact I just noticed today with chat GPT's update yesterday it is starting to pre-fill preempt prompts for you based on what you were doing in that chat thread it's like a little ghost you don't have to use it but it's kind of showing you a suggestion on what you might want to say or ask next based on what you were just doing and so I think again this is just this is the UI. These are the interfaces things we're going over to get not just people onboarded, but to get them using it in a more regular fashion. And this is the UI battle at an enterprise and a consumer level is still early days of that.

53:55Ben Bajarin: So I think that's interesting too.

54:01Ben Bajarin: Okay. Last time.

54:04Jay Goldberg: Yes.

54:04Ben Bajarin: Can we do one more thing?

54:07Jay Goldberg: All right. So this week, Lisa Sue from AMD was in Asia. And she said, I think she said to the press, or maybe she was at an investor conference. But she basically said, they expect to get greater than expected capacity from TSMC next year. Right? Which is good news for them. Right? Because obviously, TSMC capacity is very tight. It's the key constraint on all of this. And so, all right, that's good for AMD. but I also think it is a positive signal for Intel in an indirect way. This is my latest conspiracy theory. We have talked about this over the last three, four earnings calls. TSMC has gotten much more serious about how they discuss Intel.

54:58Jay Goldberg: A year ago, they were like, oh, Intel's a great partner. We love them because Intel is a big customer. now in the last year lippu has come on and intel foundry has solidified considerably to the point there that tsmc now takes it much more seriously and you see that in their language today they talk about intel not coldly exactly but much more third you know sort of more distant Right. Yes. Intel is a partner. Not there's no fondness there. And I think TSMC is now, I wouldn't say worried, but recognizes that Intel Foundry has gotten to the point where they are technically very capable of competing with TSMC long term, both in terms of EMIB and 14A.

55:48Jay Goldberg: They have, I would say, on paper at least some clear competitive advantages over COOS and A14 from TSMC, right? TSMC sees as a competitor now there. But they probably recognize or they probably are thinking that the biggest constraint on Intel becoming a more serious competitor to TSMC is Intel's ability to raise cash. And so wouldn't it be nice if Intel suddenly had a harder time generating cash flow from CPUs because it was seeing a lot more competition from AMD? Because, you know, there's huge demand for CPUs. And if the constraint on CPU demand is TSMC capacity, I think TSMC is giving that extra capacity to AMD to do what it can to throttle Intel's cash flow generation.

56:40Jay Goldberg: They know they can't stop it, but maybe they can slow it down. Okay. So I guess I have one question that I'm going to add to this conspiracy theory.

56:53Ben Bajarin: Where are those wafers coming from?

57:00Jay Goldberg: It's a good question. But, I mean, there's lots of potential people. I mean, for one thing, Intel, right? Intel still is going to do a lot of wafers at TSMC next year. They're not getting the upside. TSMC is obviously adding capacity at a pretty fast clip. TSMC is now making those decisions about who gets that capacity, and Intel is not going to get them.

57:24Ben Bajarin: Okay, second to this theory is almost everybody who tracks this space, us included, even though I've been saying this year for next year for PCs and the year after is going to be absolutely horrendous, has been upping their estimates of how bad this is going to be off. IDC, I think, now is at like 20 % for next year. That frees up wafers. And what's intriguing to me is to this Intel point, like Intel actually, this is going to sound so ironic, and I know I'm possibly stretching, but let's just again throw this out as a hypothetical. In no other prior cycle had Intel even, a down PC cycle would have been awful for Intel.

58:18Ben Bajarin: Like not nail in the coffin, but abysmal. However, when you have a foundry and you have wafers that you need, that you're happy to shift to a higher ASP server CPU, perhaps for the first time ever, abysmal PC sales is actually a very good thing for Intel in two entirely different scenarios. And I just think that's insane to think about, but no joke, PCs are going to be really bad the next two years and that frees up wafers. And to the AMD point, AMD actually did have a lot of wafers continually for PCs. So if they allocate some of that also to server and server CPUs and um helios asics um that could explain some of this too um but it's interesting to think about but uh there will be a wafer springing up because people are gonna really take back what they're gonna make in pcs think this market's gonna be pretty terrible

59:23Jay Goldberg: yeah i i think we're already seeing that um in other in other sectors where you have companies that have excess TSMC allocation for this year even who are not selling as many of their traditional end products and are repurposing that for, say, custom ASICs for one of the hyperscalers. I think we're seeing that absolutely. We're repurposing TSMC capacity for something beyond core business. I think, yeah.

59:54Ben Bajarin: Okay. Well, that's our time, everybody. hope to see folks this week. I know we will have a very busy podcast next covering a ton of ground from this week. We appreciate your time and thanks for listening. We will talk to you later.

1:00:11Jay Goldberg: Hit us up this week. We'll love to meet you all. Leave us a review. Tell your agents to leave a review and subscribe. Thank you for listening.

1:00:27Thank you.