Has the Memory Cycle Broken? Plus: Nvidia & Broadcom’s $40B+ Financing Playbook

0:05Ben Bajarin: Hello, everyone. Welcome to another episode of The Circuit. I am Ben Bajarin. Greetings, programs. I'm Jay Goldberg, and we are joined today by all sorts of avian noises. Ben's chickens. My cockatiel. I told my rooster it needs to be quiet during the podcast, but it is a disobedient rooster. All right, let's kick this off with the earnings of the week, which was Micron. And a couple of things I think are interesting to talk about. There is certainly no shortage of opinions on memory. So one, let's talk about the earnings. And then two, does anything that came out on this change, like the thesis for memory?

0:57Ben Bajarin: Because I backed into this and before we talk about the earnings, but I backed into this and essentially, while I think everyone agrees that the dollar value of memory has gone up, right? So if you've looked historically, it was a peak ebbed and flowed as memory did, but as a peak, it might've gotten to a hundred billion dollars. And now people are saying, right, our model included, you could be at between a$600 or$800 billion, depending on ASP, floor now. So the floor has changed, right? Big growth glows. The floor has changed. Nobody disagrees with that. However, it seems like the way the market is valuing these is that they're assuming that crash comes in 29.

1:43Ben Bajarin: but like either that that cyclicality value goes up and down between that new floor but that essentially the wheels come off the train like not not that it's as a is an industry that needs to go up into the right that's fine right but if bit growth continues and pricing degree leverage continues to some degree and you have this new floor that's where i sort of think the exercise is okay like are we thinking about even in some degree of cyclicality the right kind of forward assumptions here for micron and some of these other names you used the c word crash i mean but that's it that is what it's calling a crash in 29 everybody but but that is what's being modeled like what's being modeled is demand destruction and a high degree of margin compression right which which could again if they normalize to the 60s great like we can run those scenarios but i think the reality is it's not being the sustainability of bit growth and some degree of pricing elasticity is certainly not in kind of the long-term modeling i i i think yeah i i mean there's all kinds of arguments people are making right now about where memory goes like i think your scenario calling for a decline and or i understand you're not really calling for that but like that's the scenario you laid out is which this keeps going into 2029 and then conditions worse than that whether they crash or not is a whole other story but through 29 this keeps going through 29 and and then we have some kind of you know beyond that it's it seems it's too hard to forecast and we'll probably have some kind of reckoning there are people who are even more bullish who say that no memories fully structurally changed now we have all these long-term ingredients in place and that's gonna we don't have the same whipsaw cycle boom and bust it's gonna be will be cyclical but it'll be much more muted cyclicality I think that's the sort of most optimistic outlook.

3:57And then there are other people are saying, what we're seeing now is with the memory results are being driven by pricing and output bit growth is much slower. I think the company said DRAM and NAND are growing sort of in the 20s, low 20s for DRAM and mid 20s for NAND into 27, 28. And so I think the bear case would be we get to supply-demand equilibrium in 28. The market sees that and starts forecasting, starts getting much more pessimistic in sort of some point next year. and I'm not taking a stand here I don't I don't know what's going to happen but I do think it's notable that prices doubled right prices are basically doubling this year and it sounds like there are agreements in place that pricing goes up again next year but output is only in the 20s and set against that you also have all the memory makers adding significant amounts of capacity so i i think i think we get to some some parity earlier than that but how that actually plays out in the shape of that downward curve when we get there is it's very unclear right now right you know i think past cycles everyone assumes that like it's it's memory cyclical and it goes for two three years and it crashes and we go from 80 gross margins to whatever 30 percent gross margins and i think there's a reasonable argument people could make that no that's that's not how it'll play out there's lots of things that will smooth it and i think you know i think for for many people what matters most is the cost of memory is just like everything is getting so expensive right they were talking about because of agentic ai which is driving cpu demand And that's driving demand for, you know, not the fancy HBM memory, but sort of cheaper LPDDR memory and other stuff that goes into CPUs.

6:08So even that category is getting squeezed. I mean, ultimately, to me, this comes down to how long it takes them to get capacity online.

6:18Ben Bajarin: And when we start to, you know, build up a lot of capacity, possibly overbuild capacity. and I think that comes sooner but not soon right and then we haven't even talked about like where China fits in on this right because we also had lots of news reports coming out of China CXMT is pulling forward all its capacity targets they're going to have a lot more capacity next year they're building fabs very quickly I think that's the thing that people miss most about Chinese memory isn't so much that they have you know they're as constrained as everybody else, but they are able to put capacity online much faster.

6:56It takes two and a half, three years to build a memory fab in the US, Korea, or Japan. It takes 18 months in China. And so how does that play into it? I think that leads to, we're going to have a lot of memory in 2028 is I think where we get to. And how that affects pricing is the big question. I don't know the answer. Yeah.

7:16Ben Bajarin: I mean, and I think there's two parts of this also, and we should talk about the Micron sort of specific report as well. But there's pin compatible dynamics. And I think if you just look at where pin compatible standardized industry memory maintains segments, right, which is going to be phones, PCs, consumer electronics, maybe to some degree, accelerator attached, like if it's SOCAM or whatnot, and you can just put anybody's DDR right next to your CPUs, fine, right? That's not a design-in dynamic. That's going to continue wherever you believe that is, how big you believe that is, what segments you believe that standardized industry, pin-compatible dynamic is relative to, is going to remain price-sensitive, price-competitive, right?

8:11Ben Bajarin: Whoever has the best quality, so speed, reliability, all of those things, plus the ability to go pricing, right? You can just mix and choose your vendors, right? So that part, again, however big you believe that market is, like we have assumptions on that, that's going to vary, right? That part could either be cyclical and or that part could have ups and downs and pricing and margin because it's more competitive, right? The other side where you're designed in, you're specced in, you're in the advanced packaging parts of this, so HBM-ish. And then I think there's, because the other point I make is like memory innovation is not stopping.

8:54Like what we have today in memory around HBM or, you know, just again,

9:00Ben Bajarin: let's just say DDR and the whole thing. Like that's not slowing down. Like there's memory innovation coming that will change this as well. That will new products, new process technology, new transistor designs, new stacking, all of these things will come. So there'll be a segment that maintains its value, right? That I believe. Again, but then you got to come back and say, okay, how big is that, right? And what are your assumptions? But I think that's the part where you, you know, meet those in the middle, right? I would not expect margins to go back into the thirties. And again, right, this is an industry that has historically been single digit average to low double digit average in margins, right?

9:41Ben Bajarin: So even if you're at 30, you're, you're, you're more than two X where they were. But, but I think there is an assumption around like, okay, could it normalize 60 % range, you know, mid sixties, low seventies, something like that. When you do have the most expensive bits remaining the most constrained, tied to massive accelerator shipments, and tied to trying to solve massive memory pooling, and I think the other interesting element is you're going to want to keep, again, assuming that bit growth goes up by demand at just a high level, right? Because you're right, bit growth has slowed now, but it's because we just can't make as much.

10:24Ben Bajarin: Bit growth would be almost infinite if you talk to anybody who makes logic around AI accelerators. It's memory-hungry. All of that means bit growth continues to grow. So then again, premium bits, commodity bits, I think is a healthy debate, a part of that, as you think about where that attaches to. But if BitGo continues to go up, why would your memory makers do what they did in the past, which was just take old clean room space and replace it with new ones? Keep that clean room space. Keep making DDR4. If that's going to work for CXL and you can build a whole bunch of disaggregated CPU solutions, keep making DDR4.

11:10Ben Bajarin: Keep selling it. And they are. They're doing that, right? So there's not a take legacy memory offline like it was in past cycles because they're still going to need that Bikra. Everybody can still use that memory in some capacity. Not all of it needs the premium HBM-5 or DDR6 or whatever. So that's part of the capacity expansion. And again, why I say the category raises in TAM. Like the TAM is growing for memory. Where it settles is then the question of where you then balance, right, those new revenues for the main players within some ebbs and flows, which is going to happen, right, in certain pockets, maybe not all.

11:57Ben Bajarin: But that's why I think it's interesting when they talk about capacity coming online, what's going on with older generations is that it's not – it's a little bit more like TSMC, right? You still monetize legacy for a long period of time. That's never been a dynamic for memory. And I still think the demand will necessitate some degree of later generation memory fabs keeping going because there will be customers for that memory. I get that. I would like more signal from the memory makers that they hear that because in all of what you described, there's an opportunity cost. like I right you're you're you're saying that they're going to keep making legacy older versions of memory I'm I'm not confident of that right I I you know I would you know we talked about this a little bit last week there are some reports that like memory prices for consumer memory have have plateaued which I would read as a deliberate decision by the memory makers but I'm not sure that's true that's just my interpretation of it and I didn't get clear any confidence in that from from the micron call and so i i worry that they're adding all this new capacity they're investing all this money in r &d they have to have this very strong temptation to dedicate that capacity and that r &d to the to the fancy memory at the you know i i would just like some relief for the consumer side of things right um right we talked about this a couple weeks ago with with hot chips everyone is complaining, like, don't make it higher, just make more of it.

13:35There are real trade-offs, right? You start making 16 high HBM memory, the effective yield on that, the amount of wafers that it takes to produce that is significant. And it all comes at the expense of other categories of memory, as long as they're constrained. And we had this news this week that Tesla is de-specking memory I think for robots, maybe for the cars too, but I think certainly for the robots. They're de-specking memory. They're using less memory in the robots, largely because they just can't get enough memory. And so I worry about the incentives here for the memory makers and how much that leaves for everybody else, everybody else who's not using expensive, fancy HBM.

14:29well and that's why i say like again does every does every new product or even you know consumer

14:37Ben Bajarin: demand like you said like the like the the most cutting-edge memory like why could it not keep using if you've got a billion smartphones a year and you know whatever 300 million just shy of 300 million PCs, again, granted, if PCs are going to start running a whole lot more AI, then I get you might need some degree of leading edge DDR. Probably not phone. I don't know how much you're going to run on phones someday, but let's assume that legacy memory still works for some of those scenarios. And then I could keep those prices down. It doesn't need what goes on in leading But my point is, I mean, DDR4 is a good example.

15:21Ben Bajarin: It is staying in the market, right? DDR4 is still going to be sold. There's people who are going to use it that need it. And that will help normalize once they build new factories. Now, again, how long would you keep this on is going to be the question. But my point is, I think there's logic to keeping at least a generation or two going in older fabs. But my point is, this is not the way memory worked before. memory was clean room reuse and so if that changes then i think that is the argument for some degree of structural change to memory again a little bit more like like i said logic where trailing edge should we even say trailing edge memory i don't know like is that a term we're just going to invent yeah we do what i mean but something like that stays monetized for longer periods of time.

16:14Ben Bajarin: And like I said, I think you can make an argument that that's a good use of their clean room space when they're dealing with this kind of capacity constraint and to some degree of a much larger TAM. Because that's reality. The TAM is larger. It's going to be bigger. How big it gets is an unsettled question. But it's a structurally larger market for memory, and that's not changing. The memory demand doesn't go away. It might normalize and be somewhat static at some period in the future, but it's not going away. And that's the difference between how big this market has gotten. And now we just talk about how do you price it?

16:48Ben Bajarin: What gets priced in? What's your ASPs? Where do you have premium leverage, et cetera, right? So that's, again, that's why I like the main report I did this week was called bet on bits, not history, because bit growth demand is inevitably going up, which just means you've got to be able to manufacture that in a bit growth scenario up and to the right. everything else in between price vendor share etc is is negotiable i i i take your point i just i i worry that you know that the the memory makers i don't i'm not convinced the memory makers have internalized what you've just said right the idea that they're going to now run a tsmc model and maintain trailing edge capacity um when it takes you know let's let's call it i don't know four you for every four sort of trailing edge memory you get one hbm leading edge memory i actually i think that's the big difference here is if you think about in in like traditional moore's law leading edge is denser more capacity than trailing edge and here we're kind of talking about the reverse because the really expensive leading edge HBM stuff is, is less dense on a, per wafer basis.

18:11Right. Like a, like a node shrink typically. Right. It's, it's not, yeah, it's a reverse node shrink. You're actually dedicating more capacity to something that is less, you know, less productive. I'm, I'm doing the math wrong, but you take my point. And I don't, I, you know, I, it certainly makes a lot of sense for them to maintain that trailing edge capacity to keep everything stable and sort of the long-term view of it. But let's say it takes four – the capacity for four consumer chips produces one leading-edge HBM chip, and they charge 100 times more for that leading-edge chip. The math works very heavily in the favor of just throwing everything they can at HBM.

18:55That's a very strong temptation.

18:57Ben Bajarin: Right. Right. If, again, if you are up against some degree of clean room capacity constraints, right? If you have more options, like if you're sitting here looking at your land, you know, space, and you're like, look, I've got all of this capacity. And how do I then allocate, right, now that I have more? My point is you can make different decisions if let's just say, and I'm not saying this is generally the case, but everyone doubles memory capacity in clean room space. You essentially could say, okay, what then is the best way for me to normalize this, meet my demand, support bit growth? Because you're right, like leading edge is going to give you more capacity.

19:40Ben Bajarin: Like whenever they move to the next transistor designs, it's going to give you more capacity within the same space. So you'll have those factories for that. My point is, if you're doubling your footprint or more, you can make different decisions than you did in the past about how you balance both your bit growth and then who the customers are, assuming sustained demand for, let's just say, something like trailing edge memory. I take your point, but I also think that there is real damage being done on the demand side for non-data center memory right now. Right. If you look at the, you know, I was talking to somebody this week who has been forecasting mobile phone growth and they think we're in for a decade of no growth in mobile phones because if you think about phones now, the upgrade cycles are sort of globally with four years and they're going to push out because this year demand is down.

20:36phones are getting more expensive. They're less, they're less good. Nobody's going to buy a phone now. They'll, you know, they'll get to the point next year where their five-year-old COVID phone is completely unusable, the upgrade then, but then it's going to be another five years before they buy a phone. You're sort of, all of this is lengthening, structurally lengthening the PC, mobile phone upgrade cycle. And, you know, and it's something that will have lasting impact going into the future unless we get to a point where everybody upgrades because they get AI phones actually mean something someday.

21:10But there's real, you know, real, real potential there for structural shift in the demand side as well.

21:16Ben Bajarin: Okay, we've spent a lot of time on memory. I do want to just make sure, was there anything specific to Micron we wanted to highlight other than the fact it was a great quarter and they did everything they could to say the business has structurally changed we have long-term conversations with our customers we've got scas that are taking pay and so it's not the ltas of your father's past um anything specific micron wise takes um i i i would say number was good quarters guide was good uh stock is down a little bit for the week so i think it was up right after results but it kind of tapered I mean, it was flattish.

22:00Ben Bajarin: I mean, a couple percent, which I just would consider flat. Yeah, let's call it flat. The market's like, yeah, okay.

22:09Ben Bajarin: Yeah, the market is still debating everything that we just debated for the past 20 minutes. And literally until that changes, until your assumption of what happens, none of this is going to change. I don't care what the memory makers do. It's not going to change. Um, they have to reconvince people in, uh, in whatnot. So, okay. Um, a couple of other events this week, let's just say investor events. Uh, I was at HPE's networking investor event. And I think I remember bringing this up, uh, when I was at HPE's, uh, big event, vendor event in the fall. Um, you know, their acquisition of Juniper has essentially had them, I don't want to say pivot, but in drastically increased focus on networking, um, as a part of their company strategy.

23:03Ben Bajarin: In fact, I had several executives that I talked to at that event basically say, you should think of us now as a networking company, which to me just meant like we lead with networking. Networking is the hook. It's the conversation with customers that we have that then pulls the rest of the businesses through what we build in racks, what we build in infrastructure for Neo clouds or on-prem. And that's fine. I understand. I understand what they're saying. In fact, Dell would similarly tell you they're a storage company and storage is the hook that brings in infrastructure. Okay. and software. I get it.

23:38Ben Bajarin: Right. Fine. Um, what was interesting though, was the time that they spent just talking about, uh, the overall demand for, let's just say enterprises needing to completely redo their networks. Like, like this isn't even just a point of, of hyperscalers and neoclouds, which I think is a, is a good conversation itself around how all of this gets fabriced and switched and clustered and whatnot. But just that like enterprises themselves. And what they said though, and I think was super interesting, was it's security vulnerabilities in the age of agentic, which we've talked about before in cybersecurity, is what's freaking them out.

24:26Ben Bajarin: That they're worried folks can just backdoor easily because their equipment is super old, a campus is super old. And so now, like, we know everybody uses this. It's the mythos moment, right, where mythos has basically freaked everybody out. Like, oh, anybody can hack me now. And so I need to defend myself against this. And so that's causing so much of these infrastructure concerns. The HPE was like, we got customers coming out the door saying, you know, their equipment's old. It's vulnerable. And just on the backs of security concerns, which to me should have always been a concern. like it's not like you one day wake up and you're like oh crap we're we're screwed we we don't like if you're security vulnerable why now why now is agentic you know the threat of agentic attacks to your enterprise which is i guess just drastically increased the threat apparently right isn't it but anyway it was just interesting to hear kind of how their customers or the conversations with customers were we're starting to rethink networking and then again if hpe's theory is right, that that then leads to, okay, oh, you need more compute, or you need on-prem infrastructure, or you need more software and services, right, to tie all these things together, that that is their sort of leading hook into the enterprise.

25:44Ben Bajarin: And they showcase all of this with a Helios tray, which I thought was interesting. And so they are doing the networking part as a part of the Broadcom Tomahawk switches, fully liquid cooled. I took a picture of this and put it on Twitter if anybody wants to see, with pretty much the most insane amount of copper Ethernet cables I've ever seen in one single compute tray. And it's because, one, they're doing that, right, over ULOE, so over Ethernet, and it's a scale-up domain switch over Ethernet in the Helios rack. Um, there are six of these trays, the picture I showed in every Helios rack, which is, you know, how you need to throw throughput up.

26:32Ben Bajarin: But like they were pointing out like how hard of an engineering challenge it was to really solve some of those more complex networking bits with the leading edge Tomahawk trips from, from Broadcom over ethernet for, for the scale up domain. and so I just thought it was interesting that they led kind of with that engineering use case as an example for what they're doing in networking when that that's not a jupiter switch right for example they might have jupiter ip and things like scale out and uh and and other parts of that form factor but this was them doing the networking the hard parts engineering with broadcom with amd for those switches um and they had announced a deal with bolter right on the pack of that so anyway all of that to say uh it was an interesting day interesting long conversation with their engineers and executives on networking um but networking did they really say they were a networking company now i mean again they this is what executives told me like you can think of us like a networking company.

27:47Ben Bajarin: But if you just look at their numbers, like networking and Juniper is going to be one of the most sizable parts of their business. And so you could see their logic, but again, they also make infrastructure, right? They're going to sell NVIDIA racks. They're going to sell AMD racks. They're going to sell Intel and AMD CPU racks. So it's not like that's all they do. My point is I think that's their posture now. If we lead with this, it helps the rest of our businesses. You could disagree with that theory or not, but networking is going to be – Juniper has changed the revenue cycle of their business size-wise, and they're leading.

28:28Ben Bajarin: The tip of the spear is networking. That's how HPE views this. I covered Juniper for a very long time. and how do I put this politely? I do not agree. You didn't like their technology. I do not agree with the many points of the HPE investor thesis you have laid out. Just leave it at that. It's fair. It's fair. I agree. Security is a big concern. And I certainly agree that networking is important. I'm just not sure HPE is quite a path to get there. Yeah, I will say though, but I will say though, so despite that, right? And this is why like I land on, I guess what I struggle with with HPE is they also wanted to hit a, we work with anybody.

29:31Ben Bajarin: We stopped NVIDIA CPUs. We're good with NVLink if that's what you want. if you want to use non-envy link and you use ethernet like that's what we'll work with broadcom like they're sort of switzerland despite the jupyter points i just made um which at the end of the day my concern if i was going to is that if i unpack everything they walked me through in my discussions with their engineers and discussions i had with their engineers on the floor at the hp's event um are you just an engineering company at the end of the day like is your value that you solve you just solve these problems with great engineering because because so i was in this in fact this was a story i've been telling twitter you'll appreciate this so i was in like the helios tray is there and i was there and eric woodring was there from morgan stanley um a couple of the support for UBS and whatnot.

30:31Ben Bajarin: And so we were all, so I was asking supremely complex technical questions and then they were asking financial questions because they're trying to get at like, how much should we think of AMD's dollar value like here and across the board. But the thing that kept coming back to like every investor I listened to, like I just stood there and listened for a while who hadn't been a part of this conversation, come back and be like, so what exactly did you do here what is what is hpe's unique ip here and that was a very uncomfortable conversation because it's broadcom switches somebody else's copper it's you know we i'm a people person i take the i take the paper to the engineers you know So that was an entertaining one.

31:18But anyway, all of that to say that was the HPE discussion, but they did increase their guidance per year. So they are growing.

31:31Ben Bajarin: But again, I'm not saying that I hate them leading with networking. I'm just saying it's an interesting pivot from what we would have talked about HPE last year versus this year. Now, you know, post-Juniper and what they're leading with and good, bad, or indifferent, that is the tack they are taking. All right. I think we've maxed out our HPE quota for the year. Okay. I think we've talked about it as much as we need to talk about it for the rest of the year. Okay. that's i mean that's that's fine it but infrastructure infrastructure all right uh balance sheets as a service i don't mean to cut you off i don't mean to cut you off you're good i'm trying to restrain myself so uh yeah balance sheets as a service has entered the equation again we know that uh we've talked about this with nvidia in fact nvidia had a uh non-deal roadshow where I think they hit at least six of the big sell side banks, maybe more.

32:38Ben Bajarin: There was a deck that got passed around for anybody who tracks this. And I want to say this is the first time it showed up on a slide. Like we will use our balance sheet at least as a part of this deck. Obviously, they talked about it, but now it's like a formal part of the presentation. um but then also broadcom announced this week they're going to help anthropic uh fund some of these things so i know there's a ton of negativity viewed on this and and you don't think any of this is a great a great sign um well i get i get the yeah i didn't go ahead rant on balance sheets as a service so i think i think it's it's complicated it's nuanced it's not complicated it's nuanced I saw that in the R-DEC that NVIDIA passed around to investors this week, and I found it very humorous that they do have a slide talking about their balance sheet and the way that they're using their balance sheet to fund growth of the ecosystem.

33:45And this slide is basically just a picture of the footnotes of their 10K that show all their obligations, right? And I have seen people use that exact same slide, that exact same image in a negative context to say, look at how much this has grown. Look at how big these obligations are getting. and here's NVIDIA saying look how much our obligations are growing. They're bragging about it. It's like that meme from The Big Short. Why are they confessing? They're not confessing. They're bragging. This is like peak Jensen. He's like, oh, you're complaining about this? I'm going to show you why it's fantastic.

34:37So I think I mean, so I'm obviously somewhat cautious about NVIDIA's ambitions here. They're spending so much money. They're putting so much on the balance sheet. They have a legitimate counter to that, which is that, no, we're investing in growth. We're investing in ecosystem. We're enabling this huge layer of customers. And I think that there is truth in that up to a point. and it's going to be very, very hard to know how much is too much.

35:19And so I'm not going to say this is a terrible thing that they're investing in all these things. It's not circular financing. It's vendor financing. I think people should look at this and just go, this market is incredibly competitive and capital constrained, along with everything else constrained. and then we can have a legitimate conversation about how much is the right amount for NVIDIA to spend here.

35:47Because up to a point, it's going to help them, but it's just like every step ratchets up the risk.

35:56Ben Bajarin: All right, so I have a couple of questions.

36:03Ben Bajarin: Well, obviously, and we made the point. So Broadcom's doing this too. Sorry. So throw that into the context. So yeah, so so so yeah, so Anthropics Reuters got a hold of a preliminary copy of Anthropics IPO S1 filing. And one of the one of the really interesting things in it was that there's a section in which Broadcom has committed to loan Anthropics up to 40 billion dollars, 42 billion dollars. And this is around the construction of TPUs, right? Google TPUs, licensed produced through Broadcom, going to Anthropic, and Broadcom is now putting up a fairly large amount of money for that, following in NVIDIA's pattern.

36:52pattern. Yeah.

36:55Ben Bajarin: Okay. So, so I say that to say what you then have is your two undisputably large AI infrastructure players in both categories, merchant accelerator and XPU, who are merchant, fabless semiconductor companies, and they're the ones making these decisions. So I'm curious, I mean, and I have a theory, but I'm posing the question, why does it make more sense? Okay, so again, knowing that this, your customers, this is a high fixed cost scenario, that is the semiconductor industry, high fixed costs. Why does it make more sense that your fabless companies here who have those customers or want to sell to those customers do this instead of your foundry like TSMC or Intel, for example?

37:50You mean you think TSMC should be putting out its foundry too?

37:54Ben Bajarin: Somebody, somebody, I'm saying somebody is going to, is looking at this saying the demand is inevitable. I want you to make these things with us. You need help because it's expensive. I can monetize that in a decent way. Why not the foundries? Now, again, that's a high fixed cost business. I get it. But I'm saying like, but the point is there's a distinction here. These are fabulous merchant companies or custom who are doing this. why why why why are they why is it the right approach for them to do this versus the foundries where this all ends up anyway and they know that demand and they get additional money

38:34so tsmc is going to spend what 50 something billion dollars in capex this probably 60 it's probably going up yep so so they're going to spend 60 billion dollars this year in capex and you want them to take on more risk okay but again their margins what are they going to get what are

38:51Ben Bajarin: they they're trying to raise margins but they want to stay low so my point is my point is if it has to be done if you know that demand is there and you're going to make money on it you're going to make a return on that so i don't i don't think why i don't think tsmc doesn't know that right TSMC is too far up the chain to really, really know true end demand. And this is what tripped up foundries for years is they don't have a good forecast of how many phones everyone are going to buy this year. They come to people like us to get those estimates. I get it. They don't have a great forecast there.

39:31They're not touching. They're not talking to the end customer every day. um you know it's it's like it's in the news when someone from google or sam altman goes to taiwan to talk to tsmc that's it's rare enough that it's in the news um and so how how they don't have a good idea of how much the end demand is and and two i would also i think it's a pretty good argument that nvidia is and broadcom are better at monetizing ai demand if you look at the you know they have higher, well, sort of higher gross margins than TSMC does, right? So there's a lot more cash coming off NVIDIA every quarter than TSMC.

40:08I don't know that, but you take my point. Like there's a lot of the value is going to NVIDIA and the chip makers rather than chip designers rather than TSMC. And I think if I'm TSMC, it's like – to me, it's like an investor who spent his whole career investing in auto, the auto industry, and now is being asked to make – for his personal account is investing in internet stocks. Smart person, knows how to invest, but doesn't really understand the sector. um and i think that's that's what tsmc is is is you know i i i don't think they're gonna get too involved they're pretty conservative too like they don't even invest they don't have like like a venture arm like to me that would make a ton of sense that's where they should be investing is like investing in young promising companies right because that's going to feed their demand long term.

41:08Yeah. Okay. All right. So I'm getting, I'm getting somewhere.

41:10Ben Bajarin: We're going down a path. I promise. Okay. Do you remember a long time ago we talked about, uh, this was before I'm in, dude, this was like year one of the circuit. We talked about Broadcom and how Hawk operates this company. And do you, do you remember how you described that he runs Broadcom? Yeah, he runs it like a private equity fund. Okay, so if you know that, who then do you think has the best risk-reward analysis here for these investments?

41:50Ben Bajarin: Broadcom. Okay, so then at least in that scenario, why would not someone like TSMC then just share some of that risk? It's going to get made there anyway. Because that entails a high degree of trust in whoever that intermediary is. I'm not saying that Hocktan is not trustworthy, but it's a third party risk. It's a third party risk there. And TSMC doesn't understand the end markets. They can't. They're far away from it. But what they do understand is the risks involved in their own supply chain, in their own production. What they do every day is incredibly, incredibly risky. And they understand that risk and they know how to manage it.

42:42They don't know how to manage a large investment portfolio or a large strategic venture portfolio. And so they're going to have to trust somebody else. Like, I think if I were a TSMC shareholder, I'd be like, why are you giving all this money to Hawk or whoever? Sure.

43:03Ben Bajarin: When you should be – just build another fab. Which they will, and they are. They're going to increase CapEx. Like, I'm certain of this. Okay. All right. Last question on this. Then does Broadcom – they do this with Anthropic, and let's say they do this with more companies, and they start to use balance sheet as a service more aggressively the same way that NVIDIA does. Does that, knowing what we know about how Hawk does these, does that give us any more confidence in the long-term structural-ness of this build-out?

43:40Maybe you're asking the wrong person for structural confidence in the AI industry.

43:46Ben Bajarin: But I'm just saying we know his mindset. We know how he's very good at making money running Broadcom, like you said, like a PE firm. And so for them to be willing to do this and perhaps be more aggressive, I take as an additional vote of confidence to the structural degree of this build out.

44:10I don't think anybody knows where this is going. right i don't think anyone knows where this is going like yes yes ai is going to be very important and it's going to do all kinds of incredible things someday but how the economics of that actually ends up is still super unclear and i guess i guess you could say the risk is you spend all this money you're going to invest in anthropic or you're going to invest in open ai is it only going to be those two are there other other labs that might come out of nowhere that will capture the value here, I think that's still possible, right? Maybe it's a new lab, maybe it's a new business model, maybe it's a new product.

44:46Like it's also new that there's a lot of risk involved, right? Yeah. There's a lot of uncertainty.

44:55Ben Bajarin: Agreed. I would just, my main point was, I think it's significant that Hawk Tan has entered the equation with balance sheet as a service. And I think that's a sign of more to come. and now that is a dynamic. That is the competitive dynamic of the day. So I look at that slightly differently. I think it's very much NVIDIA is driving the train, driving the bus, I guess. And they have been setting the competitive terms for AI data centers, silicon for several years now. And they keep pushing and pushing and pushing, right? They added networking. They added server design. They added data center design.

45:42They have all kinds of software. And you see AMD is following suit. They've greatly ramped up their software efforts. They acquired ZT systems to do server design. The networking piece is glaringly absent from AMD. But they're following suit everywhere they can. And the same thing for Broadcom. They have the networking to begin with. they're they they don't they sort of don't do server design for a reason but like um but they're using balance sheet like nvidia is sort of forcing everybody to do more and more and more in order to compete with nvidia right yeah and that's how i see this right and um and and broadcom can afford it they have i don't know how much cash they have in their balance they have a lot of cash they have a lot of debt too though because they've done all those acquisitions but they're generating a lot of cash.

46:32But their balance sheet is less room than NVIDIA's. AMD has a much smaller balance sheet, much smaller cash flow stream. So we know that they're providing backstops in a number of data centers, but I think it's getting increasingly difficult for them to compete because they don't have a balance sheet, right? They have, I think, 10 billion in cash and a tiny little bit of debt. NVIDIA has 60 billion in cash and what their debt is, is a complicated discussion now, but not a lot of formal debt right now.

47:06Ben Bajarin: But NVIDIA is growing. They're generating. Yeah, NVIDIA is growing. And I think free cash, I was just looking it up. Yeah, I mean, NVIDIA is generating 20 times as much cash per quarter than AMD is, right? So AMD cannot compete when it comes to balance sheet. even Broadcom is going to struggle to compete here against NVIDIA, but Broadcom does have the advantage that it could call on deep pulls of private equity capital to step in. Right. Okay. All of that to say, this is one of the most interesting competitive dynamics that has entered the equation, balance sheet as a service. um it's at some point might be worth like i don't know a whole episode on this and the strategic implications and beneficiaries but it is one of the newest dynamics that i think is extremely fascinating um so more on that but i think what we've established is it's not going away uh Perhaps more players may enter the equation or Broadcom will do more.

48:19But it is now a key part of, I think, how you analyze who's got longevity and the bar of competition and how it's changed. I think if we did an episode on data center finance, we would have to title it, Running into the Street Screaming with My Hair on Fire in Sheer Abject Terror.

48:43because right because i've been covering semis for a long time as have you and like if you if you told me five years ago the the center the central competitive dynamic in semiconductors was use of balance sheet and taking on debt i would have i wouldn't have believed you i would have because like for years like semiconductor companies didn't take on debt for good reason it's cyclical and debt is very very constrained and and it's a high fixed cost business. And it's a high fixed cost business, right? And if you miss a cycle in semis, if you miss a product and you miss the cycle, that's really bad.

49:19If you miss a cycle and you can't afford R &D because you have debt payments and so you can't invest in the next product cycle, that's terminal, right? And so this industry in general has been very, very cautious about taking on debt. And obviously today's semiconductor companies are very different than they were 10 years ago. They have a lot of cash and a lot of balance sheet, but still causes this old analyst to get a little nervous.

49:49Ben Bajarin: Yes. Noted. Understand. Okay. Well, as you said, I do think this is a worthy topic to dig into more. All right. Let's talk tokenomics. Yes. Poconomics. So one of the things that has frustrated me for a long time, and long-time listeners will know this, is I get really frustrated by the way that when we talk about AI, even seasoned industry professionals talk about AI and different features of software and silicon, everybody gets a little bit hand-wavy. right remember earlier in the year the hot thing was reasoning models everyone's talking oh reasoning models this is going to change everything did it like and i or even just like tokens everybody's talking about tokens all the time but but like what is a token like what is it worth and it just it frustrates me that it's also imprecise and sort of qualitative conversations and it occurs to me that if you look at all the data that's now available out there you can construct an analysis that lets you piece it all together and actually quantify a lot of these things, right?

51:09Because you can look at token prices and you can look at token prices for reasoning models, for example, and realize that like a year ago, the labs were charging a premium for reasoning models and that that premium has gone away. It's not a differentiated feature. It's sort of bundled into the frontier models. It's just a feature. It wasn't really worth it much. As opposed to if you look at input tokens versus output tokens, they have very, very different pricing structures because the nature of compute that underlies it is very different and very different cost structures. And you can sort of piece together the ways that that plays out into the end customers, the AI labs, economics.

51:50And then you can work backwards and you figure out, oh, this is why things like NVIDIA buying Grok for$20 billion make a certain kind of sense. And I think there is a way now to quantify the economics of AI. It's possible now. And it is something that I think everyone's going to be doing in a few months. It's an important form of analysis to really just be able to talk about these things in hard dollar terms. and not just say, oh, I'm going to have this feature in my model, and it's going to make everything faster and better. No, it's going to reduce your cost of decode by 20%, and that means you can price accordingly.

52:33Right? Well, and I think that highlights the OpenAI's Dev Day this week

52:43Ben Bajarin: where they announced, I think the brand's ultra-fast for Astra, the higher tier which is i think 250 tokens per second which then very quickly was like is this cerebris because i'm not sure because they showed a much higher token model yes it was on sol but um and then it came out afterwards that it is a low batch optimized version of nvidia gpus that are actually driving that um but two things one uh impressive that they could optimize nvidia gpus to get a higher tier without Grok or any specialized premium accelerator. I get that it's not 700 tokens per second, but it's still more than one. And that OpenAI can and probably will charge more for that.

53:27Ben Bajarin: So I would add to your tokenomics point that there's the base level that I agree with you with, and then the kind of greenfield model opportunity for what you would consider premium tokens. or premium pricing, and how big of a market that could be, which OpenAI is trying to justify, right, with a higher tier. Yes. Yeah. I'm not quite there yet in my analysis. I've put a lot of these numbers together. And I think there's a lot of interesting things that are popping out about it. I'm still not totally comfortable with the idea of premium tokens. I get where people are going with this, but I think there's a better way to think about it.

54:15Tokens will be priced segmented by, will be priced discriminated by segment and end user in some way. But I think it's going to be a little bit more nuanced than just saying premium tokens and cheap tokens. But it's very interesting to think through this because I imagine that Anthropic and OpenAI at this point have economists on staff and are like sort of thinking through pricing decisions. Or I don't know, maybe they're just throwing it into their own models, but like there's a lot of nuance there. And I think it'll become especially interesting when or if the Frontier Labs go public. So to really understand their economics, you can get a fairly good insight just using data that's out there today.

55:00Ben Bajarin: No, I agree. But I do think this is an interesting beginning of the fragmentation of the market where you might see pricing tiers that are both speed-based and or possibly specific model-based. like if models fragment and a good example I've used is like I think someone it's inevitable that someone makes a dedicated frontier cyber security model that's not something everybody needs but that's something that a lot of people who do need it will pay a lot of money for and so there's the and I would also argue it's a very different type of token so that is a price slash model so it's not just pay open AI, Anthropic, get X, Y, and Z.

55:48Ben Bajarin: It's which one do you need? Oh, I need cybersecurity or I want the best engineering model. And you start to see those choices with economics applied to them, I think is a, it's a very real possibility. Yeah, but that's a product. Like that's a cybersecurity product. I don't, I don't think people end up, I mean, I'm quibbling over wording here, right? That's a product. It's a cybersecurity product. People will pay for a cybersecurity product. I don't think we're going to get to a place where there's like 20 flavors of tokens. Here's a fast token. Here's a voice token. Here's a security token. I think that's fair.

56:22Ben Bajarin: Yeah. You pay for the model. The underlying implications could be one of those things, but I want it fast. Fine. You're not thinking premium token. You're just thinking, okay, I'm going to use a lot of these and I need it to happen quickly. That's valuable to me, so I'm willing to pay a little bit more. Yeah. I think that's the other side of this too. I'm pricing things appropriately. We now need, on the other side of it, I think not just investors and analysts, but like customers are really going to want to know what they're paying for, because there's going to be a lot of room here for, you know, if you're, say you're a large enterprise, you go to Anthropic and you're like, hey, I want the cybersecurity model.

56:59That's going to be a very complicated negotiation. And I think you're going to want to understand Anthropic's economics in such a way that you can negotiate better. Otherwise, Anthropic is going to say, here, we're going to give you the super fancy luxury VIP tokens and you're not going to know what that means.

57:17Ben Bajarin: Yeah, that makes sense. Because we know we can calculate now, like we know what their hardware is. We know roughly what the specs are and how many tokens per second it is and what that costs. And you can do a lot of interesting stuff once you have that data. Yeah, agreed. All right, well, we had some other things we're going to get to, but we're up on some time. So we'll save those nuggets perhaps for another one. And we've got some nice guests as well in the next couple of weeks. So stay tuned for that. And we will talk to you next week. Thanks for listening. Thank you, everybody. Leave us a rating.

57:52Tell your friends. Tell your agents.

58:09Thank you.