Memory Pricing Pressures, Meta's Muse AI, and the Agentic CPU Crunch

0:05Ben Bajarin: Hello, everyone.

0:06Jay Goldberg: Welcome to another episode of The Circuit. I am Ben Bajarin. Greetings, programs. I'm Jay Goldberg.

0:12Ben Bajarin: Well, we return to our regularly scheduled programming of just Ben and Jay after back-to-back weeks of guests, which we appreciate the feedback on all of that. But now it's us two cranking away on hot topics of the week. The two of us and the rooster. No, my rooster, like, I shut the window, and it's midday. They're not crowing so much. I appreciate the feedback. I know some people don't love. We have very loud roosters sometimes in the morning, and they're not far from my window. So be it. um all right let's start with uh with memory and in particularly the what are we going to say the fourth and fifth horseman of the memory apocalypse the chinese players uh who are uh obviously ramping capacity we see a lot of news come across the the chatter sphere the discords and the slacks that we're in around that.

1:16Ben Bajarin: So they're moving quick, the Chinese memory guys. I know. So you had a take.

1:25Jay Goldberg: Yeah, I mean, I saw a lot of headlines this week or a lot of chatter about both CXMT, which is DRAM and YMTC, which does NAND, are accelerating various parts of their program. It sounds like CXMT has pulled forward some of its capacity expansion.

1:43Ben Bajarin: Yeah.

1:44Jay Goldberg: No, sorry. YMTC has pulled forward some of its capacity expansion. CXMT is on track to add another 100 ,000 wafers per month by the end of next year. So that's a 33 % increase for them year-on-year in capacity.

2:04Jay Goldberg: and then CXMT also sounds like they're making a lot of progress in sort of just developing their the quality of their product I've heard I've heard different rumblings of what that means but it sounds like their their yields are improving and they're starting to branch out and different flavors of of DRAM so not not like the sky is falling China is coming kind of news but But just a periodic reminder that these are serious providers of memory and they are getting better and bigger pretty quickly. Yeah.

2:37Ben Bajarin: Yeah, and I think you're already hearing – I mean, look, at the end of the day, there's a lot of OEMs. And I know it's not just the Chinese OEMs who would love some release to the pricing pressure. I saw two different forecasts come out earlier in the week downgrading consumer devices in volume really across the board. I mean it was everything. It wasn't just PCs. It was phones and everything because of the memory price bug, which I think is a fair downgrade in consumer electronics. It's rough. It's rough out there. It's going to be ugly in consumer hardware for the foreseeable future.

3:21Jay Goldberg: I don't know. I'm starting to hear different rumblings. Like, I mean, this is mostly coming from interested parties, like memory company CEOs. But it sounds to me like there is some plateauing of consumer grade memory. Pricing, you mean? Yeah. Pricing, yeah.

3:44Jay Goldberg: And I've heard that from a couple people. I don't know how much I believe it. But like, I think the consumer side would really, really appreciate that now. Just a little bit of because and I think if the memory makers are smart about it, they'll realize they really have two very distinct customer groups. One customer group is very price sensitive consumers and the other side doesn't really care about price and will take all the memory they can get in the data center. and I think the memory guys will see a way to sort of balance that. I mean, it's not an uncommon problem, market segmentation. So I think there may be at least, I mean, I don't think memory is getting cheaper in consumer, but maybe prices stop going up again as opposed to data center pricing.

4:32Jay Goldberg: Memory is just going to keep going up.

4:34Ben Bajarin: Keep going up, yeah. Well, but okay, so here's a nugget that I had found out. So there are certain SKUs of memory size, you know, gigabytes that aren't normally in a SKU or at a base slash entry SKU that are cheaper than some memory that's less than that gigabytes. But the OEMs don't want to move up into that price because once you increase your storage or your memory, there's really no backtracking from that. Even if it's less expensive than a lower tier of memory. And so you kind of have this dynamic of future-proofing planning to make sure that you're not stuck at a certain tier that historically is more expensive than a lower storage or memory tier.

5:31Ben Bajarin: and that's creating some really interesting dynamics as to where pricing increases happening, what they do with SKUs, how those prices go into certain SKUs. So it's just different navigation, right? You've got to manage through. But I thought that was interesting because it's not necessarily all pockets of memory. It's pockets of memory that someone doesn't want to bring into a certain price point because that locks them in. You can't downgrade from that going forward.

6:07Jay Goldberg: Yeah. I forgot to do the comparison for the iPhone because the new iPhones, of course, came out. And if memory serves me correctly, memory tiers on those stayed the same as the last generation. Yeah, storage went up. Storage went up, okay. But I know that that dynamic is true in China where you're seeing a lot of phone specs sort of flat line memory and storage-wise over last year. Nobody increased their – some vendors actually decreased both in a lot of SKUs because you don't want to get stuck. I take your point.

6:47Ben Bajarin: Yeah. Yeah, which again is fine. It's just you got to navigate that landscape. But no doubt, right? Even if we see some pricing stabilize, which I hope you're right, it is a higher pill to swallow. And I think that's the concern in consumer electronics right now is that we're at an already inflated state. And entry-level prices have gone up to what you just consider at mid-tier. And everybody's having to just navigate how they set these prices. And fall is going to be really interesting. Like what happens with promos, what happens with things like PCs and smartphones and other consumer electronics in fall, I think is going to be really interesting to see where they land pricing, what promotions happen.

7:37Ben Bajarin: Because, yeah, it's going to be rough and then. Yeah. Yeah.

7:42Jay Goldberg: I think about this strategically. If the Samsung Hynix and Micron are smart about it, they're going to do what they can to protect their longstanding consumer customers because if they don't sort of look out for them to some degree, that is going to open the door for those customers to look elsewhere. And I think the fact that two or three weeks ago we heard a lot of reports that Apple had tested CXMT memory and was pushing very hard to get a waiver from the U.S. government to be able to buy it, I think that sort of caught a lot of attention at the other memory makers. Because if Korea and Idaho can't provide the memories, China can.

8:33Jay Goldberg: Wuhan can.

8:35Ben Bajarin: I mean, but at the end of the day, right, they should be concerned because once you're in the door with another alternate player, like Apple's not going to all of a sudden go, oh, we're going to stop using them. Like now they're another supplier. And that would hurt. That would hurt. Barn door is open.

8:50Jay Goldberg: Yep, exactly. And I think the other thing, too, is you think back, what was it, three weeks ago with Hot Chips. one of the sort of big takeaways i heard from a lot of people hot chips was the memory presenters got a lot of very tough they faced a pretty hostile crowd people were pretty angry at them and in particular they you know this is this is a technical presentation and the the technical like there were a lot of people pushing back saying why are you moving to hybrid bonded memory why are you going higher we we don't need that because that is just going to be more and more wafers that are going to be needed yeah um like let's let's try a different approach to memory it doesn't have to be so high and yeah so many stacks of hbm let's let's try a different a different path forward that was very very loud loud feedback i heard yeah so i think the memory the memory players have to you'd think they would just like sell everything they can in this great times and no i think they're going to be a little smarter a little bit more strategic about it

9:52Ben Bajarin: yeah and and i think you know we've talked about the memory pricing you know before and it's interesting when you you know talk to some folks in in the region um different regions you know you hear them be sensitive to this right knowing that margins are going up and again there's a different margin profile for all three of your big three. But you hear particularly from the Korean players, and this is actually true of a lot of folks deep in the semiconductor supply chain in Taiwan in particular, and even in Europe, where they don't want to press margins to look opportunistic. And trying to take a consumer-friendly view or a customer-friendly view in this, I think, will be appreciated by those companies long-term.

10:39Ben Bajarin: I mean, just use the example for Apple, right? I mean, if you think these, the memory players are up in arms or have a grudge against Apple, like you just wait till you see the grudge that Apple will hold if you continue to be hostile to them and they are your biggest scale customer. Like this is a non-negotiable to, you should, you should try to still be somewhat customer friendly. And I, this is just a fascinating, fascinating dynamic um it's anyway but yes you're right if uh as long as you know ymc and cxmt do continue to scale we we know they're going to bring relief to uh china-based oems who will hopefully have some pricing um because their prices are going up their their low-end tier to mid-tier is is getting rough um but uh but yeah for even for chinese oems they can't they

11:35Jay Goldberg: They can't ramp quick enough, so we'll see. Memory dynamics remain extremely, extremely pressed.

11:43Ben Bajarin: Front. Yeah, it's wild. Okay, let's talk meta. So I have a couple of broad opinions, and then we can talk about some of the implications of this. So everybody's already like, you know, Muse is the greatest thing ever. I don't know if you all played with instinct too, but essentially what you're seeing is a very interesting, let's just say maybe more consumer use cases. Although I don't want to say it's only consumer, but things like what would it look like to have a chat bot that can go do product research for you? Or what would it look like to do at ShopBot? They can help you find cheaper flights or manage this and X, Y, and Z.

12:36Ben Bajarin: And so people play around with these and they see that element of this, which an assistant is trying to, for lack of a better word, be an assistant in a central location. And people have been very favorable to this. Like Muse has sort of taken off. I get it. I've tried it. it has some great value. In fact, one of the things that it does the best, instinct fails at this, but Muse does this the best, is try to anticipate some things for you. Like, oh, hey, this just happened, or I see this on your calendar. You might be hungry around this time. Should I find you a restaurant? It syncs together what it knows to try to be helpful or anticipate.

13:24Ben Bajarin: And there's value there. I think that's, you're starting to see the early stages this. But I'm going to comment. So our friend Ben Thompson was on TBPN or whatever. And I think he was talking about an article that he had written. In fact, in my last interview with him, he made the same point, which is, you know, consumers don't want to be productive. And to some degree, consumers like shopping, which I agree. You know, he used the example of like, people go to Walmart to go to Walmart and look around and shop around. And like, for me, that's Costco. Like I enjoy Costco for some obscene reason, but I really like going to Costco.

14:02Ben Bajarin: But those are the things like those aren't going to be automated away. And I do think there's some areas for consumers who will engage in these things and be okay with something helping them again, maybe flights or planning a vacation or things that aren't enjoyable or tedious. So I don't think it's a blanket statement that consumers don't want help or help shopping or whatnot because they want to just go shop themselves. They want to do that for things they care about. They might not do that for things they don't care about. But I also think that these little agents or muse is actually also showing you what a personal enterprise assistant could look like.

14:41Ben Bajarin: And, and that's where I think things get super interesting because not all employees have had a true kind of connected assistant who can help them be more deeply productive, do things like, manage things they don't want to manage, et cetera. And that's, we haven't really had that before. You wouldn't say that you're engaging with ChatGPT or Claude to do some product stuffy stuff as a true kind of quote unquote assistant, the way that you're seeing Muse. But I think that's interesting, right? Just this degree of, you know, could every employee also have their dedicated work assistant that does some of these things.

15:20Ben Bajarin: And then again, the implication of all of this, of course, is that it just creates more compute demand, right? Muse is doing kind of its own computer, even tells you that it says, I have my own computer, I can browse the web, I can do all of these things. And that's running in a VM, and it requires a whole host of compute, right? CPUs and GPUs. But that added dynamic of I'm not just using, you know, a research service, OpenAI Anthropic, whatever, to kind of use documents and I am the orchestrator of those things, but now something that could be another part of that and an assistant, a smart assistant of that from an enterprise standpoint, like we, I don't think we'd really had anything like that before.

16:01Ben Bajarin: So I just think it's, I just think that whole 10 ,000 foot view is, is interesting.

16:08Jay Goldberg: Yeah. So maybe backing up a second, Facebook had its annual conference this week and they announced a bunch of AI related products, the most interesting of which is, well, the most interesting of which is a little Tamaguchi AI toy that you have to keep alive. The second most interesting is their AI agent, Muse. And first, first of all, I want to say, I called this two months ago. I said, this is what the product looks like. This is what the AI consumer product is going to look like. It's an AI assistant that does all these things. and I mentioned that not just to brag although partly to brag but also because what I've been saying for a while is the the hard part now for AI is the product side yeah you have all this great technology how do you make it into a product that users engage with and want you know want to use more and I think Muse is is a very like I think OpenClaw was the first one that you looked at it you're like oh this is where this is headed and now you see muse it's a much more refined usable product that you know anybody can sign up for i i still think it's a it's only a step in the right direction i don't think we're there yet i don't think the ai models can really really do it yet yeah um but i think there is immense consumer and to your point you enterprise utility but i really think there's a strong consumer product in in the works some somebody's going to come blow up with something it's probably a couple generations of models away but where it's actually really really useful to the consumer um and it will do it'll do these things it will there's a lot more can do and i mean i think your point and is and ben thompson's point is is valid but that's to me uh just in the category of product definition right yeah figuring out what what users actually want.

18:00Jay Goldberg: And I also, I also think like you, you and I aren't necessarily the target demographic here because I started using instinct. And the second it started asking me for my, all my passwords, I was like, who is this? Like, I still have privacy concerns, but apparently lots of people out there don't. I do not. I am happy to just give my life.

18:18Ben Bajarin: it's like, I guess I was telling there, you know, there's rumors that, you know, other, other people, you know, we'll do these, these assistants too. And I was like, I'm ready. I'm ready to give it all my entire life. But I, yeah. But okay, but I want to make this point because I think this is interesting. For me, like what Muse did, I think Instinct tried this, but to be honest with you, like I just don't feel like it landed. Muse was the first one that kind of felt like once you let it to connect to some things, it started saying, I can go do these things for you. and to me this is actually a critical part of the onboarding so so what i'll back up and say is remember remember when you know ai was kind of early and i had already like gone fully down the path and then you were saying like i sit there and i'm not quite sure what to do with it right i think this little thing this little agent becomes the thing that helps you figure out what to go do with it and i say that for two other reasons there there is from our research both with checks we've had with people using enterprise AI, there is a gigantic skill gap between people who get it, who are deep in the weeds, who are like automating everything.

19:34Ben Bajarin: And then others who stare at a prompt and go, I have no idea what to do, how to use this to get the most out of it. And my inkling is that this little thing, whether it's Muse, whether it's co-pilot, whether it's Gemini, becomes this, I don't know, anthropomorphic concept that starts to guide you and prompt you and start to suggest, you know, I can go do this for you. I can make that Word document for you. Watches you and says, oh, I see you're struggling with this. Let me help you. I think that might help this concept of this agent. It might help bridge that skill gap for those who aren't right deep in the weeds here and pushing the envelope with every model because they they just want to test its limits that's not every person out there to get the most of it so to your onboarding point to your your ui point which is exactly right right you've got to productize this i think the agent this little assistant thing is the thing that might be the catalyst for adoption and uh onboarding slash how do i use this to go do the wonderful things that it can do for those that don't aren't immediately aware of that agreed but i think

20:52Jay Goldberg: i think it's not quite there yet like muses no it's definitely not but that but it is the thing the thing it is it is the thing for me it was open claw that i was like oh and opus 5 3.5 and open claw we're like oh this is this is what the consumer product could look like i think this is going to spark a lot of people's imaginations but i also think uh there will absolutely be similar versions from everybody else and sooner or later somebody will get it right maybe it'll be maybe meta but it's it's it's it's it's a step in the right direction Yeah.

21:25Ben Bajarin: I think Apple makes a play here sooner than later. I think Google does this too, right? And the thing I keep walking back from is if you're just going to pick who would you think is best advantaged, and I hate to harp on Facebook, but I do think there's people who will hesitate to give all of their secure information, even if they're okay with it, right, to your point, to Meta. I do think there's others who have higher degrees of trust in their life who they would be more willing to engage with some of these things are Apple being one of them, right? One of the areas where security and privacy is going to play off really well for them is in a Siri agent like Muse.

22:08Ben Bajarin: But I think Google's there too. I don't think Google's out of that list and Amazon might make a go also. But there's only a handful of companies who have what I would just say the right and the right relationship plus the distribution to go and succeed at this route for consumers. I agree with most of that, but I do think there is room for some third-party startup to come along and build their brand on privacy and also on some form of independence.

22:40Jay Goldberg: like I've maintained for a while that part of the agent's role will be an orchestrator across different AI models so you don't have to pick which model you're using it will optimize for that agree meta shows me that that may not be so important because the model is free right but but it's also not quite clear how how capable it is I think if you really task muse with something very very complicated i don't know i don't know what the limit would be i imagine there is one but i haven't heard it yet like if i want to have muse go build a giant research report for me it's pretty simple it can't do that it can't do that right i mean it might i haven't

23:18Ben Bajarin: tried it but i don't think it can do that or do it well it's not it's that's not its goal

23:22Jay Goldberg: its goal is different yeah but i think the ultimate consumer product will possibly be from a third party who and i think the way that they can monetize is to you you you pay them and they they basically mark up some of the someone else's model like frontier model but then they optimize around it so you don't have to bet you don't end up coming up with a big bill they you know they use free open source when they can and they use the frontier when they have to of course um somebody if anybody's working on this call me because i've got i've got the product in my head like i just give you your monetization so like it's it's so i just my point is just that there's there's room for a third party here i think i think a hot startup a smart team small smart product people could could become very powerful yeah well i mean we'll see because i think two

24:16Ben Bajarin: very big lab quote-unquote startups are going to try this also everybody's gonna try we'll see

24:22Jay Goldberg: Yeah.

24:23Ben Bajarin: And we'll see. Um, okay. Anyway, so let's, let's talk about Anthropic plus Akamai, which was an interesting deal. And then what this means for compute demand, uh, and tie it all together. So Anthropic and Akamai.

24:39Jay Goldberg: So Anthropic and Akamai announced a partnership together. I forget, it's like an$11 billion deal. And Anthropic is going to pay Akamai over some period of years$11 billion. And in exchange, Akamai is giving them warrants. So typical deal we see nowadays in equity exchange. and for those of you who don't remember akamai or forgotten about them they are a cdn they're an edge network provider they have servers all over the world that sort of accelerate local services like the cash movies that are always going to be watched in a certain neighborhood or whatever um but they also have a cloud computing business they bought linode a few years ago and they have not released many details about what exactly Akamai is going to do for Anthropic.

25:36Jay Goldberg: And so I've been digging into it all morning and there's a couple ways it could go as far as I can tell. One is Akamai is going to run inference for Anthropic as a pure compute deal. So Anthropic needs CPUs in particular to do inference. They probably haven't been getting a lot of those in the deals they've signed. They really need those now to do more inference. And Linode and Akamai has a big pool of probably underutilized CPUs sitting around. I also think there's another angle here, which is around the network topology. The way that we access agents today can be more distributed. Right now, you know, you put in a request to Claude, it probably goes to an AWS server in Seattle or Utah or wherever.

26:32Jay Goldberg: But having something more local to offload some of that and also provide some of the security for agents, I think is really important. Which kind of leads me to this theory that like Cloudflare has a big role to play here too. I think they're the next one who has a comparable offering. But they don't have a big pool of cloud CPUs. So there's a lot of ways this can go. And it was a pretty interesting deal. Because when was the last time anyone talked about Akamai?

27:03Ben Bajarin: I mean, I think they either used to host Netflix or they still host Netflix. I can't remember, but that was a long time.

27:11Jay Goldberg: But then Netflix built its own CDN. It did its own. That's right. Yeah.

27:16Ben Bajarin: Hey, maybe that's Netflix's next business. If entertainment slows down, let's start renting out compute. All right, but all of this just again continues to emphasize how short we are compute. And there's been a resurgence of the CPU demand conversation because of Muse, right? Because yes, while it is its own computer and running in VMs, that still takes a ton of CPUs. And so I think it's interesting where – and I'm working on a report for this next week so we can talk about this more in depth. I'm going to call it the agentic CPU guidebook is going to be – but everybody is taking a very different approach to their architectures.

28:05Ben Bajarin: I think there's going to be some vendors with architectures that play better with GPUs than others, which will be very interesting to play out or accelerators, XPUs. But it is true that the CPU here, like we need a lot more CPUs because they, again, they're not just part of orchestration. It's a part of the actual token workflow from reasoning to action to review back to reasoning. like it's a it's a very close loop um but we haven't we haven't yet seen muse come up against this although i've heard they're ramping pretty quickly and they're starting to get up this but what's interesting about this to me with face with meta is meta has a lot of cpus like they may very well have more not more but a lot more cpus than others out there and so it's it's interesting that they can leverage that in a way that maybe others won't.

29:05Ben Bajarin: Maybe others might run more GPU or XPU agentic workloads here for these personal assistant agents. But the bottom line is everybody you talk to in the space is continuing to pound the table on how much more compute a personal agent running its own personal computer, having the ability to go work around the clock for you in the background of doing things is going to press on our infrastructure and i think that's true yeah i there was there was even before

29:41Jay Goldberg: the meta stuff came out there was a big debate on twitter last week about cpu demand going through the roof and there were people on twitter you i think you commented some of them were saying the ratio of cp it's going to be a 30 to 1 ratio of cpus to gpus and it was one of those things where I was like, wait a second, a year ago, the same guy was saying it's going to be 16 GPUs to one CPU, and now we're 30 to one in the other direction. I was like, oh, everybody needs to take a breath. Just like, just chill. But the world's going to need a lot of CPUs. Yeah, right. It's very clear. And it's, I think it's, I think it's surprising everybody.

30:19Jay Goldberg: I don't think anybody saw this coming six months ago, which is kind of weird. Yeah, maybe open AI did. They don't seem to be struggling for compute but maybe that's the next deal who knows yeah well i mean again right

30:30Ben Bajarin: everybody needs to to to get to to get more cpus we just unfortunately cannot make as many cpus or cpus for that matter um but any anyway it's uh i you know the pace of this is is so fast like i made this joke the other day i don't know if you feel like like i i was i was not in well i was aware of the industry participating in the industry but not working in the industry um when you know there was sort of the first software evolution you know when stuff that you couldn't do digitally took you a lot of times manually you could all of a sudden do and sometimes i feel like we're similarly in this environment where like that's that's what the excitement level was when you saw something and you're like holy cow like excel right i can't believe I can do this.

31:21Ben Bajarin: Like, I feel like we're in one of these moments, something new comes out and you're like, I'm gonna try this. It worked. Are you kidding me?

31:28Jay Goldberg: Yeah. Yeah. Yeah. Fascinating. Fascinating times.

31:33Ben Bajarin: All right. Tighter episode for you all this week, but, uh, we appreciate your time and, uh, we will talk to you next week.

31:40Jay Goldberg: Thank you for listening, everybody.

31:50Thank you.

32:20Thank you.

32:50Thank you.

33:20Thank you.

33:50Thank you.

34:20Thank you.

34:50Thank you.

35:20Thank you.

35:50Thank you.

36:20Thank you.

36:50Thank you.

37:20Thank you.

37:50Thank you.

38:20Thank you.

38:50Thank you.

39:20Thank you.

39:50Thank you.

40:20Thank you.

40:50Thank you.

41:20Thank you.

41:50Thank you.

42:20Thank you.

42:50Thank you.

43:20Thank you.

43:50Thank you.

44:20Thank you.

44:50Thank you.

45:20Thank you.

45:50Thank you.

46:20Thank you.

46:50Thank you.

47:20Thank you.

47:50Thank you.

48:20Thank you.

48:50Thank you.

49:20Thank you.

49:50Thank you.

50:20Thank you.

50:50Thank you.