Google’s AI Shakeup, Nvidia Weighs Putting Less Memory in Rubin Chips, Meta’s New AI Coding Tools

0:13Welcome everyone to the Informations TI TV. My name is Stephanie Palazzolo and it's Thursday, August 6th. Today on the show, the information has exclusive reporting on NVIDIA's proposed solution to the memory crunch, specifically to incorporate less memory into its new Rubin chips. We'll then take a look into how Meta's new coding tools aim to compete with Anthropics' Cloud Code and OpenAI's Codex. We'll also unpack the AI leadership shakeup at Google and what it could mean for the future of the company. And to close out the show, we'll talk with a venture capitalist investing in the new startup founded by Google's former chief scientist, Jeff Dean.

0:51It's going to be a great show, so let's get right on into it. NVIDIA is weighing a radical step to deal with a shortage of advanced high bandwidth memory chips. Our colleagues Phoebe Liu and Chenner Liu broke the story. Phoebe is here to join us now. Welcome to the show. Thanks for having me. So Phoebe, how exactly is NVIDIA dealing with the memory crunch? Yeah, so our reporting found that NVIDIA is testing versions of its next generation GPU, so Ruben Ultra, with less memory than it initially planned. So I think in early 2025, Jensen Huang said that Ruben Ultra would have one terabyte of memory per chip, which is a lot, like a lot more than any chip it's ever produced, any chip anyone has ever produced.

1:40the memory crunch has gotten pretty intense since then and partially because of supply chain pressures like nvidia is buying so much advanced high bandwidth memory that it's just hard for the suppliers to make enough of it partially because of that partially potentially because customers have just as much use for chips with a little bit less memory nvidia is testing versions with 256 gigabytes of memory and even 192 with some customers, although it's important to know that nothing has been finalized yet. They're just testing for now. And it's kind of, you know, kind of funny, as you mentioned, that one of the big reasons why we're in this situation to begin with is because over the last year, you know, NVIDIA's GPUs have become so popular that in some ways it's kind of led to this memory crunch.

2:33So it's a bit of a catch-22, it seems. Yeah, I think so. And I think NVIDIA actually has a team that kind of looks at these supply chain and all the way down to the raw materials issues years ahead, working with kind of projections the company has for how much of each material it will need because it is one of the world's largest companies and needs so much of this that it sometimes literally wants more than the entire world can produce. So this is something they think about quite a bit. But even with all the planning they can do, sometimes they can't control every single step of the process, even though I'm sure they would like to.

3:14And how does having less memory actually affect this new chip's performance then? Yeah. So I think because today's most advanced AI models are so big, they have trillions of parameters. That takes a lot of memory to both train and run. So if one specific chip has less high bandwidth memory, to run a model with the same level of performance, if all other factors were held equal, you would need more chips, which honestly is kind of good for NVIDIA because they could also in turn sell more GPUs. But I'm sure customers aren't thrilled about that idea, right? It kind of feels like, I don't know, do they feel like the rug has kind of been pulled out from under them a little bit and they're like, oh, now I need to buy so many more chips than I originally thought.

4:02Yeah, so obviously I haven't talked to every customer, but the ones that I did talk to honestly don't think it will affect demand. They're going to want the chips anyway. And hopefully if all things shake out as I'm expecting it to, if chips have less memory, they will also be cheaper. I don't know if that will be kind of a one-to-one cost decrease, but that would be good for customers, especially like the big clouds that are getting more and more worried about how much they're spending on NVIDIA GPUs and AI in general. So it sounds like even if they have to buy more chips, hopefully they can get a bit of a discount on them.

4:44I think so. Although also important to know that NVIDIA hasn't set pricing for Minolta yet. And I'm also curious, too, you know, having this situation, as you said, you know, memory is obviously very important to run and train these giant models. Do you think this is going to push customers at all towards maybe non-NVIDIA chips? Or, you know, obviously we have TPUs, we have chips from the big clouds, we have all these, you know, dozens of startups that are trying to build chips as well. Is this enough to push customers towards them at all, or is it not really that big of a deal? I think it might not be that big of a deal.

5:21I think it depends on what NVIDIA ends up landing on and kind of if it does decide to go with pretty heavily downgraded or less memory with Rubin Ultra chips, what other optimizations they would make to kind of compensate for those chips performance. So they can, yeah, there's a lot of other things they can do, like moving the data faster between different parts of the chip and the data center and also kind of storing the data more efficiently. There are lots of other things they can do. So I think it just depends on kind of both the overall performance and the overall cost of the chip. Totally.

6:01And I know you mentioned, obviously, it sounds like there are still moving pieces here. They haven't necessarily, you know, finalized or obviously come out with pricing yet. I mean, is there any way for NVIDIA to try to get more memory between now and when they make these chips more widely available or any other things that they're considering to kind of get around this memory crunch at all? Yeah, I think they're doing literally everything in their power to make that happen. So I want to say either last week or the week before, they announced an expanded$500 billion partnership with the SK Group.

6:35So that's the parent of one of the biggest high bandwidth memory makers. in the world to work on developing this next generation high bandwidth memory together. It's kind of unclear exactly how the 500 billion will shake out, but it's basically SK Hynix, the memory maker, is going to increase its capacity for making this advanced memory pretty significantly with additional investment. And their engineers are working with NVIDIA's engineers to kind of co-design one of NVIDIA's favorite words, that next generation memory together. So yeah, definitely things could change. And also it's possible that NVIDIA starts with versions that have a little bit less memory.

7:22And then as they continue to produce more Ruben Ultra chips, I think that's set to start late next year. They could create lots of different versions with different memory specs, depending on what customers want and depending on kind of what's available in the supply chain. So it sounds like even if they're in this position now, they're doing everything that they can to avoid this happening in the future. Great. Okay, well, thanks so much, Phoebe. We'll definitely keep in touch on everything that's going on with these new NVIDIA chips. But again, that's Phoebe Liu, our NVIDIA reporter here at The Information.

8:00Meta unveiled its new AI coding agent, MuseCode, on Wednesday, alongside its latest AI model, MuseSpark 1.2, that powers it. Our next guest had early access. Joining me now is Max Weinbach, an analyst at Creative Strategies. Max, welcome to the show. It's great to have you. Thank you for having me. So what did you make of MuseCode? It is an interesting new coding harness. Meta's really trying to get into this coding game. The coding agents are, I guess, the big driver of interest money, and they do generalize into just general workplace as well. But it's a good coding harness powered by MuseSpark 1.2.

8:42It reminds me a lot of Grok Build, a lot of the not Frontier Frontier, but a step behind, maybe a generation or two model-wise from the big labs. It's a good coding harness, good tool. Mm-hmm. So it sounds like positive results, but maybe you're not blown out of the water, per se? I would say it reminded me of something closer to a GPT 5.2. It needs a lot of hand-holding. The results can be good. They can be very good. It's not great at design. That really seems to be something that anthropic models are great at. And it needs a lot of hand-holding, a lot of detail and direction. One of the major things of the newer models, the Cloud5 family, as well as GPT 5.6, is it understands more or less what you're trying to tell it to do and just knows you can give it a very brief prompt and it will go.

9:34This one needs a lot of detail. You need to really try to hold its hand to push it through what it needs to do. So it does feel a generation or two behind, even though it's very capable. So it sounds like maybe some places where it falls a little short is this ability to give it a really brief prompt and have it run from there. And also you mentioned design as well. Are there any other areas where you felt like maybe it fell short and where did it maybe stand out and really impress you? It's really, to fall short, it does have this tendency, especially for software development, of not wanting to work with real data.

10:08It loves to create its own mock data. So it makes trying to actually use it in an actual application very difficult because it's trying to fake the data rather than using real data. So if you're trying to make user accounts, it won't do it well. And that's kind of, again, a generation or two old model thing. Most of the new ones don't do it. Where it is just really good is it's a good workhorse in the same way DeepSea GV4 Flashes, GPT 5.6 Luna, where if you give it a small detailed task, like going to update some stuff for you, it will do a great job at it. It can do these little things and you can use a ton of subagents for it, which are great.

10:44It's very fast. It's about 180 tokens per second. and it's depending on the version you use really cheap. Okay, I'm sorry. I have to double click on that. What's up with the fake data? In what situations is it even making fake data? And like what, I don't know, is there something in the training data? Like obviously we're speculating, but what could cause that? So it's not, it's a normal practice. If you don't have a full database set up doing whatever, it likes new mock data so it can create its UI, make sure all the logic works and do all of this. That's a normal thing. But a lot of the newer models, again, the GPTs and the clods, and even some of the other ones in Cursors Harness does a great job of kind of pulling this behavior out of it, it will start to actually try to separate it.

11:27So it's trying to mimic the way you would actually want to do something in practice, rather than just building it to make sure it works at first. And the way it will do it is usually putting it off to a different file. So it's not monneying up code to do it. Vue Spark doesn't do that. It kind takes this again i would say it but it reminded me of stuff we saw about nine months ago in the models and try to do it but you're taking this weird behavior from like nine months ago mixing it with a very capable model today and it's just kind of a weird amalgamation of different behaviors we've seen over many different models over the past year or so it's good and you know capable if you put it in the right direction just kind of feels different and hard to uh hard to like it after using the significantly better models from the other frontier labs.

12:13So in terms of the different coding agents, obviously we have, you know, Cloud Code, Codex, Cursor, NowMuse Code. I mean, I know you have only recently kind of started testing the new meta products, but do you have a sense of, you know, which one of those you're reaching for, for which use cases? I kind of use them all interchangeably. I view the Anthropic models are good at large, just a very large thing. If you're trying to do something over a lot of files, the cloud models are very good at making workflows and having sub agents to go over everything. If I'm starting for scratch or need something super complicated and I just want a model, I can give it a task and trust it'll figure it out.

12:51That's kind of codex in chat GPT. Cursor has both cloud and codex, but its harness feels more directed towards the way I actually think real engineers do work. So it's kind of more directional with these features in there versus the others, which Codex is really good, but it's also for anything. And that makes it kind of funky where you don't necessarily trust it as much to the larger factors you would with Claude. So I use all of them, and I feel like they all have a different place. And the Muse code kind of fills, I would say, something closer to like in between Claude code and Codex. but it's not necessarily as capable of a model as the two of them.

13:37So maybe I would probably just use Muse Spark 1.2 in Cursor or one of the other coding harnesses rather than using Muse code itself. Sure. And so obviously, you know, performance is kind of one side of the equation. The other side is, you know, cost, speed, some of these that you've already talked about. It looks like from the pricing they've released that Muse Spark 1.2 is cheaper than most Anthropik and OpenAI models other than kind of their most lightweight options. So that's, you know, GPT 5.6 Luna and Haiku 4.5. How did you feel about the new meta model cost-wise? I didn't actually see the cost until yesterday, which was interesting.

14:17But I think the contributor tier, which is they give you a 95 % discount if you share your data with them, is a really good deal. A lot of the things that I'm doing day to day, I don't necessarily care if meta gets the training data for it. It's, you know, internal proprietary, but if that can then help me in the future using these models get better performance out of it, I don't mind. And I'll take the 95 % discount on it. On the paid tier, I think I would still probably go for one of the higher end frontier models just because it gets into that weird point where, yes, it's capable at$4.20 cents per million output tokens i think it's like a dollar per million in um and yes it's capable it's good but i can also use grok builds and or uh cursor where i'm getting very subsidized tokens on that and i'm not actually paying the token rate for it so you're kind of in this weird position where yes i can do pay for it as an api i can get a massive discount if i don't really care or i can just go to one of the other ones where it's hugely subsidized and I'm getting$10 ,000 worth of usage for$200 a month.

15:28I'm curious if other model providers were willing to offer that same kind of business model where they give you a huge discount if you're willing to share data. Do you think that would be attractive to users? Or do you think at this point they're already used to not sharing their data and it might rub them the wrong way, perhaps? I think the way Meta actually did is a great way. It's two different model slugs. You can choose very specifically. Normal, normal price, we don't take it. Or you can take a discount for it. OpenAI, actually, I believe they still do this. They give you a certain amount of tokens per day or week if you're willing to share data for free.

16:04They've been doing this, I think, for six to nine months now, or maybe almost a year at this point. Google gives you a generous free tier as well doing it. A lot of them have done it, but instead of just doing the 95 % discount for as many tokens as you're willing to buy, It will give you a certain amount of free per day, week, month. And with this, you agree to let us train on your data. I think that's two ways of doing the same thing. I don't mind it. I think it's a great way to do it. I mean, AI is expensive. Everyone's saying knowledge that's too cheap to even matter. It's, yeah, intelligence is there.

16:40If you can get the discount on it, and people obviously seem really excited by this new pricing, especially with DeepSeek saying they're going to increase rates as well. Sure, that's a much better way of doing it than just saying you get a limited amount of free tokens per day. And, you know, taking a step back, looking at all these different coding agents, including Meta's new ones, we're obviously seeing this shift in the last, you know, 6 to 12 months from coding assistants into agents that can work, you know, independently, sometimes for hours at a time. I mean, I'm curious just in your experience, what are some things that you're able to do today with coding agents that maybe you weren't able to do, you know, three, six, you know, nine months ago?

17:20I would say doing it would be, you're able to do, if you understand how software works, the model architect, just architecture of software or the models, you can do essentially anything. I mean, obviously there's limitations to it in terms of speed, time, reliability. But I don't think nine months ago, I would have been able to do half of what I'm doing now. And a lot of that isn't just because the models are better, the agents are better, whatever. It would take so long. Originally, it would be basically one file at a time. If you wanted to do something over multiple files, it would take 10 minutes to edit three files because you were going to one of the OpenAI 01 or 03 models.

17:58It wasn't able to use the tools. While you were capable of doing it, the models, the agents, the way they all worked didn't come together really well. Now you can do sweeping changes, rewrite things in completely different languages, make it 100 times more efficient kind of overnight. So you're getting into this area of just highly efficient software, which maybe nine months ago we had software. Now we're getting into good, highly efficient software, assuming you know how to use everything. And maybe another nine months from now, we're just getting into, you can give it a quick idea of what you want and get anything you could ever imagine built for you in a couple hours, minutes.

18:40Sounds like a great future to me. I guess we'll have to check back in in six, nine months, see what things look like then. Well, again, thank you so much, Max. And that was Max Weinbach from Creative Strategies. Google announced major changes to its AI operations on Wednesday. Google DeepMind CEO Demis Asabas is stepping aside to become chair of the lab and chief scientist at Alphabet. Taking his place will be Karai Kavachoulu, current CTO at DeepMind. He'll now lead the lab as senior vice president. Some interpreted the shakeup as bad news for Google and the stock closed in the red following the news.

19:16But the information's co-executive editor, Martin Peers, wrote his nightly briefing arguing that the moves could say more about future succession plans at Alphabet. Martin joins me now. Martin, welcome to the show. Steph, how are you? I'm doing great. I have to congratulate you on your pronunciation of Corey's last name because I have no idea how to pronounce it. No, you have no idea how many times I was practicing that before the show today. So what do you think exactly, what do you think this executive shakeup could say about, you know, future company leadership for Google and Alphabet more broadly?

19:54Well, there had been a view that whenever the CEO, the current CEO, Punda Pashai, steps down, that he could be succeeded either by Google Cloud's chief, Thomas Kurian, or by Demis. But I think it's pretty clear that Demis is not interested in actually running Alphabet. he really wants to go off and continue what he has done in the past, which is a very acclaimed scientist. And I think that's where his interests and passions really lie. So I don't think he is in the run to take over. And whenever this issue comes up, which probably won't be for a while, the Alphabet Boy will have to think of other people.

20:53And I think maybe you can see that even in yesterday's moves, right? Demis kind of taking a step back and Karai really stepping forward. And, you know, as we've reported over the last year, Karai has really been actually the person more involved maybe in day-to-day operations. I mean, do you think that's like, is that a fair read of the situation? And do you feel like people there are not really that surprised then? Yes, I think that's absolutely the case. I mean, I think what we had reported last year is that Cori was really the one running everything and that Demas was much more focused on sort of the big picture.

21:31I mean, while he appears a lot in public, I don't think he is that—he was really that involved in actually running the place today. So I think all that has happened is that they have made a reality – oh, sorry, they've sort of publicly made clear what was actually the reality. So I think the coverage in other outlets was a little overdone, really. Particularly, you know, some news outlets saying it's a huge deal. I don't think it was a huge deal at all. I mean, I think just to take the other side of the argument, though, I think, you know, a lot of the times to inspire AI researchers, they need to feel that they have this sort of like lighthouse type person that they can, you know, that's kind of guiding them and has this philosophy for where AI is going.

22:22I mean, do you feel like Demis can still offer that in his new position? Oh, absolutely. Yeah, absolutely. I think that, in fact, that's what they were saying is that he will continue to do that. So I think Google will get the benefit of his vision and the fact that he will play a role in talking about where AGI is headed, for instance. He's just not going to be responsible for the things that he was not already doing anyway. So, yeah, for sure. Yeah. So you mentioned whenever you look at, you know, people that could succeed, the current CEO, it seems like Thomas Kurian, who obviously leads Google Cloud, is, you know, one of the kind of front runners there.

23:05Are there any other people that have come up in our reporting or in discussions that could, you know, be potential future CEOs for Alphabet? You know, I've heard more about the names of people who I'm told are not in the running. I think it's possible. I mean, I did wonder whether Corey could be a potential CEO I mean, somebody told me yesterday, well, maybe at some point in the future. But I don't really know. I don't want to put forward names of people that I'm told are not in the running. So I would say right now, Curian is in the best position. But I will also say this is not an issue that has to be resolved anytime soon.

23:53I don't think there's any sign that the CEO, Koshay, has got any plans to move on. He's in a very strong position right now, so this is not an issue that has to be resolved, you know, for years, probably. I mean, it is interesting, though, that I think, you know, we obviously did a profile on Karai, you know, about a year ago. And it did seem like from that he is, you know, super politically savvy within the company, very good at kind of getting support from leadership, but also moving teams under him and gaining more influence and power in that way. So I do think, you know, I totally agree. It does seem way too early to even think about a successor for Sundar.

24:35But, I mean, he does seem like, Karai does seem like somebody who's politically savvy enough to maybe pull it off if it were to come to that. I think that that's correct. And I think it's just a question of how Google does under, now that he is formally in charge of that division, how they do. But I think he's definitely somebody that we should be paying attention to. I mean, it sounds like from your analysis that you feel like Wall Street maybe overreacted to yesterday's news, given that it was perhaps already the reality within the company. I guess I'm kind of wondering, though, if maybe the reaction from Wall Street was also maybe due to people, you know, Google maybe hasn't kind of had this moment last December, but since then perhaps has not really pulled forward as the frontrunners, people thought they would.

25:24So I guess I also wonder if maybe these leadership changes was maybe the – people were kind of reading that as like a sign of maybe Google internally struggling with AI or not playing to the front like people thought they were. Well, I think you have to unpack it a little bit. So for one thing, these stocks, I don't know if you've noticed, but they tend to move up and down a lot every day. It's true. And there are some stocks, SpaceX is a really good example, 10 % down one day, up the next day, down the next day. I mean, they just keep moving up and down. There's a very volatile period. So Google stock, I think this morning is down a tiny bit, but really, I'm sure it will bounce back.

26:13I think that the reaction was probably in response to the headlines, which were very negative about all of this. I think there are many analysts, at least, who make the point that whether or not Google has got the leading AI model is not going to really determine whether it succeeds in turning AI into a giant business. because it's got such a vast array of assets that it can use to power AI, that it can be powered with AI, that it's much better off than anybody else. And I really don't think it has to have the number one AI model, or particularly as everyone seems to agree, you don't need that, you know, really the best model for everything.

27:13So my guess is investors are not, you know, they respond on this kind of thing to the headlines, but then over time, but it's just a very short-term reaction, and over time things sort of smooth out. So, yeah, I don't think this is such a big deal, And I think over time, people will recognize that. Yeah, seems like we maybe shouldn't read too much into the performance of a stock over one single day. I mean, I think another big piece of news yesterday alongside these, you know, what we just talked about are the departures of pretty senior folks at Google DeepMind, including, you know, longtime AI scientist Jeff Dean, several others that worked on the Gemini model.

27:59You know, they're leading to go start this new company called Discovery Loop that's trying to build AI for scientific discovery. I mean, do you think there's a worry about kind of brain drain here from Google at all? I mean, I guess it's not ideal, but honestly, Google is such a large company and they've got an enormous amount of talent. And AI, talented AI people are moving around all the time. It's hard to—I mean, I don't think it's that smart to overreact to things. I mean, Jeff Dean is obviously a very highly thought-after person, highly thought-of person, but what we had heard was that he was not as important as he had been once.

28:50I think he's important in terms of being that person that people look up to. I mean, look, it's a bit hard to say, but again, I wouldn't be that concerned about his departure. I mean, as I said, they've got many people, and I'm sure they can afford to lose some. Great. Okay, well, thanks so much, Martin. And again, that was Martin Peers, co-executive editor here at The Information. Speaking of those recent Google departures, our next guest, Rob Toews, a partner at Radical Ventures, is an investor who's actually backing their new startup, Discovery Loop. Welcome to the show, Rob. Thanks, Stephanie.