Bonus: The DeepSeek Reckoning in Silicon Valley

0:00It's time for a bonus episode exclusively about DeepSeek R1 as the Chinese open source AI model royals markets and threatens to upend the generative AI industry. That's coming up right after this. Welcome to Big Technology Podcast. We're doing a bonus edition today exclusively on DeepSeek, what it means for the AI industry, what it means for markets. We're going to touch on technology. We're going to touch on business. And so thrilled that you're here for a bonus episode with us. We're joined today by M.G. Siegler. He's a writer and investor. He writes Spyglass. You can find it at spyglass.org.

0:38It's a great newsletter. It's a must read for me. And he has a great piece out called AI Finds a Way. Has Deep Seek changed the AI game or just some equations? MG, great to see you. Welcome to the show. Great to see you, Alex. Thanks for having me back. And sorry for my crazy winter beard. It is very cold and rainy right now in London, so I'm not ready for spring yet. Hey, it fits the season. I was just out in London to interview Demis from DeepMind. That's right. I listened to that. That was very good. Yeah, and very timely now. Yes, I can confirm the sun does not shine in that city this time of year.

1:10So first of all, I want to talk a lot about, I mean, only about DeepSeek and DeepSeek R1 and what it means for the AI industry right now. We are just about, the markets will open on this show. So we'll have a sense as to what it's going to do today. But it's looking pretty bad, especially for NVIDIA and some others. As we get going, I just want to thank all the podcast listeners who pointed me to DeepSeek. Because we had some comments that came in over the past few weeks. I was able to ask Demis about it. I was able to get it in as the lead story on Friday's show. So thank you. I appreciate all of you for pointing me towards DeepSeek.

1:42So let me just talk a little bit because we didn't touch on this Friday and we're going to definitely fill some holes that were left on the Friday show. We talked a little bit about how much it costs to train this model, but not necessarily about the benchmarks it hit and about the cost it costs to use this thing. So first of all, it's an open source model. It's much smaller than any of OpenAI's model. yet on the AIME mathematics test, it scored 79.8 % compared to OpenAI's 01 scoring 79.2%. So it bests OpenAI's best model on that. It scored 97.3 % on the math 500 and it beat OpenAI, which scored 96.4%.

2:23Look, these are lots of different benchmark tests, but you can tell that just by these numbers, it holds its own. And now the most remarkable part about this, It costs$0.55 per million token inputs and$2.19 per million token outputs. Just to give you a sense, OpenAI costs$15 per million input tokens and$60 per million output tokens. That's 3.5 % of the cost that it costs to run OpenAI's 01 models. And you can do it again. It's open source. You could download onto your computer and run it. So basically what DeepSeek R1 has done in a nutshell, and then we'll turn it over to MG, is it has created models that are as performant as the state of the art, right?

3:06It's ranked number three in the chatbot arena at 3.5%, 3 to 5 % of the cost. And that has huge implications for the technology, for the business, and we're going to get into those. So, MG, first question for you. If there was an AI Richter scale, right, assessing how big of an earthquake this is, what would you give this development? um so i mean it depends on on what i guess uh level you're you're you're sort of measuring the magnitudes right because as you noted the markets will open and that's going to be you know right now you know last i looked in in pre-market trading nvidia was down i think 11 10 to 11 percent um and that's the biggest hit right now microsoft a bunch of others are like in the 3 % range.

3:54So, you know, from a pure market perspective, it seems like it's, let's call it an eight. You know, it's not going to totally destroy the stock market right now, but it's going to be rough, it seems like today. From a bunch of other perspectives, I think, you know, it's probably a little bit less of a shake in these earlier days. And I think that's because everyone's still, even now sussing out what exactly this means for all different sorts of things. You noted how much cheaper it is to run than, say, OpenAI's models. And over the weekend, just reading all of these reports about the model and how many individual startups are even just swapping out already because it's so much cheaper to do what they're doing right now by swapping in deep seeks models.

4:47And so what does that do immediately? Like, you know, do we have to have price cuts immediately? And, and, you know, I think you could sort of see OpenAI doing some stuff. I think Sam Altman tweeted, you know, maybe on Friday, like about how they were like bundling, rejiggering some of the bundles, right, that they have, like what's in the free offering and stuff. And it sort of feels like we're going to see more of that, you know, as a response, obviously to some of this but then you know there was a there was a big report uh i think in the information about meta's response to this in particular which seemed pretty interesting in that like you know that's all hands on deck certainly and there's like all these different teams waiting time you and i remember that from the old school days of facebook um and so yeah it's just like all these companies are now scrambling you have satya and adela tweeting out things, you know, which seem directly aimed at the market to try to, you know, ease that pain a bit.

5:46But anyway, going back to the original question on the Richter scale, you know, overall, I think a lot of people are still figuring this out. But right now, the market thing is going to be the most acute one, because that's obviously going to open. And I think it's going to be pretty hard for, you know, this day, at least. And then I think I read some of the early analyst reports on this and they're all over the place, right? Like there, there's some folks who are saying like, oh, this is, this is awful for Nvidia. Some folks are saying, you know, this is not a big deal. This actually could be good in the longer run for Nvidia in, in ways.

6:19And, you know, and then from big tech on down what the ramifications are there. And you mentioned that some startups are already swapping in a deep seek R1 for the models they're using right now. How widespread do you think that is? Are they, are any of the startups that you speak with just saying, okay, well, to hell with OpenAI or to hell with Lama. Time to put DeepSeek in? Or is this just beginning? Because it's, again, something that dropped last week. Yeah, I think this is just beginning. I think, you know, people will experiment with it, right, just to see like how much you could, you know, get by swapping them out, given the price differentiation you were talking about.

6:52But also there's downsides, of course, like people have noted sort of the, you know, the censorship within China and of certain terms. And so, you know, I don't think everyone is quite certain what's in there. You know, it's an open source in that it's open weight, but it's, you know, it's not clear exactly everything that's going on in there right now. And so I do think that if this proves out, say if DeepSea can release another iteration of the model and it still is on the same sort of, you know, footing, I think that then you'll start to see more startups potentially taking it really seriously.

7:26I think now it's just a wait and see approach for sure. And just people trying out to see if it is, in fact, as good as they say, because I think, you know part of this like my initial gut reaction you know deep seek obviously as you noted had been around for you know basically since december and didn't really get all of the the mass of uh pylon until sort of friday right when r1 came out and in part it's like you know i've just i don't know why my mind was drawn to this but it's sort of like when they were talking about the room uh the room temperature uh conductor right like and everyone was talking about oh my god like there's this huge breakthrough that's happened, and this is going to revolutionize everything.

8:02And then it turns out, oh, you know, maybe there was some funny business in that claim, and maybe it wasn't, you know, all it was cracked up to be. And of course, that turned out to be the case. And so I'm not saying, obviously, that's not the case with DeepSeek. It seems like now this R1 release has legitimized it. And as you note, on leaderboards and whatnot, people have been testing this. And again, the startups are part of that pressure test. Right. And so the funny business, just to get this out of the way, the funny business might be on the training side. Like we think that they trained it for much less money.

8:33We think that they trained it with inferior GPUs that have been sort of the only things they can get their hands on due to export controls. We're not 100 % sure if that's the case. Right. But I think the bottom line here is that this is an open source model. It has been replicated. I mean, it has been downloaded to people's computers and used as effective as it is. And I think that the thing is the methods and the cost savings and the performance, that's all real. So even if, you know, basically all of Silicon Valley without those export controls couldn't do this or didn't do this. And maybe it's because they had a different method.

9:10And we'll get into that. But the fact is that there's no putting the genie back in the bottle right now, which is that this company has created something that can rival OpenAI's performance at 3 % of the cost. That's the big thing. So, sorry, go ahead. I also just think the overall mentality is one of the more interesting sort of earthquakes, to use your phrasing of it, that's happening right now. It's like, and I think Stephen Sanofsky summarized this well. He wrote a very long tweet thread, as he is wont to do, but then he also published it on his newsletter as well. But he goes into the history and he obviously has a lot of good historical context from Microsoft days on forward about what is going on here.

9:54But it's also, I think, important to talk through how the constraints that were put in place by the U.S. because of everything going on with chip constraints and sort of forcing AI companies not to export to China led to sort of this very interesting cauldron that I think could only happen in a place like China right now because they're so constrained. Whereas in the US, like it's still the period of abundance, right, with AI and everyone's going after the scaling. And it's and it's it's just not something they were going to focus on trying. You know, they're making the smaller models are making the mini versions of the models.

10:37And those are great. And we're seeing that. But China, you know, the folks working in China had to do this this way. And I just think it's something you couldn't have seen in hindsight arise out of the US in our current environment. Right. Okay, so I want to talk quickly about the technology, very quickly about the technology, and then get into some of the more business side applications here. So, MG, could you tell us just at a really high level, what DeepSeek has done to be able to get these results? Because, you know, it's one thing to say, okay, they were able to do it on worse chips with a smaller amount of data.

11:14But I think just it's important to very briefly highlight just the technical technological innovation here. Yeah, I mean, so and, you know, I'm not a I won't be a technical expert on this, but from my understanding, it's basically, you know, obviously, as you know, it started the DeepSeek project started out of a hedge fund that was focused on quant trading, you know, in China. and they had acquired a bunch of NVIDIA chips. I think they were H100s, you know, before all the import restrictions came in. And basically they had those servers up and running and, you know, presumably they were running a bunch of different models, including some of OpenAI's, but including also a bunch of the LLAMA stuff that Meta's been working on.

11:59And, you know, they've just used the process of distillation to, you know, effectively bring those bigger versions of the sort of state-of-the-art models and distill them down into, you know, smaller models, which eventually led to this R1, you know, the equivalent of O1 on OpenAI side. And again, for a fraction of the cost, fraction of the compute, and a fraction of the size for these to be able to run. And that latter part seems like it's sort of being under discussed right now, but is important. Because yeah, all of these models have constraints about how you can run them like on your personal machines, right?

12:41Because, you know, they're going to require so much RAM and so much memory to be able to do that. And if you can get them down to really small sizes, which again, the bigger US companies have been doing with these mini models, but they're sort of taking this bifurcated approach, whereas, you know, now we're getting to the point with this R1 model where it seems like it can run on pretty much a lot of different type of hardware, which again, they need to do in China because of the restrictions that they have there. Right. And there's also a methodology change here, which is that they've gone from effectively self-supervised learning, which is what has been used to train all of the LLMs, all the big LLMs to this point, to pure reinforcement learning where the models tend to figure out what the right answer is on their own, which is just fascinating.

13:27Yeah. And it's seemed like the, you know, sort of the American powers that be maybe felt like we weren't ready for that yet to happen, right? Like that was always the hope that we get to those points. And that, you know, we still were in the scaling point again, where, you know, you need someone in the loop to be able to check and make sure all these things are working. And China, you know, this Chinese company, because of some of the restrictions that we just talked about, like, just went for it and you know it's proving itself right and and just to harp on one more technical issue before moving on the distillation of models to me is fascinating that they could take any big model and distill it using this form of uh training and effectively be able to replicate its performance so i could take they took i could take like a llama model which has 70 billion parameters and distill it and then all of a sudden run it with this reasoning reinforcement reinforcement learning style approach and it's cheaper more efficient it's it's just i mean again like i think the entire world is still trying to wrap their head around this and there'll be more on this feed to talk about exactly how impressive this is but to me in the early innings of this that is astonishing yeah and i mean it again at a high level it it makes sense it's just it's incredible how it's happened because like do you need all of the world's knowledge you know in every single model for every single use case?

14:53Of course not. Like that's going to be overkill for almost everything that you're going to do. And so does it point to a world where, yeah, we sort of lead towards more of these specialized models that are distilled? And obviously that's been happening, but this, this one is still, you know, a model that can effectively do most everything distilled down from, from those bigger ones. So there's one sort of big question that I I think needs to be asked here, which is there's been this all Silicon Valley, and you point to this in your piece, all Silicon Valley has been operating on effectively the scaling hypothesis, which is that you add more compute.

15:29We talk about it all the time on the show, add more compute, add more data, add more power, add more training time effectively to these models, and you will improve. and now what deep seek has shown is that you can actually do all this without that and so i'm curious if you think that this invalidates the scaling hypothesis because and it might seem kind of like a you know obscure thing but it's very important because this sort of sets up the whole business conversation which is if the scaling hypothesis is invalidated then all that multi trillion dollar investment investment in NVIDIA, NVIDIA CPUs or GPUs, my bad, becomes sort of thrown into question.

16:16So what happens to the scaling hypothesis from here? And it's fascinating timing too, right? Because this is this is at the same time that everyone has now talked about sort of the quote unquote, AI wall being hit, right? And even Demis, you know, when you when you talk to He noted that he doesn't necessarily believe in a wall being hit, but he did acknowledge that things are slowing and it'll just take longer to get more juice out of the squeeze, as it were. And so that's sort of the natural evolution that's been happening. And everyone is now pointing to it or at least acknowledging that some aspect of that is real.

16:55And now at the same time, this comes along and calls into sort of more question. There's one other element that sort of I think is related to this, which was the big news story last week as well. The project Stargate, OpenAI and NVIDIA and Oracle all coming together. And one of the more interesting elements of that was the fact that Microsoft is effectively pushing off the compute costs to Oracle and some of the other players in that situation. And, you know, there's all sorts of reasons, you know, potentially why they're doing that, obviously, given the interesting relationship between OpenAI and Microsoft.

17:38but at the very highest level again if they're thinking that you know our capex is going to be we've already stated it's going to be 80 billion for the year we don't want to add another several billion you know for this this particular project and why would they do that in part probably because they're not necessarily sure that it makes sense to pay the billions upon billions to OpenAI to keep trying to scale on the frontier models. And this is sort of in line with what DeepSeek just did. Right. Yeah, it's interesting. We're also talking about Andreessen Horowitz, who sat out OpenAI's last round, and we were wondering on the Friday show, maybe they saw this coming.

18:18And it is interesting. I mean, you put it pretty perfectly in your story. You say big tech companies are now the most largest and sorry, you say big tech companies are now the largest and most well capitalized in the world, which means that they have effectively all the money that they can put towards scaling. And the hammer met the nail. But there's no point hammering the nail after it's already been put into place. And that's the point that can't be predicted, but is obvious once it's done. The question is, if DeepSeek just pointed to the nail already hammered, effectively, did they just solve this?

18:55It's sort of like going up the scaling question in a similar way. An analog for the same thing, right? And going back to the history of compute, like, right, all these, you know, the powers that be tend to spend, at the time, tend to spend a ton of capital on the build out of whatever the new technology happens to be. And, you know, there's obviously, we all benefit from it in the long run, but in the short run, you know, this segues into to, I guess, what's potentially going on with Wall Street and what it means for these larger companies with regard to the spend. Yeah. And I just want to ask the question that you put in your newsletter just to you directly.

19:37Did they just point to the nail? Like, is it done? I mean, again, I don't want to caveat this out, but I do feel like it's the exact question that everyone is sort of going to be scrambling to answer over this next week. And I think that it's not going to be as black and white as that for sure. But I do think if I had to guess at a high level, I do think that there's some element to yes, the nail is already sort of driven into the board. And we're moving on to what the next steps are. That's not to say it's over. And you know, there's no innovation from here. But I think all of these things are in a way related, like that we've just been talking about.

20:22And the fact that they're all coming together at the same time, I don't think is a coincidence. I think it's because like, yeah, we're at the point where we now need to move on to the sort of the next phase of the AI revolution, as it were. Yeah. And let's get into the business. And I'm smiling here because you're making me think of, we have Reid Hoffman on the show on Wednesday, and I interviewed him before R1 came out. And the first half of the conversation is just talking about all the billions of dollars that have been spent and when they're going to get an ROI. And I mean, I'm still going to run the conversation, but there's going to be some context in there.

20:56Yeah, it's interesting knowing after the fact. But it's also I think Sanofsky brought this up, too. And I was sort of looking into this more last week. You saw it was a smaller news item, but both Microsoft and Google had altered the way that they're basically bundling together AI within, you know, either the 365 suite and within the Google suite of apps, because they're clearly still trying to figure out how exactly you make money off of all this spend and what the right model is and how you spur on usage of it. And this just comes in and throws a grenade, you know, into that equation again. And this gets us to like some of like the real thorny business questions.

21:38So just to kick this I took a look at what all the big tech companies were doing pre-market. So this will obviously change across the day, but I imagine they'll stay directionally kind of the same. NVIDIA down 10%, Microsoft down 4%, Google down 3%, Meta down 2.6%, S &P down 2%. So this is all based off of this deep seek reckoning or this deep seek realization. And let me just put the sort of question to you, I think about as pointedly as I can, which is that the AI industry up until this point, like all the numbers we're seeing within Wall Street, the trillion dollar market caps, the billions of investment, the billions that have been raised by companies like OpenAI and Anthropic from companies like Microsoft and Amazon, right?

22:30So this is basically the whole game here. They have effectively been what's been driving the numbers. And the question is, can we, you know, basically Wall Street has been following that and saying we expect them to get a return based on those numbers. And in fact, a lot of this AI spend was just a wealth transfer, I would say, from like meta advertising to Lama, from Google search revenue to Gemini, from Microsoft Azure to OpenAI. So what happens here? Because, you know, basically, if they if a lot of the AI industry has been driven based off of subsidies coming from other businesses and doesn't need that type of spend anymore, like does the party end?

23:16so i think it's different for each company probably microsoft and google are closest you know aligned in terms of where they net out and it's sort of interesting you know the the numbers you just rattled off with where the stocks are at that feels you know just like a very um clear picture from wall street what they think now right like they think nvidia is going to get hit fast because uh in this in this doomsday scenario because obviously they're the beneficiary from everyone, from all of those companies, all those other companies that you mentioned, big tech is pouring as much money as possible as they can.

23:50They can't get enough chips fast enough into NVIDIA. And if they pause that, that obviously is bad news for NVIDIA in the short term. Again, I think there's longer term stuff that's different for NVIDIA, which we can talk about. But to just hit on the rest of this question right now, I think that Microsoft and Google, which are, as we just mentioned, you know, are trying to sort of figure out the right models for how to charge for AI. I think that this puts them in a really tricky situation if the underlying economics just totally changed overnight of what AI's, yeah, underlying economic model should be.

24:29And so they were, you know, moving around different pieces, trying to get to the right end state so that, yeah, they could ultimately prove to Wall Street, like, look, we're adding, you know, X amount on top of what we were already doing revenue wise, thanks to AI. And a little bit, there's a little bit of weird obfuscation stuff going on there, right? It's like, well, it's bundled in now to 365. And so, you know, we don't necessarily need to tell you exactly what the uplift is, but you can just, you know, assume that it's a part of this, because it's all baked in. And AI is like, you know, the new internet and blah, blah, blah.

25:03And so, you know, there's ways that they can finesse the messaging around that. But that, you know, to your exact question, I do think that there's varying degrees of being worried, certainly within Google and Microsoft. Meta is more interesting because their open source philosophy, open weight philosophy and model is so similar to what DeepSeek has done, right? And so the problem there, in my mind at least, is again, they're spending whatever Zuckerberg just threw out,$65 million or whatnot, he said at the end of last week, that they're going to spend on capex and so why are they spending that amount now if if you know deep seek can do it for you know pennies on the dollar if not even less than that um and so what does that that mean for their world so in my view high level i think that meta is probably in a bit better position than the other ones just because they at the end of the day they do want like you know their whole philosophy is to open sources, not for necessarily altruistic reasons, but because they know that it's historically helped them help their business, you know, to open source these things.

26:13The question of if it's not them open sourcing, it becomes pretty complicated if someone else's, you know, you have to use someone else's models, but they can pull back spend. It feels like a little bit easier than the other folks can. On the other end of the spectrum, open AI, like they're, you know, The entire business is sort of built around being at the frontier, and they've done a great job with that. They're a little bit different than Google and Microsoft, in my mind, just because they've done a good job getting mind share, both in terms of brand and product. Like JetTBT is number two in the app store right now behind DeepSeek for a reason.

26:52People are interested. It's a brand, and they know it. And so what does it look like, though, if they're not the ones sort of powering the models? I don't think that they would give up and, you know, go with DeepSeek's model necessarily. But what does it mean if they're not sort of the only one or the main frontier, you know, model maker providing that? Like, so there's all sorts of interesting offshoots and ramifications of that. So, MG, there's like two views right now in terms of like what could happen with all this spending, right? One is Silicon Valley will continue to spend these billions and they might get, you know, incrementally better performance and stay slightly ahead of the open sources of the world, deep seeks of the world that can just emulate their models.

27:31The other side of it is that they continue to spend and then they basically hit AGI or like, you know what I'm saying? Like if the performance increases that we've seen with such little, sorry, if the performance increases that we've seen with such efficient use of capital from DeepSea can be emulated, then imagine what you could do with 100 times the amount of spend. So the models are about to become much more powerful and all these fantasies that people have about what they can do, many of which Demis and I spoke about last week, all of a sudden become feasible because the capital is there. So which side of this do you lay on?