20VC: Deepseek Special: Is Deepseek a Weapon of the CCP | How Should OpenAI and the US Government Respond | Why $500BN for Stargate is Not Enough | The Future of Inference, NVIDIA and Foundation Models with Jonathan Ross @ Groq

25:29They could literally give every piece of technology away for free. I am completely jealous of that because if I had that right now, I would open source everything. Because then you don't have to worry about it and you get everyone helping you. So I think meta is sort of because of the network effect thing, always in a position where open source is to their advantage. It almost doesn't matter where it comes from. Now I'm sure that they would prefer to have the Linux of LLMs, but I think the more it goes open source, the more of an advantage they have inherently. If you were Matza, would you do anything different?

26:04NETA is an amazing competitor. What they would normally do, if this was some sort of proprietary social mechanism, they would try and replicate, and then they would compete, and they would say, come join or not. I don't think that the come join works here, but the beautiful thing is all of the information for this model is available. NETA has already been doing this. They have way more compute. The question is, are they willing to scrape open AI like DeepSeek did? They've been super careful on everything that they've been doing and so that's the disadvantage Do you not put morals aside to win?

26:38This is the AI arms race and I think that's gonna happen I think people will like you cannot lose and so what it's done is it's changed the game right so Okay, so let's talk about Europe for a minute. We almost forgot about Europe. It feels like with Europe There's a lack of a willingness to take risk. There's a black mark if you get it wrong everything's about downside protection. Whereas in the US, it's like, that was a great effort you failed, but I'm gonna fund you again, right? So there's that difference. But when you look at the US and then you look at China, China practices RDT, research development theft.

27:18It's just part of the culture. And it's not just against Western company, it's against each other too. The difference is if you're a Western company, then the government steals from the Western company and then provides it to the Chinese companies, which is less fair. The famous stories of turning on Huawei switches and you see Cisco's logo and all the bugs and the right. So is that a new paradigm? I really hope not. Like for Europe to compete with the US, Europe has to adopt a more risk on attitude. Does the West have to adopt a more theft on attitude? I really hope not. like that's just like this really disgusting to me.

27:58I'm like literally repulsed by the idea. Are we not being idealistic? If you're running in a race with someone who's willing to take steroids, if you want to win, you're going to have to take steroids too. And then everyone is taking steroids. Whereas if no one was taking it, then everyone's healthier and you have a real competition. Yeah, it's a real problem. And the question is, can governments get involved? Here's the thing, I would love nothing more than to compete directly with Chinese companies on a fair footing. They have really smart people, deep sea has proven this, really smart people.

28:30But when the government keeps putting its thumb on the scale, we're going to try and avoid that competition wherever we can. And now there's no avoiding it. Maybe the governments just have to get involved. But dude, I'm being blunt. Xi Jinping cares about one king power attention. And Gross is the only thing that bothers to him. And AI is central to that. He will do whatever it takes to win. and having some rational discourse about some rules of play is but in the unrealistic. Okay, and it gets worse than that. China has a lot of advantages. The chief advantage is the number of people they have.

29:01Now, number of people is not sufficient. So you also have India. And India has an advantage from the number of people, but China has out executed. In fact, India was asking China for some time to help build out the roads and infrastructure. They've really mastered that, right? but people and sort of organization discipline alignment, right? And so what is the concern with AI? The concern with AI is what if an LP or GPU becomes the equivalent of a contributor to the workforce? And you could literally just add more to the GDP by creating more chips and providing more power. Now, if that becomes the case, just China's advantage a road.

29:40They're concerned that in terms of workforce, the US could catch up, the West could catch up. And then at the same time, they have a huge population advantage. This is why so much want for Europe to get into the fight on AI. There's 500 million people who could be jumping into this. I think you would advise EU today on Europe's dominance. What would you say? So have you ever seen station F? Yeah, of course I was last week, we had a system in the man. So, I would say, by the end of this year, you should have 100 station apps, and by the end of next year, you should have 1000. Done. So, what you're doing is you're collecting up 3 ,000 people and surrounding them with other risk -taking entrepreneurs, and then they're supporting each other, they're risk -on.

30:25And, ever, when you surround yourself with other people who are risk -on, you're going to be risk -on, and you're going to take the entrepreneurial leap. What does this space look like in three years' time? I'm obviously a venture capitalist for a living. All of my friends are going, oh my god, oh my god, we just lost hundreds of millions of dollars on these foundation model companies. How many companies are you aware of that have become incredibly successful that didn't pivot? Myus pivot. Yeah, exactly. So pivot, get over it. Just pivot. I've been talking to a lot of the LOM companies. And frankly, they have some good ideas.

31:01In fact, I really like, so I watched your interview with the Suno founder. I think he sought from the beginning, like models are going to be commoditized and that's why he's focused on the product He got it from the beginning. What is your product? Not what is the model model is It's a piece of machinery. It's an engine, but what is the car? What is the experience? What do you think propel is in three years? The question I used to get asked when when we were raising money a little while ago was is AI the next internet and I'm like absolutely not because the internet is an information age technology It's about duplicating data with high fidelity and distributing it.

31:38Telephone does, it's what internet does, it's what the printing press did. They're all the same technology, just a bunch different scale and speed and capability. Generative AI is different. It's about coming up with something contextual, creative, unique in the moment. And so the LLM is just the printing press of the generative age. It's the start of it. And then there's going to be all these other stages. It's just imagine trying to start Uber when we didn't have mobile yet. Great. I'm going to book a trip over to here. How do I get home? You can't carry a desktop with you, right? So you need to be at the right stage.

32:12So when I look at perplexity, I look at perplexity as being perfectly positioned for the moment that the hallucination or really confabulation rate comes down. The moment that these models get good enough, where you don't have to check the citations anymore. That's going to open up a whole set of industries. All of a sudden you'll be able to do medical diagnoses from LLMs. You'll be able to do legal work from LLMs. Until then, it's like trying to create uber before we add smartphones. It just doesn't make any sense. However, people are willing to use perplexity today, even though you have to check the citations.

32:50So they have an actual business that gets to continue. So they're getting to sort of ride the wave. And the moment that that tsunami of lack of confabulation or hallucination comes along, they're perfectly positioned. Each company has to find their own thing. And I would look at Suno as a great example of how things are being done around the product, as opposed to just the models. You can get it all sort of pivot when you are open AI or anthropic. or any of the very large providers who've ingested billions of dollars. Disruption happens. If you're not able to pivot now, you're not going to be able to pivot later when you get disrupted anyway.

33:32One would think that with commoditization of models and with cheaper inference, that actually big tech wins, right? Have you seen this dot market today? They've been hit hard. How do you think about that? What you see is a bunch of people who are concerned about training and the need for it and everyone's still thinking that most of compute is training and that there's going to be less of it because someone trained a model on 2000 GPUs and the nerfed a 800 version with slower memory or whatever it is and they're like, oh, people aren't going to need as many chips. But again, Jevin's paradox, right?

34:11The more you bring the cost down, the more people consume. So for the last five to six decades, like clockwork, once a decade, The cost of compute has gone down 1 ,000x. People buy 100 ,000 x as much compute, spending 100 times as much. So every decade, they spend 100 times as much. So you make it cheaper and they want more. What's really happening is every time one of these models gets cheaper, we see our developer count just skyrocket. And then it comes back down a little bit, but the slope is higher than when it started. Better models create more demand for inference. more demand for inference than has people going, I should train a better model and the cycle continues.

34:52I just bought a shitload of Nvidia. They dropped 16 % on the thesis that the increasing efficiency means that obviously we wouldn't need as much Nvidia chips. And I thought exactly that, which is why you'll still need the Nvidia inference and you'll just have much higher usage. So to me, it's the most screaming by the century. Do you share my optimism on Nvidia given what you just said in Japanese power roles? So I think over the long term, the only thing I say is Warren Buffett and Charlie Munger in the short term of the market is a popularity contest in the long term. It's a weighing machine.

35:22I can't tell you about the popularity contest, but in terms of the weighing machine part, this is a misunderstanding. It's actually more valuable thanks to deep seek, not less valuable. Okay, so Jevin's paradox was actually discovered by Jevin as recently made famous in Satch's tweet. However, I did beat him to that by quite a bit. And just as Satcha likes to say that he made Google dance, I'm going to say I made Satcha dance. He might take exception to that. But less than a month before he posted that, I did a cute little tweet on it. So what's really happening here was in the 1860s, this guy, Jevin, he actually wrote a treatise on steam engines, which I guess is what you did for fun back then in England.

36:06He realized every time steam engines became more efficient, people would buy more coal, which is the paradox. But if you think about it from a business point of view, when the op -x comes down, more activities come into the money. So people do more things. And so what's happened is every time we've seen the cost of tokens for a particular level of quality of models come down, we've actually seen the demand grow significantly. Priscilla Stissity, baby. A lot of people suggest that Nvidia's incredible high margin status, which I'm gonna butt track on on board it was in the latest release. It was for something 45 or whatever it was, but it was very, very high.

36:45And then relate to your margin as my opportunity. I think give it back to the seven powers and go, their margin is their defensibility. And it makes me really just consider the strength of their mode. Do you think your margin as my opportunity or do you think their defensibility is their margin? Today, there's this wonderful business selling mainframes with a pretty juicy margin because no one seems to want to enter that business. Training is a niche market with very high margins. And when I say niche, it's still going to be worth hundreds of billions a year. But inference is the larger market.

37:20And I don't know that Nvidia will ever see it this way, but I do think that those of us focusing on inference and building stuff specifically for that are probably the best thing that's ever happen for Nvidia stock because we'll take on the low margin high volume inference so that Nvidia can keep its margins nice and high. Do you think the world sees this? No, and I was actually like we raised some money late 2024. In that fundraise we still had to explain to people why inference was going to be a larger business than training. Remember this was our thesis when we started eight years ago. So for me I struggle on why I think that training is going to be bigger.

38:01It just doesn't make sense. Just for anyone who doesn't know, or difference in training and inference. Training is where you create the model. Infraints is where you use the model. You want to become a hurt surgeon, you spend years training, and then you spend more years practicing. Practicing is inference. Where does efficiency go from here? Everyone was so short. Why, how all one is so much more efficient, what next? What you're going to see is everyone else starting to use this MOE approach. Now, there's another thing that happens here. And the Aloe approach is what I'm saying is the segmentation of what information goes is rooted to the optimal point of the model.

38:40Yeah, it's called... So, MOE stands for mixture of experts. When you use Lama 70 billion, you actually use every single parameter in that model. When you use mixed roles 8 by 7B, you use two of the roughly 8B experts, but it's much smaller. And effectively, while it doesn't correlate exactly, it correlates very closely, the number of parameters effectively tells you how much compute you're performing. Now, if I have, let's take the R1 model. I believe it's about 671 billion parameters versus 70 billion for Lama. And there's a 405 billion dense model as well, right? But let's focus on 70 versus 671.

39:22I believe there's 256 experts, each of which is somewhere around 2 billion parameters. And then it picks some small number and forgetting which, maybe it's like eight of those or 16 of them, whatever it is. And so it only needs to do the compute for that. That means that you're getting to skip most of it, right? Sort of like your brain, like not every neuron in your brain fires when I say something to you about the stock market, right? Like the neurons about, you know, playing football, those don't kick off, right? That's the intuition there. Previously, it was famously reported that opening eyes, GPT -4, it started off with something like 16 experts and they got it down to eight.

40:04I forget the numbers, but it like started off larger and they shrunk it a little and they were smaller or whatever. And then with What's happened with deep seat model is they've gone the opposite. They've gone to a very large number of experts. The more parameters you have, it's like having more neurons. It's easier to retain the information that comes in. And so by having more parameters, they're able to, on a smaller amount of data, get good. However, because it's sparse, because it's a mixture of experts, they're not doing as much computation. And part of the cleverness was figuring out How they could have so many experts so it could be so sparse so they could skip so many of the parameters But if we take that then back to the fight that's where we are saying how they've become so efficient What's the the next stage of that then?

40:55Well, you're of course all these guys but they can ruse it so efficiently what nice So meta recently released their llama 3 .3 370B and it outperformed their 3 .1 405B. So there's new 70B outperformed their 405. What was surprising to me, I thought they retrained it from scratch. It turns out you read the paper and they talk about how they just fine -tuned, so they used a relatively small amount of data to make it much better. Again, this goes to the quality of the data. They have higher quality data. They took their old model. They trained it got much better. But that 70B, that new 70B outperforms their previous 405B.

41:35What you're going to see now is now that everyone has seen this deep seek architecture, they're going to go great. I have hundreds of thousands of GPUs. I'm now going to use a lot of them to create a lot of synthetic data. And then I'm going to train the bejesus out of this model. Because the other thing is, while it sort of asses from Toats, the question is on this curve, where do you stop? It depends on how many people you have doing inference. You can either make the model bigger, which makes it more expensive, and then you train it on less, or you make it smaller, and it's cheaper to run, but you have to train it more.

42:10So deep -seek didn't have a lot of users, until recently. And so for them, it would have never made sense to train it a lot anyway. They would much rather have a bigger model. But now what you're going to see is all these other people either making smaller models, or trying to make higher quality ones of the same size, but just training it more. We've seen DeepSeat now say, hey, only now Chinese phone numbers, we can log in. That is the new sign up I think it is. What's happened and what is the result of that? So they ran out of compute. And this is why, this is the other reason why chip startups are gonna do just fine, because they ran out of inference compute.

42:47You train it once, but now, so you spend money to make the model like designing a car, but then each car you build costs you money, right? Well, each query that you serve requires hardware. Training scales with the number of ML researchers you have, inference scales with the number of end users you have. Do you think deep sea are astonished by the response they've got from the global community? I think they marketed very well. Like, you look at some of the publication and they make it sound like it's a philosophical thing and, you know, they talk about, they spent six million on the GPUs and everyone just zoomed in on that, neglecting the fact that Lama's first model was trained on like I think 5 million worth of GPU time and it set the world on fire in a good way and then ignoring the fact that they spent a ton generating the data and all this they're really good at marketing.

43:40I think they were probably surprised at how well it worked but I think this is what they were going for. Is that anything that I have an aster we haven't spoken about that we should? What's up with the $500 billion stargate effort? Okay, well it's up to the 500 -bit Invalid's dogate effort. I've gone back and forth on that. I actually did, so Gavin Baker tweeted some math. Before I saw that tweet, I came up with very similar math. However, talking to some people in the know, some of the comments are actually, they've got it. But then you keep pressing and it's like, well, maybe there's some cuteness to it.

44:15What I think it is is an acknowledgement that the models have been commoditized. infrastructure is what's important in terms of maintaining elite like scale. It's one of the seven powers. I think what you're seeing there is an attempt to move from having a cornered resource or something like that into a scale economy. Do you think it will work? I don't think you get there in a short period of time with GPUs because most of the compute is inference. And so, you know, if you're talking about building out all the power, building out, like it's going to take time, it's infrastructure, it's cat -backs, the real win here is brand.

44:54That's what I would be doubling down on. I would be hiring the best brand firms I could, I would do a complete makeover. We'll open now I have a stronger or weaker brand in three years time. Much stronger. I think they're going to double down on that and they're going to focus on it. Who will lose? People who can't adapt to disruption. Anyone who just wants to keep going on a straight line and do what they were doing before is going to lose. And the rate of disruption is probably going to increase because going back to the analogy of LLM being the printing press. Imagine if there were a couple of smartphones left over from an ancient civilization.

45:30All of a sudden the printing press is invented and you're like, ooh, Uber's coming. I'm going to position for it. I know where this is going. We are the smartphones. We know where generative age technology goes. And now everyone's like, well, we know how big this gets. Let's put money into it. I can't be the one who doesn't spend money on this because I know how big of an advantage it's gonna be. It's like getting to add more workers to the workforce. And so I think the generative age, we're gonna speed run it faster than whatever comes next because we know what it looks like. Is that any chance we see a plateau?

46:06So we saw it in a self -driving, for example, where we kind of went through this desert of black or progression. And suddenly, all events at KEM, will we see that or will we just see this continuing dollars? I think with self -driving, the problem you had was the threshold had to be way superhuman. Because if you look at the number of miles driven by these self -driving vehicles, it's an enormous number and the number of fatalities and incidents is lower per mile. But we have no tolerance whatsoever for them when it's a machine. When you're writing poetry and code It's very different versus doing a surgery or or driving a car If you're Elon and X.

46:48How are you feeling and do you feel better or worse post this? I would probably feel both better and worse. I'd feel better about my Bet on building out more hardware. I would feel worse about trying to build out my own model. Why is Elon doing that? just pick one up off the ground. Like, why are you making your own? Are you excited when you look forward to the next few years? Or are you quite nervous? You could say this is a time of heightened international warfare in terms of this new AI arms race. China's stealing everything. Us forced to steal back. Long ago, I stopped having good days and bad days.

47:27It's yes, it's how many good things, it's how many bad things, right? When you run an organization, I'm both excited and nervous. And I'm excited and nervous about different things at the same time. The thing that I am most nervous about is that unlike nuclear war, you can use AI tools to attack each other. Google just announced recently the first zero -day exploit found by an LLM that was previously unknown. Yeah, that's a scary one. So now - Why is this scary for anyone who doesn't own Sansa or the expert? So how would you like me to have access to your phone? Not ideal. How would you like the CCP to have access to your phone?

48:07Even last the night. That's a nation state, and nation states have a lot of resources. And if they stand up a bunch of compute and they start scanning for vulnerabilities in all the open sources out there, and not even the open source, just like scanning ports on the internet and trying to figure out if they can break in, they can just automate that now. They don't need to hire people to do that. And now the defense has to be automated because there's no way to keep up with automated attackers. And what happens if this gets out of control? But worse, it's not killing anyone. And it's also deniable.

48:40That's the hardest part about it. Because is it really China? Is it Russia? Is it North Korea? Is it a friendly that's making it seem like it's one of that more vice versa? Now you have this ability. So you go from where we had a Cold War because having a war was unconscionable, it was unthinkable because of the consequences. To now, yeah, I'm just hacking you, that could spiral out of control. I'm worried that we're going to have more back and forth and think of it this way. If you are a nation state and you, let's say that Harry, you're a beacon to the venture community and you want to rally the European entrepreneurs to be risk on.

49:23And I'm someone who doesn't want that because I don't want the competition. A country that doesn't want that. Maybe I sell your reputation. Maybe I make you person in on grata. How is that any worse than shooting someone? It could be worse in some ways, but you can get away with it. And so that has me nervous, really nervous, but I'm also really excited. We are seriously going to be able to innovate as fast as we can come up with ideas. Now, you're not going to have to implement things. You're going to be able to prompt engineer your way through things. Just as we moved from hardware engineers to software engineers and sped up productivity, you're now just going to be able to have a prompt engineer who doesn't even write software.

50:05One of our engineers made this app where you can just describe what you want built and it builds it. And because we're so fast, it's like that. And you just iterate and it'll build an app for you. I just don't understand why the value, sorry to just continue, but why the value crews because you mentioned there kind of, hey, they created this tool which allows you to prompt a middle build the app. I'm sure you've seen Bolt .new, I'm not sure if you've seen Lovable, where it's basically Chatchy PC, but for kind of website creation. Is there value in that everyone was like, there's no value in these wrapper apps?

50:37Everyone's like, there's no value in these foundation models. Where the fuck is there value? And that's part of the exciting part. It's discovering that. But I think people will always prefer to use the highest quality, most polished product. I think there is an opportunity for artisanship, craftsmanship, and just perfecting it, getting to a certain number of nines in the details. The eems quote, the details aren't the details, the details are the thing. I used to be a little concerned with the quote, you know, if you're not ashamed of the quality of your first release and you've waited too long, because there's a subtlety in nuance there.

51:12There's soundness and then there's completeness. What you want is an incomplete product, something that doesn't do everything. That's why you should be embarrassed, but it shouldn't like blue screen of death on you. That's not a good embarrassment, right? And so what you're going to see now is because it's so easy to come up with something that just kind of works. It's a little embarrassing, but it kind of works. People are really going to value well -crafted, high quality products. Jonathan, I can't thank you enough for breaking down so many different elements for me. And putting up with my basic questions, you've been fantastic.

51:47No problem. Have fun out there. I mean, this is a brand new age. It really is. I mean, what I showed that was, if you want to watch the episode in full, you can find it on YouTube by searching for 20 VC. That's 2 -0 VC on YouTube. But before we leave you today, here are two fun facts about our newest brand sponsor, Kajabi. First, their customers just crossed a collective $8 billion in total revenue. Wow, second, Kajabi's users keep 100 % of their earnings with the average Kajabi creator bringing in over $30 ,000 per year. In case you didn't know, Kajabi is the leading creator commerce platform with an all -in -one suite of tools, including websites, email marketing, digital products, payment processing, and analytics for as low as $69 per month.

52:35Whether you are looking to build a private community, write a paid newsletter, or launch a course, Kajabi is the only platform that will enable you to build and grow your online business without taking a cut of your revenue. 20VC listeners can try Kajabi for free for 30 days by going to kajabi .com -4 -20VC.

53:00Once you've built your creator empire with Kajabi, take your insights and decision making to the next level, with AlphaSense, the ultimate platform for uncovering trusted research and expert perspectives. As an investor, I'm always on the lookout for tools that really transform how I work, tools that don't just save time but fundamentally change how I uncover insights. That's exactly what AlphaSense does. With the acquisition of Tegas, AlphaSense is now the ultimate research platform built for professionals who need insights they can trust. Fast. I've used Tegas before for company deep dives right here on the podcast.

53:32It's been an incredible resource for expert insights, but now with AlphaSense leading the way, it combines those insights with premium content, top broker research and cutting edge generative AI. The result? A platform that works like a supercharged junior analyst delivering trusted insights and analysis on demand. AlphaSense has completely reimagined fundamental research, helping you uncover opportunities. From perspectives you didn't even know how they existed. It's faster, it's smarter, and it's built to give you the edge in every decision you make. 20VC listeners don't miss your chance to try AlphaSense for free.

54:06Visit AlphaSense .com -420 - to unlock your trial, that's AlfaSense .com forward slash 2 .0. And speaking of incredible products, what comes to mind when you think about business banking? Probably not speed, ease or growth. I'm willing to bet that's because you're not using Mercury. With Mercury, you can quickly send wires and pay bills, get access to credits sooner to hit the ground running faster, unlock capital that's designed for scaling and see all these money moves all in one place. I speak to dozens of founders every week and most of them are using Mercury because they're super smart and that's what you have to be using.

54:47Visit mercory .com to experience it for yourself. Mercury is a financial technology company, not a bank. Banking services provided by Choice Financial Group, Column NA and Evolved Bank and Trust, members of FDIC. As always we so appreciate all your support and and stay tuned for an incredible episode coming on Friday with the CEO of Monzo.