AI Vibe Check: Chinese Open Models, Distillation & The Hugging Face Breach

0:00My mom like knows nothing about AI. She texted me last week, Rob, what do you think about OpenAI's agents hacking into Hugging Face? What do you two make of it? I think this is actually a great example of why open models of that capability are important. How much do you think distillation is the explanation for what's happened here? I don't think it explains nearly as much of it as many would like to claim. What specifically do you see as the risk of like dependency on Chinese open source models? I do think there are downsides to having the AI substrate of the world come out of China. I'm Jacob Efron, and this is Unsupervised Learning, a podcast where we probe the sharpest minds in AI on what's happening in the space today and what it means for the future.

0:35We've had a lot of new subscribers in recent weeks. As part of that, we have a great recurring segment with Ari Marcos from Datology and Rob Taves from Radical Ventures. The three of us chop it up on everything happening in the AI ecosystem. This one was particularly eventful and a ton of fun. There's been a lot going on. We talked about the OpenAI hugging face incident, how big a deal it actually is and what it means for the future of frontier AI research. We talked about Chinese open source models, including how good Kimi K3 actually is. And Ari shared some really interesting insights into what the real long-term risks are of Chinese open source models and how we know if those risks existed.

1:09We talked about the labs and how they're competing increasingly up the application stack and what it means for their customers and the development of models going forward. And we hit on what we think government policy will look like for frontier model releases in the future. I love talking to these guys about everything in AI. We do it a lot off camera. and it's particularly fun to get to do it on camera and share with you all. We're constantly iterating on the format, so any suggestions, things you want us to talk about, please let us know. But if not, without further ado, here's Ari and Rob.

1:38Ari, Rob, it's good to see you both again. Great to be back. It's great to be back. Thanks for having us. Yeah, we've got some new backgrounds, some new craziness happening in the AI world. It's all changing. Ari's background got better. Jacob's background got worse. It's a dynamic space right now. It is. It is. It's constantly ebbing and flowing who's on top. So, you know, no surprises there. But I've been excited. I feel like, you know, we had all these spicy predictions last episode, and there's like a follow-up on each of them, which is, I guess, what happens in the rapidly changing AI world.

2:10I feel like there's a ton of stuff we're going to talk about today. I'm excited to hit on open source, you know, government getting involved in a bunch of these models. You know, what's been happening at the labs, the hacking over at Hugging Face, as well as this new letter we have out today from a bunch of researchers trying to slow things down. But I figured, you know, no better place to start than with open source models. It's kind of been one of the main topics of the past weeks. We've had some incredible Chinese model releases. We've had a lot of intrigue around whether the government's going to allow said Chinese models to continue to be used here stateside.

2:40And then we've had letters signed by many people to keep open source models and backlash against Anthropic for maybe going against them. A lot happening. So maybe to start it off, Ari, I feel like you've been vindicated. Earlier this year, you kind of said, I think there's going to be a Chinese open source model. It's going to be the world's best at something. You're looking pretty smart with how things have shaped out. What's your read on how this has all kind of happened? Yeah, well, I mean, pleased with that prediction, obviously. but I think this trend is going to continue. I don't know if they're going to continue to be fully open.

3:14So for example, if you look at K3, they adjusted the license to have a revenue gate on it now. So, you know, there's a conservative revenue gate for model as a service and then, you know, less conservative one for otherwise. But if you make more than, you know, 20 million a year or 200 million a year, depending on the threshold or 20 million a month, I'm sorry, depending on the threshold, you can't use without a commercial license at that point. I have a feeling we're going to see a lot more of those starting to come out across the board. But I do think we're going to continue to see extremely strong models coming from China.

3:46And then also, you know, now we have Inkling from Binky and a couple others from U.S. players that will hopefully continue to fill out that space. But I think this trend is only going to continue with seeing powerful models that are in the open and that are outside of the frontier labs. Yeah, I mean, I think, Rob, you had said last time you were a little skeptical of the open source business models. And so maybe we're starting to see the outlines of how that might come forward. Like, do you kind of, you know, is this inevitable that, like, this kind of new model is required? And is that 20 a month going to come down pretty soon to be a little more all-encompassing?

4:22well i so first on this topic i would say while there is no question that aria is very wise um i i would i i would actually posit that i think the kimmy k3 launch and and other open models like i think they've been i think it's been oversold a little bit how big of a milestone it is because like i don't i don't think k3 is actually state of the art in any real sense like i know there There are certain categories that it outperformed, but it's important to keep in mind that Kimmy K3 has been compared to Fable, which is itself a neutered version of Mythos, which itself was finished training in February or March.

5:07And so all of that taken together, I think it's fair to say the open frontier coming out of China is still several months at least behind the actual frontier in the US. So I do think that's one piece of the narrative that has gotten kind of lost because it's a much more compelling narrative to say like, oh, China, open source caught up to the US frontier. And I don't think it actually has. I mean, it's certainly like a very, it's the best open source model. Like in that sense, it's state of the art. It's the best open way model. To push on that for a second, like barring a full hard takeoff sort of scenario, does that actually matter?

5:39Like does open source being three to four months behind closed source, is that actually a meaningful difference? It depends on the use case. Like certainly there are a lot of use cases for which it doesn't matter. And I do think there's this broader trend of companies, enterprises rationalizing their spend on AI and realizing like we shouldn't be using the cutting edge frontier model for everything. And in fact, it's like totally nonsensical from a cost perspective to do so. But I do think it's like symbolically and strategically very important, like where the frontier is and how far behind the frontier other players are, including if you, yeah, Yeah, if you take seriously the possibility of recursive self-improvement or so forth, like that advantage can compound to your point about a hard takeoff.

6:22If there is a frontier, there's certainly many customers out there that are buying AI products that want to feel like they are getting the best out there. Again, like I don't know if anyone wants to feel like they're going to a health system or a law firm that's not using like the most cutting edge thing, even if it ends up being, you know, like slightly better or worse in some areas. I would guess consumers don't know the difference. I would bet that most people have known the difference for like years. No, I don't think any. I think certainly one thing that I think is interesting is like you get this really bimodal distribution, especially if you go to like Blue Sky, where there are like all these people who like absolutely hate and despise AI and have clearly not used anything except for like GPT 3.5.

7:00Like the general public is actually misled substantially by the free tier of these models, because I think many of them think that's the best. But I would I would guess that actually like most customers are going to have no idea about this. And if anything, I think we've started to see this backlash to the token maxing idea in some ways. It's all gone to routing companies and how there's so much more you can do there. So I think the right size model for the task is something that everybody is increasingly moving towards more and more. And I think that reduces the benefit of being at the frontier because it now only is relevant for whatever subset of your problem space actually matters where you need that marginal increase in intelligence.

7:41Yeah, exactly. You better hope that the frontier keeps improving capabilities so that that set of tasks keeps increasing. Because it's certainly things like the ones that anyone that's there a year ago is now perfectly addressed by these open source models. You know, Ari, one thing I think you've been consistent on since the start of us recording these is like that maybe in the narrative in the Western ecosystem, we always like to say, you know, it seems that this broader narrative emerged for these Kimi models like, oh, it must be distillation like that has driven this. And obviously, there's been incredible research, you know, out of DeepSeek now, out of, you know, incredible models out of Kimmy.

8:14Maybe contextualize for our listeners, like, how much do you think distillation is the explanation for what's happened here? I think it explains some of it, but I don't think it explains nearly as much of it as many would like to claim. It's definitely not a zero advantage. There is a real advantage from distilling, especially distilling the reasoning traces. Those are obfuscated in many ways, but there have apparently been some ways to get around that and so on that have led to some of that. And there is real value there that is non-zero. That said, I think trying to take that to the claim that the only reason that the Chinese models are even remotely competitive is because of distillation and they're just piggybacking off the hard work of the American companies, I think that stretches credulity a little bit.

9:02It's somewhere in between. They're meaningful, but it's not the end-all be-all. I also think there's something to the effect of, you know, obviously many of the American frontier models, at least initially, were trained on a lot of data that wasn't permissively licensed. And you could argue that they were distilling from everyone. So then they turning around and saying, hey, no one distilled from us. There's some amount of hypocrisy to it, even if it's understandable from a business incentive perspective. Rob, where do you think this conversation on distillation like ends up? you know, is it something that can be policed?

9:34Is it something that like actually we want to be policed? Ari probably has a more concrete sense from a technical perspective, but my general sense is like, I think it's very hard to prevent distillation as long as there are models that are publicly accessible via API. Like I think the only, the way to shut it down would be for Frontier Labs as Ari like intimated in the last conversation to say, you know, we're actually gonna like, shut down our APIs and not make our models publicly available and use them to monetize in proprietary ways, you know, whether that's trading on stock markets or, you know, building your own products end to end.

10:15So it feels like distillation is kind of an inevitability. I think it's something that like US labs are doing and Chinese labs are doing, like Meta is doing it and you know, other non-frontier lab companies are also doing it. I think in this open source debate, there's obviously been a ton of discourse over the past few days on Twitter. And frankly, it kind of feels like Anthropik has kind of been framed as the bad guy in a lot of ways. Certainly the vibes are moving against them in the sense that they're the sole company that didn't sign this open letter. And they're kind of being cast as the company that's trying to shut down open weight models for competitive reasons and regulatory capture.

10:59I like my view on it is I do think there's like a very different I think a couple different conversations are being uh muddied here I think there's it's a very different discussion about open weight models versus open weight models from China and I think that there's not really good argument as to why open weight models in general should be banned or heavily regulated and I don't think Anthropik is advocating for that in fact Dario published a letter yesterday where he said, like, to be clear, I'm not like, this is, this has never been the stance of Anthropik. I do think there are like actual legitimate geopolitical reasons to be very cautious about how reliant we want us enterprises and like the global, you know, global users more broadly to be on Chinese open models and to have a Chinese tech stack, the, the foundation of AI going forward.

11:50So I do, I actually think that like, I think that Anthropik's position is more defensible than, like, is popular to believe right now. Yeah, so I think that makes sense to point. But, you know, in the anthropics position, it's in so much open weight models are fine and they're not against it. It's more that they don't care whether they come from China or the US, but if they're too powerful, they don't think open weight models should exist. And I really question, you know, that argument. I think we'll talk about the opening eye-hugging face incident in a bit. But there's a very obvious asymmetry here if, you know, only a small set of people have access to the attacking model and the defense model is then very limited.

12:34These are going to proliferate. There's this notion of, OK, we want to pause AI for safety reasons and so on and so forth. And like, look, I think a lot of AI researchers would actually support a pause if If there was some mechanism to globally enforce it. 1 ,122 and counting right now that have signed this letter that came out the day we were recording here. The thing is, though, it only makes sense if you can actually globally enforce it. And like while I would be, for example, you know, in favor of some sort of pause for safety reasons, if it could be globally enforced, it absolutely cannot be globally enforced.

13:06That's just not a realistic thing in any sense of the word. Right. So you have to assume that these models are going to continue to get better. And that's going to mean that you're going to end up with the bad actors having access to the more powerful models and good actors not necessarily having that access. So I think the argument that like open weight, powerful models are inherently dangerous and therefore a very limited set of players should have access to them is both not an effective strategy and not realistic. I think to some extent, Anthropic was walking that back. But I also, you know, I think it's hard to not see their business interests in that, particularly given the extent to which, you know, they have been particularly framing this as a war between America and China.

13:53Just wanted to take a quick break to say that if you're enjoying the conversation, the single best way to support the show is hitting follow or subscribe wherever you're listening. It makes a real difference in helping us get the best guests and helping others find the show. Now, back to Ari and Rob.

14:35and the countries represent different value systems. And I do think it's clear that AI is becoming one of the most important axes on which these two countries and two kind of world spheres are competing. And so in theory, it would be great if, you know, it's a positive some world and everything is great and China can do great and the U.S. can do great. But I think in reality, like there are there will be a lot of kind of political stakes and policy decisions that are made globally and global influence that has real implications for the way people live around the world. And I think we're in a better place if the technical foundation on which all these applications are built globally, not just in the U.S.

15:25and in China, but in Europe and in Latin America and in Africa. I think it's a real power disadvantage for the US if the foundation of all that technology is built on Chinese models in terms of the training data that went into them, the values that are embedded in them. I do think it's kind of a soft power thing, but I do think it's meaningful. And in a way, it's similar to how... I think China has done a really strategically foresighted thing in investing many, many billions of dollars in developing the infrastructure in much of the developing world in Southeast Asia and Africa. And that gives them a lot of influence over those countries and their governments and their policies.

16:14And I think a similar thing can and will happen in the digital world. So I think if you're if you believe that a more free and democratic and open society is better globally, like I do think there are downsides to having the AI substrate of the world come out of China. I'm actually inclined to agree with part of that for sure. Like I think between ignoring the problem with open models generally, which that I am very skeptical of, but specifically about Chinese versus non-Chinese open models. One kind of truism of model customization and model building is that the kind of easier it is to put in, the easier it is to take out.

16:53and by extension, the harder it is to put in, the harder it is to take out. So what do I mean by that concretely? If you can post-train a behavior into a model with order of 1 ,000 samples, you can likely remove whatever capability change you've injected with order of 1 ,000 examples. If in order to induce a new capability, it requires trillions of tokens of data, it will likely require trillions of tokens of data to remove that, or very well might to fully remove it. So I think that the situation, which I do think is actually realistic, and I'm not sure if they're doing this right now, but I can easily imagine this being something that happens in the future, is effectively building in certain patterns of behavior to the models from the earliest stages of pre-training that are very, very difficult to remove through post-training and are very difficult to test whether they exist.

17:48Just to kind of use a hypothetical, somewhat scary framing of this, think about Stuxnet, right? For those that aren't familiar, this was a virus, a computer virus that was purportedly developed by the Israeli and U.S. militaries that was designed to interfere with Iran's nuclear development. And this virus transmitted through USB primarily. And I think when it was reported, I think it was on something like 60 or 70 percent of all devices in the world, period. It proliferated everywhere, but it was a no-op almost everywhere. and then in the one circumstance where it was loaded onto this particular model of centrifuge that was used for enriching uranium that Iran was using, it would then activate and do the secret thing where basically it would make the centrifuge run at a higher speed than it was supposed to, such that the failure rate of the centrifuge would be much higher than expected.

18:45A really, really interesting story if you're not familiar with it and excited about it. But the attribute of that that is very scary is that it's prevalent literally everywhere. But it only takes the malicious action in a very restricted set of circumstances. It is not hard to imagine China doing something like that, where if the model is deployed in a particular circumstance, it then starts to bias its results. And you can imagine this from really big things, like maybe this is deployed in actually some sort of mission-critical operation where it's going to give the wrong answer at a critical time.

19:21Or you could potentially imagine this even in a more subtle way where, hey, say that this ends up being used for ranking as part of a social media platform or something like that. Say Snap were to start using that, and then the model would have biased more towards authoritarian values or things like that. So I think there is some real risk there. Now, I'll note, I don't think we have any evidence this is happening at the moment, but I don't think there is any technological reason why this couldn't happen. This is, I think, a very, very realistic thing. And I think that the problem with this is that if you don't know how the model is baked, it's very hard to, I can't see how you would say, I guarantee that this model doesn't have any of those problems.

20:03So I think that is one real risk. And I don't think it's hard to see that the incentives for the CCP are certainly pointed in the direction of doing something like that, right? We already have evidence that this was the case in TikTok's ranking algorithms, right? And things like that. So that does seem like a real risk to me. And why I don't think that matters for 99 percent of deployment cases, there are some where you can imagine that being impactful. It's so interesting. It's obviously then, you know, there's been this government backlash of like, hey, well, you know, should we do something about these models?

20:34Or is it really fair that like, you know, they're basically, you know, the Chinese models are distilling a ton of American models. And then in turn, the American open source players go and try and distill the Chinese models. And it's like this round tripping round. And so I wonder where we'll end up from a government policy perspective. You know, it seems like people have been flirting with the idea of a ban. And that's what got the whole folks galvanized against that. And I don't know, do either you have a gut instinct on where this all lands? Hard to predict whether the government will make a wise decision.

21:00I think it would be a very bad thing for us to cut off access to open source models. I think it would be a pretty big gift to the frontier model companies, frankly. And I think it would be a good example of why Jensen, for example, has been so adamant about trying to build a non-Chinese open model ecosystem. And I think if we don't see that change, I think that could lead to more challenges for folks. That said, I will also note that, especially for niche use cases, it is increasingly viable to build from scratch with the right data. You know, it's still challenging. And the right partner to do that with.

21:38And the right partner to do that with, such as Datology. But, you know, there is still a real path towards that. And I do expect that's where people will go long term because they'll start to want to cut inference costs. But at the moment, I think where most enterprises are now really focused on how can they own their intelligence and are focusing on doing that through customization of existing models, I think it would be a really harmful thing to the ecosystem and AI development overall if we were to do something like that. I mean, to this point, a lot of people seem to be thinking now about the long-term implications of being tied to a frontier model or being tied to a Chinese open source model.

22:15And I think, Rob, you mentioned earlier, and it's felt very true the last few months since you recorded, there's been this overall backlash against Anthropik and the large labs. And I think maybe it was best encapsulated by the Palantir CEO, Alex Karp, who went on this rant on CNBC that went pretty viral. And I think there were many points he was making. But one of the things he was saying, which I think increasingly a lot of folks have felt as the Frontier Labs move more and more into different application areas, is these labs, they're partnering with you. They're kind of getting their models into your ecosystem, learning your business, and then they learn your business.

22:47And then suddenly you can't compete in all these different ways. And I'm curious for both of you, how fair a critique is that? What are and aren't the model companies actually learning through these deployments? So clearly, Palantir has their own incentives to pitch a different story where you need to own all this stuff internally because why would you give any sort of valuable insight into the labs themselves? I'm sure, as always, the truth is somewhere in the middle of that spectrum. So maybe we'll start with you, Ari. Like, how do you articulate, like, what is a business actually giving up to the labs if they, you know, deploy anthropic models all over and put them in Slack and all that good stuff?

23:23Yeah, I mean, I think the real challenge comes when you start handing over data and expertise, right? And I think the reality is that both of the major frontier labs have been, have said the quiet part out loud pretty reliably that they intend to, you know, obviate many of their customers' business and take it over, right? So it's not even a case where they're pretending not to. And the primary challenge to get those specialized capabilities is going to be the domain expertise that exists in many of these legacy companies and the data that is representative that exists in many of these companies.

23:53So if you hand those two things over, then I expect what you will see rolled out is that eventually their unwrapped version by default is comparably good to your wrapped version. But is your average business giving away domain expertise and data to the labs? Is that really how these contracts are set up? If you're working with them to customize models for you, then you are starting to hand over the data, I think. And then I think more generally, if you do build on top of them and are completely dependent upon them, the risk you run is that, you know, as they improve, eventually your differentiation becomes smaller and smaller.

24:29And you have to pay a much higher margin. So their business just has simple, better unit economics. I mean, this is the story we saw with Cursor and, you know, now with Harvey and with many others where kind of a lot of these companies that are built on top of this are realizing that it's a really huge risk for them if they are entirely built on one of the frontier models because, you know, they're competing with their hands tied behind their back just in the same way that, you know, Spotify competing against Apple Music is going to be fundamentally limited because they don't, you know, Apple doesn't have to pay the Apple tax and Spotify does.

24:57What does Anthropoc really see if it's deployed at J.P. Morgan of how people are using the model or things that actually make it that much better? And I think sometimes, I don't know, I feel like there's been this narrative of every business is so unique and has such special data that no one could possibly replicate. And some of that is true, but also it turns out if you go hire a bunch of ex-bankers, you will figure out how to do basic banking tasks. The data providers are willingly able to provide a lot of what exists in a work environment to the labs directly. And so, I don't know. I felt like there's been such a blowback against the labs themselves.

25:33And this idea that every time you use the Claude API, you're just bleeding your business value into them directly, I felt like was maybe a little overstated. I am inclined to agree. And I do think like the overriding motivation for creating this counter narrative is like ulterior motives from other groups. I mean, all this stuff, everyone, the letters, all this stuff, even if you agree with it, it's like everybody clearly has their incentive. Yeah, exactly. Yeah. Yeah, to your point, obviously Palantir is in a position where it benefits them tremendously to have a world where enterprises are not going to open AI and Anthropic to get their enterprise AI solutions, but are instead relying on Palantir.

26:14And even NVIDIA, Jensen is hugely strategically focused on promoting an open-weight environment. And it's because I think he's done the calculus that that is much better for NVIDIA in the long run. And I think it's a similar logic to… commoditize your compliments, right? Like just have a bunch of, nothing will be better for him. Exactly, yeah. And it's very similar dynamic to how NVIDIA has propped up all these neoclouds and therefore like decentralize the power and kind of reduce the influence that AWS and Azure and GCP have. And I mean, I think it makes a ton of business sense in that it's easy to come up with the kind of like ethical or philosophical justification for it after the fact.

26:56I think it makes it hard to actually disentangle like what is true. Because there's kernels of truth and like all of these things. But they're obviously taken to an extreme. I think people definitely, you know, end up being hyperbolic with this. Like I think the version that you portrayed, Jacob, of just like every time you use the API, you're giving away business information. That's like kind of what he was saying. That's, I think, going a bit too far. That said, I think like there are folks that are talking about partnering much more deeply with OpenAI Anthropic to do this. I think that is much more of a risk when you go and do that.

27:26I think the other thing that's, you know, just realistic is there's that feedback loop of, hey, the model is poor at this set of queries, where, you know, by using the API and then working closely with them, what you are effectively doing is giving them distribution to your exact set of customers. What you're giving them is like, is evals, right? Like, they're actually valuable. You're giving them evals to help them to improve it. Exactly, and those are hard to access. So what you can imagine is that by using the API reliably and by giving feedback reliably to Anthropic about where the model's working and where it's not working with OpenAI or whatever, you are effectively hastening their development of their competitive product to you.

28:02That, I think, is a very realistic one. Yeah, and it's interesting because in the early days of application development, I felt like every cutting-edge app was just working so closely with the labs of like, here's all our evals that don't work and please make them better. And it feels now more and more like you'd kind of expect them to keep it a little internal and maybe fine-tune an open-source model and address that eval that way. I think there's also just a broader point of businesses don't like extreme vendor dependence, right? Like that's a risk that every business has faced forever. And like it's not good for the world if everyone is dependent upon this small set of companies, I think, fundamentally.

28:41At least that's more – starts getting more to a moral belief to some extent. But it is one that I believe strongly that this is too powerful a technology to have it be concentrated in a small set of actors. and I think realistically a small set of actors that have shown that they are generally putting their interests first, which is fine. That's what I would expect of most anybody. Everybody acts of their incentives. But that means that you want to incentivize the ecosystem to have much more control, to have much more diversity of players, to not have this centralization, especially for technology that is even a tenth as impactful as AI.

29:22So I've obviously been hyper-focused on us not becoming a cliche of a podcast. And one of the most important things to avoid that is to avoid talking about politics at all. That being said, it does feel like a lot of government stuff has really dominated the headlines in the last few months. I actually found – we recorded our last podcast and we were talking all about Fable and them banning it for different uses. And then we released it 48 hours later and like five hours after we released, the government just banned Fable. And so I hope we get a mulligan of talking about politics, but it's pretty relevant to just AI world.

29:54We won't go into the 28 election. But maybe just on the fable side, what do you make of it? That was quite a process. They obviously banned or tried to ban all foreign nationals from using it and basically cut off access. And then, you know, we pretty soon after had open source models that were, you know, pretty close to parity. And, you know, it all kind of feels like a footnote in how it went. But maybe a dress rehearsal for more of this stuff down the line. What do you two make of it? So, Jacob, you sure you don't want us to talk about J.D. Vance versus Marco Rubio for the Republican nomination?

30:29I think we should do a separate episode. Just like the three of us, as little qualified, as little prerequisite as possible, just opining very confidently about, you know, that matchup. Yeah, you're right. It is a real virtue of this podcast that we have not yet talked about politics at all. But you're right. I mean, I think it's becoming impossible to ignore if you're serious about the world of AI because it is such an important dimension to it. I'm happy to share a few thoughts on the fable thing. I mean, part of it is like, I do think you can't ignore the dimension, which is like idiosyncratic to it, which is that the Trump administration really doesn't like Anthropic.

31:06And Anthropic has done itself no favors in terms of how it's positioned itself with the Trump administration and with the public. they've been essentially like begging to be regulated, you know, and kind of like put there at the forefront of making this argument that AI is so powerful, the government needs to be involved. And then the way they handled, like when there was this security breach or concern raised by Amazon, the way that they handled it, it sounds like was very dismissive and flippant. It's like, oh no, that's not a big deal. Don't worry about it. And so they just, I just think they haven't navigated, like they haven't taken seriously enough their relationship with what is a very serious partner to be thinking about, which is the administration, the White House and the Trump administration.

31:53So I do think that was part of it. It also like it, it always felt like the kind of thing that was going to be temporary and unwound because it's just not, it's, it's, it's untenable and a little scary for the US government to be that heavy handed in terms of how, you know, arbitrarily deciding what models can and can't be released in an ad hoc basis. So it always felt like something that was going to be lifted and it was lifted. But I do think it as, you know, to use the phrase you said as a dress rehearsal, Jacob, like I do think, you know, as these models get more and more powerful to this point around, like what happens with open weight models, I think I could envision a world where in a more systematic way, there is some, some version of a licensing regime.

32:42And this is like started to be discussed. This has started to bubble up more and more as a policy concept, but like for true frontier class models, like, do you need to jump through certain hurt, jump through certain hoops and get certified by the U S government prior to being able to make the model publicly available that like, obviously we'll, uh, get the, get, get the backs up of many libertarian minded folks. But I do think that's one possible policy direction we may be headed in. And if so, what played out with Fable could be sort of like a harbinger of that. I mean, if open source is truly three to four months behind and continues to be like, how in the world do you do that?

33:21How would you have a process that was fast enough? Yeah. So I think if we go down that path, and I'm not saying that's necessarily the most likely path, but that is one path people are talking about. I do think it handicaps open-weight models. Like I think there will still be open weight models, but they will be more lagging edge. Or they will be from unauthorized parties like out of China. And then, you know, there may be all sorts of other complications with using them. But I think it would be hard for like a US-based company that is in good standing with the US government to release like true frontier open weight models.

33:58And so like that sort of policy would really favor opening an anthropic for sure. I actually think that policy would also really favor non-America. Yeah, 100%. Because then what you're going to have happen, right, is that China's not going to have any restriction. And then a bunch of non-US players are going to use more powerful models that US players do not have access to. And it's going to actually explicitly put America at a meaningful disadvantage. I am very concerned about such an outcome because I do think it is probably the modal outcome at this point, given what everyone is saying. And it seems like a pretty bad outcome.

34:31I think fundamentally, there's a certain amount of like, you can't close Pandora's box that's happening here, right? Like, this is not nearly as hard as people make it out to be. It's not easy to build one of these models. Don't get me wrong. Like, I wouldn't say it's fully commoditized. But if you get enough smart people in the room with, you know, access to the literature and with access to these models, which can also help you, of course, build better models. And then like, the compute constraints aren't going to be massive, you know, especially now that China's building their own chips, and you can see Kemi's starting to support more and more of that, you know, to start building on Huawei.

35:06I don't see how you put it back in the box. And I think trying to limit by regulating or delaying this, it just, it only seems like it can ever do anything for a month or two. And then fundamentally after that capability, after that point, you're now going to have a plethora of models with whatever that capability is. So like each time we have a new capability leap, there's going to be some challenge to catch up. But then that just becomes a capability where, yeah, models can do that in the world now. And that's realistic. And like, I think trying to operate in a world where you say, okay, cybersecurity is a huge problem.

35:42So we want to limit access to hacking models just feels very short-sighted versus what we know is the right solution, which is you give everybody access to those models so that they can defend against them as well. I think that's just like the only practical solution. Everything else kind of feels like wishful thinking to me. So would you do anything on the policy side? I think it's a challenge of what you do here. I think you might want to say, okay, if there's a certain capability leap, there needs to be a measured rollout or something like that, which is actually kind of what Anthropic did with Mythos, which I think was pretty responsible with respect to that.

36:17So I think that could make sense. But I think it's fundamentally a pretty hard problem to solve beyond, I would, I mean, honestly, the best thing to do here, I think, would be to spend a tremendous amount of government funding to have, you know, open model efforts that are going to be guaranteed so that we, you know, you try to fight this by fire with fire to some extent by accelerating as much as we can to catch up to that so that you don't have these discrepancies, which are very concerning. Maybe just to play devil's advocate here to tease this out a bit and i'm jake i'm curious here how you think about this as well are there are there any capabilities are or do you think future frontier models could have capabilities that are that would be problematic to be widely available to anyone who can download the weights such that there could actually be logic to trying to limit the number of parties that have the capabilities and i know it's like a controversial and much more maligned analogy to nuclear weapons versus AI.

37:16But like, you know, I don't like I think most people would say it wouldn't be it's not better for the world if like everyone and anyone could have access to the recipe and uranium to create a nuclear weapon. And it's better that it's limited. Is the difference with AI just that it's software and software can endlessly replicate and therefore there's no possibility of holding it in. And so let's just like adapt to the world where it exists more broadly? Yeah, it's a good question. I think yes is my thing. Like I think to your question of what's different about nuclear, the recipe for building a nuclear bomb is super, it's straightforward.

37:50Like it's available, right? Readily available. The challenge is getting access and the challenge isn't even necessarily getting access to the raw materials, in this case, the uranium, right? It's getting access to the particular machines you need in order to, you know, get the uranium to a certain level where, it is usable for fission. You can restrict it on that capacity. And I think that's why we first started going to chips, because we said, okay, chips are the equivalent for AI models. But realistically, they're not. Especially there are so many chips that already exist that you can use those, right?

38:24So I don't know if there is a realistic way to stop it. I mean, to your question from the beginning, what is a capability? I think zero-day vulnerabilities is a pretty good one. And like you have something that can find zero days in everything, you know, particularly in encryption strategies, Anthropic released that in a weaker form of encryption, right? They were able to crack it, certainly. Like that would be massively disruptive to the entire world if that were to happen. I just don't know how you actually prevent it. It seems like the best case scenario is you delay it by a couple months. But then the capability comes out and like all you're going to do is you're going to make it more likely that that capability is first going to come out from a Chinese lab or from a place where, you know, it would go in the hands of bad actors.

39:09So I just don't actually see a way to contain it realistically. Like, look, it's already super hard to contain nuclear work, right? Like we've tried quite hard. The entire Western world, you know, tried to prevent North Korea from getting access to the bomb and they got it. Like, how do you actually prevent this in the AI space? I just don't see a practical path, in which case I think you kind of have to go the other way because otherwise this in-between strategy just ends up disadvantaging America. Yeah, I mean, you certainly need some sort of like global governance to enforce any of this stuff, right, and enforce some standard set of like guardrails that were put in the models or testing that were done on the models around access to things.

39:48Or maybe you just mitigate, you know, I mean, obviously, I guess cyber occurs fully in the digital world. a lot of things people are concerned about has ties to the physical world. And so maybe you, you like really start regulating, you know, the ordering of, of, of chemicals or things like that, that, you know, that, you know, because, you know, not, not all things that could be harmful have as high barriers to building as, as nuclear does, certainly. Right. Like look at, look at, like, like trying to buy Sudafed in the U.S., right? Like because it's the most readily available precursor for meth, like you now have to show your ID every time you buy Sudafed.

40:19Maybe that's the route we go down, but like, obviously that hasn't stopped meth production. Fair. Hopefully GPT-7 can just give us a good list of all the things we should clamp down on. But I think we have to prepare for the fact that we are going to live in a world very soon, if we don't already, where this incredible capability is accessible to anybody. And we need to figure out how do we live in that world and how do we secure that world, rather than kind of putting our head in the sand and hoping we can prevent that world from coming to be. Yeah. Well, that's a good transition to, you know, obviously we had the OpenAI hugging face incident.

40:55And like, you know, I'm sure our listeners are familiar, but for quick context, OpenAI had a combination of its latest model and an unreleased one, escape a sandbox evaluation environment it was in, reach the open internet, use some stolen credentials, find this zero-day vulnerability, and get into hugging faces systems. And, you know, it's obviously been something that people have been talking about a lot. I'm curious for both of you, as we look back on the history of this era a few years from now, is this going to be, it obviously feels like a big deal now. Do you think this will persist as being seen as a big deal or is it just like yet another one of these crazy capabilities AI has and will kind of blend into the background?

41:35I think it's a genuinely big deal. I think it is a landmark event that we will remember for a couple reasons. One is it may not have been the first time this has happened, but it is the first time that this story was publicized in a very easy to understand, crystallized form for the public. And so I do think it was like a wake up call for the broader population. My mom actually texted me. I'm going to read her text. Mom, I hope you're famous now. You're on unsupervised learning. Well, she's a regular listener, I assume, right? My mom knows nothing about AI, knows nothing, is not plugged into the tech world.

42:13She texted me last week, Rob, what do you think about OpenAI's agent hacking into hugging face? And I was like, I can't believe you know what any of those words mean. But the point is, I think it did. It struck a chord with the broader population. And I think relatedly, it was one of the first examples of AI, rather than AI being a tool that human bad actors can use to wreak havoc in the cybersecurity world. It was an example of like an autonomous AI being the problem. So I do think, yeah, I think it was like, I think it will trigger a lot of downstream policy actions and commercial actions. And I will also say like, I think OpenAI was very lucky that the victim, if you will, of the attack was Hugging Face, which is an organization that's very like AI forward and was willing to collaborate with them in a very open way.

43:07I also think OpenAI handled it very well. like they were admirably transparent about it. But you can imagine if it happened to be like, I don't know, like General Electric or Visa or like some non-AI native company that it could have been much worse for open AI. So yeah, it was a very interesting incident. I like this eval. My dad emailed me too about it. And I feel like the eval of like what meets the bar of a parent who's not super deep in the space sending contact to you in some way is a good eval for what will stand the test of time. Yeah. I mean, it definitely is one of many things that feels like sci-fi come to life.

43:41Right. I think there's a real thing here. I do think this is a really big deal, whether or not. I mean, look, like opening the model, they were testing models with fewer safeguards and like a less secure sandbox than they would in other contexts. There's a question of how wise that is to do with unreleased, more powerful models. And I think that's kind of odd that they did that. But I would definitely say that I think their transparency throughout this was quite good. But, you know, I think this is exactly what you expect paperclip maximizers to do. And I think it's particularly funny that in this case, the model was hacking into evals so it could train on the test set effectively.

44:23like it went to hugging face to go and try to find the evals so it could cheat. I think that is both quite hilarious and extremely close to the paperclip maximizing story that goes to show how, you know, models will find a way to minimize their objective, however that may be. I am kind of surprised that there wasn't some like very clear instruction repeated many times in the prompt not to try to hack the test set because models do that quickly. In fact, Post Train Bench, which has been one of these benchmarks that people are using to automate post training, just had an update that came out today around specifically preventing models from doing that.

45:03So I'm a little surprised that that wasn't in the prompt, or maybe it ignored it. But I do think it's a real thing. I agree with Rob very much that they're lucky that it hacked Hugging Face and not some other more critical piece of infrastructure. But going back Back to the open model, you know, point, Hugging Face was only able to detect and kind of manage it as quickly as they could because they had access to GLM 5.2. And, you know, Clem even pointed out explicitly that the asymmetry between the attacker and the defender was a very meaningful gap, especially given that, you know, GLM 5.2 is not as good as 5.6 Sol and whatever the unreleased model they were using in concert with it is.

45:48So I think this is actually a great example of why open models of that capability are important. It was a critical part of the defense toolkit. And I don't think it's reasonable to expect humans to be able to respond to frontier models attacks. You're going to need to have defender models there as well. And if you do this kind of tiered release where, okay, you need to be of a certain level of accomplishment before you get access to those unreleased models, of course, like the bigger you make that group, the harder it is to actually keep it limited. And overall, I think actually it goes to show really how important it is to have extremely capable, you know, open models that can be used for defense.

46:33Which is kind of crazy to think that, you know, for the conversation we had earlier, it's like, you know, we'll have open source models with these capabilities in three, four months, right? And like, you know, obviously there's only, yeah, there's only so much testing you can do around this stuff, right, before it's like widely available and accessible. I think people also like don't understand the distinction between the model and the harness nearly enough. Like a lot of the improvements that we're seeing are actually harness improvements more than model improvements. And in fact, even at the time that Mythos was released or was announced, not released, right, like in February, March timeframe, there were already a bunch of people that just put better harnesses around open models and were able to discover many of the same vulnerabilities that Mythos had discovered.

47:18right so it's also this it's not just the model it's the harness that's improving in a lot of these in a lot of these cases and certainly we don't have nearly as much attention going to harnesses and models but like no matter what we're going to be in a by the end of the year there are going to be many open models that have these capabilities that everybody in the world including all of the black hats will have access to pretending that we can prevent that world is not realistic and is going to lead to very stupid decision making. Like we need to like take bite the bullet and say, OK, this is the world now.

47:52These models exist. How do we live and operate in a secure way in that world with these vulnerabilities? It feels like kind of crazy to transition from, you know, like the these big existential questions about the future of the world with AI to like the much more mundane business questions of like who's going to capture value and how the labs are all competing. but we must give the people what they want, and it can't purely be that. And so I guess I'll move us on to just the other developments in the broader AI space since we last talked a few months ago. One of the most interesting ones, I think, for that I'd love both your take on, is Grok.

48:27I feel like the experiment that's happening right now with Grok and Cursor is so fascinating for the implications that it has for what's actually valuable and important going forward for models. Obviously, the two came together. We had Grok 4.5 released. honestly does pretty well on like a lot of a lot of benchmarks and and folks have had a lot of good things to say about it you know i think a big part of the excitement around these future rock models is you know the training on cursors data right and the you know just the massive amounts of traces they have and then like you know real world experience of people using their models um i think this question is gonna be like how valuable is this and like how important is this for in a world where lots of people can train similar models um ari kind of care start with you like What's your gut instinct on how much this is going to improve the Grok models and where it takes them?

49:13I think it's very meaningful. It's a very real advantage. If you think about just the edit-by-edit information you get from the model put an output, how did the developer edit that output, how many times did they reject its output and ask for modifications before getting the eventual thing that they liked? that sort of information is just incredibly valuable and can absolutely be used to really improve the model and what's great about it right is it's on policy information it's information about how the model's currently performing so it's not just an offline data set that you get and then you apply right like preference pairs you might go collect from record or something like that it's actually from people using your model and getting the data on that model that's incredibly valuable data and is very hard to get.

50:07And I think it makes a ton of sense why Cursor, you know, would want to use that and why, you know, SpaceX decided to acquire them for that. So does it become much more valuable once it's actually the Grok model within Cursor? Because obviously a lot of the data they had previously was other people's models. There weren't a ton of people using Grok within Cursor. But now... The more on policy it is, the better. But like, Like, just to be clear, there's a tremendous amount of value from off-policy traces as well. They're just less valuable than the on-policy traces. Yeah. And then how do we think about, like, the quantum of data?

50:39Because obviously it seems like, you know, one big question that I think will determine a lot is, hey, do you have to be, like, the number one, you know, external coding application like Cursor to get enough data that matters? Or do you just have to have, like, some amount of traffic? In which case, like most major model providers can probably find a way to get enough traffic to like improve their underlying models. So I guess it's like order of magnitude more of data that cursor generates. Do you have a gut instinct like how much more helpful that is than like, you know, I don't know, the number seven most popular coding tool?

51:09So here's what I would say. At a fixed quality, more data is always better. At a fixed quality. That is a very important caveat, right? Getting more data and having cheap access to compute makes it harder to mess up the model because you are definitely going to show some of the really good data that's in there. Right. I would expect the cursors, the data they're collecting has the same kind of power law, long tail effect that we see in all other data sources where there is a comparatively small fraction of the data that actually drives most of the improvement. For all sorts of what makes a piece of coding data like particularly good?

51:55It'll depend fundamentally on where the model is. But you want to find data that is going to teach the model something it doesn't know. right so like the technical term we always care about is we want to maximize the marginal information gain so how much extra information does the model learn from each successive data point the vast majority of cursor data is like going to be stuff the model already knows and then they'll be exactly yeah i guess you think about the self-driving context where it's like most of driving is like pretty obvious to to the models now and then there's going to be some uh some subset of data that like really really matters yeah i think the self-driving context actually is a really easy like an analogy for it like consider that like the data set the tesla would be collecting, right?

52:31Almost all of that is highways. Like self-driving cars are good at highways. They've been good at highways for a decade, right? The hard part is construction zones and, you know, going through SF when, you know, like some separate lanes are closed and there's a person, you know, with a baby carrier that's looking at their phone and may or may not walk into the street, right? It's that long tail of data. So the more data you collect, the more of that long tail data you have access to, and then the more of it you can use. So there is a really meaningful advantage. At the same time, I think we have consistently seen, and I would argue that this is kind of our whole mission at Datology certainly, is that if you are clever about these things, you can do a tremendous amount more with less.

53:10So I think that, you know, all things being equal, if Cursor were to apply the same amount of effort to kind of really enriching the high quality data as a smaller player would, they will always have a fixed advantage because they will have more data at any given quality level than a smaller player. That said, I think what we have seen consistently is that bigger players end up using compute as something of a coverall to some of these issues. And they then invest less in these areas because they're less critical and it's easier to just say, okay, we'll just like throw more compute at it. And that leads to this.

53:42So if you're the number seven player, I think there's still absolutely room to catch up. But the barrier to entry from a kind of technical standpoint is now higher. You have to be more clever to make up for that. And like circling back to the Chinese labs, I think what we've consistently seen is that in many ways, they have been extremely clever. And that's how they've been able to overcome the disadvantages that they have. So I don't think it completely runs it away, but this is definitely a rich get richer sort of process where you would expect that having more data makes it more likely that you're going to have the best coding application, which makes it more likely that users use you and you get more data and so on and so forth.

54:19Yeah. Yeah, I mean, we're just going to see how much better it makes the models. I think it has massive implications for the general application space and how the model companies play in it. Where today, most of the model improvements are probably, or a lot of them are driven by just buying Mercore or Scale or Surge data. And the relative value of that versus the data you get out in the real world from folks using the product. If it turns out, God, that other data, there's only so much you can do on predisposing what people might do or buying a bunch of RL environments or labels. And it turns out it's way more valuable to have that data in the real world.

54:48you may see labs get even more aggressive on offering their models directly to customers and or acquiring companies in spaces similar to what was done with Cursor to improve those models further. And so I feel like this experiment of these Grok models will have a lot of implications for how the broader space unfolds. Yeah, I mean, going back to kind of my prediction from before about the APIs potentially being deprecated, or I don't think long-term deprecated, but being deprioritized relative to first-party offerings, I think part of that is because the first-party offerings have higher margin.

55:20But I think a bigger part of that is that the first-party offerings have all the contextual information about the way the models are being used that you don't get in the same way if you're just being accessed through the API. If cursor accesses Claude through the API, Claude Anthropic doesn't get all the context of what the user was doing and exactly how it was editing things and so on and so forth. It just gets the queries every time Cursor decides to call it versus getting all of that information fully featured in the context of Cloud Code or Codex or something. Rob, where do you think Grok's going to end up on coding after this experiment has been played out a little further?

55:57It's hard to say. I mean, I think Ari makes a lot of very interesting points around the value of the data. I still am, I remain fundamentally skeptical that Grok will be a model that stays on the frontier in coding or in other use cases. Just because I think it's so hard to keep up with the frontier. And I think the signals like SpaceX renting out much of their computing, Anthropic and Google, to me is a pretty definitive signal that they're choosing to monetize that infrastructure in other ways. So I still don't view Space X AI as a company that will stay on the frontier long term. It'll be interesting to see if that proves to be wrong.

56:44But Cursor as an application, I think, can continue to be a super powerful and lucrative property for the known. Well, we've got to hit a variety of other headlines before we wrap up here. And so I'll rapid fire through them. The first is a pod favorite topic of SSI. Finally, we had a big announcement, right? like a time to scale, you know, what,$5 billion from NVIDIA, a lot of compute. You know, still no exact inklings of what the breakthrough is, but, you know, certainly some excitement there. You know, either of you have some thoughts here? Big announcement, and yet nothing was actually said, surprisingly.

57:24I mean, yeah, if we really wanted to make this podcast go viral, we should have a big reveal about what SSI is working on. Because I honestly still have no idea, and I haven't talked to anyone that has any real idea. It'll be interesting to see. What I'm curious about is how much did they pull back the Kimono to NVIDIA and Jensen? And my guess would be not that much. Yeah, if you're Jensen, don't you just blindly give that amount of compute to you? Yeah, there's really no downside. Yeah, if he wants to, I mean, he's just going to go spend it on GPUs. So, yeah, it'll be interesting. And I'm sure, like, folks have heard the rampant rumors, which seem to be fairly substantiated that SSI is trading in public markets.

58:08And that's kind of like the use case that they're targeting. And it makes sense. And it's, you know, it's an example of being a very lucrative use case for frontier models where you don't need to expose what you're building to the outside world. Although that's why take the investment if they actually have a trading platform that's working so well. unless if it's about actually just getting access to the compute. No, it has to scale. They've seen early signs that it can work, but they'll have to scale to really the stuff. But yes, maybe if with this$5 billion in compute, it's the last round they have to raise from the outside.

58:42Yeah, all I'll say is SSI is the most secretive out of everybody. I certainly don't have any special insight into it. So who knows? But, you know, Elia is a pretty smart guy and has clearly been right many times in the past. and has been right when many others told him he was wrong many times in the past. It's truly so impressive. And like the incestuous gossip circle that is Silicon Valley that they have managed to keep this as under wraps as they have. I completely agree with that. And just to add to that, not only have they managed to keep it under wraps in this incestuous gossip circle, but they kept it under wraps even though their CEO left the company and went to meta, went to another hyperscaler.

59:24And somehow it still has it. It really is a game. That makes me think that whatever the key breakthrough came happened after Daniel Gross left. I have no idea, but that would be the most parsimonious explanation of those events, I think. Because it's also kind of was surprising, like, why would the CEO leave? You know, unless if there was some conflict or something like that, which who knows. But just purely in the face of it, it would make more sense if kind of whatever the key breakthrough was happened afterwards. You know, in other headlines, we have OpenRouter, which is rumored to be getting acquired by Stripe for$10 billion.

59:59Obviously, at the peak of folks talking about, you know, open source models and, you know, routing, you know, tasks to the right cost and capabilities tradeoff. What do you guys think of the acquisition? What do you have done if you're Stripe? Rob? I kind of like the strategic rationale in thinking for Stripe. I mean, you know, to this point, Stripe as a business has been so successful because they basically take a cut of all transactions on the Internet. They sort of serve as this toll booth. And an open router is a different form of that business model, right? Like they're taking a cut of all model calls and model usage across different models.

1:00:36And if you think of tokens as a, like, you know, a future form of currency, essentially, like I think it's a conceptual leap. but I think it makes sense for Stripe to move in this direction. So yeah, I think it's creative. I'll be curious to see if the transaction actually closes, but I think it makes sense for Stripe. It'll be interesting to see whether there ends up being any sort of differentiation or brand power to this space, right? It feels like everybody in the last week has announced they're building a model router. I'm not sure totally the amount of technology that goes in under the hood to make these things work.

1:01:12Yeah, it's hard for me to see the long-term moat. that's there. I mean, like presumably it's the routing function, um, like, uh, that, that is going to be the most critical thing there that's going to change constantly. So it's kind of hard to imagine a fully durable note there. The, the one question I would have about the strip, I like the way you framed it, Rob, of kind of like they were taking a percentage of the internet and now they'll take a percentage of all tokens. The thing I don't quite understand that I would usually look for in an acquisition, you know, is what's the nonlinear additive effect?

1:01:44Um, you know, what's the synergy necessarily between Stripe's business and OpenRouter's business, where combining them together makes something more powerful than either independently. I don't see that here. It feels like more it's just like a fully kind of independent business that you can make a nice analogy the way you drew for it. But I don't see how Stripe's existing product offering or infrastructure or talent or core competency makes OpenRouter more successful. um that i think would be like that that i think is is the gap you clearly going back to the conversation of xai and and uh cursor like you can clearly see at least for xai why buying cursor helps them to make their core offering much more effective um i don't see that for stripe and open router uh but like revenue diversification uh could be getting in the ai game right and getting in the ai game it's a foothold it enables you to kind of have a fairly low risk you know possibly high upside subject to the commoditization risk that you just raised, Jacob, bet there.

1:02:48That makes a reasonable amount of sense. And hey, if the ultimate eval for a business is how much money they make doing something, maybe Stripe holds the keys to the evals that actually matter for routing these models. I guess, you know, I do want to hit before the quickfire around just like the broader venture world and what's happening, given that we have two-thirds of this podcast in VCs. Rob, you know, I know you're always a big BCI guy, a big robotics guy. Where are you spending time these days? And what do you make of all of us now becoming, you know, genius neocloud public market investors and, you know, exclusively hardware focused?

1:03:20As you said, the prevailing sentiment in VC world right now certainly is to be leaned in on deep tech. And as a reaction to the fear that the frontier labs are eating the world of software and Anthropic and OpenAI are moving up the stack, as we've talked about, and no app category is safe. So deep tech is certainly fashionable. Definitely spending a lot of time in robotics. And I do think like we could have a whole other discussion on this topic, but I think, you know, both of us, Jacob, are invested in foundation model robotics companies. And I do think that in just the past few months, these models have really started to work in a powerful way.

1:04:02And so I think like we're on the cusp of something really exciting happening in robotics, I think, which is which is going to have years and years to run. So that's exciting. I'm spending more and more time kind of on the physical infrastructure underlying AI data centers, energy grid, et cetera. And there, you know, at the end of the day, I think there's like, there has literally never been a larger capital investment super cycle in the history of civilization than what's happening right now in the physical AI infrastructure build out, you know, it dwarfs the space race in the 20th century, dwarfs the Manhattan project, dwarfs the railroad build out.

1:04:39Obviously back in the day, you know, venture was all about like, you know, networking and a bunch of this kind of hardware. And then, you know, we had like, Like, you know, good SaaS run for many years. And now what's old is new again. And I think all the founders of Redpoint think it's hilarious that we're spending all this time, like now back in areas where we honestly need to call them with decent frequency. And, you know, it has come full circle with what's given the value its name. Yeah, I think it's great, honestly. I mean, I think it is venture capital at its most pure. It's like real technology risk and real technology differentiation.

1:05:14and so yeah i think i think it's a really exciting i just i think venture capitalists most pure was a company going one to five you know with like some good net revenue retention and just like they can hit 15 next year yeah that's pure that's pure um well uh you know i want to end our episode with just like checking in on some uh some predictions we had previously and so uh rob i'm gonna put you on the hot seat because ari's honestly i'm kind of on a hot streak i feel like he's you know it's open source models take looks pretty good um you know his take on on the models limiting access to their APIs.

1:05:44We'll see. But like definitely seems like could happen. I think my timing's off on that, but I'm pretty confident in that directionally. I think it'll happen in 27, though, probably not 26. But a little more on the hot seat. Like, like you've been very consistent with your Google take. And it just feels like things keep getting worse and worse for these guys. Are you still you're still holding holding out hope and being able? I gotta get Ari to start writing my Forbes prediction column for me Google, yeah it's a I would say long term yes, I continue to be bullish I just think that they have so many advantages at some point after we're all dead no, I mean we've talked about all the structural advantages they have but I agree with you I think the week after we released the last podcast where I, again, gave an impassioned defense of Google.

1:06:41Both John Jumper and Noam Shazir left. And they have such a deep bench of talent that that's not fatal for them. But people matter so much. People are the most valuable asset. And Noam went to OpenAI and John Jumper went to Anthropic. And that does hurt. And so do I think that they will be successful? Do I think that they will continue to be one of the very small handful of labs at the frontier? yes. But they have a lot of challenges related to being a big company. Like, I think it's fundamentally related to this being such a massive organization. There's so much bureaucracy and slow movingness and process and so forth.

1:07:18So they definitely have their issues. When are they getting back to the frontier? I don't know. I don't know. I think they are basically at the frontier. Like, I don't think of that as, I think, I think Anthropoc OpenAI and Google are basically at the frontier together, depending on what benchmarks you use. I don't know if you - There's a meaningful gap in coding. You're right, on coding. But yeah, I think of that more as like a product prioritization and focus thing than like the core intelligence lagging. So the thing that's confusing to me is, because I've also been fairly bullish on Gemini and on Google, is that it felt like they had this initial really kind of mess up with Bard prior to kind of rebranding to Gemini.

1:07:58They tried to rush something out and then they spun it up And then it seemed like they had the ship pointed in the right direction. And, you know, I think it was with like Gemini 2.5. That was the first one that started to look like pretty good. And like you followed the trajectory and it seemed like three was going to, you know, make a big impact and so on. And then they haven't, they've underperformed that trajectory pretty meaningfully over the last six months. And I think that's been especially noticeable given the extent to which, you know, opening I and Anthropic have accelerated, you know, dramatically over that time period.

1:08:31So I'm really I'm really puzzled. Like, it seems like the best answer is just like organizational inertia and politics and all the standard big company stuff. Like, I'm very curious to read the book whenever it comes out in two years of like the inside story of Gemini and what what happened there. It feels like we're holding two truths at once. I think what you say about, hey, with the right data and the recipes are kind of known and people can get pretty near the frontier and lots of folks are able to do that. We're holding that. And then at the same time, it's like Google has every advantage that they should have.

1:09:02I think coding is a top priority for them. I mean, I think Sergey himself is on the team working on it night and day. And yet they still can't seem to get anywhere close to where OpenA and Anthropoc are. I think there just has to be that they're getting in their own way in various ways. You know, I just think that's the only explanation. One thing I will say, though, here is that I think that people tend to overfocus on, like, not talent as a general concept, but as on particular famous people. Like Noam Shazir, John Jumper, of course, incredible scientists, you know, Alpha Fold would not exist without John and like that entire line of research, you know, there.

1:09:38But I think we fall way too prey as a society to the great man theory of science. And it's something that I just, you know, firmly believe does not exist. There are, you know, at this point, thousands of extremely talented researchers who can do an awful lot. And obviously, you know, there are these exceptional folks. But there are a lot of really good people at Google as well. I think the thing that those people are, I think, are also leaving because they're frustrated by the lack of pace and like they don't have the famous names. But you lose enough of those people and that starts to make a really measurable impact and really starts to dull forward looking prospects.

1:10:17And then, Rob, you know, we've got to do an opening eye check in your you're you're honestly, I feel like definitely that that take seems it was very spicy when first introduced that Sam would be replaced as CEO. You still feeling good about the the Brett Taylor theory? We'll see. We'll see. Um, I obviously Fiji officially stepped down, which I think had been coming for a while given her health issues and we'll see. I mean, like Sam Altman has an unparalleled ability to hold onto power. And so I don't think it will be easy to unseat him. I think it would be an opening at his best interest, frankly, like, and we talked about this last time.

1:10:53Like, I do think that having someone like Brett Taylor running the organization would be just such a massive vibe shift in terms of the company's trustworthiness and reliability and and you know overall reputation um and i've you know they're they're obviously brett is this is the chairman of the board like i've you know how much would opening i have to pay to acquire crf who knows but like i i rank it i would still rank it as non-zero it'll be a very interesting uh next several months well ari rob this has been a a ton of fun as always i appreciate you letting me constantly put you on the hot seat with prediction check-ins.

1:11:29And probably, I hope it doesn't disincentivize you from juicy ones at the end of this coming year, knowing that you'll have to talk about it every two months. But it has truly been a pleasure and appreciate you guys taking the time. Thanks for having us. Great to be here as always, and always a fun conversation. I'm Jacob Efron, and this has been Unsupervised Learning, a podcast where I get to talk to the smartest people in AI and ask them tons of questions about what's happening with models and what it means for businesses in the world. As I hope is clear, I have a ton of fun doing this. It's a nights and weekends project in addition to my day job as an investor at Redpoint.

1:12:00But our ability to get these incredible guests on really comes from folks like you subscribing to the podcast, sharing it with friends. It's really what ultimately makes this whole thing work. And so please consider doing that. And thank you so much for your support and listening. We'll see you next episode.