Anthropic, Glean & OpenRouter: How AI Moats Are Built with Deedy Das of Menlo Ventures

25:21And, you know, to commend OpenAI, like OpenAI was just focused on something else, which is, you know, they've launched the most incredible consumer product that we've seen since God knows when. And so, you know, they were probably not focused on enterprise until now again. Yeah. How do you re-underwrite the company internally as you invest? So, I mean, even since we talk about cloud code, right? It's like, I think that was like a pivotal moment in like the trajectory of Anthropic. What are the things that matter to you when you're like looking at a company like Anthropic? Like, does this market share number matter?

25:54Like, how do you evaluate both the opportunity and like, what are the numbers that you really care about versus like, sure, higher market share, but that's not what we cared about. I don't think the market share number is more critical to understanding the TAM. At that stage, to be very honest with you, at the stage that we invest in Anthropic now, the only things that would really move the needle on the decision is here's the revenue, here's the margin, and here's the trajectory, and here's the other markets we may be able to underwrite that they want to go into, that they may be early in or planning on going into.

26:30I think it's really difficult to underwrite on market share other than knowing what the potential cap of the TAM might look like. So the pie will also expand potentially. But other than that, I don't think it's a nice vanity metric more than anything else. Yeah, in your mind, is it kind of like, you know, people in crypto are always about the flippening of Ethereum and Bitcoin. Is that something that matters? Anthropic can go to 50 %? or is it OpenAI was only a 50 % in a moment in time, which was a new market? Like, yeah, I'm curious how you think about that. I don't want to color like the way anthropic probably or the way all of us think about this, but I just don't think it matters that much.

27:09In my view, I'm a very paranoid person with startups and companies and technology. And so in my view, I'm like, great, now let's make it last. Or like, great, but what's next? And so to me, it's like nice to have. It's really not. I mean, look, if we're investing in around right now, which is like north of 170 billion. Sure, it matters. Some of the numbers matter. But the future of the company is all the value is really in what we underwrite as the future. And the future means that I'm more concerned about what's happening next. What are the new models? How do you gain market share? What has to be done?

27:48What are the new products that are going to be built? I'm less concerned about like where it's at right now. in terms of market share. But that's just me. I don't want to speak for others. Yeah. I think the new models are really good. I mean, Opus 4.1, Sonnet 4.5, Haiku 4.5 all released in the last few months. And it's really interesting. I think OpenAI and Gemini are in this sort of price war a little bit with the Pareto frontier that I track in terms of like LMSYS versus the pricing. And Claude can still charge a premium, but still like have a lot of market share obviously and i think like that's just because they have a better model and like people just naturally gravitate to it especially for coding but also other things and um i just think like articulating what makes a model good is just very very difficult obviously this is benchmarks and evals and everyone has like okay today it's your turn to be best at sui bench and then like tomorrow is my turn uh but like it's it's really stupid Like we're just talking about like, you know, 0.12 differences in like Sweetbench.

28:50But I wonder, you know, if you're talking about like, okay, I am investing$13 billion in Anthropic for Series F to underwrite Cloud 5, right? What does it have to do? Like, what kind of conversation does that look like? I have no idea. I'm not saying that you know, but I'm just like. I would say that despite what you said about the premium, I think everything you said is true. I still do worry. I think cost is a concern for a lot of people. And so the period of frontier does still matter. I'm glad Anthropics is where it's at right now, but who knows where that changes. When it comes to Cloud 5 and thinking about the future, one thing I think about actually that's really nice is I think we can take for granted right now that furthering the intelligence of models and chat GPT, a consumer product, does not lead to more users or more retention.

29:43It only is really applicable to a thin slice of users who care about very smart type queries. And I would say maybe under 10 million. Maybe that's just a random estimate. But most of the 800 million users on ChatGPT are asking, how do I fix my dishwasher? How do I rephrase this email that I've sent to somebody? And that's done. We know how to kind of do that. So what's interesting there is now that means we're at a point in consumer where, maybe this is too early to say, but OpenAI has kind of won. How do you catch up to something where model quality is not going to be a differentiator? You already have the users, you already have the retention, you already have great product and people are paying.

30:21But the interesting thing about Anthropic is if you look at coding, that's probably never going to be the case. There's always an increasing frontier of how good you could be at a task like that. And we're nowhere close to that frontier. So it's more possible to underwrite the quality of the future models versus like an open AI where it wouldn't be as much of a revenue driver on their consumer business than as it would be for Anthropic. Yeah. Talking about coding, let's just like talk about it because I think like this is also a very fun discussion. One, there's like the, what are the margins of cloud code?

30:54Which there's some numbers I don't want you to get yourself in trouble. But then there's also like, how do you think about the cloud wrappers, right? And we've talked to Bolt and Lovable, but then also like, I'll put Cognition and Courser in there as well, right? Like, how do you think about this market of, Basically, there's a whole ecosystem of startups. They have all done really well, built on top of cloud. I think it's great. I mean, there's - Is it sustainable, is it? I don't see why not. I mean, I don't, I kind of will allude to the margin question, which is like, can Anthropoc continue to do this strategy?

31:29Which, you know, I'm not gonna comment on the margins, but like, if you are trying to build out a enterprise-friendly business, there's like two broad approaches, right? Like high customization and high price, which is usually less scalable. And then you have low customization, low price, which is very, very scalable. In a SaaS world, I guess it's a Slack Palantir continuum. And so this is kind of different, but generally Anthropic wants to play here where scale fast, keep it cheap, get everybody on it. If we trust that most people or a significant number of people will stay on Claude if they continue to build products on top of it, then I think that's a win for the ecosystem and it's a win for Anthropic.

32:13I don't see why they would care. I think the interesting thing, and again, I don't know what Anthropic's future plans are, but Ben Thompson obviously talks about this, is classic strategy, which is every time you own the means of production, you will end up getting into the markets that your users use at you for. And so the classic Amazon example, which is first you are at the market where people sell, You find all the places that you can sell things that are commodity at high volume, and then you start creating batteries and Amazon branded batteries. And then you push out a bunch of people who sell batteries.

32:48So that's a risk, I think, for those companies that use Claude heavily and rely on Claude to think about. But at this point of time, we're too early. I don't think Anthropic is anywhere near thinking about that because you're still very much competing with other models on that layer. Yeah, playing a different game. Yeah. It's interesting. Would you rather be an investor? This is basically model layer versus app layer. So far, model layer has won. And I think there was a kind of an app layer summer. And then now it's like very back to models again. I mean, I like the discussion. I like the discussion because I was at a dinner where somebody was talking about this kind of question.

33:26And I was thinking about it more just at that dinner. And maybe this is an ill-formed thought, so feel free to push back. Yeah, we're riffing, yeah. But when I think about like moats, it's a classic like VC startup banter. In my mind, I think the moat is what is the hardest to do in any part of the stack. And so when I think about people that like tend to dismiss, there's other aspects to it too, but people tend to dismiss like, oh, you know, the app layers will capture all the value. Well, if the app layer is easier to build, I think the model layer is harder and therefore will naturally capture all the value net of competition from other model providers.

34:09So said a different way, it is far easier for Anthropic to try to go into one of the spaces of the apps than an app to try to go into the space of Anthropic, which makes me feel like one is more defensible than the other, all else equal. So I think both can thrive and that's ideally what everybody wants. But yeah. I think very brutally as an investor and as a human with my own limited time on earth, if Anthropik can go from$4 billion to$183 billion in two years, then everything else is a waste of time. You know what I mean?

34:51You kind of do want to really get this right. You can't just be like, oh, everyone's great and sort of hedge your bets. Sometimes you have to go all in on the right thing and you spend a lot of time and effort identifying the right thing. And so, yeah, that's what I'm trying to do more of these days. I think the means of production thing is interesting because cloud code only makes sense to be built if it's the best thing. Because if cloud code is mid, they're better off promoting Devon and Cognition to sell more tokens. So I'm curious, I guess, the market gets more competitive. On one way, it's like, well, we don't want you to use Devon because Devon supports all the models.

35:31And so we end up losing some of the revenue. But I think there's, right now, cloud code is obviously the best way to use the cloud models. So it drives the most usage. But I'm curious in the future, there's going to be more pressure on like, hey, this product actually needs to be great to make sense for us, again, to invest our resources into building it. Yeah, so going from model lab to model lab plus product company, right? Which is what OpenAI has done. I would push back on, well, A, I don't think everyone would agree that CloudCode is the best way to use Cloud. I've heard multiple people, even in the last few months, say that I'm a cursor guy, or I'm a devin guy.

36:08Because people have their preferences, so I don't think it's set in stone. However, CloudCode is a great way to use Cloud also. And there are nice flywheel effects, obviously, because once you capture the way people are using CloudCode, you also get so much data to then make CloudCode better over time. So I think those are the two main reasons. But at this point of time, maybe this is oversimplifying, but I can't think of too many apps that have a very meaty layer on top of the model that's very impressive yet. There are somewhat meaty layers, and it's getting there. It's a time thing as well, right?

36:46Most of these companies haven't existed for more than two years. So I think it gets there. But I don't think we're at a point where, you know, we're like, holy shit, that app has so much stuff, interesting things and technology built on top of the model, where it becomes so difficult for the model company to go and try to compete. I think tomorrow, if Anthropic decided to, or OpenAI decided to take on another app, given their distribution and their engineering and the fact that these are still not as thick as you'd like them to be, technically, they could. Whether they should or not is different, but they could.

37:19And that's something I do think about. Thank you for engaging in all this very meaty discussions. Yeah, you don't even work at Anthropic, so I know we put you on the spot. Yeah, no, but like, this is what I want to get on the podcast because like a lot of people don't get the chance to like talk about this, but then this is like a normal SF dinner. The last hit on Anthropic, I'll point out, which is more fun, which is there was a new CTO joining Anthropic from Pesit. And, you know, you're like the king of Indian posting. What's the significance of this for you? You know, last time you were on the podcast, you talked a lot about like the Indian, the university system and all that.

37:52And I need to see this guy rise up. in India, largely academics holds the same sort of prominence as sport would hold in America. Everyone talks about it. It's Asian culture, right? Everyone talks about it. It is top of everybody's mind. It is something a lot of people want to be good at. And it's extremely competitive society with a very large population. The way, and everyone on average, people are quite poor. So education is seen as the means to social mobility by a large amount of people in India. The way it works is similar to countries like China or some other countries where you take a big exam, you get ranked.

38:30A million people take the core engineering exam and the top 10 ,000 get in and the top 200 get into computer science. That's how hard it is. That's pretty hard. And those top 10 ,000 get into IIT. Everyone's heard of that. That's like where a lot of the great Silicon Valley people from Sundar to many other people come from, from IIT. And in India, often what I've seen, and this is something that I'm generally very curious about, is like, what is the motivation of humans and what is the dictator of outcomes in their life and their career? And one thing I've noticed a lot is, A, there are some societies that are inherently, I think, less meritocratic, where you get so judged for what you have in the past that you're not allowed to prosper later.

39:11And I think largely many work environments in India and other places in Asia can be like that, number one. So you're not judging on the merits of your work. You're judging on the merits of what you've done. And number two, there's a very strong self-fulfilling prophecy effect of I've seen people who underrate themselves because they think they couldn't be number one at something. It's like your own mental. It's your own mental block where like I couldn't get into like, I don't know. You know, people in the Bay Area also like this. The Bay Area is kind of like Asia. In the Bay Area, I know people who grew up who were like, I couldn't get into a good college.

39:45Therefore, I am stupid. And therefore, I should not work that hard. Right? Like, it's inherent that they could be smart. They just believe they're not. And that also has an effect, psychological effect on your long term prospects. You look at a guy like Rahul Patel, who's become the CTO of Anthropic, and he's not from a top university in India. Some people obviously debate that. In general, I don't think it's a really well-known university in India. And he's come to a society that is quite meritocratic. And he sort of worked his way up to a position of such prominence. I don't know him. I don't know what everything else he's done.

40:19But it's testament to the fact that, you know, I think this is why it resonated with so many people is even though you didn't have the opportunities early and even though you might not believe you could do it, If you work hard enough in certain environments for a long time on things you care about, anything can happen. And I think that's why I wanted to share it. I thought it was - And you choose to work at Stripe and Glean and do well. I think choosing the right company is also a very - Like, okay, if you're not going to do the credentials path, you have to be lucky and selective and working at good places.

40:54And a lot of people make that mistake. And I definitely did. I had good credentials and I worked at bad places. Yeah, it's very interesting that kind of... You work in a pretty good place right now. Yeah, but I took a long time to get there. I mean, just, you know, this is funny. I have this like automated pockets research and when it sent me the email about you, it's like, you know, DDS has a strong presence in AI and immigration for the top two topics that it talked about. Yeah, let's talk about the Anthology Fund. So it's a$100 million fund in close partnership with Anthropic. Like, talk a bit about that.

41:26I think people are really curious about how close that actually is. Yeah, so the Anthology fund we set up when we invested in Anthropic around the beginning of last year. And the sort of idea was, okay, Anthropic, again, it's so hard to think about. Anthropic was a very different company back then. It was a much smaller company. And they were like, look, there's incentive for us to run our own fund. OpenAI runs their own fund. There's a developer ecosystem that we want to create around this. It's really nice to have great startups that are using Anthropic, close to Anthropic, building around Anthropic.

41:58And we said, okay, but we had a discussion about, do you want to have it inside Anthropic or do you want to have it outside Anthropic? Because inside Anthropic would mean something, would mean a corporate venture fund, you'd have to hire for that, you'd have to have a whole role. And typically, if you look at corporate venture funds in history, obviously, besides OpenAI as a notable exception, they tend to not be very good because all they prioritize is who uses my stuff the most. And that's not a good way to invest in companies. So we thought this would be better. And the incentives on corporate venture funds are a little bit misaligned.

42:31So we did that. And now we look back at this fund. Obviously, Anthropik's in a very different place. We've funded about 40 companies. The rate, it's kind of a hard thing to calculate, but the rate at which companies graduate from when we invested in them to the next round is significantly higher on Anthropik fund companies. And we write both small and lead checks. I mean, the two several notable companies from the Anthology program have been Open Router, Goodfire. There's a company called Endia, Prime Intellect, Whisperflow. So there's quite a handful of pretty interesting things here. And yeah, I think what the other really nice thing about it is it really allows us to move fast on companies that, you know, where we may not feel immediately comfortable or ready to write the full check so we can participate in a round and then get closer and hopefully go and build a relationship and lead that in the future, lead that the next round in the company in the future.

43:37It also lets them get really close to the Anthropic ecosystem. So we have all these events with the founders and all execs and things like that. And people really enjoy getting it from hearing it from the horse's mouth. Now I think I would say Anthropic is in such a different place. It's no longer an unknown entity. So the program gets a lot of demand, but people kind of know what they need to know. And so we're still working on how do we make this program more useful and more beneficial for founders and Anthropic-like. Yeah. Also, congrats on all this. I think it's pretty successful. One reason I'm trying to highlight this for Linspace is also how does AI change venture?

44:18right and something that's something that alessa was exploring as well and that's why like i don't really know how to categorize anthology funds because it looks like a kind of like what conviction is doing what but what yc is doing maybe but like later stage right like um some of these already have their c some of these already have their a abacus is in there is that is that is that our abacus no no that's a different abacus but um what's the model like what is what are the predecessors that you draw inspiration from for setting up this fund? Or do you just not? It's like a corporate venture fund managed by Menlo, somewhat funded by Anthropic.

44:53I would say you can think of the companies that go into Anthology in three categories. One is strategically important to Anthropic, and those could typically be somewhat later around, somewhat bigger companies. Two are companies that are using Claude heavily and And are just great companies to be in. And three is just very, very early stage founders that are very high potential that may potentially be using cloud models and Anthropic and so on. We don't require people to use a certain model or the other. So we keep it pretty open. And we do everything from like a$100K check to a$20 million check.

45:33So I think it's really broad in terms of what we can do. and we wanted to intentionally keep it that way. When it comes to where we draw, there's some old, old examples, but I don't think it's really relevant. There was a fund called iFund that Kleinert did with Apple way back in the day. It was kind of similar. How did that turn out? I don't remember. I don't actually have enough data on that, but that's one example. Then you know the answer. No, I'm sure there are some great companies that came out of it. I just don't know the details about what was in it. So yeah, I mean, I think that's kind of, how it's been for us.

46:09And I think it's been a really great program. And we've had, I mean, we were excited about the companies that we could lead the rounds in as well. Yeah. I wanted to get quick hits for people who maybe never heard of Goodfire. And like, I know that because I've been, I've invited Mark to my conference and I've been to a bunch of their events. Actually, I'll just give you that list, right? Goodfire and Prime Intellect are in your research category. There's others with like diffusion-based language generation, novel architecture. It's all over the place. Research is like the most wild west of this.

46:39How do you view like sort of research investing? I can talk about any of those companies for briefly as well. But the way I view research investing is it is extremely hard to pull off. But when you pull it off, the results could be very remarkable. One of the hard parts is the tension between do you keep investing in research, hoping for something that yields a better result that leads to a better product? Or do you try to monetize and scale what you have already? That's tough. It's a really tough thing to do. It's a really tough decision to make when you're working with those founders, you're on that board.

47:11It's somewhat anxiety-inducing when you're thinking about this, even from an investor standpoint. Do I just get to a couple million A of R? Do I start doing something? Or do I keep the research bet strong? The way I think about research investing overall is, honestly, follow where the talented people have the most competence. and then have an idea around how this could be useful in what I call a top-down way. It's not really top-down, but the way I frame it is if I fast forward 10 years from the future, what do I think is very likely to exist? And what are the ways I can get there? If I do believe strongly that there's something like that and I believe there's a team very strongly headed towards that direction, I can sort of draw a dotted line and go like, okay, maybe we can see something here.

47:58So that's how I broadly think about it. So concrete example, Goodfire is like the most interesting one. Mechanistic interpretability. I didn't even think that was a market that was worth investing in, but obviously Anthropic does. And they seem like they have good vibes. What's, I guess, the summary of your take on the company? The way I think about the company is right now, almost all frontier and some many non-frontier AI models are complete black boxes. We don't understand why they produce the outputs they produce. All of the eval and studies on them are empirical studies, not intrinsic to the model.

48:34So it's like, hey, here's the outputs we saw, and therefore this is the benchmark score, or this is how we think it did. And if we believe as a society that five and 10 years later in the future, these models are going to be critically important for making pretty heavy decisions, whether it's, I call it anything from whether somebody should get a loan or insurance or a legal decision, then I don't think that the black box approach is long term scalable. It's just not how society can function, where you throw your hands up and say, well, this is what the model said. And then I asked it, explain yourself, and it said this other stuff.

49:14Great. That's kind of what we have today. That's the best thing that we have. Mechanistic interpretability is really going into the weights of the model and trying to figure out why did the model do what it did. And one of the more concrete and relatable examples of this that you guys may be aware of is GPT-4.0 had this phase of sycophancy that a lot of users really liked, but it's kind of one of those things that's not as easily detectable in an eval. Unless you know you're specifically maybe testing for it, even then it's quite hard. It's very personalized. It's not like any keywords might arise, obviously.

49:52But it is something that is quite easy to tell in even current interpretability methods. You can tell when a model is being sycophantic. You can tell when a model is trying to lie. You can tell when a model is trying to steal or persuade you of something. And so I think that if we further that research direction two, three years in the future, we will be able to understand why models say what they'd say. It's brain surgery for LLMs is my catchphrase, but doesn't apply to LLMs only, all models. And that is a pretty important insight into deploying AI at scale. Yeah. And you don't know the business model yet, don't need to.

50:29There are some ideas that we have, but not already to talk about publicly. And some that are working also, it's not right to be public. Does it feel worthwhile to do this on such small models? Because I think most of the work is done on the open source releases. How much of a gap is there between what they're able to do and then translate that into doing it for? There's no gap for scale. They've shown that even for the biggest open source models, even for DeepSeek's big models, they can do it. And in general, scaling is not the bottleneck. Obviously, access to the weights would be a bottleneck, but not.

51:03But they're in the Anthology Fund, so they can work with Anthropic. They can work with Anthropic, but they don't have cloud access, cloud weight access. For listeners who want to hear more about Mechinterp, we did a podcast with the Mechinterp team, Emmanuel from Anthropic. So that's your one-on-one there. We'll do something with Goodfire at some point. Prime Intellect, another very hype-y company. You don't have to say it, but I know it's very much in the water that they have raised a very large round. So I ignored distributed AI for a long time. It's usually crypto people coming over saying like, hey, we have these GPUs all over the place.

51:39We will somehow ignore the speed of light. And just like, you can use our GPUs to trade models. That's why I ignored Prime Intellects. I was wrong. Tell me why I was wrong. You may not be wrong. I mean, look, I could be the kind of person who goes to shills all of their companies and says this is the best thing ever. and if you don't think it's going to be a$10 billion company, you're wrong. Every company has risks at this stage and Prime Intellect has their fair share of risks. And whatever went through your mind, went through my mind when I was looking at that company. I do strongly believe in like, I'm sure you've seen this quote too, is in the quote of pessimists are probably right often, but they rarely change things.

52:15And it's an easy thing to say, but when you're investing, it's something to think about, which is there's a lot of things that could be potentially wrong with Prime Intellect, for sure. But the thing that I really liked that drew me to them is if they were right about a couple of things, what could go fantastically? Distribute training is one of them. Access to talent, I think, is one of the things that I underwrote for them. The ability to hire fairly great people away from other labs is really hard. And so I think they can do that. And the third thing I think is there's a broader vision to Prime Intellect that is not yet realized yet, where the first step of that was a distributed compute.

52:57And we'll see if they realize that. Yeah. Well, you know, Will Braun's been on the podcast multiple times and they've launched kind of like a Verifiers SaaS platform or something or a marketplace. I'm not really sure what exactly. I should probably try it out, but it's very interesting. The other thing I'll just say out there is like, everything in AI changes like every three, four weeks. So I'd be a fool to say like I could tell like what this company is going to do. Yeah, well, you know, all I'm trying to do is I try to capture for people who are like not in the loop on like, you know, these are the companies that people are talking about, right?

53:32Okay, so let's at least hit on OpenRouter and maybe one more of your choice that maybe is like less known, but you want people to know more about it. OpenRouter we have to cover. Big deal. obviously i like i i do think like this one i was like relatively early on in terms of like i i saw the i saw the products i saw what he was trying to do and i mean it clearly has done really well i did not know he was taking investment or i would have invested he wasn't okay say more say more open router was sort of my like you know like i don't want to make this about me it's really about them but in my mind it was my my darling deal because i'm just like man, I entered venture and I'm like, that is the company I would have built.

54:16I think we're skipping a bit. Let's explain who Alex is, what he did before. So let me give you the background on OpenRouter. Alex is a phenomenal, phenomenal founder. He started a company called OpenSea before, which was the NFT company. Obviously that at its peak was, I think, a$14 billion, more than$10 billion company. It did not meet that valuation's expectations, but look, there's many things out of control and in your life. Then Alex started this company called Open Router. And what gravitated me towards it initially was two things. One, it was very clear from my time at Glean that this is a perfect problem where engineers all think it's easy until it becomes so annoying to keep maintaining this.

55:01That's the sweet spot because no other person, no other company will gravitate towards it. yet it is kind of thorny to be able to maintain a portal that accesses a bunch of models. The nuances are quite tricky and annoying and boring. So that's one thing I liked. Second thing I liked is I was pretty convinced that if there was a market for anything like this, it would have to be a PLG motion. I think it goes so far as to say for in any SaaS market, if there can be a PLG motion, the PLG motion will win. What I mean by that for like if people are not familiar with venture words, like PLG is all users have to be able to access and self-serve the product and try it in order for that to be successful.

55:41Without talking to anyone. Without talking to somebody else, like the classic, like get on the phone on a SaaS website. So those two things really drew me to the business. And then of course, third one is just quality. Like there's these small details at Open Router, just like beautiful website, beautiful landing page. It's not some like SaaS trash of like, here's what we do and product solutions about us. I am so sick of that. You land on the page, it's a developer page. It's like, here's how many people are using what models. Love it. I'm like, this guy knows what his users really want. And all of those were compelling.

56:15I went out to New York to talk to Alex. He ignored me a bunch of times, forever. I'd write him what I call love letters. I'm like, hey man, love it, dude. It's so cool. I don't even want to invest. Just talk to me. I don't really care. I just want to meet you. I have so many ideas and interesting things. And it was one of those companies where I genuinely felt that way. So when I did meet him, we started jamming on things and I don't know the VC motions of how to sell. So I wasn't really even trying to do that. But when I told him, like, look, if you are ever going to raise, I will make it happen.

56:46I just love everything about this. So that's how we ended up doing the round. I think the company is interesting from a business model perspective. I get this question a lot. How does this business model scale? And I think right now the business is doing fairly well. Volume. It takes like 5 % of everything. There's that business model, but then there is a reasonable threat factor where, you know, what if the spend on the net goes down over time as tokens go up? So you do take, you do carry some risk of the prices of LLM falling to a point where the business does stops working. And I know many other companies take that risk as well.

57:22So that's one risk of the business on just pure consumer spend. Second risk would be keeping people on a, like a lot of hobbyists use OpenRouter and they tend to churn. And then a lot of enterprises will use OpenRouter to evaluate and then go pick a model that they want to settle with later. So that's a problem to fix. And so those are two of the risks, but overall, I think they've just like been executing phenomenally. Yeah. How do you think about the Vercel AI gateway, for example? I think that's been, I mean, I'm a fan of OpenRouter. We'll also do it, Vercel. Yeah, I'm interested where you already have, like I use Next.js, right?

58:00And it's like, well, I just use AI SDK. AI SDK comes with AI Gateway. It's like kind of makes sense to do it. How do you think about this market and like how tied you need to be to like the actual application development versus you're just kind of like this, what's the land? Hey, we don't have, you know, OpenRouter doesn't have a developer framework, for example. You know, if we're in a partner's meeting, that's maybe what I would ask. My simple answer is I don't think the AI gateways of other products are ever going to be their first priority. And the other simple answer is I think OpenRouter has this mindshare and momentum that just doesn't go away overnight.

58:37So it would be similar to asking like, hey, I'm OpenAI in 2020. What if somebody else does this? Yeah, they could or 2022. They could, but we are so far ahead in some ways already. I think the last thing is, I think that they have built a lot of smaller things that are non-obviously useful that other people probably won't sweat the details to go out and build. And so when I say that, I'm like, it's everything from like, here's something that nobody even cares about about OpenRouter. But they have a feature flag where you can only want to go to certain LLMs that do not retain your data. They go to that level of granularity of thinking about what do the users actually want.

59:20And that's one example. Another example is their detail on the provider level. Almost nobody has provider insights. There was a very interesting side study of how Kimi K2 did this whole study of different verifiers. But I think that's interesting. The fact that people don't really acknowledge this, but the same open source model or the same host source model can be served by different providers and have different context windows, different quality, different latency, different throughput. Where would you go to see all that information? Well, you see it on OpenRouter. And there's some elements of scale where there's enough people using the different providers to get that data.

59:58So all of those things, I think, are somewhat defensible on OpenRouter and hopefully more over time. Yeah, and I think their leaderboard charts are one of the best growth hacks. Very good graphics. Especially people that are into open source AI are always posting these things. Saying, hey, open source is up. We're back. One thing I used to joke about is OpenRouter is the only non-Elon company that Elon has tweeted the most about. For obvious reasons. Grunk code fast one, number one right now. I'm sure that's AMP code free plan. It's like a good week where I was like, every day it's like, open router, open router, open router.

1:00:35I'm like, yeah. Yeah. And so for those who don't know, that's because Grok Code Fast is like a top model. Yeah, because it's free. Yeah, because it's free. Yeah, yeah. Yeah, there's a lot of gaming, right, of this stuff where it's like, oh, we'll give it to you for free, but then we'll say we're very popular. I'm like, yeah, you're free because you're popular. Yeah. You're popular because you're free. The other way around. Okay, very cool. And okay, so there's a bunch of others. We're not going to go through all 40. What comes to mind? What do you want to talk about? What do you think maybe is a very interesting company in your portfolio that more people should know about?

1:01:07I'll talk about Whisper and Inception are the two I want to talk about. Inception? Inception's not even here. That's why I was...

1:01:18We can talk about the company without saying the name. Yeah, okay. Let's try that. Let me try that and then... But I mean, also, Inception, if I Google Inception, it's not like I'm finding it. Anyway, let's talk about these two things. Whisper, I can talk about first. That's a clear one. So Whisper is a company that does, you know, a very, in many people's eyes, something very commodity, which is voice dictation on your phone and laptop. The things that I really liked and that stood out to us about Whisper was in that quote unquote commodity market, they are, in my mind, like the fastest and best and most delightful product that kind of, in many ways, set the frontier of the nuances of how to make this easy.

1:02:04Press your function key on your Mac, talk to it. It's always on. It has fantastic accuracy. As you're dictating, if you ever stutter and go like, oh, no, I didn't mean that. I actually meant this and knows what you went and it goes and corrects it. I find that they have this metric they use called zero edit rate inside, which is, you know, amount of times you don't need to edit. Correct. And their zero edit rate, I think, is north of 80%, which is insane for a voice dictation product. So many other risks of that business too. But one thing I think I love is users love it. Users stay on. The retention is great.

1:02:41And it might make voice suddenly work. Because if you think about computing, people type slower than they talk. And so it is unlocking this new, faster way that people feel comfortable talking to their computers. That really didn't happen in voice dictation before. And it's not just a whisper model, which is a common question I get. Yeah, for people who don't know, it's W-I-S-P-R. Yes. Which, you know, you got to spell it somehow. I mean, the question here is always like, it's the same thing, right? Like, voice is very commodity. I actually happen to use Super Whisper. Same. Right. Mostly influenced by Jeremy, actually.

1:03:16And then Granola is very popular. Notion has like this Notion speech thing. Like, how? What's the plan? This is every. This is why I'm not an investor. How do you survive? Trying to reason about why you should be the winner. Even ChatGPT desktop has some shortcuts for stuff. I don't know if it does exactly the same thing, but it's not that far away. Anyway, you're excited about it. I do see a lot of tweets about Whisper. It's one of those things where the PLG is getting me, man. I'm like, should I switch? I don't know. My thing's fine, but what if it feels better on the other side? I don't know.

1:03:58We'll see. We'll see how that plans out. There's some interesting plans to get it to be a cooler product, but we'll see. The other company, and again - Okay, we'll call this StealthCo. StealthCo. One thing I find very interesting about StealthCo is it comes in the purview of research. We talk about different architectures all the time. One of the most compelling alternate architectures for AI is diffusion models. So one thing that I think is really interesting about it is you talk a lot, Sean, about the Pareto frontier of latency, cost, quality. Diffusion models today are, I would say, 80 to 90 % of the quality at one-tenth the cost and latency.

1:04:35So it has huge implications on, obviously, the stock market, which is kind of NVIDIA, and many other things. But also there is clear examples that you can show of use cases where that might be very valuable because there are many applications that work in volume that do not require high quality, but definitely require better latency and everyone could use some cheaper models. So, you know, there are, I think there's an interesting area of research there. Maybe it gets to Frontier, maybe it doesn't. The one thing I want to draw attention to with Diffusion that I think is particularly interesting is left-to-right reasoning for code doesn't actually really make sense because in code, we don't, like we might sometimes write code left-to-right, but after you write code, you go up and down and figure out, hey, is this variable set?

1:05:21Did I do this? there are many bidirectional dependencies in code. So there's a natural tendency to lend itself to diffusion models where you can imagine like as you are denoising, you fix partial issues in different parts of the code at once versus this reasoning paradigm where you kind of have to figure everything out and then go give your final answer. Yeah. Yeah. I like that a lot, especially for like syntax structures, like C-like languages where you need to open and close the bracket and all that and hold that state i think like it's i the question is always the sort of quote-unquote the hardware lottery of transformers like transformers is all you need and like uh diffusion is kind of like a different branch off of that tree of research they are related but we might be too far gone down the transformers tech tree to come back and then go down diffusion like being the point where like they might never be frontier because we've just had like four more years extra of like Transformers LLM research.

1:06:21Yeah, it's true. I think about this all the time. Like thinking about in the course of history, what are the significant moments where if only something forked off a different way that maybe there would be a completely different paradigm of outcome. Yeah, and usually the worst tech wins like Blu-ray, DVD, HD, DVD or something like that. I think there's like a lot of variations of this. Even like I think there was a discussion about AC versus DC currents, like back in Edison's days. There was this big fight between Tesla and Edison. I don't know if you've... I'm aware of the very, very basic details, but it's so interesting, right?

1:07:02Because you take something like this, and then the question becomes like, okay, do we bet on it? Or is the timing just off because something took off and we can't pull this rocket ship back to earth, and so we've lost that fight? I don't know. I'm not a purist scientist anymore where I believe like the best ideas and things win. I think in markets, it's very obvious that that's not true. I think a lot of things go into winning and sometimes it's out of your control. Yeah, it's very true. Like, you know, speaking of Anthropic and like things that happened this year, MCP happened this year. And I was, when MCP came out, I was sleeping.

1:07:38And then when they came and did the workshop with me and I think you see a lot more noise. and I was like, okay, there's something to this. And now it's basically kind of de facto won as the interop layer for all the labs and all the models. And there's no reason why this could have won versus anything else apart from like, it was well specced out, it was backed by Anthropic. It's kind of a similar thing. I don't know if it's like the best, but it was good enough. Yeah, it happens so often. It kind of makes it tricky to, not even just investing, but in general to think about ideas. We see this with startups as well.

1:08:12It's very heartbreaking. Every once in a while, you'll meet a founder where I'm like, your idea is fantastic. Your execution is great. I just don't see it work because the market dynamics are not in your favor. And maybe I'm wrong about some of them. But, you know. When you say market dynamics, is it TAM or something else? No, sometimes it's like, I don't see that. Like, you are a small group of people trying to wedge something into a market. We know how long that takes and we know the other forces at play. And if I don't, like, I just don't see, imagine a single person running in a tunnel with a light at the end, but the tunnel's closing in on you.

1:08:49You could be the fastest runner in the world and you might not make it out of the tunnel. That's kind of the analogy. And so you might be doing everything right. It's just that that window is not there, or at least I might not think that window is there. I do think a lot of companies fall into this bucket of ideas. And so to me, in a way, I almost think of companies like Mosaic ML in a way, which is like, hey, we got this amazing team. We can help you find two models. And yeah, but nobody, you know, the market dynamic, just there's really nobody fine tuning models. And part of it is like the open models are not that good.

1:09:23And part of it is like people don't really have good data. They don't have the expertise. And again, if you go back now, now there's like, you know, RL environments and like RFT is like the next wave of that. And it's like maybe they'll be able to get in the window. but it's just interesting how you know now the other flip side of that is and yet they get acquired for this amazing but yeah because the market is just so big i mean even if you think about something like yeah diffusion models for text right it's like you know it's a bet it's like if you sell it for a billion dollars right it's like 0.01 of like imbedia's market cap and so it's like okay well the amount of money being spent in this space is large enough to justify betting like the same way Instagram was like one percent of Facebook market cap it's like this is similar where it's like man Databricks is rich enough thing exactly it's like you know they really want you to know that they're an AI company exactly another worth 100 billion I mean you know like without Mosaic ML exactly it's like without Mosaic ML maybe they're not on the same trajectory it's like I don't know maybe they are because you know Ali is great and all that you guys have ever talked about like the roll-up companies which is my favorite like The PE vol-ups?

1:10:31Yeah. I didn't know that was a topic of yours. It's not really a topic of mine. I just find it quite interesting to see how, speaking of AI companies and markups, there are companies, obviously I'm not going to name them, but there are companies who go like, hey, here's a small company that does a million of ARR completely with humans. I'll buy it for two million, and then I'll do some of it with AI. But now I'm an AI company and a million of ARR. an AI company world is a hundred million dollar valuation. And so, you know, it's, it's, it's pure, like multiple arbitrage on the category that you're in.

1:11:08Um, so like, yes, that's the like cynically, haha. But then like, what if it actually works? Because like the hard part is getting the customers. The hard part is like getting the domain expertise. You drop a bunch of software engineers in there and like, you know, automate it and make it scalable, make it cheaper. And yeah, maybe it works. No, you're right. You're absolutely right. I think it's just pricing. He funded a company that bought a tax firm. Yeah. A law firm. Accounting firm or tax firm? A law firm. Law firm. If it works, it works. I just think what was interesting to me is you can 50x the value of the company before you actually landed anything with AI yet.

1:11:47Yes. But then you use that funding and the equity to hire the people. It's weird. So there's this concept I always talk about, which I'm surprised people don't really understand. It's reflexivity. The belief that something can be true can make it true, even though it's not true at the time that you believed it. Yeah, that's venture capital. Just give money and everybody's like, oh, they raised$300 million. It's a great company. I love that company. It's like, yeah, I'm an investor in it, so I love it too. And it's like all the employees are like, I love this company. My stock is worth a lot of money.

1:12:17And there's also that effect that's very clearly in venture capital where not just what you said, which I agree also happens, but imagine there's times where people funnel so much money into a company before it's really like prime time, which dissuades anybody else from entering that market. And then they become the de facto owner of the market because they canceled the competition with funding. and you can think and I'm not going to name the categories but you can think of numerable categories in this market in this paradigm that's already happened and I feel like even in AI it's like maybe two and a half years ago when ChatGPT came out it's like this is cool but like you know a lot of enterprises were like maybe skeptical of like is this trend going to continue but then once you start seeing tens of billions of dollars being put in OpenAI and Anthropic and it's like it's got to work especially you can deploy and hardware.

1:13:08Yeah. Which, you know, I think like, at that point, you're building infrastructure and infrastructure very capital intensive and like, you actually can do the math. It's not humans anymore. It's like machines and land. Yeah, exactly. Power. Like Amazon is building all these like training chips and like all this infrastructure for Entropic. It's like, do you really think they're dumb? Like, you know what I mean? I think at some point, it's like same with Stargate. It's like, do you think all these people are dumb? Yeah. And like you're saying the models are not that good? It's like, you know. uh the podcast released today with kyle like he was still kind of skeptical that they had 500 billion for stargate and i'm like not only do they have the 500 billion they have the next like trillion like lined up mostly because like the the projections i think like i i've been talking about this a lot and i'm very out of my depth because i'm not dylan patel but like i think it's the most big it's probably the biggest story of the year like beyond the models like It's just the infra build.

1:14:03And I think people don't understand the roadmap is very, very strong for the rest of this decade at least. For OpenAI to go from two gigawatts of compute this year to 30 with everything they've already announced. And then there's a plan for the next 125. The United States uses 300. It's crazy ambitious. Do you think, I guess it's a question for you guys also, because I don't have a good answer yet. The belief is always obviously bitter lesson-filled, right? Like you buy more compute, therefore you get the most models. By the way, it's an anthropic relevant thing, right? But I guess, is that necessarily true?

1:14:44There could also be a world where that's just not true. So, you know, you are kind of betting. This is what makes it bitter. It's like, what if it doesn't apply to me this time? Right, right. Right. And I think, you know, being in Sam Altman's place, it's absolutely the right chess move to play. But, you know, I do wonder what happens if like all this investment in compute doesn't actually lead to economic gain, flash better models, flash everything else. But I feel like we've reached the point where like the models are good enough that even if the next generation is not 10x better, we'll be able to use the compute.

1:15:20I mean, and again, the data center is like, you know, they're writing it down for like 30 years. so it's like you know can you run gbd5 pro over the next 10 15 years given the amount they're spending on compute this is a general question i'm not criticizing at all is even if everyone was using cloud like whatever codex cod code whatever all the time like inference demand is not that big globally right so you would have to believe so what would you have to believe for that to be true because there are 800 million weekly active users this is what greg robin says like a GPU for every human on earth.

1:15:56I'm somewhat shitposting. I'm somewhat shitposting, but they actually say this on their official comms. So I'm just repeating him. I don't necessarily disagree. I'm just trying to work backwards to like, what do we need to believe to get there? Because chat GPT compute is not that much. Correct. Right? So they're not doing like agentic stuff. Maybe they will be in the future. Most people are doing basic Q &A type queries. By the way, I put it up on chat. So if people are watching on YouTube, they can see this. which is this year, OpenAI spent$7 billion on compute. Only two of that was for all of their inference.

1:16:29The remaining five was R &D. So all of ChatGPT, all 800 million users, all of Sora, all of the sort of API volume, 2 billion. And they have two and a half times that for R &D. Right. And so my point being like, if inference is one thing, I don't know how that will scale to that volume, but then you'd have to believe that the rest of it goes into R &D and therefore produces models that are so much better that therefore have more demand, etc. But if in any case that, like, I don't know, the incremental marginal is not that big, then, you know, that's the risk of the bet. Yeah. So to disrupt OpenAI, you need to have more efficient research because right now it's pretty inefficient.

1:17:17You know, spend five to get two. So like what OpenAI did to Google is what the next OpenAI has to do to OpenAI. You know what I mean? Like Google was spending a lot of money. Facebook was spending a lot of money and like they didn't come up with anything. OpenAI did. And it was like a small, tiny little, you know, startup. and they had GPTs and I like Bradford but someone else may or may not come up with that. It's like that classic quote, your margin is my opportunity. Google was milking those margins and they didn't want to spend the compute for every search query and so now OpenAI is willing to.

1:17:52So we've covered a lot of topics. Thanks for indulging. For me it's like a survey episode of here's everything. We're also catching up with the former guests and it's always nice. maybe we can end it on this coding interview thing which literally you tweeted about today what is the situation that I guess engineers should be aware of and I think this maybe ties into LLM psychosis a little bit you know, so I tweeted I'll just cover the tweet first, I tweeted about this guy who wrote a blog post about he was in an interview from a, I didn't think it was a legit account, he thought it was a legit LinkedIn message where he was interviewing for the company they sent him a coding interview they said, clone this repo, run this code, make this edit.

1:18:34Kind of not untraditional. So it's a pretty run-of-the-mill type interview. It happens. And in that interview, he claims that he went to Cursor and asked whether the code had any vulnerabilities or anything he should be aware of. And it revealed that it had some link. It had a byte array that compiled into a link that would go and take a bunch of private information from you. So that was the TLDR. And I tweeted about that saying, you know, like the world, interestingly enough, it was solved by Vibe coding, but it could very easily, the world of Vibe coders who don't really look at code, I imagine are more susceptible to being in attacks like this and in the future.

1:19:17And it got me thinking about a lot of things. Like, what do attack vectors even look like if people aren't looking at code? There's so much that can go wrong. and what are the implications on model safety and how models behave in those environments. So that's one. But I think the broader thing, and I'm curious what you guys think about this, is what I've been noticing more and more is I was having this conversation yesterday with some of my close friends where some of the joy of coding used to really be, you're stuck on this annoyingly hard problem and you just bang your head against a wall and you want to kill yourself.

1:19:48And then eventually you're like, I've figured it out and then you solve it. And that's the muscle that you build when you improve and get better. And now I find myself even doing this. It's so hard to do if you just have a constant slot machine that might give you the right answer. And who knows if it will, who knows if it doesn't, but you just pull it all day long. Please fix, please fix, please fix. And what does that mean for the craft of engineering or software engineering in the future? I don't know, like this vibe coding stuff. I mean, great for the rest of the world that was not an engineer, But I'm now seeing how it's affecting the trained software engineers.

1:20:27And it's kind of like a drug for them. And it stops them from living their own life, which is doing the engineering. It turns your brain off. Because it turns your brain off. Yeah. I think self-driving cars, people thought about this first. This is why when you drive your Tesla, you have to keep your eyes on the road. Because they don't want you to turn your brain off. And we don't have that equivalent in developer environments yet. Maybe we should. watch your eyes. We removed one word in the code. Which one was it? Write it back. So my answer, I mean, I happen to have shipped a model today, two models.

1:21:04And part of that is actually what I've been calling the semi-async value death. And a lot of it I think is my reflection on coding agents in terms of like we started with Copilot which was tab autocomplete. And then when we went all the way to clock code which is like very async, very, you know, like it could take 30 minutes, could take 30 hours. I don't know. It just runs. And I think like something that cognition is very interested about is fast agents or something I've been writing about more as fast agents is where like under a certain level, you actually want to just be in a mind meld with the human and AI to have like fast responses so that you can get helpful assistance if it helps, you can get out of the way if it doesn't help.

1:21:48and like that is actually where you do your hardest problems and then the async agent is where you do the commoditized dumb boring labor stuff that you know how to do you just don't need to do it but when you are actually very deep work and focus and you're working on a hard problem you are you're like you should be applying your human intelligence augmented by ai in an unintrusive fashion which i think is the way that obviously i think it's like it's a pro-human message but it's also like a really interesting area of research for us but that's almost like to play devil's advocate there. That's like telling somebody, well, I'm going to put the cigarettes right here.

1:22:20I know you love smoking, but please don't do it. It's not a cigarette. It's right here. It kind of is. There's an analogy, right, to be made here. It's a cigarette for your brain because you do not think anymore when you pull that button. And over time, I feel like, you know, the brain will get weaker if you don't use it for that task. And I like your message. I mean, I would ideally like, if I had a team of engineers, I would also tell them the same thing. But I mean, I worry about the reality, which is that's not what they do in many cases. But I mean, you got to ship the thing, right? Like I agree, but at some point you got to close the ticket and merge a PR.

1:23:00So how are you going to get the code done, right? It's like they are doing it or they're going to get fired. They're just like generating, they're running the enterprise integrations. One way or the other. One way or the other. B2B SaaS. Yeah, it's interesting. Okay, so maybe I'll put it this way and I'll see how you respond. Okay, so we have the fundamental formula for coding agent performance. Okay, it basically is find the right files and then write the right files. That's it. So read and write. Read the right files and write the right files. That's it, right? So actually what fast agents can do or what I just did today was basically the equivalent of a heads-up display.

1:23:40Like give you more info, but you still take all the actions. So we help you read, read faster, read more efficiently, read with more focus, but you still write. And so I think that's not a secret so much as we try to be helpful and we're evaluated on the helpfulness of the reading and the comprehension so that you can hold everything in your head. That would be the pitch. It's true. I don't know how the product looks. I would love to eventually play with it with the sweet grep and all that stuff. But there's a world where I think the product decision also goes a long way into how people use it.

1:24:21So if it is like that, then maybe. And I think when people use, even for example, if someone uses a cursor, a lot of people like the fact that they can see the code and then they kind of have to hit the final accept. Human in the loop. Human in the loop. But I still worry. I still worry. And I worry the most about the younger kids. You think about the people growing up in college. How would you ever get yourself to think if you just had this clearly more intelligent thing than you? At least for like, and I don't want to rate myself too highly, but if I'm working in a domain that I understand, I can at least tell, yeah, yeah, yeah, model, you're doing the wrong stuff.

1:25:00Definitely don't do that. Don't write that at all. That's a terrible file. Why are you creating four files for this? But if you think about what it looks like to an 18-year-old CS major freshman, they're just probably like, I guess that's how you do things. And they can't hold it at that. So their training is just a little bit different. Cool. Yeah. Hi, Didi. Thanks for indulging and welcome back and thanks for coming back. Thank you guys. Always fun chatting with you guys.