E47: Index Ventures' Mike Volpi on AI, Open Source, and Evolution of VC

26:55So just as it's helped big companies, it's helped startups have a huge, huge advantage. And it's created a whole suite of new business models, which are beneficial to everyone. And, you know, 15 years or whatever the amount in, would you say we're still early to this trend? Or is it a mature trend? How do you think about this wave? The cloud wave? Yeah. I think we're pretty well through it. There are some companies at a certain level of scale that are still like, oh, no, I like to build this on my own. And there's a set of reasons why. Some are regulatory, some are NIH, whatever. But I think the vast majority of the technological community is very much at this point embracing cloud, even the bigger companies.

27:40And, you know, while there's still going to be some spend on homebrew data centers, I think, you know, they're on a long march back from Russia. It's that that wave is over. Hey, we'll continue our interview in a moment after a word from our sponsors. You mentioned you got excited about AI and deep in it in 2015 or so. Talk about your journey there, about what got you excited there. What was your evolution and how do you think about, hey, you know, what types of investments could make sense at what time? maybe a little bit changes to your evolution there. Yeah. Well, my original starting point in AI was when I got excited about self-driving cars.

28:18I was at a TED Talk. I saw Chris Hermsen, who is now CEO of one of our companies, Aurora, give a talk on what is now Waymo. He was at Google at the time. And I was just, I mean, I'm a mechanical engineer. You know, I love cars. And I thought this is the most amazing things in sliced bread. So it was just purely like, It was cool. Started reading and learning about it, ended up sort of stalking Chris Ermsson until we led the Series A in his company, Aurora, and began to learn the fundamental principles. Now, mind you, this is 2016. So the famous attention is all you need paper, transformers, LLMs, distant future thing.

29:00At the time, it was about deep learning methodologies applied to computer vision, but it was still very cool. And so that was my entry point. The more I learned about it, though, I would say what's made it fascinating for me, aside from the whole investment dynamic of it, is it's an absolutely fascinating technology in that it actually mimics the human brain. And oftentimes when it's hard to understand what's going on with AI, the analogy to the human brain. Obviously, we know that it works very differently, but as a black box, it's just like the human brain. And I find that intellectually absolutely fascinating.

29:43I find it completely fascinating. I found it fascinating then, and even more so now. It's a super interesting way of thinking about the world. So the famous Descartes line, I think, therefore I am, right? So our existence is based on the fact that our brain thinks. And here we are attempting to kind of recreate that as best we can through electrons and chips and all that good stuff. We're trying to recreate the one thing that sort of proves our existence. I find just continuously fascinating. As it's turned out, there's a whole business angle to it, which I'm sure we'll get to. But in first principle, that's what I found super exciting about it.

30:22So I started with self-driving cars, invested in scale, which basically was a supplier to Aurora and learned a lot about how these models learn and understood the value of something like scale, which at the time people were like, oh, it's just Mechanical Turk. It's not a very interesting company, blah, blah, blah. Now there's a million people that say, oh, I missed that investment because I didn't. Yeah, whatever. So, so. Let me stop you there if okay, because I was lucky also to be an angel in scale thanks to the Index Scout program actually. And what did you see that other people didn't? Or what gave you the conviction that, hey, this isn't just a mechanical Turk, this is actually going to be a massive business?

31:04I'd say, so there were two things. I think a lot of people didn't double click on what scale actually did. So they took the cursory view of like, oh, it's just mechanical Turk and didn't understand how much software infrastructure had to be built to translate human labor into an annotated image. There's a lot of software in there. So in other words, the analogy of this would people thought, well, Uber is just Mechanical Turk. And it's not. It's not because you need some very, very smart software to turn labor, driving labor, into a business, into a global taxi service like Uber. And what I think I was lucky enough is because I understood how some of the mechanics around AI work.

31:55And I obviously had a lot of privilege seeing that as how they were being used at Aurora. And I thought to myself, A, there's a moat. B, this is way bigger than people think. This is not a handful of people. This is millions and millions of images at the time that will need to be labeled and annotated. So I think it was a double clicking on the technology itself and understanding the value added that was being created for sure. And then, you know, the other component is, you know, you meet Alex Wang and you go like, okay, if this guy was setting up a lemonade stand, I'd invest in him. Right. So, so the combination of the entrepreneur and sort of double clicking of an understanding of the technology, which thankfully I had an inside view of were, were what drove to that investment.

32:44And thank you for doing the Scout investment in it, by the way. That's awesome. Yeah, for sure. Yeah, I think people were concerned at the time of, hey, how defensible is this? How big is this? I think they were just in self-driving cars for some time, right? For a few years. Exactly. Yeah. So scale ended up being, well, they still do, I still think about a quarter or a fifth of its revenue as self-driving vehicles. But as that wave tapered because of market consolidation, if you remember, there were 50 self-driving startups, and then now we have like three. And so essentially the demand compressed.

33:23They were lucky enough to inject themselves into the next big wave of AI, which was the LLMs. And that's what's been creating fantastic growth for them over the last two, three years. Yeah. And why did we go from 50 to three self-driving car companies? What precipitated that consolidation? I mean, look, if you think about it from a pure economics perspective, right? Industries that require very high fixed costs, capital costs, tend to consolidate to a small number of large players. If you think about it, like, you know, how many mobile telephone operators they are in a given country. It's usually three, four, maybe.

34:08How many airlines, major airlines exist in a given country? Three, four. Public utilities, same. Electricity providers, same. Why? All of them have to spend a lot of money to build the basic infrastructure that is required at scale to be competitive in that business. Self-driving cars were the same. You You need to have a lot of data. So you need to drive miles or spend a lot of money building simulated data. You need to spend a lot of money developing a sensor suite for the company. You need to spend a lot of money developing different parts of the system. There's a hardware component to it, et cetera.

34:44So essentially, building a self-driving vehicle is a very, very expensive baseline proposition. And it doesn't make sense for an industry to have that capital peanut buttered across 10 companies. And so inevitably what happens is that the companies that are better at concentrating that large scale investment into themselves win and the other ones get either go out of business or get acquired and smushed in into the smaller number of players. There's a lot of analogies to what's going on in language models today, but I think generally speaking, industries where there's a high fixed cost tends to concentrate to a small number of big players.

35:28Yeah, I was just going to bring it up. The model companies today, it seems like that's... I think you're going to see a similar trend. I think that now there are tens of companies building models. Now, I think the airline analogy is apt. You're still going to have Spirit Airlines over here, and you're going to have some regional carrier over there. So there's going to be some smaller businesses that are based on probably smaller, more commoditized models. But the main players in an industry for language models, given that it's even more capital intensive than self-driving, potentially one of the most capital intensive industries that we've ever seen, I think that you're going to see more consolidation.

36:11You're starting to see it happen already currently. If you look at sort of inflection and a few of the other deals that are announced and unannounced in the market, you're seeing compression happen in the market into a smaller number of players. And I think we're going to see a handful of small players, a small number of players in the industry that are going to be very large companies. So you're going to see the similar trend over the next two years. the let's go back to sort of the evolution of sort of the commercial opportunities of ai so you do aurora you do scale take us further into like what are you noticing are the business opportunities here and how are you looking at the landscape i mean generally speaking i i kind of in a very simplistic way there are three categories of investments in ai there are the foundation models which are still happening you know currently there they seem to be segmenting various ways.

37:06Language, you see people saying like, oh, I have a Japanese model and I have a Korean model, whatever. Size, we have little models, we have big models and so forth. Coding, like we have models that code well and so forth. There's a variety of segmentations, but there's a foundation model market. I think it's kind of maturing. I don't know that it makes an enormous amount of sense to invest a lot more in foundation models, but it is a category and there are investments that are continuing to happen there. The second is what I call picks and shovels. These are the scales, the mosaics, the togethers, the et cetera, et cetera.

37:42These are companies that essentially provide a set of tools or services which allow model builders, whether they be foundation or enterprise models or whatever, to build their models more effectively, more efficiently. I do think this is a very interesting category. There's been some early exits. There are some emerging larger and larger companies in the category. And right now, a lot of the model building is happening with the foundation model companies. I do think that you're going to see more enterprises spend their capital resources, fine-tuning, training, adapted, constructing applications that are AI-based, and they will use a lot of these tools and services to do that.

38:24So So very interesting investing category. And the third one is what I call AI-powered apps. These are Hebbia, Harvey, Glean, et cetera, et cetera. These are the companies that essentially come into a market and they say, we really understand how to use AI effectively to improve your collaboration, your workflow, your productivity. And we're going to build a set of apps which are powered by foundation models, or in some cases, they're powered by open source models that we've adapted to do this. And we're giving you a SaaS application. It's a full-fledged application we're providing you. And that's ultimately, I think that that's going to be the biggest category because basically SaaS means everything.

39:07It has the largest market exposure. SaaS probably has the largest amount of budget dollars spent by companies against it. It serves small business and large businesses alike. And I think actually you're going to see the largest number of most interesting companies in that third category. And say more about that, because we saw a bunch of companies in the past few years get to very high revenue numbers immediately, only to not be super defensible once new models were new versions were released. How do you think about the types of apps that will have defensibility and staying power over time as the sort of model providers improve and update and get better?

39:47Yeah. I mean, I tend to think that if your application that you've built is one which is an isolated productivity enhancer, you will in all likelihood be more and more commoditized by the core model builders. Because, you know, obviously core model builders are moving up the stack themselves, right? When I take core model builders, these are, you know, open AIs and Anthropics and Google and Cohere and so forth. So they're going to go up the stack. What they won't do for the time being, at least, is focus in on specific workflows that your business needs. You know, I do A in my business, therefore I do B, therefore I do C, therefore I do B and so forth.

40:37collaboration features. So how do you and I work together with the support of AI to solve a particular enterprise problem? So the more the application incorporates the fundamental business logic and nature of the business it serves, the more differentiated it can be from just being a core model with a wrapper around it. So if what you're using AI for, if your application is like, I'm writing marketing copy and I'm just pinging the, you know, GPT-4 to give me marketing copy, which then I ship out, not likely to be particularly defensible. If you are building something where, you know, there's three layers of management, which are collaborating together on a particular document, a document is being enhanced and then it's being passed to an editor and a creator.

41:27Imagine like a AI powered Figma kind of a thing. That's much more defensible because the nature of it is woven into and embedded into the workflows and collaboration nature of a given corporation. So that's where I think you create differentiation. Yeah, that makes sense. Chris Pike, a venture capitalist of Pace Capital, wrote this blog post the other day, sort of provocatively titled, I think, The End of Software. And he was saying, hey, is it possible that what AI is going to do to SaaS is what the internet did to media, which is it made everybody a journalist, basically, or gave everyone the power to create content.

42:13And that fundamentally changed the structure of how newspapers operate. And will similarly, the ability to make software become so democratized or so easy that it will fundamentally change sort of the economics of how we think about software. Is there any, I'm not sure if you saw that post, but is there any idea there that you find particularly interesting or is it, you know, a false analogy? I, you know, I'll take a step back and say this, right? Making predictions about what AI is going to do is very difficult right now because it's a technology set that's evolving so rapidly in some dimensions, and yet it's quite stunted in other directions, that it's very hard to say, like, oh, it'll do this, because, like, this bit of it is improving so quickly, but this bit not improving very quickly at all.

43:05So, you know, who knows is the real answer. But I would say that I am not a subscriber to the thesis that there won't be any software developers anymore in five years. I do think that AI will enhance the productivity of developers, but even the least skilled developers will probably gain the most from AI and the most skilled developers will probably gain marginally from AI for the foreseeable future. And I would say that today, I mean, coding is definitely an area that has benefited a lot. If you sort of talk to customers and talk to developers and you say, well, how much better are you when you use some form of co-pilot?

43:4920 to 30 % better. That's not a substitution. That's like, you know, when I'm writing emails, I'm 20 to 30 % faster because Google gives me the little guess as to what the next piece of the sentence is. But it doesn't replace me writing that email. And I think that the creation of software, if you think about it, has several components to it. One is the comprehension of the logic or the business logic of what you're trying to do with that software. Is this processing a payment? Is this creating a picture? Is this whatever? Why is it creating the picture? What kind of picture does it want to create?

44:25What's the intent? All those things are part of building software. The AI can't do that because it's the software developer's prerogative to think about what and how it's going to be done. The second component is architecture. Like how do you construct, particularly if it's a complex piece of software, how do you construct it to perform in the best possible manner? That is pretty abstracted thinking. And it may be mirrored by other software that is written in the past, but in all likelihood, it still will be done by humans for some period of time. Then there's the acting of actually coding, like typing.

45:01And I think that that part of it is probably the area that's most likely to be commoditized. But the notion that media shrunk, I wouldn't say it disappeared, but it shrunk comprehensively because of the internet, is not an analogy that I see associated with this. To be clear, journalists were never particularly well-paid. They are even less well-paid now. Developers are extremely well-paid. And my guess is they will continue to be very well-paid into the future. what we will see more of is more software. Because if you think about it, this is a labor market, there's supply and demand. Why do software developers get paid a lot of money?

45:42They get paid a lot of money because there's a lot of demand and not enough supply. So when you enhance the supply, one of two things can happen, right? There's a clearing price for that labor. What you might see is if there's more supply, you'll see more software rather than less software engineers. And that's my likely guess is that software engineers' jobs will change. They will do more of the high-level abstracted thinking, but you're not going to see less software developers. You know, in some ways, I like to drew the analogy of banks and ATMs. Bank employees didn't shrink because we got ATMs.

46:17They just started doing different kind of labor. And I think that you'll see more and more software developers do value-added labor rather than spending their time typing, which is definitely time consuming and not the most productive thing they can do. So I don't see a parallel there. I see more of a productivity enhancement path forward. Yeah. And I guess the reason why this didn't happen in media where journalists got paid more is either because demand didn't rise sufficiently to adequately compensate them, or I think more precisely that it just turned out that journalists maybe weren't the best in the world to creating content and it, you know, out emerges whole creator class of citizen journalists or just online creators or influencers that were the Mr.

47:03Beasts of the world or, you know, different kinds that were able to make a lot of money and a lot more money than anyone was making before. Yeah. I mean, look, if you think about it from a media perspective, we get information and entertainment from our media. I don't think we consume less of that. We consume different types of it. And the value has shifted to different kinds of platforms. Whereas the old platform was the one that provided Newsprint with black ink on it, the new platforms called Facebook. So, you know, but I do think, you know, media organizations 50 years ago were very rich companies.

47:42It's just that those have actually, the value has shifted, but you still have very rich media companies out there. The journalist probably was poor back then and is unfortunately poor today. I say this as my mother was a journalist. So, you know, with all due respect to the profession, it never paid particularly well. Yes. Some of my best friends are journalists. Yeah. All proper respect. And so the, you know, Benchmark has this thesis around, hey, they don't, they don't believe that for most of the economy, we're going to have this sort of co-pilot model where, you know, AI is going to help the person do their job better.

48:16Like you just described with engineers, they believe more. And I think like sell the, sell the work, i.e. it's, it's going to actually replace, and I guess what Devin is trying to do to engineers, but for the broader economy, do you think what you just described with engineers will be the norm and there'll really be this co-pilot view of the, of the world across, you know, lawyers, doctors, or all sorts of email jobs or professions? Or do you think it'll be a split? How do you think it'll play out? Well, first of all, so I would call what Vishri is saying as a false contrast. I don't think that contrast exists.

48:52I think you are going to see both. There is going to be some form of labor, which is completely done by AI. and you're going to have some form of labor where the AI enhances or increases the productivity of the individual. Both. I don't see why they are one against the other. There's probably going to be both. Which is going to be which? What's the litmus test to determine which is going to be which you think? The hardest thing that we do as humans, I think professionally, is the abstract thinking, the logic that exists in what we do. As an investor, the hardest thing is to recognize the human characteristics of a great entrepreneur and convince that entrepreneur to take your capital.

49:37Is an AI likely to do that? Not for a while. Not for a while. However, when I want to learn about how RNA development can be improved by AI, I will go to ChatGPT and talk to it for a while. Whereas five years ago, I used to go to my analyst and ask them to do research for me. So for sure, my work, my productivity has been massively enhanced by the types of research and learning that I can do when I dialogue with AI, right? But my decision-making, and frankly, as a professional, probably the most important thing we ever do is make decisions. That's the one thing that we do, I think a lot of those will continue to stay in the human domain.

50:24So the more your labor is, I'll call it grinding out stuff, the more likely that that labor will be substituted by AI. And the more your labor is abstract thinking, logic, decision-making, the more that's likely to be assisted by AI, but not replaced by AI. And so I think you'll see both. and that's fantastic. It's not like the internet replaced one thing and didn't replace the other. All of our lives change. In some cases, things have been eliminated. We don't write letters anymore. But in some cases, we use email to keep us more productive. Yeah. Yeah. That's a good framework. Let's return to the intersection of two topics that you've gone very deep on AI and open source.

51:15You were alluding to in the beginning, sort of how AI changes the dynamics of open source and maybe the commercial ability of these businesses. Why don't you outline that? Well, you know, let's go back to the first principles of what does an open source, how does an open source business model, how can it be functional, right? You write some open source software so that people, you can become, you can create adoption for it. Then you build a model where that open source software is hosted and you sell access to it. Typically, the most successful open source models tend to involve some element of computation and some element of storage, right?

51:55Like if you think about, you know, what is MongoDB or what is Confluent, it's storage and compute mixed in with a little bit of software on top. And it's a blend of proprietary and open source software. The challenge in AI, which is very different than the challenge in traditional software, is the cost of compute. You essentially have to train these models. They might be open source, but you have to train them. And the training is super expensive. For a software developer to write some open source code, it's nearly free. It's just my labor of typing. That's it, right? So essentially, the fixed cost of creating open source software is very low.

52:40The fixed cost of creating Llama 3 is incredibly high. It's like$5 billion. So if you have to spend that much money in a rational economic world, you have to get return for it. You have to get return for it very, very quickly, right? In the case of Meta, they can choose to do that because they have such a good business on the other side that they can say, yeah, nevermind. I'm going to put it out for free. But if you don't have that kind of business, why do you put out open source software, open source AI software for free? You can do it once or twice, but eventually you have to have a pretty darn good business model around it to keep doing it because of the training costs.

53:21And I don't see that right now. Like, you know, Lama 3 is great for meta, but it's not a great business. And by the way, for the next player to come along and do the thing that's better than Lama 3 or particularly the bigger models. It's super capital intensive. And open source business models tend to be a trickle. You monetize a very small part of it. If you take a look at Confluent, maybe they monetize 10 % of the Kafka universe. So I have a hard time right now comprehending how, with such high training costs, you can construct an open source business model in a realistic timeframe that covers that core ongoing cost.

54:05Particularly at the high end of the market, to be competitive, right now you're easily spending at least$500 million a year training costs. And if you do that on a cumulative basis, your revenue has to be gigantoid to cover that cost. If you look at Confluent or Elastic or MongoDB, none of those have that kind of cost to upfront, right? So the economics, the math economics have a much harder time working, I think, in AI than they did in classic open source. So at Index, we haven't invested in a lot of open source AI. We have open weights and a few other things, but it's not clear to us that the business model closes, given the compute costs that are required.

54:49And so is Meta doing something irrational? Or do you think they will keep making these gargantuan investments in open source AI? Or how do they expect the math to make sense? Well, it is not totally rational, right? Because when Zuck said, I'm going to spend 10 billion over the next year to build AI, the stock went down a lot. So basically the market is saying, we don't get it. it'll be interesting when the current state-of-the-art is Llama 3 at 70b, the new one that's supposedly rumored to be coming out sometime in July, is over 400 billion parameters. There's a variety of rumors as to how much of that is going to be fully open source versus proprietary.

55:33Meta may keep putting it out for free. They think they can recover the benefits of the AI stuff through their internal services. So that might be something they continue doing. I don't know. It depends really on their business subject, but it will have a huge impact on the market because if they continue to do that, right, if they continue to put out very high-end, super performant models in open source, first of all, it decimates the tail of the market, right? Because they're like super good and free. And if you're some other open source developer and you're competing against Meta, who's piling tons of money in compute resources, very hard to see how that business model works.

56:17And so maybe you can sort of snag what Lama's doing and make it a little bit better and sort of kind of keep up with the Joneses, but it's very hard. So I actually think Meta's strategy is fascinating. I don't know what they will do. It sounds like they're going towards a proprietary path. We'll see. If they don't, they are the open source play that I think is very, very hard for anybody else to compete with the resources they're going to deploy against this. Yeah, that's tough for Mistral and, you know, stability and others that are that are trying perhaps. But, you know, even even Mistral, like Mistral large, which is their big model is proprietary.

56:54Right. So they clearly are have figured it out and they're they're going to monetize through a proprietary path. Got it. Last question. You mentioned picks and shovels, foundation models, applications. You're most excited about applications. Do you have any requests for startups or applications that you think are underexplored or you think could be big opportunities for entrepreneurs listening or things that you'd potentially be excited to go deeper on or explore? I have a lot of fun things that I think we could look at. First, in the language arena, I think that if you think about SaaS, there's usually two axes to it.

57:31One is SaaS that is specific to a given industry, right? Harvey does law, for example. Hebbia does financial services. But there's a lot of interesting vertical industries. So thinking about pharmaceuticals, construction, travel, et cetera, how can AI be used? to build applications that are specific to an industry and have an industry workflow built into it. Would love to see more of those. The other one is more of the functional applications. So applications in sales, CRM, marketing, finance, human resources, et cetera. There are incumbents in those businesses. You have a workday and a sales force, but none of them are AI native.

58:10And so the question is, can you build an AI native functional application? I think those are super interesting. What I see a lot of right now are productivity applications, which are good, but I think generally you'll see more value in either functional or vertical applications. There are winners in productivity, but not as much as you see in the other categories. Would love to see more of that. I'd love to see some of this AI technology being used outside of the classic language domain. Recently, I've seen a couple of things around weather forecasting that use deep learning or AI, which would be absolutely fascinating.

58:43There are use cases around supply chain, which I think could be super, super interesting. Essentially, if you think about what AI really is, it's a pattern matcher. In language, there's a lot of patterns, so it can match them very effectively. But there's many, many things in the world that have patterns that AI can be used to match. And the core technology could very well be just transformers, but transformers applied to different things is a sector that I'm super interested in. And then the last bit, which we like quite a lot, are smaller models, interesting smaller models that alternative architecture, so non-transformers.

59:23The problem with transformers as a VC is that it favors the largest because it's such a compute game. And, you know, I watched the Apple announcements with interest because they're obviously showing the potential of small models. And I think that that's another sort of interesting area. So lots and lots of stuff that we can go take a look at. Great. That's a great overview. And are you a believer like Vinod that the cost of computers just, all this competition is going to lead to the cost just precipitating, you know, going significantly down? I think Moore's law is Moore's law, you know? So I think we're going to see it drop by a factor of two every two years.

1:00:00The recent trends on reduction, particularly in GPUs, has been more dramatic because GPUs were being used for graphics and then they were reapplied to crypto and then reapplied to AI. So there was very little forethought, even within NVIDIA, to explicitly target their chips for AI usage. Now that they are doing that, I think you've seen a pretty significant drop, but eventually I think it's transistor counts and you're going to see Moore's Law. And that's good, by the way. A factor reduction of 2x every 18 months is really, really good. And so we're all going to hugely, hugely benefit from it, but it isn't a precipitous drop.

1:00:39It's sort of the similar drop that we've seen in compute for a long time. That's a good note to, and before I wrap up, I always just ask my guests, is there anything I forgot to mention, or we forgot to get into that you think we should, or does this seem like a good place to wrap? I mean, we could talk about this stuff for forever. There's a lot of fascinating things and our world is changing a lot. But yeah, I mean, we could talk about non-AI venture and what happens to that. That would be an interesting theme. But overall, I think we covered a lot of good topics. Yeah. This was a great conversation.

1:01:13For entrepreneurs listening to this, you'd be very lucky to have Index as a potential partner. Mike, thanks for coming on the podcast and sharing your learnings and wisdom with us. Pleasure, Eric. It was great chatting with you. Terpentine VC is a podcast from Terpentine, the network behind Moment of Zen and Econ 102. If you liked the episode, please leave a review in the Apple Store or rate us on Spotify.