20VC: AI's Biggest Questions: The Commoditisation of LLMs, Open vs Closed: Who Wins, Model Size vs Data Quality, Why Google are Vulnerable and Apple are the Dark Horse

0:00Welcome to 20VC with me Harry Stabbingz and for this Thanksgiving special, I wanted to bring some of the best minds in AI together for an amazing panel. The only thing is, this panel never really putnt. What you're about to hear is the leading minds in AI from the head of AI at Meta, the founder of Intercom's, stability, runway, leading AI -invested Tom Tungers, all debate some of the core questions in AI. I pulled together some of those best moments and the most contrarian elements from their different episodes. Let me know what you think of this different style and format. I think it's really special and cool.

0:33You can do that on Twitter at Harry Stebnings. You have now arrived at your destination. So I want to start at the very core, the foundational model layer. And I want to ask, how do we see this layer in terms of the foundational model providers playing out and what we see the commoditization of LLMs? E -MAD, I know that you have some strong opinions here. So e -mad at stability, handing the mic over to you. I think that there's only going to be 5 or 6 foundation model companies in the world in three years, five years. I think it's going to be us and video, Google, Microsoft, OpenAI and a meta and Apple, probably, other ones that train these models.

1:08What about you, Dez? You're the founder of Incom. Do you think we'll see the commoditization of LLMs? I don't know if it's actually happened yet. If I'm clear, right? Like, we actually torture test all of the LLMs. It's not yet the case that they're all equal. And when you can pad open AI to the all -time providers, what did the test show you? It's basically a quality of conversation and does it fail? Any of our hallucination tests? Does it fail? Any of our trustworthiness tests can it infer its own confidence? Was it close? Was there a way? Yeah, it's close and narrowing and it's also a work in progress.

1:38All of these things are moving targets, right? So we haven't even gotten around maybe testing the latest and greatest of all the providers, which are increasing a number. You mentioned me, Australia is also a glamour, there's an anthropic, there's a co -here, there's like a whole chunk of them. And it's a bit of work to go around and constantly be trying to find out as anyone got them. We only really care about better, right? Right now we're not in cost optimization mode, we just like who's got the best? Jeff Sybert, you're the founder of Digit. So you sit on top of these foundational al -alems and then fine tune them with your own data.

2:07What do you think in terms of this commoditization of the foundational model layer? I certainly think we will. And this may not be a popular position, obviously opening eyes, charging ahead, sort of leading the way right now. I think the market forces at work mean there's just immense energy to have an open source equivalent. Meta appears to be highly motivated to open source its work. Many folks want to run these themselves and tune them themselves and so on. That is hard and expensive today, but I can't think of another thing in time and history where something hard and expensive in tech has lasted all that long.

2:39It's going to be commoditized. Okay, so if we go one layer deeper from just the commoditization of these models, to actually how important is the size of these models? And then how do we think about the lifespan and longevity of these models? Ima, I know that you have quite a few thoughts on this, I'll start with you. The reality is no models that are out today will be used in a year. So again, you see the order of imagined improvement. Palm last year was 540 billion parameters, then Chinchilla 67 and now 14. 540 to 14 is a big step. You see the quality of G53 versus G54. Cool. Is there an extent to how low it can go?

3:11We have no idea. You already said this is impossible. Two years ago, you're like, no way. You have a single file that's maybe a few hundred gigabytes that can pass every exam apart from English. There is no such thing as an unbiased model. Dali too, when OpenAI had that, they introduced a bias filter. Any non -gendered word that ran a random gender and a random ethnicity. So you type in Sumer -Slo and you get Indian female Sumer -Slo. This is where you need national data sets. You need cultural data sets. You need personal data sets. that can interact with these base models and customize to you and your stories because you and I both have our stories that make up our psyche and understand that context is so important to have a eyes that can work for us, not on us.

3:49Jan, help me out here. You run AI at matter. We heard their cultural data sets. We heard national data sets. Just start. Jan, how important is the size of the model first? You don't need those models to be very large to work really well. And I think it caused a bit of epiphany for a lot of people realizing, oh, okay, maybe you need 1000 GPUs running for a couple of weeks to train it, the base system. In fact, that number is going down to two because people are figuring out how to do this more efficiently. But once it's pre -trained, you can use it for all kinds of stuff and you can fine tune it really easily.

4:22And then at the end, you can run it on your laptop, right? That's going to be amazing. Or maybe on a desktop machine with a GPU in it or a couple GPUs. So I think it opened the minds of people to the fact that there is like enormous opportunities that really weren't Thought to be possible before and I think it's gonna make even more progress because if we go towards the design of AI systems Perhaps along the lines of what I described with objectives and planning. I think the system could actually be even smaller to some extent That's so interesting, y 'all so you said there about it becoming less important to have larger and larger models I'm really intrigued rich at such a foundry you I know that you have a different take on this, so do you think that it is important in terms of the size of the models themselves?

5:04It is super important. You just cannot train a single model for all of these different tasks with a small model. That's exactly why and how it would have always failed in the past. Hmm, some different opinions there. Chris, a founder at Runway. What are your thoughts on bluntly the defensibility in terms of the size of the models being used? And how do you think about that? I found myself hearing a lot about models as the mode and mode has been something that Silicon Valley has been discussing for some months now. I think models are not a mode. Models eventually don't matter. What matters most is the people building those models and how fast can you change and learn from those models.

5:44So I don't think that's why I go back. There's no one similar model that's going to rule them all. So models are not the mode. Now Jeff found it digits he we heard from earlier. I know that you may be disagree with this slightly. So do you disagree with this in terms of models not being the moat? And how do you think about that data size and quality also being a moat? At the base LLM layer, the data size so far has been very correlated with performance. And so, right, the bigger the models, the more data, the more parameters, etc., the better they do. Now, they're starting to be this counter push of like, okay, can we compress them?

6:16Can we pull that back? Like, how do we maintain the performance improvements without the size? So I think that's a super interesting part of R &D. What I'm talking about is sort of the next tier of how do you fine -tune the models? And that's where actually I think the quality of data is most important. I mean, I think before we go deeper in terms of data quality and data size, I just want to ask in terms of the models themselves, there's a core challenge today in terms of two opposing ideologies, which is open versus closed. Yann, you know, you run all things AI at Facebook or meta. How do you feel about the open versus closed discussion?

6:48I know you've got some very strong opinions. Why does the future have to be open not closed? It's very simple. It's because no outfit as powerful as the MAB has a monopoly on good ideas. If you do it in the open, you recruit the entire world's intelligence to contribute to things and having ideas and ideas that you might as have thought about, which an outfit with 400 people has no chance of thinking about, or even a large company with 50 ,000 employees. may not want to devote any resources to because they may not take it so useful in the long term or they have more urgency to take care of. So you give it away and then you have tons and tons of people, some of whom are undergraduate students or people you know in their parents' basement so coming up with amazing ideas that you would never have thought about or willing to spend the time to crunch down the seven billion -weight lama so that it runs on a Mac on a laptop.

7:37I think that's why OpenSource projects succeed particularly when they're concerned, basic infrastructure. Now I'm really intrigued to do a other founder of Contextual, which is essentially a contextual foundational model. How do you feel about this? Because I know that you do have some opinions in terms of the open source versus closed. So I'm a big fan of open source, right? I would like it to be true that with open source, we could just keep up with all of that. But I think that's just incredibly naive. Open AI has this very deep understanding of how people want to use language models. Basically, nobody else has, and they have this giant economy of scale where they can serve up language models very cheaply because they get so many requests coming in at the same time.

8:21So they have a giant mode. Richard, founder at you, you were Chief Data Scientist at Salesforce before. I'm really intrigued, how do you feel about this lead position that OpenAI has that do -age is illustrated there? And what that means then for the potential for an open source competitor to rise? I mean, certainly you can't deny that opening eye is ahead by a lot. I predicted that we'll have a GPD4 equivalent model before the end of the year that's open source. Of course, GPD4 keeps getting better and better, so my prediction was for the version we had a few months ago, but I actually think that with models like Lama 2 from Facebook and everyone there, I do think open source will take over a lot of use cases, right?

9:04It's already getting close to GPD 3 .5. When there's this much excitement in so many careers, depending on understanding these models, imagine all the researchers in all these universities, they're all the sudden out of a job unless they have an LM that works really, really well, and they can do useful things with it. They're not going to just say, oh, let's just from now on, run our entire research agenda on some closed API that we cannot analyze and understand and improve and publish papers on. So they need to have a model to exist and those are all very, very smart people now that don't have as many resources usually.

9:44They can train a single model for like 20 or 50 million dollars because they're in universities, but they're finding ways they're collaborating and they're probably going to work on foundational models that are fully open source. And we now see this with like surprisingly Facebook being at the very forefront of it. People will layer on top that make it better and then there will be open source versions you can run on your phone and those will get better and better over time and so I'm quite bullish on the LMS in particular getting more and more commoditized and yes there will be a few foundational companies you know who are here in a traffic also behind opening eye working very hard to catch up with them and they did raise a lot of money too and it's good to have some competition in that space but my hunch is a lot of people will be okay with an open source model too.

10:32Okay, so some different thoughts are in terms of open versus closed. I think the next big debate is where does the actual value of cruise is in the infrastructure layer or is it in the application layer? Now, Tom Tungus, you know, you founded Theory Adventures, which is one of the leading AI investors. When you think about infrastructure versus application layer in terms of where value or cruise, how do you think about that? I ran this analysis, so in Web2, if you take the top three clouds and you look at their market cap, so AWS, GCP, and Azure, it's about a 2 .1 trillion dollar market cap just for the cloud businesses.

11:03And then if you take the top 100 publicly traded cloud companies both on B2C and B2B sides of Netflix and ServiceNow, they have equivalent market cap about 2 .1 trillion for both. So once that the infrastructure layer wants that the application layer, market cap is basically equivalent. The difference is the infrastructure layer, there are three businesses, and at the application layer there are 100. If the analogy holds as an investor, the odds of success are going to be significantly higher at the application layer because the diversity of needs there is greater. That's so interesting to hear about the concentration of value going to those three core providers in the infrastructure layer.

11:33There's obviously with income, you sit at the application layer. How do you think about this question of where value occurs in for structure versus application layer? I think right now a lot of value is going straight into the infrastructure. Like as in we're handing it all back to our open AI in this case. In your first question about commoditization, if that happens, then the value starts to a reduced there, right? Because if there's more competitors offering the same thing, that's the beginning of commoditization, or at least you go from monopoly to an all -agoplete to ultimately perfect competition where you, you know, on price goes down each step.

12:04That's what I expect that'll play out. Unless openly I can continue to find mass market differentiation, which is always a hard thing to find, right? Like, like, there's any differentiation that all the customers care about, not just specifics, right? So let's assume over time I believed that like, just, you know, the amount of people investing in the space means that I think the infrastructure will gradually get a lot more AWS -like, right? Like, just like Razer 10 margins, well, sorry, not Razer 10 margins, that's the only way for him. It's price mixed with a strength of product as a competitive weapon, if you know what I mean, right?

12:33Like Amazon make good money off Intercom, you know? But they're also like, we're all in on them, and we're very committed to them. So that's what the competitive battleground will probably be true or four big providers is my guest. Probably GCP AWS, maybe OpenAI Directed, I'm sure as your, that's how I think that Larry will play out. and they'll end up in direct price competition with each other. Where does the value accrue beyond that? I think just differentiation, value just generally follows differentiation around the stock. Whoever has the stuff you can't get anywhere else, can charge the margin state, no one else can charge.

13:04That sometimes means quality of software products, sometimes it means quality of network or social network, sometimes it means we've got the most distributors in our marketplace, but whoever has the differentiation to no one else can get is the person who can actually basically charged the highest price. That's so interesting. As you said there about price, you said about margin. I think another big question is bluntly, what does the pricing model and the business model look like for the next generation of AI? I want to bring in Miles Grimshaw at Benchmark, General Partner at Benchmark for this one.

13:32So Miles, what do you think in terms of the next generation of both pricing and business model for AI? If AI is to be the force that it can be, I think you will get a new architecture and new business model emerging from it versus what we only see right now, which is kind of a sustaining architecture, which is just that of a co -pilot, which fits on top. What is the new architecture and what is the new business model? I think the way to encapsulate would be this idea of selling the work not the software, and that we'll move from a paradigm where you might think of it as moving from what we see right now as a co -pilot and moving to what I think about as a control center, where we'll sell an SLA on work, not an SLA on uptime.

14:11And so we'll move from a world where we all, as users of software, kind of like monkeys doing data entry, usually like what's most software? It's a database with a form on top of it for users to manage information, put information, and get information out. And so an example is use that your objectives on your marketing spend and what you want in terms of CAC and LTVs, and then actually as marketing efficiency engine will go across channels, spend, and deliver you back results. That might be an example, right? And so we'll move from a world where the users are doing all this work to a world where the application is doing a lot more of the work, right?

14:49Where the AI, the notion of agents inside of it, etc. is doing it and where right now, like you got any SLA for any software, you get up time, you get support SLA for questions and things like that. I think there's a world where we move to in the future where like an SLA maybe almost looks more like a BPO would in some sense. an SLA will be you wanted a efficiency of X on your market. We delivered that. You wanted this many leads from an SDR team like we do that. You wanted this sort of accounting and books closed by two days at the end of a quarter. We'll give you an SLA on that, not on the software's up.

15:21You'll go from Copa to controls and it'll be a UX for a worker which is dominant to UX for managers. You'll go from a seed add -on to software and labor and you'll go from SLA on like the liability to SLA on outcomes on work performance. That's what can be offered up by this. It's, you see very little of it so far. I think AI is offering the potential for that architecture shift. And if we get that, that the whole seat model, the whole the work, it does it. And the product paradigm, the distribution in terms of who you can reach changes because a CV's change. And in that way, I think it will be very different to mobile, which is mostly another UX, but the same architecture, same business models.

16:00Des, as we know you're the founder of Intercom, you're at the forefront of customer service. How do you feel about the business model that will be prominent in this next wave of AI and how you think about that today? I think a lot of work is going to get handed over to Al Alem's over the next five years. We'll start trying to price against the work that's being done, not price against the seats or the employees, which is say, how much is it worth for you to have all of your digital assets created dynamically or how much is it worth for you to have like your customers get sub -second replies to common questions.

16:31That's the actual right way to think about pricing in the future. I want to bring in a new voice here. Christian Lang, you're a former founder of TradeShift. Now, the president or co -founder of Beyond Work, I have to ask, how do you think about this consumption model pricing versus seat -based pricing question in the future? But I do think the world is changing. I think we've got to absolutely move to consumption -based pricing. It's the only way I think it's going to be very hard to hold the mode on recurring revenue as it is today. because customers would want to see more value and they'll want to see a more soft ramp up to that value But I also think on the work piece look at the demographic problem like most of the people I talk to even if they wanted to replace the workers They haven't as yet so they send us today one to one they can't because that generation of young people They don't want to go in and sit in front of a computer and type formulas into work day everyday So so we got to completely change the work experience in next five to ten years So we're gonna run out of people to do the work So everyone seems aligned that we're moving to consumption based pricing away from seat based pricing.

17:26But Jeff, so I bet it digits, we have one who thinks otherwise. So Jeff, I want to move to you. This is great. Why do you think that maybe we'll stay in the realm that we're in today? So there's maybe just me, but I, I very much see AI as a tool, not a product. And so it's a, it's a technology. It's like your database. It's like memcash back in the day. And so because of that, I don't think it'll change how people price in specific industries. Like if your market does Percy pricing, that'll probably stay. If your business and product does consumption pricing, that'll probably stay. And you'll have to work that into how you use the AI.

18:01I think it'll be commoditized and seen as technology within a couple years. I do have to ask, we see co -pilots everywhere. Miles, you mentioned co -pilots earlier. How do we feel about the rise of co -pilots? Are they an incumbent strategy? How useful are they? Christian, I wanna start with you. Christian Lang, over to you. How useful are co -pilot? Is there an incumbent strategy? I mean, who wants a co -pilot? I wanna be a pilot. And I wanna have a pilot. Like I don't wanna have a co -pilot. I don't wanna be inside an application. I don't care how many co -pilot you have. The problem is not to have an AI help you navigate at a location that's shit.

18:38The better solution is to remove that location that's shit and just talk straight with the AI. And I think the co -pilot metaphor, right? I mean, what happens when we have 10 ,000 clippies all just talking to each other? And for you to work, you have to go and working with all of these clippies and you're gonna have to tell Clibi A to go talk to Clibi B about that thing you have on when Clibi C. I mean, it's gonna be worse than it is now. Miles Grimshaw or Banchmark, you were the first in this discussion to bring up co -pilots. All co -pilots in incumbents strategy. I think co -pilot is an incumbents strategy.

19:08Incomments own distribution, their own data, They own the UX and they own a business model that all aligns to a copilot as GitHub copilot, right like in line Suggestions think of it like how most go to go to any Microsoft product right now every Microsoft product now is a copilot experience And so for sidebar an auto fail things like that right where the UX the core product is a layer on top of it Right, it's immediately added in which is also totally incumbent strategy and it's still about sort of supercharging that work but still where every user has a seed and every user during most of the work.

19:41And it works probably, you know, if you think about the evolution here, the models, most of what's rolled out, might not be good enough for some of this yet, right? But that's what will come around the corner. You know, if you think back to Salesforce, disrupting Seable, Salesforce launched like five years after Netscape launched. Like it might take a moment for that to happen. But the co -pilot, this idea of, I'm still the pilot, I'm still the user controlling everything, and it's sort of like giving me assistive suggestions like GitHub co -pilot, fits into the UX of incumbents, if it's into the business model, of incumbents, and they already control that distribution.

20:12The opportunity offered up to a startup, being a co -pilot for something else, like probably won't be that amazing. And there may be pockets of it where it can really work, but the opportunity to disrupt is to be orthogonal to the incumbents. Jan, we said there about it not being the best position to maybe be in for startups. Christian mentioned, we don't want a co -pilot, we want a pilot. But how do you feel about copilot's and intelligence assistance? And is it brighter than maybe we think in terms of what we have coming? Let's imagine a future where everyone can talk to their intelligence system.

20:44That system will have pretty close to human -level intelligence for a little more accumulated knowledge that most humans, you know, they could translate in any language and give you a quick summary of yesterday's newspaper and things like that, right? Explain mathematical concepts to you, things like that. So people are probably going to use this almost exclusively in the future for their interaction with the digital world They're not gonna go to Google or Wikipedia. You're just gonna talk to your assistant The only way to do this properly is for the basic infrastructure for those assistants There would be so pervasive so much will ride on those systems that I don't think anyone will accept that those assistants be behind the Event horizon in a private company There will insist that the infrastructure is open.

21:25There will insist also that the vetting process by which those systems are trained be something maybe like Wikipedia. We tend to trust Wikipedia, sometimes with the grain of salt, but we tend to trust Wikipedia because there is a vetting process so that whenever an article is modified, some editor can check on it and then the changes are accepted or not, things like that. So you can imagine that the common repository of all human knowledge that will be or assistance will be constructed through some sort of cross -sourcing process perhaps similar to Wikipedia, where you're going to have a bunch of people training those systems and fine tuning them so that whatever they, and so they produce are correct.

21:59Yeah, and you mentioned Wikipedia, I do want to move to the company level element and discuss which companies or incumbents are best positioned. Now, I want to start with Apple and, Des, I know you have some strong thoughts here. So, what are your thoughts on how Apple are positioned for this next wave in the next three to five years of AI? I think Apple will make massive, massive strides forward, but I'm kind of disappointed how long it's taking them. But I think - This is my point. What makes you say that? Because so far we kind of left searching. Yeah, yeah. For sure. You have to assume Apple's a really well -run company.

22:29And you have to assume that there's a head of AI in there. And you have to assume that you're training LLMs and you're looking for LLMs that can possibly run on the hardware natively. Not even have to talk to the cloud and Apple are very privacy focused so they're going to get all that shit correct. And you have to assume that it's all going to work with your AirPods, your watch, and your phone, and lots and stuff. that that's like I would be shocked if that wasn't the case. So then what will they win is the question. I think what they'll win is this idea of Siri might finally become useful. Siri is currently not useful because it doesn't really have enough smarts.

22:56But I think when Siri can be as conversational as chat GPT and can take actions under the device, it'll change the entire interaction model across desktop and iOS as well in huge ways. So I think Apple will win their E -Mabbit stability, you would dare say Apple will win that. Or do you have a different perspective? Apple's a black box, right? And so they could surprise us all, but let's face it, series crap. But they have all the ingredients in place that entity architecture, the secure enclave, other things. Neural engine, a stable diffusion with the first model ever optimized on the neural engine, etc.