20VC: Are Foundation Models Becoming Commoditised? Do OpenAI and Anthropic of the World Have a Sustaining Moat? Why Smaller Models May Work Better? Why Incumbents with Data Power Win the AI War with Christian Kleinerman, SVP Product @ Snowflake

0:00We've seen companies that with seven employees have creating models that are comparable for some use cases to what open AI or anthropic do. At the end of the day, it's a data problem. And model, they're getting commoditized until the next big innovation comes and you allocate some more value to the model. If anything, model size will influence things like cost and latency, so smaller may be better. Welcome back, this is 20VC with me, Harry Stebings and joining us in the hot seat for this deep dive on Generative AI is Christian Kleineman, SVP of Product at Snowflake. Before Snowflake, Christians spent close to five years at Google as a senior director of product management at YouTube, working on their infrastructure and data systems.

0:40Before YouTube, Christians spent over 13 years at Microsoft, serving as General Manager of the Data Warehousing Product Unit, where he was responsible for a broad portfolio of products. Before we dive into the show's day, you know all those mind -numbing tedious tasks that seemingly take up half your day will code is here with their new AI -powered work assistant that helps you and your team not just finish tasks but make progress so your product team can bring a feature to market faster by using code AI to tag customer feedback, draft PRDs, suggest target audiences summarizing product discussions and more.

1:14Your sales team can engage more customers by bringing in data from sources like Salesforce force and then use Coda AI to suggest action items meeting agendas or lead scores and your marketing team can drive an impactful launch. With Coda AI summarising user insights, creating briefs, generated from notes and writing mock press releases to visualise tag lines. With Coda AI you can reimagine your stu -list and how you collaborate so you're not only finishing tasks but really making progress. If you want to work a system that unless you get back to work. You can get started with Coda AI today for free.

1:46Head over to coder .io, slash 20VC, that's coder .io and get started for free. And to be here of amazing tools we cannot live without, travel in these bands are never associated with cost savings, but now you can reduce costs up to 30 % and actually reward your employees. How do you do this? Well, the Van rule what's your employees? With personal travel credit, every time they save their company money, When booking business travel under company policy does that sound too good to be true? Navan is so confident you'll move to their game changing all in one travel call precarred and he spends super app that they'll give you a $250 in personal travel credit just for taking a quick demo.

2:26Check them out now at navan .com forward slash 20vc and last but by no means least we need to talk cash. As of the 29th of June, you can get 5 .5 % yield on your cash, with 26 -week treasury bills. But buying treasury bills, it's not that easy and you have to navigate a website that looks like it was made before I was born, enter public .com. Their treasury accounts make it simple to earn a high yield on your cash, and it takes 20 seconds. Here's how it works. Sign up at public .com, easily purchased 26 -week treasury bills that automatically the roll over at maturity for a compounding yield, plus there are no minimum halt periods.

3:05You can access your cash at any time, with the flexibility of a bank account, of course, to receive the full guaranteed yield. Of course, to receive the full guaranteed yield, you do have to hold to maturity, but here's the thing, these are T -bills, which means your investment has the complete backing of the US government, making one of the safest places to park your cash. go to public .com forward slash 20 VC to lock in a historic 5 .4 % yield on your cash. You are now arrived at your destination. Christian, I am so excited for this. I've heard so many good things, so I would love to start with your entrance product.

3:43How did you come to the SVP of Proletics Snowflake? Let's start there, Christian. Thank you for having me, Harry. I born and raised in Colombian South America. I did a start -up there. I learned what not to do. I did another start -up in the US. I learned what else not to do. So at some point, I need to learn from the guys that really know how to build software. This is 1999. I joined Microsoft, did a long -standing data all the time. SQL Server. Appliances, when Appliances were the thing to build and then cloud services. And from then, I went over to YouTube, had Google, where I was responsible for the infrastructure, including data systems.

4:20and I think all of that set me up for understanding data, being a data junkie when the opportunity opened up for snowplik and like I could appreciate the technology and the type of company so I'm like I'm ready to be there. It was the charm and charisma of Frank. I don't blame you. I have the same feeling when he looked into my eyes. I do have to ask. You mentioned that you learned what not to do. If there were one on two things that you really learned what not to do, what would they be in question? I would say talent being the driver of truly great outcomes. I would say don't ever compromise on talent.

4:54Don't ever say yeah this person doesn't have the background But maybe has the right intention just take a bet. No, I think that nothing substitutes talent. That's a very clear Lesson learned and and the other thing that it has been very serious Building a scalable business is difficult. Well one of those startups we we built some scheduling software for airlines The thesis was you build it once, works, then you just resell it, and you can be the next Microsoft. It was not the case. There was a lot of customization. It would turn out into more of a service business. So scalability and building platforms was not really worth it.

5:29I have found some lessons learned from respectfully the startups that maybe didn't go to plan, we could say, when you think about Google and Microsoft, that's such symbolic institutions in our environment. What are ones two of your biggest takeaways from 13 years of Microsoft? I mean, shit, it's a long time. And then the four to five years that you had at Google, what are one or two of those big takeaways? From Microsoft, I think I got the ease of use and value of simplicity in products. At the time, SQL Server was coming from way behind competing with market leaders at where Oracle and IBM and the way in was not to have every single feature and capability that those technologies had, it was just simplify things.

6:09If you turn something that is a subset of the capability, but dramatically easier to use, that gets a following. And that was a very, very clear lesson learned. And as you did over and over and by the way, the snowflake story follows a big part of that journey, which is if you simplify things to a point that it is delightful to use, people adopt. That's from the Microsoft time, everything from the YouTube time maybe the biggest lesson learned consumer products have many more elements beyond just technical difficulty there's a lot of timing what are the trends we consider behavior sure you need to have a good business model you have good technology but there are things that you can control is simple always better in product I would say yes you'll say all things being equal there are points where you will oversimplify but I do think that make things as simple as possible and no more.

7:03If you could call yourself up the night before your first role in product before Microsoft, before Google, what would you call yourself up and advise yourself knowing all that you know now? The fundamentals of making something work as advertised make a very big difference. So simplicity is part of it. Even things like latency make a big difference. So there's nothing like the magical experience of you try to use a product. and what are technology or a device or an appliance at home. And if it just simply works and does what we expected, I think that is a magical thing. It's very hard to make happen in all situations, but that would be the big word of advice.

7:41I do want to focus the show on a joint passion for both of us. It's funny, we discussed a little bit in terms of topics to discuss today. And I was very excited when I got your suggestions. And I want to start from the top, which is kind of ecosystem level, and then we'll move down. But it's generative AI is kind of the biggest buzzword of today. So when we think about Gen AI as a buzzword, like top down, how do you analyze the ecosystem today? Is the hype overblown as a starting point? I would say that for sure there is hype. For sure there's a storm war on what are you doing on Gen AI? And people are rushing to somehow stick some Gen AI to their products.

8:19But if you look past that, there is fundamental innovation there. There is fundamental disruption. There is the opportunity to change pretty much every single interaction between humans and computers into something that is friendlier and simpler. So yes, there's fog and there's noise around it But it will clear up and at the end of the day is gonna be a different world. Do you think mobile is a good analogy to AI? Or do you think it's actually more significant? I think it's comparable. I think it's of the scale of the internet. I think it's of the scale of mobile. One of those things where anything and everything will get better.

8:56What do you think is most exciting when you look at the different verticals, use cases, opportunities? What do you think is most exciting? I would say that this is a real shot in the arm to the creative businesses. That's where we saw all the initial use cases and stable deputions or mid -Journey and those things. In many ways, because what in other industries we would call a bug or a hallucination, in the creative world, there are features. So it's a goodness to come up with something that has not been done before or some mix and match of existing things. So creative industries are probably the sweetest spot.

9:33I love what Adobe has been doing with their products and how they've integrated Genai, massive kudos to them. Then there's the opportunity in every other industry and every other vertical. It's only that requires a little bit of understanding, correctness, understanding, data maturity, things like that. But every customer that I talk to these days, they're looking to doing something, they're all trying to figure out how do we get started and how to get there But I think it applies to all sorts of businesses. It's interesting you said that they're kind of they're all looking to do something. You know what I'm finding there?

10:03Because I speak to these enterprises too. They couldn't do freaking clue how to do it, Christian. They're all like, yeah, so it's exciting. How do I do that, Harry? And I think the biggest businesses in AI will be built in the implementation services businesses helping large enterprises Implement AI in a meaningful way into their enterprise over the next decade Do you agree with me in terms of this lack of enterprise education on implementation and the potential opportunity for implementation services given that? I agree part of it is Services implementation education, but I also think that this stack and the way to think about this is evolving.

10:40I don't think that we have a very clean, this is the three or four components that you use and this is the type of use case that you apply. Companies are playing with, hey, the LLM is magical and you send it, send it to questions and magic answers come back. Some others have combined, vector retrieval with LLM's. But within that, there's a lot of variability, which model, which database, or which vector database, how much do you prompt versus fine tune. So I would say a lot of that is still being figured out. I think the stack continues to evolve, will continue to mature and in parallel, okay, then company need to get educated and when to use what for what use case.

11:19I get you. In terms of kind of verticals that are fast as to adopt, I think this is the other thing that we will get a little bit over excited about, which is like speed of adoption always takes longer than people think. Many, if you can believe it, Christian Enterprise in Europe still have no idea what slang is. And so my question to you is when we think about the different verticals, who do you think is first to adopt and the fast movers? Who do you think is slow to adopt? I think it completely correlates with data maturity. You may have heard some of us at Snowflake talk about there is no AI or Gen AI strategy without a data strategy.

11:51It's not just a line, it's a truth that we strongly believe in. From that perspective, I would say financial services are at the forefront. Most of the financials have figured out for a long time how to organize data and leverage data for competitive advantages. Go look at this sophistication of hedge funders and example. Retail and CPG companies have also been very wise at using data who would be on the slowest to adopt. Maybe I would point at public sector. Not that they're not data savvy, but they have a lot of regulation and constraints that may make it harder for them to adopt. How do you think that incentives being a problem?

12:30Point here is like, if you think about health care, health care is largely dictated by government budgets, but by the NHS in the UK or alternative in the US, which is run by politicians, you don't want to replace nurses with AI -driven robots and procedures, because then you'd be firing nurses, and then the front page of the newspaper is Biden or is she seeing that fires nurses. So the incentive mechanism is misaligned to increasing efficiency in healthcare, or in a lot of public services because it likely leads to job loss. How do you think about the incentive function being the problem for adoption versus implementation speed?

13:07I think it may be true that once we're ready to wholly replace nurses and other functions, probably there's an incentive problem, but I don't think that we're there yet. I think of most of the use cases right now around productivity boosts or assistance, copilot as opposed to replacement. So I think of it as I want to help people be more productive. Not a wholesale replaceable. At this point I would say the incentive should be working maybe a year, a couple of years from now what are you saying becomes true. You said there back there's no gen AI without being incredible data strategy. You said to me before, or generative AI's democratizing data access.

13:46He left me with that as a cliffhiking Christian. How so, and why do you believe this? If you think of the role of traditional business intelligence technology, it was to sort of bridge the impedance mismatch between the business and business terminology and business users and a technology that frankly is just for a few people. Writing SQL statements is not something that most people in a company do. And that was a role of that technology to do that translation, but it still requires some mapping and curation and effectively how do you inform that in Pyramids from this match. I think Genai has the opportunity to to recharge this type of translation where the language is natural language and the answers come in natural language, but along the way there's traditional database lookups, traditional retrieval and has the opportunity to democratize data dramatically more than what we are today.

14:39Another BI has not done a really good job, but I think it's going to be now data for everyone. When we think about kind of data as a competitive mode, I've had a lot of people on the show say before Bentley that data is so free to be accessible to stay. It's no longer this prize possession that incumbents can hail and use to their advantage given how free to be accessible to this. To what extent do you still place a premium on data ownership to leverage versus the freedom to access data. Yeah, I would separate there's both public data and private data, privately owned by enterprises, but the premise of your question is based on public data.

15:15And I would say if things did not change, probably the language models for the models in general, would all converge towards they're all training on the same data. And at some point this what makes of data you use, but you'll turn towards the same answer. The interesting trend to what's there is the notion of many companies realizing that their data is being used and monetized by these models. So there are companies rethinking and changing their data policies. Are you allowed to crawl me? Are you allowed to train models with this? I think all of this will shift in the next six to a month. This is all starting already because in the same way that search changed the rules of engagement with public data.

15:57Gen AIs doing the same thing and the companies that are behind that data are doing so. You mentioned that companies realizing that their date is being used bluntly and they're not being able to monetize it and now losing eyeballs because of it on their sides. How do you think about the business model of the future to ensure that they don't lose this revenue despite their loss of data control? I don't know if they're going to be able to keep all the existing revenue, but for sure they should be able to capture some amount of revenue. But clearly, their data is valuable. Right now it's not being paid for.

16:27That says that there's a value gap there. And you see it in some of the license terms that are being floated around. I think it will shift so that it changes the economics of all of this. If you would have placed a value on data versus model, what would you place 100 as your total pie size? Is it 80, 20? Is it 50, 50? How do you weigh the importance in terms of data and model? The vast majority goes to data. 90 plus percent. Certainly there is a lot of IP and technology that has gone into how do you build these models, but that is becoming less of a differentiating aspect and what's becoming bigger is data.

17:04If anything you hear folks doing the math on are we going to run out of public data to improve these models significantly? Why are models as little as 10 and why is such value being placed on the lights of open AOI, BOD and THROPIC, if actually models are 10 and not a significant chunk more. If you look at it where work things are today or where there were six months ago, maybe models would have taken a bigger edge because they paved the way on how do you model the data. You could make the case that it has been there all along, but I would stick to the 10 % or the smaller number because now you see how many foundation models have been created.

17:43We've seen companies that with seven employees have creating models that are comparable for some use cases to what open a higher anthropic do. At the end of the day, it's a data problem. And model is a strong word, but I think they're getting commoditized until the next big innovation comes and you allocate some more value to the model. Cance, what's the next big innovation, do you think? What has happened for language models is coming for computer vision, for images. democratization, how do you simplify it? Then there's the intersection of those true multi -modal languages. Which there are many of them out there, but how do you turn it into, it's completely seamless to go and intersect images, speech, text, all of it into richer human computer interface?

18:28Do you think it is the existing modeling components, your open AI, your anthropics of the world, who chase down those innovations, or do you think it's net new platforms? I'm pretty sure all of them are chasing that. I don't know for a fact, but I'm willing to venture that they're definitely chasing those innovations. What we're seeing right now has generated such a big spur of creativity. Most new startups getting created somehow they all want to chase some aspect of a generic revolution. So maybe it's a hybrid of both. You mentioned that startups chasing it. So many VCs say they're just a wrap -ron top of a GPT model and it's kind of brushed off in that way.

19:06as just a GPT wrapper. Do you think that's fair? And actually these models and the providers will create and kill off all the startups with diverse and close use cases? What do you actually think that VCs are being short -sighted with the kind of handoff of it's just a wrapper? I think there's many categories. There are some very shallow rappers on top of GPT4. I don't place much value on them, I usually ask, hey, how long did you think you'd do build this? oftentimes it's a weaker too. I don't think there's a company there. I do think that there are some very deep rappers on top of GPT -4 that apply domain specific legal or other domain that I think you end up with a true way to bring GPT -4 to a given market or industry.

19:49I think those are value. But Macometan and startups was there are actually many startups chasing the different aspects of of innovation on the core technology. I was at a dinner a few weeks ago and someone was saying, we're trying to blend fine tuning with prompting. Someone else was saying we're trying to address the limitations of the transformer model. Someone else was trying to look at how to do computer vision better. So once you start looking at the core technology of the existing models, I think there's innovation everywhere. There will be a number of new startups creating new things and probably the open -eizon and tropics are continuing their innovation, which those are effectively research companies.

20:30The AMC are in many ways research companies. I think that one of us can say often the size of the model is quite hailed. How important do you think model size is today Christian? I think it depends on the use case for a generic consumer product like Chad's GPT, where anything is fair game, the model is supposed to know about every possible topic. It speaks every language, etc. I would say that for those use cases, is a model that is large and has lots of cumulative knowledge is very valuable. For specialized use cases, which is what I see more in the enterprise, model matters less, model size.

21:07If anything, model size will influence things like cost and latency, so smaller may be better. And now there's plenty of examples that have been run where a smaller model fine tune were a specific purpose or a specific dataset, produces results better than a generic model. So it's use case dependent. Do you think we actually do that? We downsize model size to increase efficiency, you know to reduce latency to reduce cost or do you think we actually just increase efficiency of compute to be able to ingest and work with larger model sizes more efficiently? Both and as I was mentioning startups one of the companies that I talked to was working entirely on model compression only to improve latency and cost So I think there's research and development on both.

21:53Sure, the computer's getting faster and cheaper, but modern models are better. Like if you want to have one or more model calls in the serving path of a consumer product, you have a budget of a few milliseconds to go make things happen and sides will better. It's interesting that kind of thinking about kind of the efficiency, the cost, the latency. I had the normed from character on the show. And he said, the biggest problem that we have is the cost of training. It was like two million dollars to train one single model. How do we think about the cost of training changing over time? Will we see a democratization there which will allow startups to fine tune and train themselves?

22:27Would it continue to be high? How do we understand that? For sure, all of this, the computer is trending down. But the other piece is there's a lot of reinvention of the core training. If you think about how many of the data sets and data sources are common between all the foundation models out there and they're going through the exact same processes for very similar processes. Are there ways to take a common subset and then use fine tune on top of it and avoid the cost? I think we've seen reasonably good results under that path. I do think that both approach wise and complete cost wise cost of training is going to get lower.

23:04Cost of training gets lower. There's also another thing which is like the longevity of models. I have one guest email at stability and he said that no models we use today will be used in a year Do you think that's true depends on how you define a model if you say Lama and Lama two are the same model then yes, all of them will continue to be used for a while If you pin a specific version Yeah, for sure there will be new versions of any of those models cloud cloud two etc But the reality is there will be new models and new refinements on an ongoing basis But we think about those refinements. Companies that are able to transition between models obviously have the most flexibility.

23:44I think it was Alex at Nambler, another incredible founder we had on the show, said the best companies will be able to transition between models at ease and those that can will win. Do you think that's right in terms of the importance of flexibility to transition between models being a core determinant of success? A hundred percent agree. there's so much innovation in the landscape of models that anyone that builds too tightly coupled to a given model is sort of giving up optionality for the future. I just came back from Japan two days ago and a big recommendation I was doing in a large forum was make sure that you build optionality into which model you're leveraging.

24:22Even though it's easier to learn things and out of a single model, I think the optionality matters especially. There's way too much change in innovation going on. What does it mean to make sure you have optionality? What do you do differently if you're thinking with optionality in mind? Well, you'd want to be able to plug in different models, but it's not just replace the model and use everything else the same. Certain models will respond different to a specific type of prompt. In the same way that you have a hardware abstraction layer or a cloud abstraction layer, you should have a model abstraction layer that knows how to translate a specific request that your application needs to do to a model with its in trinketsies or specific characteristics.

25:02In terms of the transition between models, implementation is not that easy. You're not just handing over data with ease. All of the biggest challenges to adoption, do you think, for startups and for companies moving forwards, when they think about working with new models and data migration to them. There are a lot of issues. Probably the most obvious one is around the correctness and dependability of answers. If you ask folks, one of the key concerns is like, all these models make up stuff. So that is the obvious one. But then there are second order issues. Maybe less obvious for people, which is security and privacy of data.