20VC: Mistral's Arthur Mensch: Are Foundation Models Commoditising | How Do We Solve the Problem of Compute | Is There Value in the Application Layer | Open vs Closed: Who Wins and Mistral's Position

0:00Do you feel like you have enough cash now? I guess the start -up is always fundraising. Do you think enterprises are ready for open source? The most technical saving enterprises are definitely ready for it. We know that to widen the adoption, there's definitely some tooling to be brought to the market. What are the biggest barriers to mistrial today? We are still bottlenecked by compute for sure, but that's because we don't have many of it. We have 1 .5 THN, which is a few percent of our competitors. Can I ask, Juanley, was it a mistake for you to not scale that quicker? You can't really scale that much quicker.

0:27You can't raise like 2 billion on the seed round. I mean, at least you could have done it in 2023. Which competitive do you most respect an admire? We are surprised by the **** recently. It's a competitive year on SK. We are always mixing it up with new intro styles. Let me know what you think of that new intro style and what a show we have for you today on 20VC. Mral is one of the most exciting AI companies today at the forefront of the foundation model charge. Joining us is Arthur Mensch, co -founder and CEO of Mral, where he's raised over $520 million dollars in funding from the likes of Andres and horror as general catalyst, light speed venture partners and Microsoft, and before founding Mr.

1:07All Arthur was a research scientist at DeepMind, one of the leading AI institutions in the world. But before we dive into the show with Arthur's day, we're all trying to grow our businesses here. So let's be real for a second. We all know that your website shouldn't be this static asset. It should be a dynamic part of your strategy that really drives conversions. That's marketing 101. But here's a number for you, 54 % of leaders say web updates take too long. That's over half of you listening right now, and that's where Webflow comes in. Their visual first platform allows you to build, launch and manage web experiences fast.

1:43That means you can set ambitious marketing goals, and your site can rise to the challenge. Plus, Webflow allows your marketing team to scale without relying on engineering, freeing your dev team to focus on more fulfilling work. Learn where teams like Dropbox, IDO and Orange Theory, Trust Webflow to achieve their most ambitious goals today at webflow .com. And speaking of incredible products that allows your team to do more, we need to talk about Secure Frame. Secure Frame provides incredible levels of trust to your customers through automation. Secure Frame empowers businesses to build trust with customers by simplifying information security and compliance through AI and automation.

2:24Thousands of fast -growing businesses, including NASDAQ, Angel List, Doodle and Coda, trust Secure Frame, to expedite their compliance journey for global security and privacy standards, such as SOC2, ISO2701, HIPER, GDPR and more. Back by top tier investors and corporations, such as Google, Client of Perkins, the company is among the Forbes list of top 100 startup employers for 2023, and business insiders list of the 34 most promising AI startups of 2023. Learn more today at Secureframe .com, it really is a must. And finally, a company is nothing without its people, and that's why you need Remote .com.

3:02Remote is the best choice for companies, expanding their global footprint where they don't already have legal entities. So you can effortlessly hire, manage and pay employees from around the world, or from one easy -to -use self -serve platform. Plus, you can streamline global employee management and cut HR costs with remote free HRIS. And hey, even if you are not looking for full time employees, remote has you covered, with contractor management, ensuring compliant contracts and on time payments for global contractors. There's a reason companies like GitLab and DoorDash trust your remote to handle their employees worldwide, go to remote .com now to get started and use the promo code 20VC to get 20 % off during your first year.

3:46Remote opportunity is wherever you are. You have now arrived at your destination. Arthur, I am so excited for this. JC introduced us quite a long time ago now. I've known you for a while. I've been wanting to make this happen for a while. So thank you so much for joining me today. Thank you for having me. It's a pleasure. The pleasure is mine, my friend, but I want to start. What would your parents or teachers have described the young Arthur? I'm just always intrigued by the characteristics and traits of the best founders. How would they have described a 9 -10 -year -old author? I guess I was a bit curious and a bit stubborn, not very nice to my brothers.

4:23I was the eldest of them also. I don't know, you should ask them. I think they have good memories, hopefully. Do you know what Sandy or Mother wasn't in our reference list? So we missed that one out. But, you know, JC provided some great commentary. So I do want to start there also, you know, growing up, what was your first exposure to AI? You're a kid in France. How did you first get exposed to AI machine learning and what was that first Passion point that was Andrew Eng flying an helicopter helicopter backwards. It's a control problem Which is not easy to solve and I'm not sure if it was really AI related I think he was saying that he was using a normal network to control over this But that's the one of the first memory of me being shown what you could do with Machine learning at the time that was in 2020 2013 I think Most recently, you spent two and a half years, three years at DeepMind.

5:14What are the biggest takeaways for you from that experience? How did they impact how you think about building Miss Rale? A team of five is faster than a team of 50. Exceptive, sugar organized, the team of 50 should be 10 teams of five that are sufficiently uncoupled. One finding that I learned the hard way at DeepMind and the reason why we created the company in a slightly different way in terms of organization of the science team. And also the reason why we knew we had a chance to do interesting things with a smaller team. Can I just ask sufficiently uncoupled, do you not lose efficiency or is there not leakage between those silos and it creates actually inefficiency by having such silos?

5:54You have to share some things. So you share the infrastructure, you share the code base, you share findings. We are doing general purpose models and general purpose models. You need to evolve them in different directions. So you need to make them speak different languages. You need to make them be able to code, be able to do mathematics, be able to reason, you need to add multi -modality to them. All of these things are loosely coupled. It's useful if you use the same framework for optimization, for data, for training, but you don't want to have your team spend their entire day in meetings for coordination.

6:25It's actually pretty hard to figure out. I think so far we've managed to scale it relatively well. The team is only 25 people, so that's actually not super challenging. It will become more and more of a challenge. That's what I remember from DeepMind. It worked very well at the beginning. JimiNay was a bit too slow and I think they recovered sufficiently well since. We have optimized the team to be as fast as possible and to ship as fast as possible. Was it an easy decision to leave to start, Mistral? You're at DeepMind to one of the best institutions in the World Fair. I was some incredible talent around you.

6:56Was it an easy decision and just take me to that moment when you decided to leave to found or co -found Mistral? So it's not a zero to one decision. It's not a binary decision. You start to think like I'm 10 % leaning on living and then it grows and at some point to cross the threshold then you say, okay, well, I guess, no, that I'm sufficiently decided there's no way in which I stay more than a few days. Otherwise, I wouldn't be candid with my colleagues and so that's how you get started. You say, there's no turning back any tuning. What was that point for you? That point for me was probably around much end of March at last year where I decided to leave on Friday and I resigned on Monday.

7:35You can't stay if you've decided to resign all the way to Stodbury Fair. I totally agree with you. I do want to run this with some chronology. I spoke to so many of your advisors and investors and I want to start with actually the first model, MIR Mischar 7B, being one of the most popular released a while ago now. Why do you think it was What did you learn from that? I think it served two purposes. So the first was to show that there was a lot of slack in compressing models. And so from a scientific perspective, it was a good finding and a good learning from the community. It also filled the gap in the efficiency to performance to the space of models where there was definitely something missing.

8:217B is the size that allows to run efficiently a model on your MacBook or on your SmartSaven. And we made it sufficiently smart so that it was still useful. So there was already seven B models before, but there weren't good enough to do interesting applications. And so by targeting this specific space, we talked to the developers immediately, because developers, like the casual developers, running on a gaming GPU or on its MacBook. So it created a lot of curiosity and adoption because it was a missing spot in the performance to efficiency space. When you look at like lessons from that and how it impacts future releases, any that really stand out for you?

9:03I guess it taught us that there was a lot of interest for efficiency other than scale. And so that's why we continued of targeting very efficient models with the mixed trial at 87B and more recently mixed trial at 8622B, ensuring that for certain then a cost and for a certain size we were reaching the top performance of the market. That has been our major motivation for us to target efficiency. Well, simultaneously scaling towards the larger and larger. I spoke to Sarah Groyd before the show and she said the core question that I think is, you know, with the focus on efficiency and the efficiency frontier, does scale matter?

9:38Well, scale matters in the sense that you should spend more training on computer to make the models more compressed. So you do need to have some computer to compress models. No, scale isn't the only only ingredients to the recipe you need to scale, but you also need to have proper data. Otherwise you reach some data quality limit. You need to have proper techniques for training. I mean people call it computer computer player, I guess. How do you actually make some efficiency gain that are not costing you computer because computer is expensive? And so one of the things that we do at Mralist try and harvest is computer computer players.

10:10Can I ask, in that chasm of efficiency gain without costing more compute, is there much more efficiency we can e -count? Is there a lot for us to e -count, or are we working really at marginal improvements already? I think it's an open question. I believe there is. I believe we can make models that are much better for a certain size. But it's as open a question as can you make a much better model on the same kind of data by making it bigger and training it for longer. Things you need to discover them also on the way. You can try and predict the kind of performance you reach out. You will achieve at the end of the day You need to try it out.

10:44So that's the I mean, it's really much a research field You need to do the research and you need to try things. So I asked Sam Altman this question What is the end state for the model landscape? Most people say ah? It will become commoditized and actually there will be 12 players and it will be erased to the bottom What is the end state for models in your mind and how do you think about the commoditization question? I think the end state is to have more features on developer platforms that allows to do customization, that allows to make low latency models that serve a certain purpose, that allows to evaluate them and to improve them over time.

11:18And so the model is only like a tiny part. I mean, it's a central part, but it remains a tiny part of an application. And what you want to do across time, and when you deploy an application that you expose to users, you want to ensure that it works and ensure that it's latency reduces over time and ensure that its quality increases over time. And so I think that the end state is, models are effectively going to be a starting point for any AI application developer. They need to be surrounded by tools, by a lifecycle management platform, basically, and that's the one thing that we started to build.

11:50Like general purpose models are a bit indifferentiated, but the differentiation that you need to create for your application comes from the data you put into it, the user feedback that you gather and the intelligence that you have to figure out what the application should be doing. And that is not committed to all this. No recipe that allows to go from a general purpose model to model that is super good and better than all of the others at your specific task. This is a missing piece in the puzzle and that's one of the aspects where we're putting our strength on the product side. Sam and Brad are the other day that models just aren't actually that good at any, like yet, and they need to improve a lot in quality.

12:27What are the largest constraints or bottlenecks on model quality today and what needs to change for them to improve? I think the data quality is a constraint. How do you leverage the entire world knowledge and ensure that the model follows a certain path toward learning more and more complex things? That's a very important part and I think it has been a neglected part. There's obviously compute, but given the amount of data you have, we have a tent. Compute is already running into, it's no longer the bottleneck. The bottleneck is more the data at that point. You should get text to text models.

13:01And so the question is, how do you refine the data and how do you feed very high quality data to the model itself in order to improve it over time? And I think in that setting, it becomes a bit, one bottleneck that is associated to bringing better model performance is the question of how do you evaluate these performances. You need to have very good evaluation that targets very specific topics. You want the model to be good at helping diagnosis in hospital but in French. And oftentimes you're a bit out of domain compared to the data you have. And that's where you should identify a gap and you should try and fill it out.

13:37The pushing the model capabilities become also a question of mapping where they're failing and fearing out ways of improving it. For instance, they are failing at mathematics, how do you improve their mathematics thinking, how do you improve the way that demonstrate theorems. The answer to this is very different from the way you answer the question to how do you improve the medical diagnoses in French violence. Well, we see large scale generalized models that are able to answer huge swathes of very complex problems. What do you think we'll see much more vertically specific, smaller, more specific models that are much more vertically aligned.

14:14Yeah, we believe that. And actually these vertical models are not going to be out there. They're going to be built by the application makers because the only way you can make a low latency model that is super good at the specifics is to get rid of the general purpose aspect because a general purpose model is bit bloated, you can think about everything. But if you want your model to think thoroughly about a specific topic, so that you can call it in your AI application while maintaining a good user experience with low latency. What role do you play in that world? If it's actually in the application where you have that specific model creation, where that kind of value occurs, where do you play in that?

14:53It's a very hard job to make a specialized model. So it's actually very tight to the way you create a pre -trained model. And so bringing the tools that allow to do it in a full proofway, so allowing developers to create customized model that are performing very well at that task, but that doesn't require expert AI knowledge, which is hard to find, is definitely something where we're insisting. So I'm an investor today, and I'm pleased that you just said that there will be value accrued at the application layer, because I look and I worry that Blondney everything is going to get steamrolled by some of the players that we mentioned.

15:32How do you answer the question of will value occur at the application layer? And for me, as an ambassador, say Arthur, you know me. How would you advise me? There's two opposing directions. The first is that the models are getting better and better. So it means that creating a verticalized application, as long as you have the data for it and a good understanding of the use case you're facing, is going to be easier and easier if you have access as to the tools that facilitated. So that's the first aspect, which would make me think that the application layer is going to grow thinner and thinner.

16:01But then there's also the fact that the models are getting cheaper and cheaper because we managed to compress them, because we make a lot of improvement on their efficiency. And so that means that effectively, vispeless the competitive pressure there is on the model layer means that the price around the model, the dollar -parent intelligence unit, let's say, is definitely going to reduce. So there's two aspects of growing ability, compressed price, which on one side says that the application layer is going to grow thin and on the other side says that the model part is going to grow thin. So for us, the project that we are taking is that the model part is still going to be big enough and that we need to build this platform on top of that.

16:41Because that's where we are going to enable all of the vertical applications that will be interesting for humanity. How do you think about that positioning in brand? Because there are other players who are much more direct in saying, hey, we're going to dominate a lot of different verticals and kind of be afraid. How do you think about that, enable a two vertical applications or not in that position? We are not a verticalised company. We started to bring value to developers and to bring freedom to developers. So when we started, there was basically one API out there, soon too. And the field of generative AI was starting to look like it would be very centralised around a couple of players.

17:20And we took this platform approach where the model that we're making and the technology that we are making, we are allowing developers to own it, to modify it. And so bringing freedom to developers and AI application makers is I think the best way in distributing generative AI as widely as possible, which is our objective as a company. Making AI ubiquitous, bringing frontier AI into everyone's head is the reason why we started. We did a good job at it, but this open source part was, I believe, a good enabler for the community and made people realize that they could build very interesting technology by modifying the models themselves, instead of depending on the APIs of a couple of providers.

17:57Dude, what do AI developers care about? Everyone kind of gets on Twitter and goes, oh, did you see Axe's performance this week is better than Y's performance last week? What do they care about? Efficiency, scale, cost, what drives that usage and decision -making? They care about cost for sure. They care about customization, being able to modify the models that will. And on that aspect, I think we are only scratching the surface of what can be done. Like the fine tuning aspect that has been like the go -to solution. It's probably a little too low level from what we should be doing. They care about being able to deploy anywhere.

18:35So they operate in a certain space, in a certain cloud. They might be operating on prem. They might have some edge devices to deploy to and they want to be able to put that technology there. And so they also care about portability, which in turn offers data control. Usually LLMs AI becomes very useful when you connect it to knowledge bases or to anything that is related to certain business. In that respect, it becomes a very sensitive part of your application because it sees everything. It sees all of the data you have. And so enterprises, for instance, do care about ensuring that the property data they have is accessed in something that they can completely secure.

19:15And that's the reason why we deployed our platform on Azure in AWS, for instance, that is bringing the security layer that they need. We're going to get to enterprise. Can I just also just brand matter in this segment? You know, when we think about building brand, both in terms of developer adoption brand, corporate brand, is brand a large determinant of adoption in this segment? Brand seems to be critical and this is something that we have learned on the way. We use certain models because they are known to be good. You can't afford to evaluate everything out there. And so having some form of community watching is super important.

19:53The part we took with APHD distributed models has contributed to what I think has become, Well, at least a known brand and we believe that it's definitely going to be important. Brand is important because trust is important in that domain. And open source brings trust in terms that provide some trusted brand. You mentioned the word open source, then. I'm going to get to that. I do just want to touch on that you mentioned cost also. I want to touch on cost. How when and who will make marginal revenue that exceeds marginal cost in LLM based products? You should be telling me you are the investor.

20:30You have been your own company. That means I know nothing. I know who is doing the most margin at the moment. It's probably going to be over a bit more time. Who is doing the most margin at the moment? Nvidia is at that point. The club providers are pretty much at cost. And the club providers, we are not at cost. Hopefully, but the margin that are known to be lower than the typical software margins. AI application makers. Some of them, the one that almost used, seems to be doing a pretty good margin. I think it's going to be quite a moving space. As I've said, the capacity of models makes the cost of making an application lower and lower.

21:06I don't think there's any way in which the margin at cost and the margin of the most important part of that technology, which is really the foundation layer, become zero because otherwise there's definitely going to be a furnace problem. What do you mean by the furnace problem? Talk to me about that. Usually the value tends to accrue where most of the difficult part is and most of the defensibility is. It was for a while it has been on foundation on what is. I think it's obviously evolving with time and there's no mode that isn't disappearing or evolving with time that will remain the part where most of the innovation will be made and where most of the, well, it is the significant part of the crude value will there.

21:43Well, the value will accrue. Is there actually much of a barrage creating a foundational model company today? I know that's a really broad, stupid question in many respects, but you have so many different players now and new ones popping up every day. Is the barrier just reducing day by day? I don't think it is. To be relevant in that space is a very hard topic. You need to be dominating on the cost efficiency performance by ToFront and there isn't, there's only a few companies that are currently well positioned. So you can try and do something but if it's not relevant, if it's strictly dominated by another model or another technology, then you have a problem.

22:18There's a few barriers that are pretty hard to face. You need to raise sufficient capital to have enough compute and be relevant. You need to have people that knows how to train models, which is still the scarce resource. And then you need to have a good brand, because as you've said, it's highly competitive. And this is not something that comes out of thin air. So I think there's still a lot of defensibility on the market. Although there is a lot of noise, which is different. How quickly does the cost of compute go down, do you think? Because if you look at those things, actually, you said cost of compute, access to talent and brand, if we drastically bring down cost of compute, like many think we will very quickly, you've got access to talent and brand.

22:54Two of those are more doable. The cost of compute reduces over time, just based on hardware costs. It reduces around 30 % every two years if you show low Nvidia on map. The other thing that creates is the efficiency of algorithm. So if you look at where we train models from three years, three years ago, and the way we train today, I think we have probably made something around the 100 times algorithmic improvement. That's probably where most of the gains were actually made in the last three years. Obviously the cost of compute does reduce, but it doesn't reduce faster than the more low. So our bet is more on efficiency where I think there's a lot of improvement that can still be made.

23:32Given Nvidia's prominence there and Nvidia being the one where the gains are, As you mentioned, is bluntly one of these single most important things, not simply the quality of your relationship with the core provider, being as you're or being in video or being one of these players. Is that not the core determinant of success today? I guess it's an important aspect. There is a strategic dependency from the AI layer on the club providers and on Nvidia. The competition is heating up as well, but it's effectively important. It's effectively useful when you develop a software to also know the hardware provider because they can help you out In optimizing for the hardware issues when you're selling your developer platform to enterprises to bring that platform through their usual provider Which happens to be a cloud provider.

24:18So there's definitely some important collaboration to be made there When my Amazon invest like two billion dollars in anthropic or whatever it was Is that not just like a trade where like anthropic then spend one point eight billion dollars on Amazon and return it back to them. Do you see what I mean? Is it not a bit of a misnomer? It looks like you're on flipping, yes. I don't know about that deal particularly, but it makes sense from both perfectly. Can I ask, how does the unlimited availability of open -source LLM's impact the answer to the above being marginal cost and marginal revenue? Does it change much?