Mistral 7B and the Open Source Revolution With Arthur Mensch, CEO Mistral AI

0:05Open source AI models have completely changed the landscape of technology over the past year. One tiny team of ex-DeepMind and meta researchers in France has made a huge splash recently, Mistral. This week, Alad and I are joined by Arthur Munch, the CEO and co-founder of Mistral, who recently released Mistral 7b, an Apache 2 licensed open source model that has changed people's mental models about what can be done with small models. Arthur, welcome to No Priors. Thank you for inviting me. Very glad to be here. Okay, so just six months ago, when we met, you were leaving DeepMind to start Mistral.

0:42It takes real guts to look at the scale of dollars and compute that OpenAI and Google and others have amassed and say, like, we want to play in this game too, and it's important we do. Tell us about the inspiration to start. Me, Guillaume, and Timothée were, I guess, pretty early in the field. and it had been 10 years that we had been doing machine learning, and we did know where to start from and how to make a good model with a limited amount of compute and money. Well, not so limited, but at least more limited than where we were coming from. And so I think that's why Getter started. The various companies we were in moved into directions that we hadn't anticipated when we joined the company, and we decided that there would be a very good opportunity for creating something that would be a standalone company in Europe, focusing on making AI better, focusing on making frontier AI, and focusing on making open source AI as a core value.

1:41Maybe we can talk about each of those pieces. So 10 years in machine learning before, you were a co-author on the Chinchilla Scaling Laws paper. You worked on the sort of mixture of experts ideas early. Can you talk a little bit about what your research directions were at DeepMind? Yeah. So I come from an optimization background. So my focus has always been for the last 10 years to make algorithms more efficient and to use better the data that we have to make models with good prediction performances. And so when I arrived at DeepMind, I joined the LLM team that was 10 people at the time. And very quickly, I started to work on retrieval augmented models.

2:27with a paper called Retro that I collared with my friend Seb Borjo, who is still at DeepMind. The point there was to use very large databases during pre-training so that we didn't force knowledge into the model itself, and we would tell the model that it would have access to an external memory anyway. And so it was working quite well. We could actually lower the perplexity, let's say. That's what you work on when you make LLMs. There were some limitations that I think the community has started to address quite well. And that was at a time when retrieval methods weren't really mainstream. Now they've become completely mainstream.

3:07So that's the first project I did. I worked on a sparse mixture of experts also quite quickly because that was related to my topic of postdoc, which was optimal transport. So optimal transport is a setting where you have, I guess, tokens. You need to assign them to devices, and you need to make sure that there's some good assignment in between the two of them so that the devices don't see too many tokens. And as it turns out, the way you do it is with optimal transport. Optimal transport is a mathematical framework to do it correctly. And so I started to work on introducing this two sparse mixture of experts.

3:46and very quickly we started to move on to scaling lows. So how do you actually take the method that is working at a certain scale and try to predict how that will evolve with the scale, the number of experts, the amount of data you see? And so that's a work I've done with many colleagues as well on how do you adapt the scaling lows for dense parameters, for dense models, to a setting where you want to predict the performance not only with relation to the size of the model, but also the number of experts. I guess that was the second thing I worked on. And then connectedly worked on Chinchilla, which is, I think, a major paper in the history of LLM.

4:30Also with Seb, Jordan, Laurent, many other people. Basically, the story was that everybody was training models on too few tokens. because of the paper from 2020 that happened to be not very well executed. And so what we observed is that you could actually correct that. And so instead of training very large models on very few tokens, you should actually grow the number of tokens as you grow the size of the model, which if you think about it, makes a lot of sense because you don't want to have infinite size model looking at a finite number of tokens. And similarly, you don't want to have a finite size model looking at an infinite number of tokens.

5:18There must be some proportionality, yeah. Yeah, exactly. And that's something that we showed empirically. And I think that's something that was adopted very fast because it was like a pure win. For the same amount of compute, you would get a model that would be better, but also a model that will be four times cheaper to serve. So that was definitely a gain. And as it turns out, we didn't go far enough. That's what we did at Mistral. We realized that there was also a lot of opportunity in actually compressing models more. I mean, we've seen with LAMA that it was actually possible. What we showed with Mistral 7B is that we were definitely far away from the limit of compression.

5:57We somehow corrected that by making a model very small, super cheap to serve, super fast, running on your MacBook Pro, but still good enough to be useful. And so that's one of the first achievements we made in the company. Yeah, I think a lot of people were really impressed when Mistral came out with the 7B model because A, you did it very quickly. You know, you did it in a matter of a few months, but B, if you look at the cost of actually running these models, obviously there's the training side of it in terms of actually building the model, but then there's inference. And so much of the cost day-to-day, if you're a user of OpenAI or a user of LAMA, are you a user of Mistral is how much it costs to actually run the models, the inference.

6:36And that's often driven by the size of the model. And so I think by coming out with these small models that were very performant, it really made a huge change to how people thought about what was possible. Is there anything that you can share in terms of where you think this performance is going to go or how you think about the sizes of models, both in terms of smaller, more performant models, as well as do you folks plan to build something very large, more like a GPT-4 or GPT-5 equivalent over time. Yeah, sure. So I think what we've seen, like in 2022, 2021, you had a few companies that were really focusing on pushing the performance of models.

7:13And if you want to push the performance, the pure performance of models, you don't care about inference because you're, well, you are not going to use the model. You're just going to see whether they're good or not. And that's really for scientific purposes. But then when you start thinking about deployment and enabling downstream applications, then you need to think about what it is going to cost in runtime. So you're not only worried about the upfront payment you need to make to get the model, but you're also worried about the runtime. And so I think the coefficient that you put in between the inference cost and the training cost really is really business dependent.

7:47And as a company that intends to have a valid business model, we think a lot about inference cost. We think that it's super important to get to a regime where inference is super cheap so that you can run agents, you can basically use AI, LLMs everywhere for all of your use cases and you're not blocked by cost, which is the case for the largest model currently. So that's definitely something we had in mind and we knew that we could make a model, a 7 billion parametri model, very good. That's for sure. This is definitely not the end of the story. Now, the question is, do we train bigger models? And the answer is obviously yes.

8:27There's still a limit to what a certain model size can do. This limit was, I think, underestimated. But if you want to get to more reasoning capabilities, you do need to move into larger models. The other thing about moving into larger models is that it enables you to train smaller models that are better, which is through a variety of techniques like distillation or synthetic data generation. So these two things are quite related. If you want to make very strong, small models, you do need to have bigger models. And we are indeed training larger models for sure. Can you tell us about your approach to data and annotations?

9:08Because we kind of talked about the other two dimensions. Yeah, so we've talked about compute and obviously data is super critical. So we work from the open web. Well, we do a lot of work. I think we do a good job at getting some good data. The data quality is really what makes the model good. I mean, data algorithms, obviously, but that is super important. We put a lot of focus on that. And I think we do have a very good data set. That's for sure. Data annotation is, I guess, another topic. It's not related to pre-training. When you pre-train a model, you really want to have the purest knowledge, to the purest quality of data.

9:49When you want to align your model, instruct it, ask it to follow instructions, which is useful for many use cases because it makes it steerable, you do need to have a certain amount of either human-produced annotation or potentially machine-produced annotations. And so that's something that we start working on. We're not the top experts in the world in making good instruction fine-tuned models. We're definitely ramping up, and the team is getting better and better at that. One of the things we haven't talked about is the fact that you guys are an open source company, which is very, very different from the other sort of labs working at the state of the art today.

10:30Like, why is that important? So if you look back at the history of machine learning in the last 10 years, it went very fast. I mean, we went from a poor cat-dog detector to something that basically looks human intelligent. And it's useful to remember how that happened. It happened because you had many academic labs. You had many industrial labs actually spending more money on different problems. And there was like full communication, almost full transparency until 2020. Like whatever was done in whatever lab, even in competing lab, was actually published at Nurev, was published at ICML. And every six months, we will gather and we'll get new ideas.

11:18Ideas will circulate and everybody will build on top of the work of others. And that's the way we went from something, well, potentially interesting to something very interesting. But then the issue is that around 2020, some companies started to be quite ahead on some field and realized that some value could be accrued. and then at that point opacity made it back to the field and I think that's a cycle we've observed in software already the cycle between openness and closeness we are observing it again we think that it's too early and we think it's really damaging for the science to actually move into such an opaque regime where you have a few companies basically doing the same thing just not communicating about it spending billions of compute doing exactly the same thing.

12:12And where really the technology we're looking at is not working completely yet. So still it doesn't reason well. Memory mechanisms are not very well understood. Causality mechanisms are not well understood. It's not super steerable. There's a lot of biases. I mean, it's incomplete. There's many things to be done. We still need to invent new techniques. And how are you going to invent new techniques if nobody is speaking about it when in order to invent new techniques, you need to still spend some large amount of money to actually try things at scale. And the few companies that have the money to spend have now refused to communicate.

12:50That's something that we deeply regretted. And that's something that we are trying to change because we do have some substantial amount of money to actually spend on compute. We do have some good ideas. We know that there's a big community that is awaiting for AI players, for open source AI players to appear. and we're very grateful that meta is moving into that direction. By doing what we do, by being much more open about the technology we create, we want to steer the community into a regime where things just work better, where things are safer because put under more scrutiny. And really our intention there is to take that position and to change the rules of the game because we don't think that this is moving into a proper direction.

13:34Yeah, it's very interesting because if you look at the current discourse, the really big tech companies are claiming that open source AI is dangerous. And it feels like really a form of regulatory capture, right? They want regulators to step in so that they can constrain innovation and kind of control an industry. And, you know, the reality is, if you look at things like global health, global equity, open source is one of the biggest potential ways for all of humanity to benefit from this technology in a way that's transparent and open and people can really understand and see. How do you approach safety and policy and thinking about, you know, the right ways to think about safety in the context of open source?

14:13So I think we approach it from a very pragmatic point of view. So the question is, is open sourcing today a model that we do? Is it a dangerous thing? Is it actually enabling bad actors to misuse the model? Is it giving them extra marginal capacities in pursuing their bad endeavors? I think the answer to this question is no. That's my conclusion. we've seen well a lot of ideas around bioweapons around I don't know nuclear terrorism and the like and it's very interesting because if you actually assume good faith of these arguments and I think in many cases people are in good faith and that's an assumption that we always make and if you try to go down the arguments I mean we realized that's what we did and we realized that there was really nothing to it.

15:12So nothing is showing that LLM is actually marginally better than a search engine to find knowledge on topics that would enable bad use. And the other thing is that it's not even proven, and it's certainly very likely not to be the case, that knowledge is not the bottleneck for the actual misuse that we're talking about. So we have two things in order to demonstrate that open sourcing large language models is actually unsafe. You need to demonstrate that they have marginal improvement over web engine and that knowledge is the bottleneck for creation. And in the two cases, the answer to these two questions is no.

15:54And so that means that we believe that we can open source models today and that actually it's the best way of putting things under the highest scrutiny so that we are ready for potential new generations of models that could be super intelligent. And in that case, I think we can rediscuss these premises. But today, we're really talking about the compression of knowledge that is widely available on the web. And so marginally speaking, we're not creating anything that is more dangerous than before. So I think that really there's a trade-off. There's a dynamic conversation to be had. That's what we discussed at the AI Safety Summit.

16:39For sure, this needs to be revisited as model capacities build on. But today, banning open source, preventing it from happening, is really a way to enforce regulatory capture, even though the actors that would benefit from it don't want it to happen. But by design, if you actually ban small actors from doing things in the most efficient way, which is open source, you do facilitate the life of the larger incumbents. And that's something that would be, I guess, detrimental to MISRA life for sure. What do you make of the arbitrary sort of compute and scale limits proposed? That's interesting. I don't exactly know how they came up with this threshold.

17:26It's a high threshold by any standard, because if you compute, If you make the bad faith assumption that this is the float 64, it actually gives you approximately 300 million last run compute. So that's high. That's not something we can even afford and that we won't be able to afford for the coming years. So it's high. It's very arbitrary because who tells you that beyond that 10 to 26, you end up with bad capacities. Models start to see the emergence of bad behaviors. that's definitely not proven relating capabilities to scale is also very approximate in the sense that it really depends on the data the data set is super important if you train your model on generating i don't i mean there's a focus on bioweapons so let's say if you want to if if if we want to prevent models from generating chemical compounds because we We think it's an enabler of bad behaviors, which I have said we don't think it is the case.

18:30But if you want to prevent that, well, you do need to adapt your compute flop budget to the data set that we are working on. As it turns out, that's what they did because they actually made a specific flop budget for biology, I think. So you can see the bioweapon narrative building on. But this is completely, we should really focus on capabilities and not pre-market conditions. And I think that's, I mean, to some extent, there's still consensus around that. So everybody knows that it's imperfect. It's a proxy, which is maybe fairly correlated. But definitely we need to come up with agreeing on how we measure capabilities, agreeing on what capabilities we deem dangerous.

19:20And I think we don't agree with one another on that topic. But these are really the, should be the judge and not obviously pre-market condition or the number of flops that you do. Do you know why there's such a focus on bioweapons? I asked this as a, I worked for almost a decade as a biologist. And, you know, when I look at how some of the complexity of building viruses or some of the complexity of actually doing these things, I'm surprised that there's so much focus in the community on that specific example. Do you have any sense of the origins of why people keep bringing that up? Because it's actually hard to translate.

19:53It's not some digital thing that you manipulate. I think it's a very interesting question. Honestly, I don't have the answer. It's almost epistemology at that point. So how did this idea appear and how did it get amplificated by the policy people and how did it actually ended up being heard by national security? And I think it started somehow with GPT-4, an X-D or something like on page 46. not exactly the same these numbers but where they generated some chemical compound and then they said a small remark saying that maybe that's not the direction we want to take, we don't want to have a model that reasons about chemical compounds chemical compounds in question wasn't dangerous but then it made this just observation which is definitely expected if you train on articles on biology, you're definitely going to be able to produce some chemical compounds.

20:53So expected observation. And then somehow people built things on top of it. Nothing was observed. No scientific studies in proper form was published. But then policy papers started to cite non-scientific papers arguing that these were scientific evidences that the bioweapon narrative was actually true. And then policy papers started to cite the other policy papers. And all of a sudden, you end up with like 50 papers saying that for sure, bioweapon is going to blow us up. And this is what the policymakers read at the end. I think that's how we ended up where we are today. So there's some deconstruction to be made.

21:40Unfortunately, I think the open source community hasn't been vocal enough because they didn't see it coming. But right now, this is changing, and I'm very glad that it is. I think it's mimetic. You have the factor of, like, the world just goes through the COVID-19 pandemic. Yeah, I mean, the COVID trauma for sure played a role in that narrative. I mean, that's definitely a traumatism. 30 million people died. that's definitely something we don't want to reoccur again. I don't think AI is going to be the one triggering the next pandemic. It's always going to be climate change. That's the way it was, and that's probably where the focus should be, instead of focusing on hypothetical, non-proven biological risk induced by token generators.

22:30If bioweapons is not a pragmatic concern in the visible future, there are real concerns around guardrails about what we want our AI models to actually generate. How do you think about that? Yeah, so I think this is a very valid concern. Models can output any kind of text. And in many cases, you don't want it to output any kind of text. So when you build an application, you need to think on the guardrails you want to put on the model output and potentially also on the input. So you do need to have a system that filters input that are not valid, that you deem illegal, and output that are not valid or that you deem illegal.

23:14So the way you do it, in our mind, is that you do create the modular architecture that the application maker can use, which means you provide the raw model, so the model that hasn't been altered, to ban some of its output space. And then you propose new filters on top of that that can detect the output that we don't want. So it can be pornography, it can be hateful speech. These things you want to ban when you have a chatbot, for instance. But these things you don't want to ban from the raw model, because if you want to use the raw model to do moderation, for instance, you want your model to know about this stuff.

23:56So really, assuming that the model should be well behaved is, I think, a wrong assumption. You need to make the assumption that the model should know everything. and then on top of that have some modules that moderate and guardrail the model. So that's the way we approach it. And it's a way of empowering the application maker in making a well-guarded application. And we think that it's our responsibility to make very good modules that allow guardrailing the model correctly. It's part of the platform. And we think it's the way of... There should be some healthy competition on that domain of different startups working on guard railing the models.

24:40And the way you make this healthy competition is not by trusting a couple of companies to do their own safety. It's rather the way you do it is to ask application makers to comply with some rules. So chatbots should not output hateful speech. And so that means that now the application makers need to find a good guard railing solution. And now you have a competition where there's some economic interest in providing the best gardening solution. And so that's the way we think the ecosystem should work. And that's the way we position ourselves. That's the way we build the platform with modular filters and modular mechanisms to control the model well.

25:26It's great that you folks are being so thoughtful about that. I think when people talk about safety, they really talk about three topics. And sometimes they talk past each other. One is this sort of moderation or textual based risk. And so that's risk of hateful content, legal content, bias, et cetera. There's a second class, which we talked about already, which is physical risk. It's things like bioweaponry or the ability of AI to help derail a train or interfere somehow. And then third is like existential or species risk. And that's when people start talking about AGI and new forms of life and, you know.

26:00Resource competition. Resource competition. or aggregation or things like that? So first of all, I think it's very important to address these three things separately and to hammer home that solutions exist for the first one, that the second one, there's no evidence that it actually exists at that point and no evidence that it will exist in the near future. The third point, I think that's very philosophical. Obviously, if you can make a system of arbitrary complexity, It can start doing anything that you don't want it to do. We are not at a stage where the model has arbitrary complexity. And so this is very abstract to me.

26:41I think that still, I mean, we'll move on to a world with agents and AI interacting with one another. And that will create a lot of complexity. Anticipating that complexity will necessarily yield to a collapse. We call it a collapse in machine learning when suddenly everything stops working because you fell into a local minima. Well, it's unclear to me that complexity leads to a collapse. Usually complexity leads to doing nothing because there's no self-organization and no willpower to build something. So I'm not too worried about existential risk. Obviously, this is a dynamic conversation. If we can make a model which is growingly intelligent, then maybe you're at a singularity level.

27:30There's no evidence whatsoever that we are on the way of making that happen. So I think it's a very open conversation we should have. I personally don't see the scientific evidence. And as a scientist, I trust only what I can see. Sure. And then I guess you mentioned agents, which I think is an area of a lot of activity right now. It feels like a number of things that are related to agents are still a little bit of ways in the future. In other words, it feels like an area with enormous promise, but it's still quite early. Are there any big technological innovations or things that you're working on that you think will really help expedite a world that moves more towards agents-based use in a more broad sense?

28:11I think making a model smaller is definitely a way to make agents work. because one problem you have with agents that very quickly they start to, if you run an agent on GPT-4, you're going to run out of money very quickly. And so if you divide by 100, well, the cost of compute, well, you can start to build more interesting things. What we see with agents is mode collapse. So not very interesting mode collapse. They start repeating themselves and they fall into loops. So definitely there's some research to be made there. There's some research to be made on making models more capable of reasoning and making them more capable of adapting the amount of compute they put onto the difficulty of the task.

28:53And this can be agent solved somehow. So it's definitely an avenue of research that we're exploring. Yeah. Going back to Mistral, one of the things that you've talked about a little bit is the platform that you've been building around the models that you train. Can you tell us a little bit more about that and some of the directions that's heading in? Yeah. So we know that hosting models isn't easy. We know there's a lot of work to be done on the inference side to make solving efficient. There's a lot of work to be done on the training side because you do need to come up with architectures that are memory efficient, for instance.