Milos Rusic: CEO & Co-founder at deepset

0:00Milos Rusic:Hello and welcome back to Scaling Europe show. I'm Seb Johnson. I'm here with Milos, CEO, co-founder of Deepset. Thank you so much for joining me. Thanks for having me. Can you tell me, for those who don't know, what is Deepset? What have you built? Of course, happy to do that. So Deepset as a company was started a bit more than seven years ago already, long before the more recent hypes in AI and generative AI, right? We started the company seven years ago with probably out of two motivations. Motivation one was we felt that technologically the state of AI that was back then still kind of rudimentary will probably improve.

0:39And we're going to see major improvements, major advancements that will just simplify the adoption of AI. That was motivation one. Motivation two was we didn't know how this large scale adoption would look like and what a product could look like. But we wanted to go on the journey and explore that. And this is where we started the company initially as kind of, you know, a vehicle for us to explore AI adoption enterprises. And we did that by selling actually services. So we built bespoke AI software for enterprises in Europe and solved all different kinds of problems for them, right? For example, search engines, classification systems, search over technical documentation, process automation when it comes to automating financial audits.

1:30That was very, very broad what we were doing. Technically, any industry and any problem was a good problem for us because it was a great learning experience. In parallel to that journey of exploring enterprise problems and or actually finding enterprise solutions for AI, we experienced the technological advancements, right, that all of us are actually experiencing and using day by day today, which is the birth of the Transformer model end of 2018. When that happened, we immediately understood it's a quite big thing. It's super powerful. We can way easier build AI systems for any kind of problem out there.

2:13So we started to engage very actively in the leading communities around these transformer models. And back then, those communities were mostly all open source, right? So we built ourselves a strong open source heritage around that time. And then end of 2019, you know, we kind of succeeded in our mission to find a product because we came up with the idea of the Haystack open source framework. The Haystack open source framework is probably the first orchestration engine out there. It's think of it as of a very complete system of components that are all needed to build agents or reg apps or whatever AI system you want built.

2:52Right. And this is pretty much what we came up with. end of 2019 and it was also open source because we were big believers that open sources the thing open sources you know kind of almost a prerequisite for enterprises to adopt AI a large scale and that is why we also feel our product needs to be open source of the core and look what happened since then is I'm obviously the adoption of haystack skyrocketed the adoption of Jenny I skyrocketed overall which was a big tailwind for us we moved away from being a few services company to becoming a full product company. We raised venture money from Google Matrix, Bold and many others.

3:31And yeah, we were a company to be one of the leading players when it comes to AI, I would say enterprise AI globally. And of course, in particular in Europe as we're one of the few companies that accompanied the whole Gen.E.I.I. part of Europe.

3:49Milos Rusic:And was there a moment in time where you were like, okay, this is, I don't know, maybe we've found product market fit or we've hit that sort of like that growth curve and then sort of like enabled you to raise the money that you raised from great investors. Was there like a, was it a day where users skyrocketed or something happened where you're like, wow, this is it? I think we don't have this, this, you know, this one event where we can pin that down. It probably happened way more organically and constantly. I remember, you know, learning about the logos that were early on using Haystack. And then, you know, seeing that in 2020, right, we made a lot of successful business with helping organizations implement Haystack to solve certain problems, right?

4:37So you start to understand, hey, there are really problems that are solvable with this technology. This technology gives them something very promising. And we are authoring that. And this is kind of, you know, how step by step You gain more confidence. You feel like, hey, we serve customers with the open source piece. How can we actually scale ourselves? What does the product look like that will enable more and more organizations to do this autonomously? And, you know, it was kind of, I wouldn't say there was this one moment. And it's just, you know, you just feel sometimes, hey, now it's the right point to launch a product.

5:16Now it's the right point to commercialize a product. Now it's the right point to raise venture money. That was pretty much how the journey went for us.

5:22Milos Rusic:Yeah, yeah, nice. And I think it's such an interesting journey that you've been on since 2017 to now. You know, you've seen, you know, the pre-AI world when you're kind of building these tools for them. But I want to talk about your view on how things have changed. But before we do, I was wondering if you could give almost like real use cases of the way that people are leveraging DeepSet and Haystack to kind of build and kind of like deploy AI in the real world. Yeah, of course. one use case that probably was quite I would say quite en vogue when ChatGPT came out was that many publishers wanted to power their products with it as well right so it was pretty much you know let's say you're a publisher you have this very famous online magazine a lot of curated content now people see chat GPT and everyone expects that style of interaction with your own content, right?

6:23That is something that people wanted to replicate and they understood early on that they have probably required individual requirements in doing that, right? If you're a publisher, you care maybe a bit more about what's the tone of the answers that are given. You care a lot about, you know, that everything is politically correct, that, you know, it's not racist, not sexist, all of these things. It's essential, right, to just guardrail these solutions properly. And you realize, hey, it's not just I'm implementing a model here. You need something way more robust and stable. And that is why it makes sense for you to build on something like Haystack or Deepset, because it gives you all the nuts and bolts you need in order to really have a very reliable solution on your data that you are exposing to your clients.

7:07Other examples, also like a kind of public example, the Kitsch system that we're co-developing with Airbus, that is pretty much to help in military operations and decision making. So you have a whole bunch of data and documents. You need to make certain decisions or come up with certain plans and tactics when you want to execute a military mission. It's a lot of data that you have to process that needs to be combined. Everything needs to be super reliable. These are the kinds of systems that we really feel frankly confident with because it's pretty much where all they'll be coming up.

7:50Milos Rusic:And what's it like working with companies like Airbus, European Commission, some of these other companies? They're not typically thought of as the most tech-savvy, forward-thinking organizations, but what you're helping them to do is deploy state-of-the-art technology. What's it like working with companies like that? I would say it's amazing. and we're enjoying it a lot because we see so much honest willingness to adopt. It's not, you know, it's not POC-ing. It's not really, hey, let's see what this can do for us. As you mentioned, the European Commission, we're also working with them. We're working with a few other federal ministries and authorities, also with the Federal Ministry for Research and Education Germany.

8:38And what you see is that these organizations are really, you know, they're not thinking in an experimental way about AI, but they're really, really serious about it. And they really understand this can do a lot for the way we are acting and operating. And all of them know that everyone is constantly complaining, you know, about how undigital the government is or certain services, how cumbersome they are. Obviously, you know, a lot of that is not necessarily because of the lack of digital tooling or digitization. Often it's just the nature of the work and the nature of the procedures they have to follow and provide ultimately also to us as citizens.

9:16But AI offers them new tools, right? AI gives new opportunities to probably also implement certain technologies quicker, faster. It's way more flexible than any rule-based system. And people really appreciate that. So I have to admit and say we're enjoying it. And also the level of talent in these organizations is great. It's really, obviously, the adoption cycle takes longer, but I would say the way European organizations, governmental organizations are thinking about AI adoption is super serious and super committed. So it's really fun to work with these organizations.

9:56Milos Rusic:That's amazing to hear. I think there's a lot of criticism at the moment leveled at European politicians, bureaucrats, but also enterprises for slow adoption. Whereas what you're saying is there's actually a real appetite within European companies to make the most of this technology. Yes, in particular, like, and I see it even being stronger in governmental organizations, funny enough. Really? Yes, I can really say that it's at least for what probably, probably for technology like HATES, right? Simply because the nature of the use case is way more critical. The requirements when it comes to accuracy, the requirements when it comes to how robust an AI system has to be to really be usable in a productive way are way, way higher.

10:45And this is usually where our value is the biggest. So that being said, probably that is one reason why we're seeing so much pull out of that segment. In enterprises, a lot is about this daily way of productivity. So people are looking into, hey, how can we help people reply their mails quicker, maybe also read their mails, have better access to the knowledge bases we have on our drives and SharePoints and whatever that is. So it's more of these daily productivity tasks, whereas we see in governmental organizations many, many, many of those are thinking in key processes and how can we really leverage AI in key processes.

11:33And that, you know, is then again where we probably fit a bit better in than in use cases where something like also chat GPT just by itself is already very, very strong and strongly adopted.

11:45Milos Rusic:And when it comes to AI and, you know, governmental organizations, what's your view on a lot of the debate around AI sovereignty? How important is it to keep our data and secure servers, all the content, how important is it to have local LLM champions? What are your views on sovereign AI more broadly? It's a very ambiguous topic overall, right? If we're very, very open and honest about it, right? And I think I would like to give you really like my personal view on sovereignty, just to be clear. I think we're still lacking a proper globally defined term about what it means. But my view on it is sovereignty is something that I, in the first step, seek really independent of the geographical focus.

12:30So if that is sovereignty on the level of the European Union or Europe as a continent, or if it is on national level, or if it is on organizational company level, I think all kinds of organizations and institutions in the world have a desire for sovereignty.

12:53So this is kind of like my first entry point to the topic. Now, obviously, governments usually tend to have a bigger desire for that or at least to have maybe an earlier need than others have. But I can see that sovereignty and control is something every major bank will want, every pharmaceutical company, frankly, probably even every startup for some applications, some use cases, will want that kind of sovereignty. And what sovereignty means to us in terms of AI is mostly three aspects. Number one is the AI system must offer optionality for two reasons. Number one, you don't want to be locked in into one of the key technologies, for example, LOMs.

13:39The second aspect is you don't want to, or actually you want to use AI and tailor it and guardrail it in a way that, you know, it really fits your individual needs. I think the big problem will be this long tail of use cases that will have a very individualized flavor. All of these use cases will have massive ROI, right? Because, you know, credit decision-making is super important in banks, but each bank does it differently, right? So how do they account for these individual characteristics in these use cases? And I think this is where optionality and choices are very important. Can I introduce my own guardrails?

14:20Can I pick the model I want? Can I swap the model if I want to swap it? I think that's like the first big pull of Solanity. The second one is transparency. And I think in software, look, the most transparent way of delivering software is open source, right? And that doesn't mean that people are looking, I think often people think open source means it's free. That's technically right. But you also can be open source and have a restrictive license or you're open source and you still have a business around it. But people appreciate that technically the core processing logic of your product and offering is very transparent to them.

15:01They can test it. They can, you know, they can understand it. They can read maybe even research about it. Right. So all these things that in the end contribute to trust and in particular in AI or in any new technology, this is something that needs to be maintained. And I think this is why Open Source is the second big pillar for sovereign AI. And the last thing is infrastructure is obviously kind of the big thing in AI right now. If you look about the major, major investments, all of them are actually in infrastructure right now. We're talking about NVIDIA chips that are being bought, you know, data centers that are being built up.

15:38And what that shows us is infrastructure matters a lot to all that we will be doing with AI. People need to have the flexibility that they can move also around and that they can really make a choice on where does this run. And if it means it runs in an air-gapped environment in my basement, then they should have ours. So this is what some remedy means for us. And this is how also kind of we're addressing it. Haysick is highly customizable. We really focus on that. Plus it's built in a way that we really want to give choices to users. So it's agnostic of all models. It is open source and it is an offering.

16:13So our commercial platform is something we also offer on Crem besides the offering in the cloud. And that is what constitutes now sovereignty for us. To your question, if we see that technologically on kind of core technology level, Europe needs on champions when it comes to cloud or when it comes to LLMs, I think we can probably live in a world where we are also still consuming US models or relying on US GPUs or wherever they come from. from, right, not just the US, but any other country. And I think, you know, as long as we say, hey, our values of manufacturing and building these technologies are pretty much aligned with the country of heritage, I do not really see an issue.

17:08And I'm actually also, you know, quite a friend of saying, hey, why shouldn't we use US models and, you know, wherever they come from? And I think though that Europe would do itself a great favor by actually fostering that sub technologies are also coming out of Europe, right? Because I think the talent density is immensely high. It is the future. It will be also like the future of the wealth of a region or a nation or, you know, a continent. And that's why there should be a big control. I'll bring that and not just relying on the effort of other countries or other parts of the world. But I think sovereignty can exist also in an environment where many key components might not be European.

17:55And I think this is how we're looking.

17:59Milos Rusic:You enable your users to choose, you know, you talk about being flexible, transparent, enable people to choose whatever models they want to use. Do you see, you know, does it look like to you that people in Europe are trying to, you know, are opting for Mistral at all? Or, you know, do you see that sort of like bias towards local players at all or not? I would say bias is probably a big term, right? I would say ultimately people think in performance and what does the job and what fulfills the requirements is still what's in favor. That being said, if that is a question that might be answered in a way where you say maybe two vendors show some parity or maybe we cannot fully assess even what performance looks like right now for us.

18:53I think still that it's a big question, which is the best model that declines for much on the use case or can depend on the use case or on the type of data or maybe even on the kind of language you're using to query that model. And in such situations, I would say there is a tendency to look into European options. I've seen that myself obviously because we're a European company right I think we we see that it is something you know our clients and prospects like and that we're coming out of Europe and I can also see this for other technologies you know for example NN or also Mistral who all have a very strong open source co-op yeah but you know obviously people have a tendency to say hey if I'm thinking about workflow automation and I say the solution is great and maybe there are other great solutions in the market, but they're not coming out of Europe.

19:48Why not rather opting for the European one? We're seeing this. And I think in those situations it's fair. I think it's a good thing. And these are maybe the small aspects and a decision that will help us just accelerate the ecosystem in Europe.

20:05Milos Rusic:Amazing. I think we're all out of time, but super interesting. I think, you know, the position that you've got in the ecosystem is such a unique one. You know, you kind of sit in the middle of it all. You've been here since 2017. You've got such an interesting perspective on the state of the AI market, but also, yeah, who's using what and why. Well, thank you. Thank you so much for joining me. Thanks. Thanks a lot, sir. Thank you.

20:38Thank you.