#288 Florian Douetteau: How Enterprises Can Scale and Adopt Agentic AI

0:00Some of our customers, as they pushed employees to actually discover and create agents by themselves, start to have some form of like agent sprawl. As in, you've got lots of people that created small agents doing this or that. And there is a question of like, who is do it what and for what purpose? We're working with some customers on essentially helping them having a central access point for agents. And the core idea is that there is a need and an appetite for having more life cycle management of agents. Hi Craig, thanks for having me today. So I'm Florian and I'm the CEO and co-founder of Dataiku and started the business in 2013.

0:39Prior to that, meaning I started my career more on the tech side, studied math, but in 2000, which is back 25 years ago, I actually started to work on natural language processing and web search and enterprise search technologies and meaning all by that time of the this early in the cycle um let's say it was very hard to get anything scalable up and running in the space of an lp uh you had to build everything from scratch and uh yeah i started the taiku uh after having uh had multiple experiences with companies that we're trying to scale their data science practice and we are failing to do so um and at this idea that meaning if you can democratize ai you can actually bring lots of value to the enterprise yeah uh and uh and tell us in in short what dataiku does uh and and what its main products are yeah we have one product which is in essence a no code ai workbench a world bench where you can do everything end-to-end required to manage your layer no code so that anyone in the enterprise willing to do so can actually touch it and start to work on data and ai and it's a workbench that can work on top of any cloud any data and the goal is really to bridge this gap between no code that sometimes is perceived as being prototyping for domains and enterprise production great applications and indeed we have our customers are among among our customers you've got large enterprise that use our platform in order to build thousands of data products and models and scale that scale that in two ways meaning scale the number of things you need to build, but also scale the number of people that can participate and understand what is being built.

3:00And most recently, you have what you called an enterprise-grade blueprint for agentic AI in financial services. So on this workbench, it's built for various verticals. And can you talk about this blueprint and how it works on the workbench? So indeed, we build this enterprise blueprint for FSR with NVIDIA. and it relates to the agnostic nature of our platform. As you pointed out, our platform is used equally by large manufacturers or life science companies or banks. And it makes sense actually when you want to do enterprise AI to actually tap into all of these type of sectors because they all have these issues of big data, complex data, diversified data, side of data and people in the business don't understand or you can help them.

4:04So we want to be agnostic in that nature or can you actually help everyone? But we also want to be agnostic in the sense of the infrastructure itself. And so, for instance, we build our platform so that it's super easy to work with the various type of large language models and vector stores around in order to move from just doing ML to do a gigantic with ML with our platform. And on that front, we connected with the NVIDIA enterprise AI stack so that in particular in banks that typically have regulated constraints, want to be able to host their data and all of that good stuff, so that they could actually build this type of solution in an easy manner.

4:48and uh the what are some of you you're saying that every vertical has its uh its um you know its its particular needs what what is it with uh financial services that that uh is different from from another vertical being built on the platform. I think that for every vertical, there is a particular need to support a bit differently, security and infrastructure constraint, as in like where the data is, what is the particular type of security setup you have in place. So for instance, in financial services, it's fairly common to have the need of securing data, not just per who has access to it, but also on a project basis, making sure there is no leakage of information from one project to another, because in financial services, there are constraints in terms of who you can use the data.

5:50Meaning you can't actually create any type of insider type of issues that you could get into if you don't have proper firewall between data in your system. The way we support that in our platform is with flexibility, but also in financial services, we provide on our platform different types of solutions, as in a way to just have pre-built projects that help you get value faster. Because there is like so many use cases of AI that many of our customers actually want to accelerate their discovery of them. It's a bit overwhelming to work in AI or to have to build AI system in the enterprise today because you live between this high level of expectation coming from your board and leadership and the reality of the field where getting to select one platform for large language model can be an issue in itself.

6:53And so our buyers are essentially the people managing this high level of expectation with the constraints of the field and our platform help them do that more efficiently. Yeah. And for the agents, do you have at this, is there, because this is no code, so is there, how do you build an agent on the platform? Or is it something that an internal team would build and then make available on the platform? Yeah, that's something that the internal team of our customer would build on the platform. And in terms of agents, you can build visually agentic workflows in the platform, as in agents that would actually connect to another agent, connect to another tool, go through various steps to achieve the task.

7:51And in this platform, you can do it by connecting natively all of those platform to your data and or more recent data your more recent models so you can actually build a fairly sophisticated thing such as an agent that would use a predictive model to actually make a decision instead of like hallucinating that out uh very useful for instance in banking if you want to at some point identify business impactful uh situation such as i don't know agreeing on a loan you don't want an llm to decide you actually want a good old statistical model to decide this part, for instance. And we also build the platform with lots of agentic evaluation capabilities.

8:32And in the enterprise, it's fairly important to have a good level of auditability and testability of agents, as in having continuously this ability to tie them to the data, analyze the outputs, run type of like back testing of agents before pushing them in production when you when you want to support like more customer facing not so impactful applications you can be more like i av test type of mindset with agents and you can iterate in a certain way but when you're in the enterprise you need to have a different level of death in terms of the test you're making because the cost of one issue can actually be very high yeah uh and and then um once these agents have been built, how do different people in the enterprise access?

9:30I mean, there must be very strong permissioning controls. Yeah, meaning you need permissioning controls so that agents can only be accessed by people having access to it. You want to be able to apply the access control of the person using the agent to what the agent can do as in if as florian i can only have access to this rose in this data set when i'm using any type of regular bi tool as set up by my it i want the same permission to be applied when i'm using an agent of course and so this is the type of art things that needs to be indeed built and solved for in the enterprise a broader question that our customer has is like, how do you discover agents?

10:17As in, some of our customers, as they pushed their employees to actually discover and create agents by themselves, start to have some form of agent sprawl. As in, you've got lots of people that created small agents doing this or that. And there is a question of like, who is do it what and for what purpose? Which is, I think, another key topic these days. Yeah, well, how do you handle that? I mean, is there a master dashboard that can see every agent in the system? And not only that issue, but you end up, I mean, not simply agents sprawl, but you end up with agents talking to agents and passing data back and forth, and that creates a potential security problem.

11:11yeah potentially i must admit that in the enterprise the agent talking to agent type of scenario is still let's say very early and hypothetical currently yeah but but indeed a practical issue is first having some oversight on everything that is happening from the perspective of it can i actually tell what's happening what were the project what are the loi what are the risks and so forth and so for that meaning it's essentially providing some governance framework and ability to gather data from everywhere. The second thing, which is funny, is that we're working with some customers on essentially helping them having a central access point for agents.

11:51And the core idea is that there is a need and an appetite for having more life cycle management of agents. As in, it's kind of like back to this question of ideation. As someone in the business, I could have a great idea of like what an agent could be doing, and I want to be able to prototype this quickly so and with agent prototyping is very easy like you prompt you say this and this data and it's kind of like working but life cycle means that in the enterprise you would ideally have a step before this thing becomes shared to one thousand people in your organization a step where it can check it validate it operationalize it with like the proper data the proper permission and so forth before it gets and becomes something official.

12:39And doing that, you avoid two things. First is like issues in terms of accuracy, security, or falseness in the enterprise. And second, you create more trust because you can ideate things, but having a proper vetted catalog of enterprise agents that people start to use for real. And we've seen with this pattern, I think good progress with our customers. And we think there is for us a way to actually further out there. Yeah. You said that not many people are doing agent-to-agent systems yet. But why is that? Is it just, is it a matter of trust? Because certainly as a journalist, that's all I read about.

13:34Agents, swarms, agent societies. Yeah. So, yeah, I think because it's, so bear with me. We have, and we support, and we have lots of use cases where an agent is calling to another agent. But, in fact, the reality is that when you're, it's almost a development pattern. where when building something significant, you start by building small things and a small agent to be able to test it. And then another one, and then another one, another one. And then your real use case, you add a wrapper on top thing. You say, you will do A, B, C, and D. And that's it. And you have built intermediary agent for each in order to actually be able to test the thing as someone building the system.

14:22So it's not yet agent to agent, as in like an agent that was knowing what are the best customers and another and running on its own and an agent thinking about the talent pool in the company and the two agents talking to one another to discuss and devise what should be the strategy of the company. We're not there yet and we might never be what we're talking about agentic systems that can automate significant tasks but indeed need to need to be built in an incremental manner and we've seen good success of that in the last a couple of months we've seen advertising company using our platform in order to automate their campaign management and be able to just be able to do more campaign per employee and per manager of such campaigns like by a factor 50 which is amazing we've seen retailers using agents in order to fluidify access to information in the shops as in how people can actually access inventory can provide recommendation to customer on site and so forth by having more fluid interaction and just unlocking all of this data which is into erp and product catalog and so forth that are too hard to access by people on the ground.

15:44And we've seen manufacturers and pharmas using agents into some of their complex R &D cycles in order to massage lots of information about state-of-the-art into multi-steps process in order to refine this information and build internal drafts that serve their R &D practice. And so all of those things are not trivial, meaning they took some understanding of the business to be built and they are very early being it's a compared to what with agents compared to what agents will be doing it's still very early but very promising from my perspective yeah what is the most complex because a lot of what you mentioned sounds uh a little bit like uh like search right it's surfacing information that already exists uh you know maybe uh uh tied to a chat bot or something but what's the most complex uh agentic activity that that you've seen to date that's in production um I think that in terms of first, in terms of scope or number of employees you can touch, for instance, lots of things that are field activities as in supporting, for instance, a retailer, are indeed search, search as in like, can you tell me what should I recommend to this customer that is in front of me?

17:19It's search in a sense. But like when you dig into what you need to do, to do there you need the proper product catalog you need to understand the customer question you need actually also to know what to recommend based on prior history and all of that good stuff so you need a supercharged kind of like amazon recommender system with the proper product catalog but like being able to deliver that in front of the customer kind of like in real time and so those systems are maybe not complex in the sense of like the number of steps is not that many but you need to connect them properly to have the level of experience you need in order to deliver the value.

17:56And so in that sense, they are sophisticated. The type of thing that we've seen that were the most sophisticated perhaps were into intellectual property, for instance, where you could start using agents in order to understand the state of the art, what was patented or not in your space, mix that with your internal databases and use that in order to support or accelerate the management of your IP portfolio. And where indeed you need to understand the data and your space and the science of your space in a very sophisticated way. And again, you could say it's search, but it's not just search. It's like synthesizing all of the information of your domain in a specific context in order to understand what's new versus not new, which is indeed a complex multi-step startups works in systems that are literally calling thousands of times llm in order to answer one question yeah um and and um in terms of of agents you know i i was at a conference a couple of months ago and the guy was talking about you know speculating or about uh a headless organization someday that that there'll be agent managers in for various departments and then beneath them you know groups of agents uh doing all the various tasks that the organization needs to do and then maybe an overlord agent that watches everything Do you want to be the overlord agent or should I be?

19:47It seems to be a good position to be the overlord agent Yeah, I suppose but do you envision that sort of thing happening and what kind of more complex actions do you see agents being able to carry out? Yeah, I think that indeed it's an easy way to map agentic activities, to map them to the organizational structure of a company because it's make it easier to for people to discover them, to understand the role and responsibility of them. So, indeed, we see our customer doing that. As in, in a given team, I should think about how many employees and how many agents. And that's, yeah, that's interesting.

20:45And it's, when you, interesting agents are agents that do not just do a simple task, but that essentially accelerate or support the given process inside a company

21:02and indeed it's easier if you map if you wrap all of those agents that you um processes in a given domain let's say sales or rnd or whatsoever in a master agent that can root to them because then you simplify the way people in the company can actually access the information and start using them and so forth. And so it's a pattern of, well, essentially delegation, like in real life, like in human life. You've got the sales agent having different sub process and depending on what you ask, it will delegate to the other one they will do it's a it steps so it's actually it's actually the way people do it then uh to to maybe answer the second part of your question in terms of the the capability in the capability space i think it's interesting to realize that agents when you build them you have some expectation of their qualities that for some of them mirror a bit human qualities.

22:09As in, in some scenario, you want agents that are very good at just repeating a given task. They like to process and you want it to go through in a very predictable manner. Sometimes you and maybe you do too. I use agents for creative purposes to give me ideas, well generate an image helped me with a slideshow or a presentation so you use it as a as a creative head sometimes and i'm meaning literally all the time i use agents for more like productivity and doing some assistant type of work as in like meeting notes and whatsoever and and it can help also with collaboration or some aspect of it and so and there is some agents that are more maybe at the equivalent of a management of a function or management of a process type of capacity.

23:03As in, you expect the agent to support or automate an actual business decision. As in, do I resupply my stock or not? Do I prioritize this product or not? Do I change my prices or not? This type of things. So those things are different qualities. meaning you don't expect a given person usually in an organization to be creative and very very good at execution and very good at taking business decision or be a great developer or be a good production or manager or a great facilitator meaning you don't expect the qualities all of those qualities to be the same person most of the time i don't actually and so and i'm neither of those.

23:50So the the for the same reason, I think that when we will be building agents more at scale, we will start realizing that depending on the task, you expect different things from agents, and maybe you start using different types of system, depending on the type of agents you you want to use or create. And that's my the way I think about this ecosystem moving forward. It's very early because agent is it's like we're in 95 or 97 maybe an agent is website. MCP is a gopher or ftp. Not really not really HTTP and certainly not HTTPS. Yeah. Yeah. So that's that's and so we say agent but actually we mean

24:43blog or e-commerce or whatever else like you will have many type of agents many categories of agents that would be actually created yeah okay can you talk a little bit about the uh data IQ LLM mesh and how that works with the agent platform. I mean, does the user decide have access to all the various open source or proprietary models and make the choice or is there an agent in the dataiku ecosystem that decides what model is best for what task? Great questions. Indeed, we build the... So what I think in this agentic ecosystem is that From an IT perspective, there will be more and more the question of abstraction of large language models.

26:05Because from a cost or management perspective, they are a very peculiar type of systems in terms of their costs, their characteristics, and so forth. And the fact that you send a lot of data to them potentially, and so on. It's a new type of system. And so from my perspective, the tech who are not, enterprise, at least large enterprise, will end up having this view of having a way to abstract away this access to all of the LLMs. And there are at least three big ways to consume LLMs today. You can use directly the services of the OpenAI and Anthropic of the World. You can use them hosted into your favorite cloud platforms, either the same models or more open source ones.

26:56And you've got lots of variants in data platforms and cloud platforms to actually host agents. And you can host them yourselves, either your own servers or services you rent on a purpose manner to just self-host models. And you've got good reasons to do at scale. There will be good reasons to do all three of them, potentially, maybe, depending on security and sovereignty type of questions. So for nationally, we said like a product should be built so that from the customer perspective, it would be very easy to manage and switch from one to the other and to have a more holistic view on the risk, the logs, the audit.

27:37So the cost, the security aspect and security for agent is not just who has access to what, but it also the list of forbidden word or topics you should not send to this agent or that other agent. If I talk to about maybe you want to set up your system so that if you talk to or use a customer name, you should never do it on a public server agent and always use a self hosted one. I don't know, for instance. And so that's a type of very IT mindset we use to build our product. And I think it's actually very generic, as in like this mindset should be the type of mindset that enterprise will have when they think about those AI systems at scale.

28:23to your second question indeed today we don't think and we don't provide a magic uh model that is um choosing the best model and i think it's early to do that because it's more like on a pair use case basis where you choose the model when you create the use case instead of doing it dynamically it might be the case down the line that we do that but it's a bit early the ecosystem to do it right now yeah you were talking about security and you have a function on the platform to set guard rails is is that essentially a prompting exercise gathering is not just a prompting exercise it's also to have a dynamic guardrails are about having some form of filters that you can have on the data in or the data out as in what you send to the lm or what you receive to from the lm and this can be a mix of uh guardrails that are based on just keywords or dynamic list of keywords of the galleries that are based on topic gathered on keywords could be like i don't want my customer customer customer names um to be sent out or i I don't want something looking like a social security number to be sent out, for instance, the type of Gallerals.

29:51The kind related to topics is, for instance, I don't want my LLMs to be providing back any type of health advice or financial advice, for instance. and so you typically want to set up those guardrails in a meaningful way so that either you do them universally for any type of usage within the enterprise or pair category of applications you all are one versus not the other and so it's important for the enterprise well regardless of like whether it's agentic use case or more basic use cases of llm there is a good reason for enterprise to have a good control on those guardrails from the perspective of compliance and compliance and security yeah and on the platform is it uh is there a conversational interface to implement these things or on guardrails is there a list of check boxes that the user can check i mean how you were saying it's largely no code so how is a user uh uh telling uh you know building this agent so so when we When building an agent, a user would actually tap into a LLM set up by the enterprise.

31:23He would drop in a list the one that the enterprise provided and was actually given to him. And those LLM, when used by the platform and through the platform, are already secured. as in for instance the someone in the it of the company would have said in this company we just use let's say open ai and we can use it for three different reasons one is like super secure and one is more like internal fluid and one in the middle and depending we set up three types of guardrails that are different for each level and maybe for the secure one we are super strict in in terms of like no else advice and no financial advice and no talking about chocolate, I don't know, whatever.

32:11And in the internal one, we can do whatever. And so depending on their permissions, the user would have access to level one, level two, or level three, let's say, or like what has been configured by the enterprise. And then in order to build the agent, they can prompt their way and have access also to a set of tools and same tools are pair a setup of a set of permissions related to what they can access to and what they can give access to in terms of applications and data and so this way a user can build an agent by prompting and sequencing prompt and and accessing to tools then the user can give access can test these agents and give access to this agent to use other users and then indeed total security is about the other users when accessing the data would also have a set of security constraints been applied to them that is specific to them so that you actually don't leak information by creating agents and in the background what's happening uh uh are there blocks of code that are pre-written that are being assembled is is that i who actually writing code in the background uh or are there uh a catalog of agents that iq is pulling from uh and presenting to the user Yeah, we have a catalog of tools that contain code.

33:55We've got the prompts are enough to actually create the agent. And then Dataiku would generate, and the tools within Dataiku would generate code in the sense of, for instance, if it's about querying a database, it would generate SQL code, for instance, in order to actually support the proper activity. as a broader tool beyond agentic meaning that iq is also a platform to do any type of ml and analytics in which case it's also no code as in you can use it to transform your data and build your models and so forth and in that process you can also decide to generate code if you want we see that as part of the life cycle of the platform sometimes our users like to be able to generate code in order to actually go one step further in terms of customization and adaptation to their use case.

34:53And so over the years, we've built the platform in a way so that it's no code, but also code friendly. And it's back to this topic of collaboration and lots of people in the enterprise needed to work together in order to achieve goals. we build the platform more and more so that it can be a collaboration place between the technologists and the non-technologists in any given company because i think that the lack of collaboration between both camps is usually what creates struggles in organizations if geeks and non-geeks could be friends what would be better and and so indeed we build an local platform where you've got a full-fledged VS code and Python notebooks type of environment so that coders can also thrive.

35:43Yeah, and I mean, that's interesting. Just thinking about the organizations that I've worked in, I mean, primarily the New York Times, the people, yeah, there's a huge divide, at least in my day, between IT and users. uh it how in in your experience uh the users of the platform at enterprises what percentage are uh business experts or or uh you know operational uh people and what percentage are the IT department. I mean, is this really getting out of the IT department? Yeah, in enterprise, it's typically 75%, 80 % that are in departments and 20%, 25 % that stays central IT. And so the way I think about it is that, let's say, you've got IT and business.

37:00Great. But in fact, in IT, you've got a bunch of people in IT that are very interested in working well with the business. And in the business, you've got lots of people that are more data experts of a form or Excel gurus or like really interesting into the quantitative side of things. And so the goal of our platform is actually to bring those two groups together. The people in the business that are actually more like quantitative, I want to get my hands dirty doing AI by myself type of mindset. and people in it which don't care about building it ivory towers but actually want to build it enabled for the business enabling the business type of mindset which is not all it like bear with me it's like and so but if you take those two groups together and you can actually have them share a platform.

38:01You actually can achieve a great thing for the enterprise, which is to break silos. And is that something that you advise corporations on? Because I can imagine someone buying the platform for the enterprise, maybe the CISO or the CTO or even the CEO, and saying, here, I want everyone to use this. And it sort of sits idle on most people's Yeah, I think that there is what would this enterprise is in fact, a modern version of like what should be a center of excellence for AI and center of excellence is like loaded world and like fairly old as a concept. But it's indeed effectively creating this place where you move IT as an enabler of the business and you create a dynamic where people can try by themselves where they communicate where you you even have like sometimes hackathon real hackathon within enterprise where they can like innovate and test and so forth where instead of people being like oh my god I don't have access to data you create a proper desk or way for people to ask or people to help on that front and so that's part of the dynamic we indeed help create creating the enterprise and so this way we were successful in creating like fairly fairly large groups of people by by the thousands sometimes like thousands employees building models on the platform and so forth and And indeed, you need people in central IT to help that by having a proper center of excellence, because in large enterprise, nothing significant happens organically.

40:12In fact, large enterprises are very good at killing organic stuff. That's just a fact. That's a good thing, actually, because you need focus and vision and drive and rigor and discipline if you want to scale any type of organization. But so for this type of collaboration to happen, indeed, it needs to be in large enterprise, a proactive initiative. Yeah. I mean, some people I've heard advise that every department have an IT person embedded or a tech person embedded in the department that everyone can come to. So it's kind of a dispersed IT department across the enterprise. that makes a lot of sense to me because you're not going to have you're you're going to have varying levels of skill and interest in the department and if there isn't a dedicated person and everyone starts coming to you know joe in sales who knows how to use this stuff pretty soon his job is going to be uh just helping other people so uh yeah i mean that's that's an interesting challenge um and and you how does uh how do you sell the platform is it by seats or yeah it's basics

41:46And how do enterprises generally distribute those seats? I mean, are there some enterprises that give it to everybody above a certain level or is there like one seat per department? How do you see people organizing that? Yeah, well, there are some enterprises giving it to everybody. but not enough yet but the most common pattern is that you start having a group of users into one given team and then you expand to another by group typically not individual users the goal being to modernize the way you do advanced analytics ml or ai into this group and having a consistent set of use cases and data to work upon and a consistent way to support this group so imagine you've got a big department of thousand people you would have a few thousands maybe and kick starting using uh kicks i think using the taegu that would be more probably the people in various form of ops function and data support or ai support function within within that within that department and Central IT would actually support them by making sure that their first use cases and access to data is properly supported and that they've got a good shared view on what are the use cases and the roadmap for that.