What happens to Observability If Code is AI-Generated? The Potential for AI in DevOps, with Datadog Co-founder/CEO Olivier Pomel

0:05Welcome to KnowPriors. Today we're speaking with Olivier Pommel, the co-founder and CEO of Datadadog, the company at the forefront of the DevOps revolution. Datadog is a leading observability and security platform for cloud applications. Its execution and ambition has impressed me for years, especially since learning more about the company after it acquired Screen in 2021, a security startup I was the board member for. I'm excited to be talking about the potential for AI in DevOps. Olivier, welcome to KnowPriors. Thanks for having me. So let's start with a little bit of personal background. You're French, you've been in the U.S., working on startups since 99.

0:42How did you start to think about starting a company, and Datadog in particular? Yes. So, yes, I'm from France. I guess nobody's perfect. I'm an engineer. Also, I got into computers largely through computer graphics. When I was a kid, I used to follow the demo scene in Europe, which was all about 3D and doing interesting things in real time. This led me later on to be one of the first authors of VLC, the media player. who I think is mostly used for viewing illegally downloaded videos. And I should say, most of the people who made that successful came in after me, picked up the project, and did something fantastic with it after I left for the U.S.

1:22And then moved to the U.S. to work out of all places for IBM research in upstate New York. And I thought I would stay six months, and I've been here since 1999, so it's been a while. I worked for a number of startups through the, I would say the dot-com, the tail end of the dot-com boom arrived right in time for the bust. And after that, I worked for, I think, eight years for an education software company, education startup, that was doing SaaS for schools based in New York. And that's when I was at this company that I spent quite a bit of time with the person who was my co-founder at Datadog.

2:00And that's where we had the idea to start Datadog, basically, which was, I used to run the dev team there and he used to run the ops team and we're hired everyone on our teams. We tried hard not to hire jerks. We're very good friends. We've known each other since the IBM days, basically. And we still ended up with Dev and Ops hitting each other, people pointing fingers at each other all day long, big fights. So the starting point for Datadog was not monitoring. It was not even the cloud initially. It was, let's get Dev and Ops on the same page. Let's give them a platform, someplace they can work together, see the world the same way.

2:34Yeah, that's actually quite different than thinking of it as like a ticketing relationship, right? A quite siloed relationship between the two areas. Because I think most people assume that Datadog comes from a place that was more like metrics or like we knew the cloud was coming. And I'm sure, you know, both these things are true. But it's interesting that the core starting point is really around like dev and ops collaboration. You are, I think, pretty long NYC and have been challenged on like building in for an NYC as a thing to even attempt before. Tell me about your original thinking on that or if it even crossed your mind.

3:08Well, I mean, so I stayed in the U.S. because I loved NYC. That's why I ended up staying here. I love the energy. I love the diversity of the city. I also met my wife in NYC, and she's also not French. She's also not American. So it also made sense for us to stay in the city. So when we started, my co-founder and I, it made total sense to start a company in New York. We also knew fantastic engineers we would hire in New York. So it was very obvious from that standpoint. I would say it was less obvious when we started fundraising because we didn't come from systems management or observability or anything like that.

3:45And we were based in New York, which was not seen as a great place to start an infrastructure company at the time. So I would say for most investors, especially Bay Area investors, I think it was considered as some form of mental impairment to stay in New York at the time. It made it harder to fundraise. And I think as a result, it made us more successful because we were so scared of getting it wrong and so scared of not being able to fund the company any further that we really doubled down on building the right product, for one thing. But also, we built a company that was fairly efficient from day one and hovered around profitability throughout its whole existence, pretty much.

4:21And I think in the long run, it's been an advantage. Like everything else, if everything that's a long run or long-term advantage turns out to be very difficult in the short term, typically. How else do you think New York has benefited you? Because I feel like now it's kind of an obvious place to start a company. And to your point, when you first got started, it was very different. It feels like there was always some good talent pools there. I know Google had a giant office there and Meta set up one. And it was really sort of flourishing over the last decade. And now it definitely feels like a very strong standalone ecosystem.

4:50But are there other aspects of either recruiting or other things that you really benefited from being in New York? Yeah, I would say there's really two things. The first one is from a customer perspective, we're sort of out of the echo chamber of the Bay Area, which makes it easier, I would say, to latch on to what really matters to customers and what's not just a fantastic idea. You told three people and then repeated three others and then it came back to you and it sounds even better now. And there's a lot of companies of basically all, many non-tech companies in New York you can sell to and can get a good idea of what they need, basically.

5:22The second aspect I think that benefited us is that, so it's a bit more difficult to recruit in New York. There's less pure tech talent. There's less deep tech talent in New York than there is in the Bay Area. But the retention is a lot higher. So if you give people great responsibilities, interesting work, treat them well, they're going to stay with you for three, four, five years more, which I think in the Bay Area is pretty much on the very high end of what you can expect. What we see from looking at data, We have data from most of our customers are engineers and most of our users are engineers.

5:58And so we see basically when their individual accounts churn at our customers' organizations. And we see that it's not rare for companies in the Bay Area to have engineers churn every 18 months. We think it's really hard to build a successful company that way. I think you do have to overinvest if you want to do that. So one last background question before we ask you to talk just a little bit about Datadog today. Can you explain the name? So, yeah. So it's interesting because I'm not a dog person. My co-founder never had any dogs. In our previous company, we used to name production servers dogs.

6:36And data dogs were production databases. And data dog 17 was a horrible Oracle database that everybody lived in fear of that had to double in size every six months to support the growth of the business. And that could not go down. So for us, it was the name of pain. It was the old world. It was where we were coming from. It was Nemophane. So we used it as a codename when we started the product. And we actually called it Datadog 17. Everybody remembered Datadog. So we kept it. We dropped the 17 so it wouldn't sound like a MySpace handle. And then we had a designer propose a puppy as the logo of alpha dogs and things like that and hunting dogs.

7:19And I think that's the smartest branding decision we've made was to keep the name and to keep the puppy. Love it. So Datadog is clearly a leader in observability and security for cloud environments. You've had enormous success. I think you're now approaching a$2 billion run rate, 26 ,000 customers. You're really mission critical to a variety of different folks who spend, in some cases, more than$10 million a year on you. But for those listeners that we have who may be a little bit less familiar, or could you give us a quick background on what Datadog provides and almost like a Datadog 101? Yeah, so what we do is we basically gather all the information they have together about the infrastructure our customers are running, the applications they're running, how these applications are changing over time, what the developers are doing to it, who the users of these applications are using them, what are they clicking on, what are they going next, what the applications are logging, what they're telling about what they're doing themselves on the systems.

8:13So we cover everything end-to-end, basically. We sell to engineers. So our customers are, I would say, the folks who buy our product are typically the ops teams or DevOps teams in a company. And the vast majority of the users are developers and engineers. And then some of our users are going to be product managers. They're going to be security engineers. They're going to be all of the other functions that gravitate around product development and product operations. How do you think about translating some of those products or areas into the generative AI or LLM world? I know that, you know, obviously cloud spend is now 25 % of IT and there's been these really big shifts now in terms of adoption of AI.

8:52And it's very early, right? It's extremely early days, at least for this new wave of LLMs. Obviously, we've been using machine learning for 10, 20 years. How do you think about, you know, where observability and other aspects of your product go in this sort of emerging new area? Yeah. Yeah, so we find the AI quite exciting, actually. I mean, there's two parts to it. One part is the demand side, you know, so what's happening in the market that is driving the use of compute and building more applications and things like that. And the other side is what we do on the solution side with the product and hope we can use the generative AI there.

9:28On the demand side, it's exciting at so many levels. You know, if you think of the highest level possible and what might happen in the long run, we think that there's going to be so many more applications written by so many more people. It's going to improve productivity of engineers. And at a high level, if you imagine that one person is going to be 10 times more productive, it means that they're going to write 10 times more stuff, but they're also going to understand what they write 10 times less just because they don't have the time to see everything they do. And as a result, we think it actually moves a lot or transfers a lot of the value from writing the software to then understanding it, securing it, running it, modifying it when it breaks, things like that, which ends up being what we do.

10:13So we think for our industry, it's great in general. From a workload perspective, we already see actually an explosion of the workloads in terms of providing AI services or consuming AI services. So it actually consumes a lot of infrastructure to train those models, to run them. So we're going to see a lot more of that. We also see a lot of new technologies that are being used there, new components, this whole new stack that is emerging. So overall, it's exciting at every single level. But to your earlier point, though, it's still very early. So it's hard to tell what actually is going to be the killer app for all of that in six months, in a year, in two years.

10:58It's possible that some of the things we've seen with LLMs, where all of a sudden everything is a chatbot, it's possible that it's not the way people want to interact with everything two years from now. For example, when you start your account, you don't want to play 20 questions with it. You just want to start it. It might be the same thing with a lot of the products that are today starting to implement LLMs. But what I think is pretty certain is that we're going to see an expansion of the use cases, an expansion of workloads, also maybe an acceleration of the transformations and whether it's digital transformation or cloud migration that are making all of that possible.

11:38If you want to adopt AI, you actually have to have your data digitally. I mean, it sounds obvious, but it's not the case still for most companies. And second, you also have to be in the cloud. How else would you do it? If you tried to build everything on-prem today, you wouldn't even know what to buy because the technology is changing so fast. So I think it's accelerating all those trends, which is very exciting. I've definitely seen a lot of enterprise buyers sort of change their mind almost on a monthly basis in terms of what they view as the primary components of the stack that they're using.

12:09And that could be the specific LLM they're using. Should they use a vector database or not? The whole set of components seem to be very rapidly morphing. And you mentioned earlier, you were sort of seeing this emergence of a new stack. Are there any specific components that you'd like to call out or that you think, you know, are going to kind of stick? Or how do you kind of view the evolution of this area? So I would start with saying it's extremely hard to know what's going to stick in the end. And for us, you know, it's actually a very new place for us to be. You know, as a company, we've been very good over the past 10 years at understanding which trends are picking up, what actually are going to be the winning platforms.

12:48When the world went from VMs to cloud instances, from cloud instances to containers, from containers to managed containers with Kubernetes, from that to serverless, it always took a year, two years, three years for those technologies to gain mass adoption. And it was very clear what the killer apps and what the winners were going to be. With generative AI, it's not the case. It's changing so fast. And everybody's exploring all of the various the permutations of the stack and all the various technologies so fast that it's really, really, really hard to tell what's going to stick. You can take a guess.

13:26I think the one thing that's been the most surprising was the speed at which the open source ecosystem has been innovating and building better and better models. Nobody has quite caught up to OpenAI yet in terms of what the frontier model is and the maximum level of performance you can get. But I think we've all been surprised by the amount of new technologies that has come out in the open source that does as good or even better on smaller, more identified use cases. And I think we should expect to see a lot more of that. So when we look at our customers and what they're doing, and we do serve the largest providers of AI, but also the largest consumers, we think we see that today everybody's testing and prototyping with one of the best API-gated models, like typically the OpenAI ones.

14:13There are a few others. But everyone is also keeping in the back of their minds what they can still, what they can then bring in the house with an open source model they train themselves and host themselves, and what part of the functionality they can break down to which one of those models. So I think we probably will have a very different picture a year or two years from now. Yeah, it definitely feels like the most sophisticated users are basically asking when can they fall back on the cheapest possible solution and when do they need to use the most advanced technology and then how do they effectively route a specific prompt or user action or something else against those.

14:44So it's very interesting to watch this evolve. I know that you all released an LLM API monitoring product. Is there a whole new observability and tooling stack needed for AI? And if so, what are the main components? And if not, you know? Yeah, well, first there is an observability in the stack needed period, right? So you do need to fold that in because that's just one more component you're using. And as I said earlier, there's a whole new stack that's emerging. So if you're going to use a frontier model from one of the big API-gated providers, you need to monitor that. You need to understand what goes into it.

15:18You need to understand how it responds to you, whether it responds right or wrong, and how it interacts with the rest of your application. Then if you use a vector database, or if you host a model yourself and you use specific computing infrastructure for that, and you have GPUs and things like that, you also need to instrument and observe all that and figure out how you can optimize all this. So there's a whole new set of components from this new stack that needs to be observed. And that, you can say, can be observed pretty much the same way anything else can be observed, which is with metrics, traces, and logs, and that sort of stuff.

15:53I would say there's probably also a whole new set of use cases around what used to be called ML Ops, or not might be called LM Ops. And that's a field, by the way, we're only watching from afar over the past few years. And the reason for that was that, you know, we saw 100 different companies do that, but few of them, you know, reaching true traction. And the reason for that was that the use cases for that were all over the map. And the users tended to be very small groups of data scientists that also prefer to build things themselves in a very bespoke way. So it was very difficult to actually come up with a product that would be widely applicable and that would be, you know, also something you can sell to your customers.

16:37I think today it's changed quite a bit because LLMs are the killer app. Everybody is trying to use them. And the users, instead of just being a handful of data scientists in every company, end up being pretty much every single developer. And they are less interested about the building of the model themselves than they are about making use of it in the application in a way that is reliable, makes sense, and they can run day in and day out for their customers. So I think there's a whole new set of use cases around that that are very likely to emerge and be very valuable to those developers. And these have more to do about understanding what the model is doing, whether it is doing it right or wrong, how it is changing over time, and how the various changes to the application can improve or not improve them all.

17:24It seems like one of the things that has given Datadog enormous nimbleness is this unified platform that you've built, which is both a big advantage and a big investment. And, you know, my understanding is a pretty large proportion of the Datadog team is working on the platform right now. How do you think about resources being allocated towards the main platform, maintaining it versus new initiatives like AI and LLMs? Yeah, so the rule of thumb for us is about half the team is on the platform. But that relates to what we do, right? We sell a unified platform. So internally, as I said, half is on the platform.

17:57The other half is more on the product side, like specific use cases. But even the way we organize those use cases, those teams that work on the use cases, tend to be more focused on problems. They tend to be forward-looking on where the market's going. Whereas what we sell to our customers tends to be more aligned to categories that tend to be more backward-looking, which is how people are used to buying stuff. And that's very important. When you talk to customers in our space, there's like 12, 15, 20 different categories, always interesting acronyms. And they correspond to things that customers have been buying for 10, 15, 20 years.

18:33And that's how they understand the market. So we sell into that while at the same time delivering everything as part of a unified platform that is itself shaped towards more where we think the world is going to be. So it's very possible, even likely, that five or 10 years from now, our SKUs and our products have changed drastically because they correspond to the evolution of the market as opposed to being pinned into very specific and static definitions of categories. An example of that being observability is emerging as one big super category that really encompasses what used to be infrastructure monitoring, application performance monitoring, and log management.

19:09and we still sell those three as different SKUs today. But I think it's very likely that, you know, five or 10 years from now, you don't even think of them as being separate categories anymore. Like they really become part of one, you know, super integrated category. I would say there's a specific cost, you know, when it comes to maintaining a unified platform, you know, which is that we also do some M &A and we acquire companies such as, you know, Screen, the company Sarah was on the board of and gracefully signed the order to sell the company. But when we do so, the first thing we do is we actually replatform the companies we've acquired.

19:44So we spend the first year, really, post-acquisition, rebuilding everything that the company had built on top of a unified platform. So it's an extra cost. But again, what we deliver to our customers is end-to-end integration, bringing everybody into the same conversation, into the same place, more use cases, more people, more different teams into the same place. And we see it as a necessary part of maintaining or differentiation there. You've made a handful of other acquisitions to expand the product suite and I think the sort of talent group at Datadog. What else has made them successful? Because, you know, it seems to me that it has really like continued to drive like useful product innovation at Datadog, which is not always true with acquisitions.

20:31Yeah. I mean, as you know, making an acquisition is easy. Signing a piece of paper and wiring money is super easy. Anybody can do that. The problem is what happens next. So now you've done it. Now you have to merge those two things and make them work. I think, in general, the way we approach acquisitions is they always correspond to a product area as we want to develop. And we're fairly ambitious. There's a lot of different product areas we want to cover in the end that span from observability to security to a number of other things. At the end of the day, we're ready to build them all, but if we can find some great companies to get us two or three years of a head start in a specific area, we'll do it whenever we can.

21:18So we start with a very broad pipeline or very large funnel of companies, And then we, you know, we focus basically on the ones that are going to be fantastic fits for us post acquisitions, meaning there are teams that want to build and entrepreneurs that can really take us to the next step there with the experience they've gained in a specific area. One thing we're very, very careful of is, so we select for entrepreneurs who want to stay and want to build as opposed to entrepreneurs who are tired and want to sell out. You know, it's a fine reason to sell your company. It's not a great reason for us to buy.

21:52So, you know, that's what we're going to do. The other thing we do is when we close the acquisition, but before we've closed, we have a very, very specific and very short fuse on the integration plan after that. So we have a plan basically that calls for shipping something together within three months of the acquisition, which is very short. Because the acquisition, people celebrate a little bit, you know, and then they have to get oriented and HR systems and whatnot. Three months is a very short time. We don't really care what gets shipped within three months, but we care that something gets shipped within three months.

22:27And what it does is that it forces everyone to find their way. It also makes it very easy for the acquired company to start showing value, which then builds trust. The main issue you have when you acquire companies is, or the main risk, is that it's not that you waste the money of the acquisition. It's that you demoralize everybody else in the company. because they see these new companies being acquired and they don't understand the value or they wonder, you know, why you might pay so new people a lot of money instead of paying the people you have a lot of money to do the same thing. And it's very, very important to show value very, very quickly for that.

23:06And we put a high emphasis on doing that. So far, we've done it well. You know, I'm still expecting us to make some mistakes someday, but so far it's worked out for us. I want to go back to something you were saying, right, around like calling, let's say, environmental technology changes well, like, you know, progression from like VMs to containers to here we are with managed Kubernetes and such. But Datadog as a company has always been amazing to me because the spectrum of things that you want to, you're ambitious to go after is very broad, right? Like I was at Greylock investing in companies that were APM companies and, you know, logging companies.

23:44And you have this platform advantage, but you also are still attacking many different things, customer problems and different categories. Can you talk a little bit about how you organize that effort in your mind or as a leadership team and how you sequence it? So in terms of what we go after, first of all, it took us a very long time to go be on our first product. So I think we spent the first six or seven years of the company on our first product. And the reason for that is it was really hard to catch up with the demand for that product. We also realized after the fact that we were fairly lucky in terms of when we entered the market.

24:22We had an opportunity to enter what's a sticky market, a market that's hard to displace because of the replatforming that came with the cloud. So we could start with a smaller product and then expand to it as customers were themselves growing into the cloud. Everybody was new into the cloud at the time. So we had to spend that time getting to the, I would say, the minimum full product for infrastructure monitoring. After that, what has driven the expansion of the platform was really what we saw our customers build themselves. So before we started building APM, we had our customers, or we saw a number of our customers build poor man's APM on top of Datadog.

24:57We didn't have the primitives for it, but our product is open-ended enough that they could actually build and script around it and do all sorts of things. So we saw it. It made perfect sense. If it made sense for them to build it and for us to be part of the solution there, we thought it would make sense for us to build it for them. So in great part, that's what guides the development of the platform. The first threshold was really to get from a single product company to having two or more products that were successful. It was not easy. Once we had done that, that's what gave us, for one thing, the confidence to take the company public because we understood we could grow it a lot for a very long time.