MongoDB’s Sahir Azam: Vector Databases and the Data Structure of AI

0:00In the world of probabilistic software, the measure of quality is about that last mile. How do you get to 99 .99X quality? Will the domain of quality engineering that we typically associate with manufacturing apply to software? That really got me thinking in terms of you're not going to be able to get a deterministic result like you would with a traditional application talking to a traditional database. Therefore, the quality of your embedding models, how you construct your rag architectures, there's merge it with the real time view of what's happening in the transactions in your business, that's what's gonna get you that high quality retrieval and result, and unless it's high quality in a world where it's probabilistic, I don't see it going after mission critical use cases in a conservative enterprise, and that is a problem space we're very focused on right now.

1:03Today, we're excited to welcome Sahir Azam, who leads product and growth at MongoDB. Sahir was one of the architects behind Mongo's successful transformation from on -prem to the cloud, and he's now helping the steer Mungo's evolution in AI First World. Mungo's journey into vector databases began with semantic search for e -commerce, but it's evolving something a lot more fundamental, becoming the memory and the state layer for AI applications. We're excited to get to hear his take on the past and the future of vector databases. And what shape infrastructure itself will take in a brave new world of AI agents and applications and unlimited software creation?

1:41So here, welcome to the show. We're so excited to have you here. Thanks, Sonya. I'm super excited to be here. We're going to dig into everything from vector databases to embedding, to knowledge graphs, and much, much more on this episode. I'd love to just start with the big picture question, and maybe your hot take. Is AI going to change the database market? That's an interesting question. I think the related and probably more interesting question is whether it's going to change software development and applications. And I think that it really is. You know, I think we're seeing AI power, generative AI power and applications address a set of use cases that traditional kind of deterministic software hasn't been able to go after.

2:22And I know I've read some stuff from Sequoia around the idea of like, you know, services as software, et cetera, in that whole space. And we firmly see that in terms of our early adoption of what we're seeing in the market. And so that in turn changes the fundamental way we will interact with software. It changes the way business logic of applications will evolve over time with things like agents, and that all has underlying implications on how the database layer will need to transform as well. Can we poke on that for just a minute? So I'm curious, since you guys are operating out of layer, will you see a lot of what's being developed?

2:57What are people developing today that they could not have developed a few years ago before these capabilities emerged? Yeah, I think on one hand, one trend we're seeing is certainly that it's much more easy and efficient to create software than I've ever was before. So, you know, the fact that there will be more software in the world means that there will, that'll have implications in terms of data persistence, storage, and processing. So that's kind of one, you know, sort of related piece. But in terms of use cases, I think the fact that we can now interact with computers in completely different ways beyond just the classic web and mobile applications that we're all used to, you know, the more interactive experiences, I think the blending of the physical virtual worlds in ways that I don't think we can, you know, we've really seen yet obviously as a big trend around how AI impacts robotics, you know, there's a great blog I read from, I think it was from Langchain the other day around sort of ambient angels and sort of, you know, reacting to signals without necessarily intentional human action.

3:54I think we're at the early, early stages of that top layer of sort of human computer interaction fundamentally changing. And I think that can now tackle a whole bunch of use cases in terms of improving the productivity of our personal lives, our professional lives, and go after fundamental productivity that I don't think traditional software has gone after. I think that's the biggest meta -change that I think this all has the potential to go on. Do you have a favorite example? Either one that either a mango customer, obviously you love all of your customers, but do you have a favorite use case either that you've seen one of your customers build or a favorite use case that you yourself use?

4:30Yeah, I would say generally we're seeing more, For, like most things, we see more sophisticated advanced use cases tend to come up first in more risk tolerance, faster moving startups. But for that reason, I'll pick a couple of enterprise use cases that have captured our imagination. One is we work with a large automaker in Europe. They have huge fleets of cars globally. They have a bunch of first and third party mechanics and maintenance sites where people whether they're dealers or other sites where people go to get help and their cars are having issues. And the common problem if I hear something funny with my car, like how do I go diagnosis, means that you typically go in, mechanic who has expertise has to go kind of tinker around figure out what it is and then go through a manual to figure out what their remediation steps is or what parts they have to order to fix it.

5:21We work with them to actually identify an audio embedding model that could allow them to record with a phone, the corpus and semantically match that with a corpus of sounds that are typical problems, that are known problems with their cars or any cars, which shrinks down the actual diagnosis time, you know, typically could take hours if it was a tricky diagnosis to something that could take now seconds. It's almost like shazam for car diagnosis. And then on the other side of it, instead of looking through PDFs or, you know, physical manuals on what the approved remediation steps are to fix it.

5:56Now it's sort of a natural language interface to say, okay, this is the issue that we match to. What should I do next in terms of fixing the problem? And that's all about unstructured data, semantic meaning of the information, both in the problem with that car. And if you extrapolate the business case of that, though, across thousands of dealerships or hundreds of different models and iterations of cars, that's millions of dollars of potential savings for them. and a better customer experience and consumer sentiment around their brand. And so that was kind of definitely one cool one. Another one, and a more very heavily regulated industry, worked with Novo Nordisk, one of the largest pharmaceuticals.

6:37Obviously getting a drug approved is a highly scrutinized process. And so there's this idea of a clinical study report that pharmaceutical companies have to fill out, which typically takes a lot of manual effort to write and structure and review and you know kind of get approved. They were basically able to use a large language model again, train that against all their approved drugs, all the process they do manually, and now they can get that initial draft of a C -S -R as they call it within a few minutes. And so it shrinks a lot of just the initial drafting cycles. The quality of that initial draft is higher than what they typically see if it was, you know, manually done.

7:16And so again, you can draw a pretty quick line towards true dollar or rely savings. On use cases that are not necessarily even bleeding edge in some aspects of what we're seeing in the early stage ecosystem, but are being applied in a contact at scale in industries that obviously have big implications for them and for the customers. So now that the shape of these applications is changing and you know, where they're multimodal, as you said, they're their mean for the database layer. And if you wouldn't mind just giving us the 101 today of like the role that databases play for software as we know it today, deterministic software.

7:55And what role do you say database is playing in this kind of new evolving market for AI applications? Is a good news or bad news? We're excited. So, okay, since we'll add like, you know, a point I lightly brought up earlier, which is if there's more software in the world, which I think generative AI will just make it easier to create more types of software experiences. I think that in general as a tailwind for any data persistence infrastructure technology. It doesn't necessarily mean that MongoDB or any other particular vendor is automatically going to be the beneficiary. There's a lot of execution that goes into making sure we're technologically and for our partnerships and ecosystems, well set up for that, which is where I spend a lot of my time.

8:33But in general, like more software means more data and needs for persistence of that information. That's a very macro sort of, I think, tailwind that we're definitely, definitely excited about. I think the shift from relatively simplistic gen AI use cases, oftentimes we're just interacting via chat with an LLM, doesn't necessarily need very advanced kind of data persistence, but as enterprises need to ground the results of their AI applications to proprietary information or to control the results set so the retrieval is of high quality, now there needs to be a lot of interaction with these foundational models and their underlying about how they run their business.

9:13And a lot of that is not necessarily publicly trainable information on the internet. And so whether that's advanced or simplistic rag workflows, whether that's fine tuning, different approaches around post -training there, I think there will be more need to interact with an enterprise's data and foundational models over time, especially as these models become lower -late in C. And so they interact more with the real -time business data that's being generated in an organization. And that's really what we're seeing in the most advanced companies right now is they're building really sophisticated ways to control the output of these LLMs based on the use case that they're trying to drive towards and merging it with the operational data that drives their application or their business.

9:54And so I think we're still early days in that in terms of where I think that can go, but I really do fundamentally believe the databases will get you to get much better at high quality retrieval in particular of unstructured data. Because when I look at all these embedding models and just what we can do with probabilistic software, it takes the value out of 70 % of the world's data, unstructured data, and makes it applicable to applications in a way that just really wasn't possible before. And I think that's the real opportunity. What's the devil's advocate answer to that? So for example, I'm thinking of Jensen and our first AI Sands.

10:29I think you were at that AI Sands. And he said something like every pixel is going to be generated, not rendered. And I think of rendered as, you know, retrieved database somewhere. What is a double debit point of view to the, you know, is it good or bad for databases as general to AI takes off? Yeah, I think the devil's advocate view to me is less about whether there is a database somewhere behind the scenes. More about where is that abstraction and is that something that's a choice of the application developer building that application? Yeah. Or is it abstracted behind some higher level API or is that a choice that an LLM makes in terms of as it auto generate software or auto renders that environment?

11:05where does it choose to persist that data. But at the end of the day, we like to joke internally, an AI application is still an application. You still need to persist transaction safely to make sure people's bank balances are accurate. You still need the ability to search information based on text keywords, not only on the semantic meaning. And so I view all these generative AI needs from the data layer as additive, not necessarily substituted to the needs of a traditional application. And, you know, one of the reasons people love Mongo today is the developer experience, right? If you fast forward the clock and, you know, maybe there's X 100 million human, human software developers, but there's trillions of call it, adjunctic developers.

11:49What makes a good agent developer experience? Like, why would an agent choose to use Mongo as its database if that even makes, does that make sense as a question? Yeah, I think it does. And it's something, you know, we think a lot about sort of how the nature of software development will change. And I think One of the things is we move for more simplistic, generative AI kind of powered applications to more advanced ones with more, you know, agent -driven business logic, state will be more necessary. Because now you're coordinating, you know, a more complicated workflow where you need to be able to track the results of a particular piece of a transaction and coordinate that and all of that requires storing that somewhere and, you know, manipulating and updating it over time.

12:26So I think in general, things are becoming more state -full in generative AI applications over time, which is a drag of data and database consumption overall, in terms of where things are going. Now, I think in terms of the abstraction, I think the question is if developer experience is the thing that makes any technology really accessible today for human developers, does that same value proposition hold for AI? And I think what we're seeing, even if you look beyond just the database space, think of the adoption we're seeing of some of these, call it AI platform as a service type company, you know, look at the adoption of things like for cell V zero, or you see things like RepLit or whatnot.

13:07I think we're seeing that at least with early AI generated software, there's a preference for great developer experience, I'll hire levels of abstraction. So I think it's too early to be definitive on that, but I think we're seeing some our own promising signs. Speaking of higher levels of abstraction, I forget who had this one liner. Somebody had the good one liner, which was English is the ultimate layer of abstraction. At the limit, you will just be able to describe and plane English what product requirements you have and a foundation model will spit out the code required to build whatever application you want to build.

13:48First off, do you believe in that as a future state? Then secondly, is that great news for Mongo? because there's just gonna be so much more software and most of it's gonna need a database sitting beneath it. Or is that bad news for Mongo? Because it neuters some of that development experience that is a good advantage for you. Do you see that playing out? And what does that mean for Mongo? Yeah, I think for databases in general, I feel pretty confident it's absolutely a tailwind. I think MongoDB specifically one of the advantages we have is that our data model is really well -détuned to managing structured data, semiconductor data and now with embeddings unstructured data.

14:26So I think we have some fundamental architectural advantages, we believe are even more of a prevalent and important in AI is representing all these forms of data. Regardless of whether the software above it that's interacting with it is human generated or machine generated so to speak. Now that being said, we're certainly not resting on our laurels that that's gonna happen without us being really intentional about it. So we are working with the whole ecosystem of AI framework and model providers to make sure that we are well integrated, whether it's inference players or Dev frameworks, et cetera, to make sure that just like JavaScript and Web 2 .0 and Cloud were big tailwinds and our big drivers of our business, that the modern stacks that are being used to generate these applications might be as well integrated as a default in.

15:12So I think there's a lot of work happening there. We're also focused on this idea of what is the equivalent of quality training or even SEO for LLMs. Meaning, if you go scrape the internet to train a code assistant on any technology, is that necessarily what the best practices are? Probably not. But there's no standard way for a vendor or a technology expert behind a particular area to submit the canonical training data for a quality MongoDB code, for example. And so we're working with some of the labs on methodologies around that. we're doing things just even without, involvement to test what we can be doing to create data sets that allow for the quality of the outputs of these systems to be reliable.

15:57Last thing we want is somebody going and saying, I want to use MongoDB, help me generate some code for some functionality, and it's not high quality, performs poorly. And so there's very facets of this that I think are very intentional efforts to make sure that our technology fits well as things evolve over the next year. So actually to that point, I think there's been a lot a chatter, an increasing chatter that we're hitting a wall in terms of just public data globally available. There's a lot of data still left in private enterprise data. You guys sit in the middle of a lot of it. I'm curious how you think about your role in kind of that, as the market of all sources next leg of finding that next trillion tokens worth of training data.

16:41Do you see yourselves being a training data provider for your customers? Do you see yourselves partnering with the labs? Are your customers mostly looking to use their data in Mongo for Rags, or are they looking at also training models on the data they have in your systems? Yeah, definitely. I think just to be clear, any of the data that we manage on behalf of our customers is owned by our customers. So we're certainly not taking that data and training any models that are outside of what that customer wants us to train or use for Rags. So I think that definitely is where more of our focus is. And we see a variety of different things, very simplistic kind of use cases where people are just using core operational data stored in MongoDB or metadata as part of their kind of rag workflows.

17:28We're seeing obviously a lot of vector adoptions are fast -scrolling new product areas. They try to merge metadata, transactional data, and semantic search sort of together into a single sort of system for more quality retrieval kind of use cases which is sort of I think where the market's going. And then we see instances where people want to use the data they have in MongoDB and other systems to either fine tune or straight up train smaller models that are specific to a particular use case. And I don't believe that there'll be kind of one particular modality that suits every single use case. I think there's going to be a plethora of different things that customers will begin to optimize for their latency requirements or performance requirements.

18:08So I think you have the most fascinating seat to what's happening in the vector database market, we constantly pull our portfolio on what their AI stack is and consistently Mongo has been the number one vendor that everyone uses for vector databases. So I think you have the deepest and most interesting perspective on this. Maybe from the 20 ,000 foot view, it seems like people view are using LLMs as, you know, they have world knowledge up to some pre -training cutoff date, but beyond that, you need Ragn, you need vector databases in order to supplement knowledge to provide specific domain knowledge, almost as in information retrieval knowledge source.

18:47But if I look at vector databases, they kind of came from the semantic search world and e -commerce and do that. And so that's very different world. So how do you think about what are people using vector databases for today? Is it a technology of the past that's being improperly shoe -horns into this information retrieval use case or is it the ideal data structure to kind of be the knowledge infrastructure for LLM's? Like, how do you think this all plays out? And kind of some quick question on that too. Of course. Did Mongo, I'm aware of Mongo's vector database because of generative AI and seeing people use it for generative AI?

19:24Sure. Did you guys have a vector database pre generative AI? We started because of a more classic classic now that is a semantic search use case. So a few years ago, one of the things we noticed were that many of our customers would use MongoDB as an operational data or any operational data inside by side with it having averted index search engine for full -text, kind of lexical search. And our customers were basically like, why do I have to copy data between these systems to run two different databases just to get the search results? I want to empower my application with. And so being focused on developer experience and simplicity, we're like, this seems like an obvious problem for us to go after.

20:01And so we started there with our search product to really just simplify it. So a developer interfaces with one database, but really it has different modalities of indexing and storage that can serve, you know, a lot of Tp type queries as well as full -tech search queries. Some of our e -commerce advanced e -commerce customers were the ones then saying, okay, that's great, but I want to start to do semantic similarity search and blend full -text like a flexible search alongside similarity search because that's what's gonna give me higher quality search results. And that's where we started getting pulled into building the vector capabilities into our engine.

20:37And for us, it's, you know, one of the things we were always trying to do is remove the need for customers to have multiple systems. So when we say we added this capability, it's a lot of it goes to how do we integrate it in an elegant way to our data model? How do we extend our query language? So it's very easy for a developer to just feel like it's not a separate system. They're just interacting with it as part of their application development. So we were down that line. Then obviously, the World Explodes post chat GPT and we were like, all right, this is going to be even more relevant than we thought.

21:08So we poured the gas on things, accelerated things, expanded the strategy to be well integrated into a whole bunch of new frameworks, working a lot more closely with the AI labs because it is to a Sonya year point. It's certainly a different use case to leverage vector embeddings or even just metadata or transactional data integrate to RAG, then just a pure semantic search use case. But as we look at our most advanced customers now in 2025, they're actually seeing that the integration of all those modalities is really important because you need to filter based on metadata you know about your unstructured data, whatever it is, your building application around.

21:49There are times when you need to sort by keywords and relevance ranking, like a more traditional search engine. and then you need to understand and extract semantic meaning from vector embeddings, and there's a whole bunch of things around how to improve the quality of that. And only then can their overall application get the percentage, quality, predictability, especially for large enterprise to trust putting something in front of their customers, especially in a regulated industry. And so that's turned out to be a real advantage to have all of those in a single system. Because otherwise, it requires a whole bunch of what I call kind of rag gymnastics to try to tie all these things together, which is possible, but it puts a huge burden around the development cycle, what happens in app code.

22:31And frankly, you need to be a pretty sophisticated team to figure that out in your own. And so we're trying to democratize that all by making it just much simpler for the average application to develop. Yeah. How do you think about Vector versus Graph? Are they substitutes? Are they compliments? What are the trade -offs? Because we see Vector, Vector -based RAC, we also see Graph RAC. Yeah, and every week goes by and there's some new sort of approach to higher quality retrieval. It's kind of what I think everyone's sort of trying to chase. I think they're complimentary. You know, there are reasons why you want graph relationships because that's an augmentation of understanding that you may not be able to just infer by the vector embeddings themselves.

23:12So we view that as additive, just like pre -filtering based on some sort of metadata you know about your unstructured data and embeddings is additive and improves the quality of results. So I do view these modalities as very complimentary. Our goal is to just make it simple to combine all of those for a developer so they don't need to have their graph representations of their objects in one style of database. They're metadata and another database. They're transactional data and relational database. Then have to have a separate vector search database and try to rationalize all of that, which is kind of what happens.

23:46We're trying to just make that dead simple. Is it fair to simply think about, you know, an agentic system, the LLM as the brain, and the database, whether it's a vector database, or a super set of those, as the memory? Is it brain and memory? Is that the right mental model? I think that's definitely one way to think about it, because absolutely you need to persist memory in state, especially when you have agents that are having more complex workflows and need to drive interaction across multiple endpoints, not necessarily a single foundational LLM with a one -shot call. So you need to persist more of that state.

24:22I view them as sort of two pieces of an emerging architecture. You've got obviously compute storage, networking is sort of the underlying primitives. But now there's this whole set of use cases that foundational LLMs can go after, that are more probabilistic in nature, that can automate tasks that knowledge workers would typically have to do manually, which is super powerful. But then that needs to store at state and be grounded and interact with the transaction that the application is driving and the other information that's either semi -structured or structured. And those things together come to create a great application experience and end user experience.

24:57It's not in either or I think it's complementary in a really powerful way, which will only become more important as LLM's become lower latency and faster. Where now you can really use what's happening in a real world setting to augment the results of an LLM and much closer to real time than today, where it's just a very different interaction speed. So your thing with database is not only the memory for the LLM, but it's a reflection of world state. Yeah. Like you with LLM needs to interact with world state. Well, I think that rough framing is consistent with what we've talked about internally, which, you know, if you think about the bottom as raw infrastructure compute network and storage.

25:33You think about the top as the application. You've got all this stuff in the middle. And for anything that is deterministic, you're going to be better off with vector database, graph database, relational database, no SQL database, kind of the traditional database world. For anything that's more probabilistic, you want something that looks like an LLM. The functionality that gives you is a little bit of human computer interaction and a little bit of reasoning, which is complementary to what you get from this part. the world, but I want to take it one step further because it sounds like we're a pretty similar view on this default architecture of the future or kind of this emerging pattern.

26:09If you take it one step further, does that imply that the mental model investors should have for the API portion of Anthropic or OpenAI or the other Foundation Model Companies is Mongo, meaning they're occupying a similar layer in the stack. They both reside on top of the public clouds. They both reside beneath the application layer. Is Mongo a good frame of reference for what the API businesses of OpenAI and Anthropic water shooter could become over time? Yeah, I think it's an interesting proxy because you sometimes read like, okay, the LLM is the new operating system. That never felt logical to me in terms of application capability and functionality.