Databricks CEO on AI Pacing, Cyber Risk, and the Enterprise

22:42We are rushing. The industry is rushing to do that super, super fast. If we don't do that, I do think you will start seeing those kind of things, like sites going down, you know, whole systems that stop working for a while. And there will be consequences, not existential, but economic damage and, you know, people getting hurt and so on could happen. So we just have to race very, very fast to do all of those things. It's just you don't have the humans don't respond fast enough to the attacks that are happening. So you just you need to automate all of those. And most organizations are actually not close to doing that.

23:14The banks are doing it. Some of the people that are super security conscious are doing it. But most of the industry today is running with old school security operation centers and people that are waking up every day. And there's like hundreds of emails of detections that have fired. Many of them are just false positives. So you don't need to, you can ignore them. But some of them are not. They just don't have time to go through those. And you need to identify that. You need to have threat hunting that's automated where you're actually attacking your own systems automatically with agents and so on.

23:41It hasn't happened. So I do think if we just say, hey, this is just like the internet in the early days, bad things are going to happen. So there is a race going on. I was actually very surprised. Earlier this morning, I was on a conversation. I feel like you and I are pretty close. We talk periodically. I feel like I know a fair bit about Databricks. I was on a call this morning where a founder was basically like, yeah, listen, we're doing all of this observability, agent threat detection. Am I using Databricks? Like, I didn't even know that you had this offering, like, quite frankly. So, like, I mean, just from an education standpoint, like, how extensive have you gotten in, like, the agent AI observability security safety thing?

24:22Yeah, I mean, we gave a talk this year at RSA, actually with Ben Horowitz, but the issue is that data and AI is blending with cyber. These two markets are collapsing. Because I think most of them, at least. Yeah, and the reason they're collapsing is that it used to be like, okay, we have, like, data and AI, the kind of stuff Databricks and these kind of companies used to do, which is like, okay, you have a bunch of data and you run AI and machine learning and that just lives separately. And then you have the cyber world. Cyber world is, you know, we want to detect if something bad, if like bad people are trying to hack us, if bad people are doing things, we need to detect that.

24:53Okay. But now on the data and AI side, we have agents running. Internally in the company, people are having agents running. And the agents are also like doing things with other people's agents and they're producing a lot of data, logs, trails, you know, fingerprints that are being left. and so you know now we have internally these agents that are doing that so these worlds start merging more and more which is like okay well all the data that's being produced needs to be analyzed and the scale at which you need to do that is just like many many orders of magnitude more than just one or two years ago so things have changed dramatically like 2018-19 the time it would take from you know a CVE vulnerability being sort of published until you see it actually weaponized in the industry would be like two, three years.

25:38That went down to, you know, 2022 significantly, but it was still like eight, nine months. So that's kind of fine. You have eight, nine months from a vulnerability to that was 2022. Now, if you look at this curve from 2022 until now, now it's down to like basically hours. So it's like down to like basically no time. Like things get immediately weaponized. So you need to just do it in an automated with the data and AI sort of platform approach. So these markets, I'm going to argue, or they're just going to collapse, actually. So it's a very specific question. I actually think a lot of, like, to pull back on the existential, X-risk discussion, it feels like there's almost two camps.

26:14There's one camp which believes that this actually is an engineering problem and, like, companies like Databricks can solve it and they can solve it through product and through engineering solutions and through services. And so, like, we just, as an industry, need to solve that problem. And there's others which actually believe, seems to me, that there is no engineering solution. You have to, you know, slow it down. You know, you have to use regulation. It's more like a nuclear weapon, et cetera. So, like, does this mean you believe it is an engineering problem? Or are you not quite comfortable saying that yet?

26:46Because why do you even buy Databricks, man? Let's just put this stuff in a national lab. Just pace it is the only solution. I'm just kidding. Everybody just paced themselves a little bit. They'll be fine. The bad guys do. There is no line of inquiry ever that gets to pacing, I don't think. I think it's like you pause it or like you solve it. What we've learned here is that Martin really hates the word facing. I will never use that word with you ever again. Clearly. Duly noted. More of a frustrated Martin. Yeah. Is it an engineering problem that can be solved by engineers or is there more to it?

27:14I actually think which problem are we talking about? There's two separate problems that I think are being conflated. There is the super intelligence problem. Yeah. You know, and I think a lot of this comes from like Bodstrom 2014 super intelligence book. And if you look at the definitions, like I think people don't have these clear definitions of what super intelligence is. If you read his book, those definitions are kind of crazy. Uh, so I think what he had in mind when he said super intelligence, uh, is, you know, AIs that, I don't know, I don't know what the examples were. Something like, yeah, something like they write a whole PhD thesis with novel, like peer reviewed stuff in a couple of seconds and they can do like millennia worth of thought, you know, in like instantaneously.

27:53And, um, you know, so this is like the level, uh, of, you know, how fast they are, how intelligent they are. They can learn. Yeah. Yeah. It's just, yeah, I mean, yeah, but it's just many, many, many orders of magnitude, right? It's like, it's just the scale of the problem is just completely different. So if such a thing exists, do I think that's just an engineering problem to solve? No, I think that's actually very, if such a thing would happen, that would be very existential. Of course. And that's what everybody agrees on. So I think that's being mixed with, now we have agents that are nowhere near that.

28:25It's not even like, there's nothing like that, and we don't have anything towards that path right now. but these agents are capable and you can do something with them that you could never do before in a history of mankind. So I do think an inflection point has happened. Something has changed, which is we have good security researchers at Databricks, but I could never say let's get 10 ,000 of them in a sandbox for a month and have them do$100 million worth of salary wage work. We can do that now. We just turn on a button and we can get 100 ,000 of them. Or mathematics. We can say, hey, we want to solve a conjecture.

28:57Okay, let's get pretty good mathematicians, but let's have 10 ,000 of them collaborate, you know, and then you can like make very fast progress. So this, I think, is an, this leads to all these cyber risks. I think cyber is the major problem here. This is, I think, you can solve with engineering. And I think it's like, we are working on it. Many others are working on it. There's still risks. They're not existential. I think we should do it. There is the superintelligence thing. That's the thing that could do, write a novel PhD thesis or like, reason intuitively in 11-dimensional space physics instantaneously without writing anything down, something humans can't do.

29:32Like that kind of superintelligence. The question is, is RSI and recursive self-improvement that the labs are doing leading us there? Are we going to get there? Trying to do. Trying to do. Is that what's going to happen? And how fast is that going to happen? That's the big question. And they've suggested that, you know, hey, we should have inspectors that come in and look at what we're doing. I think that's a good idea. Have them go in there and get the data. I would love to like the question is who are the inspectors because you can like you can stack right you can stack that who do you think Martin yeah there's a bunch of people that like on either camp actually I wouldn't care if they're if they're the inspectors I would not be very impressed by what they say because they've already made up their minds even before they are they would go in there right exactly but let's say like as an example if Jan Bakun who was one of the inventors of this you know deep neural network technology right uh one of the pioneers if he said hey there's nothing to see here.

30:24There's no risk. You know, I'm paraphrasing him. This is nothing. This is just nonsense. Keep on going. Go fast, fast, fast. None of us wouldn't believe it. I'm putting words in his mouth. I mean, I'm not exactly right. Now, if he was one of the inspectors and he went in there and he had a look and he came out and he said, hey, I've looked and it's just what I said. There's nothing to see here. Just keep going. I would feel very good about that. I would say, okay, well, I would feel very good. Or if he comes out and says, oh my God, you know, he's wobbling and he would change his mind a little bit, that would also have a lot of interesting signals.

30:53So I think it comes down to who we pick as inspectors. And I think it's a good idea. Let's have some of them and pick a diverse set of people so that we can get different nuanced points of view. What do you think about this kind of Elon Musk view, which is like, it's less third party. It's more, it feels like there's kind of three proposals. Like the OpenAI, Anthropic one is a third party. Yeah. The Elon Musk one, as far as I can tell, is the labs cross-check each other like peer review, like you do in science. Yeah. And then the Mark Zuckerberg one is please yourself, right? What do you think about this middle one?

31:26That they should face each other, like, you know, evaluate each other. I mean, I think like if we have boxing matches in the ring, the boxers should just be the judges of each other. Would that work? No, they would scream foul all the time. Foul, foul, foul. Like, you know, it's like, ah, you know, the moment the other guy puts out the great model and it's like, ah, big super intelligence risk. Absolutely. Like, you know, they have not been responsible. Like, you know, when vested interests are at play and there's like IPO plans and these two companies are so competitive and they have like this history also between them.

31:59Yeah, they'll be very fair to each other, I'm sure. That's why you need a third party, right? Why do we have judges in the world at all? Why do we have third parties at all? Like, why can't just people like figure things out between themselves? But I mean, they should try. If they want to do it, they should try. But I'm skeptical that they wouldn't just, you know, be biased, you know, in multiple ways to self, like, you know, judge each other. So I have to ask Ali, do you think something Elon also said, I think it was on the All In, you know, summit, he was like, this is some elaborate 4D chess, because on the one hand, you're saying all of humanity will die.

32:34On the other hand, you're saying, hey, what do you want for your IPO allocation? Right. And so, I mean, that is probably a more cynical view. But like, how do you reconcile that? I mean, the dissonance, I think, gets a lot of people. Like, how do you think that gets reconciled? Look, I think all of these things get mixed. Like, I think there are people that are freaked out. And I do think that there are people that are saying like, hey, if there was a regulation that would pace us, sorry to use the word, that would be good for us. Right. That would be good for us. But I also think that people have that's an interest.

33:02Yeah. Right. these things like you know usually people figure out a way to always get all of these things to align in their you know harmonically in their head uh so yeah do i think that there has been a tendency in the past of in general using also marketing stunts uh by saying you know oh my god this latest model is so good that i trained it's like unbelievable it's like almost scaring me and then the whole world like kind of starts focusing on it yeah there's been that kind of marketing going on. But at the same time also, as I said, the time from CVE to actually weaponize exploit has like been going down from years down to like minutes now, just in like three, four years.

33:40So it's real. The cyber attacks are real. But there's also a great marketing ploy to, you know, whenever you train a new model, make lots of noise around how much of a, you know, crazy risk it is to the world. It helps you, right? So, you know, maybe they're not in contradiction, these things. I mean, you and I are networking folks, and there's a long history of forming third parties to help arbitrate things, right? Like IETF or IEEE or even like ICANN. I see where this is going. No, no, no. So my question to you is like, so I think it's actually, this is a very sensible proposal that they actually have.

34:20I actually agree with you. You probably want to make sure it's independent, which is not good right now. And there's going to be a lot of arguments So who you put there and everybody's going to disagree. Right, right. But you said, you know, why do we have judges? So that's like, actually like the state stepping in is actually quite a different thing than basically industry self-policing. So like at what point in time do you think it makes sense to actually consider federal involvement? Or do you think like now is the time to actually consider actual federal involvement as opposed to like more industry self-policing?

34:45Well, I mean. These are very different. They are different, but they kind of bleed into each other. Like, you know, like for instance, FINRA, you know, is not like a completely independent self-policy. It is, but, you know, it's linked to the government. So, like, I think these things, like, kind of will bleed over. I think it's hard for them. Do you think that they evolve into, historically, they've, like, the industry's up, polices, and then it evolves into regulation. Look, if they are saying there is existential risk, which they're saying, you know, and they're saying, come police us and regulate us, I think it's very hard for regulators to say, no, we're not going to do that.

35:13So far, they've said that, but I think that's not going to last very long. It's so funny. It wasn't David Sachs. It was like, I've never had a CEO ask us to regulate them. And my favorite thing is, like. I've never had a regulator that says no to that. I mean, the reality is, like, the actual, like, metapolitical machinery is actually in motion already, right? I mean, like, everyone has a talking point. Obama has came out. It is a major issue. Like, do you think that there's a reality that it's too late? This will be a major issue in the midterms, and we're actually going to, like, heavy-handed federal regulation, and this is all going to be paused, you know, and goes into the—anthropic goes into the DOE, and we're past that point?

35:50or do you think we can actually end up with like a sensible self-policing regulation? Because this is a headline you cannot. We should strive towards doing the right thing. I think there's still some degrees of freedom of how things evolve and there's still time. And yeah, you're right that largely you have these companies where we're pumping in so many billions of dollars and the way reinforcement learning works is that you, you know, give it the reward function that's verifiable. Like we're going to solve this math problem or this kind of, you know, this narrow area of programming and so on.

36:17And we pour in so much money into that. You can get quite good results in that narrow kind of, that doesn't mean that you're getting that super intelligence. No, but you can even trick yourself into thinking that like less inputs are giving you a better outcome just because you're running so many experiences and you've thought about it so much, right? But it's actually very hard to do a closed experiment this way given how many resources are going on. Well, the fundraisers are going up astronomically to your point. Yes, yes. That's why these companies are going public, right? I think they would otherwise, they would stay private.

36:46I mean, as someone who runs a private company at scale, I think they would prefer to stay private otherwise. Why are they going public? Because they need the capital. And they consider the scaling laws and the capital to be a strategic advantage. So that's why they're going public. But I would say, let's go back to the four things that I listed. If those are true, you know, if those four are true, would you want to know about it? And would that be worrisome? That could get out of hands. Now, there's no evidence that those four are happening. But if there was like, you know, no, that is actually where it's headed.

37:15Yeah, I actually think understanding for any system, Yeah. Like any sort of self-propelling property is important. I mean, we've done this in the past with dynamic systems, right? Like we've done this with like whatever, compilers. We did this with all the research on nanotechnology. Like it's been a common interest of ours. Yeah. And I don't think that that's new, that it's an interest, which you continue to have the interest. I just think the fear is that these particular systems are net economic systems that are so complex that the risk is crying that you're seeing it when you're not seeing it.

37:47And I think a lot of that's happening right now. Yeah. But of course, if you see it, you know, of course. I mean, you want to know. But it is fair to say that the labs are now focusing a lot on RSI, and that's where they're headed next. And maybe they're just unjustifiably worried themselves, just like they were worried about GPT-2, right? They were like, GPT-2 is world-ending, and then it wasn't, and GPT-3 and 4 came out. So I don't want to quibble. A lot of the times when they say RSI, they're actually talking about autocatalytic effects, and autocatalytic effects have been in our industry for a very, very long time.

38:15So, for example, there's no way you can create a computer chip without a computer chip. It's like you cannot do it. Anyone with computer science degree. Compiler writes its own compiler. Well, that becomes closer to RSI. But the Steam Engine was autocatalytic. So listen, my full-time job is people coming out of labs and starting companies, and they all say RSI because everybody says RSI. And maybe 1 % of those are Axie RSI. They're more like, we use AI for data cleaning. We use AI for making... Let's make a distinction. So you're saying it's basically a catalyst in the sense that they're using AI to speed things up.

38:52It's autocatalytic. Yes. So I would say of what we hear from RSI. Like a model trains another model, right? You're using a model to build a GPU kernel. You're using a model to do data cleaning. Just like I use a computer to design a computer. It's autocatalytic. Which is every tech. The internet was autocatalytic because it allowed people to collaborate remotely. So I would say, this is anecdotal. 90 % of the calories are autocatalytic, which is 100 % what you would expect. And that's been going on for a while, though. I mean, that's not even new. But I would say, but also now there is a focus on let's move towards actually, can we get the model to train itself?

39:27It's kind of like the auto research that Carpathia did, but now they want to do that. There are teams that do that. It is not nearly as many calories as you would expect. And I just feel like I actually have a good sampling of this because they all come and talk to us. Right, so can we, Maybe you can be one of the inspectors. Can we get all that data and all of us look at that data? Maybe there's nothing to see here. I personally don't think it's like very high probability that those four criterias are happening. Greg Brockman went on the pot this week and said we're in AGI era. Yeah, so now I asked the same question after he said that and now everybody's saying we have AGI.

40:00So, you know, people follow what they say. But people have for a very long time, when I asked this question, said that AI is smarter than most people around me most of the time. That's been almost like since Q3, Q4 last year, they've been saying that. then I ask them how many of you have, you know, hundreds or thousands of agents that you are managing that are coordinating with each other in swarms and negotiating and, you know, automating your life and everything around you. And if so, raise your hand. It's like almost nobody raises their hand. Of course, Martina has done that at home. No, no, most enterprises are on Microsoft Copilot.

40:30Yeah. Like that's the extent of their AI. Most enterprises I talk to when I ask this question, they're like, no, we don't have any of that. So we're like, what are you doing then? They're using a chatbot. Like they're asking questions from a chatbot. that's basically very, very glorified, efficient Google search of the old day. The results is just faster Google search. And then coding is happening. So people are using it for coding. Though the ROI is, you know, we can discuss the ROI there. But there's no, like, agentic, like, work that's automated the whole enterprise. That has just, like, not happened.

41:00So then why is that? And I think that the real reason is, if you actually look at it, is the models are smart enough. But they just don't have the context that exists inside of any organization. They have not been in every meeting. They don't know what's in everybody's heads. They don't know all the processes. There's always a couple of employees who know everything in every organization. You go tap on their shoulder and everybody's like, oh my God, what would happen if he or she quits? They don't have that context. And if you just fused that and gave that context into the AI models, just the frontier today, I think there's so much productivity gains you could get for any organization on the planet.

41:35For that, we actually don't need smarter models. So we don't need a smarter model that can actually solve Navier Stokes or conjectures or do better on humanity's last exam. Like, we need it to just go from 60 % to 70%. None of that is needed. So I think, actually, people are very upset on something. Like, oh, if we pace the frontier. But actually, if the frontier doesn't advance, it doesn't actually matter, I think, for the vast majority of organizations on the planet. They're just so far behind in the adoption curve of actually automating things and getting value out of this stuff. But it would be disastrous in labs because the price of intelligence is dropping asymptotically.

42:09I think it's going down by one-tenth every six months or something like that. So that would dramatically change their businesses if, like, you weren't pushing the frontier. Yeah, but this is what we should focus on, right? We should focus on, like, you know, there's two sides. We discuss here a lot the costs. There's cost-benefit analysis that we should do on everything, right? We've discussed the costs a lot here. Like, oh, is there, like, existential threat? Is there cyber risk? are there things we should be worried about and so on? That's like the cost side. What's the benefit? And I think now that this has become like a public thing and the whole public cares about AI, they're asking, hey, what's in it for me?

42:41What am I getting out of it? It seems nothing. So how did they get there? What are some of the use cases you've seen to date that have maybe surprised you to the upside? Yeah, I mean, first of all, there's like so much worry about, you know, existential risk and so on. So I think a lot of people just don't know what are, you know, cool use cases where people are actually doing interesting things. We have a lot of use cases that are, I mean, just fascinating. One that I like is Crisis Text Line. So, you know, they actually use large-angleic models with us to detect if teenagers want to do self-harm or suicide.

43:15Oh, wow, yeah. That's an awesome use case. So it actually saves lives. So that's a great company. And that organization is doing amazing work. Another one that's kind of interesting is the Omnipod, which is for diabetes patients they can put the Omnipod and it uses AI to really learn your insulin release and your glucose levels and actually exactly release. I don't know if you remember people used to like stick themselves, right? But this now happens automatically and it's like, you know, self-learned AI for your body. You know, it's a cool use case. Zipline is another one. They're doing awesome.

43:50You know, but when they started it was like these drones that had, you know, they were completely automated, all AI driven, everything from the, you know, battery optimization to the routes and everything, and they were delivering food in, you know, areas of need. Yeah, blood to refugees. Blood to refugees. Yeah, started in Africa and then elsewhere in the world. So, yeah, that's all, yeah, it's AI use case, you know, built on Databricks. So it's a cool one, but there's more advanced ones also. Like, one that I kind of like, but it's harder to maybe explain is this model that we built, a transformer-based model that we built with Merck.

44:21It's called TEDDY. Transformer Enhanced Drug Discovery. Love that.

44:56So that's a super cool use case. There are lots of these. You know, Jeannie, I mentioned, you have this ontology and you can't ask any questions. Novo Nordisk is using this. So, you know, they built this GLP-1 drug. But what Novo Nordisk is doing is now they're using it for all of their trials that they're running. And it can compress the time it takes to get insights versus if you're doing an obesity study or something from weeks down to minutes. So there are a lot of amazing use cases of AI. we should not forget these upsides also. Like we want all of these and we do not want to paste these.

45:30Yeah, exactly. You're totally right. And so how do they, let's say if you map out the next 12 months, how do the enterprises actually get value? You dropped the word context, but like how do they operationalize that? It's actually harder than most people believe. But, you know, first and foremost, we have to make sure that we have digitized everything that's happening in an organization. That actually, you cannot actually just, you know, have a magic wand and make that happen. So, you know, every meeting has to be transcribed. You know, so you have to be able to get all the context of all the meetings and everything that's happening, all the digital content has to be fed to the AI.

46:06So you have to build, we call it an ontology, we build that. But first and foremost, you have to collect that. That itself is a problem in many organizations because legal teams will say, don't record every call, don't record every thing. So you have to do that in a way where... Can you define ontology for everyone? Because I know Palantir says the word a lot, But it's not like they own the word ontology. What does that mean? And for the people listening, how should they think about it? Ontology just means that in an organization, the relationship between all the abstract concepts of all the goals and all the departments and all the people and all the projects that are going on, what do they exactly mean?

46:41And what's the relationship between them, the people, the resources, and what that company does? So it's the difference between a person who is a new employee in the company and just started today and a person that has worked there five years. You know, let's say they're equally skilled. They have the same educational background. They're equally smart and hardworking and all of that. But one, it's, you know, her first day today at work. The other one, she's been there five years. What's the difference between these two people? One has an ontology of how that organization works, who the people are, how you get stuff done.

47:14Don't look at the org chart. Don't go ask that person. He will not get anything done. You go ask this person, you know, he'll get it done for you. and that's not how it works. You don't need to file that paperwork here. And this project, this is what's going on. This is essential. So there's just a lot of ingrained knowledge that's sitting in everybody's heads. Who knows how an organization works? That's why people say in startup land, they say, hey, if you lose most of your people, that company can't recover from it. You can't just replenish and hire new people. Like the people are so essential.

47:45How do we get that context, that's the ontology and give it to the AI? Part of that is we just have to have the recording and all of that. But the second part is, how do you actually distill it down into a graph? Actually a digital graph that you can then feed to the AI. So the way a lot of the agents work today, like a cloud coder, any of them codex or pi or, you know, open code, you can go through the whole slew of them. You know, they have this loop, agentic loop, it can reason, but then it goes and checks every resource one at a time. So we'll go to this MCP server for your question and try to see, is the answer here?

48:20Is there another one? it synthesizes it and gives you an answer. But it's kind of slow. I liken this to if Google would have built Google Search this way 25 years ago, we would have said, okay, we're going to get 10 blue links. We search for key terms here. But instead of giving you 10 blue links, it would have gone to one website, summarized with an LLM what it does, found a few hyperlinks, jumped in parallel to a few of them, read a few websites, done that for 10 minutes, and then giving you like its best 10 blue links it would find. Well, that would be very expensive, costs a lot of money to do that every time go on the web two it would have taken a long time and we've got to wait 10 minutes and three the quality would be bad because you're actually only looking at a very small subset of everything that exists out there right so how do they do it they have an index you never leave google servers you search for it hits the index the reverse index immediately gets you the 10 blue links within you know less than 100 milliseconds we need to do the same thing for the AI so the ontology is that we need to compute that index offline all the time so it's almost like the page rank algorithm that Google had invented back in the day but it's more complicated because Google was just looking at a web where everybody can go on the web here there are permissions and the links existed here there's permissions involved the data I'm allowed to access might not be the data that you're allowed to access so there's privacy, there's access control also there's many different types of objects here that we're dealing with not just websites so the problem is a little bit harder but it's manageable you can actually do it.

49:52So, you know, I'm convinced you can do this and you can get massive productivity gains out of it because we did it for Databricks. Yeah, exactly. We did it for ourselves. Same more, yeah, yeah. Yeah, and we're like, the company has just completely changed. It's like not the way it was, I would say, a year ago because of this. I mean, you've been dogfooding Databricks for Databricks forever, but maybe say more about the impact you've seen as an organization. Yeah, I mean, once we got this ontology and we started working on it and we actually have probably the largest of all of our customers, we have the largest ontology.

50:18our ontology is bigger on us than any of our customers when they use us to build their ontology because Databricks uses Databricks more than anyone else uses Databricks. And so it's like millions and millions of nodes in the graph, in the ontology graph that we have. So it's just, you know, what happens in an organization? What happens in an organization, you have a tree structure organization and information flows up and down the tree structure. You know, if you can't make a decision, you escalate your boss, maybe they can tiebreak, it escalates up. They need to get up to speed on what's happening, and they need to get all the context, and then they make decisions.

50:52Once decisions get made, you have to percolate them down in the organization. A lot of this can now be done by AI if you have an ontology. Why? Because, you know, what happens in a meeting? In a meeting, you go through some, you know, someone has done the analysis. They probably have a PowerPoint deck with some pretty graphs in it. That person who did the analysis is some smart person that used Excel, made some models. There are some numericals. So a lot of that you can now just do with AI. So the AI can do the analysis for you. It has all the context. It can present it in a way that you want.

51:26You can ask questions about it. Instead of having follow-up meetings, you can directly ask questions directly from the AI. So it's very similar. It's along the lines of what Jack Dorsey has said that you can do to the organization. It's just a concrete way of implementing it. So it's game changer for us. Like, you know, it's just everybody's on their phones now in the meetings on Genie. And they're like asking Genie questions. You can see, as soon as someone says something complicated or something, it's like, you know, you see everybody go through the phone. Can you share that finance, like the finance anecdote?

51:53You mentioned once in a board meeting. Yeah, it's, yeah, sure. Internal board meeting. Yeah, so. Only in kosher. Yeah, exactly. No, it's actually needed for one of our presentations. I need to know how many customers do we have in Fortune 500? What's our penetration of Fortune 500? And I asked one of the people in sales ops because I thought she would have it. and she texted me back and said, oh, sorry, I can't log in to Genie right now. I'm on a flight. That's why if you're just going to log in to Genie, I can do that myself. I asked you because I thought you had something alternative that I don't have access to.

52:26So then I was kind of a little bit angry. So I texted the CFO instead, Dave. And so I texted Dave and I said, hey, do you know what our Fortune 500 penetration is on? And he just copy pasted a screenshot of Genie back to me. So he also asked that. So I said, does anyone do anything novel here? than everybody just going to Genie and asking the ontology, you know, for questions. It's like, let me Genie that for you and let me go for you. That's what everybody's doing. Now we just say, we say, hey, can someone just Genie this? Like, can I just get it from the ontology? So I do think it's a game changer, but it's not just you press a button and you have an ontology in an organization.

53:00And I think Palantir actually has done a great job of going to organizations and getting a lot of that tacit knowledge written down and getting it into the organizations. We automatically take that and build the graph and then we feed that graph into the agents so that we can answer the question and answer it in a way that business leaders would like to see it, which is in graphs, you know, analytical way and a way where you can interrogate that question and, you know, continue asking questions and getting answers to those so you can make decisions and then disseminating that information in the organization.

53:32Yeah, it's pretty amazing. You sort of bookmarked the developers are obviously using AI questionable value I want to follow up with you on that because I feel like you guys were one of the earliest. And I say, I don't want to use the word token maxing because it has such a negative connotation. But I think in terms of applauding people who can use AI to become more productive, you guys were, you know, at the forefront of that, right? And then, of course, there's this cycle of, oh, shoot, people are being wasteful. Now we need to value max. Like, what was your own journey on that? And like, how do you guys think about value maxing, not token maxing?

54:10And then I'm going to throw in Unity Gateway in this, right? Because I think the managing of costs piece is actually getting more important. And you guys are helping people do that. But maybe tie that in to the extent it's worth it. Yeah, so around Q4 last year was when, you know, the models got really, really good. And we started noticing that, okay, it's actually starting to give much better productivity. So I actually started using the models myself to sort of start, you know, commit code into production for Databricks. I want to take it all the way to production. So I did that and started pushing the organization that, hey, everyone needs to do that.

54:44I have done it. Why are you not? Like if the CEO can commit code to production on a very sensitive data platform that has all these security requirements, you should be able to do that too. You being any manager, anyone in the organization. So I started pushing everyone very hard and we started making leaderboards in Q4. And at the beginning of, I say, January, February, when kicked off the year, we were already full swing. Everybody was using the stuff and we were pushing and we're managing this. But, you know, the whole token maxim thing was happening around, you know, February, March period, already it was happening.

55:14So yeah, we just had the luck of being maybe a few quarters ahead of folks to see what was happening here. And it was getting out of hand. So we already had a gateway. So it's called Unity Gateway where we were already, this gateway was being used to provide token capacity. So you can get OpenAI, Anthropic, Gemini, Grok capacity. Like any customer can come to us and we'll just provide them that capacity because we have a relationship with those. And any open source model. So we started putting in budget constraints in place and giving people warnings. Like, okay, you have this much of your budget left.

55:46You're getting close to your kind of ceiling. So we started doing that per person and for group. And then we started doing great analytics so we could predict exactly where the costs were going. And then we added smart routers that could actually pick cheaper models if you're getting close to your budget or if you have simple questions. We started doing that. We also built a harness called Omnigent, which can multiplex between the different harnesses. Turns out actually the harness itself matters. Like if you use the same model, but different harnesses, there's almost 2x different cost difference.

56:18Even exactly same model, you know, same version, but different harness, you get 2x difference in actual cost. So if you can change harness, you can get a lot of leverage in the cost. So we started using all of this. That way we were able to actually bend the curve and actually a cost for AI has been basically, the tokens have continued to go up, but the costs have been sort of stagnant. So that's been actually super, super important for us. And there's a huge demand for this. I think every organization is going through this now. Yeah, for the first time, and I do a lot of board meetings, so I'm on 20-something boards.

56:52For the first time ever, a company at scale last week said that they're moving from the Frontier Models to GLM. This is a large engineering organization. do you see this do you think that that's a trend or do you think that's just like a one-off anecdote? Because I've been hearing about, I remember the first DeepSeek moment and like NVIDIA software and then that turned out to not be real and then the Kimmy moment and the next DeepSeek moment none of it seems to have actually had an appreciable impact on the market but now the amount of anecdotes that I have are pretty real and it seems to be happening love your view I mean I think people want both they want the latest model that's super intelligent for the difficult task where they get ROI.

57:31But then there's a lot of mundane dumb things. Like, you know, people literally use their harness to rename files and whatnot. Yeah. You know, like it's, you're paying, you know, orders of magnitude more for that. Please type that in yourself. Don't have the model do that. It's going to spin for five minutes and then it's going to rename the file for you and cost you, you know, cents. But I get some people, I'm just wondering, do you actually see market movement? No, people are moving on it, but what they're doing is that, you know, the pattern is either you can use this expert pattern where you have a small, cheap open source model that uses an expert model, the big ones, or vice versa, or a way in which they can ping pong them to each other.

58:12But also multiplexing harnesses and just changing harnesses so that you can control the costs is also what people are doing. People have found, for instance, Pi is very efficient as a harness. So yeah, I think there's going to be a multitude of these. It's easy to, the models themselves are stochastic, as you said, every time they give a different answer, and they're changing so much. So there's just a lot of experimentation happening. So I think we're going to get to a world where you're not always using the smartest model for everything, which has kind of been the paradigm for the last couple of years.

58:41Like a new model comes out, it's super smart, they use it for everything, even really, really simple mundane tasks. I'll tell you what I see. I see people using Fable and Astra for architecture, a cheap model for implementation, and then Fable or Astra for audit. Yeah. Like, that seems to be like this emerging. What are you guys seeing in the startups? I mean, aren't they? That's it. That's honestly the pattern. How much open source? By token or by dollar? Either. So by dollar, open source is like 5%. It's very little, but by token count, it's over 60%. Yeah, I was going to say, I mean, we talked to, let's say, a Decagon or something like that.

59:19Well, I think it's different internal use versus external for product. on the external for product, I think they're almost up to 90 % open source. On the internal, and I don't want to say for Dengon in particular, but a lot of them are like, we don't care, we'll just use Frontier. We're not thinking about cost control. But as it gets bigger, you and I were in another board meeting where they actually did bring that down just from a waste perspective. So I definitely see that moving more toward open source on the product side. And actually that's related to another question maybe around open source, but post-training specifically.

59:54I feel like you were kind of early. I remember talking to you in 2023. When did you buy Mosaic? 2023. 2023, okay. So this vision that you had in 23 kind of came true in 2026. I don't know if you guys would agree, right? Like that's sort of what we're hearing across, you know, of course, oh, just sort of like, hey, we're going to actually, you're going to own your own intelligence. You're going to be, you know, post-training your open source models, et cetera. and that's definitely what the startups are doing. I don't know if that's what the enterprises are doing yet, but like, I mean, do you feel like you were early to that?

1:00:28Yeah, I mean, first of all, you know, there was, when we started, it was also, hey, we're also pre-training for you, which that doesn't make any sense. You know, you can, there's so, they're very good pre-trained model now that you can use, but that you can do actually post-training on the model and you can do reinforcement learning. Yeah, we're actually doing it at scale. And many of those startups are actually customers, so we actually help them, you know, using early or reinforcement learning environments where we can make the models very, very good at the specific tasks that they are doing.

1:00:53It makes a lot of sense for them to do that. If you have a repetitive task, so if you have a startup and it's offering a product and a product does something specific, it's not just a general intelligence. It does something specific for you. It makes just a lot of sense to take a really good open source model and use reinforcement learning and make it really good at that specific task. You can cut the cost down. You can make it really fast. They control their own IP. So in that sense, that is possible. But large enterprises, they just need basic automation. And it's just too much for them to do this right now.

1:01:26I think one of the challenges is, you know, you need good evals. And making good evals is hard. So while the startups can do that and they're motivated to do that, other organizations, the easy button might be just to use a frontier model than having to create your own evals. We actually generated even, you know, evals for the customer automatically in the product. And we had it front and center. But then people didn't want to use it. So we said, okay, let's move it to the back end so that it's optional. And then they would never go to it. So I would say in general... Why? They just don't want to get into it?

1:01:58It's too complicated? I think you want quick reinforcement of like, hey, there's a new model. I want to try this out. I want to get this problem solved. You don't have time to go do this scientific method of let's make an eval. Let's have a great baseline. And it's sort of like TDD, test-driven development. You know, in software engineering, that people actually do test-driven development. Very few did, right? Everyone said it's the right way to do it, but nobody actually in practice did it. So that's the same. That's kind of a little bit of the curse of, you know, training your own model is the evals is the hard part.

1:02:31I know you have an FDE model at Databricks. That's a very popular word right now, or acronym. But does it, like, to get these at enterprises that large, is it a full FDE model that's required? Or, like, how do you, and how has that evolved, maybe? Yeah. Yeah, I mean, we've had these FDs and the demand for it's gone up significantly. A lot of it is, you know, how do we build that ontology? Like the ontology is automatic, but if you're not collecting any information, like you're not recording anything. Right. So that's one of the key things that we do. But also things like, you know, I want to build an agent.

1:03:06I want to put it on, you know, I want it to be customer facing and it's have really low latency and I want it to have guardrails to not, people coming to abuse it or ask it things that we don't want it to answer and so on. So we can build that, like, you know, like sports AI that Fox has. You can go chat with it about sport events. And you can try to ask it actually about politics and it's very good at rejecting you and moving and talking about sports instead. So the FD has built that. So, you know, we'll help the organizations actually get started with AI. It is important because it's just many organizations do not have the in-house expertise to build this stuff.

1:03:39So they need just a little bit of help on the side and then they get started. Yeah, makes sense. So this is more related on the agent side, but I saw recently that I think a third party, neutral third party, I think, did some tests that Lakebase or Neon was actually the Postgres database of choice for agents. And I thought that was interesting, one, because, you know, one, exciting data works. But two, I probably wouldn't have guessed that maybe a year ago. Yeah, it was a surprise. is just because there's others out there that have, you know, great developer momentum as well. But it was pretty clearly number one.

1:04:17And so I'm curious, how did you guys crack this? And what makes you win across the agents? Because if you win the agents now, you win the market. Yeah, I mean, I think a lot of credits should go to Neon and Nikita and team. And I think what they've done is they've just been obsessive about how do you make the models? How do you make the models pick and agents favor Lakebase or Neon as a database. So what do they do? The agents want to experiment. They're going off. They're trying to build a little bit of software. They need the database. So you need the database to come up quickly. So they had this obsession that everything should take far less than a second.

1:04:55So if a database comes up in far less than a second, you can clone gigantic databases, a kind of petabyte database, you can clone it in less than a second. So it's highly elastic, highly responsive. And then they built this killer feature called branching. So branching just lets you branch the database and you can have many, many branches over the same database. And they just made this very, very lightweight. We saw this with other things with agents, right? Like UV, you know, RipGrap, like basically re-implementation of a lot of the tools on Unix, making them really, really blazing fast and lightweight and also sort of fail-safe for agents.

1:05:33They just did this to a harder problem, which is database. So like now you have a Postgres database And the Postgres database has all these advantages, that it's really fast, it's nimble, it's fail-safe. You can go back to snapshots, you can do those things. So I think that's why. It's just easier for the agents to use this. They also made sure that it had a pricing model that was, like you don't want, just because the agents are building some software and experiment, you don't want the cost to run up. You're okay paying for your database if it's like production used and lots of people are using it, but just to experiment.

1:06:02So I think they were just obsessed. They were not trying to win the database war or trying to be better than some other vendor. They were obsessed with how are we the best for the agents. And that's a new persona because in databases, the obsession has been how do we help DBAs? How do we help app devs? How do we help the people that are using the database? They changed the game and said, hey, how do we focus on agents and help agents get the best database they want? And, you know, now over 90 % of their, the databases that are created on Neon and Lakebase are actually created by agents. So it's not even humans.

1:06:36So, you know, numbers speak for themselves. By the way, it's remarkable. So I've started using Neon as like my standard database. And it was bizarre to me because like normally when you enter a large company, things slow down. It's actually like the products got materially better. Yeah. Are they totally independent? Do they work with the rest of the, like how? No, it's a great team. I mean, we work very closely together. You know, we love databases and data. So it's, you know, we live that. But the team does a great job of just making it super fast, snappy, and great for agents. All right, Ali, what is your PDoom?

1:07:10Less than 10%.

1:07:14Close to zero. What about yours? I don't know. I say my only answer is my PDoom without AI is much higher than my PDoom with AI. Wow, that's a tough one. What about you, sir? I would agree with Ali on this one. Yeah. Okay.

1:07:36A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening and I'll see you in the next episode. As a reminder, the content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.

1:08:13Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com forward slash disclosures.

1:08:32You