AI Enterprise - Databricks & Glean | BG2 Guest Interview

0:00I think we have AGI. I think we have artificial general intelligence. We really have it. You hear these 95 % of projects fail, but that's actually what you want. I think the LLM is a commodity. People are not saying that, but it is a commodity. Like, you can get gas from this gas station, you can get gas from that gas station, it doesn't matter. Just compare price. Is AI in a bubble? There is an AI bubble. Okay, so then Glean is also in the bubble, everybody's in the bubble. No, I would say there is a bubble. I would say those three camps. There is a super intelligence quest camp. I would be very worried there.

0:32There's a second, the researchers doing the, you know, that's definitely not in a bubble. They're like... They're sober. Yeah, they're super sober and nobody cares about them. And they're probably the ones that are right, unfortunately. And then there's the third camp, which is us trying to make this valuable. We're not in a bubble in a sense that we're not spending huge amounts of capital on what we are doing. We're just trying to get actual economic value inside of these organizations.

0:59Two legendary builders, Ali Ervind. I'm so thrilled to get into this with you because both of you have seen every super cycle I've lived through, internet, mobile, cloud, data and AI, not just through the super cycles, but also through the hype, the trough of disillusionment, and this time it's different. Today, we're going to chop it up on the state of AI. You know, let's start with a 20 ,000 feet view. Take stock of where we are. AI, we've seen consumer AI, billions of users. ChatGPT said the guns went off three years ago. Cloud, perplexity, chatGPT, people use it in the room. On the SMB and developer side, you've got hundreds of millions of users with cursor and codex and cloud code and so on.

1:48Enterprise, on the other hand, there's a lot of divide. It's hard to see a lot of fog of war. On one side, you've got models that are earning math benchmarks and science benchmarks and engineering benchmarks. But on the other side, you've got the MIT report that's saying 95 % of AI deployments don't work. What's the reality? Bridge that gap for us. Lay it out as you see it, view from the top. So I think, first of all, I think we should know that people use AI in their personal and work lives both. So there's not so much of a divide. Everybody in your company is probably using ChatGPD and Cloud and other tools on a daily basis.

2:30The thing that I feel is happening in enterprise is you hear these 95 % of projects fail, but that's actually what you want. When you are actually experimenting with new technology, if all of your projects are failing, that means you're just not trying enough at the moment. So I think when I read the study, it was not a surprise for me. We can actually see, hopefully, similar stats next year, too, because we want everybody in the industry to be really eager and experiment and actually figure out how to actually get benefits from this technology. This would make you guys, by default, be 5 % of AI that is working, which is 1 in 20.

3:15Maybe you go to the 5%. What is the use case that is working? And not just working like it's saving me time, but like it's working and it's transforming my company. Something that you can take to the bank, to the CFO, while the CFO will notice it, but the legal won't shut it down. I mean, look, we're seeing a lot of use cases that are working. It's just that you just have to, it's not just you can just unleash the agents and it just works. It's an engineering art. Like if you're going to have a company that's going to be really differentiated, like my company or your company or anyone's company, and you want to beat the competition, you can't just quickly put something together and think that your competition is not going to do the same thing.

3:57So that's going to be something that needs evaluations. It needs something that you're going to productionize. It's going to take effort. You need a great team around it. But we're seeing a lot of them. I'll give you some examples. Royal Bank of Canada built agents with us that basically take as soon as an earnings report comes out. So equity research analysts, their job is to put together these reports that say, like, you know, this is a buy, this is, you know, a hold, and so on. The agent goes, gets the earnings report, gets all the previous earnings reports, gets all the competitors' earnings reports, gets everything that's going on in the market, does the full analysis, the news, everything, puts it all together, and it can get the equity report out in 15 minutes from the earnings call.

4:37Industry standard is two hours. Of course, it's going to get commoditized, and others are going to do that as well. But that's an actual really important use case that we're seeing in finance. So that's like finance, example in finance, right? And there's lots of examples like this, sifting through hundreds of thousands of documents, SEC reports, so on. That's finance. Let's switch gears. Let's go to healthcare. Healthcare is completely different. In healthcare, we have a customer, Merck, that in the life science space created a model called TEDDY. TEDDY stands for Transformer Enabled Drug Discovery.

5:13And this is a transformer model, kind of just like large language models that can predict the next word, but it instead can figure out which genome is missing if you remove a genome. So it really understands the gene regulatory network and can really start telling you what's happening with gene expression and so on. So this is really important for drug discovery. It's the beginnings, but this is going to actually help us do things that we couldn't do before. Let's pick retail also. So I'm picking different. Healthcare is one. I gave you finance, right? The RBC one. Let's go to retail, 7-Eleven.

5:45agents that completely automate the marketing stack. Actually, the marketing stack is going to get disrupted pretty heavily. So these agents can basically prepare. They can segment the audience. Like, this segment wants to hear this. And they can prepare all the marketing material that's directly targeting you guys. And they can put the campaigns together and do that. 7.11 was doing this before as well. But, you know, this, and we're seeing this at Databricks as well, more and more is being done by agents and being automated. so you can just do it faster, and you can segment more fine-grained.

6:16Because before you had to create the content for the groups, that was a heavy content creation was something that was human manual labor. Now you can actually do that much, much more. You can have all your web materials completely customized for a target group. So these are examples where it is working. There are also lots of examples where it's not working, even with Databricks. We're not just the 5%. We have some of that 95 % too, but some examples where we're seeing success. Ali, follow up on that. These are great examples. Thank you. Maybe if you were to take it a layer up, what is common across these use cases or these organizations or these CIOs that's making these use cases work?

6:52Is there something that we can pattern match? Yeah, look, I think the LLM is a commodity. People are not saying that, but it is a commodity. Like, and, you know, when I took econ classes, commodity was when it's interchangeable. Like, you can get gas from this gas station or you can get gas from that gas station. It doesn't matter. Just compare price. LLMs have become that way. Like, it doesn't really matter. This one is better right now. Next week, that one is better. you can't even keep up anymore, right? What's happening? So they're a commodity. So it's not about that. It really comes down to your company.

7:20What data does your company have that's special that your competitors don't have? Can you leverage that? And can you build AI that really understands that data? Because that's not a commodity. There's not an AI out there that understands all your business processes and your company, your secret sauce and your data. That's not a commodity. In fact, that's closer to the 95%. It really comes down to that. Or if you have a complicated process that just your company has, this is how you deliver your product and services in your company. And that portion can be disrupted with AI somehow. If you can do that, now you can get ahead of your competition.

7:55But it comes back to what makes your company special. Unfortunately, a lot of companies are just building commodity stuff. You should not be building that because it's a thing that every company can do. It's not special to your company. That's, I think, the problem in a lot of the industry. Another problem in the industry is a lot of demo wear. It's really easy to make cool demos with Gen.AI. And, you know, therefore we're seeing a lot of cool demos, but that's all they are. Yeah. Well, you know, something we say around at Altimeter quite a bit is your AI strategy starts as your data strategy.

8:24Yeah. So you've got to get the data house in order first. And, you know, there's a lot of reasons for use cases that are, you know, we are trying, we're not working. Maybe give us an example of the 95 % of an AI bet that either of you had at Databricks, at Glean, that did not work out and why it didn't work out? It's actually an interesting thing with engineering today is you build systems and never before have you been in this mode where you start with a great idea and it doesn't seem like a good idea anymore, like within two weeks, because we see a new development that happens. So we have numerous failures in engineering on that front.

9:04For example, some of our fine-tuning work, building models, for a specific use case within our product, like, you know, didn't really pan out for us. And ultimately, the choice was that, you know, we can go with already built models, whether they are small open source models hosted on Databricks or one of the large, you know, foundation models. But internally, like, you know, from a corporate, you know, use cases perspective, actually, like, you know, we are also like in many ways in this mode where a lot of our work actually, like, I would not say like fail, but it actually takes much longer than to actually generate success.

9:41We're actually trying to automate a lot of our business processes internally. And for example, one thing that I want is, in our company, I want everybody to actually know exactly what their top priority for the week is, what they want to work on. And maybe we want an AI agent to actually first tell them what their priority should be, and we want it all to be documented, and we want a system which actually then rolls it all up and I get a view every week where I can actually quickly see what are all the different people working on in the company and is that aligned with what I want them to work on.

10:19And it's a simple thing. Companies have always tried to actually have this. As CEOs, you always want it and it's always hard to make it happen. And we thought that AI would simply just magically do all of this work because it has all the context. It has all the context inside the company to make it happen. but I still don't have it. So things do take time to actually, you know, to Ali's point, like, you know, AI is just one more tool that you have in the toolkit. It does not suddenly make building complex enterprise systems, you know, like it doesn't make it that you can build it up in one day.

10:56Yeah. You know, the last time enterprises got this excited about a tool was called RPA. And we know how that ended. It unfortunately fizzled out. And, you know, somebody in the audience yesterday is like, hey, how is this time different from RPA? It seems like the same movie, bigger budgets, better actors. What's different this time? How is the nature or the architecture of the technology different from the previous automation cycle? Either of you. Yeah, well, I mean, first of all, RPA, like, it didn't take, you know, it didn't capture my attention at all. So I actually can't, you know. What is RPA?

11:32So I think I would not compare these two technologies at all. Like, you know, what we're seeing now with AI is so fundamental. You know, it's the, you know, when we saw it first, it was basically magic. And we couldn't believe that this is a machine that is doing this work. Machines just simply cannot do these kind of things, you know, that we saw them do. like writing on their own, having emotion, understanding emotion. So it's a, you know, it's fundamental, it's different.

12:10And that's why, like, you know, I don't think, you know, this technology is going to fizzle out. And it's not like, you know, you don't have to be like a financial expert or, you know, like sort of a deep thinker on business. This is obvious stuff. Like, you know, all of us know, all of us feel it. All of us can see the capability of this technology, and we know it's special and it's going to be around. Yeah. You want to hear my RPA? Please. You know, I mean, it was rule-based, and the problem with it, especially if you want something that automates what's going on on your desktop and automate the work that's happening, it's just that there's too much unexpected things that happen, and it's just hard and brittle to set it up.

12:48It wasn't learning ever. So there was, like, zero learning. It was like, you tell it exactly here are the rules, and if it got something wrong, you need to go and go back and expand the rules. Here, you have something that's learning, right? So it can improve and it can generalize and it can understand the patterns and do pattern recognition. So that's the fundamental difference between these two. Now, there has been many startups that have failed in the generative AI. We're going to replace RPA with generative AI models. There's many startups that have failed, actually, that I know of, like pretty some high-profile ones.

13:20It's because the paradigm we live in today with AI is there's still problems. The biggest problem is that you bake a model, and that's where it's learned everything it needs to learn, and then you freeze it, and then you launch it, and then maybe you give it some context, but that's it. It's frozen. So therein lies the problem that, you know, we need an AI that really can sort of continue learning while it's using the desktop and clicking around. So I do think this problem is hard to nail, but I think Arvind is right that it's like there's no comparison at all. It's like rule, brittle rule-based stuff versus a learning agentic system.

13:55I think it's going to nail it perfectly. But we haven't really nailed computer use yet. Yeah. Working on it. The number one shift is this move from if-then-else statements to a more regenerative solution that figures out the solution. And so you're trading breadth for maybe determinism. That seems to be the difference. And, you know, there's a lot of CIOs in the room we've got here, and they've got budgets coming up to plan. If you were giving advice to them of like, hey, based on everything I know from my customer base, here's one thing or two things that you've got to figure out and align incentives on, or it could be a reliability problem or org design.

14:32What advice would you have for CIOs who are thinking about their AI budgets right now? Well, spend more. Put it on Glean. Spend more, yeah, put it on Glean. But I think the one thing which is important in AI market today is that it's very new and there are many players. In fact, every software company is also an AI company now. You can go and check their websites. So, I think it's just hard to actually figure out where to allocate those budgets. And what we tell people is that I think the winners are yet to be identified and so experiment with more vendors, do shorter term contracts and while that's easy to say, it's hard to actually implement because every product that you try has a cost that you have to pay to make it sort of even tested.

15:32So you have to also pick products that are easy to test. I mean, those are the ones that don't require you to spend the next six months trying to implement something and you have no idea what's going to come out after that. Like the products of today, the products that are built with the right AI, they should work very, very quickly for you. Kral, walk around. We're going to take a peek into the future. Shifting gears, one of the things that keeps investors like me up in the nights is a quarter trillion being spent on NVIDIA on the semi side of things. Assuming that is just 50 % of the CapEx, you're spending about half a trillion on CapEx and then you've got to earn about a trillion dollars of AI revenue for all of this CapEx to be worth it.

16:17And just to put this in context, the entirety of the software industry earns about$400 billion of revenue. This seems like a physics problem at this point. How do you think this plays out? You've got to make about a trillion dollars of revenue to justify this present spend that's already happening. How do you think this shakes out? Maybe we start with you, Arvind. Wrong person to start with. But, you know, I'm an engineer, and I actually don't really, you know, think too much about who's spending what money. Like, you know, we're here to build our product and add value. So in some sense, you know, I've not really thought too much about this problem.

16:55But if you think about AI, you know, AI is not actually, you know, extending software in a marginal way. It's a different product. And in fact, you know, it's actually going to grab a lot of revenue that actually today is in services industry, which is 25 times larger than software industry. So there's a lot of spend that is going to move. I mean, the spend that you see happen on AI is actually sort of those service dollars that are converting into AI or software dollars. And I think the... But with that said, you know, maybe you have a more informed view on this. By the way, I do think that's just to build on.

17:34You said, you know, I'm in engineering. I want to just build something that's cool. I do think it's not binary, right? It's not like, okay, so the physics doesn't work out, so the whole thing will collapse. No, there's going to be things that work. And so it is a good idea to continue focusing on the stuff that is obviously already working. Continue expanding on that. But I think if you zoom out, I think there's like three paradigms or three kind of camps. And I put Arvind in the third camp. I actually put myself also in the third camp. But let's start with the first camp. I think the first camp is this quest for super intelligence camp.

18:05and it's, you know, I think all the frontier labs are doing this, like all three, four, or five of them, however you want to count them. And I think it's really still being, a lot of it comes from the scaling laws mentality, which is whoever has the most GPUs and the most data is going to win the quest for super intelligence, which is kind of intelligence that's like almost like godlike. It leads to recursive self-improvement of the AI, which then once you have that, it can cure cancer and solve all economical problems. And we can probably 10x GDP over a few years, period of time. So what the hell are you talking about that there's a physics problem?

18:44Like anything, any of your cost equations are going to pale in comparison to the economic value that this thing is going to provide. So that's like one camp. And the way they're developing it is bigger and bigger clusters, more and more energy. And that's how they're going about it. And that's where most of the capital is going. Right. Right. That's not the kind of capital you're spending or I'm spending, but that's that camp. And then how do they know that they're succeeding? They're not just like, oh, just trust us. They're very smart people working on this. So the way they're approaching it, they're saying, we'll throw the hardest questions we have at whatever AI we have now.

19:16And if it nails them, and we're making really rapid progress, so what's your problem? Look at Math Olympiad. We're nailing these Math Olympiad problems and Physics Olympiad. And programming contests is better than any human being. So, like, that's what they're doing. They're throwing all the most intellectually challenging. There's a second camp, which are the people that created the original technology. The scientists who created the technology got the computer science Nobel Prize for it. It's called Turing Award. And that's, you know, Rich Sutton who created reinforcement learning, which, you know, a lot of this stuff is built on.

19:48You have Jan LeCun, who was one of the three founding fathers and many others. They have for many years. Actually, I've been, you know, I've asked them for years. They've been saying that that first camp is not going to, That's like not even the right approach is their view. They're like, no, that's just like autoregressive next token prediction. It's just probabilistically predicting the next token. That's not how, and usually they will say that's not how humans learn. That's not how animals learn. You know, we operate in a different way. Your brain is not that way. And one example is that, you know, even a child learns very quickly to walk and talk and do things with very little data compared to, you know, Like, certainly no child is reading all of the Internet's data four times over before they learn to speak.

20:29So that's like camp number two. Those guys, by the way, they say it's 20 years out. So they're saying, hey, it's a physics problem, and it's going to take 20 years to get there. Which to me, it's like, I don't know. Like, leave me alone. Let me do research. Third camp, which is, I think, what we are in, is I don't think we need superintelligence. Like, you know, I don't think we need that superintelligence right now. Maybe they'll get there. That's awesome if they do. But I think we have AGI. I think we have artificial general intelligence. We really have it. We absolutely have it. It's like anyone who says we need to get to AGI, that's like it's a false premise to start with.

21:02We already have AGI. I came to the United States in 2009 at UC Berkeley, not far away from here, and I was in an AI lab. It was called Amp Lab. The A was for algorithms and AI, machines and people. And these are all AI people. And back then, the definition of AGI we had, we already have satisfied that. Like, I know the discussions we had. And I actually went back to some of those folks to see, like, is it just me? Or what was the sentiment back in 2009? And everybody that I talked to said, yeah, that's by those standards, we had AGI. But we've changed the definition now. We have those definitions, you know, ads.

21:35So for 30, 40 years, we had a definition of AGI. We've already hit that. Now we're changing it and moving the goalposts. But very obviously, we already have AGI. Just use any of these LLMs and have it do some reasoning. and certainly it's smarter than a lot of friends that you have, right? Like, you know, let's not name them or co-workers or whatever, right? So you already have AGI. Now we're like haggling over exactly how smart is it. You know, do you have a friend that's smarter or not? So if we already have AGI, we just need to make it useful inside the enterprise. We need to just expand that 5 % to be 10%, 20%, 30%.

22:08So that's why I think Arvind's answer is actually a good answer. Like, we have the AGI we need. Let us just focus on solving the actual problems inside the organizations and I think we can already, that's enough to automate a lot of the tasks and get huge economic value out of it. We don't actually need super intelligence for that. That's a good idea. If the super intelligence guys nail it, amazing. Then we've cured cancer. If they don't, hopefully the second camp comes up with a new thing in the next 20 years. That's also awesome. We already have whatever we need. So yeah, let us just do our engineering.

22:39Right. Yeah. That's really good framing. And the way this manifests in the world is there's a data layer. There's the intelligence layer, which is where Camp 1 is presumably producing a lot of great models. And then there's the software layer where the users engage with. Where do you think value will accrue if you were to design 100 units of value across these three layers, the data layer, the intelligence layer, and the software or the application layer? Where do you think value accrues in the next five years? All right. This is a tough question. I mean, I think all those three layers actually are very fundamental.

23:19Yeah. I thought you were going to add a few more, which are not. You didn't. Yeah. Yeah, because I think I feel like the, like as Ali was saying, that the models are going to be available to all of us. They are going to be a commodity. It's going to be hard to sort of see that the more spend goes to them. versus these layers on top. But how do you, it's hard to sort of come up with where the most value will be. And I also don't know if actually it changes from today's technology architecture, where again, like when you think about in a pre-AI world, any sort of enterprise application and data systems, you have data systems, you do have, I guess you don't have enough of that intelligent layer today, and then you have the application layer.

24:11So I guess some dollars will shift into it. I know we do think that the intelligence layer is actually going to be a pretty thick one. Maybe it will capture half of the enterprise value. Anything to add, Ali? Yeah, no, I think that there are more layers in the stack depending on how you want to do it, but I think, as I said, the LLMs is a commodity, as Arvind said. You can get them, like, you know. By the way, that doesn't mean those companies are not going to be valuable. They can be very, I mean, TSMC is very valuable. But I'm saying they're going to be kind of like these fab-like companies.