When Will AI Hit the Enterprise? Ben Horowitz and Ali Ghodsi Discuss

0:00The AI revolution is here, but adoption is not always evenly distributed. While we see consumers jump to try the latest AI apps, We haven't seen anybody with any traction in the enterprise. But awareness is not the issue. In fact, the Financial Times reported that nearly 40 % of S &P companies mentioned AI in their earnings last quarter. So... Why is it so hard for enterprises to adopt generative AI? As companies wake up to the value of their proprietary data sets, a whole new set of questions emerge. Are the enterprises right about not wanting to give their data, like is that correct? Fear, can they build a better model?

0:49Do they really need it to be accurate? Plus, with OpenAI recently dropping, chat GBT Enterprise, will all of this change? Today, you'll hear directly from A16Z co -founder Ben Horowitz and Databricks co -founder and CEO Ali Gazi, as they answer these questions and more, including their perspectives on open source, whether benchmarks are BS, and the scramble of universities to take part in the very wave that they kicked off decades ago. Plus, you'll get to hear firsthand how Databricks recent acquisition of Mosaic ML fits into all of this. This episode continues our coverage from A6CZ's exclusive AI Revolution event from just a few weeks ago, where we house some of the most influential builders across the ecosystem, including founders of OpenAI, Anthropic, Character AI, Roblox, and much more.

1:46Be sure to check out the full package, including all the talks, also in full at a60z .com slash AI revolution.

1:57As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please Please note that A16Z and its affiliates may also maintain investments in the company's discussed in this podcast. For more details including a link to our investments, please see A16Z .com slash Disclosures.

2:27All right, so going to Generative AI, one of the things that's been interesting for us as a VC is we see all kinds of companies, some with amazing traction, but every company that has traction is in a category like selling to developers or selling to consumers or maybe selling to like small kinds of law firms or these kinds of things but we haven't seen anybody with any traction in the enterprise. Why is it so hard for enterprises to adopt generative AI? Yeah, so look, enterprises move slow. Yeah. This one in general, right? Which is the beauty, which means if you crack the code and you get in, It's hard for them to throw you out.

3:10So you're going to have awesome business. If you do crack the code and you're in, it's more robust. You're not going to lose it overnight. So that's one. They just move slower. Second, they're super freaked out about their data, privacy, security of their data. But then in general, also, I just realized, everybody's been talking about data for 10, 15, 20 years. I just realized how valuable my data actually is. So maybe I'm actually, I'm sitting on a treasure trove and I'm going to be super successful. So I'm going to be very careful with this. Now, I finally realize how valuable this data that I have is.

3:39So I definitely don't want to give it to you or you or you. And you should be careful about this. And then there's all these reports about it leaking, data leakage, like oh, suddenly the LLM is spitting out your code. So they're freaked out about that as well. All of these things are slowing it down and they're kind of thinking through it. That's just one set of challenges that enterprises have. Second challenges enterprises have is that, hey, for a lot of the use cases, we need the data to be accurate. We need to be exact. So there's a lot of use cases where. Are they right about that? Do they really need it to be accurate?

4:07I think it depends on the use case. They're just being cautious and they're being slow as they are in the big enterprise. And then there's the last aspect which people don't talk about, which is there's like a food fight internally at the large enterprise, which is - Who's fighting? I own generative AI. Not Ben. And you go around and say, I own generative AI. And it's like, no, no, no, my team is going there. So there's this food fight internally of who owns it. And then they slow each other down. So it's like, hey, don't just Ben, because he's not handling data the right way, but I'm building my GNI, and it's unclear.

4:37Is it IT that owns GNI, is it the product line, is it the business line, so there's like huge politics going on inside the large enterprise. They wanna do it, but there's all these hurdles in the way, and the price is huge. Whoever can crack the code on that is gonna create an amazing company. Are the enterprises right about not wanting to give their data to open AI or anthropic or barter, however, like is that a correct fear? or are they being silly and they could get so much value by putting their data in a big model? They can, but I think also a lot of the leaders, but I get to talk these days to CEOs of these big companies who previously were not interested in what I'm doing.

5:14I would be talking to CIO, but now suddenly they wanna talk. And like, hey, I want this standard of AI. I wanna talk straight out of my company. Let's talk. And we have this data set. It's super valuable. Like, you know, we gotta do something with it. And it's generative AI seems interesting. What do you wanna do with it? And one of the things that's really interesting that's happened in the sort of brains of the CEOs and the boards is that they realize maybe I can beat my competition. Maybe this is the kryptonite that will help me kill my enemy. I have the data with generative AI, I can actually go ahead and do that.

5:45So then they're thinking, well, but then I have to build it myself. Yeah, I have to own that, right? I have to own the IP of that. I can't just give away that IP to interoptic open AI, anyone. Like it has to be completely prepared. I want to own that. I want to do that myself. By the way, I have a whole bunch people here that are lined out outside of my office in different departments that are saying they actually will do it and they can do it. So we're trying to figure out which of them I should give it to. So this is what's happening internally right now. Interesting. And from a strategy standpoint, when you think about it, let's say you had a big data set, be it like a healthcare data set or some kind of security data set or a meal since data set, can they build a better model themselves for that with their data or if they took their data and put it in one of the large models, but that always beat what they're doing.

6:30Yeah, so this is why we did the acquisition of Mosaic. Yes. You can. It's hard. It requires a lot of GPUs. The Mosaic guys just figured out how to do that at scale for others. You want to build your own alarm from scratch, come to me, I know all the sort of landmines and so on. It just will work, trust me. So they can do it and yeah, they've done it for large customers. They can do it. Still, it's not for the faint of heart. still requires a lot of GPUs, cost a lot of money, and it depends on your data sets and your use cases, but they're having a lot of success doing it for really large enterprises.

7:03They'll train it from scratch for them, and it just works. And the result that they get with Mosaic, so I'm doing it. So the good news is it's all mine. Nobody can touch it. It's your IDATA, screw off, competitor. But is the bigger model such a bigger brain anyway that I could get a better answer if I put that same data in the big model? or is kind of mosaic tuned enterprise specific data set specific model getting a perform better like how do you think about that for specific use cases you don't need the big one. First of all you can build the big one with mosaic and with data works. There's so much money you have.

7:41We're happy to train you under a billion parameter model if you want but hey it's going to cost more to use it even if you have the all the money to train it. Yeah. It costs you a lot to use it. So when you're using it then you're doing inference as it's called it's going to cost you more. And how do you think about the diminishing returns on a data set against how many parameters versus how much stated you have? Does like a bigger model just start to be diminishing returns post in terms of latency, expense, everything? Yeah, I mean, there's a scaling law. If you're scaling the parameters up, you kind of have to scale the data with it.

8:15Right. You know, so you just have to do that. So if you don't have that, then just scaling it, you're not going to get the bang for the buck. You're still going to get improvement if you increase the parameters or if you increase the data in any one of these dimensions. But you're going to pay. You're going to pay. It's going to come in a different way. Yeah, it's no longer a Pareto optimal, so to say. But what I'm saying is this, four enterprises that have specific use cases, which they all have. When they come to us, they don't say, hey, I would love to have an LLM that could kind of answer anything under the sun.

8:41They're saying, hey, this is what I want to do. I want to classify this particular defect in the manufacturing process from these pictures really well. And there the accuracy matters. Like every ounce of accuracy that you can give me matters. And there you're better off if you have a good data set to train you can train a smaller model. The latency will be faster to use it later. And it will cheaper to use it later. And yes, you can have absolutely accuracy that beats the really large model. But that very model that you built can't also entertain you on the weekend. Right. And ask physics question and help your kids do their homework.

9:14Why do you think it's important for you, Databricks, to build the very large model? So, the bigger models, if you follow the scaling laws, are more intelligent. Assuming if you're okay with paying the price, and you're okay with, you know, you have the GPUs, and if you can crack the code on how to fine tune the bigger model, which is kind of the holy grail right now that everybody's looking at in the research community, and in the field, and the companies, and all that. And when you say fine tune, kind of get more specific. Yeah, so take an existing really awesome foundation model that exists and just modify it a little bit to be able to become really good at some other task and there are many different techniques to use to do that But right now nobody has really cracked the code on How you can do that without modifying the whole model itself, right?

10:00Which is pretty costly especially when you want to serve it when you want to use it later, right? But if you have to go through all that. Yeah, if you have thousands if you made the thousand versions of it that's good at thousand different things. If you have to load all of each of those thousand into the GPUs and serve them becomes very expensive. The big, I would say, only grail right now that everybody's looking for are there techniques where you can just do small modifications where you can get really good results. And you can just stack on a little bit of additional, you know, just set part of the brain.

10:30Exactly, just add this thing. And there are lots of techniques. There's like pre -fix tuning, there's Laura, Q -Lora, so on and so forth. Jerry's out, none of them really are slam dunk. It's awesome, we found it, but someone will. Once you have that, then it seems in the future, in a few years, the ideal would be, really big foundation model, that's pretty smart. And then you can sort of stack on these kind of additional tuned sort of brains that are really good at this specific classification task for manufacturing errors and this other translation task. And they'll be computer efficient and energy efficient for just stealing with that task at that point.

11:05Exactly. And then you can load up your GPUs with that one intelligent brain, that one giant model, and then you could specialize it. But to be clear, no one's really done this yet. That's what I think a lot of people are hoping to do. And it might not be easy to do that. And meanwhile, we're having lots of customers who want to have specialized models that are cheaper, smaller, and that have really high accuracy and performance on that task. Yes. It's like, I can just say it, like at Databricks, So we bought Mosaic. I did not unleash ourselves for some go to market of 3 ,000 people to sell the thing that we bought because we just can't satisfy the demand.

11:39Like there's not enough GPUs. So you haven't even let all of you guys sell it? No, I'm not even letting all the customers buy this thing because we don't have the GPUs and we don't have it. If we unleash every company wants to do this, everyone wants to, okay, okay, I have 1 ,000 things I want to build, can you help me do that? In this context, sort of how much do you think these use cases will fragment? So you talked about, okay, I wanted to be good at doing my kids homework. I wanted to be my girlfriend. So how much do you think the use cases, the very specific use cases will fragment? And kind of within that, like one of the things that we're finding is getting the model to do what you want, is kind of where the data advantage is from the users in that.

12:20If I want it to draw me a certain kind of picture, that's a lot of conversations to do that. And so whoever is drawing those kinds of pictures will be good at that. But then there may be another model that wants to draw memes. But that thing that's drawing the pretty pictures can't draw the memes, because that involves words and all this other stuff that it hasn't. It just hasn't learned to get that out of the humans and map it into its model. So how much do you think we're going to get tons of specialization versus... No, no, no, once the brain gets big enough and we do these fine tunings, that's going to be it, it'll be like AWS, GCP, you know, Azure.

13:00I think the answer is closer to the latter, there's going to have lots of specialization. But having said that, it's not a dichotomy in the sense that maybe they're all using like some base models that are underneath common to many of them. Right. You're not starting from scratch every time. But you're tuning it up, but certainly. Yeah, look, I think in some sense, like right now, there's people are looking at the wrong thing. Right now, it's a little bit like 2000. and the internet is about to take over everything and everybody's super excited. And there's one company called Cisco, they build these routers.

13:28Obviously that's like the biggest thing. And the most important thing is whoever can build the best routers is gonna dominate all of the internet forever. Yeah, right? It's like that's the thing. The future of mankind is gonna be determined by who builds the best routers. And right now this company Cisco is the best one by far. It's obvious when I'm saying, Cisco in 2000 I think was worth half a trillion dollars at its peak. And people were talking about it's gonna be a trillion dollar company. It's worth more than Microsoft. So I think it's a little bit like right now like that. Who has the largest LLM?

13:56Obviously whoever can build the largest one that can turn it the most obviously will own all of AI and all of future of humanity. But just like the internet, someone will show up later and think about Uber writes and cab driving and someone else showed up and thought about, hey, I want to check out my friends on the Facebook and so on. And those end up being huge businesses. So there's these applications which many of them are obvious. like Mark talked about it and his AI will save the world, the lawyer, the teacher. They're like, there's lots of use skills. Everybody knows, probably there's going to be a lot of value in those.

14:27And no, it's not just going to be one model that open AI or Databricks or anthropocrystallium builds. And that model will dominate all these use cases. No, a lot of things will need to go into building the doctor that you trust that will be able to tell you how to cure you and your loved ones. So I think those are the companies that we will build in the future. I think there's going to be a lot of value in those, obviously. And yeah, there's a place for the Cisco router still, for LLM and so on. Cisco still is a pretty valuable company. It's not dead, but I think that's this over -focus right now.

14:59Yeah, interesting. So then how do you think about open -source? Because a lot of the large model providers are literally going in and saying, stop open -source now, you've got to outlaw it. So how do you think about that? But why are they saying that? Do they have a legitimate gripe? And then coming from data bricks perspective, how are you all thinking about open source, with respect to Mosaic and then with the other things like Lama? If the original Lama was never released, what would the state of the world and our view of AI be right? Now, we would be way further behind, right? And A, it was a big model by what existed in open source.

15:38And it was open source. And both of those things completely changed. everything that's happening in AI right now. Size kind of mattered and the fact that it was open source also kind of mattered. It doesn't stop there. It's going to continue. It's also really hard to block any of this because if you just check out the source code for Lama, it's like a couple pages. Yeah, but you have to have the weights too. Yeah, but the weights leaked and people will leaked the weights and they will get out and people will keep putting them and there's ways to also distillation techniques where you can take the weights from a, you can just take an output of a model and train smaller ones and train other ones and so on.

16:13So people are going to continue pushing the boundary of this. So I think open source will continue to do better, and better, and better. And I think more and more techniques, because there's scarcity that they don't have GPUs, they'll come up with techniques in which they can do things more efficiently, like the fast transformer and so on. At the same time, I also think that anyone that trains a really gigantic model that's really, really good, typically will not have the incentive to release it. So it's the usual thing we see, that open source kind of lags, the proprietary ones and the proprietary thing is way ahead and it's way better.

16:43And in some rare cases like Linux and so on, it bypasses. You know, and in that case that would be game changing. And what about that happen? It's hard to predict that. Right now it just seems that you need a lot of GPUs to do this. But how about when GPUs become abundant? Yeah, GPUs become abundant or certain tweaks to the transformer that lets you train that high learning rate and you know, have less issues with it so that you know, right, because they're super inefficient now. Like they can be more inefficient. Yes, and so then there will be a reason. They will be released and the universities are just tromping at the bit, right?

17:19Because what has happened right now is that the universities kind of feel a little bit that. They're aceed out. They're not even in the game anymore. What? Look, this was my game. I was playing it. I was a metronome. And now they threw me out. Yeah. And I can't even participate. Because I don't have GPUs. I don't have the funding. The universities are having a huge sort of crisis internally with the research. It's like, let's see, you hired all my guys. Yeah, I was like, so no, but their guys are leaving and their gals are leaving because they want to work close where they can train the models and do this kind of stuff and what the data is.

17:45And at the university, there's none of this. So then what are the universities doing there? Of course, looking at, okay, how could we crack the code on this? How could we make it much easier, cheaper, and how can we release it? So there's going to be innovation there. So I think this sort of race will continue between open source and proprietary and eventually open source kind of catches up. So, you know, I think it's going to be the finishing returns. I think we're gonna hit walls with scaling walls and you just move down those. You go to the right on the x -axis and you move the pair to the right and eventually you get a GI.

18:15And it's just happening, it's guaranteed. It's gonna happen. I think we're gonna hit the mission returns on walls that kind of... Do you think we'll get stuck before we get to a GI in a fundamental... We'll need an actual breakthrough as it goes to just more size. That, and I also think that almost in all the use cases, we're seriously trying to use this like for medicine or for like anything where you're really for lawyers and so on it quickly becomes clear that you need a human in the loop. You need to augment it with the human in the loop. There's no way you can just let this thing loose right now.

18:44It's stupid. It does mistakes and so on and maybe that can get better and better and better but it does better on the medical exams and like DACA's to. This is a funny thing. I kind of think all the benchmarks are bullshit and so all these cello and benchmarks. Here's how it works. Imagine in all our universities we said We're going to give you the exam the night before. Okay, you can look at the answers. Then the next day, we're going to bring you in and you answer them. Then we'll score how you did. Suddenly, everybody would be asing their exams too. For instance, MMLU is what a lot of people benchmark these models on.

19:18MMLU is just a multi -choice question that's on the web. Ask a question is the answer ABCD and then it says what the right answer is. It's on the web. You can deliberately train on it and create an LLM that crushes it on that. Or you can inadvertently, by mistake, in the pile or whatever you used to train your model, happen to see some of those questions that happen to be elsewhere. So the benchmarks are a little bit... Yes. Well, they're benchmarks for taking the test, but presumably the test correlates with being able to make a medical diagnosis a decision. Yeah, but they memorized all these, you know, they memorized it.

19:54Yes, yes. So, I'll have there's no transfer learning from the memorizing the exam to actually diagnose the system. No one really knows the answer is everybody's playing the benchmarking game this way right now. Yeah, I would love it if, you know, a whole bunch of researchers that do old fake database benchmarks, but it's like, look how fast they're databases, but it's only good at the actual benchmark. Yeah, I would love it if there was like a bunch of doctors that get together and come up with a benchmark that's super secretive and they don't show it to you and you give their model, you were modeled to them and they'll run their questions on that and then they'll come back and tell you, I scored, but that's not how it works right now.

20:23So then let me go to the question that you dodged, which is, okay, what are the ethics of the large models versus open source or just in general? What is the responsibility? How big is the threat? Is open source an ethical threat? Yeah, look, I don't have all the answers. There's like different categories. There's like the jobs are gonna go away kind of category. We've been doing that for 300 years. And the nations that are at the doing the best high -speed GDP, they're the ones that automated the most. And the ones that weren't able to. And they have the most jobs in the house. So that's happening anyway.

20:54There are ways to deal with that problem. And the way to deal with it is not to just stop all progress. That's stupid. The nations that win are the ones that are doing well on automation, not just AI, in general, efficiency, improvements. There's like, economics is about efficiency. So anyway, so that's that category. Then there's like, how bad things that humans can do deliberately because they're malicious, which is the one I think Mark was the most worried about. But I would just say, look, ever since the invention of the hammer, we started misusing technology that, you know, in a bad way. So that's going to be, when you have a hammer, your head looks like a nail.

21:30Exactly. Right? So that's happening all the time with every technological improvement, especially the internet. So there's a really big question that I think kind of like, mark a little bit maybe dodged in his essay, which is, is are we going to get this super -age AI that decides to destroy us? And I don't know. The side part is the part where I got a little lost, because free will is not something we're on the path for, for machines. Like a machine doing many, many, many computations, which we never have machines to do this many computations in the history of humanity. That is amazing, but it's very different than like no LLM has ever decided to do anything.

22:16Yeah, like that's not what they do. And so it does seem like, okay now they've got free will. But maybe they don't have free will. Yeah, you know, I've just been my way and I need to kill you all, right? It's like, and that's just what I'm going to do unemotionally without any, I don't even reason of, I don't have consciousness or anything, so I'm just doing stuff. Yeah, that's the paper clip. Yeah, kind of. So I do think like those hypotheticals, and if you had something, this is a big if. If you have that thing that has that level of intelligence and can control things and so on, then I do think that's a big risk.