Do You Even Need That Trillion-Parameter Model?

0:07Aaron: Good morning, good evening, OVR, and welcome back to the Enterprise AI Show. This is your host, Aaron, and today we have another episode in our Models and Markets series. Brian and I ask the question, do you even need that trillion parameter model? We talk about the current and future state of Frontier Class models. We focus on model size, model harnessing, and intelligent model routing. And we discuss whether bigger models are always better, the economics of AI, and how enterprise applications can benefit from tailored AI solutions. And all of that is coming up right after this break.

0:46Today's show is sponsored by Nasuni. There's a growing gap in AI right now between what's possible in theory and what successfully works at scale inside an enterprise. The difference comes down to unstructured file data. Many AI initiatives struggle because the file data they depend on is scattered, unstructured, and disconnected from where and how work actually happens. Nasuni changes that. It brings your unstructured file data into a single, secure foundation so AI, both generative and agentic, can access it with the context, governance, and performance it needs in production. Bring AI to where your unstructured data lives.

1:20See what it takes to activate your data for AI and request a demo at nasuni.com slash AI. Aaron, you and I back again doing another one of these short shows that we have not yet come up with a name for. But the first couple we did, the last couple we did, were kind of ownership and cost and control focused. What do you have on your mind today? Are we going to get outside of that harness, if you will, of cost and control?

1:46Aaron: Yeah, yeah. Yeah. So the idea was there is like, okay, we kind of want to mix in some business shows and some future in industry kinds of stuff, some macro level shows, if you will. But I also want to dig into some more, you know, more tech technology focused and a little bit of like, okay, state of tech in the industry and then where it's going. And we're going to do our usual like, hey, why are we talking about this right now? What's been in the past? What's the present state look like? And where do we think this is going in the future? And so this topic that came up with was like, when will the models and harnesses be good enough?

2:18Aaron: And why am I saying that right now? And why am I asking that question? Well, there's this concept out there of bench maxing. So basically, you know, there's lots of big models out there. And oh, by the way, there's lots of models that like they're, you kind of like building this. And if you build a trillion parameter model, I mean, that thing costs a lot of money to build. It costs a lot of money to host. It costs a lot of money to maintain. And oh, by the way, do you even need that? Like we were talking on some of the other ones, like, hey, if I compare that to the Ferrari or, or, you know, we compare it to like, okay, what kind of queries are we doing against?

2:53Aaron: A lot of them aren't necessarily doing deep research. And so like the past piece I wanted to talk about was like, when was the last time we had a really big wow moment in versions? like for me personally, it was like a GPT four level maybe, right? Like I noticed to jump from three to four, but you know, from all of them since I wasn't a big, like, yeah, like you kind of said, Oh, you know, some of the folks in the industry can tell the difference between the models and it might have a different vibe or it might have a different this or it might do, you know what, if all you want to do is, is have it, you know, question answer kind of thing, or agent go off and do a task and tell me when you're done, hey, guess what?

3:40Aaron: You don't need a trillion parameter model, right? And so like, are we reaching a state where everything's good enough and now we're just feeding in the industry of up and to the right? Yeah. I mean, I think that's a discussion that it's been, I feel like it's been going on for now, you know, at least a year is, you know, right now the economics are still incredibly driven by, you know, the model, the model being big, the model being, quote unquote, frontier on however many levels. And again, I'm trying to think of the equivalent of this because it would be going back quite a long ways. But I mean, I'm sure back in the day when Oracle's first database or the first time series database or whatever it was came along, it was pretty revolutionary.

4:27And it was like, oh, wow, we're accessing data really fast. We're able to do all sorts of merges and queries and other stuff that was data-centric. And then at some point, the industry sort of said, the thing that really makes accessing this data or the information powerful is the user experience, the way it's blended into other workflows, the value moved up the application chain. And I think people are constantly questioning, okay, is that where we now start going? Is that where we now start going? And I feel like a little bit, we skipped it a little bit because we kind of went from chatbots were the first sort of application.

5:06Then people couldn't really figure out a whole lot of things other than chatbots. And then they've sort of moved on to agents and agentic. And we really haven't built too, too many, you know, kind of classic applications on top. And maybe that's because that's not the way we're going to, we're necessarily going to do these things. But, you know, it does feel like you're right, What you get out of the model is as good as it is, and I think people still expect it to sort of, you know, it's going to evolve and it's going to have more recent data. But it does feel like all the attention these days has been focused on the harness, on narrowing the experience, on perfecting the experience, on trying to reduce hallucinations or, you know, make it seem more deterministic.

5:47I mean, it feels like that's where people feel like they can differentiate themselves, especially if they don't own the model, for example, right? I think they're trying to, and then I think they're trying to figure out a way to make that harness portable so that, you know, if, if they want to move off of one model to another, I don't know how realistic that is yet. You, you probably know that a little bit better because you, you live in the model world, but I feel like that's another top of mind thing people are trying to figure out. Yeah.

6:14Aaron: Yeah. And for me, like, okay, if I kind of look at the present state of all of this, and this kind of goes to where I was the core message or the core idea behind the question was, I almost feel like, okay, the models is a race to the top, meaning benchmarks. but that race to the top is leading to a race to the bottom all at the same time and because at the end of the day i mean you know we have no way to prove it but i mean a lot of the models especially the oss models you know the chinese oss models in particular you know there's lots of acquisite accusations of just hey we're just distilling off the big frontier models anyway and oh by the way it probably costs millions of dollars to generate these things and oh by the way uh if you're state of the art or soda, as it's referred to in the industry, a lot of times, if you're soda, you're soda for like two days.

7:00Yeah.

7:01Aaron: And Oh, by the way, you probably trained it to specifically be soda against that benchmark. And so like, I don't know, like, I, I feel like maybe I'm getting already what a couple of years into this, I'm already old man yells at AI or old man yells at cloud here of like, you know, I kind of have to step back every once in a while and go, but does this solve enterprise problems? And, you know, with us, like kind of us, but coming from these enterprise backgrounds, it's like, what's the business outcome, you know, scoring just a, you know, a couple points more on a benchmark really do? Well, you know, and I think if we're, if we're trying to, with these, these little short shows, if we're trying to draw some analogies to things, I mean, if you think about like consumer types of applications, right, the ones that are super, super finely tuned and always sort of have to be latest and greatest are things like ad serving, right?

7:54Like, you know, recommendation engines and ad serving that are very, very tied to like your most core business activity. But, you know, for a lot of the other stuff that you do, you don't need latest and greatest of stuff. And if you think about most like enterprises, there's probably only a few things that they do. You know, maybe it's, you know, if you're an airline, you're putting your best and brightest on like fuel cost and like seating optimization. So you can, you know, get people to pay you more money for a better seat or a better, but like, you're not putting that on the other 89, 95 % of the applications.

8:27And I think we're going to see probably a similar thing with AI is there will be a small set of stuff that the frontier level things probably make the most sense for, because you know, you look at your competition or you look at the thing that's creating your moat for you and And you're like, yep, I see the value of staying on Frontier or state-of-the-art because it's something I can continue to create great margin with. But for everything else, if it's something that's just doing employee productivity or it's doing customer outreach, how much differentiation can you really do there? And even with that, how much are you willing to pay to differentiate that next thing?

9:07right? If you can't guarantee that you can get somebody to click back and buy something with one email as opposed to six emails, reminding them to do stuff, you don't need AI to do A-B testing on those six emails. You're just like, no, I just know the system takes six repetitive tasks and then they finally buy something from me.

9:26Aaron: So - Well, it reminds me of there was, oh gosh, I think it was, by the way, everyone go subscribe to the podcast This Week in AI. It's a really good news podcasts, but they also do some good breakdowns of things. But it was really interesting. They had this theory once upon a time, and I don't know if it was their theory or somebody else's, but it was basically like, Hey, the only reason we still need to go up into the right is for like these frontier labs should honestly just take their models and go do like day trading with them because like that way you can at least make some money back. And it goes back to your core differentiator and like, what's the, what's the idea of where AI is going to move the needle the most kind of thing.

10:05Aaron: And it just, it kind of made me like laugh, but at the same time go, Hmm. Yeah. Like, where do you really, really need all of these things? And, uh, you know, at the end of the day, the enterprise, a lot of times is like, okay, I need to solve a problem, right? But I need to solve a problem in a way that's economically viable for the company gets the best ROI and is the fastest, right? It's your typical project management constraints, right? Quality, time, and cost. And what do you want? And you can only have two of the three kind of thing. And is AI helping that or hurting that in something like this?

10:43Well, and the other part of it, it's going to come back to, you know, if, you know, going back to one of the shows we did a couple of weeks ago, you know, if you feel like you need a little more control over your environment, and that ends up being that you're either, you know, buying or renting GPUs, you don't necessarily want to start with a trillion parameter model on, you know, what might be limited GPUs or limited size GPUs. You want something that's going to fit, you know, fit on the GPUs, be able to do a query within that one GPU, not have to ship it all over the place, do all sorts of load balancing and stuff like that.

11:17So there's, you know, there's going to be both, you know, quality of the work decisions, but also economic decisions that, you know, have some, some ripple effects system. It's what you choose.

11:27Aaron: And, and by the way, too, so we, I want to go into the future state for a little bit here. And here's, here's how I see the future state of all of this working out is yes. Like we just said, there is, Hey, guess what? There is use cases for these, you know, really, really big models, but Oh, by the way, there's lots of use cases for small models. There's lots of use cases for a mixture of experts and dense and all the other different technologies that are out there. But I think this is where the model world really goes into, this is where agentic and agentic workflows and more specifically models and model routing comes into play of there's going to be one central interface.

12:06Aaron: Like I really firmly believe we're going to get to a point in the industry. It's like, do you remember when you had to think about the model you used? Like you just have a task and you put it in the thing and the thing decides where to route it and how to, how to route it. And, and maybe it's kind of a combination of our last couple of shows of like, Oh, by the way, that wraps up the FinOps and it owns a, it wraps up their own versus renting your AI of like, Hey, guess what? You might own the small models, but you might rent the big models. Like I've, I feel like there's this future. We haven't even begun to scratch the surface on, which is intelligent model routing.

12:42Aaron: So that's my idea of where all this goes. Well, and that's, and that, and that's going to be an area that'll be interesting because that's the kind of thing where what would normally happen in the marketplace is, yes, there would be some sort of semantic routing capability, but it would be owned by the model provider itself, right? It would be owned by an AWS or an Azure, and they would be routing you between, you know, their five or six models at some tiered pricing. And then maybe there's this last resort of, oh, okay, somebody said they need to be able to try something else. or, you know, an open source community would come along and build the equivalent of like Nginx, right?

13:17So that you could do load balancing that you could have better control over that, you know, was a lower cost price point or something like that. Because the reality is, if you offer customers something that, you know, kind of mediates between what your profit looks like, you're going to want to try and get some extra profit out of that mediated thing, right? Because it's like the equivalent of somebody being like, well, you could pick your own stocks for your portfolio. But if you have an expert do it for you, then – For just a small cut. Yeah, you should pay them a 1 % fee for basically doing – so yeah, I mean that's the – that'll be an interesting thing to see that evolve.

13:55I think you're right. There's going to be a demand for that. The question becomes how does that show itself in the marketplace? Yeah.

14:02Aaron: And, and I'll, uh, and by the way, for everyone out there that, that actually uses open call on a regular basis and knows the technology inside and out, I may completely mess this up because I've been researching some open call stuff here recently for actually for some show related stuff. But the idea is, yeah, okay. Open call runs on a harness. It runs on a workstation, but Oh, by the way, when it needs to do stuff, it will route out and call bigger models for bigger things based off of certain decisions and connectors. And I almost see that kind of happening of like, you know, AI is calling AI is kind of thing and just trying to break it down into something that's easy and consumable and folks have access to today.

14:41So, yeah, I think we're, we're, we're still a long way from like in a perfect world, this is what it would look like for me. And, you know, this is, this is the reality of it, but we can, we can dream, we can, we can hope. Agreed. Agreed.

14:54Aaron: All right, man, you want to wrap this one up there? Yeah, absolutely. Absolutely. Everyone out there, thank you so much for listening. And as always, reach out to us at any time. If you enjoy these shows, we'd love to hear your ideas of what you want to hear as well. So definitely send us an email, reach out on the socials. We'd love to hear from you. Thank you. Great. All right. Talk to you next week, folks. Thanks for listening. Check us out at theenterpriseaishow.com for past shows, newsletters, and all things Enterprise AI.