Dan McCarthy | University of Maryland - The Unit Economics of AI - Can LLMs Actually Make Money?

0:00The CMO Confidential podcast is a proud member of the I Hear Everything podcast network. Looking to launch or scale your podcast? I Hear Everything delivers podcast production, growth, and monetization solutions that transform your words into profit. Ready to give your brand a voice? Then visit IHearEverything.com. Welcome to CMO Confidential, the podcast that takes you inside the drama, decisions, and choices that go with being the head of marketing. Hosted by five-time CMO Mike Linton. Typeface has completely changed the process of large campaign executions. Everyone knows AI can help with images and headlines, but the real impact is making the leap from a creative brief to a live multi-channel large campaign at speed.

0:52Typeface just announced its marketing orchestration Engine, the first platform built to automate campaign workflows. You can roll out campaigns that used to take months in just a few hours. Typeface uses agentic AI to orchestrate the entire process, taking one campaign and instantly orchestrating it into thousands of personalized experiences across ads, email, and video. Major brands like ASICS and Post Holdings are already transforming their marketing with Typeface. See how to move from brief to personalized campaigns in hours, not months at typeface.ai slash CMO. Welcome marketers, advertisers, and those who love them to Chief Marketing Officer Confidential.

1:40CMO Confidential is a program that takes you inside the drama, the decisions, and the politics that go with being the head of marketing at any company in what is one of the most scrutinized jobs in the executive suite. I'm Mike Linton, the former Chief Marketing Officer of Best Buy, eBay, Farmers Insurance, and Ancestry.com. I'm here today with my guest, Dr. Dan McCarthy. Today's topic, the unit economics of AI. Can large language model companies actually make money? We know they have high valuations, but today we're going to talk about how they make money through a customer lifetime value lens.

2:18Now, Dan is a professor at the University of Maryland. Previously, he was an assistant marketing professor at Emory, where he created what may be the first ever customer lifetime value course. His specialty, and you want to take notes on this, is the application of statistical methodology to contemporary marketing problems. He has been one of our most popular guests over the last few years discussing CLTV, the business impact of privacy policies and the power of loyalty programs. So it's great to have him back to discuss his latest work. Welcome back, Dan.

2:57Dan McCarthy:We're going to make it a four-peat. There we go. That's right. So, hey, Dan, just to assure everyone is starting from the same place, give us a quick overview of customer lifetime value and customer-based corporate valuation and how they're used to evaluate businesses. Yes, the customer lifetime value is this number that's kind of over every single person's head that represents the overall profitability of that customer to the firm. And as you'd imagine, there's a lot of customers that aren't worth very much. And there's a few customers that are worth a lot. And that could be really, really powerful to know.

3:38Dan McCarthy:And when you kind of roll it all together, that gives you a sense of when this company acquires its customers, are they actually generating a return? And a lot of the nuance comes in like, what costs do we include? And that's where I think CLV is kind of extended to what we call customer-based corporate valuation or CBCV. that typically what CLV is supposed to represent is when I bring in that next marginal customer, what is the incremental profitability that I'm going to get from them? And so I've got all my accountants, I've got all the lawyers, I've got the headquarters, I'm paying the CEO.

4:19Dan McCarthy:Those are all relatively fixed. But when I bring in that next customer, what is the additional incremental profitability that customer will throw off? And so I need to know how much I'm spending to bring them in. I need to know the revenue I'm going to get. And then I need to subtract out all of those variable costs that are associated with the revenue that are bringing in. And one of the keys here is that all customers are not even close to equal and understanding which ones are worth more. Say you're an airline and you have someone that flies 200 times a year versus someone that flies 10 times a year.

4:52Knowing that and knowing the descriptor really makes a difference and it should make a difference in how you market and also the valuation of the company, right?

5:02Dan McCarthy:Yeah, and instead of thinking of kind of like your typical customer, or I've got my ideal customer profile, you know, there's these personas, and we have like the, it's this type of person that I want to acquire, you know, that they're my best customer. Typically, what you find is that the customer base from a value perspective is like a barbell, that 80 % of the customers are worth very little, nothing, or maybe even costing you money. And then you're making all of your money on like the top 5%, 10 % of the customers. And so knowing that, it's like, well, what's so different about them? What are they like?

5:39Dan McCarthy:What's bringing them in the door? Where are they geographically located? You know, what was it? What was the acquisition channel that I acquired them through? You know, suddenly it's like, oh, that's stuff that I really want to know. But yeah, I think the key in a case like these AI companies is that there's customer lifetime value, which is like the quality of the customers that you bring in. But there is not, at least with CLV on its own, there's no notion of quantity. you know how many am I going to bring in and what is the kind of like the cost of the the running of the customer acquisition machine you know so every period I need to spend a certain amount of money to you know to pay all of my overhead expenses whatever those might be and you know ultimately that really matters like if it's really really big then you need to earn a much higher rate of return on your customer for the whole business to eventually be able to grow its way in the profitability.

6:39Dan McCarthy:And so, yeah, so that's kind of, when we talk about the LLMs in particular, you both need to think about lifetime value, but then you also need to think about, you know, well, how much do they have to spend to kind of keep the whole thing going? And how many - Especially when you have a valuation of, you know, hundreds of billions of dollars on something where there's only several years worth of data. Is that what made you want to dig into this analytically? Or what made you want to go look at this through a CLTB lens? It's really, it's kind of two things. One is, I think that when you're thinking about it from the perspective of, is this business going to justify its valuation?

7:22Dan McCarthy:The CLB lens is the most valuable when you have a business that's losing money, but growing quickly. Because then the big question is, are they just buying growth and they won't ever be able to grow their way into profitability? Or is it a business that's actually doing very well, like they're acquiring customers profitably, but they're not yet overall profitable yet because they're still scaling? And that's an incredibly important distinction. And some companies that we run the numbers on, we find that they're good. Some companies, we find that they're bad. And so, for one, it's like a really nice industry where this sort of analysis is like specifically suited to it.

8:06Dan McCarthy:But then I think the second thing is I am like a voracious user of all of these services. And yeah, I do suggest - You're like a tiger. Yeah. So it's like, I want to know, you know, should I be worried here? You know, is this company going to have, does it have a viable business model? Because, you know, if it doesn't, then it's going to have real implications for me because either the price is going to need to go way up And then it's like, oh, well, that's kind of a bummer. Or, you know, in the worst possible scenario, you know, is this company going to crash and burn and I should really make sure that I'm splitting my usage across multiple services?

8:45So, yes, because when I get dependent on one thing and I build it into my, say, entire MarTech stack and all my work and then it disappears, that would be a big problem, right?

8:55Dan McCarthy:I think a lot of companies should be thinking about that, you know, that they're going to need to know because a lot of companies and a lot of people within those companies are building, you know, workflow after workflow around certain things that they do. And if this, you know, if the company is going to run into major issues, then they're going to need to think of contingency planning. Got it. So let's talk about the work you did with, because you looked at all the major AI companies, right? And tell us what you learned for products like ChatGBT and Anthropic and everything else. yeah the big thing um i'd say one of the big learnings was that among the the paying subscribers the economics are actually pretty good so um yeah the way that i would kind of conceptualize these businesses is that they have free plans and the free users cost these businesses a lot of money and that's one way where this is pretty different actually from like a pure software business because a software business like slack there are costs associated with their free tier but software is cheap and so you know the marginal cost is going to be very small but here every time you you know write that chat to create that anime picture of your family uh it's actually a real expense for the firm and so these free tiers are actually quite expensive and uh and i would consider them to be kind of like the on-ramp to the paid plans.

10:38Dan McCarthy:And when you look at expense, you're talking about, I mean, I have to have everything working and then I'm using up huge amounts of energy, et cetera. I have a lot of money I'm spending just to be on, right? Is that what you're thinking? Well, not only that, but even just the cost of running the queries, Every time a query is run, there is what's called kind of inference costs. And inference costs are one of the biggest expenses right now of the LLMs. And so, yeah, so those are purely variable. You hit, you know, you type in that thing. It was an example of how inference costs work. Like, just take us through one where I ask about something about the University of Maryland, say.

11:20Dan McCarthy:Yeah, all of the tokens that are used. So there's tokens that are, you know, you're asking the question and that's basically they pay per word. And then everything that you might have had in context, you know, like keep this document or keep these images, keep this paper as context to help, you know, to help answer this question that I'm asking you. All of that corpus of data basically represents a cost to OpenAI or to Cloud or what have you. And then it's going to spit out its answer. And the length of the answer, again, represents kind of token usage. So you can think of the direct costs associated with inference as being a function of the number of tokens that are used by the service.

12:13Got it. And so I have this free thing. And then I obviously want people to trade up to buy stuff. Tell us what you're learning on the acquisition and then the retention and the customer behavior you're seeing across your cohorts.

12:31Dan McCarthy:I think a lot of people find the free tiers to be quite attractive. And I think one of the issues that OpenAI has been facing is that there's like all these users and then there's some percent of them that actually bite the bullet and say i'm going to pay 20 a month or 200 a month um and that percentage back at the end of 2023 it was almost you know call it six percent and then it went to five percent and right now late 2025 it's below four percent so they've had a really big increase in the number of users it's gone from like 100 million to like 900 million over that period of time. But the number of paid subscribers has not gone up to nearly the same degree.

13:21And so there's this question,

13:23Dan McCarthy:like maybe is the free plan too good? You know, like people just want to hang out on the free plan and, you know, they don't need to upgrade. So again, there's this whole question that a lot of companies face that have freemium models of how ultimately you want people, you want to make money. We had the same thing at Ancestry, yeah. Yeah, so it's like, well, what can we do? Either we can try and lose less money on the free people or we can make the free tier worse. We can make the paid plan better. And then there's like a whole bunch of other things that we can kind of inject in certain parts of the process that really will make it the most likely for people to want to take the jump.

14:14Dan McCarthy:And so again, when we think about ads, everyone's talking about ads right now. Ads would be one way that they can both simultaneously lose less money on the free plan and they can create more of a gap between the free plan and the paid plan. Right, because they can take ads out. Yeah, it's like, this is kind of annoying. I don't want to be exposed to this. So, so it kind of kills two birds with one stone. But yeah, I think, you know, there's been a lot of articles written about how OpenAI in particular, their gross margin, actually, this was true of both OpenAI and Anthropic, their gross margins were a lot lower than they even were saying it was going to be.

14:59Dan McCarthy:You know, it's going to be, it was 40%. Now it's down to 33%. They're like, why the heck is this getting so much worse. And part of the reason why is because they have all of these free users that are just consuming so much cost. And so I think the ad plan is a way for them to help kind of defray some of that expense associated with a free plan. We are taking a short break from this show for a word from our sponsor, Typeface.

15:31to orchestrate workflows across channels meet arc agents your ai teammates handling complexity so you can focus on what matters all within a reimagined workspace where you and ai create as one spaces works the way you do create anything from documents to videos to entire campaigns at scale. Our agents help you craft campaign storyboards powered by your brand hub, giving you the perfect starting point. Transform one asset into countless variations for every audience and market.

16:12Join industry leaders already reimagining their content lifecycle with Typeface. Welcome to marketing's next chapter, where every story finds its voice.

16:25Now back to our discussion. How does this play out? We're talking now about valuations. A lot of these companies have enormous valuations. Tell us, take us through kind of the unit economics, where they end up and how people should be thinking about it.

16:47Dan McCarthy:Yeah, so the one, you know, the big question, you know, to me comes down to CAC. That's customer acquisition cost, everybody. Customer acquisition cost, the value of customers after they've been acquired, and then just how many can you bring in the door, and what happens with your overhead. And so I think the CLB perspective is actually pretty optimistic that the paid subscribers are actually pretty profitable, even when you factor in pretty generously the cost of the free tier. I mean, even at$20, just$20 a month, I'm still profitable, even given massive token usage. Because, yeah, the margin is actually quite good for those people.

17:34Dan McCarthy:Okay. it's about 70 % and it's way up. So obviously in a lot of other businesses, that margin, it doesn't change very much. Like if you're selling food, you know, and you're twice as big, you're probably still paying the same amount on food. Here, the efficiency of the models has been improving and improving and improving. And so, so that margin has been going like this. So, so if they can get people to paid, they're getting a nice kind of 70 % ish variable margin on those people. Now they're not only getting 20 a month for some people, they're paying 200 a month. And then there's the API usage as well.

18:22Dan McCarthy:And the customer acquisition costs ends up being around, I was estimated to be around 30 bucks. Okay. And retention is amazing. So once you have this tool, you're probably not going to dump it. Right. You're just even if you have more tools, like if I have clock and anthropic and I can use I can use anything I want. Right. Gemini. Yeah, I'll put a big asterisk on this. So, you know, as you know, because I've talked about, you know, using this data before on your show, credit card panel data is a really nice data source for certain types of activity. And this is a great use for it because people, they kind of build their credit card for this.

19:03Dan McCarthy:And so we can directly observe customer retention through the credit card panel data. And companies like OpenAI, they have best in class retention. The people, they just stay. Now, obviously, it's funny. I know we were almost going to do this show a little earlier. I feel like the passage of time has just made this more and more interesting. so just this past weekend we had this humongous kerfuffle where anthropic was deemed a supply chain risk and then the you know it's like right after the department of war shunted them to the side it's like five minutes later open a is like yeah i'll take the business and now everyone is you know saying i'm going to dump my chat gpt subscription and i'm going to subscribe to clod So that is all, you know, I think that is, trust is a very big issue with the LLMs because of the nature of the data that they're collecting.

20:04Dan McCarthy:But holding all that aside, the base rate of retention is very high in this category. And as you're kind of alluding to, people like me, if we find a lot of value in ChatGPT, and then we find a lot of value in Claude, it's not like people will only sign up for one. There are a lot of people who do what's called multi-homing, where you kind of have multiple subscriptions. Well, it's no different than streaming services, right? I mean, I have Netflix. I have Amazon Prime. I have HBO Max. It's not like I only have one here. You're saying that this could be really a big deal. And I hear almost beneath that is a lot of consumers are probably not going to drop chat GPT if it's really functional for them.

20:57You might really like Claude, but let's say you want to generate images.

21:03Dan McCarthy:Well, sorry, you know, because Claude can't really, they don't really do images. And I think it's smart of them. I think that image generation is extremely expensive. Yeah. And they've chosen to focus on these knowledge worker use cases. And so it makes a lot of sense. But if you wanted to do that sort of thing, well, you're kind of stuck. Also, if you wanted to do like super high-end research where you want these models to think about a problem for an hour. Well, Claude is not amazing for that. It's really good for questions. It'll give you an answer, you know, in a relatively short amount of time.

21:49Dan McCarthy:But it won't, you know, it won't think for an hour like ChatGPT Pro 5.4 extended thinking will. And so, again, it's not that they aren't really good for what they do. But there are these, you know, kind of there's like this Venn diagram of the use cases. And if you need certain use cases that Claude simply doesn't do, you're not going to just not do those things if they're of value to you. So, yeah, I think that we're already empirically observing it. Like when you look to the data, you'll see a lot of people are using, you know, I'm subscribed to probably, you know, four or five different paid plans right now, just for different things.

22:32and um and i can't see how you know for at least three or four of them i can't see how i would give them up so yeah yes if you can't make caricatures with claude i mean geez you gotta have something yeah where am i gonna get my anime pictures you know that's right um so you're saying all these look like pretty good businesses to you and it's just a matter of time before they have a path to big profitability, right? Well, actually, no. Okay. Tell us what you're saying.

23:08Dan McCarthy:Yeah. So the CLV is not bad, but there's a question of quantity and there's a question of what they call these training costs. So inference costs is one of the big expenses. That's again, you do a query, you got to pay that cost. That's like the cost of food for a meal kit company. but unlike the meal kit company these companies are spending a lot of money on training costs and r &d and those are they're not a variable cost in the sense that you know if you bring in that next you know claude subscriber it's not like you're going to need to spend more on training up you know opus 4.7 or something like that but they need that next model and in that sense the Tell us how the training models work and then come back to this point, which is why they have to keep spending.

24:04Dan McCarthy:That's the R &D that brings them to the new models. So if you go, if we think about chat GPT, there's, call it like 3.5, 4, there's 01, there's 03, there's chat GPT 5, and then 5.2. And then again, just over the past week, they just broke, you know, chat, chat GPT 5.4. They need to keep coming up with these new models to fend off the competitions models. And yeah, they, they need to be smarter. They need to be better. Everyone's focusing on these benchmarks. And the models have come such a long way, you know, over the past, over the past year. So they feel like they're in kind of this like nuclear arms race.

24:52Dan McCarthy:really it's primarily Chachi PT, Claude, Gemini. And I guess if you think that Grok is part of the set, then Grok. But they're all duking it out. And when I train a model, tell us what happens when I'm training from 5.3 to 5.4. What do I have to do to train the model? I'm going to leave that to the people at the company. I wish I could say all I can say is the nature of the expense doesn't vary as a function of subscribers. What it is, is it's just finding ways to make the algorithms better. Maybe it's some combination of that, you know, training on better data. And I'd say the other thing that often gets lumped into that category is building out additional functionality for the services.

25:42And so, yeah,

25:44Dan McCarthy:I remember, you know, back in the day, I didn't used to be able to have Claude generate any sort of Excel file for me. Now it can generate Excel files with all the formulas in it. And it can analyze all of my Excel files, you know, all the inner workings of them to find errors in them. And so, you know, that wouldn't necessarily, that'd be like some combination of, you know, better model, but also just like better functionality, better interoperability with, you know, with the sort of tools that people use. And this costs a lot of money is what you're saying. It's like, if I'm running a restaurant, this is the cost of the stakes.