Token Series Part 1 | How AI Created a New Budget

0:00It's a brave new world. This is not your grandfather's finance. We're embarking on a new way of doing business. So I had to take out the old CFO bat phone and get my friend Ali Hussain on the phone. Ali, who owns AI spend inside the company? What are you hearing? It's a big debate right now, CJ. This problem is much bigger than just what the CTO can do anymore. Are you paying for more than one LLM provider? I'm paying for more than I have fingers on my hand and I have no, it's a pretty normal situation here. The volatility and the number of tools we're now seeing, I think it's just getting way too complicated.

0:38Open Router, which was acquired by Stripe, maybe the most prolific acquisition we've seen in the AI era, just in terms of multiple. It's a way to help CTOs and engineering leaders administer this cost to the org. My hot take is.

0:57Is this thing on? Yesterday's price is not today's price.

1:12Probably to start 2026, we still had a lot of companies out there with these, what they would call experimentation budgets. And it was, hey, just activity for activity's sake. See what you can build. try to just show that we are learning here. Companies are heading into 2027 saying we need to have a real plan around how we incorporate AI into our product, how we invoice and collect for the job to be done and how we price that product. Companies suddenly have a major new variable cost. We're calling it inference, and that is behaving very differently from traditional software spend. So maybe just to start there, what are you hearing from customers?

1:52Yeah, I think in this new world, the reason it's a new world is not that we've changed the word. It's still AI. But the difference to your point was it was kind of exploratory. Like these were side budgets, which CFOs, CEOs, boards were saying, look, like, let's get every function, particularly engineering. But I think it started to sprinkle in other departments throughout the last 24 months to kind of test and try. But I think the turning point is now we've passed the threshold of materiality. The costs of this is showing up pretty universally, and it's not just isolated to product and engineering.

2:32And I think it's opening up new types of questions on how do we get here so fast? How much does this actually cost? And what do we actually find valuable? Well, from a first principles perspective, Ali, who owns AI spend inside the company? What are you hearing? It's a big debate right now, CJ. I think if you asked this question this time last year, it would typically be some type of technical leader. It'd be the CTO for larger organizations, maybe someone who owns IT or the CISO. I think for the first time, there's this big question on, is this a technical lead, CTO, cost for the entire company, which to be fair, it has been in other domains like cloud for a lot of companies.

3:16or is this now a real CFO and finance problem? And I don't actually know if the verdict is out. The best example I have is we have two big products that came out. Open Router, which was acquired by Stripe, maybe the most prolific acquisition we've seen in the AI era just in terms of multiple. And I think that product is highly used and the philosophy of Stripe has always been the engineering team. And I think it's a way to help CTOs and engineering leaders administer this cost to the org. And on the other hand, we saw our friends at Ramp slowly thereafter announced their own offering, which very much felt like it was geared towards the finance and accounting leader.

3:59And so I think the jury is still out. I'm excited to, throughout this episode, kind of talk through who we think wins. But I think I'll start by saying the answer has not yet been resolved in the market. It makes me feel like I'm aging a bit. And it may be the fact that I have three small children. and they don't let me sleep. But I remember when I was working at a company that was moving from perpetual licenses to cloud. And as part of that journey, I saw our gross margin just get completely annihilated. Now it was still like plus 75%, but I was used to something that was over 90%. And that's because we had all these cloud costs in there.

4:35I think there was this initial volley back and forth. Who's going to own this? Finance? Who's on top of AWS, GCP, Azure? or is it going to be the CTO and this new infrastructure team? And I think the finance team actually not gave up, but they conceded it. I don't know if it's the same right now for AI costs. I think it is different for two reasons. One is just the overall quantum we're talking here is even greater than cloud. The relativity of how much is going into the token and inference bucket, I think is already on a path to well exceed cloud. I think it's just hitting a level of comfort or lack of comfort where the CFO must get involved.

5:18The second is, I think the thing with cloud and inference, a lot of the predictability, a lot of influence, the way to kind of optimize very much could sit within the CTO. Like they could put behind more SREs. They could rethink the product. They could configure AWS or Azure or whatever to optimize. this problem is much bigger than just what the CTO can do anymore. As an example here at Tabs, like I looked at my latest Anthropic bill and I thought it was going to be very engineering heavy, but I've got power users ripping through tokens and sales, finance itself. And this problem is multi-domain and multi-dimensional, which I think often is a CFO or CEO problem.

6:01And then the third is it's highly unpredictable. Like at cloud at some point, and look, AWS and all built a bunch of tooling. You'd get that initial jolt. You would have some social contract with the CTO and say, look, I'm good. You own this, but this is your budget and you're going to stay within it. The volatility and the number of tools we're now seeing, I think it's just getting way too complicated to have the CTO do their day job and also manage this and have the same heartache that a CFO will have with the level of just budget inconsistency and financial modeling black box that this is all created.

6:39I love how you laid it out at the department view because the AWS GCP bill, that was very much in your cost of good soul, but you could tie it back to what the CTO's org was doing. And then maybe you had an allocation that you could say, hey, this is for the development of our product that belongs in R &D. But either way, it's not like you had cloud costs showing up in your sales, right? It's not like you had cloud costs showing up in your finance team. But now I think what you're saying is that AI is more like the wiring throughout the house. And there's a certain amount of electricity that's going through all these departments.

7:14So it kind of made sense in that cloud era and the cloud boom for the CTO and to a certain extent, the product team to wrap their arms around, hey, we kind of own this and we're We're more technical so we can forecast it to a degree that may be more reliable. But now we're in this world where I think you need someone with a financial background and who understands usage-based pricing as well to wrap their arms around it. I think you're spot on. I mean, it's almost more like headcount planning or T &E planning. The nuance of what a CFO or a strategic finance person can bring in terms of the level of change, the level of department-level partnership.

7:54trip. That's not to say that the CTO and R &D is not a big part of it. It definitely is. But I think it does a little injustice to try to make this feel more like, oh, this is highly technical. It has to sit under the CTO. I mean, I'll give you my own example. Our head of finance joined. We had a predictive plan for our inference and token costs. It was owned by the CTO and our IT leadership. it's just become out of control. Like the CTO's job was not to run around and have conversations with each of the functions. And I love our CTO. I mean, he's the co-founder of the business, but he's way above his head on how to manage this all.

8:35Well, also in the cloud era, and we can talk about how to budget for this in a second, but just anecdotally, I remember when we moved to the cloud, we had one vendor. Like it was pretty extreme to go multi-cloud. That was not an overnight thing. Right off the bat, you can look at the data. I mean, Ali, are you paying for more than one LLM provider? I'm paying for more than I have fingers on my hand. And I have no, it's a pretty normal situation here. And so I would be very unsurprised if folks not only have more than one, I think we're talking at least five to 10 for most companies right now. It was an amazing observation that OpenRouter comes at it and Stripe comes at it more from the engineering angle.

9:15You could argue Ramp comes at it more from the accounting and finance angle. But can you just break down what problem those are solving? Because I think it helps set the context of the rest of this discussion. Yeah, I think there's just an overall view that we must start to manage kind of the tokens and also the optimization of the models. It feels a lot like cloud right now, but I think ultimately beyond the provider at the model level, there are going to be multiple options and multiple price points. And I think the concept of a router is how do you route those with more choice? and with a little bit more price optimization.

9:52So you don't ultimately feel tied to your specific relationship. It's kind of bringing a kayak type feeling, whether you want to get on a flight to Cancun. It doesn't, you don't need to be tied to just Delta. And within that, you have a couple options to figure out, do you want to take a layover or do you want to go direct there? And what actually solves for the best price and the outcomes you're trying to drive? What do you think the finance layer that administers all of this ultimately looks like? There's a couple of things. One is just an acknowledgement that this is part of the job. My hot take is I do think this ultimately lands more finance and accounting than CTO and so and IT.

10:34But I do think it's going to have to be a partnership. I do want to call out there are certain things that my CTO does, such as deciding which models are actually impactful enough versus is what may be overkill. And I'm not sure the CFO's job is to optimize the specific models for what an engineering team should be building with or what a technical person in marketing should be building with. And so it is a partnership. But in terms of the ultimate long-term stack, I think at the end of the day, there are core accounting functions that have to still go on, whether it's the GL, billing, AP, spend, etc.

11:12But I think there's two broader pillars of tooling everyone needs. One is actually the way you administer and manage the costs at the corporate level of AI, which is a lot of what the routers and the open routers and others are starting to do, which is try to help us get control of so many different options and predict how much we're going to spend and have a little bit of choice around where we lean more or less to kind of manage a budget. And I think the second is, as we all, as organizations start to metabolize and use these costs, how do we actually measure that at a customer level and more sustainably pass this on to the end customer who's ultimately benefiting from these costs?

11:58And so there's a, how do I manage the costs in and be a better steward of finding the value and the utility for my organization. And the second is, as I'm metabolizing these powerful inferences and tokens and LLMs into my products that I'm building, many of them are now being powered more by inference than by cloud or by humans or by software. How am I right-sizing it in a way where I am in a fair way passing that on to my customer. Beyond that, we don't have to overcomplicate it, but I do think it's a completely new way of thinking about the infrastructure needed to be a part of this next wave.

12:39Ali, what I think makes this such a nuanced discussion is that most of the people listening to this are both buyers and sellers of this stuff. I mean, I even look at your company. You have to think about your own P &L and where it shows up, but you also have to think about how it impacts your customers' P &L. I've spent a lot of time thinking about this, is to try to simplify this problem. Part of the chaos you and I are talking about is to also give our listeners very practical advice. The two areas that I do think this rethinks your tooling and how you think about systematizing this is before you had your ERP, you had managing your vendors, your employees, and your customers.

13:17I think now there's like two higher order principles to follow. One is what is the tooling and process you need to understand the types of inference and models and AI you as a company are consuming. And I think that's been a lot of our discourse today is how do you route and optimize the token costs for your company beyond just the engineers, the entire org. And I think the second is, as you're using all of these amazing new ways of building, how are you passing that on to your customer? And I think that's like something certainly we've been spending a lot of time with at Tabs is like you're gonna need a lot of flexibility and pricing and packaging.

13:57You're going to have to figure out what is the way to pass on the cogs where you can hit a gross margin profile that is sustainable. And how do you master the art of the deal with your customer to get them to buy and pay for these services in ways that they've never had to before? And I think that is the next wave of where this all is headed. And just like that, we're going to go 90s sitcom on you and we're going to do the cliffhanger. And we'll be back to explore that on the next episode. Run the Numbers is a mostly media production. Yelling an intro by Fat Joe Artwork by Meg D 'Alessandro Nothing said on this podcast is intended to be business or investment advice It's the sole opinion of me A guy who feeds his dog way too much ice cream And has a history of net operating losses LOL If you like this podcast, hit subscribe and give us 5 stars It will take like 2 seconds and our algorithm Overlords love it Drink water, call your mom and have a great day Peace