The Reality of AI Economics With Paul Kedrosky

24:17These are tokens. So the idea that you're having to pay a fixed obligation, these notes that have been issued with respect to the debt to finance data centers, with a thing that's falling 70 % or 80 % year over year in price, just try and run an auto company that way with a significant debt obligation. But the price of tokens coming down wouldn't affect GPU compute in that way, though. Also, that price coming down isn't necessarily a result of cost savings, but it's a result of the companies cutting the prices. Isn't the problem that they're also full of these depreciating GPUs as well? Yeah, yeah, yeah.

24:52So there's a double whammy. There's both sides of it, right? So to a degree, if you're working with a model directly through the API, which the largest issuers are, or at least if you're in a production position, you're actually paying an API price which is metered at the token level. So you do see those token prices directly. And then secondarily, you have the problem that the GPUs themselves, insofar as their capital investment around which the investment is predicated, the depreciation of those items is relatively rapid, too. The analogy I always make to people is that it really depends on what they were used for historically.

25:25So if they're just being used for inference, to a degree, they have a longer lifespan. But if they were ever used for training, which is like flat out, pedal to the floor, 24-7, huge workload, it's kind of like you had a car that was only driven to church on Sundays and a car that was raced at Le Mans one weekend. They have the same number of miles. I know which car I want. The same thing applies to GPUs. yeah and the other thing as well is i've really been looking at this so you've you brought up the wood max study which was awesome i don't know if you saw the sightline climate one where it was like of the 16 gigawatts that were meant to come online this year only five are actually under construction i think that there's a big problem with just the speed of the rollout and the upgrade cycle because we are still going to be installing blackwell gpus into 2027 if not 2028 that's insane I worked it out as it's like, it takes six months to install a single quarter's worth of GPUs.

26:24But it's like, at some point, NVIDIA has to slow, not even because of me wanting it to or not, but because where are they going? Like, where are we putting these things? I mean, Taiwan warehouses, I guess. Yeah, Taiwan warehouses. Yeah, and that's a big problem, is the build-out. So this problem of – and one of the Northeastern – I've forgotten which utility in the Northeast just recently put out some data on this, showing that something like the data you suggest, which is like 25 % of the total commit was actually ever produced and likely will ever be built out. And that's in part because a lot of these things are speculative projects, naming no names.

27:02There's a very large Texas company that's doing this directly. That is a very speculative position. We're going to power it all behind the meter with nuclear reactors and all these kinds of things. But this is a game we've seen back to, I mean, the analogy I make all the time is back to Chinatown. This is like Chinatown, right, where I'm buying up real estate with numbered companies in hopes of securing water rights. I saw this. I saw this when Jack Nicholson was wandering around. Jake Giddies was wandering around in the deserts of California. I don't know what you mean. Tell me. What do you mean the Chinatown example?

27:31So what's happening is increasingly a lot of what's going on under the hood here that's creating the impression of a buildout that doesn't exist are these things called powered land companies. So powered land companies are these speculators, they don't call themselves that, who look for strategic locations where using numbered companies, they can purchase real estate that has access to peering points. So high speed interconnection to the - What is a numbered company as well? I'm really sorry. So a company that doesn't make it obvious to who the actual direct owners are. So it's not clear what their purpose is.

28:02So I think of it as like Cayman Islands, but it isn't. So the idea that you're trying to at least loosely obfuscate what the ownership and purpose are. But even that's less important. The idea, though, that they're buying on a speculative basis, this tracts of land could be hundreds of acres in some location that has access to power, access to water, potentially access to a peering point, a high speed interconnection point to the broader Internet. And then they lock that up. And then they go out and say, okay, I've got this position. You guys need to build up more data center capacity. I'm talking to the hyperscalers.

28:35Look at me. You got nowhere else to go. I've locked all this up early on. Kind of like locking up the water for the orange groves. Right. Is that widespread? Is that widespread? That's so bad. Because this whole time I've been looking for the speculative part. I will fully admit that's been it. If it's the land, that's not great for anyone involved. But even then, though, GPUs are still being sold. That's the real thing that's getting made. Right. But part of the problem is, and there was a great piece from Trendforce, I think it was the other day, one of the market research firms in this. They were pointing out how many firms are now buying LTAs, long-term purchase agreements, because they're being told that if they don't lock in demand now, lock in now, they won't get product in two years.

29:19So this is the other layer, that a lot of what you're seeing as purchasing is completely speculative by people who are worried that if they don't lock in a long-term purchase agreement now, they will never get supply later. I'll worry about that other stuff later on, but for now I need to sign up. So that's the other piece that a lot of people miss here is a lot of the demand now is increasingly tied into these sorts of long-term purchase agreements, which have nothing to do with actual units being shipped. The mob boss thing again. Yeah, yeah, yeah. but that's but that's i was joking the other day i told someone this the other day who was i talking to i saw the wall street journal was talking they were saying like it's a it's kind of like the old mafia threat like really nice air market you have there something happened to it oh you want you don't i guess you don't want vera rubin anymore i guess we'll have to give the vera rubin it's exactly like that so this is the thing so those two pieces are really important because it creates the impression of unit growth where unit growth doesn't actually exist because it's predicated on locking supply in the hopes of something later.

30:20But at the other side of it, you've also got the speculative land component. There was a company the other day, there was a great Bloomberg story about it, who raised, I think it was$3 billion in junk bonds for exactly this purpose. So this is highly speculative stuff that's actually seen - Terrible, maybe? No, no, not terrible. It was just literally last week. I was called Tract. Tract, I think it was called T-R-A-C-T. Yes, Tract. I remember this one. This is the insane thing. They're still able to raise those bonds, though. They're still able to get the money. Well, that's because, again, this is the insidious problem here is there's this idea that if I'm successful, my counterparty, the counterparty in the data centers, they're good for it because Microsoft, Google.

31:03So instead of having a bunch of dodgy Florida strippers or something on the other side of this, like in the financial crisis, what I've got on the other side, my counterparty has a high credit, very focused, very small group. It's the Microsofts and Googles and others. So people are willing to take much crazier risks because the counterparty looks so good from a credit standpoint, which is very different from what happened in the financial crisis, where I wouldn't have issued junk to create something that was going to be purchased by who knows. But if it's Microsoft, Google, and the other hyperskills, and all of a sudden, I'm like, you know what?

31:34If this works, they're good for it. But what if it doesn't work? Well, then, of course, you end up with a lot of room, a lot of extra buildings for, you know, laser tag or something like this. I'm excited about the laser tag arena future we have. Just America's the laser tag capital of the world. That's right. We got a lot of extra space for you. Store it and laser tag.

32:27I'm I'm I'm I'm radio, and podcasting. Call 844-844-IHEART to get started. That's 844-844-IHEART. 2%. That is the number of people who take the stairs when there is also an escalator available. I'm Michael Easter, and on my podcast, 2%, I break down the science of mental toughness, fitness, and building resilience in our strange modern world. I'll be speaking with writers, researchers and other health and fitness experts and more to look past the impractical and way too complex pseudoscience that dominates the wellness industry. We really believe that seed oils were inherently inflammatory. We got it wrong.

33:12Many of the problems that we are freaked out about in the world are the result of stress. Put yourself through some hardships and you will come out on the other side a happier more fulfilled healthier person listen to two percent that's t-w-o percent on the iheart radio app apple podcasts or wherever you get your podcasts on the serving pancakes podcast conversations about volleyball go beyond the court today we have a little best friend compatibility test okay how long have we been best friends for since the day we met as the league one volleyball season heads towards its final stretch there's no better time to tune in.

33:51We really are like yin and yang, vodka and tequila. You'll hear unfiltered analysis, behind the scenes stories and conversations with leaders making an impact across the sport. Today we have Logan Lednecke. I feel like our fan base in general is very connected. It's like a comforting feeling getting to play at home. Whether you're following the final push of love season or just love the game, serving pancakes brings you closer to the action and the people shaping the future of volleyball. Jordan Thompson had that microphone out. God forbid we make mistakes or cuss at our coach like when time or two times open your free iheart radio app search serving pancakes and listen now this has been serving pancakes and we'll catch you on the flip side okay presented by capital one founding partner of iheart women's sports hey i'm norah jones and i love playing music with people so much that my podcast called playing along is back i sit down with musicians from all musical styles to play songs together in an intimate setting.

34:47Every episode's a little different, but it all involves music and conversation with some of my favorite musicians. Over the past two seasons, I've had special guests like Dave Grohl, Leve, Mavis Staples, Remy Wolfe, Jeff Tweedy, really too many to name. And this season I've sat down with Alessia Cara, Sarah McLachlan, John Legend, and more. Check out my new episode with Josh Groban. You related to the Phantom at that point? Yeah, I was definitely the Phantom in that. That's so funny. Share each day with me, each night, each morning. Say you love me, you know I love you. So come hang out with us in the studio and listen to Playing Along on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

35:39because the only thing we know for sure is that the the efficiency of inference if you buy the argument that we're transitioning rapidly to inference the efficiency of the inference is going on is rising rapidly because of things like distillation models are shrinking chips are becoming more efficient um there's less memory required though because that's the thing i've the people that have told me inference is getting more efficient are usually referring to nvidia demo is based entirely on oh yeah yeah no no that's a ridiculous position right right right look at companies like um fractile out of the uk um an interesting company with i think some really groundbreaking inference technology that we'll be shipping this year look at a company like talus in Toronto, TAA.

36:21I'll look into them. So Talos is a good example. They're doing some innovative stuff showing, so a high-speed inference chip today might do a few hundred tokens per second. That's considered a relative, leaving aside the wattage required. That's a relatively high-speed token producing chip. So Talos is demoing 16 ,000 tokens per second. So we're seeing step function increases at lower power from some of these next-generation silicon vendors. Granted, they're not going to dominate the marketplace But the idea that we're not going to see step function changes that we can project into the future based on what we've seen in the recent past is just dead wrong.

36:57But the thing is, what models are they able to do that with? Because again, a lot of, I mean, all of the benchmarks we see are based on open source models because open source models are open source. Those are the ones they can test on. I feel like everything, every time I get any kind of leak out of Azure or AWS about these models, it's, I have a Microsoft person who told me it's like four to six, sorry, four to 12 GPUs for one generation of the smaller reasoning model, 04, I think it was. Yeah. And that's just for one generation over several minutes that someone's doing a particularly different coding task.

37:37It doesn't feel like these inference chain things will trickle down there. So maybe it will be the future of LLMs is this small industry run on these smaller chips or something like that. I think you're going to see all of the above thing. Let's say, for example, we're seeing inference happening inside of EVs where I'm doing rapid ingestion of video tokens for the purposes of deciding whether I'm going to back into my neighbor's trash can. And these kinds of things will - Are those large language models? Oh, absolutely. Because you can imagine I'm ingesting all of that. The video, I'm going to have a lot more granularity with respect to the tokens flowing back to me and I can do more with it and know more about what's happening in my environment.

38:14I think most of video ingestion is going to move towards edge tokens, but done on small language models. Very, very small stuff. We've already had edge compute AI vision stuff for like a decade or more. Like I was working on stuff like that in 2014. It just feels like a lot of this is trying to make tokens do stuff that we already kind of do. So it is, but the thing I, and again, you know, I'm deeply in the skeptical camp here. The thing I will say in favor of tokens in terms of absorbing a lot of this stuff is that it makes it all less ad hoc because you now have this sort of universal commodity for ingestion and production of information.

38:56That makes things a little more interesting because I can now abstract away some of the hard problems of video processing. I can abstract away some of the hard problems of speech synthesis because they all kind of disappear and become, in a sense, this universal token. And to a degree, I think that's true. And I think it will lower the barriers to more people playing in the worlds of video and speech and other places. But that doesn't create the kind of marketplace that people who are pushing huge numbers of tokens run on frontier models want. This is just edge stuff, cheap and dirty stuff. Also, the problem with tokens as a commodity as well is it's very hard to know, like a million tokens per million tokens.

39:34It's impossible to actually measure how many tokens a task will use because of the inherent unreliability of large language models. And it's like, so it's hard to even, like right now you're seeing with this, have you seen this people complaining about anthropics rate limits, for example? I'm not sure if you've seen that. I see it constantly. Well, right now, people are mad that they can only spend$1 ,000 on a$200 a month plan. But the thing with that is people very clearly do not know how far a token goes or a million tokens goes. Like, it's difficult to evaluate and measure that, which feels like kind of economic poison at some point.

40:13Because if you can't say how much a task will cost, you don't have a miles per gallon, like 16 ,000 tokens per second. wow, you can do inference fast. But if a customer can't afford it, if a customer can't actually reliably say, I'll be able to use this in this way, what uses tokens as a measurement? I mean, I know what they're used for as a measurement, but it's like, you can't say what even a million tokens might do. No, you can't. But that's kind of innate to the world of like a true commodity. I don't know how much you're going to put in your car. I don't know. It really, that becomes an engineering decision for you not a production decision for me right so yeah that makes that you have to separate those two pieces and so that that i don't can't tell you here's how many tokens it will take for you to do x that's no more my job than it is for me to tell us you know a copper mine how many how much copper it's going to take for gm in a particular car so you know how much yeah but they know how much i get what you mean it's like they they know how much copper they need to use but right and then and then they'll make constrained you know constraining decisions where they'll say, I only want to use this much copper because copper is really frigging expensive.

41:17And so, you know, we're going to like, I just saw, I think it was Rivian the other day said they'd cut out like, I don't know, like 10 miles of electric cable inside their cars, which seemed ridiculous to me. But it was again, because of the price of copper. So there's, yeah, so they turn into, these things turn into engineering decisions. Right now we're in this kind of subsidized wild, wild west where everyone thinks it's a land grab. And so they're subsidizing it to a degree and people are overusing tokens. Oh, I've had a model running for three days and it's doing all these agentic things.

41:44And it's like, well, what are you trying to create out of it? And it's like, I don't know, it's some nonsensical thing. So people are being subsidized to do non-economic behaviors to an incredible degree right now. And I find that remarkable, which is a statement about this kind of land grab mentality among the frontier model vendors. And Dario Amadei at Anthropic has been very upfront about this as he believes that we are in a land grab mode. There's only going to be a couple of frontier model vendors left standing. And so we need to make sure that we're the dominant provider, if you will, of coding harnesses and frontier models.

42:15No, I think that's a misnomer too, but that's another problem. Yeah. I think, but my core economic, I mean, one of the many core economic things, such as it's totally unprofitable. My thing right now is these rate limit changes are more severe on an economic level than people give credit for. That's totally true. Not just because of the economics, but because of the habits. Yes. If you believe, like the way I analogize it is like, if your car can drive 15 miles and then one day it can only drive three, can you get to work? Yeah. And is it because - Or even worse if you bought a house predicated on being able to drive the 15 miles to work, right?

42:48So it has externalities, it has outside consequences. Yeah. But the thing is, they've trained everyone in this land grab to act in a way that doesn't make sense long-term. And I don't think they can become profitable. I actually truly don't think that there's an economic way for it to happen. But they've trained people to use the product in a way that doesn't make sense. Like it's not even a, oh, they can charge more. Your habits are not built for this. That's exactly right. So I have a wild-eyed theory. Are you ready? Go, go, go. Hell yeah. So my theory is the first frontier model company to abandon frontier models wins.

43:26How do you mean? So my theory is that all of this stuff, most people in a Pepsi Coke challenge kind of way can't tell the difference. They claim they can, but they can't actually tell the difference between most frontier models for a typical test. Certainly normals can't. Coders claim they can, but if you actually do it in a blind way, most of them can't tell. This is just ego. And so increasingly what people see instead is these coding harnesses, these tools like CloudCode and Codex and OpenCode and all these sorts of things. So most of the value they see and most of what they actually think of as the model is just the harness.

43:55And that's where most of the innovation is happening. So my argument is, and that's why it was so dangerous for Anthropic this week when they accidentally did a whoops and it released CloudCode, most of the value is in the harness. And so the first company to say, you know what, we don't need to spend this kind of money anymore on training new models because we're going to just sit on top of models from all of these loons who are out there spending crazily on new frontier models that aren't improving very much anymore. Now, certainly not like they were four or five years ago. And they will be rewarded for that, no different than saying, you know, I've let go all of my employees or I've decided to stop spending money on hydroelectric dams.

44:28You've cut CapEx. You've made your business more financially appealing by taking away the single biggest piece of cost because you're recognizing that the world has changed, that I'm not getting incrementally as much value for a dollar on training a model as I was five years ago. And that's very clear in the data. If you look at any of the composite benchmark models, getting away from like, can they solve math problems, but literally real world composite models, it's been a sharp decline from like, you know, 18 % year over year improvements in models to like four or 5 % at vastly higher costs and more time.

45:00And so that really matters. And so my completely will never happen theory is that the winner here is the first one to stop doing it. Isn't that just describing cursor? So cursor is an interesting example. So Cursor doesn't actually embrace the full idea of being a harness across all of these white-collar applications. They're still kind of trapped in a coding world. Right. So I just think people get trapped in coding because it was the first place this stuff emerged. So you have to think about like, co-work is a good example of at least an attempt to break, again, escape containment and get out of the world of coding and say, okay, this is actually for all white-collar workers.

45:36Like I watch people struggle with cloud – pardon me? But it didn't work. It doesn't work, but it's at least it's directionally the right idea. If you buy my theory that all of this stuff is commoditizing so fast and is a loser's game financially, that maybe the right sort of game theoretic strategy is to be the first frontier model company to stop making frontier models. And in a weird way, Apple kind of showed the way, right? Because early on, they were getting pilloried for why isn't Apple spending more on AI? Why does Apple not have a model? And now, of course, it's reverse where it's like, look at Apple.

46:07They're so smart. Look how smart they are without their CapEx. I don't know. I just feel like what you were describing is just AI model wrapper companies. Yes. But my whole thing is, unless someone is able to break out of coding, there isn't really a hope for any of this. Because to this point, every time I read about an integration with a Goldman Sachs or somewhere, I can never actually find out what it does. I can never... The further you get into the reason... I have a good one for you. So I was talking to a very large investment bank the other day about their prodigious AI integration efforts.

46:43And so they built it out. They told me across equity research, sales, trading, and investment banking. And they asked me, which one do you think has seen the greatest benefit? I said, oh, God. I said, I used to work on the sell side. I said, my first instinct is none, that they're all lying to you. But my second instinct is I'll say equity research because, you know, no one likes to build spreadsheets and maybe it helps them build spreadsheets. And they said, no. And so the answer, of course, was investment banking. And I said, why investment banking? These are like knuckle-dragging dinosaur, arcade men.

47:12What are they doing? And he said, the answer, of course, is the main thing junior investment bankers do is build – well, they get shouted at. That's the main thing they do. But the second thing they do after being shouted at is they build pitch decks for companies that don't want them. So they build a pitch deck because a partner wants a pitch deck built for some rando company somewhere. And that's a pain in the ass. And so now that used to cause all these sleepless nights and blah, blah, blah. They're doing them all with AI. So junior investment bankers love AI because it lets them do this completely unproductive, largely inconsequential task of building pitch decks for companies that don't want the pitch deck.

47:47And so these are the kinds of applications that have really minimal economic value and yet sort of superficially appear really exciting. Because if I'm someone who otherwise had to stay up all weekend building a PowerPoint deck to pitch to some random small cap company, I'm like, yeah, this is terrific. So that was the answer. It was really interesting. But that's also kind of worrying because like the best example we have for this thing that has taken over everything, at least optically, even though it hasn't in economic terms, is like we can do PowerPoints kind of. We can do PowerPoints for junk bond raises for micro cap companies way better than we used to.

48:25Wow. And it's like helping junior analysts. So it's like, are you really, the time they're saving is just lowering their work days from 15 hours to 11. Well, and they're still being shouted at, unfortunately for them, but that's the way it goes. That's part of the job. That's part of the job. Paul, it's been such a pleasure having you. Where can people find you? PaulKodroski.com. Thank you for joining us. And yes, we'll be back with a monologue this week. I'm of course Ed Zitron. Thank you everyone for listening.

49:00Thank you for listening to Better Offline. The editor and composer of the Better Offline theme song is Matt Ossowski. You can check out more of his music and audio projects at You can email me at or visit to find more podcast links and of course my newsletter. I also really recommend you go to the chat.wheresyoured.at to visit the Discord, and go to r slash betteroffline to check out our Reddit. Thank you so much for listening. Better Offline is a production of Cool Zone Media. For more from Cool Zone Media, visit our website, coolzonemedia.com, or check us out on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

50:052%. That's the number of people who take the stairs when there is also an escalator available. I'm Michael Easter, and on my podcast, 2%, I break down the science of mental toughness, fitness, and building resilience in our strange modern world. Put yourself through some hardships and you will come out on the other side a happier, more fulfilled, healthier person. Listen to 2%. That's T-W-O-percent on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

50:50We got the Grits and Eggs podcast, Deontay Kyle and Big Ice Cup Cat. We got Club 520 with Jeff Teague and the gang. Don't call me white, girl. Mona will be there. Keep it positive, sweetie, with Crystal Renee. We got Reality with the King with Carlos King. And yes, Drink Champs will be in the building. Plus, you know, we're going to have a lot of guests. So you need to join us. And we got the Black Effect Marketplace, the picture podcast, and everything you expect from the Black Effect Podcast Festival. Tickets are on sale right now. Go get yours at blackeffect.com slash podcast festival. Don't play yourself, okay?

51:24Pull up.

51:29On the Serving Pancakes podcast, conversations about volleyball go beyond the court. Today we have a little best friend compatibility test. Okay, how long have we been best friends? Since the day we met. As the League One volleyball season heads towards its final stretch, there's no better time to tune in. You'll hear unfiltered analysis, behind the scenes stories, and conversations with leaders making an impact across the sport. Whether you're following the final push of love season or just love the game, Serving Pancakes brings you closer to the action and the people shaping the future of volleyball.

51:55Open your free iHeartRadio app, search Serving Pancakes and listen now. Presented by Capital One, founding partner of iHeart Women's Sports. Will Ferrell's Big Money Players and iHeart Podcast presents Soccer Moms. So I'm Leanne. Yeah. This is my best friend, Janet. Hey. And we have been joined at the hip since high school. Absolutely. A redacted amount of years later, we're still joined at the hip. Just a little bit bigger hips. This is a podcast. We're recording it as we tailgate our youth soccer games in the back of my Honda Odyssey. With all the snacks and drinks. Why did you get hard seltzer instead of beer?

52:28Oh, they had a BOGO. Well, then you got them. Listen to Soccer Moms on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. This is an iHeart Podcast. Guaranteed human.