Dylan Patel - The Infinite Demand for Tokens, Claude Mythos, and Supply Constraints - [Invest Like the Best, EP.469]

21:39Dylan Patel:Glasswig. Anthropic Earwig, where they only release mythos to certain companies for cyber, that's just going to be something that continues. Models will have less broad and less broad deployment. I know OpenAI and Anthropic and all these people are like, we want to have great AI for everyone. and AI is very fucking expensive. Who's going to pay for the trillion dollars of infrastructure? People who have money and we can build useful things with AI. And then you don't want people to distill your model so you don't release them broadly. You release them to fewer and fewer set of customers. Those customers are also now wrestling over the tokens unless Anthropic jacks them.

22:15Dylan Patel:They could double their pricing on Opus and I would continue to pay and I bet most users would continue to pay. I bet that wouldn't solve their humongous capacity problem that they have. So then the question becomes, where does this cycle end? where token usage and therefore the benefits of those tokens, the additional value generated on top of those tokens, aggregates among fewer and fewer and fewer companies. I don't have mythos. You know who has mythos? Top freaking banks. Now, they're only using it for cybersecurity, but at some point I can envision a world where, hey, maybe I, because I have an enterprise anthropic contract and because anthropic people kind of like me, they're willing to give us slightly earlier access or slightly higher rate limits or something for a model.

22:53Dylan Patel:I hope that's what happens. And then my competitor, whoever that is, doesn't have that. And I'm able to fucking crush them. There are people like Ken Griffin of Citadel is super well connected and super rich. He goes and signs a deal with OpenArenthropic that's like, yeah, I'm going to get access to your models and I'll buy the first$10 billion worth of tokens each year. So whenever you release the model, I'll spend the first 10 billion tokens and then everyone else can get the model after that. And it's like, okay, well now what does that do? Now he's going to crush everyone in the markets. That's just an example.

23:21Dylan Patel:It could be any number of things. It could be cyber like Anthropica is worried about, oh, now I can hack people. It can be information services business like myself where I crush someone else. I think it's such a broad base. We don't know what these models can do. Anthropica doesn't know what these models can do. No one knows what these models can do. It's up to the end user to figure out where they can leverage the tokens to see what they can build and imagine, which is tremendously productive and uplifting for humanity. But then what happens to the concentration of resources and usage of it?

23:45Presumably right now, robotics or robots consume relatively zero tokens versus everything else. What's your view of that? if that's like a second demand curve that could start to ratchet, there's a new startup every single day within a mile of here trying to build something interesting in robotics.

24:02Dylan Patel:So there's this concept of software-only singularity, which is that the world has AI singularity, but only in software. And now what about the rest of the world? The vast majority of the world is physical. You can see the world orient around hardware, not software. That's actually why I think software-only singularity is like just a blip and not like we do get everything else. Because once software is super easy, what makes robots really hard? It's programming microcontrollers and actuators and controlling all this stuff is very difficult. And right now, the interesting thing about models, AI models is they're actually really inefficient in learning.

24:38Dylan Patel:It's just we're able to give them so much data that they're able to learn and pass us in certain ways. Currently, the robot models, VLA's, Vision Language Action Models, which is very popular right now, is probably not going to be the thing that ultimately scales beyond. They're inefficient in data, and we can't scale the data for them fast enough. There is going to be some way to large-scale pre-trained robot models where, just like humans, see all this data throughout their lives. And what's interesting is humans, the reason why we're so good is we're sample efficient. One example, two example, we're good.

25:10Dylan Patel:So applying that to robotics. So once you have this software-only singularity, implementation is super cheap. Like anyone can start to build these models that now robots are actually useful. And so I think in the next six to 18 months, we'll start seeing real breakthroughs in robotics that enable few-shot learning, i.e. there's a pre-trained robot model, and now there's a robot that you have hired or bought or whatever. You showed a few examples and it's able to do it. Right now, you know, there's a lot of companies doing robots for advertisement or robots for simple stuff like that, but it'll be like, oh, folding clothes, sure, sure, sure, no, but it's gonna get really niche.

25:43Dylan Patel:robots just for cleaning chalkboards and it's a rental service or it'll be a model package that you download onto your standard robot that then does that. And anyways, there'll be a huge explosion in physical good acceleration and deflationary effects there. But that's ultimately going to keep token demand going crazy. I don't think token demand slows down personally. Did you learn anything else about the world based on Mythos' results and how it was built? It's my way of asking if you break down the components of the scaling laws. Mythos is a materially larger model than prior models. And so, yes, it is a much larger model.

26:18Dylan Patel:What chip it's trained on is not really relevant. It's the scale. And obviously, to 100 ,000 blackwells is a coolant to hundreds of thousands of prior generation chips. TPUs and Tranium have their different release cadence, so it's not exactly like mirrored one to one. But ultimately, yes, Mythos is a significantly larger model. It's proof that the scaling laws still work. Everything about it shows the trend line continues of models. more compute into model makes model better. And along the whole way, it's not just more compute into model makes model better. Along the whole way, we're also getting these compute efficiency wins, which are as all this research compute that the labs are spending is actually turning into if I want X capability to your model, every six months that cost or every two months that cost is dramatically decreasing.

26:59Dylan Patel:But then if I scale it up massively, I get a humongous capability drop as well. And so yes, it's proof that this is still happening. Google and Anthropic are not heavy, heavy users of GPUs on the training side. But OpenAI, they'll start having their new class of models. I think they're taking a more sensible, principled approach to scaling in small steps. Anthropic really went for a huge jump. We'll see better and better models throughout the year, and the release cadence is only going to get faster. We've gone a long way in the conversation with saying almost nothing about OpenAI, which would have been so strange 12 months ago.

27:30Dylan Patel:So this is the interesting thing. Everyone's like, okay, so Anthropic's just won, right? They had Mythos in February. They never even released it because they didn't feel the need to. They're already sold out. Their revenue is already adding$10 billion a month. And then you've got Opus 4.7 today, all before OpenAI's alleged spud release, which media such as The Information and others have posted about. So clearly Anthropic is in the lead and OpenAI is cooked. What's interesting is because Anthropic has such bounds on compute and they can only grow it so fast and to the point of, Dario used to gloat about how OpenAI was being too aggressive on compute and Anthropic was more sensible in their scaling.

28:07Dylan Patel:And now Anthropic is like, fuck, I wish we had a lot more compute. OpenAI is able to pay the bills perfectly fine. In fact, they've raised a ton of money to get incremental compute in addition to the irresponsible levels of compute that they were buying from Oracle and CoreWeb and SoftBank and all these people and Microsoft, such as Tranium. Now they're getting Tranium as well from Amazon. They've done this insane thing on compute. They also know they need more. But what's interesting is, if you were to say Opus 4-6, let's ignore models getting better over time. Let's just take diffusion of this technology.

28:36Dylan Patel:You and I may jump on the model immediately day one, but other businesses take time and it takes time for people to learn. And the spark of, oh shit, quad psychosis moment doesn't hit everyone at the same time. And so by the end of the year, let's say a 4-6 Opus tier model, the economy would spend a hundred billion dollars on. I don't think that's unreasonable. It's spending 40 billion right now. That's like a linear extrapolation. It's a linear extrapolation, not an exponential. To get the exponential, you need the better models. Anthropic won't have enough compute to do that. And presumably OpenAI and Google will hit that tier soon enough.

29:10Dylan Patel:Whoever hits that tier next, sure, Anthropic may get to charge 70 plus percent gross margins. But if OpenAI hits it next, they charge 50 % gross margins. They still get all of this incremental demand. And probably they also won't have enough compute to serve all the users. Sure, maybe Mythos is a model where if the world had enough compute, it'd be$500 billion of revenue or something crazy. There is such demand for these tokens and such limitations on compute. We see this with H100 prices skyrocketing and all these other things. The useful life of these GPUs continue to extend. It's pretty clear even the tier two lab is gonna be sold out of tokens, let alone the tier one lab.

29:46Dylan Patel:The tier one lab will have better margins, but the tier two lab will be sold out and probably the tier three lab will also be close to sold out. The economic value that the best model can deliver is growing faster than our ability to actually serve those tokens to people via the infrastructure. And so this gap will continue to grow and the model labs will continue to have expanding margins until people in the hardware supply chain, infrastructure supply chain are like, wait, no, why don't I just jack up my margins? So suffice to say, I think the assessment today or your assessment of the demand side is completely explosive in your own particular example here at SemiAnalysis.

30:16but just more broadly that you call it AI psychosis as people fall into this experience of what they can do, the implementation difficulty going completely away. I've certainly felt that. My own token spend is just through the absolute roof just in a matter of weeks. So that feels like a pretty good assessment. Anything we're missing on the demand side?

30:34Dylan Patel:If you don't use more tokens, you'll never escape the permanent underclass. Either you use more tokens and you generate economic value, outsize economic value for the use of those tokens. A lot of people are doing it the boring, lazy way. oh, I guess I'll just work one hour a day instead of eight hours a day and I'll have AI do most of my job. That's the boring way. The cool way is I'll still work eight hours a day and I'll do 8x the work and maybe I'll make 5x the money. You can't do this with a job, obviously. There's people who have multiple jobs and there's people who start companies and start selling stuff.

31:02Dylan Patel:There's people who are hustling, which is what I view like you and I is doing is we're mostly hustling. Get that economic value on this AI before everyone is using it and it's table stakes because it's still not table stakes. So if you don't use more tokens and generate the value from them and capture that value. There's three different problems here. Using more tokens, generating value from those tokens and capturing value from the value that you created from the tokens. If you don't do these three things, you'll never escape the permanent underclass, i.e. as models continue to skyrocket capability and the concentration of resources potentially happens.

31:33All right, let's talk about supply. What is changing at the frontier of supplying the entire stack that's required to serve all these tokens as the demand curve explodes?

31:42Dylan Patel:As demand skyrockets, Prices are going up for everything on the supply side, whether it be the end GPUs, their prices are going up. In addition, their useful life is extending. H100 prices look like this. Yeah, exactly. There's people who have argued GPU useful lives are less than five years. Complete nonsense. There are clusters now re-signing. Three or four-year-old Hopper clusters re-signing for three or four more years. There's A100 clusters that are re-signing for another couple of years. So the useful life is clearly not five years. It's maybe even seven or eight years, arguably. We don't know yet.

32:13Dylan Patel:We'll see when Hopper gets there, but it's clearly not five years. So useful life is extending and the prices are going up on that renewal. In effect, the gross margin was not 35 % on a cluster. It's beyond that. So margins are expanding in the cloud layer. Margins are extremely healthy on the hardware layer with NVIDIA still charging 75 % or whatever percent gross margin. As we move down the stack memory, obviously margins have skyrocketed there. Places like Optics and Logic, there are large prepayments and margins are growing slowly. More so, the companies that are making chips like NVIDIA are paying huge prepayments.

32:48Dylan Patel:So in effect, the cast of capital or timing of cash flow, the return on invested capital is going up, even if the gross margin isn't. And you see this across the whole supply chain. You see ASML is completely sold out and they need Carl Zeiss to expand faster. Everyone's either sold out and margins are going up or they're getting prepayments, which increases the return on invested capital because the invested capital is lower. And so this is a consistent trend across any part. It's even like to make a PCB requires copper foil. And that copper foil is sold out and people are making prepayments for it.

33:17Dylan Patel:Anything and everything that has a pulse and is sold out, people are jumping to get more incremental supply and fighting over the supply for the years after. What do you think are the most important bottlenecks? Typically in economic history, when there's this kind of demand, supply reorients and rises very, very quickly. to meet the demand. It seems like it's almost impossible for supply right now in this moment to keep up. Famous last words, every shortage is followed by a glut historically. But what are the most interesting bottlenecks to you across the supply side? Supply chains are usually very fast to react.

33:52Dylan Patel:One unique thing is that our supply chains now are more complex than ever, and the things we're building are more complex than ever, and therefore the lead times are longer. And it's not like we haven't seen 18-month-long lead times in other industries. It's just building incremental supply didn't take years. And this is the case with memory. Memory can only grow capacity, low double-digit percentages a year, right? 20, 30 % a year, even less for NAND, a little bit higher for DRAM, but whatever. Even though the demand signal was very strong at the end of 2025, the memory companies immediately started reacting.

34:23Dylan Patel:None of that incremental capacity really gets here until the second that they've decided to do in addition to the typical 20 to 30%. They can stretch a little bit, but really the true incremental supply doesn't come till 28, which is a very unique thing. Even if they wanted to build as fast as possible, it doesn't come till 28, late 27 at best. So the result is memory prices have gone through the roof. And guess what? They're going to double and triple again, at least on DRAM, especially. People are like, oh, the memory story is overplayed, everyone gets it. It's like, no, no, no, you don't get it.

34:53Dylan Patel:DRAM will double or triple from here still because that's how much capacity is required and they have to steal capacity from somewhere else. And the only way to steal capacity from somewhere else in a capitalist economy is demand destruction via higher pricing. We're not rationing stuff here. And so ultimately that's what's gonna happen. And so margins continue to go up. I think Logic also has humongous capacity problems. TSMC just had their earnings. They keep upping CapEx. Ultimately it takes them quite some time to build fabs. And they're trying to do everything they can to squeeze every little output out of every fab that they have.

35:24Dylan Patel:But ultimately, they're not raising prices fast because they're good people, it seems like. Single-digit price increases instead of triple-digit price increases like the memory guys have had. So you ultimately have this market where, yeah, TSMC is a great company, but are they actually going to extract all the value? I mentioned things like copper foil, glass fibers for PCBs, lasers. These are things that are well understood in niche supply chains, but they're very, very tight. And ultimately, upstream, the semiconductor wafer fabrication equipment supply chain is one that's gone up a lot. but it's still very underappreciated.

35:56Dylan Patel:TSMC, CapEx this year, they say 56. We've had 57.4 billion since January, and we may up it slightly more just because we see some ways that they can get incremental CapEx. But what people aren't focusing on is what does that mean next year? And what does that mean the year after? And it turns out three years from now, TSMC is going to spend$100 billion on CapEx. Maybe two years from now, it might be 28. Sincerely, they may spend $100 billion on CapEx in 2028. And people just can't fathom that. But what does that mean for their downstream supply chains. Companies like Lamb Research or Applied Materials or ASML, or their further downstream supply chains like MKSI and all these other companies, the tail whip, it just gets whipped harder and harder and harder.

36:36Dylan Patel:And that's a shortage if TSMC wants to spend$100 billion in 2028, which is a real possibility. I think people would think that's insane, but that's a real, real possibility. What about other parts of the chip ecosystem where GPUs have been completely dominant? What about like CPUs or ASICs or things that start to pop out as both opportunities and bottlenecks beyond just nvidia's gpu dominance yeah i mean asics are obviously taking off but i'll pivot away from ai chips to talk about these other things there's a project we did on fpgas and it turns out there's 120 fpgas per next generation rack ai rack and then what about all the fpga names cpu eyes all these reinforcement learning environments plus all the slop code you and i are generating that is now running on some vercell instance or whatever it is or some AWS Instant or some bucket that we've spun up, all of that requires CPU.

37:24Dylan Patel:And so CPUs are completely sold out and demand is skyrocketing there. Do it help people understand the role that CPU plays and everything? There's two main reasons why you need tons of CPUs. One is when you're doing reinforcement learning, the CPU is very critical to that. So before you would throw all the internet's data into the model, train it, and it spits some stuff out. Now you train all the world's internet, you put all the internet data into the model, then you put it in this environment. This environment is like, hey, model, try this out and it tries stuff out tries a bunch of different things and in the end there is an environment which scores whether or not what it tried out is successful and it grades it and these environments can be anything it can be hey check if the text was output in the right way structured outputs it can be very simple stuff it can be very complex stuff and people are starting to get into very complex things like hey i want you to open this file change it edit it update it submit it to this website.

38:17Dylan Patel:I want you to open up this physics simulation from Siemens and edit this CAD model. So the environments can get more and more complex and those environments run on CPUs. They don't run on GPUs. They don't run on ASICs. The ASICs run the model that takes the input data from the environment, runs it through the model. The model creates outputs of various different trajectories, ways that I think it could solve it in different instances. Those trajectories are graded slash scored, and the ones that are successful, you train on and you update and you iterate, iterate, iterate. And so CPUs are very useful for that one.

38:49Dylan Patel:And then once you have these great models and you're deploying them, those models are generating code. They're generating useful output. That useful output, it doesn't go from a GPU straight to the human brain. It goes from a GPU or an ASIC through to a deployed app that you're deploying somewhere that actually just runs on CPUs. So that's another area where there's a lot of demand and things are sold out in a large, large way. as you continue to. Your finance team isn't losing money on big mistakes. It's leaking through a thousand tiny decisions nobody's watching. Ramp puts guardrails on spending before it happens.

39:23Real-time limits, automatic rules, zero firefighting. Try it at ramp.com slash invest. As your business grows, Vanta scales with you, automating compliance and giving you a single source of truth for security and risk. Learn more at vanta.com slash invest. Every investment firm is unique and generic AI doesn't understand your process. Rogo does. It's an AI platform built specifically for Wall Street, connected to your data, understanding your process and producing real outputs. Check them out at rogo.ai slash invest. The best AI and software companies from OpenAI to Cursor to Perplexity use WorkOS to become enterprise ready overnight, not in months.

40:01Visit workos.com to skip the unglamorous infrastructure work and focus on your product. Ridgeline is redefining asset management technology as a true partner, not just a software vendor. They've helped firms 5x in scale, enabling faster growth, smarter operations, and a competitive edge. Visit ridgelineapps.com to see what they can unlock for your firm. To assess and try to be the world's best informed person on both the trajectory of supply and demand, what are things that you wish you knew to make that understanding that you don't know?

40:34Dylan Patel:I think the hardest area for us and for everyone is understanding tokenomics, economics of tokens. I think we have a really tremendously good insight into how much it costs to run infrastructure, what the cost of tokens are, what the cost of models are, what the margins of these labs are. But the usage and adoption is what's really difficult to model. Continuously, right? January, we had crazy estimates for February. Anthropics smashed them. How do we calibrate this model? What are the data sources for this? February, we had crazy assumptions for March. I know people were like, you're crazy, Dylan.

41:06Dylan Patel:And then they smashed up. Everyone sees the number of 10 billion and they're like, what the f - how do they add 10 billion of revenue? Who's using all these tokens? Why are they using them? What are they building with them? And then more importantly, with what they're building with these tokens, how is that actually diffusing into the economy and what value is that generating? Because it's not really something that you can capture in any GDP statistic. All of the value of the tokens that I use get transformed into better information, which I then sell at a discount to what people used to sell information for relatively.

41:35Dylan Patel:And therefore, that information is now making its way throughout the economy and people are making better investment decisions or better competitive decisions if they're a semi-care company or a data center company or hyperscaler. What is the value of this? What has that done to the economy? It's clearly by every subjective metric amazing. But where is the phantom GDP? What is the phantom GDP? How do we track the real economic value? Because the GDP metrics are not accurate if you were to say, what is the GDP that Dylan Patel is making? It's tiny compared to the value that I think is being created.

42:08Dylan Patel:And I think you would say the same for Patrick. What is the value being created by these tokens? Not on a basis of simple, what is the knock-on effect of all the things that these things are doing? And I think that's the real question and challenge that's hard to measure. I think we've got a tremendous reading on the supply side of things. I think we've got a tremendous reading on even a lot of the demand side signals, but it's what is the value these tokens are generating? That's hard to quantify and measure. I hope we get a chance to do this every three months because this changes so quickly.

42:36What do you think is going to happen next? When I come back three months from now and we're in San Francisco together again, what do you expect? Large scale protests. Really?

42:44Dylan Patel:Yeah, I think there will be a large scale protest against Anthropic and up at AI. People hate AI. AI is less popular than ICE, less popular than politicians. With Anthropic adding so much revenue, that's going to start causing business changes downstream. People are going to get more and more scared of AI. They'll start blaming more and more of their own problems and things that are global, have been deep-seated problems for a long time. Those will bubble up and be blamed on AI. Probably some politician or some influencer will be able to start taking and weaponizing AI against people. You look at the comments of news articles where Sam Altman had a Molotov cocktail thrown at his house twice in two weeks.

43:23Dylan Patel:People are cheering it on. And this is just the beginning. So I think we'll see large-scale protests against AI in three months. What is the counterweight to that? How should the AI industry head that off? First of all, Sam Altman and Dario have to stop getting on interviews. They're so uncharismatic. I don't know what they're doing. Every interview they do is like, wow, normal people are going to hate you even more. Sam being on Tucker Carlson probably made all Republicans hate open AI. I'm just guessing. Same with Dario. I think that's first. Two, they need to start showing uplifting things that can be done with AI.

43:56Dylan Patel:Three, they need to stop talking about how the capabilities are going to change the whole world constantly because then people are going to get fear of that capability because they have no connection. They don't know how to use it, yeah. There's no connection to it either. The average person doesn't know an Anthropic employee. The average person doesn't know an OpenAI employee. The average person doesn't know who these people are, what their goals are, and they just view them as a sneaky cabal of 5 ,000 people at this company that are going to change the world and automate all the jobs and destroy society.

44:21Dylan Patel:That's what they view it as. And as people who are funding the building of all these data centers and power plants that are going to pollute the world, they don't quite understand what's happening. So they have to stop talking about the future thing that's going to happen and only talk about present, how uplifting AI is. I think it's a huge reorg and rebranding that needs to be done. This is so much fun. I love doing this with you. Thanks for your time. Awesome, thanks. If you enjoyed this episode, visit Colossus.com. You'll find every episode of this podcast complete with hand edited transcripts.

44:50You can also subscribe to Colossus, our quarterly print, digital, and private audio publication featuring in-depth profiles of the founders, investors, and companies that we admire most. Learn more at colossus.com slash subscribe.

45:48Thank you. Ridgeline is redefining asset management technology as a true partner, not just a software vendor. They've helped firms 5x in scale, enabling faster growth, smarter operations, and a competitive edge. Visit ridgelineapps.com to see what they can unlock for your firm. Every investment firm is unique, and generic AI doesn't understand your process. Rogo does. It's an AI platform built specifically for Wall Street, connected to your data, understanding your process, and producing real outputs. Check them out at rogo.ai slash invest. The best AI and software companies from OpenAI to Cursor to Perplexity use WorkOS to become enterprise-ready overnight, not in months.

46:27Visit WorkOS.com to skip the unglamorous infrastructure work and focus on your product.