What NVIDIA’s bet on OpenClaw means for the future of AI and your token budget

0:00Today, I want to talk about what Jensen Huang, the boss of NVIDIA, said at NVIDIA's GTC Jamboree, where the firm lays out its view of where we are and where we are going. Now, I want to focus on one particular dimension, and it may not have been obvious if you had watched any of the videos from GTC or read any of the reporting that came out of it. It's a dimension that will accelerate all the things that we are being promised about the future. And that dimension is AI inference. Inference is when an AI model responds to us, ideally with what we want it to respond to, whether it's a recommendation for a lawnmower summarizing your boss's emails so you don't have to read every interminable word, or whether it's getting an image for a presentation you have to give.

0:52All of that arises because of inference. I've been peeling back the layers of what Jensen Huang said to figure out what it's really telling us about where AI is today and where it's heading. So do stick with me because I think the shape of the future is really starting to become visible.

1:20NVIDIA is the world's most valuable company. It's a$4 trillion behemoth. that value changes with the winds and the waves and the ends and the oars of the market. It is the leader in providing AI accelerator chips, the GPUs upon which virtually all of our AI interactions reside. Now, like me, Jensen has seen something in OpenClaw, that piece of open source software that lets you run an AI agent that's as close to the AI agents of 1980s science fiction, as you can imagine. Now, a couple of weeks ago, I said that OpenClaw is the most exciting piece of technology that I have seen since the web browser back in 1992.

2:04Now, I really mean that. I remember really clearly using it back in that summer of 92. I had to go to a computer room that was in the basement. It was a VDU display, so green screen. There were about a dozen, maybe 20 websites at the time in the mostly university physics department. And if you were using the internet, the action was elsewhere. It wasn't on the web. It wasn't through the web browser back then. But it was really clear that the browser was offering a new way of interacting with the internet. And just in that one year from 92 to 93, the number of websites in the world expanded by a factor of 50 or 100.

2:44And just keep that in mind, like 100x back in 1992 to 1993. And in a sense, my whole career has been defined by that experience, by falling into the internet a year earlier and then making sense of the web browser. So when Jensen brought up that parallel, the most important thing since the web browser, of course, it really spoke to me. He's had the same epiphany that I have, or perhaps I've had the same one that he has had. But he went on to say that every company now needs an open claw strategy. And I think that that is the key that unlocks how to make sense of a lot of what was said by NVIDIA earlier this week.

3:28Now, look, it's completely wild because before January the 30th, nobody had heard of OpenClaw. I mean, it wasn't even called OpenClaw. But even on January the 20th, 10 days before it got renamed, the project on GitHub, which is where open source projects live, had about 5 ,000 stars. Now, that's not a nothing burger. It's impressive, but it's not radical. It's not top of the leaderboard, which is where OpenClaw now sits. And it is wild. It is crazy. It is unprecedented. I am running out of words that by March the 16th, 45 days later, the CEO of the world's most valuable firm, the firm powering roughly 90 % of all AI compute, said every company needs an open claw strategy.

4:20Now, of course, he is largely right. And I'll talk to you about our open claw strategy a little bit later, so hang on in. But to give you some sense of that, most of the team in Exponential View now has an open claw agent. I've talked a lot about my AI chief of staff, the agent R-mini Arnold. Now, these open claw agents, they need somewhere to work and they work on new Mac hardware that we've bought for them. We've actually increased the compute the company owns by about 50 % in a month. And if you look at the amount of RAM that Exponential View has on its balance sheets, the amount of memory, roughly the AI agents have the same amount of RAM dedicated to them as the humans working in the company.

5:09We've built shared repositories for skills and tools for the agents, and we've got dedicated services that they can call, for example, if they need to run a simulation to figure out whether an argument is strong enough. The investment that we're making, it's making real something that we've argued for a long time, that the demand for intelligence is essentially infinite. Any company, any person trying to solve problems doesn't really get to a point where you have too much intelligence in the room. I mean, in our day-to-day, quotidian, if I'm brushing my teeth, maybe I don't need an enormous amount.

5:47But certainly in my work, in trying to solve knotty problems around my life, you really do want to sometimes be able to leverage lots of intelligence. Now, if the demand for intelligence is essentially infinite, that means the demand for compute, which is what produces manufactured intelligence, that thing that we call artificial intelligence, well, that's also effectively infinite. And we're experiencing this within exponential view, that the more we use, the more we need to use, because it opens new horizons, new vistas, new avenues and new opportunities. And no one is experiencing this more.

6:24No firm is experiencing this more than NVIDIA. Last year, the CFO said we've got about$500 billion of committed orders. This is effectively promised revenue. It's orders that investors and the markets can see. And it's a really good measure of the health of a company like NVIDIA. right? Semiconductors are highly cyclical businesses, they're prone to booms and they're prone to busts and being able to see an order book out like that is a real sign of health. Well this year we learnt at GTC that the scale of the committed orders was now a trillion dollars for Blackwell's, for the new Vera Rubin products just out through to 2027.

7:13And the thing to understand is, yes, NVIDIA maintains a dominant market share, but it's not 100 % of the market. There's also competition and supply from Google's TPUs, from Amazon's Tranium, from AMD, and from others in the wings. But there is a trillion dollars of orders, at least, and certainly more than that, every dollar going on chips to build manufactured intelligence. And those chips are more power efficient and they're more powerful, delivering more for each dollar than the dollars spent last year and the dollars spent the year before that. So hold that thought for a moment. The reason that's happening is because the shape of AI is changing.

7:59The way AI is being used is shifting. When ChackGPT burst on the scene, the bulk of usage of compute for AI workloads was in what's known as training. Training, building these large language models that we would then go off and use in the inference phase. And these large language models really did require large amounts of compute. Astronomical is not an exaggeration, 10 to the 23, 10 to the 24, floating point operations or more to train. And data centres were being consumed by the training rather than the inference stage of the process, which is where, of course, people like you and I actually end up using these models.

8:40But that is changing. Now, to understand that, let's talk about how manufactured intelligence makes its way out there. The atomic unit of manufactured intelligence of AI is the token. Now, there are lots of limitations in the framing I'm about to give you, but this is a really helpful one to take away. Think of it this way. The token is effectively the unit of AI output. It's a little bit of something that you can do something useful with. A token is roughly three quarters of a word. So if you get a 500 word email summarized into 100 words, that back and forth is probably a couple of thousand tokens.

9:21And as we get more advanced, we move from back and forth models, we move from summarizing emails to doing much more sophisticated things like, please analyze these 10 vendor proposals and find what's common, what's different and why I should prefer one over another. I mean, that's going to be tens of thousands of tokens. When you move to the reasoning models, usage increases really, really significantly. And when you move to agentic systems, it moves even further. Now, I discussed this a couple of weeks ago in an essay called Magnitudes of Intelligence. So you can go back to the archive on Exponential View and find that essay, I made the point there that our mini Arnold, which is my open claw agent, the AI chief of staff, had just consumed 100 million tokens in a single day.

10:10And that was my average running. And when I look back to the summer of 2024, so less than two years ago, I was at about maybe 100 to 150 ,000 tokens a day. So that's a three order of magnitude increase in less than two years. So the shift to reasoning models, that so-called test time compute, which started with OpenAI's O1, was really the start of the increase in the amount of tokens that we as end users would use. It changed the compute workload. What Jensen said in the last week is that as people move to reasoning models and using reasoning much, much more, they saw a 10 ,000 fold in the increase in compute demand from each user but at the same time usage increased a hundred times so that was a million fold expansion in compute demand in just two years and again hold that idea in your head that's a million x in two years what other market could go out and serve that so the question to think about is what does the next two years look like Does it look faster or does it look slower?

11:19The trillion dollar backlog gives us the answer, tells us what NVIDIA's customers actually think. They think at the very minimum it's going to be the same, quite likely more than that. So in this changing market, as we move from a world where the compute is dominated by training and move to a world where there's a lot of inference, things do change. And at the end of 2025, NVIDIA acquired a company called Groq, G-R-O-Q. It was founded by Jonathan Ross, who was the original designer of Google's Tensor Processing Units, which is Google's specialist chips for serving AI workloads. He'd done that roughly a decade ago, maybe a bit longer.

12:00And it was a big, slightly weird acquisition of about$20 billion. People moved and IP was licensed that the company wasn't formally acquired. It's kind of complicated. But that Grok acquisition really, really pointed to the changing shape of the AI market. Up until that point, NVIDIA had survived on a single, albeit evolving, architecture. That is the GPU, the graphics processing unit. It was its heritage coming out of video games. And GPUs are great at many things, but it had been coming clear over the last year or so that they might not be fantastic for the changing shape of AI use as we move towards inference.

12:41So now, this is a technical bit of my discussion. So when you think about what happens in inference, I think it's worth just unpicking this because it'll explain what's going on. There are a couple of phases. The first phase is called pre-fill. That's when you send a prompt to a model, whether it's a question or a document to summarize or some complex instruction. The model reads and processes your input token simultaneously in parallel. This is enormously compute intensive and it is where GPUs shine. They were built for graphics, throwing huge matrices of pixels at thousands of cores at once, doing it all in parallel.

13:17And that is the shape of the pre-filled problem. So when you're feeding context in, GPUs are doing what they were made to do. But the second phase of inference is called decode and that is the generation of the responses. And you know exactly what that is like because you have used these tools word after word after word sequentially at a time like a slow teletype from the old days. The model is producing tokens one at a time each depending on the previous one. This can't be parallelized it's structurally sequential. The bottleneck here now is no longer raw compute it's memory bandwidth. How fast can you stream the model's weights from memory for each individual token step.

13:59So now you have lots of GPU cores sitting largely idle waiting on memory reads just to produce one token at a time. So GPUs are not fantastic here. They weren't designed for this. And as we move from the training era to the inference era, well, the workflows are shifting from building models to running them constantly at scale for billions of users and lots of agents. that efficiency becomes a serious problem. That's why you acquire Grok, do whatever deal you did with Grok. It's a fast move, a high conviction move by an incumbent to ensure it can serve that changing market. A quick note, if you want to support us in bringing more of these conversations to the world, please consider subscribing to the show.

14:44So NVIDIA is going to make some new chips or systems with Grok technology embedded later this year. The point is that that combined architecture uses NVIDIA's homegrown Vero Rubin GPUs and GROC processing units, and it will result in a 35-fold improvement in the throughput per megawatt of power versus NVIDIA's current Blackwells, which are the posh chips of the moment. This isn't the first time NVIDIA has done something like this. They acquired Mellanox, it was a new networking company, and it turned into a real advantage for NVIDIA. And it's important to understand that NVIDIA is more than just a chip company.

15:20It's actually a systems company and it sells systems, not just chips, platforms, not just processors. But the inference market is really different to the training market and that's really different to the graphics market that NVIDIA came from. And I think it's really impressive to see the firm figure this out, not just figure it out, frankly, actually deliver on it at the scale at which they have to deliver. We'd figured out, we'd thought like NVIDIA, like lots of other people that inference was going to be an important part of the market, that workload volumes were going to shift and ratios were going to shift.

15:53And I've personally invested in a couple of startups that build chips to tackle this inference opportunity through much higher throughput or much lower energy usage. It was clearly going to be important. And I think that what we've seen with NVIDIA is almost going to be a case study of how a large firm can move really quickly with conviction, which is not something firms can normally do. At this point, let's go back to what we're doing with all that inference. What am I doing with it? What are you doing with it? What are your colleagues doing with it? And this takes us back to Jensen's observation that every company needs an open claw strategy.

16:30So I have to go back a layer still. What's driving all this growth. Well, it's really happened in the last few months where agentic systems, I think Claude Code, the coding tool that Anthropic has built, was a good example of this, started to work well, which means that you could effectively leave them to their own devices. And I think that that shift and OpenClaw also symbolises it, helps us understand the distinction between breakthrough research and development and the things you need to diffuse technologies into the market. The debate that's gone on about the AI investment wave and AI in general, I think, has confused this.

17:10It's often all about whether the biggest frontier models are hitting a wall. You know, the boosters and actually also the more sober minded, calm people in the foundation model lab say they're not close to a wall. And the evidence does suggest they're probably right there. The doomers or the sceptics say there's obviously a wall and this thing can't go on or it's already stopped. but both of those miss the point because what we're doing there is just looking at a performance curve the innovation curve and you're arguing especially if you're a doomer that if the innovation curve can't be guaranteed to continue the whole of this shebang is just a house of cards it's a glass house in which you shouldn't throw stones but what it does is it misses the importance of the systems that you build around a technology to make it useful.

18:01And these are the things that drive diffusion. They're the things that make technologies helpful. That real inflection point, I think, happened at the tail end of 2025, which was the maturity of clawed code. It's almost independent of the model that sits below it. It's a harness that helps you get work done. That harness is traditional code. It's also the rules around it. Sometimes traditional programming, sometimes specifications. And it's what helps that AI system do so well in the particular context of coding, or as we've discovered at Exponential View, lots of other things. Maria, for example, uses clawed code when she's editing essays, which is a kind of a novel use for me.

18:43But that is the second curve, right? The first curve is a performance improvement curve. The second curve is the diffusion curve, the harnesses, the things that sit around a piece of technology that make it more applicable and make it more useful. Well, I'm going to treat you to a classic Azeem analogy here, and I want you to think about the internal combustion engine. Now, if you're a smart engineer, a mechanical engineer, and you get an internal combustion engine, it's useful to you. You can do something with it. You can put some diesel into it or some gas, and maybe you can connect it up to a generator and produce some electricity.

19:24Or you could figure out how to connect it to a rotating blade and build a hedge trimmer. It's brilliant and well done. And I'm so glad you did mechanical engineering. But the rest of us can't do that. What really gets internal combustion engines to take off is when you put them in a harness, a harness that makes that technology a useful product. Now, historically, we've called the harness for an internal combustion engine an automobile or a car. When you put a combustion engine in a car, in a harness, it becomes useful. Let's call it open car. And now people will buy lots of them and they'll use them a lot.

20:01This is the full product solution. And you've got to remember that research companies don't start with the full product. Doing the research is hard enough. They start with the technologies. and those technologies are not that easy to use. And ultimately they or business partners figure out how to productize them. And that harness is the productization. And it's the harness that makes these AI systems so incredibly useful. But you could look at an agentic harness and I've seen this on X when people have looked at OpenClaw or the zillion of clones like PicoClaw and ZeroClaw that are out there. And they say, well, there's just not much in there.

20:41It's just a way of scheduling your AI system to run different prompts at different times and to look at the right files on your file system. It's a scheduler. It's a set of consistent rules. I mean, that is a lot of what something like OpenClaw does. But equally, you could look at a car and say, well, so what? It's kind of an increasingly cheap metal frame and there are some cheap seats in it and there's a dodgy hi-fi sitting there. That's not very clever, but it's the harness that makes us use it. and it's a harness that makes us use them more and more and more. Would you rather have a race-tuned Ferrari engine sitting on your lawn at home or a 10-year-old petrol engine, gas engine, actually in a car that you could go off and drive and do something useful with?

21:26And it's a harness that makes these things valuable. So I'll talk to you a little bit about how the open-floor harness has completely transformed my own personal experience. So on this show a few weeks ago, in one of the essays, I talked about how our Mini Arnold, which is my AI chief of staff, it is an open-claw agent, has been helping me with more and more of the things that I want to get done. And I talked about how it had got to 100 million tokens per day. And that was really the average through the earlier part of March. And to give you a sense of what 100 million tokens is, you know, if you use chat GPT a few times a week and you do a bit of document summarization, you're probably at the tens of thousands of tokens per day.

22:13If you are doing some software development, you might be a couple of million a day if you're doing this regularly. So 100 million is sort of at a different level. But since then, and I think it's been two and a half weeks since I told you about 100 million tokens a day, I bumped up a little bit. My record day working with R-Mini Arnold, 870 million tokens consumed and applied to business problems that I'm dealing with. That's close enough to a billion tokens per day. I mean, there was lots of other workloads being done on other systems where we don't get the token counts. So maybe it was a billion tokens.

22:49You know, it feels like something. It's a billion a day and it was 100 million two or three weeks ago. My average days are now well above 200 million tokens per day, seven days a week. It's because if you have an internal combustion engine sitting on your lawn, you don't use it that much. When you stick it in a car with some seats and a windscreen and a terrible hi-fi, you use it quite a lot and you use it because it's useful. People always ask me, what am I doing with my 850 million tokens? tokens. I mean, how much value are you getting with that? Well, look, let's acknowledge, right, I experiment with these things.

23:25It's my job to experiment with these things. It's my job to make sense of them. But also, I'm kind of rational about my choices. So I wouldn't be using them this heavily if I wasn't finding them useful, if I wasn't getting substantially more value from them than the cost of actually running them. And, you know, that cost might be my time, or it might be real sort of commercial dollars and cents. I talked a little bit about this in a previous discussion where I explain how I work with these AI systems to improve my thinking. And this is true. This is where part of that 850 million tokens go. So there are some really obvious things that we do.

24:00Of course, there is coding, right? There is building new bits of software. You'll see some of this. If you looked at our solar super cycle model, which I recommend you go and take a look at at solar.exponentialview.co. That was built by my colleague Hannah. That's a kind of thing that I'm also building. But there's infrastructural things that we are building. The agents are helping with that. They're also helping with writing the documentation and going through and clearing technical debt and doing security audits and bug fixing. Our mini Arnold, as of this week, now has four open claw agents that it can hand work to.

24:36It's got our Veblen, it's got our Simmons, it's got R. Bradley, it's got R. Gulbenkian. Now, I'm going to regret some of those names soon, but there is some logic to it. So R. Veblen is, of course, named after the Norwegian economist Torsten Veblen. He was an outsider who never quite fit in either the academy or in business. Most of his work was really about technology as the primary driving force of social change. He was an interdisciplinarian. I recognise Veblen in my work and R. Veblen is the agent that is helping me with my book research. R. Simmons is named after one of the world's greatest investors, Jim Simmons.

25:22So R. Simmons is the agent that helps me manage my portfolio in different ways. To help my writing and analysis for Exponential View, I turned to R. Bradley. So Benjamin Bradley was the editor of the Washington Post, and he's one of the greatest newspaper editors ever. He backed people, he valued friendship, and he remained sceptical, not ideological. And I hope that R. Bradley will help me do a better job in exponential view. R. Gulbenkian is perhaps the most controversial name. So Kalust Gulbenkian was a business architect who at the turn of the 20th century catalyzed the formation of the oil economy.

26:02Now, I have a few misgivings about Gulbenkian, but his role was essential to shaping the 20th century. And our Gulbenkian's focus is on the data and the frameworks that help make sense of the burgeoning AI economy. But equally, I've started to do some really large scale simulations to figure out difficult questions. So we can simulate any kind of audience of readers or of potential customers or of clients who I might be talking to and we can test ideas before we get there. Doing it in silico in much the same way as the aerospace industry has used CFD, computational fluid dynamics, to test the shape of wings and planes and other surfaces before they actually build them.

26:48A large-scale simulation for us can take 150, 200, 250 million tokens. And there is some cost to something like that. You know, 250 million tokens isn't free, but it's about 10 to 50 bucks. And it's not nothing, but it's certainly 100 times cheaper than doing it in the real world. And perhaps priceless if we get to a better outcome for an argument we're making for our community. But in doing this, I've also started to notice a shift in the way that I use the AI models. And it's something that we have pointed to and argued as sort of theoretically being the case, but now I'm seeing it in our own behavior a couple of years later, which is that now that I have an AI chief of staff in R-Mini Arnold, I'm also cognizant of using the right model for the right task.

27:32I mean, we never thought that a single monolithic model would win the AI race. My core agent, R-Mini Arnold, uses the anthropic models of sonnet and opus, and for some really lightweight things, it uses haiku. But it also has access to other systems, so it can pull on manus. So manus was that Chinese Singaporean company that Meta acquired. It was pretty good at certain types of research. It could do quite long workflows. Actually, it's also quite good at making PowerPoint presentations. I thought part of the point of AI was we wouldn't have to deal with PowerPoints, but what have I done? And of course, Armine Arnold also uses OpenAI models through Codex, which is its code development product.