2025 AI reality check: Are we in a bubble?

0:00So let's get started though with marking my work for this year. So the first prediction was that there would be no AI wall. There was this battle between whether scaling AI models was still a strategy that would work, that would deliver results, whether they could be actually even built to that size or not. My second prediction was about the speed of deployment, making certain predictions about how fast things would spread and what would happen to the price of tokens. I made a prediction that bots would out-talk humans this year in the production of natural language. I also predicted that Waymo would overtake Uber in San Francisco.

0:44I noted that I expected climate extremes to intensify significantly, and that alongside this, and despite the change in the political environment, renewable deployment, particularly of solar, would continue to surprise to the upside. And alongside that, that again, despite changes to the political environment, electric vehicles would significantly shift up a gear in their market. So there were seven predictions, and I had some watchlist themes around geopolitical volatility, demographic decline, and climate and capital, which I didn't have strong tests against. But let's start with the first one, which was that there would be no AI wall.

1:26And I said, look, research is accelerating, not plateauing. And we would likely see a 10 million token context model and reasoning breakthroughs across some of these reasoning benchmarks. Now, both of those matter because the context window of a model is a little bit like its working memory. It's a bit shonky as a working memory, but it's the bit that you put into your LLM and it can manipulate back and forth. And when you get to the end of the context window, it tends to hallucinate much, much more. And the reasoning model, the reasoning tests like Frontier and ArcGi are very useful tests for whether a model is capable not just of producing text, but also reasoning through problems.

2:09Now, in both cases, I asked ChatGPT03 to mark my work, and it said, you are accurate. There's been no slowdown in AI progress, that ultra-long context models have emerged and AI models are shattering reasoning benchmarks. And that's true. I mean, there is this 10 million token context window with Meta's LLAMA for Scout model. But I want to say that even though that model was released, Meta had had all sorts of issues with its LLAMA models and there were problems at the time. So while ChatGPT has very gracefully given me that as the evidence point that I was right, I don't think that's the best evidence point.

2:48There are a couple of other models out there, Megadev, and I believe Gemini has one in testing, which are up about 10 million level. And at the same time, O3 itself has really, really pushed its benchmark scores. So on the Frontier Math test, which GPT-4 used to score 2 % on, O3 got 25%. And I think that this is clearly what we are seeing. And if you're using the tools, you have noticed that they've got better. Although, as people also know, they're really, really quite unstable. Something that happened with our team was that we had standardized on Gemini Flash 2.0 for some of our internal workflows.

3:28It was quick, it was fast, it was bright enough. I messed around and I moved to 2.5 Flash in one case and 2.5 Pro, which is a more thinky model. And in fact, the system started to break because those models overthought the problem they were given. And I had designed the bits of the process on either side for that kind of output you get from Flash 2.0. So hidden in this idea of kind of consistent progress is also this idea of instability and change costs. The second thing was the idea of warp speed deployment. And I said that AI use would skyrocket and token prices would collapse, that AI agents would become more common, that Gen AI startups would exceed$100 million in revenue.

4:11And I explicitly said, outside of the main foundation models. And this has clearly, clearly been the case, the warp speed deployment. Sundar Pichai went off and said that Google saw a 50-fold increase in demand for tokens across its various product surfaces in the year. Microsoft, which had a head start, was reporting a five-fold increase. We did modeling on the Chinese market and we're seeing a 30 to 40-fold increase in a 12-month period. So the use has absolutely expanded and we know that ChatGPT has approached a billion users from 100 million or so a year ago. And on that revenue number, Cursor, the AI coding app, has exceeded$100 million in recurring revenues and a number of other firms are following hard.

5:04So, you know, score very, very highly there. Of course, markets can't always grow at that speed, but I don't really see the speed, you know, materially slowing down. The third of my predictions was that bots would outtalk humans. So I'd made the point that about more than half of web traffic was bot-based on the pre-LLM internet. Bots outnumber humans. In fact, Matthew Prince, the CEO of Cloudflare, made a point that LLM bots are incredibly hungry web surfers. So they surf the web about 90 times more each page than Google does for every human they send to the page. So that bot traffic is quite greedy.

5:45But the test I was really thinking about was the point at which LLMs generate more natural language text, whether it's in English or French or Farsi or Mandarin, than humans do. I can't remember if I've published this in the newsletter yet. I've certainly spoken about it in a number of my speeches. but by our internal models, we reckon that at some point this summer, LLM generated natural human text by volume of words or tokens will exceed that of humans. So we will get to that crossover point, which I think is going to be pretty significant. So I think I was right with that prediction as well.

6:24Waymo would overtake Uber in San Francisco by the end of the year. They'd seen tenfold growth in 2024, and they're bigger than Lyft. We're only halfway through the year, so they obviously haven't overtaken Uber at this point. But I do think that by market share, there is still a pretty decent chance that by December, there will be more rides on Waymo by revenue than on Ubers. And the other thing that we've seen in the market has been Waymo cascading out to other cities. Of course, it was in Atlanta and Austin, but now we've seen announcements in New York and a bunch of others. So they've got their operational design right, they've got their standard operating processes, and they're able to expand very, very quickly.

7:11I didn't at the time talk about Tesla's robo-taxis, which have just started operation in Austin. Just a note on that, because the Tesla approach, the Tesla modality is different to the Waymo modality. The Waymo requires lots of sensors, most famously LiDAR. They have a very, very strong safety culture in Waymo from the time I've spent with them. Within the Tesla, what you have is just the use of cameras, but you do have tons of data. And so you have these two competing approaches. Tesla is end-to-end deep learning, use of cameras. Waymo has a different architecture, uses many, many more sensors.

7:54Tesla has driven fewer miles autonomously, but has more data from miles driven. And it's going to be very interesting to see how those two models play out over time and how they end up converging. You know, I expect Waymo is feeling a little bit of competitive heat here and will aim to go faster as a result of that Tesla announcement. So I got that one right as well, or so far, I've got it right. In fact, what ChatGPT says is not yet because they haven't yet surpassed Lyft, but we're only halfway through the year. So then the next one was on climate extremes and seeing climate extremes absolutely intensify through storms and wildfires.

8:40And this year, we have seen record-breaking climate disasters. Last year was Canada's most expensive. And if you look at some of the temperature anomaly data that you can still find on X, you see some quite frightening numbers in different parts of the world where there are really high anomalies. In honesty, this was a pretty easy bet because this is baked in, right? it's baked into the system. We are at that point of climate acceleration. My sixth prediction was about renewables. And I said, look, solar is going to continue to keep breaking records, getting better and cheaper, and batteries will do the same, you know, even in the light of the change in the political environment in the US and in certain parts of Europe.

9:21And absolutely, this is a slam dunk. Solar PV output has jumped 30%. Incredible numbers coming out of China for the amount of solar that was put on last month. I mean, that was driven by some changes in the market where feed-in tariffs were going to change at the end of June. So there was a rush to get them out. But we're starting to see that solar story play out in other parts of the world, you know, from places like Vietnam and Pakistan, because solar is a cheap and reliable way relative to where the grid is. And that's being followed up by what's happening with batteries. So solar will have another record year as to whether any of the solar manufacturers are making money given the price competition.

10:02That's another question. And the final thing really was about what would happen with EV markets, electric vehicle markets. And I just said they would step up a gear. In the British press, particularly the right-wing press, there had been some noises saying, oh, people aren't buying these things anymore, mostly by innumerate journalists. There has been incredibly rapid growth, of course, amongst with EVs in many, many different markets. I think the record one is that Indonesia has gone from 20 % to 80 % of all vehicles sold, being electric vehicles in a two-year period, which is the fastest period ever.

10:38And we've had this hiccup with Tesla because Musk took his Washington detour. And what that's done, I think it's created breathing room for the European marks to bring some products out and for the Chinese to start to make themselves available. And we've seen in the UK, the BYD brand at the higher end with the seal and the lower end with the dolphin start to become more and more visible. Incidentally, I don't think that helps the European brands at all. I think I talked about this last week. I think they're in deep, deep trouble because they don't have cost leadership. So when I go off and rate myself, I did pretty well with my predictions.

11:16I hope some of you acted on them. That would have been fun for you if you had. But I think there's two things I should say about that. The first is we build models of market penetration, transition, cost curves, and we use those models to inform what we think is going to happen over the next year or two years. So it's actually reasonably easy to look at those models and say, okay, where do we think things are going to go? But there is also a problem with coming out with predictions that you get seven out of seven right, it means you didn't try hard enough. It means I didn't push the boundaries enough.

11:52Was I really just, you know, sandbagging a little bit and giving myself an easy job? I mean, I'd love your views on that. There were also things that I think in reflection, on reflection, I didn't talk about that I could have done. I think the biggest one, which is implicit in the first two predictions, was just how the capex boom for AI data centers, AI chips, and the electricity that powers them was going to really, really dominate the headlines. We talked about this quite a lot last year, but I didn't put it in one of my predictions. So that was an oversight, and I apologize about that. But, you know, it's so remarkable what we have seen in the change in the discussion about what AI is.

12:40Back in December, I wrote an op-ed for the New York Times saying the US has all this amazing AI research, but it's going to need power. And its inability to deliver lots of power to data centers is going to leave it with some tough choices, might even cause it to fall behind deployment of this technology relative to China. And I think that's been a really important part of the narrative. I think one of the things that has really surprised me, although it makes sense in hindsight, was the deal that was done in the Gulf, both with the UAE and Saudi, by American firms to build data centers and to build AI capability out there.

13:20You know, from the perspective of the UAE, what I think of that as is a way of creating a third pole. China, the US, who's number three. Europe can't get its act together. India is thinking about other things. So the UAE, the Gulf has stepped in saying the 21st century infrastructure is going to be AI and we're going to power it. So I missed out on making that explicit. In honesty, I don't think I would have said in January that we'd see this breakthrough in the UAE. I think the other couple of things I didn't talk about in my predictions, one was about humanoid robots. So, you know, I don't think we're at the chat and GPT moment for humanoid robots yet, but there's definitely been a lot of news coverage with it, you know, figure expanding the use of humanoid robots in BMW factories.

14:10So did I miss out on that as a trend? I'm not sure, but I do know we need to share some of our modeling around it, share some of our cost curves and why we think from those data that humanoid robots will be a big thing. So that's something that you should look out for in exponential view. And I think the final thing that I didn't really talk about, and I think I should have done, is made some kind of prediction of what would happen with the workforce, with productivity in companies, with jobs, a range of associated questions, right? Will companies be able to use this well? Will they see productivity benefits?

14:46Will they start to grow more quickly because of that? Or will they start to shrink their workforce? And, you know, I think that was an oversight. Just think about some of the things you've heard in the last couple of weeks. So Benioff, the CEO of Salesforce, saying that AI performs 50 % of all the work at Salesforce. I mean, chortle-chortle around Twitter, what does Salesforce actually do? And that AI agents were succeeding in 93 % of the tasks they were given. That sounds very high, but that's what he claimed. Satya Nadella saying that Microsoft's code assistants write 20 % to 30 % of all code, and Microsoft going through a range of headcount cuts, I'm always careful about how to interpret headcount cuts in these large tech companies.

15:31They hugely, hugely over-hired in and around COVID. And of course, naturally, as the market changes, as you see growth in different areas, they end up trimming their sales. So it's hard to interpret at this point what's going on. But it wasn't just Satya. Sundar Pichai from Google saying about a third of all code as AI generated. Andy Jassy from the CEO of Amazon saying that AI is going to lead to fewer jobs overall. And just in the last couple of weeks, Amazon passing a million humanoid robots. And I think that was that significance that it's as many humanoid robots or robots, not all humanoid, pardon me, as there are people.

16:09It's not, of course, just the tech companies. So Jim Farley, who is the CEO of Ford, he's a guy I really like. He's a very good straight talker, been very, very authentic about the capability of Chinese electric vehicles, has recently said, look, he thinks AI could eliminate up to 50 % of current white-collar jobs. And Jamie Dimon from JP Morgan has anticipated this 10 % reduction in headcount due to AI tools. So that area was not something I put in a prediction about, and that's remiss of me. So to my readers and to listeners, I apologize for missing that out. The thing that I would say on that particular question is, of course, we're going to talk about it over the summer.

16:54We've got a couple of really good essays coming out on that topic. But the other thing to note is, so I've run a model, I haven't had a chance to write it up yet, where I looked at the automation of several classes of white on a desk work from invoice processing to payroll, accounts receivable, HR, basic legal drafting. And I looked at the volume of work that was done in, you know, 1945 in those areas, in 1975, 2000, 2025. And, you know, it's, you can say one thing coming out of this, and I'm rather preempting what I will write when I finally get around to writing this, which is that the amount of work that is done in those categories by automations today exceeds the amount of work done in those categories by humans in 1945.

17:48So in some sense, we've already replaced all of that by a factor, right, a multiple. Of course, we're doing much more of it because work begets more work, the global economy is bigger and more connected, we're more demanding. But there is some data there that we can look at and we can say, actually, we have already seen some part of this play out. And so this is not to downplay the risk of job cuts on a company-company basis that aggregate up to quite a lot. But it is to say that interpreting what's going on is more complex. So I will pull that essay together in the next couple of weeks and send it out.

18:29So that's my mid-year assessment. I was pretty accurate, which I think is therefore a flaw in what I predicted. and I will try to address that the next time we get on with this. But I do want to talk a little bit about the next couple of years for a couple of minutes and then just leave you hanging with a question right at the end. So by the time we get to 2027, 2028, I think a few things will happen. If you look at technologies like this, they go through a phase where all the excitement is in the infrastructure. And that's where the internet was in early 90s to mid 90s. And then when the infrastructure's in place.

19:07People understand how to build with it. The excitement happens in the vertical apps. Now, that's not a clean separation, of course, but it's a helpful heuristic. We're still in that infrastructure phase with AI, and the vertical apps that are emerging are still young. But within two years, they won't be young. I mean, the speed of growth of these businesses is pretty insane. Anthropic is now at a$4 billion run rate entering July 2025. In January 2025, it was at a billion dollars. So it's quadrupled, which is really in a six-month period, which is really unheard of in a company of that size. I expect the period of fireworks to start to hit in 2027, 2028.

20:00So why is that? That's the point at which startups that started a year or two ago will be four or five years old in this accelerated timeframe. And they will start to be big enough to eat meaningfully in aggregate into incumbent companies, forcing those companies into either defensive postures, cost cutting, or just having to spend more money to maintain market share. It's also the sort of time by which companies who started early and seem to have a grip of this technology, incumbents I mean, like Moderna or JP Morgan, should have gone through proof of concept, pilot, scale, multi-scale, deepening, and should start to see the deeper, wider results of having the technology across their companies.

20:53So you're going to get this effect in the economy where the startups are now big enough to be challenging, to be difficult, to have some control of distribution, of how customers get to incumbents, or perhaps they've got competitive products that incumbents can't compete against. And at the same time, incumbents who got their act together will be getting the results they expect and having that impact in their sectors as well. So that will be quite a hairy time, I suspect, 2728. And I think that's predicated really on this technology continuing to improve. As you know, I don't really like the phrase AGI, and I'm not predicating any of this on some sort of miraculous technology that crawls out of a data center.

21:39But the other reason 2728 will be really interesting is that if this is actually a speculative bubble with way too much irrational exuberance, that might be the period, point at which there would be a significant pop, right? Why do I think that? Okay, super unscientific. Line is still going up. There are still strong incentive buy signals. Balance sheets of the big tech companies are not strained at this point. There's a lot of capital in the world. You just saw Meta do a debt deal, even though it's got tons of cash on its balance sheet, a debt deal to fund some data center expansions. So there's still enough vertical momentum, I think, for this to run for a little while.

22:21And then when bubbles do burst, if they burst, they don't happen immediately, right? So the popping of the dot-com in April 2000 took a while to get to its bottom. And so that's why I think 27, 28, if that scenario happens. I really don't think that that will happen, at least not in that time period. Of course, there's going to be overexcitement in investment and overexuberance. But because of the scale of customer demand from consumers and enterprises, and because enterprises, when they get this right, and many of them are getting it right, want more and more and more, suggests that we've got something that's actually driven by fundamental business value rather than by speculative hype.

23:02That is really everything that I have to say right now on Friday with Azeem on the 4th of July. And happy independence day to those of you who are celebrating. I said I'd leave you with one question. I mentioned that we are halfway between the year 2000 and the year 2050, and we're also halfway between 1950 and 2100. But just looking at that first period, between 2000 and 2050, the question I'm going to ask you is what different way will you look at the world with the recognition as of today that we're closer to the year 2050 than we are to the year 2000. So what different way will you look at the world with the recognition that we are closer to the year 2050 than to the year 2000?

23:47And with that, I will sign off. All of you have a wonderful day, 4th of July. Enjoy yourselves and everyone, please have a great weekend.