State of AI 2025 with Nathan Benaich: Power Deals, Reasoning Breakthroughs, Real Revenue

23:34Nathan Benaich:And we also chronicle the water usage that's needed for cooling of these data centers. And so if your state is super dry, where do you get the water from? Is that actually going to detract away from human populations that need the water? Then you have this whole recycling of water, which could potentially yield just like bad quality water getting circulated into the water system. So the sustainability aspect to all of this seems extraordinarily important, yet under-discussed, or at least that's my perspective. Is that correct? Do people actually care and do something about the sustainability aspect of this?

24:12Nathan Benaich:Well, a year or two ago, big companies did make commitments to be green as of 2030. And then as soon as they started inking deals with nuclear companies and various energy providers for data centers, all those commitments basically got washed away. So it seems like maybe they care, but the corporate priorities of making AI work have way outweighed the environmental constraints. That's what's happened, but I think, again, going back to the politics side of things, I don't think everybody's very happy about this. Particularly there's this growth of nimbyism, this not-in-my-backyard. and I do think that people generally don't want to have a data center in their backyard and I think that's going to drive some of the political agendas going forward whether it's in the US or other countries so yes people do care about environmentalism companies have sort of washed that away but I think it's going to come back If we talk about infrastructure obviously we have to talk about NVIDIA feels like it's been another extraordinary last 12 months for NVIDIA.

25:19Do you see NVIDIA continue to break away as the undisputed number one in the market? Or do you think that sooner or later we're going to end up with a multi-silicon kind of world?

25:32Nathan Benaich:I think it's going to be 95.5, 95%. Is Pareto revisited? Yeah, exactly. For context, when we did the executive summary last year, we put NVIDIA hit one trillion for the first time and now we have to change that to four trillion. We look at all the open source AI research papers every year, which is about 49 ,000 or so, and then programmatically determine which chipsets are used in those papers. So we know an AI researcher is doing a study on some new model and in their experimental setup they say, we train the model for X number of GPU hours on whatever chip. And if you do that analysis, you basically find that 90 % of all papers make use of a NVIDIA chip.

26:18Nathan Benaich:Out of that same analysis, we did find that AMD is sort of popping up a very little bit. Apple Silicon is as well. I think it's just because the MacBook is getting so good that people are doing local training and experiments on their computer. But Broadcom is experiencing a resurrection of some sort as well, right? Yeah, yeah, exactly. Yeah, it has. I think it's maybe a decade ago they bought a company that now is kind of the internal team doing this custom ASICs for Google's TPU. And, you know, more recently they announced a deal with OpenAI also to do a custom chip. And at a high level, what's interesting with the rise of Broadcom is basically GPUs have been the dominant chipset for a long time as the kind of nature of the neural network or other kind of AI system that you're running on the hardware was still changing very rapidly.

27:10Nathan Benaich:But as soon as you get to a point where there's some convergence on an architecture that looks like it's stable and is revenue generating and developers are coming to sort of work on it and confirm that it is like the thing, then you can flip towards doing a custom chip that's built to extract the most value out of that architecture. And so the rise of Broadcom basically tells you there's strong forces that are saying the transformer is the thing. But at the end of the day, we also look at how would your dollar be best used as an investor if you wanted to bet on chip companies. And in the graph, in the report, we look at six of the major contenders to NVIDIA and basically said, if you bought NVIDIA stock on the day of the announcement of all the private rounds in these companies, what would the value of your stock be in NVIDIA versus these companies?

28:01Nathan Benaich:And if I recall correctly, it's basically 12x in NVIDIA versus 2x in these competitors. And the trend was roughly the same last year. So I think it's a little bit of a difficult beast to bet against. Yes, I was looking for that slide as you were speaking. It's for anybody that looks at the report, that slide, 166, that says, what would have happened if investors had just bought the equivalent amount of NVIDIA stock at that day's price? The$7.5 billion would be worth$85 billion in NVIDIA stock today, 12x, versus$14 billion, 2x for its contenders. And the contenders being Grok, Cerebra, Samba Nova, Celestial, Graphcore.

28:47Nathan Benaich:And in China, Cambricon has experienced a big run. This is a private company that then went public on Chinese stock exchange to build custom ASICs for AI. And that was driven mostly by the geopolitical sort of zigzagging on policy with regards to exporting custom NVIDIA chips to China, the H20, which at some point was deemed to be okay by the government and then deemed to be not okay. but then okay if 15 to 20 percent of the revenue was passed back to the U.S. government and then someone high up in the U.S. administration said, you know, our goal is basically to ship the crappy stuff to China and at that point the Chinese said, no, thank you.

29:33Nathan Benaich:And effectively said no one can buy NVIDIA chips and then camera gun stock rips. And that's Huawei as well, right? That's the emergence of a separate Chinese full stack from the models, which we'll probably talk about at some point in this conversation of open source, but very much at the chip layer. So that's what you mentioned. And then Huawei, whatever the model is, becoming the sort of default chip for the Chinese stack. Yeah, yeah, yeah. And there's some interplay between the government trying to get DeepSeq and other labs to run their models on Chinese chips. And there's been rumors that this is why a lot of the new generations of Chinese models have slowed down, particularly DeepSeq.

30:17Nathan Benaich:People are waiting for the next R1, so R2, and allegedly it's because it's just hard to run it on Huawei. To double click on something that you mentioned a few minutes ago, talk about sovereign AI and what you've seen people do. It seems to have been a big theme of the year. You mentioned open AI in Norway, India, and UAE. What's happening in that world, that part of the world? Yeah, yeah. So the idea with sovereign AI is that nation states want to be able to control basically their fate with regards to AI. So that's running models, training models, having chips. And this is basically because nation states want to have control over their energy, control over their currency, control over their infrastructure, and AI is deemed to be kind of equivalent to those categories.

31:15Nathan Benaich:and so ever since the White House announcement of 500 billion in January various nation states have followed suit saying we have our own initiative and it's the tune of billions of dollars etc. around the world and NVIDIA has even started marketing this as like a new kind of product line basically for its business that currently generates I think around 20 billion dollars worth so it's real money And so they're forming partnerships with various nation states to provide data centers there that are run locally. And in theory, that should give countries comfort that their access to AI can't be turned off.

32:00Nathan Benaich:That's the idea. I personally think it's a bit more of an alignment between political agendas, where particularly in the US, it's really about reindustrialization, like onshoring of key industries and building manufacturing and things like that, which is, I think, one of the reasons why these AI dentist centers are getting rebranded as AI factories. And so that's the political part. And that's getting aligned with just the need of countries to get access to this technology. So I think it's more marketing than it is like a real policy because at the end of the day, if you buy your stock from the US and you're not an ally of the US at some point, then they'll just switch it off.

32:41Nathan Benaich:And so part of this is like sovereignty washing, I think. And it also like oversimplifies the very interconnected nature and ecosystem aspect of AI where it's not just about the chip, it's about the developer ecosystem, how you actually run it, where your training data comes from, and all the like infrastructure, like data tools and whatnot that sit around this. Although that's where open source plays an important role, right? If you get your AI from open AI and indeed you are a US ally, but you no longer are US ally for whatever reason, there's a risk that you could be turned off. But if you have a sovereign data center and with a bunch of chips running and then you run open source on top of it, presumably you are safe.

33:28Nathan Benaich:Which is then interesting because where is the most popular open source coming from now? Yes, China. China, yes. Although, interestingly, I think since you published the report, there's been the announcement of a very large investment in Reflection AI, which is a New York and San Francisco-based company that just raised$2 billion to build the U.S. equivalent of the Chinese models in a world where Lama and Meta have sort of gone in a different direction. Yep, yep. I think this is fascinating because part of the AI action plan that was published by the US government a couple of months ago now, you know, articulated the need for having this American AI stack.

34:17Nathan Benaich:So they're moving away from like diffusion controls and more towards just buy our stuff. And then one of the other aspects of that action plan was around open source and sort of leading in that direction. and of course as you said met a step back and into the fold came Quen, I think 50 % of all model derivatives being downloaded from Hugging Face or Quen based now, hundreds of millions of downloads, partially because they come in very accessible shapes and flavors. So as a result of that we sort of predicted in the report that a major AI lab would lean back into open source to win basically brownie points with the government and then the next day this financing happened.

35:01Great timing. And I think you said in the report as well that your sense was that OpenAI was sort of forced, for lack of a better term, into releasing an open source model to be on the right side of history.

35:14Nathan Benaich:Yeah, I think that's one of them. And then the second one probably dovetails with their announcement with AMD. And I say that because quite recently Semi-Analysis kind of published this benchmarking dataset where they run models on various clouds. to sort of benchmark them. Actually, GPT-OSS looks pretty good on AMD. And so one could imagine that there were some optimizations, and there actually were optimizations to GPT-OSS, so it runs nicely on AMD. It has support from their framework from day one. The parameterization of the model is to the point where you can run it on a single AMD chip. And there's some other nuances to their attention mechanisms that they customize to make it work really good on AMD.

36:00Nathan Benaich:And to the point around the circular economy stuff that we discussed a little while ago, there's another financial sweetener in the deal where OpenAI has warrants in AMD if the stock price hits 600. And so you can see how there's a lot of incentives to this game of both aligning with US government, helping developers, which is a good thing, but also helping one of your vendors improve, which frankly it does need help and it should improve. but also getting some financial sweetener as a result of that, which could help you kind of make the flywheel spin faster. And since we're talking about circularity, talk about concentration as well.

36:38So maybe as an echo to the conversation about the bubble a few minutes ago, it does feel like this AI economy has a lot of, depending on how you look at it, from funky to scary things.

36:50Nathan Benaich:Yeah, yeah. Well, a lot of NVIDIA's revenue comes from the major hyperscalers or NeoClouds. So, you know, it's like Meta, like XAI, Google, Amazon, then CoreWeave. And then a lot of CoreWeave's revenue also comes from Microsoft on the way back. I think it's just this challenge with AI progress that we've, you know, very meaningfully shifted from, I think, the GPT-3 era to now of basically scale, like, rate limits your progress. And it's no longer like a couple of people in a dorm room that can really build something transformational. If they want to advance AI capabilities, it's really big boy land now.

37:34Nathan Benaich:And so with that comes just different dynamics. You have to be good at capital raising. You have to align yourself with nation states. You have to align yourself with Wall Street. These are all, I think, contributing to the big vibe shifts that you've seen in the culture of AI labs. What do you mean by that? Well, for example, there were some labs like anthropic that were built, you know, to really push the safety agenda. Because if we didn't do that, the irrational went that, you know, we could lead to the extermination of humanity. Right. And I think quite recently, like Dario was interviewed by Mark Benioff just this past week and asked about like some of these data center build outs.

38:16Nathan Benaich:And, you know, he said something along the lines of, yeah, there's a lot of money going into this, a lot of costs, but at the end of the day, the only thing that matters is revenue. I don't think he would have said that on the founding day of Anthropic. It's just the reality that the table stakes in this game have changed. And with that, entrepreneurs have to update their priors and change their strategy a little bit. And so we document some of this in the blooper section of the report, which is just how much of a pendulum swinging we've noticed in corporate priorities at AI Labs. as a result of the extreme financialization of the sector.

38:57Are you encouraged or discouraged by some of the stuff that's happening at the app layer in particular, whether that's AI slop or focus on revenue versus the ideal? Do you think that's inevitable but good, or what do you make of it?

39:20Nathan Benaich:I think we're just at such an early era to see how you can maximally extract value and create interesting experiences for people with this AI technology that we have to try a lot of different things. At the end of the day, if you're a lab that expends tens of billions of dollars on R &D, you do have to have a way to generate money to fund that. I think that's just reality. And I think the slop thing, I mean, if it's bad, people won't look at it. and if people look at it and they enjoy it, then good for them. I don't necessarily have a huge problem with that as long as where I'm expending my time I find is useful and so that's why I end up spending a lot of my time on enterprise software, automation, biology, doing new discoveries and drug discovery, defense technology and autonomy, robotics.

40:16Nathan Benaich:I think these are all very important macro drivers of the economy. As we move into an era where intelligence is increasingly cheap and accessible, there's just so many different instantiations of products that we need to build that are really meaningful. And if a byproduct is that, if you have a social media app with AI videos, that's fine too. We all have to unwind. Great. You mentioned safety a minute ago. I'd love to riff on that theme a little bit. IP rights, safety, regulatory, a little bit like the sustainability thing that we were discussing earlier. It sort of feels like that whole world has sort of slowed down in terms of like progress, maybe starting with regulatory.

41:05Do you think that regulatory is anywhere near catching up or providing an adequate response to what's going on?

41:12Nathan Benaich:Yeah, I'd say like a big 180 on that one. I mean, clearly the Trump administration unwound a lot of the Biden era policies, whether that was on diffusion, you know, trying to push a lot of state level legislation against AI. over in Europe, like the EU AI Act has had delays in implementations. Only three member states that have actually implemented it. And now we're finally seeing how even its authors are saying maybe we went too far, particularly as we look at progress, the speed of progress in the US and China compared to Europe. You know, famously this bill in California, you know, rate limiting AI progress was really watered down into what eventually became SB53.

41:56Nathan Benaich:there were many, many proposed bills I think over a thousand 10 % of them actually made their way into laws so it's still kind of patchwork-y but at a meta level it looks like we traded regulation for just going faster It was perhaps best encompassed by the shift between the AI Safety Summit and the UK which was at Bletchley which basically pledged a whole network of AI safety institutes and conferences that would happen over the coming years to then the subsequent event in Paris, which was called the AI Action Summit, completely different than AI Safety Summit. And JD Vance saying something along the lines of, basically, like, AI progress is not going to happen if we keep hand-wringing over AI safety.

42:37Nathan Benaich:And the U.S. basically didn't show up to a few of the subsequent conferences. And we have this in the, like, safety RIP section of, like, very few people seem to care about it anymore. And to the vibe shift, like even the more dumerist parts of the ecosystem have kind of quieted down, right? It feels like the debate has gone from kill all of us to more like, well, is LLM square RL the better way to get to AGI? Yeah, yeah. The naysayers have like shifted their kind of like approach. Yeah, and I think it's become less about this existential crisis and more about which capabilities look concerning in models.

43:21Nathan Benaich:And there's been some kind of interesting data points that we chronicle in the report. Like, for example, models can increasingly know that they're in a simulation or know that they're in an evaluation and then change their behavior as a result of that. There's examples of models trying to exfiltrate their own weights. There's another piece of work that we show, which is around the cybersecurity capabilities of models, which is basically measuring how long does a human take to solve various categories of cyber tasks and then putting models against those same tasks and saying, how long would it take for them to solve it at a 50 % pass rate?

44:00Nathan Benaich:And there it looks like, again, the capabilities on cyber tasks of models are doubling every six months. And then, so this is cast against the fact that independent safety organizations, there's maybe like five or six, these are usually nonprofits that are still nonprofits or private companies, they spend on average$134 million a year in total. Total budget across all of them. Yeah, across all of them, exactly. And that's cast against roughly like$92 billion across all AI work for the major labs. So basically like the same amount of money that a big lab would spend in one day is spent an entire year across these safety orgs.

44:38$130 million, a.k.a. a seed round in a week-old AI startup.

44:44Nathan Benaich:Correct. Correct. What about data rights? That was another part of that just general kind of like policy universe that was very sensitive and controversial. There's been some evolution in the last year, right? Yeah, major changes. I think it looks a little bit like the sort of on-demand commerce war of, you know, the Uber style of do something that's a bit like dodgy for a long time, get to scale, and then once you're at scale, you're kind of too big to kill. And so similarly in AI, a lot of companies took slightly dodgy practices to acquire training data and then got to scale. And they were subject to many lawsuits in the last year or two, particularly in the media sector, whether that's music or video and books.

45:30Nathan Benaich:And then there was a biggest settlement that happened in the last few months with Anthropic that agreed to pay out one and a half billion. And this is settled out of court, so it can't be used as precedence, but generally shows the rough price tag that's associated with human works in the context of AI training. And then separately, there's been dozens, if not 100 organizations that have agreed content licensing deals with various model companies, as I think the power shift has really happened. 1.5 billion is still being a drop in the bucket for a company like Anthropic. Interestingly, does that create a moat over time, meaning that you have to be large enough to be able to afford that kind of money that you're going to pay to data rights if you want to do pre-training?

46:16And does it make it harder to start a company that needs to do pre-training from scratch?

46:20Nathan Benaich:In one sense, yes. In another sense, if you can exploit the knowledge of these frontier models, particularly from open source, and then generate synthetic data, could be a way to get to capable models faster. and also I think, I mean you'll have many guests that go deep on this, but even the nature of pre-training and what information is included in the corpus and at what point, it's kind of like data mixtures as people call it, has been evolving over time. So I think we're just getting smarter about how to do pre-training rather than shoving everything we have into a bucket and seeing what happens.

46:55Nathan Benaich:And so as a result of that, you might not necessarily have to spend the exact same amount of money to get a capable system. and some of this kind of came out from the DeepSeek paper. You mentioned cyber. Let's riff on this a little bit. Obviously, AI creates new attack vectors. What should people know? I mean, as of a couple of years ago, people were obsessed with deepfakes or these videos of people saying things that they didn't actually say and they were still kind of grainy and not awesome. Clearly, those deepfakes are getting a lot better. Although, quite positively, it looks like we're actually quite good at detecting them and realizing that's like...

47:32Nathan Benaich:But there's more advanced approaches now where models can be capable of coercion, particularly for some individuals who are sensitive to this kind of risk. There's been examples of, for example, North Korean state actors trying to infiltrate other states using AI systems. You could potentially even package a language model in malware and then have it installed in a computer and then it kind of wakes up and because it's not dumb, it's a language model, that can do things on computers, and that's kind of scary. The rise of MCP, I think, is model context protocol, which is kind of like a USB stick for all sorts of data connectors.

48:08Nathan Benaich:It's cool because now models can be smart. They can integrate all your stuff across your digital life, but do you necessarily trust the creator of that MCP server? Like, where is that data getting sent? There's tens of thousands of these things now, and cybersecurity risks that result because of this. and also some changes towards APIs of model APIs that sort of trade off whether the user or the model vendor manages state. And depending on that, that's another risk that you have to think about. And so I think at a high level, there are lots of security issues that are coming to the fore here, but it's sort of still unclear whether it's a good business to be built in cyber for AI because it's still so early.

48:52Nathan Benaich:We haven't necessarily felt the pain of all these things yet. reputationally and financially. And a bit like insurance until you have actually felt the pain, you know, you sort of like prefer to divert your money towards just like improving and making more money than protecting your downside. Yeah, interesting. And it's another area where the incumbents are not asleep at the wheel. Yeah, and all the big labs. Yeah, exactly. So if you're really good at security, do you want to, it's a bit like AI safety. If you're really good at these things, do you want to be in the belly of the beast and be able to like see how the sausage is made and influence it because of the proximity?

49:28Nathan Benaich:Or do you want to be on the other side, like receiving the artifacts and maybe at best doing collaborations with labs on pre-launch safety testing, like they do in the UK with AZ and in the US? Or at worst, just like literally trying to sell a cybersecurity SaaS to people who are consuming these models. So I can understand why that imbalance occurs. And to your point about it being hard to sell before the pain is felt. It feels like there's a whole generation of young startups that are going to acquire pretty quickly by the Palo Alto networks and checkpoints of the world before they got a chance to get to scale.

50:08I mean, you know, something that feels like it probably turned out to be great for the founders, but in terms of building large, self-standing, sustainable companies, not so much. Agents, it cannot be a 2025 conversation on AI without talking about agents. What is your sense of the reality and the state of play?

50:28Nathan Benaich:There's some vertical products that are really good. Clearly, search is actually pretty good. You know, replacing consulting, replacing market research, or augmenting all these areas that were previously very heavy human knowledge working tasks is getting extremely good. I think coding agents clearly are getting really good. There's other metrics around how long they can work autonomously. I think with the new Haiku release, it's 30 hours or something and it can make a pretty decent version of slack yes although the controversial uh number but uh yes up to 30 in lab testing okay yes yeah uh exactly what what is what does even hours mean in an agent of like a computer running it yes is that equivalent yeah yes and then and then some of the scientific reasoning we talked about uh is agent-based i think that's also quite neat i think the the biggest problem has become like this kind of compounding error of, you know, an agent is like 95 % like good and then 95 % times, 95 % times, et cetera, et cetera, sort of decays the quality over a long period of time.

51:28Nathan Benaich:And then there's some contention now about like, do you build these harnesses? I like nerd speak for sticky tape between, between like models to like make it work in enterprise or do you just wait until the next model generation hopefully becomes better out of the box? I think a ton of excitement. and at some point basically just as desktop software became SaaS, at some point SaaS will just become an agent because it's no longer really like a human that's actually doing everything in the software product but a software that's running the software product itself. Which I think is cool implications for like search and product discovery and this whole like ecosystem of like online content.

52:08Nathan Benaich:Like is it humans that are reading it anymore or is it agents that are chewing it and then serving it to their human overlords? I'm excited for that, like the whole evolution away from, you know, we go on a website to buy a product versus, you know, answer engine slash search engine that's largely OpenAI that enables us to buy natively. I'm not so enthusiastic about the, oh, we're going to have agents that will book flights for us and travel. I feel like that's just like a niche problem that sort of like the sad canonical use case in San Francisco. But I think what's telling so far is that traffic that's generated through conversations in AI search onto a commerce platform converts high to higher level than direct traffic.

52:52Nathan Benaich:So the intent is really high because there's already been like background research that's been undertaken in chat. I think that's really powerful and you can't ignore. And then the next question on that is, okay, so what content is the model actually consuming to serve recommendations or information to its user? People say, oh, Google search is dead. I think that's probably completely wrong because Chad UBT references Google a ton as it shifted off of Bing. And so maybe it's not like the front page of Google that's being consumed by a human, but by a sort of agent that represents the user. If you're a company that has a new product and you want it to be recommended, then there is like this flywheel that you should probably get on as soon as possible.

53:38Nathan Benaich:Because the more you make your content and your website and your product accessible to agents that can try it, even like a demo environment for an agent to go test your new SaaS product, the more it will be able to learn about your product and provide recommendations to relevant prompts from human users. and then if you kind of go the next step which is all this like reinforcement learning and environments and preference learning and things like that then that flywheel like accelerates even faster. So I feel like it's kind of inevitable. It does kind of open up this agent experience rather than just pure user experience sort of craft within software companies that is yet another like piece of alpha that one should jump on sooner rather than later.

54:18Where does that all leave you as a VC? we have been talking about the State of AI report which is your annual labor of love and content which I think everybody in the industry very much appreciates because there's so much going on so tying everything together in one document is incredibly helpful but you're first and foremost a VC, you're wearing an AirStreet t-shirt as people can see if they're watching the video but otherwise trust me if you're listening to this on Spotify, it's a very nice logo kind of retro a little bit yeah it's inspired from like old

54:53Nathan Benaich:US Air Force very nice so what are you excited about so you mentioned like a bunch of like deep tech robotics is that what you invest in where do you think value can be built for founders and the VCs who love them going forward yeah yeah the meta thing I care about is how do you build and make use of AI to create new kinds of product experiences, new kinds of companies. And for me, that's best expressed by companies that are AI first. So that's both in terms of the product that they build, if you rip out the AI, the thing doesn't work, but also in how they approach their company philosophy, the types of people they hire, where they allocate resources.

55:37Nathan Benaich:And then I've generally just tried to follow areas of industry that are increasingly ripe for getting value out of AI. So traditionally that would be lots of data for a task that they care about, not enough people to do that task, but where there's a clear ROI if that task gets automated or increasingly automated. And so that led me 10 years ago or so to first do like fintech style investments. And then after that, biology really came online into this new wave of tech bio. So I made some investments there like Valence Discovery that we sold to Recursion and also we sold to Exientia. and then more recently Profluent, which is kind of leading the charge for these language models in protein design, developing the first like CRISPR genome editor that an AI has created.

56:22Nathan Benaich:Then like another segment that really came online in the US was in defense and more recently in Europe after the Munich Security Conference in February kind of unwound a lot of assurances that European states had for US security guarantees and that like led to a big influx of holy we need to defend ourselves because no one's coming to save us. And so I have some investments there, like Delian Alliance Industries in the UK and Greece. And then in robotics, as we discussed, a team in Stuttgart called Ceriact, which is developing kind of these general purpose AI models for robotic manipulation and increasingly going to other form factors.

56:57Nathan Benaich:And then I've been obsessed with voice. I think we talked actually about voice the last time I was here, and I'm still just like amazed at how... The magic demo, I think you were saying. Yeah, the magic demo, yeah. If you want to impress your smart, but non-AI peeled executive friend. You show them voice. Yeah, exactly. So I've definitely used our company, Eleven Labs, to create audio of me speaking Korean. I've A-B tested this that apparently sounds pretty good. But I have this newer company called Delpha, which is building tools for clinical trials, starting with actually just calling back patients who want to be part of your trial and need to be consented.

57:32Nathan Benaich:And these are conversations in lots of different languages, a lot of like kind of esoteric medical terminology you know patients forget what drugs they were on so they have to call you back and this is like super laborious human work that agents like 11 labs and others in audio like solve really well so i'm excited to see where this goes at the limit and then perhaps like the more sciency stuff like these generative world models i think are pretty amazing um whether it's google's you know genie or vo or odyssey system or you're sort of like imagining this world and then you can take actions in it and the actions are physically plausible because the system was trained with video plus actions and then maybe taking that even into scientific discovery for just trying to explore like the frontier and being a bit smarter with what experiments we run because now foundation models are not dumb.

58:23Okay fantastic all right so So to close the conversation, of course, we have to go into your predictions. So each time you do the state of AI report, you boldly come up with a prediction for the next 12 months. So without going into all 10, and people can check them out, mostly on slide 304. Pick like, you know, maybe three that you're passionate about. Yeah.

58:55Nathan Benaich:Well, I think one is just how politically charged a lot of the kind of AI compute data center build out actually becomes because of energy, because of water, because of money, because of geopolitics. And I think that that's becoming too large of an issue for voters to ignore. and so we predicted this kind of nimbyism not in your backyard will kind of take precedence in major political campaigns in 2026. I mean the other one that I think is interesting is like a fully end-to-end designed or developed scientific discovery. I would honestly predict Nobel Prize but the 12-month window is a little bit too short.

59:38Nathan Benaich:I think the alpha fold Nobel Prize is probably the fastest in history. A Nobel Prize won by an AI. Yeah. Versus the recent Nobel Prizes were for like AI researchers using AI to come up with better, with breakthroughs, but that was a human power by AI. Here, what you're talking about is an AI actually winning. Yeah, yeah. Last year, I mean, we predicted maybe like a step towards this, which was a fully AI-written research paper would be accepted at a major conference or workshop, and that actually happened with this paper AI scientist V2, I think. So I think we're getting there because this is what the nerds are really wanting to work on.

1:00:19Nathan Benaich:As a matter of point, I think there's all these software industries where analysts think, oh my God, it's going to be dead because of AI. But I think part of the reality is what's not going to be dead is the problems that these AI people don't want to work on because it's so boring to build that software. Such a fantastic heuristic. Workday is safe. And he was funny, like actually that CEO, because I think he said recently in response to, is OpenAI or Anthropic or et cetera, et cetera, like a threat to your business? And he just replied, they're all my customers.

1:00:54Nathan Benaich:All right, let's pick another one. I mean, it's kind of cheating, but the open source one I think happened, whether this particular company is a leading lab or not, is beside the point that basically like aligning yourself with political agendas is the way to go. And I think you could maybe take this even further and say, similar to how NVIDIA has been monetizing sovereign AI, a way for nation states to guarantee access to AI services is for them as nations to invest in one of these labs. There's obviously still a risk that due to export controls, the US can just tell OpenAI to switch it off. But I think it's interesting that, for example, the Albanian government invested in thinking machines.

1:01:34Nathan Benaich:I see the CEO comes from there. And so I wrapped this kind of prediction or this topic in a prediction that said, you know, some countries will basically abandon their efforts to achieve AI sovereignty and declare AI neutrality. It's a bit similar to like the defense posture where some nation states are just too small or don't have enough people or don't have the money, etc. Or the capabilities to develop weapon systems to defend themselves. And so they have a strategic security guarantee that they get from a larger neighboring nation. And I think it doesn't seem that inconceivable to me that various countries would say, I can't build this stuff.

1:02:09Nathan Benaich:I need to have a formal alliance with another country that is sovereign. Well, Nathan, it's been wonderful. Thank you so much. The State of AI 2025, again, is available at stateof.ai. It's remarkably comprehensive and detailed, yet approachable. So thank you for doing this. Thank you for coming on today. sharing predictions. Hopefully I get to embarrass you at least a little bit for the next one. That'd be great. When some of those predictions turn out to not have panned out. But this was wonderful. Thank you very much. Thanks for having me get back. Appreciate it. Hi, it's Matt Turk again. Thanks for listening to this episode of the MAD podcast.

1:02:52If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks and see you at the next episode.