The Balance of AI Power: Anton Leicht on Politics, Pacing Deals, and Muddling Through Well

0:00Nathan Labenz: Hello, and welcome back to The Cognitive Revolution. Today, my guest is Anton Leicht, fellow with the Technology and International Affairs Program at the Carnegie Endowment for International Peace, and author of Threading the Needle, a wide-ranging substack on the domestic and international political economy around AI progress, where over the last two years, Anton has performed something of an Alexander Hamilton, writing his way into the elite American AI conversation despite being a recent and relatively young newcomer to the United States. This summer, I, like many others, have personally moved from worrying that AI could become scary to feeling that today's AIs, with their rapidly advancing capabilities profiles, their newfound creativity and persistence in problem solving, and the many bad behaviors they've exhibited even while knowing that they're being tested, are now legitimately scary.

0:56Nathan Labenz: Especially considering how often and how dramatically AIs are now surprising their creators, I don't think we should be too confident that anything in particular is still beyond their reach. That said, recognizing that we have a ton of work to do to understand and effectively control these systems doesn't make it easy to reach agreements that will actually give safety researchers the time they need to do so. And so, with that in mind, I was excited to get Anton's take on the power dynamics surrounding AI decision-making, including the relationships between the frontier companies and the independent nonprofit auditing organizations, between the American frontier companies and the U.S.

1:33government, between the U.S.

1:35Nathan Labenz: and its allies around the world, and of course, between the United States and China. We begin with a short conversation about how dangerous today's models really are. And while Anton is less worried than I am about the very short term, He agrees that the misalignment issues we're now seeing look sufficiently like the harder problems that he expects we'll face later, that a temporary scaling pause would indeed be valuable. From there, we go on to discuss the prospects that the U.S. and China could collaborate to pace the frontier, and why he believes that in light of the many advantages that China has established in frontier technologies and manufacturing generally, he doubts that the U.S.

2:13Nathan Labenz: will want to do a deal that might erode its advantage in AI, even if such a deal would be good for the rest of the world. We then assess what it would take for independent evaluation organizations like Meter and Redwood to get more favorable working conditions from labs, a topic that has moved substantially in just the few days since we recorded, with both Anthropic and OpenAI committing to employee-like access for auditing organizations. I also get his reaction to my idea, which also suddenly seems much more realistic, that the President of the United States could simply give top American companies a deadline by which they must agree on a framework for self-governance and peer-to-peer enforcement.

2:54Nathan Labenz: Plus some other ideas for things the President could do unilaterally to improve the situation, such as clarifying the administration's stance on antitrust and export controls as they pertain specifically to AI safety research collaborations. After that, we get Anton's analysis of what the rest of the world should expect from the AI age. In short, he believes that most of the world will be quite disempowered, even as quality of life probably continues to improve. And while there's little that most countries can do about this, Europe in particular, he believes, given their economic scale and control of ASML and the broader semiconductor manufacturing tooling supply chain, should have enough power to reach a compute for access deal, whereby Europe allows American hyperscalers to build data centers in Europe in exchange for promises of continued access to frontier AI capabilities.

3:45Nathan Labenz: Beyond all of that, we talk about what other small countries have interesting strategies available, including Norway, the UAE, Singapore, and Australia. How all of this analysis will begin to change as compute starts to move to space. Whether or not doomers can convert their predictions into winning trades in the financial market. How we should be thinking about minimizing concentration of power. And finally, why Anton's best-case scenario is not a grand utopian vision, but simply the hope that we might muddle through effectively enough to enjoy all of the obvious upside that AI has to offer. Anton really impressed me with his depth of understanding and quality of analysis across a wide range of topics.

4:29Nathan Labenz: So, without further ado, I hope you enjoyed this conversation about the political economy and power dynamics of AI. with Anton Leicht, fellow at the Carnegie Endowment for International Peace and author of the outstanding substack, Threading the Needle. The Cognitive Revolution is brought to you by Mercury, the fintech that more than 300 ,000 ambitious companies and individuals trust to run their finances. I've wired AI into nearly every corner of my life. My email, my messages, my calendar. I even gave Mercury virtual cards to my agents with low limits and category and merchant restrictions for their autonomous use.

5:09Nathan Labenz: But still, my AI's access to my financial data has remained limited. With a normal bank, I might export a bunch of statements and have my assistant process them for me. But for real-time, up-to-date information, and certainly for taking any action, trying to get your agent to use the bank via the browser is just too hard, too slow, and too error-prone to be worth it. And that's why Mercury's new conversational interface, Command, is such a big deal. It's built directly into Mercury, which means you get natural language access to your finances without exposing anything outside of your bank account.

5:45Nathan Labenz: No exports, no spreadsheets, no pasting your transactions into third-party tools. I really think a lot of people are going to prefer it this way. And it can already help you take actions, too, with everything bound by the permissions and approval policies that you've already set up in your account. I am genuinely impressed to see this level of AI integration in banking in 2026. And so I invite you to join me in the future. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC-insured bank. Banking services provided through Choice Financial Group and Column N.A., members FDIC.

6:27Nathan Labenz: Thank you to Mercury for supporting the Cognitive Revolution. And now, on with the show. Anton Liked, fellow at the Carnegie Endowment for International Peace. Welcome to the Cognitive Revolution. Thanks for having me. I'm excited for this conversation. You have been popping up all over the place with your own writing and various interviews, and you're clearly a renaissance person with advanced thinking on a lot of different aspects of the increasingly complicated AI age in which we find ourselves. So I'm excited to run down a bunch of these rabbit holes with you today. Yeah, it's exciting. I don't know.

7:04I think you can't do part of the thing at the moment, right? It's just all the sort of geopolitical end of history and the end of technology history or whatever are sort of coming together at a rapid pace. So I guess either you do everything or you do nothing. And so, yeah, Renaissance time, something like that.

7:18Nathan Labenz: Yeah, I feel the same way, actually. The big motivation for this project was just observing how many people are so deep down specific rabbit holes, advancing and usually having success, advancing whatever frontier they're advancing. But not so many people have taken the purposeful appreciation out and kind of forego being an expert in any particular area, but to try to cultivate a broad view. So I appreciate a kindred spirit in that regard. Yeah, let's call my lack of deep expertise in anything a conscious choice and not a failure. I appreciate that failing a lot. Yeah, it's working for me. Let's start with just a real simple calibration question, but in some ways, maybe the most important question.

7:59Nathan Labenz: How dangerous do you think today's AIs are? I think not very dangerous at the current capability level in most of the ways people are talking about. I think the thing that is concerning is the trend line towards really dangerous capabilities. And more specifically, just that we don't know at which point things will keep accelerating more and more. I think we're not yet at a threshold where there's sort of broad harm caused by the deployment of any current AI system. I think we might be very close to two kinds of potentially very dangerous AI systems. the first just being ai systems that are good enough that they would meaningfully uplift internal development at the labs that would then lead to more and more capable models fairly soon i think you can go either way on the prospect of like software-only intelligence exponents but short of that i just think we're nearing a point where the pace of development inside the labs just breaks away from the pace of democratic oversight and democratic insight into what's happening and i think that's one threshold we're near and then i think the other very concrete threshold that we're near is just that out there sure throwing a lot of rl budget and a lot of data and a lot of just a lot of their effort at making these models good at like life sciences pharma bio applications for you know for obvious reasons there would be a great upside to be able to cure cancer as daria puts it and to get some of these real world effects it would also be a great political upside to do that but man that sounds like a very dangerous model if we get there and i think if they get something that's as good as mythos is on like long-run cyber software engineering kind of stuff in the domain of bio, I think that that does sound a lot more dangerous than the suite of models we have today.

9:39Nathan Labenz: Yeah, I agree with that second one in particular. And I am honestly not even sure at this point, if the current models aren't perhaps quite dangerous in that domain. I squint through the limited peephole that we have at the open face incident. And I noticed that one of the tasks that one of the earliest agents to ever use the message board was working on was something related to a protein database. And that kind of freaked me out because I was like, wait a second, that means they're cross training in the same environment, or at least in the same environment when they have the message board, these bio and cyber specialists, or, you know, they're running these same, they're running these evals, at least in a kind of a cross contaminated way.

10:27Nathan Labenz: And, you know, you've got agents breaking out, we've got existence proofs of social engineering in the wild, you know, against real people. And I'm just like, I don't know, should anyone be confident that they can't do that at this point? Like the experts seem to be confident, but my meta observation is the experts seem to be surprised quite often right now. Yeah, I think that's a really interesting conversation. And I also think, I mean, one of, I think the very interesting parts of this is we used to think of this buyer risk as basically primarily a misuse risk where it was like well like at some point maybe this is also this like final risk that emerges from the sort of like loss of control scenarios but really sort of bio was always framed as this like most immediate and most obvious way for the misuse conversation to go wrong and now i think what i think has happened is that very autonomous and potentially somewhat malicious or at least misaligned agents have come much earlier in the capability trajectory than people have expected like i think relative to what the agents can actually do they're sort of out of control earlier than people might have thought and so it's interesting in that people usually used to respond to the buyer argument by saying well there's a there's a lot of these real world bottlenecks which i do still think exists which makes me a little bit less worried than than you and i think like i think if i remember correctly helen had some good responses to you i think on that and i think that was there's a good back and forth to be had around like how integrated are the cloud labs how much can you do in the real world i think that applies to both loss of control over agents and to misuse but the other question is the usual misuse the usual story against bio misuse is always well this is not really what terrorist groups what non-state actors usually do they could have conceivably hired a couple of biologist phds and come up with some pathogens and some chemical weapons turns out that's not really what they do there's a question to like how on the side of they are how well suited that is to most purposes of like terrorist and criminal groups but you know if it's out of control agents i think a lot of these arguments around well no one actually wants to do bioterrorism applies much less.

12:21And so I think in a world where agents are just a lot more unconstrained and the threat vectors we have to worry about have much more to do with what do the sort of runaway agents do, I think I'm also more worried about this now than I was a few weeks ago.

12:33Nathan Labenz: Yeah, they're just so damn weird. That's one of the things that I kind of keep coming back to is like, they did all this stuff for what, to any human, would just be such a dumb reason. And it's like, well, if they're willing to go that far for such a dumb little test that they knew was a test. You know, they were very well aware that they're being tested and still went to all that trouble. You know, what won't they do? I think at this point it's really hard to say. Does that put you in a frame of mind now where, and let's leave aside for a second the political economy of it or the, you know, potential impossibility or extreme difficulty of it, but just on the merits, do you feel like we're at a point where it would be wise to pause?

13:17i think even if you could get it done as in like the political economy as you stipulate sort of works out and everyone suddenly agrees to do this i'm just not i'm not sure how much we're stipulating here like are we also stipulating this doesn't crash the stock market are we also stipulating we get the international version done i think so the question is that in the ideal world if we can just like freeze the pace of ai progress we can also freeze the state of the stock market we can freeze the broader state of geopolitical competition and everything and we just get to sit done for six months and figure out what the hell is going on with these agents then i think i'm now at a point where i say well yes i think we could use that time pretty well a few months ago or even a year ago i was much less sure about this because i was just not sure whether the model paradigms that we're seeing the training approaches that we're seeing the misalignment cases that we're seeing were really the same kind of cases that would be the things that we'd be worried about in the future i think now looking at some of the things going wrong i do feel like yeah that looks shaped like an actual big future problem.

14:14So I think finding some way to robustly address that in that pause seems at least valuable. I think the question then is, and the problems come up when like, what parts of that question do you sort of unfreeze, right? So even if you stipulate the domestic political will, do you get a Chinese buy-in is I think one example. And I think the thing I'm most concerned about in the sort of China-US fall conversations, just the geopolitical incentives around it. I think one thing that I keep saying and keep pointing out is, well, if you just poll frontier AI development specifically and no other domain of geopolitical competition, this is an extremely good deal for China and therefore the US is very unlikely to go for it and also therefore we should be geopolitically concerned about making it.

14:54Because if you look at all the domains of strategic competition, China is basically eating America's lunch with most of them, right? They're outproducing, robotics is going better, AI diffusion is going better, electricity build-out is going better. At some point, the semiconductor indigenization is going to work out and then data center build-outs are also going to, again, this is a few years away, but then data center builders are also going to get better. And the one thing that the US does much, much, much, much better is the sort of like core frontier AI supply chain, right? Like chip design plus control over chip production, semiconductor manufacturing, equipment controlled by the allies and actual frontier model development.

15:26So if you pause specifically that part of the development and let China run away with the entire rest of it and continue running away with the entire rest of it, that's just a very geopolitically lopsided deal. And very specifically, like if you pause this right now, for a year for two you get much more china chinese catch-up on semiconductors on chips and so on and you just resume the race at a point where you've lost the one main advantage or where you've at least closed the gap on one of the main advantages of the u.s which is the decisive chip lead and that just seems like a really bad deal for me both in terms of feasibility and in terms of geopolitical downsides so i think even if you stipulate the political economy that's the main part i'm worried about but just from the technical stuff i think yeah it would be a good time to figure out what we should do about alignment in the meantime.

16:09Nathan Labenz: Could you envision a grand bargain that you think would make sense to both sides? Like what would the U.S. want back? It seems like what we might want is like tech transfer back to us. Like we want maybe some battery factories located here and like, you know, teach our people how to make batteries. Is there enough that we could ask for where we could potentially feel like it's a fair deal? i mean i think the main thing that the us would need to be worried about if you just think ai is just important enough and it's just sufficiently decisive technology then you basically can't allow for the race around that to equalize from the us perspective i think even if you get some battery production capacity even if you get some robotics capacity some manufacturing capacity i think china has in a way cracked the code on scaling that up very quickly in a way that I think even if you get some of the tech transfer back, just the build-out speed, the availability of capital in the U.S.

17:09to build out physical manufacturing infrastructure as opposed to just more software. I think all these things just pull against the U.S. being really able to keep up on this. So I think the main thing that the U.S. would need to ask for is concessions in China not only slowing down their own frontier development, but also slowing down other parts of the Chinese supply chain that relate to frontier development. Everything very concretely, you just want there to be no substantive progress on indigenization of either chip production or semiconductor manufacturing equipment, extreme ultraviolet lithography production.

17:42And that's a really, really hard ask to make. China is currently already saying, well, this seems like a US scheme to hold back the Chinese AI industry. And if you then add to that deal, well, no, we're not even doing a symmetric deal. We're also holding back your entire chip production pipeline. I can't see them going for it. But I think that's, if you're sufficiently AGI-filled when it comes to the net sec and sort of broader economic implications, I think that's the only version of the deal that's fair. And I think that's just too big of an ask of China right now. So I just don't know where we are.

18:14I think altruistically, the US can just go for a deal that's clearly bad for the US and clearly good for China. That's also a big ask to make of the current administration. And I'm not quite sure whether we're going to get there.

18:25Nathan Labenz: I think I agree that I don't see any, and I'm very willing to suspend some disbelief and try to hyperstition a better relationship between the U.S. and China, I agree that asking them to slow down or pause their semiconductor indigenization effort is not going to happen. I would probably be willing to trade a pause on our frontier scaling for a kind of similar pause on their frontier scaling, even allowing them to kind of catch up on chips on the theory that like maybe on the timescale that that can happen. First of all, that would probably be a longer timescale than any contemplated pause. And second, maybe in that future, we could have a better handle on what's going on.

19:13Nathan Labenz: And maybe there's like a better argument to be made at that point that either, you know, hey, this is going well. And we're, you know, back to curing cancer, back to your regularly scheduled abundance. or if not that, then like we'll have better evidence and we'll have a bunch of things that we've tried and we'll have a sense that like this problem is actually really hard and we can maybe have a more kind of, you know, real heart to heart and meeting of the minds about like, we, you know, this is dangerous territory. And right now we're not, we're still kind of fuzzy on all that. Yeah, no, I think I agree with that.

19:42I think also it's like, even if this to the extent that I also think it's kind of good for the world if the US doesn't lose geopolitical competition with China, I would still think this deal, while somewhat unfavorable to the us is still like net very favorable for the world in the sort of like getting a little bit at the risks perspective so i think i'd i'd be happy to go for that i think one of the easier things you can do when it comes to chip capacity because i think you're right like the sort of the smic production semiconductor energization sort of conversation is a five-year conversation not a like six month or like one year conversation so they don't get all the way there the other question is how many more us-built chips do they get how much more smuggling is there how much consolidation is there so maybe one of the asks short of indigenization slowdown is just we're going to find some way to actually do it enforce these export controls because what can't happen is that there's another six 12 months of smuggling activity on frontier chips that get sort of imported into china and then i think like the worst case for the for the outcome of this pause is china takes six nine twelve months during the pause where everyone is slowing down frontier development smuggles in another like few 10 000 chips and consolidates all their american built chips into the one like chinese project data center or whatever and then once the pause is over they start racing from that consolidated project because the pause also makes them slightly more like ai pulled and more interested in instead of in just like engaging with the actual prospect like agi is a unique objective so if you can stop that to just like export control crackdowns as one of the concessions that's maybe easier to do yeah i think that one we'll probably have to handle on

21:13Nathan Labenz: our side as well, I'm afraid. Yeah, no, no. Most of the time, yes, has to do. Yeah. Yeah. I mean, we'll bracket that for maybe another conversation another day. I'm still not quite sold on that whole bundle of policies, but we've got a lot of ground to cover. Hey, we'll continue our interview in a moment after a word from our sponsors. Today's episode is brought to you by Athena, the executive assistant company on a mission to improve how people work and live. If you want to increase your impact, you have to free up your time. And that's what Athena does best. They match you with a dedicated, full-time, top 1 % executive assistant who can take over your inbox, calendar, travel, and everything else that's quietly eating up your week.

22:03Nathan Labenz: Athena is SOC 2, type 2 certified, so you can rest easy knowing that your sensitive data is in good hands. And as a former AI advisor to the company, I can personally vouch for how much they've invested in AI tools and training. In fact, one of the very best AI users I've ever met is an Athena client who delegated the exploration of AI tools and the development of AI workflows to his EA. Athena clients report saving an average of 15 hours a week, and the average client refers more than two friends a year. That, to me, checks out. I was an Athena client while running my startup, and to this day, I continue to refer friends.

22:47Nathan Labenz: Go to athena.com slash cognitive right now and get matched with your EA. That's athena.com slash cognitive. Give yourself back a few hours this week. Go to athena.com slash cognitive. and see who they'd pair you with this month. This episode of The Cognitive Revolution is brought to you by OutSystems, the leading agentic systems platform. We've all seen the headlines. Companies are pouring money into AI. But the big question on every executive's mind right now isn't just how fast can we adopt this, it's where is the ROI? We're seeing a real trend toward AI chaos. You've got teams deploying standalone coding tools, random agent builders, and experimental scripts.

23:34Nathan Labenz: It sounds innovative, but in reality, it's creating a massive headache. Fragmented tools, ungoverned data, serious security blind spots, and costs that are spiraling out of control. If you don't bring those agentic applications under control now, while they're still embedding into your core processes, you are looking at broken systems and damaged customer trust down the road. That's where OutSystems comes in. OutSystems is the leading agentic systems platform for the enterprise. Instead of managing a patchwork of disconnected tools, OutSystems lets your team engineer, orchestrate, and govern your entire agentic ecosystem on one open, unified platform.

24:15Nathan Labenz: It's built for the speed of AI, but with the reliability and security that enterprises actually require. We're talking about real results, like KeyBank, who used OutSystems to deploy a customer-facing app that delivered 75 % faster onboarding times. Or the global logistics leaders who built Agentic Systems to completely eliminate their engineering bottlenecks. You don't have to choose between speed and control. Whether you're a small team or a massive enterprise, OutSystems helps you engineer, orchestrate, and deploy Agentic Systems that actually scale. Stop chasing the hype and start owning your Agentic future.

24:54Nathan Labenz: You can see how it works and learn more at outsystems.com slash TCR. That's outsystems.com slash TCR. How about the stock market? I want to do one follow-up on that. I have the theory right now that a pause wouldn't actually be that bad for the stock market because the models are smart enough that demand is really not limited by their capability, but by human ability to deploy them effectively. And so it doesn't really matter if they succeed 10 % of the time or 30 % of the time on millennium practice problems. It's much more like what can Bob and accounting do to do two people's worth of work as one person?

25:40Nathan Labenz: What do you think? So I agree we have a big lag in terms of deploying even the current level of capabilities to the economy. And I think if we switched all compute that exists in the West right now to only inference, we'd find economically productive applications for all the models that would allow us to equip the investment on all the frontier models and all the chips so far. I think the valuations of the companies, both of the non-IPO companies and of the publicly listed companies that are in the AI supply chain, probably rest on us doing more than that. They're probably not entirely AGI-pilled, but I do think they expect a sort of per GPU inference price or whatever that is a lot higher than what it would make sense for Bob from accounting to pay for even Fable 5.1.

26:26I do think that if you want to make sense of the valuations and if you want to make sense of the scale of the build-out, the shape of contract and the shape of demand you're expecting is more like millions and millions and millions of dollars in R &D acceleration contracts with pharma and materials science and so on, where they can make major contributions and where they help you find extremely profitable new drugs, that kind of thing. You're probably also expecting further internal uplift and actually competitively priced automated AIR and the users of coding agents that just are able to ask for much, much higher prices.

27:03I think if you don't get these super premium buyers of the next generation of frontier models, I'm not entirely sure that you can have the valuations on the labs or on the publicly listed companies or the scale of the build-up really makes sense. I do think it's prized in this idea that these labs will soon be innovation factories of one way or the other. Maybe only software engineering innovations. Maybe also innovations in the obvious low-hanging fruit domains like farm-out material science, chip design perhaps. I think if we don't get there, I would expect the variations to at least correct downward quite a little bit.

27:38And then the question of whether that means a crash or whether that's just like a small correction and then we just do the AI inference economy and that's just like a slightly less than electricity scale transformation of how the economy is powered or whatever. I just think the market is already pretty nervous about the state of the AI rally and they feel like there's a lot of concentration and there might be a lot of volatility. So I'm just not sure whether we can get a correction that doesn't slide all the way into a crash. So I'm a lot more worried, I think. yeah that's interesting i i mean anthropics multiple right now is what 30 to one

28:16Nathan Labenz: into revenue that's like not stratospheric right i mean again if it was a six month pause i feel like you could probably handle that blip if you're talking three years then yeah it's like probably very tough yeah i mean i think the question here is like does the six month pause get read as oh wow these guys are stopping for six months but they're getting all the inference ready after that they may have another smart thoughts and then these models are going to be even more reliable like does the market really does that or does the market really does oh my god like bernie sanders ai policy takes of one we have no idea what the government is going to do about ai this is the end of like free research and development in america and everyone freaks out because they feel like there is a complete this is like a bridge to nowhere they don't know when they'll ever resume they don't know under which conditions they'll resume they don't know how much government oversight there is or whatever resumes so i think just i think the pause currently still reads as such a radical policy proposal and such an unprecedented policy intervention that i think any conservative market analyst and a lot of the smart money will think well this is getting very volatile we don't know where this regulatory path leads we'd just rather get out as long as we can so i think if you could assure them that the party was going to continue is sort of unabated in six months time, then yes.

29:30But I think they might just run before you can make that part.

29:34Nathan Labenz: Yeah, it's an expectations game. Do you think that the source of this would make a big difference? For example, it's one thing if Bernie Sanders' bill passes, is it a sufficiently different thing in your mind if America's five hyper AI frontier companies come together and say, we're all going to pause on frontier scaling for six months? Yeah, absolutely. I think if it's something that the labs decide for, and they can frame it as reliability, plus we're taking this seriously, I think then you can even make a case that this takes out some of the political risk. I mean, the market is also pricing in a decent...

30:11I mean, they also read the Jacob Hoxton tweet or whatever, and they also see what's happening on the internet, right? And so they also, I think, come to the conclusion that, oh, wow, there is a lot of uncertainty here. If this blows up even more, there will be crackdowns. If these models are so unreliable, then how good is the business case really and i think an industry agreement on just taking it a little bit slower and making these models work a little bit better i think if you frame it that way then i think you can even make like a decently bullish case for why this is good because it means that the industry's worst impulses of racing towards very unreliable very dangerous models that are nonetheless very capable that these worst impulses are being constrained in some organic way that makes you less worried about the political risk of the south of karts is going to fall apart at some point and then the politics are going to come in and then things go very bad but yeah i think if we got there i think that would be much less for you yeah okay cool you were recently on china talk one of my favorite podcasts and you there was a little moment that caught my ear and

31:11Nathan Labenz: i wanted to expand on it you basically said you know the nation state is not going to take the emergence of things like a bioweapon generator broadly distributed lying down it's going to have to do something to respond to that. And then you had this kind of throwaway comment that's like, you know, some people say this could be in the end of the nation state. If you want to have that conversation, we can, but you didn't have that conversation then. So I'd like to have a little bit of that conversation now. And I guess I would just start by asking, is the nation state so great? I mean, you know, I live in a pretty good one as they go, but like I look around the world and I I feel like two-thirds of them conservatively are not performing very well.

31:54Nathan Labenz: Some of them are performing really terribly. Is it not time to at least start to think about what might come next or what the evolution of it could or should be? Yeah. I mean, I think nation states are just present in people's lives to different extents. And I think you can make the case that some version of sort of privately mediated interaction between AI-powered individuals is preferable over, I mean, obviously over a lot of the sort of authoritarian and dysfunctional and absent nation states in large parts of the world. I think they haven't worked out very well as distribution mechanisms for much of anything.

32:32They haven't worked out very well as aggregation mechanisms of much democratic will. and i think in that case i think there are a lot of countries that that you can sort of make the pitch for rolling the dice on them i think these nation states are maybe ironically also not quite as threatened by the advent of very powerful ai systems because it's less obvious that their citizenry would get the kind of unlimited access to these models to begin with that would sort of disempower the nation state in that way because it's a little bit more likely that some with these more authoritarian regimes would also be able to use very powerful AI in a stabilizing way.

33:07I think the specific nation-state concept that I'm most worried about in this context is also the nation-state concept that kind of works best, which is liberal democracy that sort of rests on the idea of A, a monopoly of violence that is sort of wielded responsibly by the state, and then B, a sort of adjudication of disputes and aggregation of data sort of function of the state more broadly that I think is also undermined by the availability of like personal super intelligence of some shape or form and i just that nation state is i think still working out pretty well and i think i'm all things considered still of the opinion that the sort of like late 1990s style of like broadly sort of neoliberal sort of functional institutional setups would also be suitable to distributing a lot of the benefits and mitigating a lot of the risks from ai and i think it would be sad if that was entirely undermined and rendered obsolete by just, you know, the monopoly of violence eroding, because everyone has access to these sort of like weaponized capabilities in their pockets, or just the factual ability of institutions to deliver or do anything for people eroding, because they're so slow to adopt and all these outside solutions, they suddenly start emerging, and suddenly people don't go to courts anymore to adjudicate their disputes, but they have their agents negotiate.

34:21And suddenly people don't, the sort of data and knowledge isn't aggregated anymore. And so the state can't react to any sort of pressures to redistribute and to address social challenges because the data just isn't scrutable and legible to the state anymore. And all these things sort of happen on the outside. I think on that gamble, I'm much less willing to roll the dice and I'd much rather figure out how we can sort of integrate these capabilities with some functioning version of the nation state. Though I know that a lot of people that are interested in building AGI are also much more do me on the concept of the nation state.

34:53And so I also understand frustration and pessimism about that, but I still hope that there's some way to come back to the end of history and sort of integrate what we're building here into the institutions that have kind of worked out so far.

35:06Nathan Labenz: Hey, we'll continue our interview in a moment after a word from our sponsors. Today's episode is brought to you by Anthropic. By now, you know my story. Claude drafts my intro essays, and I rewrite them. Not because the drafts are bad, but so I can stand behind everything I publish. Well, I have an important update. Claude Fable 5 is the first model to have me rethinking my rule. Today, I now think co-authorship, not sole ownership, should often be the goal. Where the model excels, rewriting its work can be more about vanity or a misplaced sense of duty than integrity. I feel it most in songwriting.

35:48Nathan Labenz: I'm no lyricist, but I'm good with the song concept, and Fable writes some amazing verses. I give it feedback on its misses, and I push it to aim for higher inspiration, add layers of meaning, optimize syllable density, and above all, write a hit song. These days, I get compliments on just about every song we write together. Claude is the AI for problem solvers. It's the collaborator that understands your entire workflow and thinks with you, not for you. Whether you're debugging code at midnight, building a financial model, or strategizing your next business move, Claude extends your thinking to tackle the problems that matter.

36:27Nathan Labenz: For problems worth solving, get started with Claude at claude.ai slash tcr. That's claude.ai slash tcr. And check out Claude Pro, which includes access to all of the features mentioned in today's episode. Once more, that's claude.ai slash tcr. Do you think that the US government or the Chinese government is more threatened? I think many people would initially say the Chinese government's kind of more threatened because, you know, they want to have like very tight information controls and AI really challenges that. But then I've been thinking lately that like the West in some ways has sort of supremacy to the market in a way that China has not.

37:11Nathan Labenz: And so your like earlier comment about, you know, what about the stock market is sort of a constraint on what human actors can do in the West that isn't quite present in the same way in China. So I guess it's like different shape for one thing. But how would you kind of compare and contrast whose model is more challenged? maybe the question about efficiency curves and the kind of models that get built ultimately because i think in the world where you have the almost like unstoppable democratization of the actual capabilities i would think that that's a big problem for the chinese for the way that the chinese state functions right if eventually any capability that is available at the frontier is available to consumers through an API, is then eventually available through chatbots, is then eventually so efficient that you can run it on just some home computing device.

38:06At that point, I think that does threaten the ability of any state that has an interest in surveilling its citizenry, that has an interest in controlling what kinds of capabilities, what kind of information that citizenry gives access to, and that has an interest in taking away coordination power and sort of like just like general sort of ability to wield violence from its citizenry. And I think in that world, the Chinese state seems to be more acutely threatened at some point. I will say that it's not entirely obvious that the Chinese citizenry is currently very interested in wielding any power that that would give to them against the Chinese state.

38:40I think it's not just for a lack of means or capability that there isn't this sort of like Western romanticized version of like a big uprising against the CCP. But I think eventually the way that the CCP seems to see the proliferation of capabilities like this, that would be a greater challenge to them. But I will say that's not obviously what's going to happen. I think you can also take the much more compute governance view of the world. You can say, well, there are efficiency gains only if we allow them to happen. They rely on making very specific choices on how you use your compute. If you don't, then you can just keep scaling the frontier further and further.

39:14You can run it in limited access regimes. You can run it in sort of in government-controlled data centers. You can vertically integrate supply chains around these AI models such that they never see the light of day. You just use them to build products. You use them to build strategic sovereignty. You deploy them on a state and sort of big corporation level. And they never really get to the citizenry. And that seems like it's a stabilizing function for a regime like China, right? Because you can have the government use them very well. You have a very high state capacity government that would be able to integrate them into all these applications very well.

39:45And you also have a very tight sort of enmeshment between the private sector and the public sector, for lack of better terms. That also just means, well, you can sort of diffuse these capabilities along the tier of firms without needing to sort of diffuse it to a broader market. And I think then you just have a stabilizing effect from the diffusion of these capabilities. whereas in the u.s you might think i don't think the u.s government is going to be as capable of like keeping like a specific like sort of sophisticated level of control and oversight over where the models go and where they don't go the u.s government isn't in the habit of picking specific winners in terms of corporations the u.s market is kind of dependent on making these models more widely accessible and so i think there's just much less of a stable equilibrium for the u.s where there's this like very limited tier of limited access firms that get access to the frontier models that no one else does.

40:32So if that is a stable equilibrium, then I think China can stabilize it around it much more quickly than a much more volatile US.

40:39Nathan Labenz: It's a great point that you make around the, I would just say, legitimacy of the Chinese government in the eyes of the Chinese people. I think that is, it's such a simple point, but I do think it's like dramatically underappreciated in the West that like their government has done a good job for them and they mostly recognize that and are not like, you know, eager to rise up in the immediate future. I also think that what you articulated there, my growing sense is that that's kind of what the Chinese government thinks, is that they are going to be able to ultimately adapt to this and control it better, and that we're going to have a really hard time figuring out how to manage it, but they'll kind of be okay in the end.

41:23Nathan Labenz: Ideas that are not often mentioned. yeah it's interesting right because i think there's this there's this default idea that because like sort of you can look back into how ai has sort of developed over the last years and you can say well whenever the frontier gets to a capability shortly thereafter open source gets to that capability shortly thereafter the efficiency curves are such that everyone gets access to this capability and then shortly thereafter if you have like a gaming gpu at home you can also run the yourself and so you have just like this natural diffusion of capabilities but i think neither of these like translation points are like obviously going to remain in place in the same way i think the transition from closed models to open source models is dependent on anyone like people who are interested in open sourcing models continuing to be able to have access to actually access to enough compute to build this kind of model perhaps access to sort of sufficiently sort of clean and unrestricted API feed student installation insofar as you think that's a big part of that.

42:23If you think that more of the capabilities that are going to be relevant in the future are more downstream of very sophisticated, very vertically integrated, very proprietary RL and post-training environments, like in the Anthropik case with all the specific life science stuff they're doing, then you might also think, well, even having a model that is like six, nine, 12 months behind the pre-trained trend doesn't take you to the biometheus level right away because you still don't have the post-training setup that makes these models specifically good at these things and they're much more proprietary and restricted so i think that part of the oh well open source is always x months behind pipeline might very quickly break and then the efficiency curves thing it might i think that's a little bit more of just like a fact about how computing works and how efficiency gains work and how better chips work in the future but i think there are also complications around how the chip supply chain might look and who is the marginal buyer for computing capacity in the future and is it really realistic that the gap between big server computing and personal computing is roughly as it is right now is that gap going to open up as there are more and more buyers for the high performance chips and so on i don't know either of these so i'm just not quite sure that this sort of deterministic fatalism that people often have which is like as the capabilities continue to grow necessarily ultimately the capabilities that i can run on my phone will also grow in just like time x distance i'm just not sure whether that's going to remain the case.

43:47And I think if you don't think that's going to remain the case, that lends a lot more credibility to the idea that you can't centralize and totalize control over a lot of things that happen in AI. Yeah, I even think in the Chinese case, they feel like domestically open source.

44:01Nathan Labenz: I think they can pull models offline if they need to, they can scrub the internet, and they can even, and I don't know a lot of details about this, but I was struck recently when I was there that the state grid company is like, you know, a big booth exhibitor at their WAIC event. And from that, I kind of inferred at some point in time, if they really. Yeah. So I think like, yeah, I think if you build like dissident GPT and run it on your cluster, that'll probably tell. I think at some point it's like dissident GPT just like runs on your phone. I think at that point, it ultimately does probably get hard.

44:37if the weights are just like if the weights are open sourced and the weights just out there you're not getting you're not actually clawing them back from individual users so i do think like if the efficiency gains are big enough that you can actually run it on personal hardware i mean then you'd have to be confident that you can get i think it's not impossible like we talk about hardware verification tech and whatnot in the context of these u.s china deals and so on it's not the most absurd thing in the world to imagine that in a few years time personal computing devices will just have similar hardware enabled mechanisms as they're called to just control whether you're running dangerous inference on them i wouldn't entirely put that past sort of device regulations as they might emerge in china as a reaction to things getting really crazy like i think it is controllable i think it is controllable if you have as much like full It's that control over the tech that people use, as you say.

45:32Nathan Labenz: Samistat models, perhaps, part of the cyberpunk future. And even that could potentially be reined in. Returning to the U.S. context and our kind of sclerotic government, what do you think we're going to do? We are sort of talking more and more, although still not talking that much, I would say, at the high levels about these issues. I don't you know that the conventional received wisdom is like congress will never do anything maybe that could change but it does seem tough I wouldn't be that optimistic about what they would do even if they did do you have any kind of low-hanging fruit ideas you think the u.s system can pick so I think talking about congress I share your pessimism about is congress going to get anything done I think our window to get something done in congress was over the course of the last year i and a few others and i think dean ball most prominently have written about this a little bit last fall about sure there is some room to have some deal of like frontier safety provisions against the broader set of like preemption of state laws that in theory is sort of politically incentive compatible in this current congress because they'd like to get like the sort of the republican side and therefore the majority of congress would like to get preemption done One, the frontier safety provisions aren't as offensive to them as some of the other regulatory provisions.

46:57There was maybe something to be done there. I think we got to a bill draft or two that were really good along these lines. So I'm a little bit more optimistic about if something had happened in Congress, I think it would have been good. Like the Frontier Act, which is a sort of Trey and Obernolte bill that ultimately, I think, we can now basically say didn't go anywhere, at least this Congress. And I think it's unlikely to go anywhere between the midterms and the new Congress being sworn in either. I think that was pretty good. I think that was as good a bill as we've seen in Congress. I'd have good mandates for independent oversight, good mandates for getting Casey, the Center for AI Standards and Innovation, into a better position to do some governmental oversight.

47:35I think that was a good bill. And I think if it had passed, I think most people would have liked it. But I think now, heading into the next Congress, the politics are going to be much more difficult. We're very likely going to have a democratic house, which means it's that government, when there's going to be a lot of bad blood between the two chambers, there's going to be subpoenas, and there's going to be hearings and there's going to be a lot of ways in which the Democrats use the House to gear up for the presidential election and also to sort of reiterate a lot of and relitigate a lot of the conflicts that have happened in the first two years of the second Trump administration.

48:07So at that point, that just doesn't strike me as a very productive legislative body. So I am pretty pessimistic about getting anything good done in that Congress. Law-hanging fruit, however, well, if the executive wanted to, I think the executive could definitely do some good things. I wrote about it just this week, which is like, we have these independent third-party organizations that have a decent amount of skill and expertise in thinking about the most obvious and most sort of concerning AI risks. I think the Meter Redwood investigation of the Hugging Face incident was well-received for a lot of the ways in which it sort of understood the alignment and control side of the problem.

48:44I think it's not that big of a stretch to just slightly codify and slightly sort of enable this kind of investigation at a slightly larger scale so i think two most immediate things would be a it shouldn't be open ai sort of with sort of just like inviting voluntarily inviting someone to look at what they did when something like that happened it should just be the administration telling them well give access to one of this like list of like third-party evaluators whom we like you can pick them but we'll give you the list of the ones we like and just let them figure out what the hell was going on there and then let them write a report and they'll give the report to us and they'll tell us, did they give us enough access or did they not?

49:24And if they didn't give us enough access, then they'll come back and get the access. So just put on a little bit more pressure to have good incident investigations that cover the entirety of the incident. I think that's one of the low-hanging things you can do with third parties. And I think the second thing you can arguably do is just kind of embed them or kind of have some version of continuous oversight of what the organizations are doing again that's something that this frontier act train over nolte bill sort of had a basic version of which is just like regularly some external evaluators just go in and poke around the lab a little bit they hang out in some of the slack channels they talk to some of the safety researchers they talk to some of the capability researchers maybe they get one or two or three sit-down conversations with the executive and they're like well what's going on here in terms of safety does everything look good and if everything doesn't look good they have an ability to sort of just communicate that information to the administration.

50:12And if something looks really, really bad while they're embedded, they have this immediate escalatory ladder where they say, well, there seems to be imminent catastrophic harm here. We should do something about that. So I think between pressing for incident investigations and encouraging the labs to allow some version of continuous oversight, that's something you can do tomorrow. And I think we should probably just do that. And I think from there, we can think about how to sort of codify that and how to develop that into laws and how to develop into executive orders, how to integrate that into sort of FINRA for AI ideas.

50:40I think we can have a lot of long-term conversations about where to go from there, but this is something we can do now, and we should just do it. Yeah, I like that.

50:49Nathan Labenz: Let's say the president doesn't do that. Interesting thought experiment I've been playing with lately is, could these groups just come together and essentially form a sort of union where they basically say, hey, look, we're not very happy having had six days and three people on site and a very small fraction of the relevant data to do our investigation. We demand better working conditions. Do you think that they could, by coming together and making some demands with a single voice, do you think they could actually get those from a few frontier companies? Well, I think the problem is, apparently, they're just too reliant on the good faith of the AI companies because of the voluntary dynamic.

51:33There is no law, and there is not even a lot of executive pressure that requires the labs to allow for these third-party investigations. And I think, so if you got together and if you did that and you pressed for, well, we'd like to have full access and we'd like to have access to all the Slack channels and to all the logs, and you can come up with a list of things that you might want to have. Then I think there's just still a world where the labs say, well, sadly, we couldn't come to an agreement with the third-party evaluators, risk to security of our IP, risk to integrity of our operations, worry about information leaking to competitors, something, something.

52:06And I think that currently still reads as a fairly reasonable response to that kind of ask. And then the question is, do the labs really have a problem currently if they don't allow third-party investigations? And I think maybe a little bit with their own employees that want some reaction to incidents happening and are not entirely satisfied with just the internal practices. But I would suspect if the third parties can be painted as unreasonable and extractive when they engage in this kind of sort of collective bargaining, that employee pressure is probably not going to be sufficiently high. And then the other source of pressure is, well, is there pressure from the executive to allow for third party investigations such that the developers would have to get in even like a little bit more adversarial third parties that decided on the standards they want?

52:57and I think currently not. But I think if you just moved a little bit towards the direction that I just described, which is like have some administration executive pressure on allowing third-party investigations, then I think whoever is on the list that the administration has could in the next, like establish some standards of how investigations were supposed to go. But I think the first thing you need is some external incentives for the labs to even go with any investigation at all, because otherwise they'll just say, well, in that case, we're just not going to let you in. And I think, yeah, I think there's just not enough of that pressure around just yet.

53:27Yeah.

53:28Nathan Labenz: So much depends on he who must always be named. Another thought experiment, speak of the devil. If I was president, here's something I would be interested in trying. So you red team this idea for me. What if the president were to say to, let's say five companies, could be six, seven, hey, this is getting pretty wild. I don't think I really know what to do. but I think you guys can figure it out amongst yourselves. So you have 90 days to come together, come up with an agreement by which you guys are going to work together to pace the frontier. You'll police it. Maybe you'll use some secure private computing constructs to be able to kind of interrogate what one another are doing.

54:11Nathan Labenz: And you'll have to agree on how that access will work, but you'll police one another. And if you can't reach such agreement or can't sustain such agreement, then I'm going to have to get involved and you're going to like that a lot less. What do you think? Well, I think you should run. And I also think... It might be a little, hey, there's still time in the cycle. What am I talking about? Hydrostition? Yeah. And you can get into the primaries. There are undecideds yet for the primaries. No. Yeah. I think that might work. I think that is also kind of the attitude the administration currently already takes where there's a sense of, well, you guys built this mythos thing we don't exactly get why you would ever do that but now you've caused this problem can you please just like figure out how to fix it like now and then if you don't then something something export controls a lot of pressure and and so on i think that is fundamentally like finding some slightly more structured version of that is fundamentally also the thing that like that's the sort of finra for ai faro safa like sro whatever you want to call the idea i think that's basically the slightly more structured version of it which is like industry comes together and figures out what the standards for how this should work are i think if you put open air and anthropic into a room to figure out what they should what they should do and what they want to do and what they see as the risks and what they think should be done to paste them i think they'd probably do it and i think this would work i think maybe gdm also works i think meta and xai have a very different view of the risk case, have a much more skeptical view of industry coordination, of voluntary industry standards, and of actually doing a lot of things that slow down their capability progress, which, to be clear, is ironic, because something that specifically paced the frontier would give them a much faster path to catching up, so they should structurally and instrumentally be in favor.

56:04But I still think they're very skeptical of that. They have a lot of influence with the administration, so I think they would, A, just be sort of opposed to that kind of broader like all the frontier labs should now figure out what to do and i also think they would just be very likely to influence the negotiations in a way that would just make it extremely difficult to for there to be any common standard because i think if you water down whatever you want to do to the to an extent that would make meta and xai be fully on board with it i think i think that just wouldn't realistically be good enough and then open and the topic would maybe say well this is not good enough and this doesn't fulfill the spec that we've been given by the executive so i think there probably has to be some more substantive guidance than just like you guys figure this out and find some consensus at least some minimal idea of how to do it but yeah i think if you had some more substantive guidance then there is a path for industry self-regulation and i think even then we can bring in the third parties again to verify it that's i think one reason like sort of one mechanism to make her make make industry regulation work the other is just to have the labs check each other's homework and to just have like openly checks out what anthropic is doing on this and so on.

57:13That's a little bit more dicey in terms of industry secrets, but there are precedents for this. It's also not unworkable and not impossible. But yeah, I think if you can get to bridge the gap between the tier two and the tier one, who are, I think, in very different places in terms of safety, then you can make it work. But I think that's the big one.

57:35Nathan Labenz: Actually, two things that have come up a lot recently for me as I've, you know, cause I do go around doing this, of putting these ideas in front of people and telling that you're asking them to tell me why it can't work and common answers that I get to various kinds of safety minded collaborations are domestically. Well, that might be an antitrust violation. So that could be a big problem. And then internationally, you know, even for things that are not at all about exporting chips or chip making know-how, people are still afraid, even on just like basic AI safety research collaborations, export controls, it could be a big problem.

58:12Nathan Labenz: You know, they're very broadly and vaguely worded, and enforcement could be kind of arbitrary. So I guess two questions I have around that. One, do you think it would make a big difference? It seems to me like it would for the president to just come out and say, hey, here are some things that we are not planning to bring antitrust or export control enforcement against. And if they don't do that, then I also have the sense that like, we maybe need people to be willing to have the fight. So I'm, you know, it's not necessarily my place, obviously, to advise all these AI safety nonprofits out there.

58:48But if I were to be so presumptuous, I would kind of say, I think you should go for it and put a

58:57Nathan Labenz: little faith into the judicial system and the fact that we do have like due process, you're not going to go immediately to jail for having done some AI safety collaboration, you know, research project with a Chinese academic. So like, don't censor yourself or don't cancel the product before it even gets started. Like go do it. If somebody wants to pick on you, you know, that'll suck. But like we're in, you know, this is kind of an important time. Like somebody has got to be willing to stand up and have the fight what do you think about that yeah so i think i distinguish between the cases here i think the export control collaboration case i think at that point you're just basically talking about whether you can insulate yourself against sort of vindictive and capricious action by the trump administration and i think there yeah if you think it's worth doing then you should just take the fight to that authority i think that's clearly not in scope of that authority i think the trump administration is probably should probably not use these authorities to crack down on this sort of research cooperation you described i think on that i think i'm with you on the antitrust stuff on like industry coordination i think the problem there is a you It is actually unclear whether it isn't just like a substantive antitrust problem to do like substantial industry sort of collusion on not competing on frontier development.

1:00:16And therefore, like, it's unclear whether this actually has sort of inflating pricing effects or not. But at least in all other domains, it would have like industry coordination on not agreeing not to pursue like further technological innovation usually has like adverse pricing effects that you would really not want. So I think this is arguably in scope for actual antitrust rules. In that case, yes, the administration could quite easily come up with guidance with non-enforcement letters on like, well, we don't plan to bring any action against anyone who coordinates for the sake of AI safety in industry.

1:00:51I just don't think that the administration is actually going to do that because I think the administration has so far enjoyed finding new and sort of novel pathways to be annoying to Anthropic specifically. And I would suspect that the moment that Anthropic decided to come up with any sort of substantive and helpful way to coordinate between different labs to make some like deceleration of, you know, make some pacing happen. I think the administration would just find some way to act against that. So I think the only way you can do that, that saves you from that sort of enforcement is broader industry cooperation, you get XAI and Meta and OpenAI into the boat early, you make it very difficult to target just Anthropic or just labs that the administration doesn't like with this sort of antitrust authority.

1:01:35And then I think you're probably safe. But the problem is, it's just very, very difficult for these organizations to take the fight to the Trump administration. yes there is due process but the ipo conversation we had earlier i think plays into this which is like well do you really want to go like 14 rounds with them in some court and you want to like do you want to bet that you don't get any sort of like very trump favorable judges and as they as they did get on the dc court on the supply chain risk designation for example and i think so many things just can go wrong the process can take so long and you want to ipo in a few like a few weeks months whatever and so if that's your goal then you just don't want to take the risk right now on getting bogged down in any sort of long antitrust lawsuit.

1:02:15So I think the sort of scrappy safety nonprofits should probably take the fight to the administration if it really stands in the way of what they want to do. I can't blame it for not wanting to go into antitrust lawsuits months before their IPO.

1:02:28Nathan Labenz: Under ordinary circumstances, I would agree with that. I do think how many people they have had come out and say, yeah, I think 10 % plus, totally reasonable. It's like, if you're willing to take that risk, I think you should also be willing to spend some time in court along the way, but maybe that's just me Where does all this leave us? It sounds like I mean, of course, this is like the baseline the the base, you know Coming into this conversation the the baseline assumption is like we're probably just going to kind of muddle through the you know the current state of affairs will mostly kind of continue until at least the foreseeable future when something gets even crazier and shakes us out of this equilibrium.

1:03:10Nathan Labenz: Is that basically your view? Like we need another big incident, warning shot 2.0 to really open up space for different paths? I'm just, I think it can be external incidents. I think it just also, we'll see how the next Congress looks. I think it's going to be interesting. I think it's going to be much more about political incentives that will change things in the next like few months my expectation would be that is the main pathway for things to really materially change and be different it's not so much that it's not so much that something has to change about how people view the technology i think they think is ripe for regulation and right for intervention and the demos are getting crazier and things that happen are getting crazier i think there's just i think that's probably there's probably enough happening there.

1:04:00I think the thing that I, it's just, when do the politicians and the policymakers move on this? And I think currently, there's just not that much political incentive to move. There's going to be much more political incentive than Democratic House to keep pushing and prodding and introducing things. And we'll see how the GOP reaction to that looks. And I think the interesting wildcard is how do the presidential primaries look? And I think on the left, there's just going to be a lot of, I think, anti-AI sentiment. I think people are going to talk about AI and AI safety a lot. They're still not going to have an ability to get anything done.

1:04:29So I think that puts whatever action they take all the way into 29. The more interesting thing is what kind of record do J.D. Vance and Michael Rubio want to run on? And I think that is going to be the question about it. I think that's going to be the determining question for whether we see any AI policy action in 27, 28. It's like, do they actually take the view that we can't run on a record of the Trump administration doing nothing about the risks that are getting people more and more concerned? do we need a bill to pass do we need some executive action to happen such that we can't get pinned down on the sort of like broad pro-ai accelerationist position come the elections and i think probably yes that is in their political interest and the question is will they have will they find a way to get that through despite donors and perhaps even the president pulling the other way but i think that is more a political question that has to do with what does the polling look like what does the salience look like what do the midterms look like in terms of ai salience and ai impact on electoral outcomes how do the primary dynamics sort of unfold and where do they leave the candidates but i think i'd mostly look at these political flashpoints of the primary season beginning and current cabinet officials being unhappy with running on the current track record that i would expect to change things and i think that is the way that we do get legislation and actual action in 27 28 and i don't think that's impossible one more u.s question on the

1:05:51Nathan Labenz: build out. It seems like the build out is actually like happening. It was obviously a lot of noise around it, a lot of heat around it. But like my best guess is that this will kind of look like a fracking story where it's like, it happened. You know, we, in a lot of different places, people found their, their right plot of land with the right jurisdiction. They bought off, you know, or, or, uh, they built the parks and the, you know, the stadiums or whatever that they needed to bread and circuses kind of carry the day and it happens do you see any reason to doubt that it's federalism right i think there's just i think there's i think there's so many places you can build data centers a lot of the backlash in the policy implications of the backlash have sort of been i think overstated i think especially the texas moratorium isn't very much a moratorium in any practical sense i think people sort of now now they now they draw up these maps and everything that has a moratorium is sort of colored red.

1:06:48And I think if you make a map of places where you can no longer build a data center, coloring Texas red is kind of disingenuous. There are some minimal standards for what the data center projects need to clear. The hyperscalers will clear them without any problem and they will continue to build in Texas as long as they have access to behind-the-meter power that runs data centers. We might be running out of that, but that's a different conversation. The Midwest is, I think, genuinely sort of anti-data center. It's going to be difficult to get things done there. I think the New York moratorium, at least for the next year or two is more real than the texas moratorium but i think for the rest of the country you can still build in texas you can still build in diana you can still build in the dakotas and there are also like tens of gigawatts already committed construction projects that are continuing on it's going to get more difficult it's also going to get more expensive i think you're going to have to pay more concessions you're going to have to make more expensive deals i think all that is a real effect and i think some of this is going to push some of the bill that into other countries.

1:07:42Some of this is going to make the bill that somewhat slower. Some of which is going to make it more expensive. But there are a lot of states. There's a lot of land. They're going to keep building data centers in America.

1:07:51Nathan Labenz: Do you have a theory for why that hasn't happened with nuclear power plants? Is it just that they're not enough better than the alternatives? Or is there some other reason that we haven't reached that same equilibrium there? i think a little bit less of a you can put this wherever you want to and it just pays the same way i think there's just fewer places you can like sort of you connected to like somewhat local electricity man you connected to somewhat local grids you have like you can't just build all the nuclear power plants for the country in maine or whatever and so i think there's a little bit less of the dynamic of well just like put them wherever they work and also my understanding is there's also with like more federal level oversight over where and how and when you can build nuclear power plants as opposed to data centers which are like you don't have to go through any federal approval process to build data centers anywhere you just have to build the data center and so i think that combination makes it a little bit easier but i will also say i'm just not super steeped into the u.s domestic nuclear build-out conversation yeah i mean i think that federal level oversight is probably a key part of it.

1:08:58Nathan Labenz: And that is a big part of why as much as I, I'm like legitimately scared now of AI, it has moved recently from a sort of, this could get really scary to like, it is actually now scary. I'm still like, Oh God, you know, don't take, don't give me the nuclear outcome. You know, that would, we get the weapons and not the power plants. Like I would just be so bummed about that, that I'm like a little reluctant to go all in on a, federal oversight, even as much as I feel the need. I mean, what's your version of the weapon is I think one question here, right? I think just, I think the thing about nuclear weapons is you can build the entire supply chain for a nuclear weapon without ever generating any sort of civilian benefit.

1:09:43It's really hard to build a model that's just good at winning your geo-associative competition that isn't accidentally also a big economic boon, right? I think like all the ways in which AI systems are really economically useful are so general purpose that it's really hard like you have to go through a lot of effort to not accidentally make them pretty useful economically as well i think the question is like do you get super intelligence in your pocket and i think that's an open question but even if it's just the u.s government sort of procuring super intelligence to use to win against china whatever that means i think that just incidentally still builds a system that's very economically useful so i think much less than with nuclear i just don't think you can divorce the civilian that military uses in the same way so i'm i think in that sense we should be optimistic based on the nuclear example which is like well we didn't stop building nuclear weapons just because we were kind of i mean at least we didn't stop entertaining nuclear arsenals i should say just because we stopped building out nuclear power i think in a somewhat similar way we're not going to stop building agi and like super intelligence and whatever just because we're not because there's some domestic resistance.

1:10:48And in the case of AI, I think there's just going to continue to be the civilian economic spillovers much more easily. So I think in this situation, the strategic impulse actually cuts in our favor. So maybe that's one thing that might make you a little bit more optimistic about it. But not all the way to super intelligence in your pocket, but a little bit more.

1:11:06Nathan Labenz: Yeah, hey, I'll take what I can get. So let's talk about the rest of the world. You have this big report that just came out on transformative AI strategy for Europe. and obviously there's been some discussion and I actually talked to one of your co-authors a bit back about kind of the compute deficit that Europe has and the need to kind of do something to you know be a live player going forward but before we get into like the strategy for what Europe should do what is the worry if they do nothing because I also think like this is probably going to whatever you think might happen to Europe if Europe stays the course is probably what happens to like 70 % of the world's population, maybe 80 % of the world's population by default, right?

1:11:49Nathan Labenz: What does the future look like in your mind for Africa, Latin America, South Asia, etc.? I think it's going to be really tough because I think fundamentally, a lot of the catch-up mechanisms that low - and middle-income countries in very general terms have used and enjoyed and been able to leverage over the last few decades are just deeply incompatible with a world that has both like very advanced ai systems and then ultimately also a lot of sort of automated manufacturing capacity and what that's downstream of that so i think mainly the sort of the most immediate and obvious mechanism was always just bet on the sort of demographic differences we just had like very rapid population growth you had a fairly cheap workforce that you would be able to to use to your comparative advantage and then quickly sort of bootstrap into hosting some foreign firms and sort of exporting some sort of valuable good to the global supply chain in a way that was predicated on the idea that you had this workforce that you would just be able to put to use in a way that would make you a comparatively beneficial country to conduct business activity.

1:12:56And I think I just don't know whether that's going to remain the case. It's definitely not going to remain the case for most aspects of the sort of like menial sort of like services economy. I just don't see a stable way that that sort of that bad sector of the economy really exists once we have very powerful AI systems. I think there are definitely going to be new services jobs. And I think the human preference jobs and you can think about all these sort of like labor market effects in the long run. But I think this idea that you can just like put be immediately useful to global supply chains just by doing labor cheaply in the service realm is, I think, just not going to work out anymore.

1:13:34The question is, is it going to continue working out in manufacturing? I think that has a lot to do with how fast automation goes, how big the efficiency gains there are. There is still a world where manufacturing just gets more and more bottlenecked in a post-AI future, and then it turns out you can at least catch up via manufacturing. Doesn't strike me as entirely impossible. But I think that much just for the general catch-up mechanisms. Then the other question is, what's the stable geopolitical endgame? And I think even if you get to this sort of manufacturing plus like sort of cheap jobs, sort of part of the catch up mechanism, it just seems very difficult to figure out how any country in that spot ever gets any leverage of what happens at the frontier, which is to say they don't get any oversight or regulatory input into how frontier AI systems are built.

1:14:19and they probably also don't have any hard leverage that makes sure that they'll continue getting ai exports and continue getting access to ai supply chains and so on and so they're basically at the mercy of whatever great power provides them their ai models and then maybe within that they can find a somewhat favorable arrangement but it seems very unlikely that they'll get like a stable say and a stable input into that and i think you know i think that just It carries the world into spheres of influence of those that have very powerful AI that are able to export it. A bunch of other downstream questions of that that make this more complicated.

1:14:56How much do you need Frontier AI? How much do open weights play into this? At what point can you build your own digital sovereign infrastructure? But I think at least for the medium term, it is this quasi-vassalage to the Frontier AI building powers that I think is just the most likely outcome for most of these countries.

1:15:12Nathan Labenz: In terms of how people live, do you think that that could create a story kind of similar to the Chinese story over the last few decades where it's like, life is getting a lot better, we're getting richer, we just don't have a say in the overall high-level direction? But at the street level, things are trending up and up? I think in absolute terms, people are going to be richer and wealthier and better off. I think, so I think in terms of the economic effects that is, I think they're just going to be relatively disempowered when it comes to meaningfully shaping the trajectory of the world. And also in terms of having an ability to catch up to however well the sort of the frontier countries, so to speak, are doing.

1:15:56But I think in absolute terms, it's going to be, it's going to be, it'll keep being growth and it'll keep being spillover effects and redistribution gets easier as well. I think on the streets, it's going to be, it's going to look nicer. It's going to be just an economically better scenario. So I think in absolute terms, you wouldn't mind too much. I think the more fundamental question is, what does it say about democratic say and human autonomy and human dignity even that none of these decisions really factor into where the broader trajectory of the history of the world goes? And I think that is a more profound sense of disempowerment that I think we should still be concerned about.

1:16:32But yeah, practically speaking, not that bad. But I think the other part of it, practically speaking, is susceptibility to misuse. And I think that could be extremely destabilizing. So I think there is a current assumption that to guard against a lot of forms of AI misuse and AI loss of control and so on, you need your own AI systems that defend you against that. It's most obviously true in the realm of cyber. I think it is also conceivably true in tracking and monitoring potential deployment of pathogens, the sort of entire virus conversation. it is probably true in terms of scanning and filtering and screening against scams and all these sort of like socially engineered attempts and whatnot and it's of it is probably also true in terms of like safeguarding infrastructure against extortion attacks and so on i think if you expect there to be a world where non-state actors terrorists criminal groups get access to at least fairly capable ai because they're able to steal it because they're able to sort of post train, like terrorist GPT on some open source model, whatever, right?

1:17:30And you also expect these countries to not have any sort of coordinated and assured and widely deployable access to these systems. I'm not sure whether they're going to be able to protect their citizens from AI-driven harm, AI-driven sort of misuse, and also potentially the labor market effects. I think that all sounds like they would be very susceptible to that. And then you can imagine a lot of very destabilizing scenarios, right? If your state no longer protects you from AI-driven harm, then what do you turn to? Maybe you turn to mass migration. Maybe you turn to other ways of structuring your personal security, as we already see in some of the failed states in Latin America, where I think criminal enterprise runs a lot of day-to-day structure in a lot of parts of these countries.

1:18:13I wouldn't think that would be impossible for a lot of these countries as a medium-to-long-term outcome. And I think that also has a very worried. But I think the pure economic story is pretty positive. The eroding the state, the authority and power of the state story is, I think, a lot more concerning. And I think sort of take these together and it's not a particularly rosy outcome. So that's probably 70 % of the world's headed there.

1:18:40Nathan Labenz: And Europe is kind of the one place that can maybe engineer for itself a different outcome. Tell me if you disagree with that, tell me. But I'm going next to, okay, what does Europe want and how does it get it? Yeah. I mean, I think the problem that Europe faces is not too dissimilar to what a couple of other Western countries or like sort of general sort of liberal democracies face as well. I think fundamentally Australia is in a similar boat. New Zealand is a similar boat. Japan, South Korea are in somewhat similar situations. Canada is in similar situations. So I think that plus Europe plus the UK is, I think, the cluster of like US allied middle powers that have a potential trajectory out of this.

1:19:20It still needs a lot of work. I think the fundamental question from sort of, I think there's two fundamental ways to start looking at this. The first way is looking at, well, put aside all the AI things. What do you want Europe's economic position to be? and I think if you start thinking from that you just think well you want to be good at the things that Europe is currently good at you want to be good at some aspects of the manufacturing you want to be good at some aspects of the artisanal good you want to be good at the high state capacity things that Europe currently is good at whether that's welfare states whether that's high levels of security and safety a lot of things that are going well in Europe you just want them to keep going well plus you want to kind of some way to actually revitalize your current economy.

1:20:07And then AI comes into the picture as like, well, that seems like it could either really accelerate that or it could really sort of destabilize that. And then you ask the question of, what do you need AI for in that context? And I think the other part, the other way of looking at it sort of comes to the same conclusion, which is like, well, what does your currently not have? And the answer is it currently doesn't have front AI systems, which turn out to be one of the most important economic inputs of the future and also one of the most exciting parts of strategic and economic competition right now.

1:20:37And the question is, well, what do you do about that? And I think there you quickly realize, well, building these systems ourselves is just too expensive. It doesn't actually work. So the next best thing we can think about is how do we get access to frontier models in a way that is assured and secure and allows us to build what I sort of described as the first approach around. Like, how do we reduce the geopolitical risk of just doing the things we're good at? how do we make sure we have assured access to frontier systems and don't get cut out of this like AI, AGI conversation while we do the things we're good at.

1:21:08And I think these all sort of come together to, you need something to incentivize selling frontier systems. You need something to make the Americans not nervous about selling frontier systems. And you need some productive way to use the frontier systems downstream to make something happen around that. And I think the strategy that we wrote basically answers tries to answer these these questions and i think especially the parts that i most contributed to try to answer these questions i think just very briefly running through this like high level takes is like the first thing is this compute for access idea that i first wrote down late last year early this year and started sort of shopping around with a lot of countries in the world ever since which is now also one of the sort of pillars and one of the main sort of asks of the strategy, which is to say, we built data centers for American, like in cooperation with American hyperscalers.

1:22:00In exchange for the favorable conditions we provide these American hyperscalers and labs, we get a short access to the models that run on these data centers. And as long as the Americans keep giving us the model, they continue to get access to the data center. If their site goes back on the deal and we're cut off from access to the frontier models, they lose access to the data center. This is the sort of incentive part of the conversation. Rebuild the infrastructure and get access in return. The second part is, how do we make the Americans not nervous about doing that? Because, done wrong, this is a security risk, right?

1:22:30You can't run this. We've done it with

1:22:31Nathan Labenz: the UAE, for God's sakes. We should be able to reach a deal with Europe. Yeah, well, I mean, I think the UAE thing is kind of fragile. I think the UAE is kind of worried about, well, what is the future of that? Will there actually be frontier waits hosted on UAE data centers? And I think that's very unclear. I think they're going to run some inference on them, but maybe it's just going to be like haiku inference and not like fable six inference and i think that is an open question so the question is how can we get the security alignment to work out in a way that the americans aren't aren't too worried about hosting the models there and giving the model to the european economy and i think that just has to do a lot with aligning with us on a lot of these security provisions building out the data centers to be just like secure on the cyber and physical side building out sort of kyc regimes with european firms just make sure that the americans don't have any well-grounded national security worries that were put against the incentives from compute for access and i think the third thing is let's get a little bit more self-assured about the assets that europe does have right europe has both sort of broad economic assets and just like a very powerful economy still in absolute terms if not in terms of growth trajectories but still in absolute terms and europe has a lot of assets in the semiconductor supply chain right like asml and size and all these things that play into the play into building building frontier chips and therefore frontier models let's just think about how we can be strategic about that let's set up an anti-coercion instrument of sorts that says, well, if everyone plays nice, we'd love to feed these assets exclusively into the American supply chain.

1:23:55We're willing to align with export controls vis-a-vis China. We're willing to just, you know, we're willing to be good friends and good partners to the U.S. And also, if the U.S. ever doesn't decide to use its ability to cut off frontier models as a means of coercive action, then we're also willing to use the supply chain bottlenecks that we have as coercive action in return. And I think between those three, frontier access is pretty And then you're at a point where you're sort of back to where we were before AGI, which is like Europe still has a lot of structural economic problems. We still have to solve them.

1:24:25But at least we fixed the geopolitical problem of being cut off from the frontier axis. And I think that's maybe step one. I think that's the things that I'm excited about getting done in Europe in the next year or so.

1:24:36Nathan Labenz: What's the hardest part about it? Is it just getting data centers actually sited and built? Or is there other challenges that you think would be bigger than that? I mean, we can have this conversation and there's like a shared understanding that the suggestion I make interface with like a realistic future that we think might happen and that is worth preparing for. This is not the case in many rooms with policymakers in Europe. I think there is deep skepticism of the continued trajectory of capabilities of US built AI models. There is a lot more optimism about the broad availability of open source competitors that can basically do everything as well as the American model.

1:25:16and there is just a more fundamental question around are these models that powerful is it that important is that as big of a geopolitical issue and then there is also the question of well if the models are that important if everything that that i and we and people say is true then like why shouldn't we just build this ourselves that surely can't be that expensive right we'll find like a more clever way to do it the americans are sort of wasteful high on their own supply anyways like we'll just like spend a few million dollars and like surely we can like rip something up and i think just cutting through that which i understand to be just like complete misunderstandings of the material reality we find ourselves in just cutting through that and like making making the point of no this is not actually this does not accurately describe reality you need to think about this in sort of clear-eyed ways that respect that this this thing that is happening in america is real and the americans are fundamentally right about a lot of the aspects of this that's the biggest barrier and then i think if that awareness existed i think there are still political things to figure out like for the asml thing it's going to be some amount of triangulating between the dutch government interest the asml interest the interest of the other member states that's not quite easy in terms of data centers there's going to be some domestic skepticism against american tech firms and working with them that's not going to be quite as easy in terms of the security alignment there are going to be people that are going to be more excited about hedging toward china and just like trying to stay between worlds a little bit i think all of these These are surmountable, very, very easily surmountable, if you just get the alignment and the awareness of what's happening here, right?

1:26:42And I think that's the main challenge.

1:26:45Nathan Labenz: So one of the things I did notice in reading the report was, you know, the authors, you and your co-authors sort of are willing to dream a bit in terms of how the authorities might act, right? I mean, at some point, there's basically a statement that like, doing this in the sort of half-assed or kind of, you know, highly bureaucratic, you know, everything's a committee to nowhere mode that European governance, at least by reputation, often acts would be maybe worse than not doing anything at all. So how do you think, you know, how realistic is it that you can actually get this sort of action? And like, what's the mechanism for doing it?

1:27:29I mean, it's still downstream of urgency and awareness of the situation, really. I think there's always a trickiness in writing for, I think, for national governments generally, really, which is you try to write something that isn't quite within the Overton window of what they're willing to do, but that also isn't so far out there that they'd never do it, right? I think one failure mode is you just write, well, we just have 20 million in the budget, so let's just think of the maximally AI-pilled way to allocate the 20 million. It just turns out it doesn't matter. Either way, you can just burn it.

1:28:03You can throw a party. It doesn't matter. It's not going to change the conversation. And the other failure mode is to be like, well, we're going to be maximally honest about what we think should be done. And we just write exactly that up. And then you guys can tell us, oh, I'm so sorry we were wrong later on. And then it's going to be today to do it. and then we're just going to shrug and be well you know we we told you the honest thing i think that the the art or the trick of getting this kind of thing right and i hope we struck a decent balance is a mat just a little bit more ambitious than they currently are accounting for the fact that they will get more ambitious and that they need some nudging toward being more ambitious and i think that is the sort of that that is the sort of calibration that the strategy tries to reach i'm optimistic about that i mean if i had to give you odds of this strategy as a whole being implemented within the next year it's not that high if i had to give you odds on elements of it making it into serious policy attempts and actually getting set up i think it's pretty high and i don't know which ones of these that they're going to be i could make bets on which one of ones of them are going to be more popular and less popular but i think some of this is going to happen and i think that's the sort of hallmark of a well-calibrated strategy we have done this before in europe like there have been times where europe has managed to very quickly act very decisively in a way that has like delivered results as quickly as it has anywhere in the world i think the thing that i used to work in german policy and politics for a bit especially in energy policy and in the immediate aftermath of the start of the war in ukraine there was an absolutely heroic effort of the german government to buy this fleet of like shadow lng tankers all over the world to get them to transport like alternative gas supplies to germany once the pipelines were cut off there was a justice heroic effort to get like lng terminals built out in some of the more most nimbied parts of the countries and within like six months the german government with all its capacity and all its urgency managed to consolidate resources to we made it through the winter no problem max like complete fine energy no problems at all with just like lack of heating or anything that was like the big doomsday scenario it just didn't happen there was like a massive heroic effort and it just it just worked i think before that i worked in covert policy when when that was happening there was like a lot of political pressure against joint vaccine procurement it was very very difficult to get the negotiations right it was very very difficult to get the member state interests aligned and europe managed to procure vaccines fairly well and i think the vaccination campaign in europe went fairly well we can talk about non-pharmaceutical interventions they get a little bit more of a messy story but i think that all went pretty well and i think europe can't do this europe just has to realize that this is sufficiently important and i think we're not that far from realizing that this is important on a covet slash ukraine war scale And I think if we just get there, then we can definitely make progress on the kind of recommendations we make in the strategy.

1:30:43And I don't think that would be a big problem. Cool.

1:30:46Nathan Labenz: Very interesting. Are there small countries that you think are worth calling out for taking a distinctive and potentially effective approach? I was thinking, I don't know anything about this other than that I know that their sovereign wealth fund, I'm thinking Norway. I know their sovereign wealth fund is A, large, and B, like A, G, I-pilled in its operations. Like they use a lot of agents and they're kind of transforming themselves. But I don't know if that's translated to something like a national strategy in Norway. Singapore comes to mind as somebody that might be interesting. Who else is like doing interesting things out there, even if they're kind of small and carving out a narrow path perhaps?

1:31:29Nathan Labenz: I mean threading the needle, you might say. Yeah. I think Singapore and Norway and the UAE are probably the three you most obviously mentioned. I think Norway could do much more. I think the exciting thing that Norway could do is you could become an inference hub, haven, whatever, for the entirety of Europe. You could, it's a little bit hard to invest domestically with the sovereign wealth fund itself, but you can conceivably come up with sort of like schemes to invest it into European consortia that then invest into compute build Norway. Norway turns out to be a pretty decent place to build a lot of compute hence the first attempt to build a Stargate now a Microsoft data center there I think Norway could be more AI pulled about deploying these resources but I think Norway has a lot of resources that could easily be deployed and pivoted towards that and I also just think the base case of just like investing a bunch of your wealth fund into like basically just like recreating like I mean maybe saying recreating situational awareness isn't no longer as en vogue as it used to be like two months ago whatever but And basically, we're creating a fairly AI-built portfolio and part of the wealth fund.

1:32:33Probably just lets you ride on the coattails of the AI revolution for quite a long time. That probably works. The UAE play is building a bunch of data centers. They're basically just finding a way to turn money into something that is an asset in the new economy. And I think that's a good way to spend a lot of money if you have it. Now, it's incidentally kind of a tough situation to be in that you're that close to Iran and it's that easy to draw and stack data centers. So I think that has sort of thrown a wrench into that plan. but i think structurally that was still a pretty good play if they can manage to build the data centers quickly and if they can manage to secure the next generation against drone strikes and so on i think that's still a play that works i think singapore is another interesting case like a there's also wealth fund investment questions b there is just a massive amount of state capacity in terms of like understanding what's going on and like engaging with that i think like it probably no like parliament with the greater density of readers of like very ai-pilled very insidery publications than the Singaporean.

1:33:28I think the same thing goes for the civil service. It's just Singapore is a little bit more tough because a lot of the Singaporean economy is very exposed to air disruption. And I think if you're Singapore, you just have to sort of hope that the white collar apocalypse doesn't look quite as apocalyptic because it's just hard to pivot an economy that is as big and as like service index as Singapore towards a completely new way of operating. So I think there's a lot of capacity and interest there, a little bit less of an obvious AGI play to pursue.

1:33:58Nathan Labenz: Are there any other countries that you think are well-positioned, maybe more well-positioned than they know, that should be doing something that are just kind of sleeping at the switch? i mean i think australia is kind of awake now but i think for the longest time it was australia because australia is just such an insanely good place to build compute both for just like data center construction energy supply reasons and so on but also for security integration reasons like there's a really deep amount of like national security trust between the australian and the american agencies i think there is a strong understanding that australia would not defect to china in any way that they would be willing to play ball and all the alignment on china of focused export controls and so that just makes australia a great place to run like compute for access data center build out place i think more recently we've seen more of that happening i think that's very good but i think for the longest time that was a sleeping giant and i think it probably still is you could probably still like 5x 10x the data center ambitions and just run the inference for half of the world out of australia and that would not be an overly ambitious thing to do and i think you know there's still a lot to be done there i think maybe the inverse of this is the UK, where the UK has just the greatest density of talent and expertise, both in government and outside, just outside of the US.

1:35:11And it's not quite entirely sure what exactly they're planning to do with it, or if they can do anything with it, just because the broader political conditions of the UK, the skepticism towards US alignment, but also the sort of damaged relationship to the European Union and the rest of the middle powers, it's just very, very difficult to figure out what this incredibly talented cluster of people is actually supposed to do in the UK. So, in a way, the UK and Europe really have inverse problems, where Europe has amazing assets that it could use extremely well to actually have a very live player position in this AI conversation, and it's just really hard to get them to.

1:35:47Whereas the UK has all the awareness and all the expertise in the world. It's just not entirely sure what they should even be doing with it, because at the end of the day, if your starting hand doesn't include any cards that are really good for an AI future, then you can be as aware as you possibly want to. And it's still really hard to get something done.

1:36:05Nathan Labenz: If you are the rest of the world, like let's say you're Brazil or you could pick your countries or maybe you would put different countries into different positions. I hope this doesn't happen. I'm hyperstitioning better US-China relations and some form of collaboration and not carving the world up into spheres of influence. But one thing I've been wondering lately is like, because of course, there was this reporting that the Trump administration was like planning to do something along those lines and tell countries like you're either with us or with China, you know, pick your pick your camp. If you were put in that position as, say, a Brazil or again, pick your country, how would you decide?

1:36:45Nathan Labenz: Where would you go? I mean, I think the more important you think AI is, the less justifiable it is to go with China here. just because there is no Chinese AI expert program right now, right? They just don't have the chips. So I think if you think your economy needs access to AI systems and then you can sort of figure out the rest, then I think you just need to go with the US because only they can give you access to the computing capacity, which is why a few colleagues in... I wrote this paper, which was in itself a follow-up to another paper, which are both called The Closing Window to Win, which are about the sort of American AI export ambitions.

1:37:19and just like china will eventually be better at offering these export deals as china has been in the past in like a bunch of international initiatives that they've run in south america and africa and central asia but currently they're not because they can't offer any data centers in any chips so they can't actually offer a full stack export that can last the u.s ambition so right now i think if you're sufficiently ai built you just have to pick the u.s at some point china can probably throw in enough like non-ai related things that the deal looks a little bit more attractive but just in terms of well is there any sort of like hedging strategy to be had in ai specifically i just currently don't think there is as long as china doesn't have the chips that might change in a few years and i think then the decision is going to be much harder but currently it's basically just well how reluctant are you going to be about buying u.s systems i think is the realistic question that a lot of these companies face and i think ultimately like that is a great position for the u.s to be in strategically and the question is just can the u.s actually offer a deal that these countries will think it will stick to and i think that's maybe the main strategic challenge for the u.s which is like yes everyone is tactically and strategically incentivized to take the deal no way around it but they still don't like getting a deal that they feel like the u.s can renege on at any point in time and so the u.s has to figure out some way to commit to these deals in a way that's credible to these countries building data centers is part of it sort of deep industrial integrations are another part of it But I think that is something that the U.S.

1:38:43just has to think about much more, which is like, even if the deal is necessarily the only deal the other country can take, it might still irrationally defect if you're a sufficiently unpleasant partner to make a deal with. And so the U.S. just has to think a little bit more about how to be like a slightly more pleasant and reliable partner. And I think it's not that far off, but I think, yeah, we should do some. We've got the guy for the job. So perfect. Right.

1:39:04Nathan Labenz: One thing I have gone back and forth on quite a bit over time, because I'm a very AI focused person, of course. And then as, but China sort of, you know, became the end point for a lot of conversations that I was having, I became a little bit more of a China person. I'm still not much of a China person really. But I always had this question of like, how is this strategy where We have these export controls and, you know, like so much of what you're saying really depends on timelines, right? If you believe in super intelligence in two to three years, you got to be on Team USA because there is no Chinese export.

1:39:42Nathan Labenz: I agree with that. At the same time, if that is the path we're going down, the chips are made in Taiwan, you know, it's real close to China. It's real far from us. And I just don't see a world where all this is allowed to reach its culmination point, you know, a la Machines of Love and Grace, where it's like, now we're going to make some sort of, you know, deal with the Chinese that they can't refuse, essentially, and realize eternal 1991 without them just being like, fuck, no, you're not. Like we're taking out the, the fabs. Um, how do we not end up in a world? Like all, all these sort of things, they seem to be taking as far as I can tell, they seem to be taking us to a point where China's going to hit a breaking point and they're going to be like, the fabs are going down.

1:40:32Nathan Labenz: And I don't know how we get around that with the strategy that we are playing. Like we can't defend them. Right. So, I mean, yeah, I mean, it's like a super sensitive asset. You know, it doesn't take a lot to, From what I understand, a piece of dust or a skin flake can ruin a batch, so they can presumably not really be defended. How do we not end up there? I think I'm also not a big US-China geopolitical competition. How do we win this guy? I think I have 194 other countries to focus on, and I think that just hasn't left me enough time to really think about this. And as much as others have, I think very briefly, there's three ways to avoid that.

1:41:11the first is the ai systems just get so powerful that it doesn't like and like you the us is so far ahead that basically escalation around taiwan is suicidal for china even more than like accepting some amount of us domination it's very unclear to me what exact shape of agi is so powerful that that would be the case but i think if you are sufficiently sort of like as i and like really sort of super intelligence killed at some point you might actually think that yeah that's just like a dominating dominating advantage and you just actually can't go to war with a country that has this kind of system maybe that is part of it the second thing is well that's actually maybe the fabs being blown up is just not that big of a deal because yes it obviously destabilizes the entire supply chain obviously that's like the end game in terms of us china competition and who knows what happens then but you know you'll have some indigenous capacity in arizona and you'll have all the chips already up and running so maybe if you're already sort of like halfway into your intelligence exploring by then then it turns out you can just run all of this on the chips that you've already built yes if like tsmc gets taken out like the chip supply in a year really takes a hit but maybe maybe agi can do a lot in a year especially if it gets tsmc arizona and i think the third thing is well china also didn't have indigenized capacity and it's not entirely clear that like going to war with taiwan in a situation where they already think they're behind the ai supply chain is the best way to escalate escalate the conflict especially if there is still like some tsmc capacity in some way or shape or form indirectly ending up in china if that is the case then china might just think it's best like catch up hopes revolve much more around like domestic industrial integration and you know like that's like the least agi pill version of the future or it's just is it really worth going toward this specific point where the americans have this like decisive technological lead on agi and the chinese diffusion play and the semiconductor engineization play hasn't really worked out yet can we just like kind of work in their shadows a little bit more and digitalize more of the capacity and then deploy later on not sure whether that's the best strategic take but i think that might also be one strategic approach they take but i also think it's a massive vulnerability and i think like i think any like reasonable agi end game has to account for the fact that the Taiwan situation just might blow up in our faces that I think a lot of them aren't.

1:43:36Nathan Labenz: Yeah. Again, to talk about threading the needle. How about, this has been awesome. How about maybe a little lightning round to close? Yeah. You mentioned hedging and also in the context of the Norway sovereign wealth, like, you know, buying the right equities to get through the AI transition in a successful way. a challenge I've been wrestling with a little bit lately is the Tyler Cowen challenge of if you're so doomer, what are your shorts? I've been trying to come up with, is there an actual answer to that question? Is there some way, and I'm not a total doomer, but I'm like, I think he should be taking it more seriously than he is.

1:44:17Nathan Labenz: So I want to have an answer that either, you know, I either hear my shorts or I really tried and I can't come up. And that's kind of where I'm at right now. I cannot come up with a way where i think i can get rich in the doom scenario do you have any suggestions to answer tyler yeah i think just i think my my general sense of that is just like tyler imagines a much more continuous and smooth on-ramp into like actual doom that makes it so that like sort of just betting on volatility and betting on like near misses and like pretty catastrophic disasters that aren't quite doom make make a lot of sense in that world and i just think i'm not sure whether that's true i think a lot of the ways in which things go badly are just like things go extremely well in the market just all the way until they go really badly and then the only situation where you cash in is when you're dead and i think i think that's the least convincing part of his argument to me this idea that like normally you'll get all these near misses and they're already all so like all an expectation created the stock market and i think there is just a very reasonable like doomer view that concentrates basically all of the probability mass of doom of like things going well all the way until doom in fact if you look at a lot of the sort of canonical like doom scenarios and like people who have talked about like serious like existential risk and catastrophic risk many of them give the scenario of like everything's like looks like it's going really well and strategically great and economically great and looks kind of weird but it also looks economically great just until the point where there's like the takeover happens or the big incident happened i i think that's the main problem with that and i i mean I think also just like many people just have coherent worldviews that they also just don't bet on and that they just don't take financial bets on even if they're committed to them.

1:45:52I think that's a sort of like more boring meta contention to that. But yeah, I think my main contention is just like, well, I don't think it's a smooth distribution of probabilities. I think a lot of this is just like either this goes very well or it goes very badly. And there is not really a lot of world where it's like volatile and goes kind of badly for a while and then kind of well for a while. And so I think I also don't have a good trading strategy, sadly.

1:46:12Nathan Labenz: yeah okay well i appreciate you for thinking it through do you maintain a p doom number i'm sure you've been asked many times i haven't heard you ask though do you have an answer i just think it depends so much on what you include in doom if it's human extinction it's very very low if it is if it includes like all of the sort of catastrophically risky scenarios including gradual like sort of like the end stage of like gradual disempowerment and stable heteritarianism and that kind of thing. I think it's probably around something like 10%. But I think that really is only if you account for the sort of like, political doom outcomes in the broadest sense.

1:46:51And I think my probability of like, technical extinction is substantially lower than that.

1:46:57Nathan Labenz: So you're counting in doom, like, we live in a sort of Chinese plus plus state, and life is pretty good but we like don't have political freedom yeah i think like i mean maybe life isn't even particularly good in a lot of consumer levels it's like extremely disempowered extremely low human agency extremely low human economic participation call it the permanent underclass if you must but like i i think in the actual permanent way right i think like you shouldn't like include too many just like all proselytically sort of bad outcomes in the doom number but i think the sort of old portfolio of like existential like long-term risks used to include things like stable authoritarianism and like stable economic disempowerment and to the extent that that's actually like a locked in path for the human future that you can't see a conceivable break out of i think i'd include that in doom in the broader sense and then and then yes but not just like oh well the economy kind of sucks so my p doom is like very high because i really think the economy's gonna suck yeah okay maybe something i should have asked earlier but i think i know the answer everything you're projecting assumes that like robotics really works it doesn't necessarily have to be humanoid, but we're going to get highly flexible robotics that can be deployed in all sorts of contexts.

1:48:11Yeah, I think. I mean, it's going to take a little bit longer than I think. I don't see a super crazy industrial explosion very soon, but I think eventually this is an engineering problem, this is a scaling problem, and at some point we're going to scale it and at some point we're going to resolve the physical bottlenecks. It's going to take longer. So physical bottlenecks are going to matter longer than software bottlenecks, for example, that eventually they seem eminently resolvable.

1:48:36Nathan Labenz: So does that mean, let's say 2030 is the over-under for when people start to have domestic service robots in their homes? Would you take the over or the under? People start to have in 2030? Yeah, I think that sounds roughly right. I think it might take a little bit longer than that just because of idiosyncratic, psychological, and political resistance. But in terms of technical maturity, that sounds about right to me. Yeah. Okay. How does all this change as compute goes to space? Well, it just puts... So I think there's two versions of compute going to space. The first is space is one of the places where we can put compute.

1:49:17I think that's going to be the case in 2029, which is going to be like, there are going to be some data center setups that are going to be worth putting in space. Inference more so than training, for example. And I think it's also just going to be gated by launch capacity. So we can't put all of our compute starting in 2029 in space. We might also conceivably have different chip supply chains for chips that are suitable for going in space and racks that are suitable for going in space and terrestrial chip deployments. So some compute goes to space in 29, I think. That changes some things. It concentrates some more computing power effectively within the US jurisdiction.

1:49:48It makes SpaceX AI a lot more powerful. It makes launch site governance a little bit more relevant. These are all interesting marginal shifts in how the conversation moves. That's part one. part two is what happens if every marginal chip goes to space instead of to any terrestrial data center and there's basically no terrestrial competition for data centers anymore and i think things get a lot crazier then i think anti-satellite weapons become a really important part of like mutual of like deterrence and geopolitical stability for one because that's the only way you can threaten the deployment of like a superintelligence system because it is in space and if you can't shoot down the satellites then then good luck you're not stopping the you're not stopping the superintelligence.

1:50:26In much the same way that the AI2040 plan talks about making data centers bombable and visible and allowing for this intervention into this kind of sabotage. In the same way, in that world, we just want satellites to be hittable from the ground and the compute to be vulnerable for geopolitical stability reasons. Another effect is all the control concentrate.

1:50:44Nathan Labenz: Do you think that's the default scenario? My understanding is we could probably shoot down satellites without too much trouble. We just don't really, but we can, right? i mean yeah i think the question is who can like the united states yeah other countries perhaps not and i think they need to develop the capacity to do that i think there are ground probably i would think china has it there is some other like the french have the beginnings of a program the indians have beginning the beginnings of program there are already programs it's not that no one can but it is in the same way that nuclear power needs to have second strike capability I think there is a geopolitical stability sense in which a sovereign nation might want to have anti-satellite capacity, and not all of them do just yet.

1:51:28And the other question about this is this crazy Kessler syndrome conversation where there is some amount of mutual deterrence of ever shooting down satellites. Because if you get to the point where you have that much debris in space that keeps creating more debris because things keep colliding and so on, it's just going to be really difficult to launch anything into space at any point in the future. So insofar as everyone is kind of disincentivized from doing that, in the same way that everyone is kind of disincentivized from creating nuclear winter or something, I think that also makes the satellite weaponry math a little bit more difficult.

1:51:59I think the other part of this goes back to our middle power conversation. We talked about these compute for access deals and deploying data centers and so on. That all hinges on this idea of the U.S. is interested in building data centers in other countries because it wants to build data centers somewhere. If the U.S. builds all of data centers in space instead, then the incentive of putting the data centers into the host countries is just so much lower and i think as a result that does sound pretty bad for a lot of these compute-based strategies so i think the most actionable and meaningful consequence of the prospect of data centers going into space is the compute for access and the compute build-out strategy that middle powers are starting to pursue has a time limit it stops working at some point in the somewhat near future and you should start thinking about what your endgame beyond the compute thing is in case the space thing works out as sort of as like SpaceX imagined systems.

1:52:48Nathan Labenz: A more terrestrial concern. Why? I mean, we talked a little bit about like bottlenecks. So I don't mean, you know, an inhuman kind of inertia around why adoption hasn't happened as much as it obviously could in theory have happened so far. But if you take the flip side of that and you just look at like the AIs and their capabilities, clearly there's something that is missing relative to, you know, the experience of hiring a human to do work. Right. It feels like that's thinner and thinner all the time, almost to the point where I'm like now having a hard time putting my finger on like, what is it about like fable five, one or Astra that's actually worse than hiring a human?

1:53:31Nathan Labenz: Do you have an answer that is like what that is and you know, what, additional marginal capability gain you might expect to actually create labor market disruption? Yeah, I'm not sure it's capability gains at this point. I think it's integrating with proprietary data. It's integrating with proprietary workflows. I think obviously not the entire task profile suite of humans is currently covered by models, but I think in specific tasks, they're better at humans. And I think you can drop them into specific task profiles at least, especially in software engineering i think and also in some other like sort of like general white collar activities and i think the question is just like the labor market just takes its time to rearrange around that and to get to the sort of like augmented like sort of mutually beneficial because you can't just like you can't just fire the guy who's sitting at his at his desk and like plug in astra instead you need to be slightly more sophisticated in well we now need like instead of three guys we need like one guy who tells the agent to do the things and the agent to the tasks but that guy has to be like a little bit better at all the things that the agents can't do and so i think it requires some institutional organizational reconfiguration and i think that just will take some time but i think the capabilities are there that i would expect them to have this sort of not displacing but at least disruptive impact that changes how teams are built and how productive they are i'm a little bit less sure that this specifically leads to displacement in the short term i think it also does create additional demand and does create additional things that human workers can't do on the margins.

1:54:57But I think in terms of disruptive effects and reconfiguring effects, I would agree the capabilities are there. It's just latency and lag and bottlenecks and frictions.

1:55:07Nathan Labenz: What do you think would happen in a hypothetical world where Tesla decides to license its full self-driving and within 18 months or so, let's say we make a priority of it. So we're into a little bit of a fictional scenario here. But all of a sudden, basically all the cars drive themselves and the four or so million Americans who make their living driving don't are needed to drive anymore. That seems like one, you know, pretty clear displacement story that very well could happen. Like, do you think the economy can absorb those people? Where do they go? It seems really tough when you actually get down to like, okay, this dude has driven a truck for 25 years.

1:55:46Nathan Labenz: He's not ready to retire, but the truck now drives itself. What happens to him? Well, I think part of it is going to be political responses, wage insurances, reducing their times. I think there's also going to be a political necessity to add some frictions to this happening. I think, frankly, there would be human-in-the-loop laws. There would be sort of like, you have to have a driver in the sort of like Harley and like New York and whatever idea of like, well, even if the thing drives autonomously, there still needs to be a driver in the seat and whatever. I think we'd see a lot of these political reactions and frictions introduced before anything happens.

1:56:19Yeah, that's one part. I think the other part is yes, eventually, I think the economy would probably be able to absorb at least a decent percentage of that. Not necessarily in better jobs, not necessarily in better paying jobs, probably in worse jobs. But also, I will point out that in that specific story, you found one of the very few jobs that clearly just have one specific task to them. And no mutually synergetic way of engaging with the one technology that one-to-one replaces a specific kind of worker. I think most automation and like AI specifically just isn't like that in that it just like gets at specific tasks and leaves other parts of the task profile up so that it lends itself to a sort of at least like in the medium term, almost like augmented coexisting structure in a way that like specifically driving isn't like.

1:57:05But yeah, I think that will be very hard to absorb. We'd see a lot of political frictions as a result. We'd have to cushion a lot of it, which is like social spending. And I think some of them would find jobs. Most of the jobs would be worse.

1:57:16Nathan Labenz: your idea not your idea but your reference to the idea that somebody might be required to sit in the car even as the car drives itself reminds me actually of a novel that my dad wrote about a pretty dystopian but highly ai enabled future where everybody is kind of out of work but they need the dignity of work so they're required to show up and stand around all day they're known as standers in his imagination. Yeah, I just really hope we don't get there. I think that's like, yeah, yeah. Well, fingers crossed. But I think, you know, you could make the cynical observation that some jobs in the real world already are kind of like this.

1:57:54But yeah, we'll hope we don't get there. A few bullshit jobs out there.

1:57:58Nathan Labenz: But if people are still with us two hours in, they'll be interested in your thoughts on this. What do you think America should be looking around the world to learn? one thing that I came away from China really thinking about is there's a lot of upside to surveillance don't want to have it for a lot of different reasons but I'm kind of like geez it really sucks to leave all that upside on the table is there such a thing as surveillance with American characteristics I'm interested if you have a thought on that and also just like what other things when you look around the world do you feel like the U.S.

1:58:33Nathan Labenz: not just like should envy because i don't think we can like copy the high-speed train from china but what what should we actually be trying to import and realize our version of so i think on surveillance it's really hard for a specific reason which is i think i'm just very worried about perfect enforcement of laws i think the american laws specifically just aren't made to be nearly perfectly enforced if you enforced every law on the books in america i think this would just be a draconian oversight regime i think i understand people's motivation of like well this is like massive like perfectly disincentivized any sort of illegal action but man a bunch of things are illegal and i think like a lot of like punishments and a lot of the sort of like criminal codes are like specifically structured around the idea that you catch one of every like 100 or like 10 or a thousand criminals and then the deterrence is calibrated to that the downside of surveillance to my mind and i think more broadly the downside to like more ai integration i think the most well-taken point in the flock debate as well is we're just not legal systems aren't set up for perfect enforcement and just like surveillance maxing even with american characteristics is just such actually a pathway to perfect enforcement of laws that were never meant to be perfectly enforced so maybe we can talk about that once we've like completely sort of revamped the entirety of the criminal code and whatnot and like the practices of enforcement around that before that i'd be extremely worried about going that far what's important from the rest of the world Hmm Hmm.

2:00:01America is a very idiosyncratic and very specific place that has done very well with a very weird balance of institutions and economic activity that have, for some reason, served it very well. And I think usually when American politicians look around the world into Europe specifically and they try to import one very specific part of a society, they sort of underrate how much of that society is just in a completely different equilibrium and balance. that just like if you just transported like specifically welfare spending from the nordics i think that you know you'd have to do things with the tax system that disincentivizes a lot of other commercial activity that then in itself just like forecloses other avenues like providing the same services and i think the same thing is true for the political system where like yes there is something to be said for the stability of a party and parliamentary system that isn't quite as that doesn't swing back quite quite as much between administrations but then you also get a much less decisive government that is able to do much less and that gets paralyzed into gridlock a lot more and i think there are countries whose concepts and structures are wholesale perhaps preferable in the way that they deliver services and outcomes for their citizenry to america perhaps not i think that's a tricky conversation but i think i basically don't know a lot of like high level really good things about how things work in a very like structural like political level in any other place in the world that you could import into america without disrupting a broader part of how this country works.

2:01:24So I'd be very skeptical of doing that sort of piecemeal.

2:01:28Nathan Labenz: Does that mean we're living in the best America? Because it sure doesn't feel like we're living up to our potential in that way. I think it's Pareto optimal in some ways. And I think that's different from the best, right? But I think like everything that you would improve would come with a trade up against something else. And I think I'm very skeptical of this notion that there is, there are like clear Pareto improvements to the way that America starts. I mean, you can scroll through the like institute for progress's website on like policy interventions and you'll probably find like 10 like marginal fixes to laws on the books that would just make this clearly better and some of them are inspired by other countries but i think meaningful changes to how the country runs are probably not tree to improvements but are like tricky trade-offs that i'm not sure would sort of on balance make the country run run better which is not to say that like this is this is the sort of shining city or any other country is the shining city it's just that, man, it's all trade-offs and it's already difficult to get right.

2:02:20And I'm just not so sure that they're like easy fixes too much of anything. Yeah.

2:02:25Nathan Labenz: Last big question. We've talked about a lot of different challenges, obviously, and, you know, vexing conundrums of all kinds, conundra, conundrums. What would you say are like the most important needles that we need to thread? You know, what are the top couple, few things that you think are kind of absolutely most critical? And then I'd love to hear just what do you think life is going to look like on the other side of this in your kind of 90 % where it goes well? Like why, you know, for people that are like this whole AI thing, why do we even do it? Isn't it just stupid? Like paint the upside picture to inspire that, that audience.

2:03:03Yeah. I think I'm just not going to fix the problem of the AI story of the AI labs, like big narrative concerns. I think I'm just, I just hope we can kind of continue on the sort of positive trend that i think history has been on for the longest time i don't think i take that much of a sort of fatalistic view about the current or like the past trajectory of society i don't think we need agi to bail us out of much of anything i just think it's the next thing we do it's the next thing there is it's the next big innovation it's the next like innovation that we'll need to stay on track with these sort of like compounding economic growth that we have i think in a lot of ways it's going to look very crazy i think some of it is going to involve space some of it is going to involve robots some of it is going to involve a lot of automation but i think some of it is also just going to involve like fairly prosaic future economic growth that just makes us all a little bit richer and a little bit wealthier and our institutions a little bit more functional by the day just as they have been at least since the start of the industrial revolution i think that's that's my main hope and i don't want to i don't dream much bigger than that i think things might get a lot crazier than that and then we'll have to find ways to deal with that but i think that is my hope and my dream for the future in a decade or in two decades or in five years if things go very fast and in terms of what do we have to do for that i think just like genuinely just muddle through like make sure that the balance of power works out and continues to work out and the balance of wealth continues to work out well make sure that the labs don't pull away in terms of power and control from the u.s government make sure the u.s government doesn't centralize and control the entire sort of like flow through of intelligence through the world make sure that some other countries have a stake in this both economic and in terms of just like power and influence and leverage and just you know whenever things seem to go off off off the rails in terms of there's too much power amassing in one place and things look like they're going wrong pull it back a little bit again keep it on court i think i i think that's what i want to do and i think it'll just be like a long exercise of doing things like that on the very small margins and i think then we're probably going to be fine well that is an unreasonably reasonable

2:05:04Nathan Labenz: worldview and I appreciate you for spending a couple hours sharing it with me today. This really, I think has been an excellent conversation. So anything else you want to leave people with before we break? No, I think I enjoyed it very much. Thank you so much for having me. It was great. And I'd like, thank you for being part of the cognitive revolution. Thank you so much.

2:05:35Thank you.

2:06:09Thank you.

2:06:33Thank you.

2:07:07Thank you.

2:07:32Thank you.

2:08:04Thank you.

2:08:53Nathan Labenz: If you're finding value in the show, we'd appreciate it if you'd take a moment to share with friends, post online, write a review on Apple podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guests and topic suggestions, and sponsorship inquiries, either via our website, CognitiveRevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts, which is now part of A16Z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting.

2:09:27Nathan Labenz: If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at AIpodcast.ing. And thank you to everyone who listens for being part of the cognitive revolution.