China's Not the Problem. We Are.

24:03Ross Douthat:Are China's leaders looking at AI through that lens and worrying about, you know, the AI girlfriend, AI boyfriend future? Definitely. They are very worried about that. And in fact, they are already rolling out policies and regulations around AI boyfriends and AI You know, it's so funny. They have a very sort of negative view of wasting time, basically, of what they see, the folks in Beijing, what they see as sort of nonproductive activity. And in that earlier era of a tech crackdown, you know, they saw video games as not really part of the Chinese vision for a, you know, high growth, technologically powered future when everyone's at home playing video games.

24:51And they also cracked down on the education market. So there was a lot of private tutoring, ed tech startups were sort of sprouting up. And they saw that as also kind of wasteful because it was sort of a race to the bottom in terms of preparing for exams and feeding into that kind of cutthroat academic environment. So I think right now we're seeing something similar happen again with worries that AI companions could end up being a big time sink for Chinese youth when they should be engineering the future and building the startups and the future Chinese versions of SpaceX, for example.

25:26Ross Douthat:But is there also a sense that this is the solution if China never fixes its birth rate, that robots are just the way that aging low birth rate societies compete? Is that also part of the theory or the mindset? Definitely. That's a big part of the story. So China has a shrinking workforce. I think their labor force size peaked actually over a decade ago. And they're heavily dependent on manufacturing. They don't want to let that go. They see that as the engine for the whole economy. So how do you reconcile those two factors when people don't want those factory jobs anymore and young people want sort of different jobs?

26:10And there's just not enough people to fill the factories. One solution is robots. One solution is to increasingly automate factory production to put robots of many different kinds, whether they're your classic six-axis industrial robot arm.

26:27Ross Douthat:The classic six-armer. That can lift up a car in one go. Or now this big push with humanoid robots is seen as being yet another potential solution, if not a perfect solution, to this ongoing labor issue. So China wants to continue to become more and more competitive, to move up the value chain, and to make better and more high-value stuff. But they don't have the workforce. So AI and robotics is seen as the way to fill that in. Yeah, it's interesting just thinking about, you mentioned like robot waiters, right? So one thing that has been sort of encouraging, I think, to people worried about job displacement in the U.S.

27:12Ross Douthat:is the extent to which robotics in restaurants, fast food places, supermarkets, and so on has not so far radically displaced human workers. And in fact, places like McDonald's and Starbucks that have tried to sort of really, you know, move to kind of automatic ordering and so on have often found themselves sort of maintaining human staff beyond what they expected or expanding human staff even. In a context, though, where the Chinese birth rate is maybe two-thirds the U.S. birth rate at this point, depending on which stats you look at, you're just in a different landscape, right? Where maybe you're worrying less about whether the robot waiter displaces workers and more about whether you have a waiter at all.

27:58Ross Douthat:And so the robot waiter is welcome and necessary. I mean, that seems like it could be a big point of sort of the divergence ultimately between how the U.S. and China relates to robots. Yeah, definitely. It's like you're going to have to err on this on one side or the other. You're going to have to err on the side of going too slow. And then you may not have the ability to do all these things because there's not enough workers there. Or you might err on the side of going too fast. And I feel like that's the concern in the U.S. more. Let's pull up to the back to the AGI superintelligence question.

28:33Ross Douthat:How do you think China's leaders actually think about the American fixation or the tech world, Sam Altman, Dario Amadei fixation on AGI? Is it two options? You can tell me if there's a third, right? One option is that the Chinese basically think that our tech companies are high on their own supply, that there is not, you know, that there's never going to be some insane return to superintelligence. And it's always going to be fine to be, you know, three to six months behind, but then you have catch up. Another option would be that China is actually worried about superintelligence and is basically trying to figure out what are our contingency plans if the Americans seem to be pulling much further ahead.

29:21Ross Douthat:Do either of those describe China's mindset to the extent that you can sort of read the tea leaves in Beijing? So, I mean, one, you know, sort of interesting corollary question is, is China trying to do an AGI Manhattan project somewhere buried underground in a bunker with data centers that can't be seen by satellites and powered by... Yes, are they? And my inclination is no. Do you think they could do something like that without the U.S. being aware of it? So I don't think that they would be able to do that without the U.S. being aware. I think that it would require such a scale of production, of amassing resources and construction that we would detect something and we would start to wonder what is going on.

30:10And I mean, we're already watching everything about the nuclear buildout, for example, in China, nuclear weapons buildout. So I would be very doubtful that we would miss something of that scale because you really would need massive scale in terms of computing energy to power something that would be like a Manhattan Project for AGI.

30:27Ross Douthat:So they're not secretly trying to win the race. Whatever they're doing, they are sort of accepting this position of being in our draft on the racetrack or whatever metaphor you want for now, right? But is that just making a virtue of necessity? Or do they think that we're diluting ourselves in our race to superintelligence? I think they just see the technology quite differently and they just don't have that kind of transcendent view of technology. I think that you can see this in other approaches that they've taken to the Internet or to the IT revolution, which they were obsessed with as well.

31:11Um, so they were really focused on just trying to integrate the internet and IT infrastructure into just basic services, education, healthcare, government services. And I think they see something similar with, with AI now, you know, one thing that I, one kind of thought experiment I often think about is what would be the signs that they were trying to do a secret AGI program. And one of the signs, I think, would be about those NVIDIA chips that I mentioned earlier, where right now Trump has relaxed some of the export controls and allowed H200 NVIDIA chips to be sold to China. Those are better than what China had gotten before, but not the very best.

31:55And China has basically said, thanks, but no thanks. The AI companies, to be sure, in China really, really want those chips. But here's the divergence, because Beijing, they don't necessarily want to be dependent on the U.S., and they want to bolster their own semiconductor program. So if they were really sprinting today for AGI, I think they would have gobbled up those ships as quickly as possible, not knowing when that window might close. So that is one sort of indicator that they are kind of seeing this as a medium to long term bet.

32:27Ross Douthat:So there might be people at DeepSeek who believe in the superintelligence future more strongly than people in Beijing. Yes. Yeah. I think the AI... The closer you are to the machine god, the more its voice whispers in your ear, right? That's right. Yeah. I don't think Beijing is AGI-pilled. What about espionage, which obviously played a big role in the early Cold War arms race with nuclear secrets? Is there an equivalent sort of spy-based solution for China if the U.S. seems to be pulling too far ahead? So there is something called distillation. And that's where you take a weaker model and you actually train it on the outputs of a stronger model.

33:13And distillation is a common practice for AI developers when it's done with full knowledge and full disclosure and total authorization. What seems to be happening now is some of the Chinese AI labs seem to be distilling on American AI models without authorization. And they're using, it seems, a number of different sort of proxy accounts so they can get around efforts to block these campaigns.

33:40Ross Douthat:But that doesn't require stealing secrets from Anthropic. It just requires using the Anthropic model in a way that you're not supposed to be able to use it. That's right. It's sort of its own category. It's not quite like outright IP theft. It's not like taking the source code from Anthropic or OpenAI. It harkens back a little bit to an era where Microsoft was always trying to cut down on black market copies of Windows and Microsoft Office. Does it work in the sense that like, can you just have a Chinese clod distilled that works as well as clod? So it can help somewhat, but you need to have that foundation to start with.

34:22So I think that this is probably one area where it'll be hard still to get concrete data on exactly what the net effect is. But I would say that if you or I were building a model from scratch, we would not be able to use distillation as a way to catch up to the frontier. If you were one of the better Chinese AI labs, you might be able to use some of this to improve your model, especially on areas where you're weaker, like on coding, for example. You might be able to use Anthropics Claude models to support your long-term coding capabilities. So there is that aspect to this whole AI race.

35:37In a world where there is some kind of takeoff,

35:41Ross Douthat:And I should say, one of the theories that animates the American AI companies is the idea that at a certain level, the AIs start training the new AIs and you get this kind of acceleration where suddenly being three or six months behind, it becomes impossible to catch up. Again, this would be the theory. Suppose that starts to happen. Does China just invade Taiwan? Well, seriously, right? I mean, it's just a kind of fascinating circumstance that you have a kind of arms race. Maybe China doesn't think of it as an arms race, but it is sitting next door to a central hub in the supply chain that makes the arms race possible.

36:30Ross Douthat:Right. Right. Like, is that the natural Chinese move in the event that they seem to be falling incredibly behind? So I think, ironically, if that were really starting to happen, taking over TSMC would be a move too late because the chips are already made and installed and are already running and training the models and feeding into this feedback loop in the United States. So at that point, all bets are off. And you're kind of out of options for what to do. The big question here is how fast that can happen. And whether this could happen without being detected, you know, there's always speculation about, you know, is there a version of the latest AI models that hasn't been shared or even disclosed to the public in, say, the U.S., you know, or maybe even in China, where they have gotten the inkling of this recursive feedback loop that will lead to the super intelligence explosion.

37:33So that question is sort of hard to know. And then how quickly can you actually get there?

37:39Ross Douthat:But I want you to be prescriptive for a moment because we're having a summit. We've been talking about sort of what China is doing, how China is thinking and so on. What does all of this mean for the United States in terms of our policies? Does it mean that we should treat China as a fundamentally more benign actor than our current policy treats them as? Or is it an indicator that, in fact, our policy is working by shaping a Chinese perspective that is not as engaged in the race as it could be? Yeah, I think at this point, what we should do is take a step back from this all out race framework, because I think right now that race mentality is driving a kind of recklessness, I would argue, from the American side to bring up like the threat of Chinese AGI.

38:36you know, we should think about that. But I don't think that that's what they're so focused on. So but if we're only focused on that, that means we need to get rid of the guardrails, we need to not bind ourselves, we need to not have any kind of regulation or restrictions, we need to have as many data centers as possible everywhere. And I think right now, that approach is starting to run into some some problems in the United States. And, you know, whether you're talking about the backlash to data centers, or you're talking about now some of these models getting so capable that they might not be at, you know, whatever AGI level, but they are at the level potentially of causing greater damage, either in terms of cyber attack capabilities, or maybe even in terms of augmenting what a relatively unsophisticated group could do with bioweapons.

39:26So there are all these sort of questions that the AI community has been talking about for a long time. But certainly for the Trump administration, if you recall, you know, J.D. Vance's speech last year, where he said, basically, we should not have hand-wringing over AI safety, slow down the progress of American AI development. In other words, in this trade-off, and he viewed it as a trade-off, we should err on the side of going faster rather than putting on a seatbelt. And I think now we're reaching that point where we need to think about still making progress as fast as possible, competing with China, making sure we do have the best AI models so that we can keep.

40:05But does it have to come at the expense of wearing a seatbelt or having some basic safeguards?

40:12Ross Douthat:Would you also suggest that the U.S. should adopt a more Chinese vision of the goal of diffusion and sort of building the best possible AI-enabled technology right now? Because, I mean, a different way to frame this is that the U.S. and China are in a race, but China thinks it's running a race to build the self-driving cars and the robots that every single country in the world will use. And the U.S. will be stuck sitting here with its pretend machine god while China, you know, sells to India, Africa and Latin America successfully. Do you think the U.S. in being less breakneck should also be pivoting to a strategy of essentially integration and sales?

40:59Yes. I think we need to focus a lot more on deployment. One of those areas is actually open source, which because of the commercial incentives is not a high priority for the top American AI labs, right? They're focused on selling access to their models through subscriptions, through APIs. And the thing is, that open source approach has been really, really powerful for these Chinese AI models to gain adoption, not just in China, but around the world. And so it feels like right now the U.S. is seeding a really important channel of competition when it's so expensive. It can be the most powerful AI model, but you don't want to pay for it.

41:41That can put limits on your growth.

41:43Ross Douthat:Do you think you get that shift organically if there is a slightly stronger regulatory hand? Because, again, the U.S. does not, we have industrial policy, I'll put it in quotation marks, right? But we don't have the kind of steering of economic strategy that China has, right? So it's not like you can say, oh, you know, the United States should be more focused on deployment and there's a button to push in Washington, D.C. that makes that happen. But do you think it would happen naturally if it was a little bit harder and a little bit more challenging just to sort of maximize compute and capacity for existing AI companies?

42:23I think there's a way to tweak the incentives in a way that is not like the Chinese approach. That is not about a top down steering of the whole industry. but is more about trying to create maybe some of that commercial or even research space for, say, open source models. Yeah, I just think right now, right, you can think about a number of different markets where this is happening, where there's a focus on the high end of the market, on consumers or businesses that are willing to pay a lot. But there's less focus on sort of mass adoption and sort of that broader marketplace. And we're seeing some of this, right?

42:59Like, I I should be clear that, you know, NVIDIA is trying to release open source models. They have a commercial incentive because the more AI gets adopted, the more their chips are needed, right? So there's that closed loop there. And Google DeepMind, they have some relatively good open source models. But the commercial incentives as they stand are not quite there.

43:19Ross Douthat:Do you think we should sell more chips to China? Like as a sort of token of a different model? It's a very difficult topic because anyone who tells you yes or no on chips to China is really flattening the whole story. On the one hand, you do have real near-term effects on China's ability to produce the most cutting-edge AI models. So by limiting chips, that does slow down China's AI development in the near term. And that can be useful, for example, for giving our companies that edge in cyber attack capabilities. With mythos coming out, even a few months of being able to test on our own systems first is very useful versus a Chinese model having this capability and they're testing on our systems.

44:05So that's important. But at the same time, there's the other side of this whole equation, which is accelerating China's own chip development. And that's an area that they've been really focused on and they've been focused on because of our export controls. So it cuts both ways. In the near term, it will slow down their air development. In the longer term, it could speed up at least their ability to have a more resilient, self-reliant semiconductor supply chain that is not as affected by U.S. actions. So somewhere in there is a sweet spot. And it's really about where you draw the line rather than just saying more chips or less chips.

44:41Ross Douthat:And also how short timelines are overall. Absolutely. And I'm just going to make the hawk's case against your case and see how you respond. Right. Because the hawk says, you know, look, we've been at this for an incredibly short amount of time. Right. Since the chat GPT appeared in the pandemic, there's been tremendous acceleration. The people who have predicted acceleration keep being vindicated. Right. Right. And yes, if you're talking about like a 20 to 25 year time horizon for the point at which, you know, you sort of hit maximum super intelligence capacity, then, yeah, you have a lot of room to sort of figure out the optimal regulatory balance and all of these things.

45:26Ross Douthat:But if you're talking about two to four to six years, then maintaining a three to six month lead over your leading rival, by the way, is an authoritarian government, right? Seems like it may be really, really, really important. And the slowdown that you're advocating is one that could give up that advantage, right? So how would you respond to that kind of argument, which seems to be the mindset that certainly not just people at the Pentagon, but a lot of people in Silicon Valley have? Yeah. So that timeline comes up again and again, like in so many different debates within the U.S. as it relates to the U.S.-China AI competition.

46:15And fundamentally, it's impossible to say, right, how that timeline will play out. So I think, for example, that is what I I've discovered that in interviewing people. Yes, it is impossible to say on the timeline question. I mean, then it really boils down to what your views are about this AGI timeline and how likely this is to happen. And another factor that I will throw in there is, as a thought experiment, imagine that China did have access to the most cutting edge American AI chips. Would they be more AGI-pilled? Would Beijing be more AGI-pilled? You know, forget about DeepSeq or the actual tech founders themselves.

46:56And even on that, I'm not so sure that they would be so AGI-pilled. My guess would be that they would try to deploy certainly better models, but basically run their current playbook just amped up a whole bunch. Um, and, and I think it goes back to even their current playbook, right?

47:17Ross Douthat:Like includes cyber warfare includes a lot. Like you just mentioned the fact that just a three month advantage in the deployment of a cyber warfare capable model like mythos makes a big difference. Right. So it's not, it's not as though the current Chinese playbook is sort of innocent of conflict with the U S that's right. Yeah. So that's why I see it in as a different sets of risks. One is this AGI risk that you're talking about. And that I think is, I would argue, has been sort of overblown. But what I don't think has been overblown, and in fact, maybe even underestimated up until recently, is the cyber risk and the biosecurity risk.

47:59These are sort of more, I mean, it's kind of crazy to say this, but those are sort of like more medium risk relative to the AI catastrophic, like total takeover by superintelligence. So those sort of more intermediate risks I do worry about. And I do worry about U.S. competition vis-a-vis China. And so I think that would be, in my mind, a reason for maintaining the export controls that we currently have and not kind of fiddling with them and not agreeing to these side deals with Xi Jinping, for example. So that's why I try to find that balance. But in terms of the AGI question, that's where I'm just less convinced that we're really all in the sprint towards AGI, that China is really all in the sprint for AGI.

48:41Ross Douthat:But even on the medium risks, which I agree seem to me to be the most plausible risks, right? You are then making a calculation where you're saying, what am I most afraid of? Am I most afraid of China with the capacity to do unprecedented cyber warfare against the U.S. or a rogue AI or disastrous AI model that, you know, crashes the entire U.S. power grid for some inscrutable, you know, AI relationship? reason, right? Like it's that balance that you're worrying about. Yeah, yeah, exactly. And it comes to this question too about how the US should engage with China about AI. Because if we are focused just on China's cyber attack capabilities relative to our own, then you might say, don't bother engaging, right?

49:29We're both in this arms race, essentially on cyber capabilities. But if you're thinking about the rogue agent or say a non-state actor using either a set of American models, a set of Chinese models, or maybe they even do sort of arbitrage across, you know, even, you know, this is sort of like maybe 4D chess, but, you know, they deliberately are playing this geopolitical competition against each other and trying to distribute an attack across all these different models in order to disguise their origins, right? Those are areas where I do think that, one, it would be useful to talk to the Chinese side about these.

50:07And two, where I think it would be in the US national interest. It wouldn't just be about binding ourselves and slowing ourselves down relative to China. It would be about this extra third factor that we want to take seriously.

50:21Ross Douthat:And this is a good place to end because a lot of people in Silicon Valley will say, oh yeah, in theory, we could engage with China and and negotiate a sort of mutual AI slowdown. But in practice, either it's not clear that China wants to do that, wants that kind of negotiation, or it's just unimaginably complex to verify some sort of AI control agreement in the way that we did with nuclear missiles during the Cold War. Do you think a kind of Cold War style ongoing AI control negotiation with China is possible? I think we should not have high expectations, and I certainly don't. I think that we should start by talking.

51:07We should start by sharing our approach to AI safety and AI risk mitigation. We should try to convince the Chinese to take this more seriously, and they are starting to take this more seriously. We should also have a discussion about open source models, actually, because as those get better, right, on the one hand, we want those to diffuse more. But on the other hand, they could also pose a risk if they get into the wrong hands. So we can talk about all those areas, but I would be very hesitant, certainly at this stage, to even think about binding constraints, verification agreements, a kind of arms control treaty for AI between the U.S.

51:42and China. At this stage, it's way too early. Let's just start talking.

51:45Ross Douthat:If it's too early for that, is it just because of the sheer difficulty of imagining such a thing? Or is it a dynamic where precisely because Beijing's attitude is that we're not in some Cold War style race, they're actually less interested than they otherwise would be in that kind of negotiation? I think overall, it really boils down to one thing, which is an extremely low degree of trust between the U.S. and China and an unwillingness for either side to subject ourselves to invasive verification, monitoring, surveillance by the other party. And yeah, there could be interesting technical solutions that would make that more feasible, but it boils down to this geopolitical reality where we don't trust them and they don't trust us.

52:36And so we might be able to make progress on areas that affect both of us. But when it comes to letting, say, Chinese regulators come into the U.S. or letting American regulators go inspect data centers in China, I think that is pretty, pretty far out there at this stage.

52:53Ross Douthat:And do you think that that only changes on the far side of some disaster, conflict, some sort of event? Because I mean, one theory that I sort of, I don't just toy with, I guess I hold is that a lot of the negotiations around nuclear weapons were only possible because they'd been used and people were aware of how destructive they are. Is there a world where the only way that the US and China come to terms is a world where something tragic has to happen first? Yeah, that's a scenario I think about too. And I think about what would be the level of incident and what could the response be. You can think about sort of a most extreme case where you have some major cyber attack incident or even bioweapons incident related to AI where there are real lives at stake, for example.

53:42and that could cause both countries to just unilaterally put a pause on all their AI development because they realize that this is such a big issue with such huge risks. That is possible. So I do wonder and I do worry that we might be waiting for that incident to happen before we take action in advance, before we even start to talk to each other about how to take action. All right.

54:08Ross Douthat:On that somewhat dark note, Kyle Chan, thank you for joining me. Thank you.

54:42Thank you.