Can the US and China stop AI from killing us all?

0:00This BBC podcast is supported by ads outside the UK.

0:30We do the things we do and how to make better choices to help avoid costly mistakes. Each episode covers the latest research in behavioral science and dives into themes like the power of self-control, shaping your mindset for success, navigating new beginnings and why starting over can feel so hard. Listen to choiceology at schwab.com slash podcast or wherever you listen.

1:00John Deere is putting in work and putting up highlights. Oh man, look at those passes. Tillage, planting, harvesting. Three years in and these pre-owned machines are still at the top of their game. We're talking reliability, performance and long-term potential. And get this, these veteran machines can pick up the latest precision tech with ease. You know it and fans ready to add John Deere certified pre-owned equipment and more options to their lineup can shop at machinefinder.com. Booyah! The fears around artificial intelligence have truly reached a fever pitch. Even tech leaders within the AI industry say they're worried about what this technology they're devising could be capable of.

1:43Over the weekend, the CEO of Anthropic, Dario Amadei, published this lengthy essay laying out a plan to rein in and slow down the pace of AI development to, quote, reduce the risk that something goes seriously wrong. He talked about his concerns in an interview with CBS News. It's a warning sign. It's a warning sign that we need to slow down. We need to make this technology carefully and we need to make sure that our safeguards, our ability to understand it, our ability to control it, keeps up with the pace at which the technology is happening. And perhaps surprisingly, his biggest AI rivals, Sam Altman and Elon Musk, chimed in to say they agree.

2:31This came on the heels of recent apocalyptic warnings of superintelligence, including one from a former researcher at Anthropic who resigned with a warning that AI could kill us all by the end of the decade. But slowing down might be easier said than done. On Sunday, President Trump was asked about these recent warnings. We're leading China in AI. We're the most sophisticated country in the world. And frankly, I want to keep it that way because whoever wins AI wins. And we can quit guardrails. We can do this and that. But I think you have a lot of negative forces in bringing it up that shouldn't be bringing it up.

3:09And they're bringing up things that won't happen. Any action to truly regulate AI would need to be global. And global cooperation is really difficult when the world's two biggest powers are in an arms race to build the most advanced AI as fast as they can. Next week, here in Washington, President Trump is slated to meet with his Chinese counterpart, President Xi Jinping. And artificial intelligence is expected to be a part of the agenda. But guardrails or not, Trump has made it clear he believes America needs to win the AI race. From the BBC, I'm Asma Khalid in Washington, D.C. And today on The Global Story, what would it mean for China and the U.S.

3:52to agree to pump the brakes on AI development? And can these two countries realistically cooperate to prevent an AI catastrophe?

4:05My name is Kyle Chan. I'm a fellow at Brookings, and I do research on technology and AI in China. Welcome to the Global Story, Kyle. We really appreciate you taking the time. Xi Jinping and President Trump are meeting here in Washington, D.C. We are also expected to see American and Chinese officials meeting later this month to discuss AI safety. And this is coming shortly after a former anthropic researcher said that AI, in his words, could kill all humans within the next decade. That's sort of dire warning about AI and its ability to potentially kill us. I don't know whether or not you agree with that assessment.

4:42But Kyle, can you help us imagine how that could happen? What is the nightmare scenario? How would that play out? Yeah, I think the nightmare scenario that is imagined by some in Silicon Valley, especially many in the AI industry, is one where these AI systems make themselves more intelligent. to the point where they can do things and come up with ways of disrupting our world that even we couldn't imagine. So I think probably the sharpest example relates to cyber attacks. The idea here is imagine an AI system gone rogue that attacks critical infrastructure, say the grid or vital computer systems that run our hospitals and our medical systems, attack financial markets and cause a major shock to the financial system.

5:40These are things that could be large-scale risks. And then I think in the furthest, sort of most extreme version of this, these superintelligence systems could sort of take over everything that was digitally and internet connected. All of our devices find vulnerabilities and kind of take over our digital world, as it were. and essentially through that, our physical world. Okay. I mean, that does sound rather alarming. Do you think it's likely, Kyle? I am in the camp where I see definitely growing large-scale risks from AI, but I'm not of the mind that we're already on this steep slope towards sort of runaway superintelligent systems.

6:25One thing I do worry about is the focus on the really tail-end, almost sort of sci-fi risks, I think can sometimes pull away attention from all these other also very important risks, but maybe not quite as existential as the runaway superintelligent systems. Some of them are actually major threats to global public safety. Like if an AI system were involved in, say, nuclear command and control systems, that would be a huge deal. I mean, I just saw this headline that was pushed about Anthropic saying that it had disrupted some potential plots by scientists, I believe, to conduct research that could have helped develop biological weapons.

7:08Yeah. So AI-powered biological weapons design and even production would be really, really a big deal. And that is something that I think that we should be all taking more seriously. I don't know, again, if that is sort of the sci-fi scenario. I think that's more of a today, what can they do? And now there's already been studies showing that we can use AI systems now to develop new viable viruses that thankfully we have not released into the real world. We want to spend the bulk of this conversation talking about the relationship between the United States and China and their relationship with AI.

7:47We hear a lot about AI as being a race between the U.S. and China. This is the rhetoric that is often used here in Washington. We heard President Trump say, We are way ahead of China. I have to say, China and other countries are racing to catch up to America having to do with AI. And we're not going to let them do it. Does China subscribe to this idea of AI being a race between the U.S. and China? I think China sees it differently. So I think in the U.S., it's framed as not just a race, but a very specific kind of race. And that is a race to AGI. AGI, or artificial general intelligence, is this idea that we might have AI systems that are capable of doing everything that humans can do, at least for cognitive tasks, or even more than that, superintelligent systems.

8:40And what that would mean in practice is AI systems that could power, say, new military weapon systems or develop new biological weapons or create new cyber attack capabilities that had never existed before. And that could be decisive in having an advantage over other countries. I think for China in general, and then for Chinese policymakers in particular, they see AI as something very powerful and very useful, but not necessarily a linear path towards this superintelligence takeoff. And that is something very different than I think how it's talked about in Washington and Silicon Valley. So in China, there's much more focus on trying to use AI like electricity or computers or the Internet, trying to put AI everywhere.

9:37So the sort of collective good that doesn't have a zero sum. It's not that if I have electricity, you in Germany don't have electricity. That's right. I think the key phrase here is winner take all. I think in the U.S. view, whoever gets to AGI first wins everything, essentially, militarily, economically, technologically. And I think China's view is there could be some zero-sum competition, but that it's not like a single company or single country will come to dominate everyone else.

10:13I'm curious what you think is the right way of framing this, Kyle, because I've often heard the competition between the U.S. and China described as sort of a modern arms race. Is it comparable to other technologies in the frame of nuclear weapons, where there's a really big advantage to being there first? Or is it more like running water, electricity, where there is less of a zero sum? I think a lot hinges on whether you believe in a steep takeoff is what they call it. That is, whether you think we'll hit this inflection point where the technology doesn't just get gradually better and better, but gets better exponentially to the point where humans no longer have control over how AI develops.

10:55If you believe that's the world that we are in or barreling towards, then you would see this as a sort of not just zero sum, but winner take all situation. But if that's not how you see it, if you see it more like electricity or like utility, and honestly, that is probably more of my view. The technology is changing very quickly, to be sure, but I don't quite see us hitting that inflection point in the near future. And because of that, I think about AI capabilities today or in the next couple of years as being potentially very, very useful for a wide range of economic sectors, for education, for drug development.

11:35And sort of commensurate with that, I see sort of the risks that might go along with that. But yeah, I just don't subscribe to this sort of utterly transformative view of the world that I think a lot of people actually do see in Silicon Valley. If we accept the premise that there is competition between the United States and China, maybe they don't both agree they are running in the same exact race. But if we broadly accept the idea that there is competition between the two nations for the development of artificial intelligence, who would you say is winning the race in this moment? That's a really interesting question.

12:12If we define the race in terms of who is the most powerful AI models, then it is the U.S. And I think, ironically, many in China's AI industry would agree with that. But on the race to figure out how to integrate AI into society in a productive and economically useful manner, also in a way that mitigates some of the potential shocks, say, to the labor market. On that, potentially, one could argue that China is pulling ahead. They have certainly a much more comprehensive effort to integrate AI across a whole range of domains and industries. I want to understand how we got here. What is the story of China's rise in AI?

12:58How did it reach this point? AI has been a priority for China for a long time. You have researchers going back now decades working on this technology. And China felt like it was ahead for a while. But there were two Sputnik moments for China. The first Sputnik moment was AlphaGo. This is an AI-powered player developed by DeepMind that could play the game of Go, where you sort of place pieces on a board to surround your opponent and sort of remove their chips from the table. And what makes Go so interesting compared to chess is the complexity of the game. One, two, three, four, five, six, seven, eight, nine, nine, nine, nine, ten, ten.

13:49I think maybe the game ended. Wow, I think you're right. Wow. Wow, wow, wow. AlphaGo has won again. Three straight wins. Three straight wins has won the match. What was so striking with AlphaGo was that it seemed to exhibit a kind of creativity in how it was able to play against humans. And ultimately, it beat the world human champion at Go. Korean champion Lee Se-do emerges in defeat after losing the second battle in a row against a Google supercomputer called AlphaGo. And this happened in 2016. So that was the first spotting moment. The second spotting moment was ChatGPT in 2022. And this was a huge wake-up call because if anybody thought that China might be ahead on AI, this put that theory to rest.

14:47All of this is just to say that Chinese AI labs were in full sprint to try to come up with their own competitor to ChatGPT. Finally, you have, at the beginning of 2025, the DeepSeek moment. Let's talk about DeepSeek because it is mind-blowing and it is shaking this entire industry to its core. DeepSeek, an AI lab that actually came out of a hedge fund in China, released an open source model that suddenly performed almost as well as the very best models from open AI. Not only was it cheaper and quicker to build, but it's also cheaper and more efficient to run, pushing it to the top of the App Store charts and dethroning chat GPT.

15:30And that sent shockwaves, including through the global stock market. It has wiped more than$1 trillion off the world financial market. Welcome to the AI race post-deep seek. This is a breakthrough so significant, it redraws the battle lines. And it makes the winners of the previous era, OpenAI and Microsoft, makes them look like sore losers. I remember that. It seemed really shocking. People didn't seem to know that China could do this, at least here in the United States. That's right. There, in fact, was a long perception that China could do hardware well, like making smartphones or consumer electronics, but struggled with software.

16:15And DeepSeek really put that belief to rest because it showed that here in the most powerful software system of all, these AI foundation models, China was able to nearly catch up to the U.S. I wanted to ask you about that because, you know, the United States and China, very different systems of government. And when it comes to AI here in the United States, you have a handful of really powerful tech companies led by really prominent individuals. You know, folks like Sam Altman, Elon Musk, who have enormous power and influence. Does China have something similar with extremely powerful private companies or is this more of a state run project?

16:56project. In China, there's ultimately a sense that the state is going to be running the show. While the technology has evolved very quickly within private AI labs in China, and they get all the credit for developing these very powerful AI models and nearly keeping pace with the United States, ultimately the broader direction of the industry, the regulatory framework that everything operates under is pretty clearly laid out by Chinese policymakers. So in many ways, there is a tacit understanding between the AI companies and the government agencies, where the government agencies want to give enough leeway for the companies to innovate, but the companies know who is really in charge at the end of the day, and that's Beijing.

17:49Are there really prominent equivalents to, say, an open AI or an anthropic in China? Yes. There are companies like DeepSeek that have become a national champion. So in addition to DeepSeek, there's Z.ai, there's Minimax, there is the larger tech companies like ByteDance, which people might know with TikTok. TikTok, yeah. That's what I was thinking of when you Exactly. And you have companies like Alibaba and Tencent, which are also very, very large companies, kind of like the Google or Amazon of China. So it's a range of companies from startups to big tech. And many of their founders have now become celebrities in China, in addition to becoming very wealthy.

18:36So they are kind of akin to a Sam Altman you're saying in China. Yeah, they have a certain celebrity status. At the same time, though, they are quite careful. because of what has happened historically in China's tech industry. In an earlier era, there was actually a big government crackdown on China's tech industry. When some of the Chinese policymakers felt that the company leaders were getting a bit too vocal and pushing back against policies, most famously, Jack Ma, the founder of Alibaba, had to leave the country for a while and has since come back. But that sent a chill through the whole industry.

19:15And ever since then, Chinese tech founders have been careful to champion the development of AI in China, but to not overstep what they believe to be sort of the red lines of the government.

20:00We'll be right back. ready to add John Deere certified pre-owned equipment and more options to their lineup can shop at machinefinder.com. Booyah!

20:31So you've outlined the rise of China's artificial intelligence industry. And we are having this conversation now here in 2026 at a moment where, you know, the U.S. and China seem to be in quite a bit of competition when it comes to AI. We heard the U.S. Treasury Secretary Scott Besson at a Breitbart News event just the other day say. There is no day after tomorrow if China wins this. Yes. Like everything, a trillion and a half dollar defense budget, the Iron Dome. If they were to pull away for us on AI, then nothing would matter. I mean, he's speaking about this in really, really dire terms. Can you help us understand why that way of thinking is so prevalent here in Washington?

21:17And it seems like parts of Silicon Valley as well. I do think a big part of this has to do with the analogy to nuclear weapons in the Cold War. I think that in many ways, though, AI doesn't quite fit that nuclear analogy. AI can be dangerous, for sure. And as I mentioned earlier, some of these risks we need to be taking more and more seriously. But at the same time, though, it is not a kind of technology that you can sort of seal off in a facility and limit, say, a particular input like uranium, nuclear-grade uranium in the nuclear weapons industry. Instead, it is something that spans the mundane, the chatbots that many people use, to the truly formidable in terms of its capabilities.

22:06Biological weapons, as we were saying possibly earlier. Exactly. And so given the wide range of potential uses for AI, the question becomes not who's going to get there first, but who can reap the benefits while also trying to address the real risks and the real potential harms coming from this. Let's talk more about that question of how do you jointly manage this and what U.S.-China cooperation would look like, because we are having this conversation in the context of upcoming meetings between American and Chinese officials, as we say they are meeting later this month to discuss AI safety. In the past, the Trump administration seems to have taken a more relaxed approach to AI regulation.

22:53In January of 2025, we saw President Trump revoke former President Joe Biden's 2023 AI Safety Act. In the summer of 2025, the Trump administration issued its AI Action Plan, in which it says, and I'm going to quote here, AI is far too important to smother in bureaucracy at this early stage, whether at the state or federal level. That all being said, there has been, I would say, a change of tone a bit from the Trump administration, which says that it will introduce a framework to review certain types of AI models for risk. But that being said, you know, the president has also expressed hesitancy about regulation over the summer, he told journalists.

23:33So we have to be careful in both ways. We don't want to restrict them. We're all of a sudden we're coming second to China. Whoever wins with AI is going to win. That's how big it is. And so we're talking about the idea of Chinese and American officials getting together to talk about AI safety. But at the same time, we hear the president of the United States expressing reservations about regulation for the specific reason that it could hurt and disadvantage American AI companies. How do you square that? So I do think that China is sometimes used as a reason to not have regulation. It's used as the boogeyman out there where if we don't run faster, if we don't unleash American AI and accelerate American AI progress, China will get ahead and we will lose the race.

24:29And the consequences of that could be tremendous. I think that what this framing of the race with China does is it allows us to step away from dealing with AI risk head-on. And it allows us to continue this hands-off approach to the industry and let the companies basically decide the terms of how this technology will develop. And I think that's a mistake. I have one final question for you. Xi Jinping and Donald Trump are planning to meet later this month, and one of the items on their agenda is expected to be AI. What do you think could come out of that meeting? And do you see any risks to there being no progress when it comes to AI between the Americans and the Chinese?

25:21There is a risk that this meeting could be one and done, that there are so many tensions and thorns in the relationship, especially over AI competition, that those issues could dominate the discussion and it could be a more acrimonious push and pull between the two countries. But on the more hopeful side, if the U.S. and China can at least begin talking about some of these issues, keep in mind that we don't even agree on what the definitions are for some of these terms like AGI or loss of control. If we can start to talk, then perhaps we can slowly, steadily build up the trust and framework for something more comprehensive.

26:06and maybe longer down the road, after we each also get our own AI regulatory house in order, then we can perhaps reach some broader bilateral or even global agreement about how to regulate and ensure that AI doesn't cause the kinds of destructive events that we now increasingly fear. Cal, it sounds really optimistic to me, but maybe that's just because I'm a acidic creature of Washington now where I think that there's so much distrust between Washington and its counterparts here in China that I just don't think, whether we're talking about AI or anything, I don't see that political conversation changing here domestically at all.

26:49It's true. I think there is definitely a deep degree of skepticism that's warranted here about how far any kind of cooperation might go between the US and China on AI. The one response that I would give is the nuclear analogy. Even though I said earlier that as a technological analogy, it doesn't quite hold. I do think that on a geopolitical level, it offers a useful historical lesson, which is that even in the depths of the Cold War, where the U.S. and Soviet Union faced off with missiles pointed at each other, threatening the existence of the entire world. Even in that level of distrust, they were able to find some kind of agreement on nuclear arms control.

27:40And perhaps it was the sheer devastating potential of this technology and what it would mean to have an all-out nuclear war. And I think as the U.S. and China come to appreciate both the benefits, but also the growing risks of AI, that could tip the balance in terms of getting the two sides to at least start to talk to each other, if not agree to something outright.

28:14On that note, I just want to say thank you, Kyle, for bringing your insights to us. It's a really, really fascinating conversation. My pleasure.