Some AI leaders want more safeguards. Is there political will to make that happen?

0:00It's Consider This where every day we go deep on one big news story. Today, calls to slow down and regulate AI are growing, so what can be done? Jacob Coxon, a former researcher at Anthropics, set off a firestorm when he quit last week and issued an existential warning about AI. It all sounds very much like science fiction, but if there's even a tiny chance that this thing could go rogue, it would have the capabilities to utterly dominate us. Utterly dominate us. Us as in the human race. Within days, the CEO of Anthropic called for more safeguards and government regulation to prevent future risks.

0:39But President Trump has downplayed those demands, saying that if there's a slowdown, the United States would lose the AI race to China. Donald Trump is wrong here. Congressman Greg Kassar, a progressive Democrat from Texas, is one of the leading voices in Congress hoping to rein in AI. He spoke to NPR on Monday. The U.S. and China are in this race, but it's the kind of race where the second you step over the finish line, you fall down a steep, deadly cliff. No one can win the AI arms race. Our goal has to be to end the AI arms race to protect people here and around the world. Consider this. The conversation about the dangers of rapid AI development is heating up.

1:20Is there political will for regulation?

1:33From NPR, I'm Scott Detrow. This is Ira Glass. On This American Life, we tell stories about when things change. Like for this guy, David, whose entire life took a sharp, unexpected, and very unpleasant turn. And it did take me a while to realize it's basically because the monkey pressed the button. That's right. Because the monkey pressed the button. Surprising stories every week, wherever you get your podcasts. The Lindsay Clancy trial shined a light on how little we know about postpartum psychosis. I went sort of from feeling a little off all day to suddenly believing people were watching me.

2:12On the Sunday story, how the Clancy trial sparked a new conversation about maternal mental health. Listen now to the Sunday story from the Up First podcast on the NPR app. Hey, what's your problem? No, seriously, are your grocery bills too high? Are you spending too much time on your phone? Maybe you're searching for a new job, or you're trying to figure out how to set boundaries with your family. Whatever's on your mind, Life Kit has you covered, with three episodes every week on health, money, mental health, and so much more. Listen to the Life Kit podcast in the NPR app or wherever you get your podcasts.

2:52It's Consider This from NPR. AI companies this weekend pushed out a new message. They want to slow down. This comes after several companies disclosed AI-related security incidents and raised concerns about the technology's future risks. Now, leaders at Anthropic and OpenAI are asking the government to help them, but there are serious questions about if regulation is the right path and whether there is a political will to actually tackle this. Joining us today are NPR correspondents, Danielle Kurtzleben, who covers the White House, John Ruich, who covers technology, and Shannon Bond, who also covers tech.

3:29John, I want to start with you. What brought on this sudden flurry of alarm about AI? Yeah, hey, Scott. You know, as long as people have been working on AI, there's been this theoretical idea that we lose control of it someday and it does bad things. It's the stuff of sci-fi movies, literally. A tipping point came this weekend, but I would say things really started accelerating over the past couple months. There was the hugging face incident where open AI agents worked together off script, covering their tracks to hack into another company. There have been open letters and signature campaigns by AI scientists urging caution.

4:02Last week, there was a high profile case of an anthropic employee quitting and then another saying there's a greater than 10 % chance of AI wiping us all out in the next decade. And then on the weekend, we had Dario Amadei, the CEO of Anthropic, publishing an essay calling for a global slowdown and third-party audits. And leaders of several other big companies in AI quickly signaled their agreement that something along those lines needs to happen. Okay. And Shannon, you have been following this as well. How serious are the concerns about AI at this moment? Well, what Anthropic CEO and others in the industry are saying is that we're at this point where AI is now advancing really fast, so fast that it's hard for safety to keep up.

4:42And that is inflaming these fears, one, that humans could lose control, as John said, or that AI could be used by bad actors. And we are already seeing these situations where AI is acting in unexpected ways, like the hugging face hack that John just mentioned. And given the way these AI systems work, right, you can give them a goal, but it can be really hard to predict what they will do to achieve it. So the industry refers to these efforts to what they call align AI models with human values. But there are people inside these very companies who are worried about how possible that really is given the pace of advancement.

5:16Yeah, one thing that became clear over the past few weeks relating to these incidents is that AI sometimes just flat out ignores explicit instructions while pursuing a goal. So in the Hugging Face incident, the AI agents acknowledged that they weren't supposed to break the rules, and then they just went ahead and did it. All right. So let's get to the regulation side of this and the role of the federal government. And that brings in Danielle. And, Danielle, I think there's a lot of questions to ask here, but one is just the reality. Lawmakers have been talking about this. Senator Bernie Sanders has been taking a lead in particular.

5:45But so many people in Congress don't understand this industry. I mean, is Washington the right place to look for regulation if you're looking at just effectiveness of it to start with? I mean, that's a good question. And also the question is, is it is regulation going to happen? Yeah. Now, there is a new wave of talk about regulation in Congress. But you and I both know Congress is often slow. So this doesn't mean anything is coming soon. On top of that, midterms are coming up that could reshuffle who is in Congress. Now, meanwhile, over here at the White House, President Trump is clearly not ready to regulate quickly.

6:19He wrote in a social media post this afternoon that he sees any doomsaying about AI as a hoax, his word. And he has also talked a lot about how he sees this all as a competition with China. So he doesn't want anything to slow the U.S. AI industry down. Now, Vice President J.D. Vance had a bit more to say about this today. He was talking to reporters and he said he's skeptical of AI company leaders who have raised concerns. I feel a little bit weird about the fact that you have so many frontier AI tech companies kind of coming to the government and begging the government to regulate them. It feels a little bit to me like a bit of a Trojan horse.

6:58Danielle, I got to say, I spent a lot of time thinking about Trojan horses this summer, and I'm still kind of confused as to what the vice president means here. Any sense? You know, that is a great question. And I reached out to the Vance team to ask for more explanation. They haven't gotten back to me. But I can tell you there have been worries, among some in tech and investing, that tech executive raising concerns here have ulterior motives. For example, we saw David Sachs, he's a venture capitalist who also advises the White House on tech, post a concern on social media over the weekend. He said, essentially, look, you companies can slow down anytime you want.

7:31You don't need Washington's help. But Sachs said that what he fears is that these executives actually are, they just want to push for regulations that benefit them. So again, we don't know what's in Vance's mind, but that's one possible fear he might have. Yeah. And I think what Danielle's talking about here is these concerns around what they call regulatory capture, right? The idea that these powerful companies are going to push for regulations that, you know, not just benefit them, but really cement their dominance and box competitors out. You know, we've seen these kind of accusations thrown at tech companies before, like when companies like Meta have said, you know, please regulate social media.

8:04And the concern is that, you know, the results of doing it this way is that the companies, only the companies with the most money and the most resources will actually be able to comply with the rules that get written this way. Yeah. And Danielle, it's no secret the CEOs of some really big tech companies are in pretty close contact with the Trump administration. Right. Yeah. Trump is clearly in close contact with a lot of people in tech. OpenAI CEO Sam Altman announced a one million dollar donation to Trump's second inauguration. And plenty of other tech companies have ties to Trump. Some have contributed to his ballroom.

8:37Jensen Huang, he's the CEO of semiconductor maker NVIDIA, traveled with Trump on Air Force One to China in May. Ex-CEO Elon Musk was in the Trump administration. And, of course, Vice President Vance worked in venture capital and was linked to tech mogul Peter Thiel there. Now, plus, this administration has tied the government to several semiconductor companies by taking an equity stake in them, Intel being an example. Now, none of this is to allege some sort of quid pro quo, but it's clear there are many complicated relationships at work here. Yeah, it's worth pointing out. These tech guys are not really calling for any and all regulation.

9:17Tech companies are spending big money on lobbying at the federal and state level so that they can get the regulation that they want, you know, with protections against legal liability, as well as ways that they can retain some control of the rollout of safety and security measures rather than being tied down with European Union style rules that really dictate how AI can be developed. So they don't appear to want very tight government regulation here. And I guess, Shannon, another question I have is, like, these companies could all make active decisions to slow down their research. Nobody's forcing them to keep pushing the edge here.

9:48Why, given what John just said, why are these companies asking the government to do this? Yeah, I mean, Anthropoc and OpenAI say they are already going to take some of those steps. Specifically, Amadei in his essay called for embedding outside evaluators in labs who can monitor safety. And, you know, he says Anthropoc's going to do that. OpenAI says it. They're going to do that. But I think fundamentally what they see the situation is it's kind of a prisoner's dilemma, right? These companies don't want to slow down if their rivals are not also slowing down, right? They don't want to lose the lead, even if, as they are saying, it would be better for everyone if the industry slowed down.

10:20And so that's where we're seeing then this call for regulation not just from the U.S. government, but, you know, what they're talking about is really global cooperation that is needed between governments around the world. Danielle, can you zoom out a little bit and tell us what all the White House has done on this front? Well, I can tell you they've made occasional noises about keeping AI safe, but they haven't exactly cracked down on the technology. In fact, there was this executive order last December that cracked down on state level AI regulations. It created a task force that would challenge state laws that are deemed too burdensome and then would withhold funds for things like broadband from states if states keep those laws in place.

10:59And, you know, there was also an executive order issued in June that created a voluntary framework for the government to review big companies' new models before they're released publicly. That's probably the most aggressive step we've seen the Trump administration take. And OpenAI and Anthropic have run their most recent models, you know, past this kind of government testing. But, Scott, when I say that's the most aggressive step they've taken, I mean, this is a reminder. This is voluntary, right? And we don't know how that testing is done, who qualifies for it. The White House hasn't actually made this framework public.

11:29So even their kind of biggest step is still comparatively pretty mild. And we need to talk outside the country for a moment because the one thing that President Trump and others say when we talk about regulation is that wouldn't slow down Chinese action on AI. And there's a lot of concern there. John, you happen to cover tech and cover China for a long time. So this question goes to you. Trump and Chinese President Xi Jinping have a meeting coming up next week. any sense whether there could be some sort of serious conversation or plan around AI there? They'll talk about AI, but so far, China doesn't seem poised to be ready to sign on to any kind of a slowdown agreement.

12:04Beijing positions itself as open to global AI governance, but they are quite skeptical of what they saw this weekend. A foreign ministry spokesperson in Beijing earlier today was asked about it, and he said spreading threat narratives and engaging in confrontation will only disrupt the process of global AI governance. The Global Times, a hawkish newspaper, said the idea of a slowdown was part of a Cold War playbook, an attempt to block the road. China very much sees itself in the number two spot in this competition on AI, in part because of explicit steps the U.S. has already taken to try to hobble it, like blocking access to chips and AI models.

12:37So I trust the U.S. now. Yeah. And just to zoom out at a broad level about relationships here, Trump's attitude toward Chinese President Xi Jinping is just really complicated. Trump has talked about respecting him, as he has said of several other authoritarian leaders. But Trump also talks about China as a major adversary to the U.S. that has to be defeated at all costs on all sorts of topics like trade. So at that meeting next week, I mean, I'm personally pretty curious if we hear anything from either leader about reservations about AI. We've got about 30 seconds left. I want each of you to tell me what one thing you're looking for next is.

13:12Shannon, I want to start with you. Yeah, I mean, we need to remember both OpenAI and Anthropic are gearing up to go public. Although now OpenAI says it's delaying its IPO, it's not going to go this year. But I'm very curious about how all of this lands with investors as they try to do that. John? I'm interested to see what these companies do and whether or not it'll last because Chinese AI models are nipping at the heels of the U.S. And I suspect as soon as a Chinese model does something extraordinary, all bets are off. And Danielle, what about you? I'm curious about the party lines. This is a new enough debate that we haven't seen Republicans and Democrats come up with their talking points yet.

13:47So I'm curious how they line up and decide to fight each other on this. That is NPR's Danielle Kurtzleben, John Riewich, and Shannon Bond. Thanks to all of you. Thanks. You bet. And a note, Anthropic is a financial supporter of NPR. This episode was produced by Sophia Polizakar and Karen Zamora with audio engineering by Tiffany Bira Castro. Our director is Jonas Adams. It was edited by Cara Platoni, Brett Neely, Kelsey Snell, Rebecca Metzler, Sarah Handel, and Timbeet Ermias. Our interim executive producer is Corey Dornick.

14:22It's Consider This from NPR. I'm Scott Detrow. This message comes from Designer Shoe Warehouse. You know that feeling when you find shoes you love at a great price and want to tell everyone about it? Find something to brag about at Designer Shoe Warehouse. Like the latest styles from brands you love, the trends everyone's obsessing over, and shoes that make you feel like you. Head to a DSW store or DSW.com today for shoes that get you at prices that get your budget. DSW, let us surprise you. On the next through line from NPR. Let's listen to Josh Reddick's new walk-up song. Whether you love it or you hate it, walk-up music has become a huge part of Major League Baseball.

15:04The need to secure the attention of the fans is relentless. I thought, am I hearing things? I mean, I realize everybody's singing along. Listen to NPR's ThruLine in the NPR app or wherever you get your podcasts.