Who decides when AI is too dangerous?

0:00Nilay Patel:Support for the show comes from ServiceNow. AI is moving fast across the enterprise, but without visibility, it's just chaos. Different tools, different models, different teams using AI in completely different ways. ServiceNow turns that chaos into control. With the AI control tower, you see all your AI across the business in one place. What it's doing, what it's done, and what it's about to do. So you stay in control. To put AI to work for people, visit servicenow.com.

0:35Nilay Patel:Support for Decoder comes from Uber Advertising. Whether it's ordering dinner, booking a ride, or shopping for groceries, we live in a world where we make decisions the moment a need arises. But these moments aren't just when people choose Uber. They're when people choose brands. Uber Advertising helps brands understand the context behind those decisions. the moment, the need, and the mindset across mobility, delivery, and commerce, so brands can show up with greater relevance when people are ready to act. Uber Advertising, where life's movement becomes your brand's momentum. Learn more at uber.com slash advertising.

1:17Support for this show comes from BetterHelp. Summer can feel like a sprint. Kids home, trips to plan, routines flipped upside down. it's easy to slip into survival mode just trying to get through it. Then suddenly it's over, and you're wishing you enjoyed the days just a little bit more. Therapy can help you slow down and actually be present for the moments that matter. With BetterHelp, you can connect with a licensed therapist from anywhere on your schedule. Don't just survive this summer, thrive. Visit BetterHelp.com slash Vox Pods.

1:51Nilay Patel:Hello, and welcome to Decoder. I'm Nilay Patel, editor-in-chief of The Verge, and Decoder is my show about big ideas and other problems. My guest today is Hayden Field, the senior AI reporter here at The Verge. Often when Hayden shows up on Decoder, it's because something has gone wrong in the world of AI. And last weekend, that something was a pretty intense mix of Anthropic, the Trump administration, and Anthropic's new model, Claude Fable 5. On Friday, not even a week after Anthropic released Fable 5 to the public, the United States government said it was imposing export controls on that new model, as well as the underlying mythos model that Fable is based on.

2:24Nilay Patel:Those controls restricted foreign nationals, even those working for Anthropic in the United States, from accessing these models. Anthropic then took Fable and Mythos offline for everyone, because the company said it was worried it would not be able to restrict access and reasonably comply with that order otherwise. As you might imagine, this is all a giant mess. Hayden actually just published a fantastic play-by-play on The Verge about how this all went down last Friday, and the scramble through the weekend from both sides to figure out exactly what happened and how it might all get resolved. So I want her to come on the show and walk me through the timeline and what it all means.

2:57Nilay Patel:The situation is ongoing. As of Tuesday, when we're recording this episode, Fable is still offline. In fact, if you boot up Claude, it tells you right above the chat box window when Fable 5 is currently unavailable. Yet, as you'll hear Hayden explain, whether Fable comes back online this week or not, the ripple effects of the government's feud with Anthropic have far-reaching consequences for the tech industry and the United States' AI regulatory regime. There's also a big irony here, and you'll hear Hayden and I talk about it quite a bit. Anthropic has spent years arguing that AI might soon be powerful enough to be dangerous, and that the government needed to get serious about regulating AI sooner rather than later.

3:34Nilay Patel:Well, now we're here, and Anthropic doesn't love the way it's playing out. And now everyone, but maybe especially the Chinese government, is watching to see whether the United States AI regulatory approach takes the shape of a serious safety framework, or whether it's just another weapon for the White House to use against companies and people that don't bend the knee to the Trump administration. Like I said, it's a real mess. Okay, Verge senior AI reporter Hayden Field on the Claude Fable ban and the new AI regulatory landscape. Here we go.

4:16Nilay Patel:Hayden Field, you're the senior AI reporter here at The Verge. Welcome back to The Coder. Thanks. Great to be here. It's always chaos when you're here, Hayden. That's what I've come to discover. It really is. Also, my rule is still in effect in that it's always a Friday. Stuff always hits the fan on a Friday, and this was no different. Particularly when it comes to the government regulating AI, there's a real, what if it all went crazy on Friday? So we all had to think about it over the weekend. And then I think this has been basically true throughout the Trump administration. The scramble to put it all back together on Monday is immediate.

4:49Exactly. It's crazy. And the fact that all these talks were happening all over the weekend, and then Monday, still no resolution. I'm surprised that things aren't resolved yet. But yeah, lots of drama.

5:03Nilay Patel:Well, this podcast is going to come out on a Thursday. We're recording on a Tuesday. We'll see what happens over the next two days. But I think the big picture here of how the United States government should regulate AI and what that means for American AI companies on the world stage is open. I don't think it's going to be resolved with whatever happens with Fable. So let's try to understand the whole story and then pull those themes out of it, because they're just going to keep coming back around again and again and again, I think with every new model released now. So let's start at the start.

5:35Nilay Patel:What is Claude Mythos? What is Fable? How do they relate to each other? Great question. And I asked them the same thing when they came out with this like a week ago, or by the time we publish this, it'll be like a week and a half ago. Because, yeah, AI companies are really bad at naming things, and it's always very confusing. But the situation is Mythos is the underlying model that is powering both Mythos 5 and Fable 5. Fable is a new thing. They haven't had Fable 1, 2, 3, or 4. They just immediately started with Fable 5. Very confusing. But Fable is the watered down or safeguarded version of Mythos 5.

6:14So when they came out with Mythos preview in April, Anthropic kept saying, you know, they were really hyping it up. They're saying this is a potential cyber weapon. It's got to be never released to the public, at least until safeguards catch up, et cetera, et cetera. We're only going to release it to enterprise and governments, a.k.a. cyber defenders is what they called them, places that need to patch up all their frameworks before bad actors come in and exploit those vulnerabilities. Now, fast forward to last week, and they released not only Mythos 5, which is the first official Mythos, before it was Mythos Preview, so we skipped straight to 5.

6:56Then they also released on the same day Fable 5, which was the very first public version of Mythos Class Models, a.k.a. same underlying framework, but now tons of safeguards built on top to hopefully stop the public from accessing this super dangerous bianthropics own admission model that could mess up everything. So, yeah, that's what they released, and they spent a long time hyping this up and talking about how dangerous it was, and then it kind of came back to haunt them later.

7:26Nilay Patel:So Fable is a mythos with the security guardrails. And when it first came out, there was a wave of controversy, right? AI researchers, security researchers found those guardrails were actually preventing them from doing research and seeing what Fable was capable of. Because if they tried to push the boundary, it would downgrade them to Anthropik's previous models. Yeah, it became kind of like a meme. Like people were making fun of how strict the guardrails were when it came out. So that's the thing that's confusing me, where there's a controversy because the guardrails are so strict, and then something happened that made the government think Fable was so dangerous that it needed to be pulled.

8:04Nilay Patel:What happened? I talked to some independent red teamers about this, and before the Amazon white paper was circulating, before this allegation of a jail break was public, I talked to some red teamers about how safe Fable 5 was, and they were all pretty impressed with how the guardrails held up. I was expecting it to be a little bit easier to jailbreak. They were saying they had tried a lot of stuff and nothing was working. So that's not my experience when I usually talk to independent red teamers. They're like, yeah, I got it to break here. I got it to break there. In this case, they were saying it held up pretty well.

8:39Now, what happened this time is, according to a source familiar with the negotiations who was directly involved with the negotiations that I spoke with yesterday. This is kind of the timeline of last week. Mid last week, Anthropic was made aware of some research that Amazon researchers were doing and that they had uncovered a potential jailbreak, something they were pretty worried about. They sent it over to Anthropic. Anthropic was kind of going back and forth with the Amazon researchers talking about it. And they were discussing whether it was really a jailbreak or whether it was like not that big of a deal, essentially.

9:18Apparently, reports say that Amazon CEO Andy Jassy then got worried about it and called maybe Scott Bessent, some member of the Trump administration, on Friday and talked about his concerns. The Trump administration freaked out immediately. They immediately call Anthropic. They send them a message, actually, and they say, look, you need to shut this down in 90 minutes and we need to figure out how to - 90 minutes. Yep. 90 minute ultimatum. And they said, we need to figure out how to fix this jailbreak that's come to light. Anthropic got on the phone with the Trump administration within 15 minutes of that first call.

10:00And they were asking for more details. They're like, okay, let's talk about this. Are you talking about the research we're already aware of? Are you talking about something different? What is the jailbreak you're talking about? They're trying to get more details and you know, talk to the administration. I think the administration was also mad that Dario Amadei, the CEO of Anthropic, didn't call back until an hour and 15 minutes after that first call. So Anthropic called back 15 minutes in. It took Dario an hour and 15 minutes to call back. So there was a lot going on here. By the time, you know, they're talking, you know, whatever, the 90-minute deadline passes, the Trump administration says, okay, that's it.

10:38Yep, we're hitting you with export controls. And so that is when they told Anthropic, no foreign national can use either of these models. And you have to just make sure that that is the case. No foreign national, whether they work at Anthropic or not. AKA, you know, if you're say Bank of America and you have access to Mythos 5 and you're using it to plug vulnerabilities in your system and you're not a US citizen, you can't use it. Same with, you know, a random, just regular individual using Fable 5. Same with an Anthropic researcher who is not a U.S. citizen. So, Anthropic did the only thing it could really do in this situation, and they said, okay, we got to sideline both these models.

11:21We can't confirm that, you know, we can't in like one minute figure out a way to make sure no non-U.S. citizens are using the models. So, that's what happened. I mean, it was a crazy couple of hours. Anthropic comes out with a statement later that night saying they're trying to work with the Trump administration on this, spent all weekend virtually meeting with the Trump administration and then flying three employees out, who I detail in the piece we just published, to D.C. to talk to them in person. Notably, Dario was not there in person. He is virtually joining all these meetings. I can confirm he was not at a wellness retreat, which is what some reports say, but they won't tell me where he is, but he's not at a wellness retreat.

12:00That's all I know. So yeah, there's a lot going on. Very dramatic. All the sources I spoke with over the past couple of days were saying the whole tech industry and the cybersecurity community in particular are just talking about how, you know, one source told me, Beijing is laughing at us right now. It was interesting because some cybersecurity leaders in the space came together to write a public letter and they said, look, not all of us believe in regulation, but if you were going to regulate it, this is not the way. which I think is just kind of indicative of this whole situation.

12:36Nilay Patel:We're going to pause here for a quick break. We'll be right back.

12:47Nilay Patel:Support for the show comes from ServiceNow. AI was supposed to handle the parts of the job you hate. Instead, it just describes them, suggests what to do about them, and then leaves you to do it. That's not help. That's homework. ServiceNow's AI specialists are different. They're not a tool. Think of them as digital teammates who actually do the work from start to finish. Cases get resolved, requests get processed, loops get closed, and most importantly, no extra work for you. Because when you can truly delegate to AI, you can get back to the work only you can do. The work that requires a person with ideas and judgment.

13:27Nilay Patel:And you know, a pulse. To learn how to put AI to work for people, visit servicenow.com.

13:37Nilay Patel:Support for the show comes from Even Realities. Think about the last time you walked into a big meeting and wished your talking points could just stay with you without losing your place or your train of thought. Even Realities built exactly that. Even G2 are productivity smart classes designed to keep real-time support right in view with teleprompting, conversation support, real-time translation, AI assistance, and more. They help you stay on top of work and daily life. And unlike most smart glasses, they're designed to look and feel like premium eyewear with no camera and a lightweight 36 gram design you can wear all day.

14:17Nilay Patel:The more contacts you give them, the smarter they get, adapting to how you work and what you need. To learn more about Even G2, Go to evenrealities.com and see how everyday smart glasses keep helpful information in sight so you can stay productive and hands-free throughout the day. And for our listeners, use promo code DECODER at evenrealities.com to get 10 % off Even Ring 1 and or Even Clip when you add them to your Even G2 order. That's evenrealities.com promo code DECODER.

14:59Nilay Patel:we're back with verge senior ai reporter hayden field discussing anthropics new showdown with u.s government over ai regulation maybe you believe in aggressive government regulation maybe you don't there's a way to operate the regulatory efforts that is rigorous and thoughtful and careful and predictable nothing about this seems like it was rigorous or thoughtful or predictable. I actually want to start at the beginning. Andy Jassy, Scott Bessent might be smart guys. They obviously have very senior positions in our society. They're not experts in AI. How did we get to Andy Jassy, the CEO of Amazon, relaying concerns about AI security research to Scott Bessent, who does not really oversee AI policy in any meaningful way, and that resulting in a 90-minute ultimatum delivered to a company whose CEO is absent.

15:54Nilay Patel:Like, all of that just—I don't think your politics should matter. If you just examine that, you have to say either this is the worst security breach in history that the CEO of Amazon is going to call the treasury secretary, or this is just a bunch of guys who want to feel important. And I can't quite figure out which one it is. I think it's a combination of a couple things. One, it's the fact that, like you mentioned, a lot of these guys are not experts in AI. And something that I'm not saying it's not, like any jailbreak of this is obviously, you know, serious and worth considering. But I think that it's the type of thing when you don't have direct knowledge of how these systems work, you're going to take a really black and white immediate.

16:39Like, I think that these guys weren't experts. It's Andy Jassy hears about this from his own researchers. His own researchers, by the way, who had been going back and forth with Anthropic for a couple of days on this. So they weren't, you know, calling the president about it. They're like, OK, let's discuss this and kind of, you know, figure out what we can do here. The researchers allegedly didn't seem to think it was, you know, the end of the world. They're just like, oh, we uncovered this. You know, let's let's discuss. Anthropic was working on it. And then Andy Jassy hears about it. He makes this call.

17:10Everything hits the fan. Which makes sense because if you are not an AI expert, and even if you are an AI expert, maybe you're not an expert on cybersecurity and how exactly these systems work, of course. So you're going to freak out. I mean, it's kind of understandable, but what is really strange is what happened after that. So, you know, making this phone call, the administration becoming aware, okay. But what I think is really weird is the 90-minute ultimatum, them not giving them additional details when they said, hey, let's talk about it. Are you talking about the vulnerability we're already aware of?

17:40or are you talking about something else? And can you just tell us about it so then we can talk about it? They said, no, 90 minutes, do it or die. And then making it so that no foreign national can access the system ever. That is what's really strange. Just kind of the way all these cards fell is everyone in the tech industry thinks is just really strange. And it's kind of indicative of, like you mentioned, the way that the Trump administration has been operating with tech. A lot of times it seems to be President Trump has one conversation and he's kind of swayed from that most recent conversation.

18:11So he's been very anti-regulation, very pro-American AI, advancing, exporting American AI, which is something that the Commerce Department is requesting, it has an RFI out on right now. They're all about exporting American AI and advancing and beating China. But then they do this thing that kind of kneecaps everyone with no notice and could very well impact not just Anthropic, but also OpenAI, Google, every competitor in the space that also has what they call a semi-equivalent mythos product. So yeah, it seems really disorganized. And it's very interesting that this often ends up in a scramble with all these CEOs kind of trying to race to kowtow to the Trump administration without really being grounded a lot of times in science or evidence that really matches what's going on.

19:06I feel like the level of freakout is not usually matching, you know, the evidence there.

19:11Nilay Patel:Amazon is an investor in Anthropic. They're a major partner. Why did Andy Jassy kneecap his own investment in this way? Well, Amazon typically like does a lot of red teaming just normally. You know, it's kind of just part of their process. So it makes sense that this would have been tested and then unearthed. but what's interesting is why Andy Jassy called the Trump administration about it. I'm not sure. Maybe it was a typical call that was already happening and he just happened to mention something offhand. Maybe he was calling them and saying, yeah, good thing you did the executive order because X, Y, Z.

19:53Don't know. But what is interesting, and I'm glad you asked this because Because a lot of the sources I spoke with said that the stuff in the alleged jailbreak and in the Amazon white paper that's been circulating, all of it could be achieved by OpenAI's GPT 5.5 and some other models. So that's the allegation that these skills are not unique to Fable 5, the skills in question as part of the jailbreak. Mythos is obviously more powerful than Fable because it doesn't have safeguards. But when it comes to Fable and the problems that were brought to light by Amazon, it seems like those problems are not unique to Fable and that GBD 5.5 can also achieve those.

20:38So it's like something that a lot of people in the industry are asking is why Anthropic?

20:43Nilay Patel:So a real element of this is that it's always Anthropic. Anthropic has the most contentious relationship with the Trump administration. They were designated as supply chain risk by the Department of War. But the posture here is different than the usual Anthropic drama. Historically, Anthropic has said, we don't want you to use our tools for X, Y, and Z. And the Trump administration has said, no, we get to do whatever we want with the AI tools. That's why they were designated as supply chain risk by the Department of Work. They said, we have some red lines about how we want to be used in the military context.

21:13Nilay Patel:And Pete Hegseth said, nope, I'm going to do whatever I want. You have to allow all lawful purposes. That's the line. And because you won't, you're a supply chain risk. There's a lawsuit that's being argued about. Here, it's the opposite. The Trump administration is worried that the model is too powerful and can be used for too many things. Ananthropic seems to think that that belief is overblown, that the administration is overplaying a thing that they can fix. They just need the time to do it and some rigor and conversation about how effective their fix is. How do you put that together? That on the one hand, the Trump administration is anti-AI regulatory efforts.

21:48Nilay Patel:On the one hand, they say, we get to do whatever we want with the models, make them as powerful as you can. And then here, the model is too powerful and has to be pulled before any foreign national can even look at it. The administration is trusting these companies to kind of tell it what it should be doing. We've seen that a bunch with, you know, any regulation that has happened or, like, has become voluntary. You know, lots of drafts have been adjusted after the Trump administration talks to AI companies. They already don't like Anthropic. You know, there's already a big trust breakdown between the two from the supply chain risk thing, from the lawsuit, from drawing red lines.

22:33And so I can't imagine that this exact situation would have played out if this had been a competitor. I mean, OpenAI has a mythos semi-equivalent. again, like, you know, no company is really saying theirs is as good, but they say it's, you know, pretty good. Microsoft, Google, I can't, I think that the freak out would have still happened, but I cannot imagine the rest of it would have played out the way it was. I think they would have given them more time. I think they would have had more of a conversation. And I don't think they would have made this foreign nationals rule that has like, you know, sent shockwaves through the whole industry and the world.