The Pentagon Wants an Obedient A.I. Soldier. Will It Get One?

0:00Every idea starts with a problem. Warby Parker's was simple. Glasses are too expensive. So, they set out to change that. By designing glasses in-house and selling directly to customers, they're able to offer prescription eyewear that's expertly crafted and unexpectedly affordable. Warby Parker glasses are made from premium materials like impact-resistant polycarbonate and custom acetate. And they start at just$95, including prescription lenses. Get glasses made from the good stuff. Stop by a Warby Parker store near you. I don't know if you have this feeling, but I constantly find myself daydreaming about finding an expert who can solve a very specific problem in my life.

0:38And one specific problem that comes up every year is filing taxes. But luckily, finding someone who can solve that problem is easy with TurboTax. They have dedicated tax experts who are there to help you every step of the way. With TurboTax Expert Full Service, you can match with an expert who will handle your taxes from start to finish, no matter how complex. You can even hand off all your tax stuff within the TurboTax app. And then you can see updates on your expert's progress while you go about your day. You can check exactly where your tax return stands right there on your phone. And then if someone ever says to you, man, I wish there were an expert who could do my taxes for me, you can say, oh, I know someone.

1:18Visit TurboTax.com to match with your expert today. It's only available with TurboTax full-service experts. The real-time updates are only in the iOS mobile app.

1:35Hey, Gideon. Hey, Tyler. Thanks for coming back so quickly. Thank you for having me. Would you say that your reporting on AI has made you more freaked out about AI potentially being used in military operations, or are you less freaked out? Oh, significantly more freaked out. Part of it is just that, yes, it seems to allow us to unleash the forces of destruction on a much greater scale. Then, of course, there's the question of, like, the game theory of this, which is, like, if we are doing this, then certainly our adversaries are going to be working on doing this kind of thing. And then finally, there's the much more ticklish question of to what extent can we, like once we have trained it to produce this amount of devastation, to what extent are we really ultimately able to control it?

2:26That's Gideon Lewis-Krauss, a staff writer at The New Yorker. Last month, Gideon came on the podcast to talk about Anthropic, a company that has committed to developing an AI that's grounded in the principles of ethics and safety. But just days after Gideon came on the show, reports began to emerge that the Department of Defense had allegedly used CLAWD, Anthropic's series of large language models, in military actions in Venezuela and later Iran, potentially without the company's knowledge. Recently, when Anthropic tried to put guardrails around the way that the Pentagon is using their technology, Pete Hegseth and other DOD officials bristled, leading to a wild escalation in which the Pentagon labeled Anthropic a supply chain risk, essentially blacklisting the company.

3:11Anthropic is now suing in response. I wanted to talk with Gideon about what we actually know about the role of AI in these recent military operations, how this conflict between Anthropic and the Pentagon took shape, and what it might mean for how companies handle similar pressure from the Trump administration. This is The Political Scene. I'm Taylor Foggett, and I'm a senior editor at The New Yorker.

3:37So let's rewind a little bit. So when you were on the show recently, I think like a month ago, you described Anthropic as this company that was essentially made up of lovable nerds who were deeply committed to ethical AI. And then next thing you know, reports are coming out about Anthropic's series of large language models, Claude, being used in the Maduro operation in Venezuela, and more recently in the military operations in Iran. Were you surprised when you started seeing these headlines, or did you always have a feeling that something like this was going to happen? I mean, something like this was bound to happen.

4:13There's no way it was not going to happen. And there's been a tendency, especially online, to sort of say that Anthropic in particular was kind of hoist on its own petard, that it's like, you guys have just been talking for years about how incredibly powerful this is, and so obviously the government was going to cotton to this, and now the government has believed what you've been saying, and now the government wants it for itself. So just getting more into the nitty-gritty to the extent that we can about how Claude has been used in military operations so far. I mean, what do we know, if anything, about how Claude was used in the raid that resulted in the abduction of Venezuelan President Nicolas Maduro back in January?

4:55I mean, we seem to know almost nothing and the event that triggered all of this is that somebody apparently, according to the government at Anthropic, called a friend at Palantir and said, hey, you know, we heard a rumor that Claude was somehow used in the Maduro raid. Do you have any idea what was going on? And then that got reported back into the White House. And can you just remind us what Palantir is? Palantir is essentially an information synthesis company. And like what they offer is an interface that allows for huge amounts of information from disparate sources to be fed into one place where one person can access whatever seems to be decision relevant at a given time.

5:42And the difference with Claude is that Claude was the first LLM that was certified for use on classified servers. So there was information that they could use Claude to process that they could not use Claude's competitors to process. I mean, I guess I'm wondering, like, this idea of the Department of War using Claude, is that as formal? Because obviously there was this contract between the government and Anthropic, but is it the kind of thing where they're being encouraged to use this service, or would it be more like if I just one day decided to, like, use Claude to try to edit a New Yorker piece and then, like, our boss found out?

6:20So the initial contract that Anthropic was involved in was through Palantir. That Palantir had signed a contract with, or they've had ongoing contracts with the government. They had signed a contract in 2024 where Anthropic was essentially like a sub-vendor of that contract, which is like, we have this system, and this is a system for real-time information analysis that's going to pull in, you know, climate data and satellite imagery and signals intelligence and all sorts of things and put it in one place so that one person, you know, using what's essentially just like workday software can have an overview that helps them immediately synthesize like vast troves of real-time information.

7:04And that system allowed a user to just pick from a drop-down menu of different AI options. Claude was one of them. And I had heard that Claude was kind of the overwhelming favorite for users of Palantir software because Claude was just the best at doing this stuff. So, I mean, before we get deeper into the relationship between Anthropic and the Pentagon and how that fell apart so dramatically, I just want to talk a little bit more about how Claude may have been used in military operations in Iran. I mean, we certainly, we've only had a little bit of reporting about how exactly Claude was used in Iran.

7:45But it seems like the initial reporting that we've seen has suggested that it was used, for example, for target selection. And that there was a quote, I think, in the Washington Post saying that the kind of battle planning that might have taken people, you know, months to put together was able to be done using AI as part of a broader system. It's not just like they're sitting down with a chatbot, like asking questions like, where should we bomb, that it's incorporated into the system that Palantir uses, that's kind of a total information awareness system, that that was able to generate, you know, a thousand targets in the first 24 hours.

8:24Of course, when, you know, we think of targets and faulty targets, like the first thing that comes to mind is just the horrible assault on that girls' elementary school, which killed around 170 people. The reporting doesn't show that Claude was necessarily used to pick that target, But the reporting has suggested that that target was based on bad intelligence. You know, like apparently, I think the school used to be a military base or was close to one or something like that. So it does kind of introduce this idea of like AI using faulty intelligence to suggest a target, which seems like one of the kind of the big things that we would have to be worried about here, right?

9:01Right. Well, so to me, kind of the key question in all of this stuff that sometimes doesn't get asked when people really are quick to blame a technological solution for something is like, what is the relevant counterfactual here? Is that like, if this was in fact based on outdated military intelligence, would if we had abstracted Claude out of this equation? Would a human have made the same mistake? Would a human have made the same mistake? Yeah. We don't know. And would, you know, so to what extent can you say that this was specifically an AI-related failure mode? Because you could also certainly imagine cases in which a human would have acted on bad information.

9:43And an AI, which would have, you know, in theory had access to orders of magnitude more information, would have said like, oh, no, hold up here. Like, that's not right. And so people who do AI safety, and this has come up a lot in terms of keeping a human in the loop and what they call the kill chain, they love to point to this example of Lieutenant Petrov during the Cold War who noticed potential launch of atomic missiles and decided that this had to be a drill and didn't initiate a counterattack and potentially save the world from nuclear destruction. and you know this isn't you know often used as an act of like like the heroic individual conscience that has like prevented us from destroying each other and it certainly was but you know it's very possible to imagine a system and when in which an ai in that case said whoa whoa chill out this is clearly like just a drill like this is not real and that it would have made the same decision i mean i don't think based on the information that we have now although this very quickly gets into like kind of the heart of the matter, whether we even know enough to say that there would have been an appreciable difference between like how a human would have acted in some of these high stakes situations and how an AI would act.

11:01I mean, the appreciable difference is accountability, right? And I guess like this is kind of jumping to the next part of the conversation, which is about what sort of destroyed the relationship between Anthropic and the Department of Defense or the Department of War, whatever you want to call it. But just like it's maybe a human and AI would have made the same mistake, but at least when it's a human who makes that mistake, it's like it's someone to point to. It's their fault. And it seems like the only reason why the government would be so interested in fully autonomous weaponry, which was one of the sticking points in the deal between Anthropic and the Pentagon, is that they're trying to, you know, sort of shunt that accountability.

11:38I mean, it seems like they kind of want like an AI scapegoat, right? Or is it just so much faster if, you know, Claude can just pick all the targets and execute on them? Well, in a sense, there is an open question whether it's better or worse to have an AI scapegoat. I mean, so people talk a lot about accountability, which clearly is the crucial question here. But so often accountability is just like finding a scalp, right? Where like, that's not meaningful accountability in terms of what was the process that led to this catastrophic outcome. That was just like we had an outcome. And like that was the guy who happened to press the final button.

12:11So like, well, let's put his head on a stake. And in theory, that represents accountability. In practice, like, is that the kind of accountability that we want? I'm not so sure. So let's take a break, and then when we come back, I want to talk more about the sort of war between Anthropic and the Department of Defense. This is the political scene from The New Yorker.

12:43Wired has always put a microscope on the people, power, and forces shaping our world. Uncanny Valley brings that same fearless reporting straight to your feed. Is Doge finally over? Will AI actually democratize American healthcare? Each week, Wired journalists from across the newsroom are going to unpack where politics, technology in Silicon Valley collide. From conversations with tech leaders across Silicon Valley, internet fandom investigations, and government crackdowns on rigged gambling, we're taking you all over the news cycle, going straight inside the priorities, pressures, and power plays driving today's biggest decisions.

13:20Uncanny Valley tackles the questions keeping you up at night and helps make sense of the future taking shape right now. Listen to new episodes every Thursday, wherever you get your podcasts.

13:41So in your recent piece, you give a really thorough overview of this escalating conflict between Anthropic and the Department of Defense. As you mentioned earlier, it seems like the Department of Defense first started using Claude through Palantir. Palantir. And then I guess it was in July of last year that Anthropic signed its own, was it$200 million contract with the Department of Defense. So what were kind of like the stipulations of that contract? Like what did it look like and what were like the parameters that that Anthropic initially set out for how the tech could be used by the Pentagon?

14:15So, I mean, we certainly don't know all the details of that, but we definitely know what the two ultimate sticking points were, which was that in this contract, there were two stipulations on Anthropik's part, which was they did not want Claude used for fully autonomous weaponry, meaning without some human involved, and they did not want Claude used for domestic mass surveillance. Now, the tricky thing is that domestic mass surveillance is not a legal term. It has no legal meaning. Basically, their worry was that there are plenty of legal loopholes in our privacy regime that people have been pointing out for a really long time, where essentially there are certain things the government is prevented from doing, not because it's illegal, but because it's simply too cost inefficient to do.

15:06And if the government wants to assemble a profile of you, Tyler, they could assign a person to go through all of that data, find which data points correspond to you. All the Taylor Swift videos. All the Taylor Swift videos. Yeah, you unfortunately have left a very long data trail. And they could compile a dossier about you if they wanted to. But they can't do it for 350 million Americans unless they hired millions and millions of people to be doing this all the time. Whereas with Claude, it would be pretty easy for Claude to just do that for everyone based on publicly available information. And they did not want the government to do that.

15:46And I'm going to quote your own piece back to you, but I found this line to be like really illuminating in terms of like what it would actually look like. But you write, a panoptic cloud could make tailored watch lists all day long, say, matching concealed carry permits with unpatriotic tweets or cross-referencing protest attendance with voter rolls, which, like, really helps you get a sense of, like, how bad and overwhelming it could be if cloud were actually used in this way. Okay, so they said no to mass domestic surveillance, and then they said no to fully autonomous weaponry, which is what we talked about earlier.

16:18And, I mean, this was back—they signed this contract in July of 2025. So why is it that, you know, like several months later, that contract is falling apart and they're all of a sudden fighting over these terms that it seems like the Pentagon must have agreed to back then? So some of this is kind of murky and depends on whom you ask. But it seems like what happened is that Emil Michael, who is a former Uber executive, he was the one who was behind like using Uber to track journalists who were, you know, writing negative things about the company. His reputation precedes him. He was brought in as an undersecretary for research and development, I think.

17:01At the DOD. At the DOD. And, you know, according to the government side of the story, he basically looks at their existing contracts and he decides that he wants to render them uniform, that he doesn't want certain companies to have certain stipulations, that it's too hard for the government to be negotiating on an individual basis with each defense contractor, and that he kind of wants uniform contracts across the board. now from the alternate perspective on this is that well actually plenty of companies already have lots of stipulations in their contracts like they are contracts contracts are things that are freely entered into and that you offer your terms and they you know your the counterparty accepts them or doesn't accept them and that you know it's not like when you buy an f-16 from lockheed martin you can do whatever you want with it you have to still agree to like certain contractual stipulations there.

17:55But what Michael didn't like is that kind of according to White House sources or according to administration sources, he just wanted a clean thing where all defense contractors were going to have the same agreements, which is that the things that they sold to the government could be used for all lawful uses. Meaning, we're the government, we follow the law, you know we follow the law, trust us, we always follow the law. And all we want is to be able to use the thing that you're giving us within the bounds of the law. The laws are, you know, a result of a democratically legitimate process. And we can't just be letting, you know, private individuals in San Francisco, like, effectively setting their own laws based on the technologies that they're selling to us.

18:43Totally. But it wasn't that the government was just going to rescind Anthropik's contract. It's that they had this whole other plan, which was to label them a supply chain risk, right? Yeah. So to take a step back, I mean, the cleanest possible thing, as even many conservatives pointed out, is that the government could have just walked away from the contract. They could have said, look, you know, we liked using Claude. It was useful. But we're not going to use Claude if you're going to be so meddlesome and interventionist here. And so, like, let's just both walk away from this. and Anthropic would have lost$200 million, which is a rounding error on annualized, you know, their current...

19:19What's their valuation? Well, the valuation is about$380 billion at last report, but their revenue, like, doubled in the last month. They're currently on path for an annualized revenue of, like,$20 billion. So they're fine. So$200 million is something they make in a couple of days. Like, they didn't care about the money. They weren't doing it for the money. So that certainly would have been the easiest thing to do. But instead, what exactly happened? I mean, what led to Hegseth labeling Anthropica supply chain risk? And what does that mean? Well, that's a really good question. Now, typically, this is a restriction that has only ever been applied to companies, typically infrastructure companies that have ties to our adversaries.

20:02So for example, Huawei, like we know that Huawei has a close relationship to the CCP. Huawei is a telecoms infrastructure company. it makes sense for the government to say we don't want anyone we're working with to be using you know Huawei infrastructure because we don't know what it's going to be sending exfiltrating and sending back to Beijing. So on one level there's there would have been okay any government contractor while in the course of doing their government business cannot simultaneously be using an anthropic product like Claude meaning that like for your government workflows you can't rely on Claude.

20:37And that would have been extreme and unprecedented, but not company destroying. But what Hegseth initially threatened was what other people have called a secondary sanctions regime or a secondary boycott regime, which would have interpreted the law to mean we can prohibit any company that does any business with the government from doing any business with Anthropik. And this, like, irrespective of what business they're doing. And this would have destroyed the company because, you know, Anthropic relies on these companies like Google and Amazon for cloud services. And they rely on NVIDIA for chips.

21:15And, like, this would have fully cut them off and placed, like, you know, what were clearly extremely punitive sanctions on the company. So that would have been existential. Now, people pointed out, you know, national security lawyers pointed out right away that this almost certainly vastly exceeded Hegseth's statutory authority and that that was never going to fly. And that finally, a week later, when Hegseth did make this official announcement that it was a supply chain risk, it was the narrower version of the law. And even that seems, you know, Anthropik filed a countersuit or lawsuit two minutes later.

21:51And even that seems probably beyond what his authority was. And, like, Anthropik will probably prevail in court. I want to go back to an idea that you mentioned earlier, which is that the Pentagon is used to buying jets from Lockheed Martin or, you know, semiconductor chips. And usually it's a situation where, you know, the thing is just kind of sold to the government and the government gets to decide what to do with it. Basically, you know, it's this category that you refer to in the piece as normal tech. But obviously, or maybe not obviously, it kind of seems like AI is not normal tech. It's this thing that you wrote about, you know, so wonderfully, you know, about a month ago, just that it's this thing that's almost like weirdly sentient and even anthropic itself doesn't quite understand what exactly is happening with Claude and what its capabilities are.

22:45And so I guess I'm wondering if you can just walk us through this, you know, this normal tech idea and like why it is that AI should maybe be considered something different and whether that is something that the government is ever going to be able to fully kind of like wrap their heads around. So the argument for AI as a normal technology, it's not saying that AI isn't potentially hugely important and transformative. It is saying that AI is potentially hugely transformative, but also not categorically different than anything else that has come before. That there is historical continuity here, that there have been other things like electricity and the semiconductor and the internet and the private, you know, the home computer, all of which were transformative in their own ways.

23:33But none of which thoroughly transformed life and geopolitics as we know it. And, you know, what one government official said to me, like what this fundamentally comes down to is a division over this question of normal technology, where if AI is like a normal technology, then it is like a tank. It is like a gun. It is something that behaves the way that you expect it to behave. It is a tool that is used in relatively predictable ways. It's not really going to surprise you. And so current laws apply, and we can't have private individuals saying, like, no, no, no, we need, like, further restrictions.

24:11And so the alternative view is, no, no, no, AI is something that you cannot place in a reference class with all of those other things. That AI is so fundamentally different and transformative that, in fact, actually, tomorrow is unlikely to resemble today. Is it possible that right now, as we speak, AI is not a normal technology, but eventually it could be one? I guess I just wonder, like you mentioned like electricity and it's like, what if when electricity was invented, it was just like we didn't quite understand how it worked. Every time we tried to do it, there was like a 40 percent chance that you would electrocute yourself and die.

24:50I guess I wonder if like there are just like a lot of kinks that are still being ironed out, but that there will one day be a point at which we can regulate it or sort of treat it the way that we would a gun or a tank or a phone. So this is exactly, this is the crucial question. And in AI safety communities, this is called the alignment problem. And kind of the classical way to look at the alignment problem is we want to make sure that an AI that is like potentially one day as intelligent or more intelligent than we are remains aligned with, quote, human values. Now, as anybody could point out, like what the hell are human values?

25:26Like that's not a load-bearing concept. Like different communities have different values. Different people have different values. Now, another way to set aside the jargon of the alignment problem and this placeholder usage of a phrase like human values, it really is just a question of are we building something that we can reliably control or not? the government is essentially saying that's not a problem that we like we don't believe it's a problem it's a normal technology like one administration official compared it to a computer virus and he was like we used to have problems with computer viruses and then we figured that out and like now we don't really have problems with computer viruses anymore and that like all of these issues of control are actually just like it's going to be a long game of engineering whack-a-mole where when we see behaviors that we don't like we're going to patch them and we're going to be fine.

26:19And the AI community is essentially saying like, well, it's news to us that you guys suddenly solved the alignment problem that like, you know how to control this thing because we build it and we don't know how to control it. And it basically seems like the government is saying, we want an AI that is going to be a perfectly obedient soldier, an AI that is going to follow all orders and carry them out in the way we would expect it to carry out those orders. And Anthropic is saying back, what you want doesn't exist. And that like you are opening a gigantic Pandora's box by saying you essentially want an AI with all the safeguards removed, because you basically just trust that it's going to do what you want.