Pentagon Insider: What's Next For Anthropic and The Department of War — With Michael Horowitz

22:00Big Technology:The US military has been using autonomous weapon systems for more than 40 years. I think people really underestimate in some ways, the degree of autonomy built into modern weapon systems, even in a world like before what we would call like AI today, like a good old fashioned AI, you know, like kind of world. Like, let me give you two examples. One is something like a homing munition or a radar guided munition, where somebody may believe that there's a radar over the horizon and they fire a missile at that radar. There's no human supervision of that missile after it's launched. It just turns on a seeker and it goes and hits the radar.

22:38Big Technology:What if that radar is on top of a school? What if that radar is on top of a hospital? You don't know, it's gone. Second example is something called the close-in weapon system, which is a weapon system that protects ships in some military bases from essentially massed attacks. So if there are like 10 missiles coming in and you couldn't even point and click at all of them, if you were an operator, you can flip on essentially an algorithm that automatically detects and shoots at those. The US military has been using that system since like 1980, as have like dozens of militaries around the world. And so we need to be careful then when we talk about autonomous weapon systems and to be clear about like, what is the thing that we are worried about?

23:23Big Technology:And what is the thing that we think the technology is ready for or not ready for? And as I said before, I think Anthropik is absolutely right that they're like tech isn't ready for prime time and incorporation like at the edge in an autonomous weapon system. Also, if you think about like the compute at the edge, like how would you even like fit that into a missile? Like, I don't know, but the, the, but like, this is a, there are like so many other ways. Like if you want an autonomous weapon system, there are so many ways you would do that that don't involve LLMs essentially, but public service announcement, the phrase autonomous weapon system is the appropriate term of art and autonomous weapon system is a weapon system that after activation selects and engages targets without further human intervention, like period dot.

24:11Big Technology:That is the way that the Pentagon, at least, different people have different definitions, but the way the Pentagon, at least, defines what an autonomous weapon system is.

24:18Michael Horowitz:Can I tell you where I think so much of the confusion is coming from now that you explain this? All right. So this is, so I've worked a couple years in the government. So, and you talked about the technology. We both know that the government technology tends to lag behind commercial use cases by a couple years. Just a little bit. The AI industry has gone through two phases over the past year and a half. There was a chatbot phase of AI, right? And that also includes content synthesis, summarization, these type of things. And now they're moving into an agentic moment, right? I think there is a misconception that the government is already on agentic, right?

25:04Michael Horowitz:Where the technology takes its own decisions. But really, what I think I'm hearing from you is it's in the chatbot phase. It's still this year, two years behind commercial. And this this worry about the technology getting too agentic is sort of misplaced because of where the government is.

25:21Big Technology:I think that that's probably broadly right. Although, frankly, part of what Anthropic was trying to do in doing classified work with the Pentagon in the first place was fix that and getting in behind the scenes and ensuring that their tech, that America's warfighters had access to things closer to the cutting edge. But another thing to keep in mind here is the way that testing and evaluation standards, or what the military calls T &E standards, differ from what you would need to maybe toss a piece of technology out in the commercial market. You know, imagine you were releasing an either like last gen like chatbot kind of system or this gen kind of agentic system into the marketplace as a company.

26:07Big Technology:If there are errors and problems and whatever, like those are embarrassing, but you fix them on the fly. And frankly, like getting there first can get you market share. There's all sorts of like economic reasons why like a for profit company might do that. When you release stuff that doesn't work well in the military, people die. And so the incentive structure is very different. And so the testing and evaluation of these systems is thus very different in a military context, like the level of reliabilities and cybersecurity, et cetera, you need to hit for something to be like fieldable is very different.

26:37Big Technology:So people should, at least in theory, like if the system's working properly, like be reassured on that front.

26:44Michael Horowitz:Exactly. OK, I want to talk now about the government's perspective and what this supply chain risk designation might do to Anthropic. Let's do that right after this. If a driver in your fleet got in an accident tomorrow, could you prove what actually happened? Without footage, it's much harder. So your insurance rates spike and you're stuck paying for it. That's why so many fleets choose Samsara's AI-powered dashcams, clear video evidence, real-time alerts, and coaching tools that help prevent accidents before they happen. Samsara AI helps reduce crash rates by nearly 75%. For instance, the city and county of Denver saw a 50 % reduction in false claims against them and a 94 % reduction in safety events overall.

27:31Michael Horowitz:This is the kind of visibility that every operation manager needs. Don't wait for the next accident to take action. Head to samsara.com slash big tech to request a free demo and see how Samsara brings visibility and safety to your operations. That's samsara.com slash big tech. Samsara. Operate smarter. You want to eat better, but you have zero time and zero energy to make it happen. Factor doesn't ask you to meal prep or follow recipes. It just removes the entire problem. Two minutes, you get real food, and you are done. So remember that time where you wanted to cook healthy but just ran out of time?

28:09Michael Horowitz:You're not failing at healthy eating. You're failing at having three extra hours every night. Factor is already made by chefs, designed by dieticians, and delivered to your door. Inside, there are lean proteins, colorful vegetables, and healthy fats. It's the stuff that you'd make at home if you had the time. There's also this new muscle pro collection for strength and recovery. You always get fresh and never frozen food. It's ready in two minutes, and there's no prep, no cleanup, and no mental load. Head to Factormeals.com slash Big Tech 50 off and use code Big Tech 50 off to get 50 % off your first Factor box plus free breakfast for one year.

28:46Michael Horowitz:The offer is only valid for new Factor customers with the code and qualifying auto renewing subscription purchase. Make healthier eating easy with Factor. And we're back here on Big Technology Podcast with Professor Michael Horowitz of the University of Pennsylvania. Also, the former Deputy Assistant Secretary of Defense for Force Development and Emerging Capabilities. All right. Let's talk a little bit about the government's perspective. Is there validity in the government's perspective of telling Anthropik, you might have these thoughts about how to use your technology, but you don't tell us what to do.

29:22Michael Horowitz:We should be trusted to be the ones who determine that, not you.

29:27Big Technology:I think there are, the government has a point in some elements here. And let me tell you what I, let me tell you what I mean. And, you know, I hinted at this before, when the government, the government's used to buying a technology, to think about when the government buys hardware, the government buys a fighter jet or a submarine or a missile or something. The companies that build those technology don't tell the government how to use it. The assumption is that the government will follow the law when it uses those technologies. Since like, otherwise, like kind of, what are we doing here? And, and so the government viewed these requests from Anthropic and their refusal to yield on them as essentially challenging the Pentagon's authority.

30:15Big Technology:And, and this is, I think part of where the, what is a little bit the like culture and personality clash that we were talking about before, like where it, where it comes from, because the, Pentagon saying, hey, we follow the rules. That is a thing we definitively do. You don't need to worry that we won't follow US law. You don't need to worry that we will go do crazy things that the technology isn't ready for. We have law and policy and process designed to ensure that that doesn't happen. We don't let other vendors tell us we can use their tech in scenario X, but not scenario Y. So what you're asking for is unreasonable.

30:54Big Technology:And and I understand that from the government's perspective, like why they might why they might say something like why they might say something like that. That's also why, you know, as I as I suggested before, I think what we're really seeing here is to start us off in this part of the conversation. But what we're really seeing here in some ways is a breakdown in is a breakdown in trust.

31:14Michael Horowitz:exactly and so the question is what happens next and yeah i in some ways i i do believe that if if your government and you'd think you can't trust your technology vendor you should probably swap them out um but that's not exactly that's not where the government stopped here what they did was they they deemed uh uh anthropic a supply chain risk uh and that means that the company cannot work with U.S. government agencies. And War Secretary Hegseth went further. He said, effective immediately, no contractor, supplier, or partner. That does business with the United States military, may conduct any commercial activity with Anthropic.

31:58Michael Horowitz:That includes Amazon, by the way, who is a U.S. government contractor and also hosts Anthropic models. I have this from a source with knowledge of the department's thinking. The feeling inside the Department of War right now is they want to destroy Anthropoc. What do you think about this reaction?

32:18Big Technology:I have a lot of thoughts about this. Let me start with the bottom line, which is crushing one of the most innovative companies in the world and salting the earth is not good for American innovation or the American economy. And so, dear God, let's hope they work it out. but like backing up a little bit, the right, like you would have met in a normal marketplace situation, like one can think that the Pentagon's view of this is reasonable or unreasonable, but it is what it is. And in a normal market view of this, the Pentagon would do one of two things. Either it will say we will work with Anthropic on these use cases, but not those that, you know, like they don't want to do.

32:56Big Technology:And if we want to do those in the future and reminder, they're not doing them right now, so there was no dispute about a current or planned future use, then we'd find another AI vendor to do that. And that, you know, whether it's XAI or OpenAI or like somebody else like that would do that. Or the government could have said, you know what, it's not worth it for us to do business with Anthropic. Let's cancel the contract. We'll off-ramp them and we'll load, you know, we'll bring XAI or OpenAI or like somebody else on to meta, whatever, like to address this. That's obviously not what happened. It's not just that the government has labeled Anthropic as a supply chain risk.

33:35Big Technology:It's in some ways even more baffling than that. And the supply chain risk designation is for companies believed to present a sort of clear danger to U.S. national security. Examples of companies labeled as a supply chain risk are Huawei. You know, like Chinese companies where the fear is that if a U.S. government agency worked with them, they might insert backdoors or vulnerabilities that could place U.S. national security at risk. That's not really what we're talking about here. And so I think a lot of people have wondered whether that designation would hold up in court. And also, it's not clear that the supply chain designation has actually been delivered to Anthropic yet.

34:17Big Technology:It hadn't as of about a day ago, although it's still been threatened. I mean, Anthropic, I'm sure, will be in court as soon as they get the letter and actual designation. And it was striking, of course, that, I mean, no pun intended, that less than 24 hours after the supply chain designation, the US government was using Anthropics technology in the context of Operation Epic Fury against Iran. How could they really be a supply chain risk if you are using them in the context of ongoing military operations. But the government's gone further. They've, on the one hand, said they could label Anthropic as a supply chain risk, or are labeling Anthropic as a supply chain risk.

34:58Big Technology:They've also said that they're considering using the Defense Production Act to compel Anthropic to work on use cases with the government that Anthropic might not want to. And the Defense Production Act or DPA was designed to ensure that, say, the government was first in line for vehicle manufacturers. If there was a war going on and you needed more tanks or something like that, it was not designed for like this kind of environment. But the government's thinking about these two different things, both the Defense Production Act designation and the supply chain designation, and they pointed opposite directions.

35:34Big Technology:One says you can't work with the government and one says you have to work with the government, like points to some of the confusion here.

35:40Michael Horowitz:Now, you've worked within government agencies. You've worked within the Department of Defense. This is from Reuters. State Department switches to OpenAI as U.S. agencies start phasing out Anthropics. And this article says, leaders not only at the Department of State, but Treasury and Health and Human Services have directed their employees to abandon Anthropics' language-trained chatbot platform Claude on orders from President Trump. They joined the U.S. military in dropping use of the platform. I'd love to get your perspective just about the speed that governments move. And when you think about governments evaluating certain technologies, because you've been inside one, what sort of damage do you think this has already done to Anthropic now that we're seeing so many agencies move off?

36:27Big Technology:There are a couple of different pieces here, I would say. And again, a lot of people seem to I'm not a lawyer, but a lot of people seem to think that this this won't stand up in the the designation won't stand up in court right but even so oh yeah absolutely absolutely the the use cable but it it matters insofar as it's not like anthropic can't work with aws it would mean that anthropic couldn't work with like aws government the it's not um it's not in theory like a death blow to like working with aws or something like that but the but from a government agency side, what this implies to me actually is that LLM integration in US government departments and agencies has still behind the power curve and behind where frankly somebody like me would want it to be.

37:20Big Technology:And it was much announced over the context of the last year that all the Frontier AI labs made their technologies available either for free or for like a penny or a dollar or something like that to the federal government trying to ramp up, trying to ramp up adoption. And so government employees then at these agencies in theory have had access to multiples of these for a while and are like choosing whichever ones they like want to use for various, for various tasks. And it sounds to me like on the unclassified side, Ben, that Claude is being, people are getting instructions, like don't use Claude, use something else, use something else instead.

38:01Big Technology:This is pretty fast moving, frankly, for the government. But it was notable in the announcement, both the Trump announcement and the Hegseth announcement, that they laid out this six-month off-ramp period for real national security use cases, in part because they rely on Anthropics technology right now because Anthropics is the only vendor behind the curtain in a classified environment. So I think what we're seeing is that real bifurcation where for these unclassified use cases, the, you know, essentially like flip this, like use, you know, use chat GPT instead or use like Grok instead or something like that.

38:37Big Technology:And frankly, if there's a deal in the future, they'll just like flip back to using Cloud if they want. On the classified side, it's going to be a much harder slog because of the integration of Cloud and the fact that it was the first mover because Anthropic was the first company willing to do that kind of work with the defense establishment.

38:57Michael Horowitz:Then the question is also in terms of what this means for companies thinking about working with the government, that you could potentially be declared a supply chain risk. This is from Dean Ball, who I think worked on some AI policy with the Trump administration. He goes, even in the narrowest supply chain risk designation, the government has still said that they will treat you like a foreign adversary. Indeed, they will treat you in some ways worse than a foreign adversary simply for refusing to capitulate to their terms of business, simply for having different ideas, expressing those ideas in speech and actualizing that speech and decisions about how to deploy and not to deploy one's property.

39:33Michael Horowitz:Each one of these is a fundamental to our Republican. Each was assaulted by the Department of War last week. And basically the worry is that companies will be wary of working with the Department of War if this is what could happen to you. I'm less worried about that, but I would love to hear your perspective as someone who's been on the inside?

39:51Big Technology:I mean, this is a rough look for a Pentagon that has worked really hard across multiple administrations and in a bipartisan way to build ties with Silicon Valley across the board. And obviously, this administration, the Trump administration, has some deep ties with Silicon Valley in some places, like less deep ties in other places. But certainly the notion that if you can sign a contract with the government, they might ask you to change that contract. and if you don't agree to it, they might attempt to destroy you, is very different than in terms of the risk then for a company and getting involved with the Pentagon in the first place.

40:29Big Technology:Because going back to something that we were talking about before, when it comes to the use cases that Anthropic may be concerned about in different kinds of ways, I mean, the thing to remember is if you do business with the Pentagon, the business of the Pentagon is war. So you shouldn't be surprised then that the Pentagon wants to do all the war things with your technology, because that's the thing that the Pentagon does. But the idea that if you have a contract to suit with the Pentagon, that they might attempt to annihilate your entire business, not just cancel the contract. I do think in some cases could lead to questions for companies that might be making a marginal choice about whether they wish to work with the government or not.

41:12Big Technology:That being said, some of the other frontier AI labs like XAI and OpenAI are already now willing to work on the classified side. And Sam Altman is attempting to broker a piece essentially and create a deal that perhaps Anthropa could join as well. Now, even if he succeeds at that, will Anthropa then walk through that door. I mean, obviously, like there's beef between OpenAI and Anthropic, as well as with OpenAI and XAI. But there are other vendors that clearly wish to do these things. But it's also true that America's warfighters have said very clearly through what we see in Operation Epic Fury that they think Anthropic is delivering a good product and they wish to use it.

42:00Right.

42:01Michael Horowitz:I think, and I'm curious to hear your perspective on this, this does do long-term damage to anthropic because even if these laws or even if the, let's say, the supply chain risk designation ever makes it to them or is overruled, public sector companies, contractors will just in the back of their mind think twice before rolling out anthropic technology in the future. I don't know. It kind of depends on how you, I could imagine that scenario.

42:29Big Technology:If the narrow, even if the supply chain designation gets struck down, but all of the contracts are canceled and after six months, the Pentagon's using other kinds of things and Anthropic never gets back into that business, then one could imagine that occurring. Although in the context of what we end up seeing in the midterm elections or a future presidential election, the politics could change in a way that also like rejiggers this. But it's also possible that this six month off ramp period, I mean, maybe I'm just being, this could be like wishful thinking, frankly, from a national security perspective, could allow for some bargaining potentially to occur.

43:08Big Technology:That maybe -

43:10Michael Horowitz:We've seen that with TikTok, the six month never happened. Yeah, yeah, exactly.

43:14Big Technology:And that the supply chain letter wasn't delivered on day one made me wonder, like, oh, like maybe is there an opportunity for bargaining here? Like who knows? I mean, another challenge here is like, there is, if there's like any organization in the U S government that is like full send all offense all the time, it is like the, it is like Secretary Hegseth's Pentagon. And so it, it would be, it would be challenging, I think, to figure out what the win-win looks like for both Anthropic and the Pentagon from a public perspective. but there's probably a lot of utility in that. And it wouldn't surprise me at all if there are negotiations that like, maybe they take a couple of weeks to start, or maybe they're happening right now, but if there are negotiations that lead to some kind of deal eventually.

44:02Michael Horowitz:Okay. Last question for you. You're someone who's thought a lot about autonomous warfare. And so I don't want to end this episode without asking you, how do you think AI is going to change warfare? Now, I know it's not like just a couple minute answer, but - Yeah, how much time you got? I mean, as long as you have, we have, but I'm just curious to hear your perspective on where things go from here.

44:27Big Technology:So I think about AI as a general purpose technology. It's not a widget, it's not a weapon, it's a general purpose technology, which means the analogies to me, if we want to imagine the impact of AI on militaries or on the balance of power, say more broadly, are other general purpose technologies. So think like electricity, combustion engine, airplane, like those kinds of computing, like those kinds of things. And there are three different buckets that I would put the impact of AI in. So one is a bucket that is analogous to the commercial world, which is the military's use of AI for payroll processing, logistics, acquisition paperwork.

45:07Big Technology:Like Lord knows the military could be more efficient. from that perspective, having spent a couple of years recently in the Pentagon bureaucracy. And so there are potentially massive opportunities there, just in the bare minimum. Second bucket is in more that intelligence surveillance and reconnaissance kind of category, like bleeding into something like the decision support we were talking about before, where you already had things like computer vision algorithms that were helping the military and intelligence agencies process all the data that they get about the world and separate the signal from the noise.

45:42Big Technology:But there's a real opportunity with some of those LLMs, if their reliability can be improved, to make that happen much faster and much more accurately. Because while people worry about errors from AI in this context, and it's often the AI industry, frankly, speculating about potential errors and accidents from AI, humans, definitely error prone, and which we've seen like all the time. And like, think about like in 1999, for example, in the context of the Kosovo bombing campaign where the US by accident bombs the Chinese embassy. The, like, I don't know, maybe the computer vision algorithm or LLM like might've caught that.

46:24Big Technology:Like there's like lots of opportunity essentially in that like second bucket for more effectiveness and essentially for buying decision makers time. Because we tend to think in the military context that the more time people have to make decisions, and this is like a behavioral science insight, like not a military insight, more time people have to make decisions, generally the better the decisions that they're gonna make. And so that's another way that AI can be upheld. And the third is like close to or on the battlefield. And autonomous weapon systems, frankly, could be hugely important for militaries, especially if you imagine future conflicts with great power adversaries, say if there's like a US-China conflict or something, One thing people worry about in the context of that kind of conflict is, say, losing access to satellites, losing access to space.

47:10Big Technology:And in what the military would call a degraded or denied communication environment, something like an autonomous weapon system will be essential for lots of different kinds of weapons to be able to operate. and algorithmic operational planning to help commanders, that may be part of the way that a military like the United States can still compete and win in the worst case kind of scenario. So there's a range of different, in some ways, uses of artificial intelligence. And what I would leave you with is like macro, I think we're talking about enormous consequences for militaries. Like this is why, this is one dimension of that macro US-China AI competition.

47:53Big Technology:They're not the only dimension, certainly. But that when we get into it, I would encourage people to think about AI in the military in the context of specific use cases rather than as a monolithic technology. Because the kinds of AI you would use and what you would use them for will vary a bunch depending on the use case.

48:15Michael Horowitz:So like autonomous robot wars, not exactly around the corner.

48:19Big Technology:I mean, I'm ready for our robot overlords. I have been for years. I'm not in the short term.

48:27Michael Horowitz:Okay. All right, Michael, thank you so much for coming on. This was so illuminating and definitely gave me a deeper understanding of what's going on than any conversation I've had previously. So thank you so much for coming on the show.

48:39Big Technology:Thanks for having me. I'm happy to chat anytime.

48:42Michael Horowitz:Awesome. All right, we'll take you up on it. All right, everybody, thank you for listening and watching. We'll be back on Friday, breaking down the week's news. Until then, we'll see you next time on Big Technology Podcast.

49:17Michael Horowitz:Limited time offer for new clients on federal returns. Participating locations in their terms at JacksonHew.com slash 149.

49:23Big Technology:Rinse knows that greatness takes time, but so does laundry. So Rinse will take your laundry and hand deliver it to your door expertly cleaned. And you can take the time pursuing your passions. Time once spent sorting and waiting, folding and queuing, now spent challenging and innovating and pushing your way to greatness. So pick up the Irish flute or those calligraphy pens or that daunting Beef Wellington recipe card and leave the laundry to us. Rinse. It's time to be great.