How AI is changing warfare

0:02Good morning from the Financial Times. Today is Friday, November 28th. And this is your FT News Briefing. I'm Mark Filippino, and we're doing something a little different today. We're devoting our whole episode to a very tricky topic, artificial intelligence and warfare. How big of an impact it's going to have, and what the guardrail should be. I'm going to hand it over to FT investigative reporter Helen Worrell, who recently wrote about this. Here's Helen.

0:30Artificial intelligence has the potential to disrupt almost every industry we work in, from manufacturing to stock trading. Defence is no exception, and at a time of rising global conflict, the question of how different militaries are using AI is increasingly important. As part of an FT collaboration with the MIT Technology Review, we're going to be debating the ethical, political and practical questions around AI warfare. I'm joined by James O'Donnell, who's a senior reporter at the MIT Technology Review covering AI. Hi, James. Thanks for joining us. Hi, thanks for having me. Before we get into the sort of future applications of the technology, I think we should start off with giving listeners an overview of what's actually happening right now in the world of defense AI.

1:19Can you give us a sense of where AI is being used and what sort of capabilities it has? Yeah, I think there's a scale here. I think there are startups and defense companies using AI for a number of different areas. So one of them is about logistics and planning. So figuring out how to move things around the world, how to make military logistics more efficient. There's also efforts to use AI in cyber warfare, both on the sort of offensive, so in implementing cyber attacks, but also on the defensive. So I've spoken with companies that are trying to build AI models that sort of analyze foreign intelligence.

2:00And then, you know, the most controversial application of AI is really on the battlefield, figuring out what sites militaries should target. It's also the most controversial because it really involves high-level military decision-making. We're talking about attack targets and lives. And I think what's really pertinent here is that those scenarios that you've just talked us through are all situations where there is a human in the loop. And I think when we actually think about the controversies of AI, a lot of this focuses on a sort of future conflict and a future scenario where potentially there isn't a human commander there, or rather the human commander is maybe overseeing things but not directly making decisions.

2:50Do we have a sense of what that sort of future conflict might look like? The biggest anxiety that people have around AI being used for warfare is this scenario where the AI is actually making decisions. And though that's not happening right now, per se, I think it's a future that military leaders are planning for. And so you might imagine an AI sorting through a list of potential targets and not only recommending particular targets, but actually sort of pushing the button, so to speak. And I think as you think about the future, it comes up against some limits of the technology, right? You have hallucinations that are a problem with all large language models.

3:34You know, the ability that an AI could come up with an answer or create something that sounds reasonable but is actually false. So that's a prevalent problem for all large language models. It's really only as good as the data that it's been exposed to. And the geopolitics of war involve a lot of confidential, top secret information. These aren't things that are easily learned from a data set, right? So it's not something that I think most technologists right now, even the most optimistic ones, feel that AI would be particularly good at. Sure. Sure. And I think this point that you make about hallucinations is something that a lot of people have warned about, not least somebody who you mentioned in your piece, Missy Cummings, who's a former U.S.

4:17fighter pilot, who's now a professor of computer sciences. Yeah. So I think some of it is there are biases or mistakes in that data. Large language models are likely to sort of replicate those mistakes in their answers. For example, that I spoke with a group of U.S. Marines who were testing large language models that could analyze intelligence, right? So the promise was that these Marines were doing an exercise in the South Pacific, and they're tasked with analyzing intelligence from the surrounding area. So these are newspaper articles, audio, newscasts, documents, things that are in different languages.

4:55And the AI model was tasked with doing translation work with that, analyzing it, surfacing pieces of intelligence from all of that data that could be concerning to the service members on this ship. But as AI takes on more and more responsibility, and you can imagine as a human operator, perhaps you're leaning on this AI to make sense of more and more data and more and more documents, it becomes a lot harder to fact check the outputs of AI. And I think that's kind of what someone like Missy Cummings has pointed to, the flaws of this large language model that work really fluently with text and generate convincing outputs, you know, when you're relying on them to make sense of things and make really specific and sensitive decisions about what this piece of text means, I think that's where things can go awry and it becomes pretty unwieldy to imagine fact-checking all of its responses.

5:48Well, I think one of the really key questions that a lot of people are asking now is what the guardrails should be around this kind of technology and what the limitations should be on how we use it. Do you think there are specific areas where we should be striving towards proper international agreements like the ones that we have currently, for instance, on non-proliferation? So I think countries are approaching this in different ways. Some countries are saying, well, the best AI computer chips and GPUs, those are sort of the building blocks of really powerful AI. So rather than writing military rules about how AI is used right now, we're going to be controlling those chips as a sort of national security issue.

6:34The idea being that if you prevent the export of certain chips or you control them in a certain way, perhaps you can control who has ultra powerful AI in the future. In terms of writing the rules around how militaries use this, I think it's a little bit soon to say, but I think the biggest question is how you define AI when you start talking about how much of a role it should play in the decision making process, because I think that's a little messier to define than people on the outside of this might think. There's a huge discussion, an ongoing discussion about who's actually winning the AI arms race as far as military applications of AI goes.

7:16Between, for instance, the US and China, which are the really big players in this area, do you think one or the other is very firmly ahead? I don't know if we can say right now, I mean, since the war in Ukraine began, you've had this shift, which is that the U.S. had for years sort of owned the really expensive, exquisite military technology space. But as you saw in even in the beginning days of the invasion of Ukraine, you know, the war really quickly became about cheap drones that were sort of expendable, really, you know, cheap, uncomplicated drones that were being used in really significant military operations.

7:58So this whole question of who's ahead and who's winning the race, I think maybe years ago that would have been about budgets and who's spending the most. But I think AI, for better and for worse, sort of upends that paradigm a bit. It's become really cheap to get ahead in some ways if you have the right drone and the right algorithm for it to navigate itself. You know, that's not the story of the U.S. building really high-budget, sophisticated technologies through much of the 20th century. That's all really changing, and it's changing quite quickly. Well, it sounds like we can come back and debate this in five years' time then.

8:35Thank you so much, James. James O'Donnell is a senior reporter covering AI at MIT Technology Review. Thank you.

9:11and our theme song is by metaphor music.