AI in security and defence: is tech making us safer?

0:04Hello and welcome to the Times Tech Podcast, where every week we unpack how technology is reshaping business, culture and everyday life. I'm Katie Prescott covering all things tech here in the city of London.

0:18Danny Fortson: And I am Danny Fortson out here in Silicon Valley. Which is, Danny, the place to be this week. I am so jealous. You are at what I think can only be described as the tech trial of the century, the millennium. The millennium. Elon Musk versus OpenAI. The beef between them is Elon Musk and Sam Altman co-founded OpenAI, among others. And Elon Musk said, we founded it as a charity and he's turned it into something commercial in very basic terms. yeah basically i got bamboozled i was a prime funder i put in 38 million dollars to kind of get it off the ground and fund it in its early years and then they fell out over strategy elon musk left and said and this is in an email gave them quote unquote exactly zero chance of survival once he left he's like yeah and of course they went on to very much survive turn into a for have a you know establish a for-profit subsidiary which is now worth 800 billion dollars launch chad gbt kick start the ai revolution and so this fight is really around he's like hey i donated this money to a charity now it's now it's one of the most valuable companies on the planet that can't be okay pony up guys exactly so he's wanting just a cool 134 billion I mean, you know, that seems reasonable.

1:56What was he like in court that first day with Elon Musk giving evidence like and what's been going on?

2:03Danny Fortson: Yeah, so it was a total zoo. So I got there at like 8 a.m. and there was a line of maybe 75 people waiting to get into the building. All kinds of camera crews, etc. So just a real spectacle. you got in there and there was like no room in the courtroom because it was fully packed you have to show up kind of 5 6 a.m to try to get a seat inside the actual courtroom then they have like queuing for Wimbledon exactly exactly but without the strawberries but then you get but they also have an overflow room that has a live video feed and audio so and that's just like literally down the hall so i was in the overflow room with probably i don't know at least 100 people a lot of journalists there's a lot of lawyers and in the foyer are all these like beefy looking guys with earpieces which are all of the security detail for the various billionaires who were in the room that day so they're all just kind of hanging out looking kind of big and buff and mean but you had Elon Musk in the room.

3:13Danny Fortson: You had friend of the pod, Sam Altman in the room. You had his co-founder, Greg Brockman in the room. And so I was there for opening arguments of both sides. Astonishing. Well, we should say OpenAI has strongly disputed Musk's claims and characterizations. Anyway, Danny, it's really exciting to hear what's going on. And I can't wait to hear more over the coming weeks. But let's move on to today's episode, where we're going to be talking about all things defense and the price we pay for security, both globally and at home, because this has been a massive issue at the moment, hasn't it? Yeah, exactly.

3:51Danny Fortson: And this is all, of course, Jermaine, because what we're really talking about is AI and how does this all work is a very interesting dilemma around national security, safety at home and abroad in this age of AI. And the reason we're looking into this This is because several stories have come together at once in the last couple weeks that really shine a very bright light on this moment in tech and defense. So this week, Alphabet, which is Google's parent company, signed a deal that allows its AI to be used by the Pentagon for quote-unquote classified uses. A little vague, but one can speculate.

4:29Danny Fortson: And this was despite a letter of protest signed by more than 600 Googlers delivered to CEO Sundar Pichai. And in signing the deal, they joined really a pretty fast growing list of Silicon Valley companies currently making agreements to kind of do work for and sell stuff to the US military. Yes, it does feel like quite a moment as most companies grapple with what their relationship should be with government when it comes to defense. There's one company that doesn't grapple with that. And that's pretty clear on what their view is, of course. And that is Palantir. Yeah, zero grappling. No dilemma.

5:07Just a firm view. And they put out this post on X, which has had about 32 million views, setting out its manifesto, which is just worth going into a bit. It sets out a future, basically, where AI warfare, it says, is inevitable. And it calls on Silicon Valley tech bosses to, and I quote, participate in the defense of the nation. I mean, it sounds like something from World War II.

5:30Danny Fortson: Yeah, and it positions companies like Palantir as kind of now helping engineer national security for this kind of new AI era. And they envision a world that grants, you know, really a handful of unelected tech executives, huge influence over military strategy, domestic safety. And critics have called us, you know, what we're kind of, we're a step toward, if not already in a surveillance state. So this week, we're looking at how AI is transforming defense and security and asking the critical question, are we actually being made safer because of AI? And in keeping with that theme of defence, one of the most controversial aspects of AI use.

6:14Our big interview is with the chief technical officer of defence company BAE Systems. Julian Cracknell is going to talk to us about what the future of defence looks like and if it involves killer robots going to war and scarily making their own decisions. But should we start off by talking about Google's deal with the Pentagon? Danny, can you just take us back to what's happened?

6:37Danny Fortson: Yeah, so this week, Google has reportedly signed a deal with the Pentagon to use its models, its AI models for classified work. And the agreement allows the Pentagon to use it for, quote, any lawful government purpose. You may recognize that terminology, because that was the kind of the key phrase when OpenAI and also XAI, Elon Musk Company, signed similar deals to kind of start selling their wares to the Pentagon. on. Anthropic famously said, we're not comfortable with this because effectively they don't trust the government in its current form to only use it for, quote, lawful purposes. And the agreement sparked employee pushback inside Google.

7:21Danny Fortson: 600 members of staff signed a letter and sent it to Sundar Pichai. In it, they say, quote, we want to see AI benefit humanity, not to see it being used in inhumane or extremely harmful ways this would include lethal autonomous weapons and mass surveillance but extends beyond end quote and this is somewhere google's been before um and grappled with before so back in 2018 they scrapped a contract with the pentagon project maven after thousands of employees protested against the use of ai tools to analyze drone footage so now eight years on they've made another agreement and it feels like employees are pushing back again let's have a closer look at what the deal entails so essentially it says google has to adjust the company's ai safety settings and filters at the government's request and it doesn't give google the right to control or veto lawful government operational decision making yeah the pentagon has not commented on this and it's very similar to what you know we had sean gurley on last month talking about the whole anthropic kerfuffle and he's like look if you're selling something that can be used as a weapon or for weapons or with weapons to a military you can't say hey don't use it to fire a gun in that way you know like you're kind of handing over power of these tools you can't set rules about how it can be used you're basically selling a product and then you know the government the military whomever is going to use it however it wants as you mentioned this is really significant because back in 2018 and i covered this at the time over the project maven ai drone contract 3 000 people signed a letter of protest it was a really tumultuous time in silicon valley especially around defense and then at that time eight years ago the company backed down and they basically declined to renew the contract and who picked it up palantir and what's really interesting more in a minute yes indeed and what's really interesting back then 3 000 people signed this letter of protest today alphabet is twice as large so it's now 190 ,000 people.

9:45Danny Fortson: Only 600 people have objected in this way so far. So I'm not a mathematician, but that feels like on this very unscientific way. Numbers, even chat GPT could get its head around. Exactly. And this one unscientific metric, that's like one-tenth the outcry, at least in this form. And again, I think it speaks to this moment of just this vibe shift we were talking a lot about a year ago where it's just whether it's due to the war in ukraine what happened in israel and trump being in office like there's been this real turn it used to be turning away from washington and now there's a real turn toward washington when it comes to you know big tech like i said it's the vibe shift and if you talk and if you look at like andreese and horowitz one of the biggest um investment firms out here venture capitalists out here they have their what they call the American Dynamism Fund, which is really all about funding a whole new generation of American defense startups, which again, if you go back five years, that idea would just not really even be talked about in polite company out here.

10:59Danny Fortson: And now they're like stepping into that and be like, yeah, we're going to build a new generation of defense. And we're proud of it. You know, So again, it's just it's quite a change. It does also feel naive, doesn't it, to think that you could sell technology to the government and then put these guardrails about how it uses it onto it. I think it's either it's quite binary, really. And it also begs the huge question, should US AI companies be allowed to say no to Washington? And Palantir, which is the company that interprets massive amounts of data, shared this manifesto a few weeks ago when it said, no, absolutely not.

11:43Silicon Valley owes a moral debt to the US for making its rise possible. And led by its boss, Alex Karp, said national service should be a universal duty. I mean, it's worth saying this sparked so much controversy when it came out and some critics called it the ramblings of a supervillain.

11:59Danny Fortson: Palantir put it out on X. It's 22 points, kind of point by point about kind of really, it's a summation of the Technological Republic, which was the book that Karp put out some months ago, which kind of laid out like basically his life story and also building Palantir and his kind of worldview, which is, has long been, let's call it distinct from everyone else. like you know palantir delightfully diplomatic of you yes uh palantir was basically created by him and peter teal back in 2003 in the wake of september 11th because if you remember peter teal he was a founder of paypal they'd create all these anti-fraud systems and he's like hmm maybe we could use these to solve a problem of like you know finding those needles in the haystack of intelligence helping governments you know use their data better find the bad guys before they do bad things thus was born palantir and it's been a pariah from day one it's worth just talking a little bit about the scale of palantir i think so just before we came on i saw its market cap is currently 326 billion dollars shares are up more than a thousand percent since it floated and um just 17 % in the past year, but probably even more interesting than those numbers is the boss, Alex Karp, who is, should we use the word distinct again?

13:28He's distinct. You've met him. I have. What's he like for people who don't know him?

13:35Danny Fortson: So I did a profile of him a few years ago, and I went down to their office in Palo Alto. more i met him and i walk in and he's like he's doing he does kata which is a form of kind of uh martial arts and where it's you're kind of moving very deliberate movements through the air kind of looking like you know something out of a movie he was doing tai chi or kata in the in the foyer and people were just kind of veering around the boss of you know the billionaire boss of their company as he's like very deliberately doing his tai chi in the middle of the space and then you know we go into his office he has this wooden sword that he goes and you know talks in great length about this wooden sword that he has and what he uses it for i would say he does not suffer from a lack of clarity of his views he views silicon valley as this like island of moral bankruptcy where all of the best people all of the best talent at least this is a few years ago was going into optimizing ad spend getting people hooked on apps and he's like this is just all garbage you guys are giving up your duty to the country to actually get involved help the government you know help quote-unquote war fighters and equip them with the best kind of 21st century tools that you all know how to build and you're choosing not to um and so palantir had been the subject of a bunch of protests in you know because silicon valley famously progressive so eventually he left he relocated the company to colorado he now lives in new hampshire out in the middle of nowhere by himself and he runs this company that has just grown dramatically in the past four or five years, basically since he and I sat down with the wars in Israel, Ukraine, now Iran.

15:34Danny Fortson: And really, they've been one of the kind of stars of the stock market. And also his star has risen, especially with, you know, the advent of the Trump administration, their kind of, let's call it more muscular approach to international relations. And they process vast amounts of information yes as you said for defense and for governments but also for organizations like banks and over here in britain people probably know them best because they won an enormous contract with the british government to process medical data for the nhs worth 330 million pounds so some campaigners have said we're not sure is that dangerous is this the sort of company we want to be handling our public data louis mosley who's the chief executive here has pushed back very hard on that saying, look, you need to see what we're doing.

16:22He said he welcomes the extra scrutiny that the company is getting. And it is getting a lot of scrutiny here at the moment. But yeah, it's growing and growing. Well, that feels like a good time to introduce this week's guest who's from the largest defence contractor in the UK and Europe, BAE Systems. And they just work with everyone, the UK Ministry of Defence, US Department of Defence, Denmark, Sweden, Saudi Arabia with a market cap of 60 billion pounds. This is a really incredibly topical interview given the conflicts going on around the world. Defence isn't something that is easy to discuss.

17:02And I think it's fair to say, Danny, this is certainly a different side to a lot of the consumer and B2B tech that we talk about.

17:08Danny Fortson: Yes. And I just want to call out you did make a small mistake. It's not the Department of Defence. It's the Department of War. It's been rebranded don't you know no way it's doe not dod okay dow department yeah anyway so who better to answer that question and about what the future of defense and national security looks like than julian cracknell he is the cto of bae systems and we started by asking him just to go back a pace or two and explain exactly what the company does. We're the biggest defence company in Europe, pretty much, it depends slightly on how you measure it, one of the largest in the world.

17:52And the way I like to think of it is from outer space down to the seabed. So from a space perspective, we build satellites for communications and observation. Then in the air domain, we have the combat air programmes that we're delivering at the moment, as well as all sorts of uncrewed vehicles and drones and similar things. On land, we're the largest provider of land vehicles in the Western world. We provide complex warships. We build submarines, as well as other uncrewed vehicles that float or go on the land or under the sea. And then there's a whole set of systems that connect those things together, whether that's communication systems, command and control systems, things like electronic warfare.

18:36so it's a very kind of broad portfolio and that's delivered internationally so you know very large in the UK big presence in the US very large in Australia as well as lots of other countries around the world. And give us a sense of what your role as chief technology officer entails because I just can't even begin to wrap my head around that I mean the research and development must be enormous for BAE before you get on to everything else. Yeah so I'm responsible for four things the group strategy overall and thinking about how we run the strategy process a number of the operational functions so project management engineering manufacturing quality technology now that's not every bit of technology we do clearly there's a lot that happens within the businesses and within the sectors but I'm my team are thinking about some of the areas we might go next particularly how we incubate new ideas and think about taking those you know establishing whether they've got legs and might go somewhere and then also on the CIO as well so the IT cybersecurity, all of that kind of stuff.

19:34How has AI changed what you do over the past few years? Because I imagine AI has always been integrated into what BAA Systems does, but with the advent of generative AI, ChatGPT launching in 2022, for everybody, things have changed quite dramatically. I mean, you're absolutely right that AI has been involved, or defence has made use of AI for an enormously long time. If you think about the Typhoon platform, which you may recognise as a combat jet that's used by the the uk it has two very distinctive little flaps at the front of it that keep it stable because the platform itself is inherently unstable in order to be more performant and those flaps are controlled by an ai system and that's the only reason that the plane stays up and is capable of flying is because there is an ai augmenting the pilot at all times so the kind of use of ai is definitely something that's been prevalent in defense things like generative AI and LLMs are clearly having an impact in some ways in similar ways that you would see in other non-defense organizations you know helping people write reports and similar but also in things like controlling drones being able to interact with them with you know natural language and ask it to perform a function and they'll go off and and you know task the drone to behave in that way there are other ways that we're making use of AI so things like physics based AI models to allow us to design complex components more quickly rather than having to do lots of iterations with humans trying ideas out and then testing them using the physics based AI models to try and get to an answer more quickly.

21:12So it's definitely something we're, you know, making lots of use of in defense.

21:16Danny Fortson: I just want to get your sense of a kind of the moment we're in. Because if you go back to 2018, it was another moment out here in Silicon Valley in particular, where there was this very kind of infamous Project Maven drone contract with the Pentagon that Google had. It was revealed. There was a big walkout. And you had all these employees being like, we don't want to be in the business of war. Google gave up the contract. Palantir picked it up. Palantir famously left Silicon Valley because they're like, basically, you all need to be more patriotic. Stop worrying about better ads, make weapons of war to defend the West, et cetera, et cetera.

21:56Danny Fortson: And there's been a real shift in especially the last, I would say, 18 months to two years, where all of a sudden kind of defense is seen as something that is okay to do now. From where you guys sit as one of the biggest defense contractors in the world, how has the playing field changed in terms of what the big tech companies in Silicon Valley are doing and how they're showing up in your universe compared to, you know, say five, six, seven, eight years ago. I mean, clearly that question of whether you're comfortable to work in defense is one that, you know, we've thought about a lot. And, you know, we're really proud of our role in defense.

22:36Our motto, if you like, is we protect those that protect us. And I think it's a very inspiring thing for us as an organization. In terms of access to technology, my sense is it can only be a good thing that more technology is available to be used in the in the defense space so so where we are today I think is clearly helpful I think the risk is if that appetite changes and it starts to take away some types of technology that we you know have started to rely on but certainly from my perspective it's clearly a good thing that people are in both applying their technology towards defense and also you know actively investing in that area.

23:13Danny Fortson: Are the tech companies, whether they're startups, because there's a bunch of new kind of American defense startups like Anderol and others, are they showing up in pitches in a way that they just weren't there before? There are a lot more companies interested in defense. And I think that has changed fairly significantly over the last probably five years or so. I suspect you tie it back to the events in Ukraine and that that has in some ways it has re-established the reality of the threat, made it clear to governments around the world that they need to invest in defence and therefore that has attracted investment and technology into the space.

23:53And how important is it now then in warfare to have the right technology? I mean is it a question of whoever spends the most, has the best tech, can win wars? Or as we've seen with, I guess, Iran recently and some of the cheaper technology that they're using, which has been relatively successful in some fronts, is that, you know, do you feel it's still an open playing field or is that a sort of daft question? Well, it's not a daft question. There is a balance to strike between kind of exquisite tech at one end and the mass of volume that you can get at another end. And the focus on affordable combat mass is something that I think has become much more apparent coming out of Ukraine.

24:32In the Strategic Defence Review in the UK, which was published a few months ago, they talked about a balance between exquisite capability, reusable capability and attritable capability. And exquisite meaning top? Crude, C-R-E-W-E-D, with people inside it. And therefore, with the protection that's necessary around that to protect people. But typically, the view expressed in the SDR is probably about 20 % of your capability would sit in that space, probably about 40 % reusable and 40 % attritable. So, you know, you're certainly seeing a recognition from governments, again, driven to some extent, I think, by what's coming out of Ukraine, that more focus on volume and the kind of lower cost capability is necessary to support and augment some of that more capable technology.

25:25And apart from the government's review of this, what do you think BAE has learnt from Ukraine? We're very proud of our support of Ukraine. The pace of technology adoption is really high. There's been a great or significant use of commercial capability in Ukraine. And you see that in a number of different places in the use of commercial drones, in the use of things like commercial satellite imagery, commercial communications. But those technologies tend to get competed out very quickly. And we're talking about kind of days and weeks rather than months and years for those technologies to get effectively sort of outcompeted by the opposing technology.

26:11and so that pace of evolution the pace of adaptation to be able to respond to new threats embody that in hardware and software and then deploy it is a real change i think there's always been that pace element in defense and we talk about urgent operational requirements which is essentially you know the customer has a burning problem that needs to be solved and you know defense companies are very good at being able to respond to those things but i think what we're seeing is probably more of that on a more consistent basis and you know obviously you

26:45Danny Fortson: make bombs and bullets and tanks you also make bugs like cyber bugs you know because we're in this moment of ai and we just had all this strum and drang about mythos preview and how it can hack virtually anything and everything and those are quite powerful weapons right especially when you're talking about you know things that have been deployed to like uh hamstring iran's nuclear program and things like this is that part of your operation of like beyond the kind of the big kit that you're also building kind of cyber weapons we've got a really broad portfolio as i talked about earlier on and i'm not going to get into any specific capabilities if that's okay Classified.

27:28I can neither confirm nor deny.

27:32Danny Fortson: Right, but I think what feels like really interesting and different about this moment is that it's kind of everything is being democratized. When you talk about a$200 drone that can go do real damage versus a$2 million Tomahawk missile, that feels like we're in the midst of this kind of qualitative and quantitative shift in how wars are fought. Is that how you guys view this moment? I mean, that comparison is clearly true. You know, relatively low cost drones are being taken out by relatively expensive interceptors. I think what gets missed in that comparison, though, is what's the thing that you're defending?

28:06You know, if you're protecting a nuclear reactor or a city, actually, you're going to spend a lot of money to do that, regardless of the cost of the thing that's coming at you. So the straight kind of pound for pound comparison in terms of the cost of things, I'm not sure, tells the whole story. that said it's clearly important that given this you know volume game of of you know large numbers of relatively low cost attack drones that we can respond to that so one of the pieces of research that we've been doing and have just announced actually is taking a a low cost rocket attaching a laser guidance system onto the front of that and then firing that from both a drone and also combat air typhoon platform.

28:52So suddenly, you've got a really low cost interceptor that's able to respond to that. And I think that that kind of, I was going to say arms race, that's exactly the right term, that kind of back and forth between technology, and then the counter to that technology is going to keep going and is one of the things that drives innovation in the defense sector.

29:09Danny Fortson: And to Katie's earlier point around, you know, how AI has changed things, when there was the blow up between Anthropic and the Pentagon, we had a guest on, Sean Gourley, out here. And he said a lot of people within the defense space view AI a little bit like nuclear weapons. Like if you have clear superiority in this very powerful technology, it is what they call the next kind of offset. Like you render your opponent defeated before you even have to fire a shot because they just know and you know that you're just going to win and win decisively and cause real damage. Is that accurate or is that how you guys view the world?

29:53I think what's really being talked about there is speed of decision making. So, you know, that concept that if you can make decisions faster than your enemy, that you can get inside their decision cycle, outthink them and therefore outmaneuver them and at some point get them, you're so much faster than them that they're unable to respond. That's been a constant in warfare forever. And I think what we're seeing here with AI is the opportunity to really accelerate that. I'm not sure I would describe it in quite the same way in terms of that kind of, you know, life is fundamentally different. But clearly what it is doing is creating opportunities to make decisions much more quickly, to take in volumes of data that would have been harder previously.

30:41and synthesise those, bring them together in order to make better decisions faster. And that does clearly have a strategic benefit. So we are in a geopolitical arms race then when it comes to developing AI. It sounds like from what you're saying. To try and get ahead of other people, to get it, as you described it, get inside their heads. We've got to make use of the best technology that we can in whatever space that we can in order to be able to out-compete our adversaries. And AI is clearly a significant element of that. I don't think it's the only element. There are lots of other areas where technology is advancing.

31:20I'm going to ask you a question that you'll probably hate, but I have to ask it. How has your job changed with Donald Trump in the White House and the sort of split in the special relationship in the last year or so? I do hate that question. So I look at it from a couple of different angles. We have a very large US presence because of US federal legislation that is entirely separate from the rest of the business. And so they are working with the US administration and working through the impact of that. For the rest of the business, we're thinking about where our customers are going in terms of their decision making around new equipment, the types of capabilities they want.

32:06I think it's not entirely related, but the issue of sovereignty, who has control, particularly who has the ability to make decisions about how you use equipment, how you can modify equipment. Again, towards ever thus, in defence, there's always been a concern. But certainly in the recent UK SDR, Strategic Defence Review, you see a much greater focus on sovereignty and the element of industry as part of the defence enterprise. And again, I think that's drawing on the lessons of Ukraine, that deterrence is not just about the capability that you can deploy today. it's about how you can regenerate that how you can iterate that and how you use industry in a close relationship with industry as a critical part of the overall defense estate I guess.

32:57We'll be right back.

33:02Nothing tests my patience quite like sitting on hold listening to the same four bars of music on repeat. It turns out all that time adds up. The average person spends over 40 days listening to that dreaded hold music.

33:16Danny Fortson: 40 days? And this episode is sponsored by Parloa, the AI platform built to make that a thing of the past. Parloa's AI agents are available 24 hours a day, seven days a week. Fluent in any language and smart enough to remember every customer across every interaction because no one should have to explain themselves twice. From payment processing to roadside assistance, flight rebookings to appointment scheduling, Parloa delivers the kind of customer experience that turns a one-time buyer into a customer for life. See Parloa's AI agents in action at parloa.com. That's P-A-R-L-O-A dot com.

34:02Danny Fortson: julian you mentioned control which brings to mind autonomous weapons and palmer lucky who i'm sure you know runs andrel which is like now worth 13 billion dollars you know the this defense tech startup yeah i love that you describe as a startup exactly it's very big now um but he's his kind of line is i love killer robots like that's what they're building and i'm just wondering how you guys think about that like where are we on the continuum of just you talk about decision making control and especially in the context of china where they have this huge manufacturing base where they're building more drones than the rest of the world combined and all of and they're racing ahead in AI, is this where we're going?

34:51Danny Fortson: And how do you guys think about killer robots, this idea of just letting the AIs go to war, make decisions, and blow stuff up? I don't really love the phrase killer robots. What we're seeing at the moment is actually a lot greater use of remote controlled systems. So you've got human in the loop in most of the systems that have been deployed, certainly in Ukraine. And that reflects the fact you've got a very mobile battlefield that's changing on a regular basis and therefore having humans actively engaged is the right choice. We will inevitably see more autonomous systems and indeed we are doing a lot of work looking at common autonomy systems and the deployments of those in all sorts of domains, land, sea and air and below the sea as well.

35:37The use of autonomous systems is going to be a political decision as to the acceptability of that and the role that humans play in that decision making process whether you have a human in the loop or a human on the loop who is making the decision about where you deploy the system and the rules of engagement that are put around it so i think you know we clearly will see more autonomous systems over time i have to say i'm not sure sure i do share the vision of armies of autonomous systems fighting against each other to determine who wins it i don't think that makes sense actually in a political context but you know being clear around the the rules that you can put around systems so you can have confidence in how they're going to be used that that's where you know an enormous amount of thought in government and in ministries of defense is is going on and you know i think we

36:28Danny Fortson: will see that if all over presumably this is like i don't know self-driving cars like the technology is already there right it's just the question of how do we roll it out in other words there has to be a decision to your point by governments or the west or whatever to cross that rubicon to actually be like okay we're going to take our hands off the wheel and maybe that moment never comes well we do have autonomous systems now so you know defense of a maritime platform will be done some of it will be being done autonomously that as a missile comes in there will be systems that spin up see that and fire a response faster than a human could engage with but there's a lot of testing around that to clear understanding of the situations in which it will respond but the Rubicon in that sense has been crossed there are autonomous systems I think the question is how far you then go with that and and ultimately that's a policy decision and does it mean the end of boots on the ground you know one of the things we can use autonomous systems for is is trying to take humans out of harm's way wherever wherever possible but you know does it eliminate boots on the ground at some point you have to be able to demonstrate political will and you know that's about being able to hold ground katie always gives me flack because i love a robot it's just a male thing that's i love robots the humanoids there's so many humanoids and you know the humanoids are coming etc etc and it feels a little bit ridiculous in science fiction but to the point around boots on the ground if you have build up an industrial base where you can pump out 500 000 a million robots yeah exactly um you know troopers is that a realistic thing is that like you know when you talk to the pentagon they're like yeah we're just we really love to have a droid army in 10 years where we don't have to and that it sounds ridiculous but also it's keeping humans out of harm's way it's a little bit hard to know how to respond to that we will clearly see more autonomous systems i think it's unlikely we will see armies of human-eyed robots because why would you design them to look like humans in the first place for a start you would design for the mission but you will see more autonomous land platforms more autonomous air platforms at working junction in order to take humans further back from danger where you can but that will be i go back to my earlier point that is in conjunction with other systems and that element of in the moment decision making I think will always be will always be necessary.

39:01What sense do you have about the way politicians and governments are leaning on that issue of sort of trying to use robots more and taking boots off the ground like which way do you think we're going if you're a betting man? Yeah, the SDR was clear about more use of reusable and attritable platforms where you would expect to see autonomy. The Navy is talking about its future vision for the Navy, which is the hybrid Navy, which will have more autonomous platforms alongside leaner, crude vessels. So, you know, I think we will see that direction of travel continue. But I don't think we'll get to the point where the humans aren't there.

39:44well i thought he was incredibly diplomatic i mean it it's it is a really difficult thing to discuss and that line that he said you know in defense our motto is we protect those that protect us i mean a lot of people i think will will wonder about that issue of protection but if we put that aside for one moment the future that he painted of the defense industry industry was was really

40:09Danny Fortson: interesting yeah and like i said there's just a whole new wave of companies in defense like a whole new wave of defense startups and it's just uh it is quite extraordinary how dramatically it's changed and the one thing i would say two second history lesson is this is really the start of silicon valley out of you know coming out of world war ii there's a bunch of pentagon contracts You know, Lockheed Martin was out here. Like a lot of the kind of the tech industry started with big military contracts, building stuff, whether it's chips or missiles or bombs or radar systems or whatever kind of gear.

40:52Danny Fortson: and then over the course of the years the kind of industry was like yeah we don't want to really do that anymore if we do we're not going to talk about it and now it's kind of come full circle and you have this whole new generation and this whole new attitude that is kind of proudly patriotic about this whether you you know people can have their own opinions about it but it is quite extraordinary how the kind of the shift has happened i would say again over the last five years going from you know big walkouts and protests to hey check out our new cool like you know bomb robot drone thing or whatever it may be on the on the tech front actually i mean obviously bomb robot drone things are sort of interesting but the most interesting thing to me was the the question that he didn't answer when you asked him about cyber weapons and you said are you building cyber weapons?

41:46And he said, we've got a really broad portfolio. I actually wouldn't go into the details of that because I think you're spot on. In terms of the future of warfare, as quantum computing develops, the acceleration of generative AI, it is absolutely that cyber warfare that not only governments, but obviously businesses and whole organizations are absolutely terrified by. But yeah, he dodged that question quite neatly.

42:15Danny Fortson: Yes, indeed. I think we can infer what he really meant. Yes, exactly. And he has no kind of, what's he going to say, right? He's like, yes, actually, we are building cyber super weapons. But clearly, that's part of the mix. Do you know what? I'm sorry to say, I'm not sure we have answered that big question. Is AI actually keeping us safe? I don't know how I feel about that. I actually feel like it's scary. Yeah, I don't think we know. I don't think we know. I think it's just like, it's like everything with AI. Actually, going back to something that Elon Musk said this week in court, he's like, AI is double-edged because you can do all of this new stuff.

42:50Theoretically, you can have all these scientific breakthroughs, but you can also build really dangerous things.

42:57Danny Fortson: And I think that we're in the middle of this process of trying to figure out what we enable, what we're okay with, what we're not. and obviously when you're talking to BAE, which is the largest defense company, one of the largest in the world and certainly in Europe, they're right at that bleeding edge of figuring out, what are we going to use this for? What level of autonomy are we good with? What are we not? All that stuff. So it's really tricky. It's a really interesting, weird time to be alive. If you guys like the show, tell us. Go to techpod at thetimes.co.uk. that is techpod at thetimes.co.uk leave a comment good or bad whatever we love feedback and we will see you back here next week see you next week bye bye