Between Two Nerds: Can AI defend critical infrastructure?

0:03Tom Uren: Hello, everyone. This is Tom Uren. I'm here with The Grugq for another Between Two Nerds episode. G'day, Grugq. How are you? G'day, Tom. I'm fine. And yourself? I'm very well. This week's edition is brought to you by Authentic, the open source identity platform. Find them at goauthentic.io. So, Grugq, I came across this wonderful little, I guess, interaction on Twitter. So the first post is from Heather Adkins, who I think she's the VP of security at Google. Some very senior position at Google. She's been there for ages. And it's just, pst, reminder, defense is going to win. And so she is talking.

0:48That, by the way, means this is going to be the last episode we need to do. It's officially the end of BTN. We found out the defense is going to win.

0:57Tom Uren: So she did it in reference to a particular post from someone congratulating CrowdStrike for coming up with a particular AI model that is good for cyber defense. That's not really important. I actually agree with Heather in the sense that I am optimistic that in the long term, AI will have positive effects on security in the long term. I think it's fair to say that all of the potential for benefit does lie with defenders. It's... I wouldn't say all, but... Realising that... Okay, there is significant potential. It probably outweighs... Yes, yeah. Yeah, I think so. It's just... Yeah. So anyway, like, nice tweet.

1:44Tom Uren: I agree with the sentiment. And then there's a reply from an account, Mr. Reboot, UK Daniel Carr. Yeah, Daniel. Daniel Card, yep. Technically, we already do, I think. We can defend way better than most attackers can attack. Where we struggle is from an organizational will point of view, exclamation mark. And I thought this was a wonderful reply because... I mean, in a way, we're all millionaires. We're just not quite there yet on the money front. And so it reminded me of Hal Laugh Flakes, I guess, I assume it was on Twitter. I don't know where I came across it. I think it was his defensible internet keynote where he might have done it first.

2:27It's been his thing for a while.

2:29Tom Uren: Where he says that all attackers' problems are technical and all defenders' problems are political. And this kind of reflects that, expresses the same sentiment. So we went from the optimism of Heather Adkins to the, oh, it's a political problem. We're never going to win because politics doesn't change in the space of a tweet and a reply. And I'm not sure if Mr. Reboot is being sarcastic. I think it's a very entertaining tweet or he's just being honest. and that it's... So we're going to talk about whether AI actually changes the dynamic because I think there's a deep truth. Right. I think the problem is both of them are right.

3:18Right. And it doesn't easily reconcile one way or the other, right? Defense can win and probably will win, sort of given enough time maybe. all we need to do is solve this political issue first

3:40Tom Uren: yeah so the lens i kind of think about these things or about cyber security in general is that if you want to be secure if you dedicate the resources if you actually spend time and money getting good people and thinking about it you can be acceptably secure and so the way that demonstrates itself in Australia, for example, is that banks are pretty secure because the big organizations, it's easy for them to see when they're losing money, when they're being ripped off and they can draw a straight line between cybersecurity spending and profit. Right. I mean, that's literally what they do, right?

4:22It's about risk appetite for them, right? It's like how much, like this is a risky loan, we won't do it. Whereas this, like we're willing to accept, as long as the payout is this much, then it's worth the risk of whatever. Like that's literally their whole way of thinking. So it's easy for them to apply that logic to cyber.

4:39Tom Uren: Yeah, but for many other organizations, the link between security and profit is much more tenuous. And in fact, for a lot of, until I guess encrypting ransomware came along, a lot of organizations, it made no difference. They could get hacked up the wazoo, and the effect on their share price or their profits was mostly minimal. So there was no actual incentive to do anything about security, so they didn't. And that, to me, is the politics that we're talking about for a lot of organizations. It actually never made sense to invest a whole lot in security, therefore in security. Right. I agree with that.

5:19And the way I've seen it expressed is a thing called Martin's Law. and Martin's law states that first you need the will, then you need the resources, and then you get the technical solution. It's very much a one, two, three. It has to go in that order. If you don't want to do it, it doesn't matter how easy or how little it will take. You're not going to do it. And if you desperately want to do it, but you won't put any money into it,

5:47Tom Uren: then it's still not going to happen sort of thing. So it seems like the one thing in the recent past that's improved security is actually ransomware. We've spoken about this before because it gave many organizations a reason to care. Right. So it created the will in that now there is finally a reason to be concerned. And once there was a will, then you could allocate resources based on how much you thought it was worth. And then you could do the technical solution from there because that's the easiest part of it, I guess. Yeah, yeah. The technical solution always existed, right? In the sense that much of the answer for ransomware has been just have good backups that you can actually restore from.

6:32Yeah, and so much of it is also just like security hygiene basics, right? It's very few of these things come out where it's like, okay, and, you know, then they used three chain zero days that had just been discovered specifically for this one target. And then they used modulation of the hard drive to create a virtual Wi-Fi that was able to communicate to the next rack over that they then, you know, hopped over. like it's always um they went on an underground forum and spent five dollars on the list of known passwords then they spent three hours throwing thousands and thousands of usernames and passwords at us which we didn't notice until one of them worked and then they were in immediately

7:18Tom Uren: it's all depressingly simple i guess yeah so the the question i'm thinking about is the the The fundamental problem is that organizations have to make decisions about where they spend their resources. And typically, many of them underinvest in my thinking in security because it was actually a rational decision. Because security is the most important thing, as we all know.

7:45Tom Uren: They underinvested in security because I'm a poor, benighted cybersecurity person who thinks it's important. Everyone's wrong but us, Grock. Right. And so the rise of AI changes the trade-off in that it becomes, at least in theory, easier and cheaper to have better security. Maybe in theory. I guess we can talk about that more. But it also, they're still asked to make trade-offs, right? So the political decision or the resource allocation decision, it doesn't go away. It's just that the numerator and denominator are changed. Right. So it's basically, I guess the way to look at it is, you know, before, you know, if you want to be this level of secure, you're going to need three full time people and a manager to look after them.

8:41And that's going to be, you know, this amount of spend per year. And if you don't feel like doing that, then it's very easy to not do that. whereas I think with an AI, maybe what's coming out is you've got, now you need one full-time person, and they can be a fractional manager looking after them and doing the other stuff that they do, like the CTO type, whatever. It doesn't need to be a dedicated security thing. And they can do the same amount of work in terms of just configuring things, deploying patches, writing policies that no one reads, It's just, you know, that sort of thing becomes a lot easier to do.

9:21So the resource cost goes down. And that might make, I guess the logic is, if you bulk at the idea of hiring three new people, maybe you don't bulk at the idea of hiring just one person. Right.

9:36Tom Uren: I mean, I think in this discussion, we're talking about the kind of receiving end of cybersecurity, the organizations that are trying to do something else. And I think part of the reason I'm optimistic is that the big organizations, the Microsoft, the Google… Oh, you mean the support function for the cybersecurity department? Like all of these other organizations exist to support a cybersecurity department in some fashion. And, you know, they probably, whatever they're doing that supports cybersecurity, you know, whether it's banking or commerce or aviation, it's, I suppose, it plays some role.

10:08But, you know, a lot of times they get confused and they misallocate resources into these side hustles as opposed to where it belongs.

10:19Tom Uren: What I was going to say is that the reason I'm optimistic is that the big organizations that are building all the infrastructure, they seem to have adopted AI and they are making more secure products. Probably maybe from now. And I think that'll probably accelerate. And so there's good stories about Google, for example, incorporating it into their standard development processes so the products come out more secure from the get-go. Now, I'm talking about – we have been talking about organizations like, I don't know, say, water sector utilities or energy infrastructure where they're very much on the receiving end.

10:59Tom Uren: They've got very limited impact on what they've got now. so if they've got things now that's basically what they'll have until they replace them right so the in whatever the lifespan is you know 50 years or 20 years or to piggyback a little bit on that google thing chrome is not part of their critical infrastructure right yes you know the fast updating better security it's it's far better now than it ever was sort of thing doesn't impact the Windows 2000 server that's running everything.

11:35Tom Uren: Right, yeah. So we're still left with four, and I guess that would be the vast majority of organizations, right? This long tail of cybersecurity dross or insufficiency. So even on the politics angle, I'm just thinking that in a lot of cases, we're making fun of, for example, that the Polish, in December of 2025, there was a hack on the 30 Polish substations or something like that, where they were disrupted, they were mucked around with. And the report was quite damning in terms of like, they had these systems that didn't have logging, didn't authenticate the firmware images that were uploaded, had default SSH root passwords, had SSH enabled...

12:22Tom Uren: Was this a recent report? Yeah, this was in the beginning of January that it came out, the Polish one. Like the solutions were things like upgrade to one of the firmware patches that's come out in the last three years. Right, yeah. You know, like any single one of those since like 2022 would be fine. And, you know, we look at that and we despair like, oh my God, how could anyone do that? How come they're not running the latest or whatever? but I think that in a lot of these places they're not they're not built in the technology right like that's not their sort of first thing right for them the technology is an enabler that has to do its job and a lot of the time that would mean that there's contractors and support contracts and like things that are very much like as long as you don't touch it we will guarantee that it keeps working like we will warranty it as it is right but as soon as you do anything right yeah It's your problem, right?

13:17And so you get to these situations where it's like it's working. We could dick around with it and then become responsible for any problems that happen later. Or we could leave it in a state where it's working and just, you know, continue as is. And the rational choice for them is leave it as it is because it's not broken. It's still supported. They know what it's doing. They don't have to test whether the changes that come from an upgrade are going to disrupt everything and cause knock-ons. It's a very logical decision if you have this naive assumption about no one will ever try and attack me, so it doesn't matter if I don't do this, I guess.

14:01Tom Uren: Right. I mean, you say it's naive, but it was also true for the vast majority of... Most of the years that they were running, they never... Yeah. Yeah, it's like an experience that's outside of what has happened to them before. Right, yeah. So, I mean, it's naive from my perspective because for all of my adult life, I've known about hacking. Whereas I think for them, it's maybe come as a little bit of a shock. Right, yeah. You know, I thought this happened to me sort of thing, yeah. That is very similar to the kind of water infrastructure story I wrote about today where it's, You actually need people to do things like change passwords and firewall systems or remove them from the internet.

14:45Tom Uren: I kind of struggle to think, at least right now, that AI would be able to do that. And I don't know what a compensating control with some sort of clever technology would be. Would it be like some sort of magic deep inspection AI machine that understands everything? Like that seems not very plausible anytime soon at all. Yeah, and I'm just thinking, like, these are not necessarily systems that you want to hand over to an AI to take care of. Oh, you mean like critical infrastructure providing power and water? Yeah, like just in general to be like... Oh, I'm sorry I turned off your lights, but there's no electricity.

15:22You're absolutely right to push back. I should not have dumped all of that chlorine into the water. That's on me.

15:32like that's one one sort of catastrophe that can happen but to me i think the other one is just you will lose the institutional memory of what has happened and what has changed so that when things go wrong you won't even know how things work anymore to be able to try and troubleshoot it so um you know i suppose we're supposed to say like yeah but the ai will do that for you I think that you still need to know what's going on because you like AIs now and I don't think they ever will be in the future will be able to take responsibility for you know if they get

16:06Tom Uren: things wrong well if you look at that hugging face hack and it seems like part of what enabled it was that there was a win condition that you could tell had actually occurred so I think the story was they had hundreds or thousands of agents running at a time and I think so sort of logically most of them failed but if there's one that succeeds it knows it succeeded because it you know it's measurable got the success flag of whereas if you're mucking around with critical infrastructure it's like how do you know that you've succeeded with the right plan i don't know like there's no test that you can have other than empirically like flick this switch and see what happens right so it's i mean i think it's it comes in like in a way it comes back to halvar's thing of like offensive problems or technical problems which means that they have sort of binary solutions it works or it doesn't work like it's testable and ai is very very good at things where you can test that the solution is correct and if it's not correct then it can try again and And if it is correct, you can stop.

17:14Like it's got a very good mechanism for that. Whereas how do you test that you are secure?

17:21Tom Uren: Because nothing happens. And so technically up until about December of 2025, all of those systems were secure. Perfectly secure. Absolutely secure. Nothing had ever happened. right so yeah it's like it's very it's very difficult because uh on the one hand the political problem is the like how do you allocate resources and whatever but the other thing is how do you know that you're secure enough when have you put in sufficient controls that it has reduced your risk to a level that you're willing to live with right yeah so one of the things i came across in this week's research was the uk national cyber security center they published in the last month a worked fictional example of here's how you implement our secure connectivity principles for OT, operational technology.

18:17Tom Uren: And so they've got this. This will be set in the fictional country of Oland Perry.

18:26Let's just say there's like 30 or 32 power substations that we could do.

18:31Tom Uren: Yeah, so it came out at the beginning of August, very prescient. So principle one, balance the risks and opportunities. Two, limit the exposure of your connectivity. Three, centralize and standardize network connections. Four, use standardized and secure protocols. Five, harden your OT boundary. Six, limit the impact of compromise. Seven, ensure all connectivity is logged and monitored. That one actually seems pretty straightforward. That's tractable, right? Yeah. That's quantifiable. yep eight establish an isolation plan now what struck me about so many of those is that there's no black or white answer right right so in the quantifiable right you can't say unlike the the logging one where it's like if your logs are enabled everywhere they exist or they do not exist right that's it but it's like you have risk or you don't have risk isn't the case right it's not a it's how much risk.

19:28Yeah, and in the very first principle it talks about

19:31Tom Uren: you need to know what your risk appetite is. Right. How do you balance something if you don't know what you're weighing on either side? Like you can't do that. I mean, I'm just thinking AI is not good for that either because they're sycophants. They're really, really not good at pushing back. So if you say like, yeah, I think given our position, our risk appetite is, you know, it's obviously very, very low. But given that we've never had anything happen to us, I think it's absolutely fair to leave these very, very old systems with default passwords online. And we go, you're absolutely right. Based on historical trends, you have like, you know, they're not thinking things with agency.

20:14They don't have their own agency to come up with their own ideas to push back.

20:18Tom Uren: They do what they're instructed to do. Even if they're instructed to push back, they will push back. and fabricate reasons to do so, if that's their instruction and they can't find a real thing, they'll just come up with one because they're following their instructions. Yeah, so all of this felt like both a lot of work, but also a lot of work that relied on coming up with some sort of consensus position based on the context of your organization, what you're trying to do and what's acceptable and appropriate. I was just going to say, these are principles for the principles to figure out. yes yep right it's like the principles at you know your company have to sit down and say here are our beliefs on these things and they they're going to get fed information to make informed decisions hopefully right they're not going to do this naively with you know you know assume or whatever they're going to have some information and then they're going to sit down they're going to hash it out and say like okay here's the the risks that we know about here's how much it would cost to do 80 % security versus 60%.

21:21What are we willing to live with and how much are we willing to allocate? Balancing all of these things, here's the solution we're going to go with based on these guiding principles of how we should...

21:33Tom Uren: Yeah, so it seems to me that maybe AI in that kind of situation will be, okay, we can do that for cheaper, like you said, with less full-time staff. and so that maybe changes what you're willing to do. Right, it's going to change the calculus. When you're doing your balancing act of political will and stuff, you're looking at like, well, we're going to need a lot of will to spend a huge amount of money. But if we only need to spend a small amount of money, then it's a lot easier to get consensus on that. Yeah. Now, on the other hand, it may be that using that same FTE, you could make a whole lot more money.

22:17Tom Uren: So, FTE, full-time equivalent, same number of staff. So, it's possible that it doesn't change anything at all. Like, you're left with the same fundamental dilemma. Do we make more money or do we make things more secure? So, nothing happens. So, I mean, I agree with that, but I think critical infrastructure is different in that the way that they make money is quite strictly regulated. so like if you're a software company and you're saying like we can delay our launch by three months for every release cycle by adding security or we could go out three months early and then patch it later like that's money you're leaving on the table if you wait three months and it might that might be the survivability of the company that's like the frontier lab strategy right now isn't it right it's just it's pedal to the metal all the way right but i don't think that exists for like an electric company.

23:14It's not like if we don't spend that on security, we can raise our rates by having someone figuring out how to raise rates. And then like it doesn't seem like it trades directly into more money for them. It doesn't equate to profit, whereas it will for the private sector for a lot of things.

23:33Tom Uren: Right. So I guess that's pretty optimistic, actually. Now, to be a bit of a downer, I guess the things we're talking about Taking systems off the internet and making sure that they're firewalled or behind a VPN or something like that. That someone logs all of that. There's monitoring and logging for all the people who SSH in this route from the internet. And changing passwords and stuff like that. So that seems to me to be pretty fundamental stuff and not necessarily stuff that I think AI would be all that helpful with. Here's the thing is. Or maybe it is. I think it would be helpful, and it's like, look, I've got 32 substations.

24:14There's 200 systems I'm going to have to SSH into. Claude, SSH into these 200 systems and change the password to this one. The problem is there's no way of guaranteeing that it actually does that for every single one. I mean, you could get it to write a script that would do that, right? Yeah. And then, you know, it'll stop working and it'll go in and manually fix the one and be like, but that password was insecure. So I used a better one and I didn't write it down. So no one could copy it. the disk. I mean, I think, obviously, there's technical solutions to that specific problem. But I think, in general, what I'm getting at is that AI by itself, like, you're not going to buy an OpenAI$20 subscription to secure your substation and be like, you know, chat CPT, secure everything, and go away and come back, and it's all done.

25:09You're going to need a lot of other stuff that then maybe AI can orchestrate it, but You're still going to need all of that other stuff first. And so there's going to be spent.

25:20Tom Uren: I suppose a booster would say that it will get better and that it's a bit flaky right now, but over time it will be able to do these things. Yes, sure. I mean, as soon as we get flying cars, we will also have working AI. It's not falsifiable, right, to say in the future AI will be able to do anything. It is falsifiable, but only in the future. At some point in the future.

25:50Tom Uren: Just not right now. It seems so much like, now that we're putting it this way, it seems so much like there was the joke that's, so this is a Soviet joke, which is communism is always on the horizon. What's the horizon? It's an imaginary point where the sky meets the land. No matter how far you walk, it never gets any closer. So it feels like to me Martin's law makes a lot of sense to me, right? You need the will. And I think we both agree AI doesn't change the will. But will is not a binary thing. It's not like, yes, we want good security. No, we don't want good security. No one is making a binary decision about that.

26:33Tom Uren: They're talking about how much do we want to dedicate to it. Right. And the optimistic case, I guess, is that AI and its ilk will make the costs of having good security less. It'll be able to help you do more with fewer people. That kind of seems realistic. Yeah. So the resource spend would go down and the technical solution becomes easier. Therefore, the amount of will that you need to accomplish a goal is a lot less. Yeah. Making it. it's like a zempick for weight loss except for security the ai a zempick for cyber thanks greg