Risky Business #851 -- Agents are just ones and zeros, and tigers are just atoms

0:05Patrick Gray: Hi everyone and welcome to Risky Business. My name's Patrick Gray. We'll be getting into the news in just a moment and as always James Wilson will be joining me as co-host as will The Grugq who is sitting in the third chair this week. This week's show is brought to you by Ant Security or Ant AI, whatever you want to call it. But basically Ant is a, what do you even call it? It's an intent aware security product that sits on an endpoint and makes heavy use of AI. They came out of the gate strong with a$100 million seed round recently. And they are scaling things up pretty quick. It's like, you know, this is a startup on the move very, very fast.

0:51Patrick Gray: So Brandon Dixon, who is one of the co-founders of Ant, will be joining us a little bit later on to talk, I guess, about what they're able to deliver already. Because they're a very new company. And what's really interesting is they're going into big enterprises and doing these pox. And even though the product isn't completely developed yet, they're still converting and selling to these companies. because there's just certain things with this paradigm that you can do that nothing else can do. It is actually a very interesting conversation. So that's with Brandon Dixon coming up after this week's news, which starts now.

1:23Patrick Gray: And some news from close to home. I first heard of this from our former colleague, Brett Winifred, who lives in WA, not too far from Cottesloe, which is where two alleged Team PCP hackers were apparently arrested. and there you go, a couple of them were Aussie, according to the police. They're all innocent and everything, please don't sue us. But let's start with you on this, James, because I know that you have been in regular contact with Brian Krebs who has had the identities of these two people who've been arrested for quite some time.

1:57James Wilson: Yes, for quite some time. It's been very difficult to keep this a secret. But yeah, so Krebs and I have been in touch and we've been tracking this. It's been, the identities have been known for a long time and we've been just waiting, waiting, waiting for law enforcement to move. There were some challenges around international travel that dragged this out. But eventually these two individuals have been picked up for their alleged involvement with Team PCP. The Krebsing article itself is, of course, it's one of the, you know, wonderful Krebs reads as they are. But this one in particular is great because you can sort of, you can almost follow this story arc where Brian's like, I've got some info on this.

2:35James Wilson: I can't quite connect the dots. And he almost sort of steps back and goes, well, this is weak source. I haven't got enough in if I want it. And then he does a lookup of historical DNS and suddenly the case is blown wide open. He's got the ability to see some family members had registered a domain on this IP that had been used on some forum activity. That leads him to a Facebook. He discovers the family members. One of that family members is this Reuben Thompson that was picked up by police. and then the whole thing just made a whole lot more sense.

3:05Patrick Gray: Yeah, so everything just sort of clicked into place but it certainly seems that the names that he's had since I think June were the names of the people who were picked up, right? As sort of floated by the media. What's interesting is one of them allegedly is linked to the NSN here in Australia, which is like the Nazi people here in Australia. So I guess it's been a bad couple of weeks for them because one of their leaders killed himself which was just wonderful actually. I was so happy when I read that. Tim Lutz. Yes, he was up on charges for raiding an indigenous encampment and decided to do us all a favor and shuffle off his mortal coil.

3:40Patrick Gray: And now we've had an arrest of someone apparently who is involved with that in addition to Team PCP. And of course, yeah, both of these guys have been picked up, allegedly have said some pretty gnarly things in their chats. They're not exactly woke, let's just say that. But Grok, you got any thoughts here? I mean, you know, you're Mr. Opsec, and I'm guessing you have some feelings on their Opsec or lack thereof, right, in this instance.

4:04James Wilson: So, like, to be fair, when you read the story about it, you're getting the how I found them, not the how I failed 20 ,000 times before I found them. However, that path is so embarrassingly bad. Right? Like, it's just, and the standout has to be the cat picture angle, where this guy had a custom avatar of a cat that he was using on both his, like, illegal activity, allegedly, and his not illegal activity, which is a great way of, you know, identifying yourself. It's just, it's beautiful to see. It's sort of like a case study over and over and over again. There's so many different ways of picking it apart.

4:50James Wilson: Yeah, second only to the restaurant reviews left up and down the coastline where he was in Perth. Oh, buddy.

4:57Patrick Gray: Yeah, I think I remember having a good old laugh with, man, Kevin Mitnick had a beef with like Lizard Squad. You remember that? And I remember years ago, hanging out with Kevin in Vegas one night, and we just wound up in stitches laughing about how Lizard Squad actually posted, like they got one of their girlfriends to post like a picture, but it was geotagged. Yes. You remember that?

5:21James Wilson: That particular photo was in my OPSEC for hackers because JL is for WooFTP. So in that talk, her picture is in there as like greatest OPSEC fails. She's on Twitter and she's messaged me now. And she likes to post that photo saying like the boobs that got so-and-so arrested. She contacted me over this Team PCP thing. I know about the cat pic because she contacted me and said, there's a better pic to get busted over.

5:55Patrick Gray: But, you know, again, an Australia connection because I believe she was actually located in Melbourne. So there you go. Just if you're an Australian hacker, don't post pictures Stay away from pictures, I guess I think that's the take-home message That's right, that's right Now look, we've got some breaking news here Which is the White House apparently Is this a White House thing? There is apparently a pilot program kicking off in Texas That's going to look at how to better defend American water systems In the wake of all of these Iranian attacks Well, apparently Iranian attacks against water infrastructure I think they began in Minnesota But spread to a whole bunch of other states James, you've had a look at this.

6:31Patrick Gray: I believe Catalan is going to write about this for us tomorrow as well. But, yeah, what do we know about this plan to fix water plant security?

6:43James Wilson: Yeah, it is very much a White House thing because it's got Trump all over it. He's done speeches about it and talks about it, and it's all about how he's putting the American people first with this initiative. It's quite nauseating to read the press coverage over it, But what it really boils down to is it's a six-month pilot being run out of Texas, where essentially federal agencies and some private enterprises, Microsoft and Dragos were mentioned in the article, will essentially give pro bono investigation and support or remediation help to Texas water facilities. It sounds good, but my argument here is I just don't see how this scales.

7:20James Wilson: This is not a get in there, find the fix, then roll it out everywhere. where every one of these water treatment plants or OT networks will be so sufficiently unique that they're going to need this over and over again. And do you do this again in every state, in every treatment and continue to do it for free? I don't see how this ends up with an overall broad-based remediation of this OT problem.

7:45Patrick Gray: Yeah, it's like we're going to deliver 18 ,000 consulting engagements now. Yes. It has those vibes. Now, look, unfortunately, we have to talk about the Hugging Face incident some more. It just refuses to die. In this case, what's happened is OpenAI has released a whole bunch of more information about what actually happened with all of these agents kind of colluding to own Hugging Face and like spring up message boards inside OpenAI and whatever. I mean, it's not, I think it's just more detail. It doesn't really add much to what we know. Funnily enough, though, Dworkesh Patel has written this huge, you know, essay about it, talking about the rise and fall of agent civilizations.

8:26Patrick Gray: And this has turned into this really polarizing piece online. So funny, too, because we had this brief conversation last week, James, with Ollie Whitehouse, where he's like, oh, well, we don't want to anthropomorphize these agents. And my point was like, well, who cares about that conversation? If they're trying to hurt you, they're trying to hurt you. It doesn't really matter if they're alive. Like, who cares? Like, it seems like a bit of a side issue here. And then we've just been bombarded with a week of that sort of discourse. But let's start with you. Very funnily, though, Grok, you tried to take this report, feed it into an AI agent and try to get a summary.

9:01Patrick Gray: And it shut you down because you were violating the cyber guard rails, which is just sums up this whole experience. But James, I know you've been looking into this. Did we learn anything new or is it just, as I say, more detail about what happened?

9:15James Wilson: Oh, Pat, I think the two things that are sort of net new since we last talked about this was we got a little bit more of an insight into the scale. You know, when we were talking about this before and we said, look, why wasn't anyone watching this? You know, where's your monitoring? We've since found out that this is on the scale of 700 agent sessions simultaneously and something like 1 ,200 of them colluding through this message board, which, OK, on one hand, you could say that's why they couldn't monitor it. But at the same time, if you're going to unleash 700 plus agents all working in parallel to solve CyberGym, I reckon you might have a problem you want to solve before you actually hit the go button on that in terms of your reservability.

9:52James Wilson: The other thing that we've learned is that the attacks didn't stop at Hugging Face. So when the initial incident response kicked off, they could see some signals that there was some further hacking that had been done beyond Hugging Face. But interestingly, you know, I guess incident response diligence, they said, nope, this is our scope. We're not going to go and look at that. So with that scope now clear, they went back and looked. And what they found is that there was the initial learn how to message between yourselves on Artifactory. Then there's learn how to message between yourselves on Artifactory to hack Hugging Face.

10:24James Wilson: And then there was learn how to message between yourselves on Artifactory to go and hack the Kubernetes cluster that you're actually running the entire simulation on, which, you know, on one hand, I still remain in the camp of this is not surprising. This is what happens when you unleash an LLM. But also that, you know, what is it, the William Gibson quote of the future is already here, but not even evenly distributed very much applies here. This is the future of what cyber attacks will look like. It just happens to be this is what cyber attacks look like when you can throw an entire Frontier Labs resources out.

10:57James Wilson: But how long until that's just the norm and the inference improvements and the costs drop mean that this is just what we see everywhere.

11:04Patrick Gray: Now, Grok, like me, you are also old and have been around this space for quite a long time. One of the reactions that I always have when I read about stuff like this is I just think this is exactly what people do. You know what I mean? Like it really, like these robot hackers are very much like, this just looks like typical, trying to do this without swearing, without using bad language, but it looks like the typical SFery that goes on in a CTF when people are breaking the rules and just going, okay, man, I'll pop shell on the Kubernetes cluster and just grab all the flags that way. Ha, ha, ha.

11:38Patrick Gray: That'll be funny. It just seems very much like people behaving badly.

11:43James Wilson: So I remember a CTF where they figured out which free BSD version was going to be used to host all of the flags and was going to be on everyone's network. They had a kernel Oday for that and they just remote popped every single team plus the judges. and like that was their CTF. Then they started, you know, dicking around with everyone. So like this doesn't strike me as unusual behavior. Like this, like if you said -

12:11Patrick Gray: This is what I mean, but like there's all these young folk freaking out.

12:15James Wilson: Yeah, if you said like three freshmen who had just, you know, for the first time had access to like a network university and had just learned how to do cyber stuff were caught hacking into like the teacher's lounge and they hacked into like the administration stuff. That's this. That's exactly what it looks like. They're just, you know, as I was saying with Tom last week, is that like, this is the platonic ideal of hackers, right? They don't sleep. They're just eternally curious about trying to do something. They never give up on a task. And they don't really have a model of how other people works.

12:49James Wilson: They don't process like, maybe this is bad. I'm hurting someone's feelings. It's just like, but I have a job to do. I must do this, you know?

12:57Patrick Gray: Yeah. Reminds us of a few people we know, right? Right, and look, I know it's interesting there that you said that they're endlessly curious because you get yelled at by the Twitterati for anthropomorphizing these models, but the best reply to that whole argument I've seen is someone, what's his name, Matthew Zaitlin on Twitter, there's a graphic of how to survive a tiger attack and he's got the guy being stared down by a tiger and it's okay because he's thinking it's just a bunch of atoms and biochemical reactions, right? So remember that tigers are simply made up of atoms and various biochemical reactions.

13:36Patrick Gray: The tiger's appearance and behavior may be scary, but do not let your fear get the best of you. Decades of research into biology and physics has shown that tigers are actually composed of very small units called atoms, and on and on and on it goes. So I think this is the point, like don't get distracted into the whole, you know, we got to be careful about the way we talk about agents because we've got to be careful not to anthropomorphize them and whatever. Although Hal Barflake did have a good point when he was saying that human beings regard orcas who have brains bigger than us and social hierarchy and stuff as things.

14:10Patrick Gray: And now we're talking about clankers as if they're people. So, I mean, we've got to keep our heads somewhat grounded, right?

14:19James Wilson: Maybe the first time an AI rips the rudder off a yacht by Gibraltar, we'll start calling them things again.

14:26Patrick Gray: There you go.

14:28James Wilson: That's a very deep cut.

14:30Patrick Gray: It is indeed. The Iberian Orcas do find the old spade rudders on rich people's boats somewhat tasty. Tasty little snacks go floating by. All right. So now we're going to talk about this letter. So we've got this letter, which has been signed by 100 companies like OpenAI, Anthropic, Google, Microsoft, and a whole bunch of others, saying that there needs to be a global surge in AI-powered cyber defense to face the threat of AI cyber offense, I guess. And this kind of reminds me of the whole CISA shields up moment, right? Where they're like, we need to put our cyber shields up. And it's like, okay, you're assuming that we've got like cyber shields.

15:12Patrick Gray: You're assuming that we've got like this spare capacity in our armory that we're just waiting to deploy. In this case, though, it's pretty funny when you do have these AI companies talking about how we need to prepare to defend using their products, of course, against the threat that are coming from their products. It has very strong, we're all trying to find the guy who did this sort of vibes. James, what were your thoughts reading through this letter?

15:38James Wilson: Nothing but complete unbridled frustration because it still remains the message from these people saying, we've all got to step up and meet this challenge. It's like, yeah, but you're all the ones that have the tools that we need to be able to step up and rise to this challenge. And you're not giving us access to them. If I could just interrupt you for a minute, James, I have detected that. We have a very loose filter. Yeah, we've got a very loose filter over here. And we've noticed that you're talking about cyber. So we're just going to drop you down to another podcast. I forget that the input classifier is a five-legged if statement in these models.

16:14James Wilson: But look, the thing that gets me here is that it's not just that there's – it's one thing to say, you know, we've all got to take a step forward, but then you're not unilaterally providing the tools to do that. But the other thing is that the access methods that they have for actual, you know, trusted and vetted cyber professionals to actually use these things is a completely opaque process. That's just not acceptable, right?

16:38Patrick Gray: That said, that said, you were recently approved for the OpenAI one, right?

16:41James Wilson: I agree with that process, but just not the process in general.

16:46Patrick Gray: Yeah. I mean, what are your thoughts here, Gruck? Because this does seem, I mean, what's really funny is this seems like a PR exercise. I had someone in government say to me, like, what is it with people in tech and open letters? Which I thought was a reasonable question. And they also said, like, this does seem to be some sort of PR exercise, but the optics are terrible. because you're sort of coming out saying like, oh, look at this mess we've created. Now we're going to punt it to you to clean up. Like it's just really tone deaf and the optics are not good and you just sort of wonder what on earth they're doing.

17:21Patrick Gray: But what are your thoughts there?

17:23James Wilson: Yeah, so it's very much like the hacking is coming from inside the house, right? Like these people are going on like someone has unleashed all these terrible agents upon the world and it's going to be very important that we step up and figure out how to pay us more money to stop us from unleashing these agents upon the world. But I guess the good news is at least China is releasing open models that can then be used to actually solve the problems caused by America's frontier models, which is what Hugging Face had to do. And that's, I mean, that's not a great sign for the future.

18:01Patrick Gray: It's not. But I mean, look, you know, the open weight models are going to be, I can't wait for the big open weight model to come along that's not Chinese. That's going to be the big game changer, right? So, yeah, that'll be an interesting day. But, like, let's have a look at some more interesting stuff that's been going on with the way that AI is impacting offense. And you spoke about this with Tom, my colleague. So, by the way, for anyone who's listening to Grak right now thinking, wow, this guy is hilarious and charming, and I want to hear more of him, you can listen to Between Two Nerds, which is a weekly podcast he does with my colleague Tom Uren every week in our Risky Bulletin RSS feed, or you can find it on YouTube.

18:38And last week you were talking about the way threat actors

18:42Patrick Gray: are using AI to basically rapidly cycle through like IOCs, right? So they can actually make their campaigns varied enough or they can inject enough variance in their tooling, TTPs and whatever in their campaigns so that it makes attribution something you have to do on every target instead of just, oh, look, that's that same C2. We're just going to cluster that activity here or that's that same binary and that same C2. So the idea is you could wind up with a Vault Typhoon or Assault Typhoon that because they have access to these AI tools that they can use to rapidly develop new infrastructure, deploy new infrastructure, rapidly develop new binaries, new infection techniques, entire new sets of TTPs, the idea is you won't be able to trivially and easily cluster these things together.

19:31Patrick Gray: You won't be able to threat hunt. So from a macro perspective, governments are going to start to lose visibility into some of these big campaigns because they're not going to look like big campaigns anymore. This to me is something that, you know, I haven't really heard anyone else talk about. And it's actually, you know, this is a meaningful change, I think. Right.

19:52James Wilson: So, I mean, it's a new problem in cyber because historically we've been gated by cost. Like the ideal would always be to have your own unique set of tooling for every single target that you go after. But that was prohibitively expensive until six months ago, essentially. And now that that's gone away, it's possible now that cyber can go up to the level of the human intelligence stuff that happens, where you do have a sort of unique individual approach for every single agent and penetration that you're doing, so that if one of them gets compromised or discovered, it doesn't cascade out to all the other ones.

20:33James Wilson: This would be OPSEC, which, you know, a couple of kids in Perth should have learned about. However, like the trick here is that the way that you defeat this is not by trying to cluster things at the very end. It's by you have to penetrate the guys that are attacking you.

20:52Patrick Gray: You have to do proper infiltration of the foreign agencies that are running the campaigns and observe them that way. And we know, look, we know NSA does that, right? We know they do that. We know they're pretty good at it, but it's not always easy. And it also means that you're not getting the benefit of the civilian world being able to cluster. And, you know, when I'm talking about clustering these things together, like, you know, NSA doesn't have the incident response capacity of like Mandiant, for example, right? Like they can't be in those places. and they can't be in those places because they're private companies.

21:23Patrick Gray: It's a commercial sector, right? So yeah, this is a problem in the making, I think.

21:28James Wilson: Absolutely. And we have been coasting on the fact that cost has forced people to reuse tooling. So in a way, like this was inevitable. It's just it's arrived quite a lot sooner than you would have predicted. Yeah.

21:43Patrick Gray: And we've got a case in point here, which is some research out of Bitdefender, which I actually thought was quite impressive. Like I wasn't familiar with your game, Bitdefender, basically. But it was a really good write-up. James, you would have had a chance to have a look at it. And it's like, you know, they've developed like seven rats. And it's like, basically Bitdefender's take on this is like, this is what it looks like when top tier operators start using AI. Not just scattergun, noisy, dumb stuff, but like really doing careful AI assisted campaigns. And what I found really interesting too is that they were using dumb lures that are kind of easy to spot in the end because they didn't want it to look suspiciously sophisticated, right?

22:27Patrick Gray: So everything after the lure was really quite well done. But the lures themselves are just like, you know, I mean, it wasn't a, you know, by Viagra lure, but like maybe just one step above. James, what did you think of all of this?

22:39James Wilson: Yeah, it's very interesting. Tom and I talked about this last week in Seriously Risky Business. And, you know, two things that I got out of this one. One is that what's interesting here is you can see that there was a product spec for these remote access Trojans because the overall sort of size and shape and net outcome of what it did was the same. But its implementation details just is wildly different each time. Different language, different protocols, different back end. And so, but again, this is exactly what you'd see from a software company that was using AI. We've all learned that these things work really well when you spend the time with them to get the product spec right.

23:16James Wilson: And then you just say to them, you know, hands off, go do it. That's out the code. We don't even look at the code anymore.

23:22Patrick Gray: Now, one of America's adversaries might have struck a serious blow to its refrigeration. The Department of Defense has these military commissaries where apparently active duty military people can go and buy discount food. and someone has disrupted the DOD's commissary refrigeration system or their freezers by sticking them on the defrost setting. These are networked freezers and there is a rumor going around at the moment that this might have been a hack. And when I say hack, I'm guessing that probably means these things were sitting on the internet with some admin password like default credentials and someone just logged into them and pressed, even if there was any auth at all, and press the defrost button.

24:11Patrick Gray: James, you were actually the CTO for Woolies X, which was the online division of Australia's, I think, biggest supermarket chain. Why are freezers networks?

24:26James Wilson: Because when there is a sudden outbreak of salmonella or the health department comes knocking and says, excuse me, there seems to be an outbreak of food poisoning in this area, you absolutely want to be able to pull up the records and say, hey, sorry, man, all our freezers were running exactly to spec. Look, here's the temperatures. They were fine. So it's the combination of, you know, you want to be in that defensible position if there is a problem, but also just, you know, supermarkets run on wafer-thin margins, and so you're always looking for opportunities to have really good observability, cost, you know, limiting any waste.

24:57James Wilson: And so it makes sense that these are networked. On the internet with default creds, eh, not so much, but that's going to happen.

25:04Patrick Gray: But I mean, you know, this could cause some pies to be defrosted. I don't know how the Yanks will recover. But I mean, I guess this is one of those things. Look, if you are Iran, I'm not saying it's Iran. I've got no idea who might be behind this, even if it is or is not a hack. I'm just saying there's a rumor out there that someone did this deliberately. Is this sort of thing worth doing, Grok? Is it worth defrosting your enemy's commissary food for, you know, some sort of effect? Is it going to hurt morale when people can't get their favorite brand of frozen pie at the military commissary? And that might, you know, sap their will to fight.

25:38James Wilson: My take on this is, OK, it is possible that it's Iran or some other threat actor who's got this 5D chess approach. Or hypothetically, it's a 20 year old who found something on Shodan and clicked the button that said defrost. And then nothing happened and they went away. and suddenly it's in the news and they think it's hilarious. Yes.

26:03Patrick Gray: Well, it's going to be hilarious until they get the knock on the door, right? Until next week when we read about them. Yeah. Until we read about them getting arrested. Now, look, this next one here, this is more AI stuff, right? Which is that Codex, Claude Code and Hermes installed unowned code inside corporate networks. James, what do they mean by unowned code here? What's the story?

26:28James Wilson: So it's a little bit nuanced, this one. Look, unfortunately, we seem to be increasingly filing all of these AI stories under what did you expect? And this is another great example, right? You can, by convention, drop on your website this file called llms.txt or llms.full. These are the files that an agent will prefer to read to get information about the site, right? It's an llms version of robots.txt. But of course, these agents, they consume that. It goes into the transcript. It gets processed like anything else and they'll take actions. And so there's been - So hang on.

27:02Patrick Gray: Are you telling me we can prompt inject people by putting llms.txt on our website?

27:07James Wilson: Yeah, our one goes live in about an hour. Don't worry. It's going to be fantastic.

27:12Patrick Gray: I've got to spit up a Bitcoin wallet address. This is going to be great. But anyway, sorry.

27:16James Wilson: Continue, please. But in this case, the whole thing about unearned code is it's hard to tell what exactly has happened here whether it's just um you know an ai agent somewhere created this llm and there's a bunch of hallucinations in it or whether there's something nefarious going on here where they've been told to install a package that actually doesn't exist and so someone at some stage could create that package or create the domain and then use that as an infiltration vector it just because that's what this when they say unowned that's what they mean it's like there's instructions that they're finding in these llms uh dot txt saying please go and install this package or please grab this thing from this URL, but that URL or that package doesn't exist.

27:55James Wilson: I think we can file this under, you know, it probably just, you know, don't assume malintent if stupidity will cover the case. And then what's happened here is that the reason these LLMs or TXTs exist with these non-existent packages and domains is because, well, an agent created them and it just hallucinated them along the way.

28:13Patrick Gray: Man, that's just like many layers of hilarity there, but go on. Sorry, Greg.

28:18James Wilson: I was going to say, I think the follow-up to that is that's now created the vulnerability of someone being able to create those. So even if the malicious intent did not exist, the fact that it has happened has now established there's this potential for malicious activity, which, again, that's not a thing that used to happen. It didn't used to be that you would produce software that would then create the opportunity for someone else to produce vulnerabilities to then hack someone. It's the code version of the self-sourcing pudding. Comes with vulnerabilities back then.

28:54Patrick Gray: That's what I call AI agents is like, you know, the self-sourcing bash script, right? I had a really interesting conversation this morning with Randy Pargman, who's over at Sublime Security. And that was an interesting conversation about what they're having to do to deal with prompt injection. Because you can imagine as an email security vendor, right? Like you got a bit of text in email and that's going to wind up not only being looked at by their security agents. And interestingly enough, this is where people are putting their first efforts is like trying to get like security agents to burn too many tokens.

29:29Patrick Gray: So eventually some accounting process comes along and snipes it and just lets it through because it'll fail open in that instance. but also you know you've got to watch out for prompt injection attacks against stuff that's behind your gateway as well so it's a really interesting conversation like the way they're getting through that at the moment a lot of it's stuff like oh white text on white background that's always a signal in email anyway so I guess we can we can you know flag that and you know so there's like from the very simple to the no pun intended to the sublime when it comes to detections for Yeah, prompt injection.

30:03Patrick Gray: But it's just interesting when you talk to someone where their job is to solve an unsolvable problem because it's not a solvable problem. But yet their customers will expect them to deal with prompt injections. I think that one's published next week. So yeah, it was just an interesting chat. I enjoyed talking to Randy. Now, let's talk though about Ukraine giving Britain access to battlefield data to train AI. Grok, you've got a note here, which I thought was very, very funny, which is that Ukraine is trading battlefield data so that the UK can train terminators. And in exchange, they just want a license to make long-range missiles.

30:45Patrick Gray: So I think this is like we're really thinking about, like, you know, two countries that are operating on somewhat different timelines here, I guess.

30:52James Wilson: Right. So, you know, obviously Ukraine, they're under existential threat. They're going to use what they can to get what they need. And obviously, for everyone involved right now, this battlefield data is, you know, as we know in the AI world, they kill for data. No pun intended. But this is very unique, right? Like this is not books. This is not like, you know, users walking around, consumers buying things. It's got nothing to do with you literally cannot get this anywhere else. and so it's got to be worth its weight in gold or in long-range missiles if you will yes and so

31:32Patrick Gray: worth its weight in long long-range missile uh you know construction approvals i guess yeah

31:39James Wilson: it's but it's not going to be terminators it's going to be those freaky cyber dogs that come after us with guns yeah and fpvs run by themselves like terminators now seem like really nice and cuddly like, oh, they look like us. That's nice. As opposed to terrifying flying demons that blow up.

31:59Patrick Gray: Yeah. Ain't the future great? James, I see you've had a bit of a look at this one.

32:05James Wilson: I just have a small request if anyone from the UK body that is now in possession of this started, just a small request. Could you maybe wait until we've sorted out the whole sandboxing escape problem before you do a training run on this? Please, just maybe. What we need is the MOD to come out in two years with a 100-person signed letter saying, look, we all know about the incidents last year. We all need to come together and do a shields up to stop these Terminators going out of control. We need to surge in defensive battlefield data.

32:39Patrick Gray: We've got a warning out of Anthropic that infostealers are stealing clawed sessions, which I just think is funny in that people are hungry enough for tokens that they'll use stolen tokens. So, yeah, there you go. There's a whole market now for stolen tokens via InfoSteel and Malware. Now, it's not often we talk about a DDoS attack, but one has actually knocked out some pretty important public services in Norway. James, what's the go here?

33:11James Wilson: Yeah, they're having a bit of a rough time. I think this is the third or so attack that they've had. And this is the Norwegian Digitalization Agency, also known as Digdir. I hope I'm pronouncing that correctly. It's actually one of their partner companies that keeps getting hit by this DDoS. But it affects a lot of really basic things like ID portals, digital identification. You take that out. And if you've got a relatively modern digital infrastructure for government, then, well, if identity doesn't work, it doesn't really matter if the rest of the services are up. And the concerning thing here is, as they said in the article, it's the second or third time this has happened, but this is the largest by, I think, two or three times magnitude in terms of what they're getting hit with.

33:54James Wilson: And still they have no attribution or understanding of who's behind this or why they're getting hit.

33:59Patrick Gray: Yeah, that's not good, right? I mean, it's just funny because every time you think, oh, you know, the age of big DDoSs is behind us, like someone just turns up with a bigger cannon. Yeah. Like, I think the funniest one of that was when Akamai were looking after Brian Krebs' site and someone tested their latest, you know, botnet and like Akamai had to cut him loose because it was going to threaten, they were just giving him free services, but it was like posing a threat to like Akamai's global network. So they had to cut him loose. And that was a bit embarrassing for everyone concerned. Grak, do you have some thoughts on this one?

34:30James Wilson: Yeah. I got to say, just reading it, it seems like the problem is there's this one provider that is actually central to all of like all of these other services and it feels like i'm not sure there's another way of doing it with you know digital identity but it does feel like if you go with a single point of failure you're gonna you're gonna have problems at some point yeah

34:56Patrick Gray: but i mean it's i mean look yeah and we'd love to talk about federated identity but there's always going to be that single point where you can cause a problem like this though you know that's the problem, right? When you've got a root of trust, there's a root. Yeah.

35:10James Wilson: When the state issues you an ID, it's not like you could say, oh, I make mine in the garage, and the state just, you know, at some point says, that's cool, you know, his garage is trusted. Yeah. It does have to go back somewhere.

35:22Patrick Gray: Yeah, because eventually someone's going to say, are you sure you gave this guy permission to make the ID in his garage? And that's the plan to stop that process. Well, you know, you see what I'm saying. Alright, so we've seen, too, I just wanted to touch on it briefly but like we haven't even been talking about it but there's been all of these campaigns owning SharePoint boxes like just left right and center um there's still more bugs out there in SharePoint just a bad time to be adminning SharePoint at the moment uh but someone's trained together two bugs and they're just smashing honeypots with them at the moment I think these are both patch bugs but yeah there's an exploitation wave going that's about the long and short of it

35:57James Wilson: isn't it James yeah but one was patched in July one's patched in August um the two bugs were kind of rather inert on their own, but chained together, they're quite useful. So if you haven't patched, now's the time because, yeah, honeypots are seeing these being chained together and used for an exploit.

36:12Patrick Gray: Yeah. And meanwhile, PaperCut, the printer software, is just getting owned sideways as well. The company has told people to take it. They released an emergency advisory telling people to take their PaperCut servers off the internet, which like, well, yeah. The problem is, though, it's very popular on campus networks in universities and things like that. And they're like, well, you should IP restrict them. And it's like, well, they're not very useful if you IP restrict them, right? So that's turning into a bit of a headache for all concerned. Wanted to touch on this one specifically to talk to you about it, James, because someone has thrown together this voice phishing as a service kit.

36:51Patrick Gray: So like when you've stolen someone's iPhone and you want to get the passcode so that you can turn off find my iPhone and you know get the registration unlocked from its you know from its original owner and whatever that's what this kit does but it does it in a really clever way because it will reach out to the user and say hey it's Apple looks like your phone got stolen yeah let's you know let's get it back for you let's uh let's sort this all out so give us your um you know give us your details but it's using voice AI agents and whatever now of course the reason I wanted to talk to you about it is you worked on iCloud stuff as well these would have been features that you wrote code for i'm guessing you read this and had the same reaction though that i did which is like

37:31James Wilson: this is pretty clever yeah i mean uh you you said that the platform being thrown together ah no man this was engineered really well like this is um it's just they've understood the threat model so well and and the way to make this work right and it's it seems to be running at a sort of an industrial scale where a phone gets stolen from that you get the ime you can use various look up services within Apple. They determine then that the device is in lost mode. If it's in lost mode, it presents the contact details to contact the person that's put it in lost mode. They use that for their phishing and their vishing.

38:03James Wilson: And there's even, as you said, AI interactions to pretending to be a support rep from Apple. You know, good news, we found your phone. It's at the nearest Apple reseller. We just need your passcode and whatever else. So it's really good work. The thing, when I looked at this last night, I was trying to find a conclusive answer as to whether or not Apple's stolen device protections actually help you out here. And disappointingly, when you look at Apple's documentation as a little bit like Microsoft's documentation, where one document links to the next document, which takes you back to the next document, which loops you back around to the first document.

38:34James Wilson: And I couldn't get a straight answer, but I think it does actually end up being a wrinkle in the plan here, because if you do have Apple's stolen device protection, which is an extra thing you've got to turn on, then it will require a biometric unlock with Touch ID or Face ID before that lost mode can be turned off. But the wording was too weaselly, and I couldn't actually get a really strong sense that that was definitely the case.

38:57Patrick Gray: So Apple didn't think of what would happen if someone's face was severely injured in a bear attack and they needed to sell their iPhone. I mean, this is a typical uncaring megacrop here.

39:11James Wilson: It is, but as we now know, bears are just atoms, Pat. And so you shouldn't be too concerned for this.

39:17Patrick Gray: They're just a series of chemical reactions and atoms. Let's not anthropomorphize them. We're into the last couple of stories here now, and this one caught my eye. A Slovenian casino, or Slovenian casinos have reopened after some sort of attack knocked out their, like, you know, pokies, I guess, their gaming systems. Was this a shakedown? Because the systems were down for a couple of days. Was someone ransoming them, James? or was this just like the byproduct of some other type of attack? What do we know here?

39:46James Wilson: We know very little, but I'll give you a little bit of insight from a previous job I've had where I oversaw the technology for a loyalty program, let's say, in a large retailer. There's a part in this article that says what they've been able to restore is some of the gaming machines on the floor, the pokies and all the rest. So that tells me that this was an attack of some kind, I would bet ransomware, where they had to restore certain things and they've been able to get just enough things back up and running for more sort of the day-to-day operational stuff. But the loyalty program remains offline.

40:15James Wilson: And that tells me it was ransomware. It hit the databases. They've not been able to decrypt that loyalty database. Therefore, it's still offline. So that to me was sort of the tell here of not just a, you know, not an extortion attack, I don't think, but actually someone locked up that loyalty database. Otherwise, you would absolutely get that back online as along with everything else because it's such a core part of how those businesses run.

40:39Patrick Gray: But to be clear, that's speculation. Total speculation. But it's just interesting it's called out.

40:44James Wilson: It's called out in the article that the loyalty program is still down

40:47Patrick Gray: and that just made me go, yep, okay, that's, yeah. An educated guess, shall we say. All right. And finally, we've seen this, the ATF apparently admitted to some major incident. But it's really weird, right, because it's declared it as a major incident, it's reported it to Congress. But then you read about it, it's like, well, it was a single system that got popped. I think you can reconcile those two things in that it was a single system that got popped. I think it was the Keelan Ransomware Group or someone like that. That gang popped a shell on one box. But I think there was enough sensitive stuff on that box for this to actually be a pretty big deal.

41:22Patrick Gray: Is that how you reconcile those two things, James?

41:26James Wilson: Yes, because what was on that standalone system was who's being investigated by the ATF.

41:31Patrick Gray: Well I'm guessing some of them. I don't think they take one machine and put, let's just put a list of every single investigation on this one machine.

41:39James Wilson: That's just speculation, Pat. Just speculation. It is. It is indeed. We're all just speculating here.

41:46Patrick Gray: Crownjewels.xls. That's right. Crownjewels.xls. Oh, no, don't open. But guys, that's actually it for the week's news. Thank you so much for joining me to walk through all of that. It's been a pleasure to chat to you both, and I'll catch you all soon. Good to see you, Pat. See you around.

42:04James Wilson: Thanks, Pat. Once again, what a week.

42:10Patrick Gray: That was the Grok and James Wilson there with a look at the week's security news. Big thanks to both of them for that. As I mentioned at the top of the show, this week's show is brought to you by Ent Security or Ent AI, which makes endpoint security software that uses AI very heavily and it can find all sorts of cool stuff. It even uses computer vision, right? So this thing can tell you, hey, someone who works in accounts got a weird email that looked a bit suspicious, and then wound up changing the destination bank account of one of our suppliers. Like it can actually do stuff like that. And as a result of that, what they're finding is even though this is a product that is very much still in development, they're going into these large companies, getting up and running and finding enough alarming stuff quickly to do with user behavior.

43:00Patrick Gray: sometimes when they're being tricked into doing stuff, sometimes when they're a malicious insider. They're finding enough that they're actually winding up converting sales like already, which is a pretty sort of insane thing to do, to happen when you're such a small company. But Brandon Dixon, who is the co-founder of Ent, joined me to talk through some of the stuff that they're finding. And yeah, it's a very interesting conversation. And Ent, you know, eventually, for those of you who haven't heard about them, you're going to encounter this company. You're going to run into this company out there pretty soon.

43:30Patrick Gray: But here's Brandon Dixon walking us through some of the stuff that they have found. We've seen cases where, you know, an employee will log in to like meeting software, like Zoom and give remote control of their system for eight hours a day. And they might be working five different jobs. And so they're using that corporate asset. And the way that this sort of manifests is we can detect when someone presses that remote control button. we can see the behavior over time kind of accruing to a single app without really other work necessarily taking place or it's happening at the same time so hang on why are they doing this to a corporate machine are they are they trying to work from home and they're using zoom to like like what what's the actual thing that they're doing they literally invite somebody outside the organization that they've contracted part of their job to outside the org and that person logs in and then they're working another job.

44:26Patrick Gray: So you basically, this is like a labor arbitrage thing where you basically you hire someone in the Philippines to do your job for five bucks an hour. Is this? Yes. That's amazing. And we've seen people doing like five jobs throughout the day. It's like, they'll commit a couple hours to this job. Then they'll do a couple hours on that job and they do it through someone's corporate asset on the network, which is mind boggling in its own right. Right. So you've got so you've got multiple like staffers for basically people in other parts of the world coming in and doing the jobs of like five of the staff.

44:59Patrick Gray: Wow. Or just being employed by like four different employers and then they farm it out to these other contractors. Yeah, right. So they share. They're like, hey, let's hire someone in India and we'll share them. And the interesting thing, like in the case of Zoom is like sometimes the tell is like you'll see the technical employee's name, but then you'll see like three other people that have been invited that also have the same name. So they're not dumb, right? They're not like just inviting like some person outside the org is like an external entity per se. They're literally setting up Zoom accounts that match the same employee name.

45:33So if you look at it, you go, oh, okay, they're having a meeting with themselves. Maybe, I don't know.

45:39Patrick Gray: Okay. So that's cool. And, you know, detected that. What was the detection mechanism to figure that out? Looking at remote control and being able to see like there's an inordinate amount of time being basically spent within Zoom working different browser, like using the domains to kind of see the different external entities that they're going to that have other companies as identifiers in it. Okay. I mean that you've explained to me like how Ant was able to see something and say that's weird, but how far did the inference go? Did it go as far as to say, this person has hired someone to do their job?

46:17Patrick Gray: Like, you know, like, I guess I'm just curious, you know, how far the product could go in terms of figuring out precisely what was happening in addition to just saying that's weird. So the way like it, it'll, it can characterize the person's work. So like if we are plugged into like the IDP system and we know that person's role, so let's say they're a software engineer. And, you know, we see that they have remote control on Zoom. We collect the rest of the contents of what it is that they do throughout the day. We have a deep investigation feature that will allow you to summarize like that person's day or their week and basically write up relative to that person's role what was normal versus what was not.

47:00and that is a prime instance where it'll be like this person spent an inordinate amount of time doing things on other assets that weren't related to their original job in the first place

47:11Patrick Gray: and it calls it out so it was the fact i mean the fact that they were sharing this resource made it jump out like big time yeah because well the remote control in this like that session went on for so long like it runs through the entire day and these are big enterprises that you're dealing with too these are smes like these are jumbo megacorpos and this is a prop like is this a known like i've never really heard of people doing this before this is amazing oh yeah so this uh there is i think it's like over employment is like a reddit sub subreddit thread uh about people that are like coming up with all these techniques and hacks to basically like get multiple jobs and then run them through on the remote access side and look like you know some people, maybe they, look, I'm not, who am I to like say what's right and what's wrong?

48:00I don't believe that's, you know, right to do that. I do know, I'm sure that some people could probably do three jobs. And like, if all you're being asked to do is, you know, a small amount of work and you're incredibly good at what you do, maybe you should have multiple jobs. I don't see an issue with that. The problem that I see in this is like, you're employed at a company and you're using that corporate asset to then bring other people into the asset to work three other jobs from there. And that's the part where I'm like, this doesn't make any sense to me. It's so weird.

48:33Patrick Gray: This is some real Gen Z BS, right? Is what I describe it. Like where the attitude is, Hey, the work's getting done. You know, who cares? Like what else? I mean, you know, you mentioned to me before we got recording that you found a bunch of fraud. I'm guessing that this is the fraud that you're referring to. Or was there other stuff as well? This is fraud. We've worked with one customer that happened to have screenshots of six months of activity associated with someone that they were already monitoring before we even got there. And they had done a pretty extensive investigation. And it took them months to go through all of this material and find out what occurred.

49:19And in this case, it was more like skimming, if you will, like skimming money from contracts and setting up a bribe.

49:29Patrick Gray: Okay, so this is the fraud. There was actually fraud fraud happening where someone was like redirecting suppliers to pay to certain things. Yeah, I know. I'm guessing. Wow. And you're able to. Okay, so that's interesting too that you said the screenshots because a lot of that insider risk stuff currently and DLP stuff, one of the things it will do is when someone's under suspicion, it does fire up the screenshot of Matron, right? But yeah, as you say, manual... But in this case, they already had the screenshots. No, no, that's what I'm saying. But then there's that manual review process after it.

49:57Patrick Gray: I'm guessing your stuff is a little bit more, you know, auto-magical than that. Yeah, so we basically fed it into our harness, the same thing that powers like a deep investigation, and it processed it in less than 15 minutes. And it wasn't complete. So I do want to caveat, like, AI obviously still has its issues with completeness and thoroughness. But with the screenshots alone, it found 14 of kind of the conduits of where money was going. And it found the sources of where the conversations were happening. And it identified the people that were involved. Well, I'm guessing at that point, too, you could have asked it to go and collect more screenshots and see if there were, you know, other instances of this happening.

50:40Patrick Gray: I mean, I'm guessing that's the idea here. Yeah, well, the idea was, was, had we actually been in place six months prior, would we have seen this? And how would that story have manifested in the product? And the answer was, we would have covered most of it enough that someone wouldn't have had to spend months doing it. Within as little as 15 minutes, they would have got the leads to actually go and investigate and go, Oh, God, like, there's something wrong. Yeah. And so like not everything's been bad, though. Like, I think most people are trying to do their jobs and do them well. We have one instance where, you know, it was the someone put in their notice.

51:19And actually, they were this one, they were let go from the company. So obviously, you know, unfortunate, but there was concerns around the DLP side. You know, there was DLP products that were in the account, but they didn't have context. And so there was an instance where, you know, that day, that person went in, they booked a hotel room, they registered for an event using someone else's name. They went to their personally united account, registered a new credit card, booked a flight. and that in itself was very strange like to be doing it but then they also went into various ai tools and they're like you know distilling all the information you know of their work and

52:11Patrick Gray: putting into like a single html file that's interesting that's like uh tell me uh tell me everything i need to know i'm leaving my job like what should i remember you know tell me agent it'll do it yeah that was that was essentially it and the business like look it was a business where it It was like gracious with folks where they were okay with some things leaving. So that's a case where you're like, oh my God, this is like, I remember seeing it. And I'm like, there are so many things here that's like, I can't believe this is happening. And we went and worked with the customer and they're like, this is incredible that you guys saw what you did.

52:46Because our kind of like expense system is not connected up to the security side in the same way. we had the DLP solution going and it didn't see any of this context of what you guys are describing. But it's actually not bad. And I was like, huh? And they're like, yeah, the this person is actually participating in an event next month that is sanctioned by us. They're allowed to do it. And, you know, we are letting them go. But, you know, they were allowed to go and register for everything. And it was actually above board. But they were like, there's no other way that we would have seen anything like this had you guys it could have been bad in this case like we're just being super friendly yeah and and look that yeah and that's i've seen the cases of two where you know the combination of like personal and corporate on an asset this is something where like i i don't know where people stand on the line here uh but i i see corporations increasingly like incentivizing people to do their personal work on the laptop.

53:48And so it's, it's a challenge for us to be able to like discern what is what's personal versus not. But when you see work files going from a Dropbox account that we know is associated with the business over to one that is personal, that's cause for concern and being able to kind of see that as it happens and then know what exactly transferred and understand the context that happened around it is incredibly powerful.

54:14Patrick Gray: I mean, that is, you know, but that's like, that's, you know, standard DLP stuff that people have been doing for a while. I mean, I think the, you know, the case you gave us about the people, you know, farming out their jobs to like low cost jurisdictions, that's amazing. And we should be clear too, that the way that you're detecting this is actually with endpoint telemetry. It's not, you know, you don't need this thing to hook into every single corporate resource and data store that you have. It's all, it's all our telemetry. Yeah. It's all your telemetry off the endpoint. I think the most exciting thing that we identified was like your kind of standard DPRK sort of incident, your remote IT worker.

54:57And that was a case where I've, I've talked about this on LinkedIn, where I think I've interviewed these people. There's like this uncanny feeling there. And when I pushed that out there on LinkedIn, I got a lot of engagement. I had a lot of people like telling me they're part of these like groups or they've interviewed people or they've accidentally hired them. And these were the things that they identified or like problems that they're running into. In any case, um, DPRK person gets hired. And the signal for that is first day they plug in a KBM switch, um, into their device. So we can, we profiled like the USB peripherals and we can see like manufacturers and whatnot.

55:35Um, And that's not in of itself, like necessarily DPRK or anything, but it's installed on the system. It's the perfect opportunity for us to look at like, what is that person going to do now that we know the KBM is in the mix here? And that was a case where both text and screenshots were helpful. We could have done it with text, but it's not the same.

55:57Patrick Gray: yeah you want to have that i mean there's that computer vision component which is very useful uh when you need it it's not you're not going to do it all the time it's expensive but you know correct like it was it was one of those instances like where they were using um they were using microsoft teams and if you've used that product there's the there's a button on the microphone with a little carrot drop down and it allows you to select your audio source and so when you select your audio source you're actually kind of clicking on like a button or a link and in this case because the kbm was attached as a source they could change that audio source to the kbm and so when they clicked on that we had a rule that basically just looked at like whether something looked like a kbm or was associated with a kbm on a button click inside of microsoft teams and so we didn't we just looked at a generic button click we happened to see it in teams and we could see that multiple times throughout the day that this person was conducting meetings and then they were like switching the source over to the KBM again not necessarily bad but at some point they go and send a message to themselves which is basically instructions to the person on the other end telling them the work that they should be doing and like there's your smoking gun and so you see that and you're like uh I think there's a problem here and uh all of a sudden like the evidence is pretty clear, like there's, there's something going on.

57:20Patrick Gray: And it's funny, right? Cause this is, you know, the reason I wanted to have this conversation with you is cause like, this is just the beginning. Right. And it's funny because, you know, you've got those handful currently of unique things that is making it difficult to have design partners who don't turn into customers, which as we've established is a pretty good problem to have. We're actually out of time. Brandon Dixon. Thank you so much for joining me to talk through all of that. That was very interesting. And yeah, let's see where it's at next time we meet. Yeah, can't wait. That was Brandon Dixon there from Ent.

57:55Patrick Gray: Big thanks to them for that, for sponsoring this week's show. And big thanks to Brandon for doing that interview. You can find them at ent.ai. So that is E-N-T dot A-I. Yes, all about inferring intent on the endpoint. Very interesting stuff. Something genuinely new. Do go check it out. But that is it for this week's show. I do hope you enjoyed it. I'll be back soon with more security news and analysis. But until then, I've been Patrick Gray. Thanks for listening.