Risky Business #855 -- ShinyHunters done sowing, enters reaping phase

0:03Hey everyone and welcome to Risky Business. My name's Patrick Gray. We've got a great show for you this week. We'll be joined in just a moment by our regular co-host, Mr. James Wilson. And in the third chair this week, we have Brad Arkin. Brad, of course, was most recently the Chief Security Officer for Salesforce. Prior to that, he was the CISO for Cisco and before that for Adobe. And a long time before that, he was even working for At Stake. He's part of that whole at-stake mafia crew, so he knows a thing or two about cybersecurity. So those guys will be joining me in just a moment. This week's show is brought to you by Corelight.

0:41And in this week's sponsor interview, James actually interviewed Greg Bell, who is one of the co-founders of Corelight, and spoke to him about a white paper they put out, which was really looking at, for their purposes, when doing sort of automated SOC work, like what's more important, a sophisticated model or really nice structured data, you know, being processed by that model. I don't think it'll surprise too many people to discover that the data side of that equation turned out to be more important, but there were some interesting lessons learned through that research process. And Greg and James get into that a little bit later on.

1:17As people joining, listeners, viewers, I guess, joining us on YouTube will be able to see that I'm not actually at home at the moment. Also, I am in Singapore. I'm over here for a joint Decibel and Tamasac Frontier AI Security event, in case anyone's wondering why I sound different and why I have very Asian-looking buildings in the background if you're joining us on YouTube. But look, let's get into the news now because we did just have a couple of weeks off, so the news backed up on us somewhat. So I guess the big news so far this week, James, we'll get your thoughts on this one to begin with.

1:54A French supermarket giant had its freezers apparently hacked and turned up to 50 degrees and whatever. So a whole bunch of food was spoiled. This is very similar to that incident that affected US military commissaries, which, you know, we were told through the grapevine was a hack. I actually got some angry email for saying that I was trafficking in rumor when it wasn't confirmed. But, you know, all of this has come after in August, some vulnerabilities were disclosed in Danfoss, like, freezer controllers that are usually connected to the internet. And then lo and behold, freezers start having this sort of stuff happen to them.

2:37What do we know about this particular one, though? Because it looks bad, right? because this is a big supermarket chain. A lot of food has been destroyed.

2:44James Wilson: Yeah, this is a big chain and it's affected hundreds of stores. And I think the thing that, you know, casual observers might not get about this is that supermarkets operate on just-in-time inventory in such razor-thin margins that any sort of event like this of having to, you know, throw out food, not only is a huge financial impact to them, but this will take weeks if not months to actually restore the supply chain properly. So it's going to have a huge impact for people. From the technical side, there's a lot of conjecture as to exactly how these Danfoss controllers were compromised. What we do know is there was a lot of refrigerators.

3:19James Wilson: All these controllers were exposed directly on the internet. It could have been some of the bugs that were reported. I've also seen some reports, you know, can't really substantiate them of perhaps even the password was laughably bad. All of which honestly seems completely plausible because this is what happens when you've got these internet connected, you know, controllers. They're probably not within the IT purview of the head office. It's run by some contractor. And, you know, I think this will be a hard lesson of what happens when one of these, you know, internet connected control systems slips through the cracks.

3:54I think the why here is going to, well, could possibly turn out to be interesting, right? Because when we've seen a similar thing happen to a military installation, there's nothing to suggest that these two incidents are connected, mind you. But, you know, in whose interest is it to spoil a bunch of food at a US military commissary shop and also European supermarkets? And there is a suspect that comes to mind. But obviously, you know, there is absolutely zero information out there on how to attribute this. But Brad, I know that you thought similarly, which is this feels like, you know, could it just be like a Russian chaos campaign?

4:32Yeah, for me, the question of who benefits from doing this. So, you know, people that are doing it for the lulls. So that's one category of threat actor. Russia just, you know, making everything miserable for everyone. That's, you know, another common category. and then it at first i was thinking you know maybe this was part of some poorly executed ransomware campaign but i guess you'd have to make the thread in advance and then if you don't get paid follow through on it and then demonstrate some kind of capability but like what james was saying this feels like a pretty sloppy environment and so i don't know proving that you can do it to this like is this going to make all the other supermarkets pay because they're not going to be able to fix these types of problems overnight so i don't know to me ransomware feels like a like a long-distant third option.

5:17And so my pure speculation would be either Lulz or Russia. Yeah, I mean, that's kind of how I feel it could be as well. It will be very funny if it turns out this is yet one more OpenAI test. You know, because it's seeing if it can do freezer melting benchmark or something.

5:35James Wilson: This is the supermarket gym test that it was trying to token max on. And we'll get into that later, but there appears to be a benchmark for absolutely bloody everything, which is just mind-bending. Now, this is not the only thing affecting European supermarkets. There's this company, and I'm going to completely mess up the pronunciation here, but it's written as Ahold Delhaisy, I guess. Dutch, I think it's a Dutch company, isn't it, that has subsidiaries everywhere and operates a lot of supermarkets. Of course, James, you were the CTO for the technology division of Australia's Woolworths supermarket chain, which is even by global standards quite large because there's only two major supermarkets control a staggering percentage of supermarkets in Australia.

6:20So you are actually familiar with this company. You were sort of, Woolies is friendly with one of its large subsidiaries, right? Yeah.

6:28James Wilson: So I worked a lot with the folks at Albert Heine, which is one of their subsidiaries in Europe. And actually, it caught me by surprise because I saw the news on this and thought, I wonder if this whole De Heise is related to Albert Heine. And then when you look at that, you know, yes, they are a subsidiary. And then you look at the overall scale of something like 9 ,400 stores across Europe, United States and Indonesia, 400 ,000 people. It's like, wow, the scale and footprint of this company is huge. What's not clear is the actual nature of this attack. Yeah, I mean, I should have set that up better because I said, I'm having a bad time.

7:01But, you know, what I should have said is they've taken a bunch of systems offline in the wake of some sort of cyber incident. I did have a listener in the Netherlands sending me messages saying that this was showing up like at retail shops. You know, there's some serious disruption here. Yeah.

7:16James Wilson: So and what's interesting is the, I guess, the Venn diagram overlap of the impact seems quite broad. So US employees at Stop and Shop said that they were having trouble accessing their schedules and pay stubs. So that would sound like it's a failure to access or authenticate to an HR system. Whereas in Europe, the Albert Heijn, Eto 'o, St. Gall and Gall stores in the Netherlands were reporting that actually it's their own website and ability to do e-commerce orders and customer service platform is impacted. So it kind of feels like something pretty central has been impacted and infected here. But that's all we know, just speculating at the moment.

7:52Well, yeah. And like the other one, we don't know who, we don't know why. We just know that there's been a serious impact. I do find it interesting that we've seen massive disruptions to, you know, two separate grocery concerns in Europe in short order. Whether that's meaningful or not, who can say? We just don't know yet. Now, geez, you know, sometimes you have to make a bold prediction in this game. And last time we spoke a few weeks ago, I made the incredibly bold, brave prediction that shiny hunters might experience some blowback for having hacked the FBI. At the time, we only briefly mentioned that story because the story at that point was like, Shiny Hunters had hacked some sort of FBI Oracle PeopleSoft system and taken some data.

8:37It turned out that data was extremely sensitive. So I think it's fair to say that Shiny Hunters have moved on from the sewing phase and they are now squarely in the reaping phase because we've seen a bunch of arrests, right? So we saw one guy who's purportedly goes by the handle of Ray, R-E-Y, who's someone quite significant. I think he was the one that got picked up in Jordan. Someone actually did get picked up a few days before the public learned of the FBI breach as well. So that was sort of before it was announced. And we've seen this interesting arrest too in the United States of a guy who runs like a ransomware negotiation firm and they've arrested him in connection with the FBI hack.

9:22But it's all pretty vague at this point. You kind of get the impression that maybe this guy had done negotiations with shiny hunters. Maybe he has some useful intelligence and maybe they're trying to squeeze him to get that intelligence, or maybe he crossed a line or maybe this or maybe that. But you certainly don't get the impression that his core shiny hunters crew because he just doesn't sort of fit the demographic doesn't fit the profile brad what are you what are your feelings on what's going on here i mean because it does look like what you would expect to happen is happening which is the fbi is asking its partners to pick up people it knows about it's kicking in a few doors it's going after anyone who's even tangentially connected to them uh you know they want to cut off the money they want to collect intelligence and they want to finally deal with these guys is that your impression as well yeah so you know they do not like being embarrassed in public.

10:10And so the idea that the FBI is pushing every single resource they can in order to get something going on this investigation, the, the kid in Jordan has been fascinating to me because Brian Krebs was like interviewing him and his father, uh, long time ago, like a year ago plus. And so he's just known like everything, like I knew what his name was and where he lives and what his dad does for a living and everything else. And so my guess here is that they're going after anybody they can and leaning on all their partners in all these different countries to pick up anybody they can in order to now try to make some progress in the investigation.

10:49And then somebody from the FBI is recording videos and sending them out, basically taunting the adversary and saying, we've got some things rolling. This is your chance to come in before we get to you next. um so they're definitely setting up a game here and it sure doesn't seem like it's gonna end well for the shiny hunters guys um and i guess they're just banking on you know their sloppy opsec and different jurisdictions that have kept them safe up until now will you know their luck will hold but it sure feels reckless lack of will it's been lack of will and they've never realized that it's been lack of will i remember years ago after the like casino breaches which was like com kids same sort of group, right?

11:33Same sort of kids and speaking to someone quite senior at the FBI. So this would be about three years ago and saying like, why, why is nothing like, why are these guys not in cuffs yet? And it's like, quite often it can be difficult for the FBI because not everybody wants to get on board for an FBI investigation. They don't want to have to do all of the evidence collection and this and that, and then testify and, you know, you know, going through that, cooperating with the FBI and something like that is a, is a, is a pain in the, you know, what, right? So there have been issues for the FBI just in terms of like getting these incidents reported to them and then getting the cooperation from the people who've been impacted by them.

12:09But when the FBI itself is the victim, that's not a problem, right? So they can roll pretty heavy. The political will's there and that's what's happening. So yes, our very bold out there, unlikely prediction that shiny hunters might experience some negative effects from having, you know, been a little bit high on their overconfidence and going after the FBI. Yeah. You triggered some dark memories for me where six guys in suits with guns show up and you've got to put them in a conference room somewhere. And we don't want them to actually have access to anything or do anything, but we can't make them feel like they're being left out.

12:44Otherwise they're going to subpoena us. And then, you know, things get much more less pleasant. And so we had people on the team whose job was to go in and basically like buy them donuts every two hours and keep them talking so they felt included, but not actually give them anything. Did you do puppet shows? Yes, exactly. Yeah. So trying to keep them out of your hair so you can do real work and not risk the spiraling. Distract the G-men as a job on Brad's CISO teams at mega corporations. That's pretty great. But we've seen another massive data breach as well, which is millions of US personnel records were stolen from some US military system.

13:24It's the Defense Manpower Data Center, it's called. Someone broke into it, stole an absolute gajillion things. Like, was this Oracle PeopleSoft as well, James? No, I think this was just a file transfer appliance. File transfer appliance, yeah, right. So it could be CLOP maybe stealing data on millions of active... Like, why? Look, I think the better question here, okay, And for those who are not watching the video version of this, I am rubbing my face.

13:52James Wilson: It looks considerably pained for those that can't see the video. Why is the FBI putting its most sensitive information about what its personnel do in an Oracle PeopleSoft system that is connected to the internet? Now, we've seen the news that they fired the Accenture contractor who didn't apply the patch. But if the security of all of these critical FBI operations, including like counter, you know, counter espionage personnel and their details and stuff. If all of that hinges on someone applying a patch to Oracle PeopleSoft, you know, that's on the internet, like something has gone wrong. And now we've seen, you know, millions of details of people in the US military stole from a file transfer appliance.

14:33Brad, does this, I mean, does this upset you as an American? I mean, I'm Australian and it upsets me. It must be maddening for you crazy making like the the first thing that comes to mind is there was some bureaucratic uh path of least resistance that led them to solve a problem using like the worst technology possible and just just they got pushed in that direction uh because of like all the rules or whatever was going on and they just had to solve the problem they had to get like something had to be accessed by somebody else and this is the way to do it and then the real tragedy is you You know, this thing might have been stood up to solve a problem 15 years ago, and then they just left it there or something like that.

15:13Like that's entirely plausible. So those are the things that popped in mind for me.

15:16James Wilson: Actually, Brad, I can add to both of your pain significantly. When I was researching this, I found that the FBI themselves in 2022 posted a published a privacy impact assessment for FBIjobs.gov and its candidate gateway. This document actually details the fact that FBIjobs is a public facing recruitment website built in Drupal. Awesome. it talks about the candidate gateway which is the oracle people soft implementation and also how it connects to hr's hr source which is fbi's own internal hr system and it actually details quite thoroughly how the data governance is supposed to work what gets purged where what gets moved how there's these cross-domain systems but seemingly none of that was actually being done it's like it's all written up but the whole purge the data move the data mustn't have been happening because it seemed to have all been sitting there waiting to get snuffed up.

16:07I mean, yes, this does not sound, that's surprising really, does it? Yes. But it's still just, it's a little bit mind-blowing. But also, you know, the Shiny Hunters crew, you know, they've been having a good time. They've been getting their mileage out of this PeopleSoft thing. And one of the funny things they were doing is bypassing the mitigations, which were like WAF rules. And all you needed to do was whack in an old Unicode, you know, percent five zero instead of a P. And that would bypass the WAF rule. So that's the sort of level of technology and countermeasure we're dealing with here. It was funny watching you discover that one, James, in particular, because you're like, oh my God, I can't believe they're so dumb.

16:46And it's like, yeah, welcome to a bad WAF, which is most of them. This is what they do. Well, it's not just a bad WAF. It happens to be Oracle's WAF.

16:53James Wilson: So if you've got the whole Oracle stack, it hits you both sides. It's amazing. The credit default swaps on Oracle bonds are looking real juicy, right? At the moment. So like, if anything, there might be, we might get to finish this year watching Oracle getting completely destroyed by the, being at the epicenter of the AI bubble. And let us all just pray for a moment that that's what happens. Anyway, Brad, you had something there. Oh, I was just going to say, I've always been a WAF skeptic that the presence of a WAF creates more of a false sense of security because you'll put a rule in place and then somebody will tell somebody oh the rule's in place we don't have to patch anymore we can delay the patch defer the patch and so i've i've often asked to not have any laughs at all to just avoid this kind of confusion because it just feels like it's too big a game of telephone and anytime somebody's going to write a regular expression and that's what's going to keep you safe like you know it's just not going to go well um and so this is made me smile when i read about it because it's the perfect illustration yeah i mean it's funny though because there was one WAF that I liked, only one, but that was the Signal Sciences one.

18:00Because they were like, that's how they actually wound up like making a whole bunch of money is by actually making a WAF that didn't do stuff like this. But unfortunately they sold it to Fastly and you don't hear about it much anymore. So I guess now WAFs are just a CDN feature and that's where we were. But yeah, I mean, look, that's the same point, right? Like what are you doing using an Oracle, an ancient Jurassic era Oracle WAF and expecting it to do absolutely anything for you anyway? So look, I'm just scrolling through all of the news that we've spoken about, that we're talking about this week.

18:33And yeah, like I think we've covered most of that. Oh, there was another arrest too of a CEO of another ransomware recovery company because he said, like, give me your money and I'll decrypt your stuff because I've got amazing technology. And it turned out he was just paying the hackers, negotiating the ransom, you know, whacking some margin on top. made a decent, you know, made millions of bucks out of doing it. But unfortunately, that sort of misrepresentation of your services like could be construed by certain prosecutors in certain jurisdictions as fraud. So he's in a spot of bother. Now, look, something else that happened over the last few weeks when we're off.

19:08And, you know, I had a bit of exposure to this one through being on the board of Knock Knock, which is the Citrix Netscaler stuff has just been a rolling disaster. There's been a series of zero days discovered. Well, I mean, it started it off with in the wild mass exploitation of a zero day in NetScaler, which is like not a good time if you're running NetScaler and you don't have any way to mitigate this thing. And of course, you know, people, you know, there were patches came out and then more Oday and patches and Oday, and it's just been an absolute disaster. So that's just been horrible for people running NetScaler.

19:44The reason I say it connected to Knock Knock is we had like people who were doing POCs or wanting to do POX to protect their net scaler from exactly this sort of thing. And you know, those sort of deals, like they take a few months for people to get them in place. And like, let's just say a few deals got shook loose by this thing. But we had Kiteworks as well, you know, with a similar issue where they were saying, stop using our platform. Like it got bad enough that they said, just pull it offline. And it's so rare that you see a vendor actually say that. Now they've said, oh, we've patched it.

20:14Everything's fine. I mean, I would argue that everything is probably not fine. So those are probably the big ones over the last few weeks. But then, you know, you spend a couple of weeks away and then you come back and you do your news triage to prepare a run sheet for a podcast like this. And look, I see a lot of contrarian types on social media saying, oh, look, there's a bugpocalypse, but there's no breachpocalypse. It's just not true. Like you look at the amount of in the wild exploitation of enterprise crapware at the moment, and it is absolutely mind boggling. This is not an exhaustive list, but this is just from the last few weeks, right?

20:47So we've got in-the-wild exploitation of Zimbra. So people using a bug that lets attackers remotely issue operating system commands without authentication. Hey, good one, Zimbra. So yeah, a bunch of different CVEs in Zimbra being exploited. We've got Checkpoint warning about in-the-wild exploitation of its secure VPN gateway, ironically named, I guess, as it turns out. Then we've got bugs in SharePoint getting exploited, and there's been something like 16 of them in the last month or two, which is just wild. We've got exploitation of Atlassian floors, a pot going out and then it's in the wild.

21:26We've got CISA warning of, what is it, was it Team City by JetBrains. Bugs there getting exploited. I mean, it's just been, it's nonstop at the moment. And I think people really need to get serious finally about reducing their attack surface because you are not going to patch your way around this. This is something we've been saying on the show all year. You need to get serious about reducing your attack surface. Brad, you're the seesaw among us. Do you agree that that's the only way to do, to go with this sort of, to deal with this? I mean, this is going to keep happening until you clean it up because you've, you know, the fuel is out there and you either remove it or it's just going to keep going on.

22:08I was hanging out last week at a CISO gathering. So I still get invited to these things. And the feeling is that you see the bug counts and the patch volume is up a hundred or a thousand X and the attack levels are not up a hundred X compared to what they used to be but pretty consistently across the room it seems like it's about doubled from what people previously had as their previous high watermark and so in normal times if your attack volume doubled consistently across every single defender this would be like an unbelievably big deal but it just feels really tiny in comparison to the patch volume of the bug volume and so so i think people were prepared for absolute armageddon and when things double basically like existing processes and your instant response procedures and all that can still handle double, but they wouldn't be able to handle 10x or 100x.

23:00And so people were expecting to get completely tsunamied and swept over. And instead, they're able to keep up with it with the way they've always been doing things. But I think, you know, going back, like a lot of these appliances, the Kiteworks stuff, and, you know, Citrix, things like that, they're just not fit for the internet anymore. You can't put any, maybe that's been true for five or eight years. But the number of things that you should feel comfortable putting on the raw internet is shrinking really, really fast. And you can still use it potentially safely, but you got to put it behind, you know, some network based off before you're allowed to touch it and limit the population from the whole internet down to just people that have credentials that should be allowed access in the first place.

23:37Yeah. I mean, it's, it's, you need to re-architect your remote access, or you can put some sort of mitigating control, you know, knock knock's one of them. But like, I'm working on another startup at the moment, which is around, you know, a better approach to remote access, because it just feels like this is an area that just hasn't got enough attention, which is wild when you think about how fundamental it is, you know, and when you look at how many people are actually getting owned by their like remote access gateways, this is not a good situation. Yeah, just, just crazy. But yeah, agree with absolutely everything you've said there.

24:09I'm sure you do too, James.

24:10James Wilson: Yeah, absolutely. And I think just do a shameless self-promotion here. The thing you said there, Brad about the bug count not telling the story was exactly the solo pod that I did a week and a bit ago now because it it frustrates the heck out of me that the headline is you know this patch Tuesday has a thousand fixes it's like yes but how many of those are exploitable meanwhile these things on the edge that are supposed to keep these windows boxes and whatever else stuff safe all it takes is like a failure to sanitize input and then bang that's it RCE off you go so we we've got to have better telemetry and better metrics and reporting around this to be able to properly manage this risk going forward.

24:46Yeah. I mean, I think though, ultimately it is just what Brad said. You've got to get this stuff away from where people can touch it, especially unauthenticated people. Like even putting it behind some sort of like authentication challenge before you get network, a la knock knock or some sort of remote access. Like even if you do that, it still only gets you so far because, you know, we've seen hackers are able to look through cred dumps and actually get themselves authorized. You know, you've got to be a little bit more comprehensive. You've got to be looking at like using phishing resistant or using device, you know, device bound certificates where possible.

25:23Like, you know, there's so much you've got to do now, given that the fragility in these systems can now be trivially exploited. They've always been fragile, but now the effort required to do stuff with those things is, yeah, is a lot lower. I mean, I've been thinking a lot about this lately. And I think the real challenge we're going to run into is, okay, it will be well and good to put this stuff behind some controls. But as models, particularly like open weights models, get more and more sophisticated, the workflows and the TTPs for like, you know, AI assisted hacks, they're going to be more complicated than just hitting stuff on the edge.

26:05Right. And we're still going to be in a pretty bad spot. So yeah, I think ultimately throwing the worst enterprise software that we have into the bin is probably just going to be the way to do it. But like that's not a realistic solution. So I don't know. It could be an interesting few years. Lucky I've got this job, right? Because we get to talk about it. Now, look, let's talk through. Look, again, we're not going line by line over this. You know, models behaving badly has just become like, you know, there's a headline every day. we've seen an anthropic ai model sending a tip a homicide tip to the philadelphia police one thing i found interesting about this one though is the police department involved was like absolutely bucketed um absolutely bucketed uh anthropic over this even though the spam filter caught it and like no one ever saw it right but they're still like this is like as if they've never got a bad tip before right uh then you've got anthropic agents trying to fill out visa application forms on the State Department site.

27:03I mean, it didn't pay the application fee. It didn't actually lodge the applications, right? And I'm thinking, look, is it just trying to crawl these forms so it can learn more about the forms? You know, this must be something about training the things. And I spoke to you, James, and like, this is the joke I made earlier about how there's like a gym for everything. It turns out like there is a benchmark that where one of the things being measured in the benchmark is the ability to help people do things like apply for visas with the US State Department. So it's like, oh my God, I can't believe I built this machine for filling out visa forms and then entered it into a visa form filling competition and it filled in a visa form.

27:41Oh my God, Skynet, we're all doomed, right? I mean, that's basically what's happened here, right?

27:47James Wilson: It is exactly what's happened. And I sort of found this out by a hilarious accident because after you and I were talking about, is this scraping or not? I just said to Google, hey, are there any popular AI benchmarks that use filling out Visa forms as part of a test? And it came back and said, yes, the wildly popular OS world includes DS-160 Visa applications as one of its tests. And I'm like, are you serious? And so this has led me on a whole rabbit warren dive of what exactly are all these benchmarks? And for pretty much every example of models going rogue or being misaligned, which is nonsense there, neither, you can actually find the benchmark that probably describes the behavior.

28:31James Wilson: Like in the Visa case, yep, that was literally using OS World. This is the meme. This is the meme. Tell me you're alive. I'm alive. Oh, my God. Right? Like, go apply for a Visa. It applies for a Visa. Oh, my God. Like, what are these guys doing? Like, what are they? What are they? What? I know all of this just frustrates me that we're still talking about it because it hasn't left the realm of what did you expect, right? This is exactly what is going to happen. Yeah. Yeah. And we've got like OpenAI and Anthropic. They've both paused trainings because the models have gone rogue and whatever. Right.

29:06And then there was this incident where Albo, our prime minister, who I got to admit, I have a pretty dim view of our prime minister, came out and said that we are, you know, our health websites were hacked and stuff. And my immediate instinct was he's talking out of his hat. And it turned out he was indeed talking out of his hat because it looks, you know, we've got the headline here, doubts grow over claims OpenAI agent hacked. Australian medical website or whatever. But it looks like there was some scraping involved that might've been bypassing anti-bot stuff. And there were exploitation attempts thrown at various government websites by these agents.

29:42They were doing things they clearly shouldn't have been doing, but it sort of falls into two categories, which seems to be like getting around bot controls is one category. And then the second category is like, yeah, okay, that's actually, you're trying kind of like SSRF, like that seems a little bit dicey. That would be illegal if someone did it, not just in terms of service violation. But again, like all of it seems pretty predictable, pretty understandable, like, and, you know, it's not unexpected behavior. I guess that's the point you're making as well.

30:12James Wilson: Yeah, and it's, I guess the other element to throw into this is this is happening because models are primarily judged on these benchmark scores. If you look at every press release that comes out for a new model, it's here's the name of the model, here's the version number of it, here's its benchmark scores. And so, you know, you follow the incentives and of course the frontier models are throwing all this massive compute essentially at an agent saying, look, here's your benchmark. Stop at nothing to get the best score. You know, keep going relentlessly until you can complete this benchmark. Because if they get an 80 on the benchmark and some of the model gets a 90, they're dead in the water.

30:45That's going to move their bonds in the wrong direction by 50 bips, right? Like it's just, I mean, the whole thing is just nuts, man. Like This whole thing is just nuts. We've got Wikipedia discovered a whole bunch of really bad agentic activity as well. They were using the Wikimedia query service or whatever it is to just scrape all of Wikipedia. They were using, look, it turns out AI agents love having somewhere to write down notes as well because it was using their note takers. No coordination in that case. But, you know, either way, they were causing headaches for Wikipedia, which is not so good.

31:15We got reports from the New York Times talking about how OpenAI ignored employee warnings about safety. And look, the really funny thing about that New York Times piece is it reads like every single high growth technology sector ever, right? Like whether it was a social media website, whether it was Microsoft scaling up its cloud services, whatever. The people who were saying, whoa, whoa, slow down, got ignored. I mean, this is hardly new. Brad, I mean, I'm sure you had that, you know, history doesn't repeat, but it sure does rhyme, right? Yeah. I think that basically you've got the company is saying, you know, Anthropoc and OpenAI are out there saying this is so dangerous.

32:01Someone's got to stop us. Someone's got to slow us down. We can't help ourselves. But then when internal safety staff say the same thing, they get fired, you know? Yeah. Yeah. So they're clearly they're moving as fast as they can. And that's the only direction they want to go. nobody nobody wants to hear about how this is dangerous we need to slow down and then after the bad thing happens they get mad at you for not explaining it in a way they could understand because if you did properly document and say like this is dangerous we need to do it and then the bad thing happens despite you being overruled you still get in trouble for it they'll just say oh you didn't explain it to me in a way i could understand it so it's still your fault so you're annoying before and then now you're making me look bad after so nobody likes the like security voice wet blanket you know cautioning that this isn't safe i mean did we just read a page out of your diary because it sounds like this sounds a little personal uh brad i gotta say how many times have you been this person brad i've seen this movie before uh now meanwhile anthropic published a completely ridiculous blog post i think just overnight uh entitled ai models may be capable of self-replicating without access to a data center.

33:09And the idea, I think, is that you could run some of these models in a distributed way on, like, smart toasters or something. And it's like, to what end? I don't think I really understand. And, like, what is the risk here and how could it be achieved? Sort of reminds me, we were talking about this before we got going. It's like that scene in Indiana Jones where the guy is, like, with the swords, you know, waving the swords around and then Indiana Jones just shoots him. Like, I feel like the person who wrote this blog post is the guy with the swords at the moment, like trying to look scary and it just doesn't seem that scary.

33:39Like, what was your feeling reading this, James? Because it just, I mean, look, on one level, I think, hey, this is really cool that you've constructed like a theory here that shows that you could theoretically run a functioning, you know, AI model in a distributed way across a whole bunch of IoT devices. That's cool. But I'm like, I'm yet to understand the part where I'm supposed to be scared of that, right? Yeah, exactly.

34:02James Wilson: Like I read it first and I was like, that's interesting but ridiculous. And then I went back and did a detailed read and I went, no, this is actually even more ridiculous when you put it in that framing of what exactly am I afraid of here? It's like the most scary thing out of this could be a whole lot of Ralph Wiggum level intelligent models running on your smart TV producing 0.39 tokens per second. It's like, oh my God. Mrs. Hoover, I ate my paste. Yeah, exactly. it's just uh it's comically it's comically comical it's it's it's funny because it's like okay um i love that you get the time to sit down and construct these in theory scenarios and prove them out what a wonderful job that must be but there's nothing frightening or scary about this this is not the self-replication that you should be worried about yeah i think when i was reading it the the thing that i think they've done wrong is they're too mentally hooked on the idea of a monolith that has to self-replicate in its entirety from one thing to the next.

34:59And they're trying to figure out a way to make this relevant in a low resource world. And so they ended up with sharding across, you know, lots of Android TV boxes. But I think what would really happen in the real world, instead of thinking of like a self-replicating worm, it's going to be like an ant colony. And so it'll send out different skill level, different personas that have different tasks. and you could have a uh you know just flat storage with a bunch of md files but it's got a credential into some stolen uh you know just frontier model and so then it's like keeping itself going and it's sending out you know attack strings or things like that and writing code and executing and so i think that is actually really interesting so i want to see that paper that anthropic published but written by dave vitel because he's all about like ants and colonies and like how ants work and that sort of thing and so so i think there is some like really good work to do here but that paper didn't do it for me.

35:50No. And to be clear, just you mentioned Dave Itell, who's a very well-known person in security. He actually works at OpenAI at the moment, which is why I'm guessing you mentioned his name there. But look, let's look at actual harms from AI. We've got South Korean officials saying that they believe some recent hacks targeting banks were agentically driven. They were AI-enabled attacks. We've seen these jade puffer attacks targeting Azure and destroying cloud resources. So that's pretty interesting. I had a chat. We're teeing up an interview at the moment. I had a chat with a guy who worked for quite a while at the Ukrainian CERT and runs a cybersecurity company in Ukraine, does a lot of incident response and whatever.

36:35The stuff they are seeing in terms of the way Russians are using agents now to hit targets in Ukraine. It's causing real problems for the Ukrainians, right? Because they are just able to scale up and do everything at once. The crazy thing is though, you could actually see if you're doing incident response across a whole bunch of these different incidents, you could see where the Russians ran out of tokens via Open Router because everything simultaneously on 10 different targets just stops. It's dead for three hours and then someone plugs a credit card number or whatever into Open Router and bang, up it goes again and it all starts.

37:06So, you know, there's a lot happening at the moment in terms of real harms, not these theoretical self-replicating science fiction harms. And, you know, the public discourse around this stuff on like X, for example, is just so bizarre. Yeah, very, very science fiction. And meanwhile, we've seen GLM 5.3 released as well from that ZAI, right? GLM. And people are saying that GLM 5.3, which is an open weights model, you can download it if you would like to. I mean, it's kind of going to be pretty big and you're going to need a lot of hardware to run it. But it is an open weights model and people are saying it is kind of parity with mythos.

37:45James, you are our heavy user of AI models at Risky Business Media. Do you think that is accurate? Do you think GLM 5.3 is sort of a mythos parity model?

37:58James Wilson: My first instinct is to say unequivocally and emphatically, yes. But I have to qualify that and say I've got a huge information disparity here. I've used GLM 5.2 and 5.3 extensively. I've got access to Mythos Preview. The first time I tried to use it for a bug hunt thread that Daybreak had been churning away on, I stopped Daybreak and said, hey, agent, we're going to switch it now to using Mythos. And Mythos came back and said, I refuse to proceed. This is biological. Like, it's bug hunting, dude. Like, at least hit me with the rights guardrail. so I have to work off like you know chats I had with like Niels Provost and Carsten and what they saw with actual real access to these models but what I've seen personally out of GLM 5.3 is it's not just necessarily that the capability it's got it's that Claude gives up and you've got to really harness the heck out of Claude to make it actually progress towards finding a vulnerability and weaponizing it.

38:55James Wilson: GLM knows such trouble it is just relentless it will just go on its own and so I think that's one of the the big differences here it's not just the capabilities it's got but it's like it's it's it's it's innate desire to actually do the full vulnerability weaponization exploit creation end-to-end without complex harnessing and a lot of subject matter expertise having to be plugged into it as well yeah right i mean i think this is a interesting moment because the mythos moment was such a big deal uh everyone made a big deal out of it and the fact that you've got a Chinese overweight model that is even in the same conversation.

39:29And something like Mythos would suggest that they are going to get there, even if this isn't it, they're definitely going to get there. And this whole idea of like, why hasn't the world ended yet, James? The Chinese have got Mythos. You know, like it's, yeah, kind of cray-cray. Look, I'm just going to whiz through a bunch of these because we are, you know, time's getting a little tight. Apparently there is a new bill being floated in the US that would create an investigative body for AI-driven hacks. This sounds a little bit like the CSRB that Trump disbanded. I mean, they could have just kept the CSRB running and tasked it with looking at extending it to AI incidents.

40:07Take a lap, all the critics of the CSRB who managed to, they thought that it wasn't perfect because it didn't have subpoena powers. So let's criticize it until Trump kills it. And of course, it's been replaced with nothing. Well done. You guys did such a great job. That's sarcasm for anyone who can't tell. just in case you can't tell. What else have we got here? Oh, there's been a court ruling that says Trump actually did have the legal authority to designate Anthropic as a supply chain risk and ban it from like use by the Defence Department and also its contractors. So that's actually quite impactful.

40:45There's been one court that said he didn't have the authority, one now that has said he did. I'm guessing this is going to a higher court, perhaps even the Supreme Court, So it's interesting. And this is just a typical Trump thing where he does something and people say, well, hang on, he can't do that. And then it goes to court and sometimes they discover actually he could. Right. And that's been just a story that's been repeated over and over and over and over through both of his terms as president. uh now we've got one uh we want to get into a bit with you james which is uh apple is looking at the permissions it grants to ai models on its products like in this case uh mac os because you know meta releases muse of course and it goes around sort of hoovering up people's messages and not kind of telling them and like kind of you know violating their expectations of what it what it had access to.

41:37So now Apple's making some changes here. And, you know, Google Gemini is like looking to do similar things and get access to everything on a Mac computer. But you, as somebody who worked at Apple, you follow like podcasts and whatever and newsletters from like all about the sort of Apple ecosystem. This is a bit of a philosophical challenge at the moment for Apple because they are so privacy focused and whatever, but they also have a user base that loves to embrace new stuff like this. And those two things are in conflict and nobody quite knows what to do.

42:08James Wilson: Yeah, exactly. You know, it's basically the, it's the collision of, you know, macOS was based off Unix. It's always been an open platform. Along comes iOS. It's very closed and privacy insecure by default. There's been a decade of effort to have macOS catch up to that same level of protections. But what that's resulted in is macOS is now a great OS for the normies, shall we say it, It was very privacy-centric. But for power users, it's become increasingly more painful to use with all of these dialogues that pop up. Can this app access your downloads? Can this app access your documents from other things?

42:43James Wilson: Can this access? It's Windows Vista syndrome, basically. Yeah, exactly. And that pushed a lot of developers to just adopting this sort of coverall sandbox privilege, I would say, of full disk access, which Apple intended to be for backup apps, basically. But now just, you know, if you're an app that needs to do stuff on the disk, you request full disk access and users are actually guided through how to turn that on. Apple has said they're going to change how full disk access works, but not told us how they're going to change this. And this has made a lot of commentators and even me, I guess, to a degree nervous because this could be the pivot point where Apple either digs in and continues to make macOS great for humans, not great for agents.

43:25James Wilson: which is going to really push power users into a position of, well, like, what platform do I move to, though? You know, Apple's interests are no longer fundamentally where my interests are when I want a highly agentically driven. You know what this means? Linux on the desktop? 2027, year of Linux on the desktop. I think that's where this is. Have you guys been playing with, is it Omarchy? How do you pronounce that? This is, you know, the like AI first Linux host for your desktop. And so for people that get fed up with macOS, they move over to this thing. Got it running on a VM already. Yeah. There we go.

44:00And how is it?

44:02James Wilson: Man, look for some of the spend. You said it all. You said it all with that breath. How is it? Deep sigh. Yeah. It's making me challenge. It's challenging some deep held passions, Pat. It really is. Yeah. Yeah. Now, meanwhile, we've got, look, again, I'll just whiz through some of these because we are running low on time. BitGet. this is some sort of crypto concern, got hacked like 300 and something million dollars. They got owned via bugs into security appliances, apparently. Gee, I wonder what they could be. I'm guessing it's something like a SonicWall or a Fortinet or some absolute garbage.

44:39Staying with the crypto world, Ledger has confirmed that a Southeast Asian authorized reseller was selling hardware Trojan Ledger wallets. These have been linked to something like$87 million worth of crypto thefts. So, man, you know, they've got to sort out that supply chain. Why are they using resellers? I have no idea. That seems crazy. But, yeah, that is an interesting story in the crypto space at the moment. Kim Zeta has an excellent write-up as well. Brad, please. Yeah, getting a trusted computing base that can self-verify all its various components in a way that the end user can understand.

45:14Like, having come from a hardware world for a portion of my career, this is, like, an incredibly hard problem. and almost nobody has the patience to really pay attention is the box i bought like just what i wanted and nothing extra and and then the ui problems of how do you convey that and um it's it's such a hard problem so when i was reading this the the resellers like it's tempting to blame that but basically how can anyone produce a piece of hardware like this that's so small with like limited ui and then have confidence that it is what it should be and nothing more and so i don't I don't know how they fix this.

45:49Like, I don't know how you could build something that's supposed to be that important for managing, you know, huge amounts of money in your Bitcoin, you know, crypto wallet. It seems like a really tough problem to fix.

45:59James Wilson: And if you don't use resellers and you consolidate your logistics, that logistics get popped and then your customer list is leaked and then they go after that. Yeah. I mean, it's super easy as an adversary to see how you would attack this. And I do not know as a defender how you can effectively do it. And for the price point, they're selling these things at. Like, I just have no idea. This does not feel like it's going to work. We used to talk about supply chain interdiction being a thing that NSA does, right? And now it's something that we worry people who want to steal Bitcoin can do. So it's just interesting how every TTP that starts off in a SIGINT agency eventually trickles down to criminals, right?

46:31Trickle down TTPs. Yeah, crazy, crazy times. We've got this story here too about the trader-traitor group using a Trojan-like Terraform provider, which I thought was an interesting vector. James, you had a quick look at this one.

46:44James Wilson: Yeah, it's interesting just to see yet another modular ecosystem with third-party modules and indexes being used, but also it's inevitable. If you've got something that's modular, you allow third-party submissions to it, you're going to become a supply chain risk. We've got some new Spectre research, Spectre V2, which can leak a Linux root password hash in minutes. So that is interesting. We'll just link through to that in this week's show notes so people who are interested can take a look. There has been a data breach at Denmark's National Population Register. Details for something like 8.8 million people have been leaked, and that includes their equivalent of what the Americans would call a social security number.

47:23So that's crazy. ASOS, the British clothing company, someone got access somehow and pushed a push notification to all of their customers saying, you know, we've taken control and we demand. And I don't even know how serious the incident is, but it's pretty funny that they managed to drop a push notification to absolutely everyone because everybody was screen capping that one and putting it all over social. So it's a bit of a PR disaster. Google has halted bug bounty submissions for open source stuff because they were just getting too much AI slop. No surprises there. We're going to keep seeing more and more of this.

47:58And I think open source in particular has got to work out what to do about all of this. I actually just had a chat today with an open source maintainer, you know, whose full-time job is maintaining open source software. And they tried like plugging, you know, security at bug descriptions, like tried throwing Jev at them to see if it could chip away at that problem. And it's a garbage in garbage out issue. It did not help at all. So, you know, no one really knows what to do. We have seen also Also, some authorities seize tools used by the so-called Flax Typhoon APT crew in China. This is infrastructure maintained by a cyber contractor that works for the Chinese government.

48:38I don't really understand what the point of this seizure is in that they, you know, took down a couple of domains. I don't think it's going to be hard for them to stand up that infrastructure again. But, hey, maybe there was more to it. And we're just seeing what they're talking about. We've also seen someone hijack some ccTLD registries in order to, like whole country codes, in order to be able to redirect domains to obtain certificates for a bunch of services, including Google services. Not sure how useful that was going to be to the attackers, given that most browsers now, they will pin certificates for sites like Google.

49:13But it's a reminder that you can't take for granted that like a ccTLD is actually not owned, right, which is a pretty crazy thing. there's been a bug, there's a bug in something called GoBalance, which is like a load balancer for Tor Onion sites. So I'm guessing the FBI is having a wild time with that one. Again, we've linked through to it in this week's show notes, but that one, you just see it and go oof. And we're going to finish off this week with a fun story, which is Ukraine hit a data center, a Yandex data center near Moscow. And oh my God, the impact that had on social media was absolutely incredible.

49:53People were quote tweeting like White House tweets and saying, scroll this and see if you notice anything different. And all of the like MAGA bots, gone. All of the pro AFD bots in Germany, gone. Like all of the most caustic bot accounts apparently were being hosted by this data center. Brad, did you get a chance to actually wander through the empty halls of Twitter after this thing got blown up. It was amazing. So I saw the charts and I saw the anecdotal coverage on it. I saw some pushback on that analysis later saying that the charts of bot volume going down was really like crawlers and not the social media bots.

50:36And so I guess I'm not in those information spheres. So I didn't personally notice like anything changing. And so I don't, I'm actually really curious now what, what's really going on. Like, was it all those bots or not? Well, I'm telling you, man, like scrolling some of these, some of these tweets in like the Megasphere in particular, it was pretty, it was, it was pretty incredible because it was just, I mean, you're scrolling hundreds of replies on a White House tweet where the replies are basically GFY and not, not a single supportive message. It was just really weird. Right. And obviously this is not going to be a permanent effect, but it was noticeable.

51:10Definitely. We reached out to Texas Patriot 596273ABB, but they were unavailable for comment. They did not get back to us in time for reporting on this. But guys, we've got to wrap it up. We've got to wrap it up there. We have covered an awful lot today. So yeah, sorry that the last 10 minutes of that turned into basically me reading the news, but it's like that sometimes when we've been away for a while. Thanks to both of you for joining me. And actually, because you set up the, because you did the sponsor interview, James, for this week's show, let's set it up now. You spoke to, who was it? Was Greg, what's his name?

51:49Greg? Greg Bell. Greg Bell, that's right. He's a co-founder and chief? Chief strategy officer. Chief strategy officer of Corelight. And what's interesting is the conversation you had with him was really about that garbage in, garbage out issue that I was just referring to with the open source maintainer. And so really Corelight put out a white paper where they had looked at, you know, doing sort of automated SOC triage and whatever. But they certainly had looked at what is more important, the model or the data. And they sort of landed on it being the data in the end, right?

52:22James Wilson: Yeah, which is, you know, not surprising really, as you said, I think at the start of the show, because more model is generally, you know, more data is better for any given model. What was interesting is two things is they deliberately used different open source data formats. So you can sort of, you know, track the sort of, I guess, level of value that the different data types provide. But what was really interesting is just the, it's not like a linear increase of a little bit more data gets a little bit result. There's certain breakpoints where certain types of data being entered with certain structure just gives like a 3x improvement in how these models perform.

52:54James Wilson: So interesting results in terms of the, just the scale of the improvement. All right. Well, Brad Arkin, thank you very much for joining us. James Wilson, thank you as always. I had fun, Pat. Thanks for having me. Thanks, Pat. It's great to be back. And now we're going to throw to James's interview with Greg Bell. So here is Greg Bell talking through the research that went into Corelight's white paper, which looks at, you know, data versus model, which will win? Enjoy. In some ways, the thesis we're making in this white paper, which I encourage folks to download and read, it's only moderately technical and I hope it's interesting.

53:30The thesis is one we made when we founded the company 10 years ago, which is that security is evolving into and will become a highly data-centric activity, like most scientific inquiry, like education, financial analysis. At the time, that was a little bit eccentric, I think, and maybe a little ahead of its time. Now it's become a more relevant claim. And we got thinking about it again after a decade of building the business and supporting defenders, not at this last Black Hat, but the year before when it became really popular to enlist language models in capture the flag competitions. And we saw that those models were doing really well.

54:12They were doing better than some of the best humans and doing that work quickly. And this was a year ago when the models weren't nearly as good as they are now. And we began to wonder if we could use an agentic test harness comprised of models and skills to answer, to really explore the question of how important data is, the richness of data is, in getting to hard security problems.

54:38James Wilson: Interesting. And so I like the framing there around how this was the thesis of the company 10 years ago and you wanted to sort of retest it. But is it right to say that 10 years ago you were thinking about the importance of data in the hands of humans? Now you're thinking about the importance of it in the hands of AI agents that maybe running autonomously or assisting humans. How does that sort of change your view on that original company thesis and framing? Yeah, it was half of the right idea, I think. We were also thinking of it in terms of detection methodologies that could include machine learning, behavioral analysis.

55:12So we definitely thought about automation in some sense, not just human investigatory power. But certainly our earliest customers were really interested in threat hunting. They loved the rich detail in the data and the fact that it's interlinked and it's also open. It's an open standard. And so they use that to explore theses about security. And we supported those workflows very well. The difference now is that a lot of that work is amenable to amazing automation. And so what took a really skilled human and maybe a person of rare abilities, maybe hours to complete 10 years ago, can be automated pretty trivially by agentic systems now.

55:58And we can run those analyses in real time over thousands of alerts and get to the truth much faster. And it's a good thing we can because AI is bringing the same kind of automation to attacks. So defenders really need to use fire to fight fire.

56:14James Wilson: Yeah, 100%. So let's talk about the study itself. On one hand, not particularly surprising that you put more data and more context into a model, you'll get a higher quality or perhaps more valuable result. But talk me through the actual sort of the rigor and the setup here. What were you trying to prove? How did you go about running this test? We approached it with an open mind. We knew that there were many components that could go into an agentic SOC system. And so what we did is zero in on the question of whether the relative importance of the model versus granularity of data and quality of data.

56:51We did that by bending over backwards to build a fair infrastructure and one that was pretty robust against hallucination and against other pathologies that could affect the test setup. And we used a capture the flag exercise that we had delivered to hundreds of real humans since COVID, but we had never released that to the commercial internet. So that particular capture the flag, the questions and the answers aren't in any training data set. We also applied a schema, an OCSF schema that normalized the data. So the fact that our own data, Zeke data, and even CoreLite data was probably in the training data sets for the models didn't play as much of a role.

57:34So we tried to treat it like a real scientific experiment. And then what we did was we gave the system a real volt typhoon capture the flag exercise, 40 questions of increasing difficulty to answer that we knew a lot of humans had gone through. And we held one parameter constant while varying others. To begin with, we held the data available to the model constant, and we just used different frontier models. And we discovered that they're all pretty good. At the time when we tested a few months ago, they all came within about 5 % of each other in answering the capture the flag questions.

58:10James Wilson: Yeah. Okay. So you started off with keeping the amount of data that was available to the model static and constant, and then you just vary the model. But then my understanding is you kind of flipped that around and said, no, let's lock in the model, and then let's vary the amount of data available. So what sort of things did you start to learn when you flipped it to that model? Yeah, exactly. In phase two of the experiment, we held the model fix. We just chose one that happened to be the best at that moment in time. And then we varied the data available to it. And we're a company that transforms network traffic, raw packets, into well-structured, well-ordered data for a variety of purposes, including agentic purposes.

58:47So we exist at the intersection of networking and security. So what we did is provide the model with different forms of network-derived data, starting from NetFlow, which is widely available and it's cheap to acquire and it's better than nothing. but it's quite reduced. It's a very reduced summary of what happens on the network. Then through open source IDS logs, through open source firewall logs, we weren't baking off commercial competitors against Corelight. We didn't want to do that with the study. We really wanted to look at openly available data. And then we used Corelight data. We also took care to make sure that we had not configured the open source data sources in such a way that we hobbled them.

59:33And we even took care to make sure that our data was not configured optimally. So we didn't use all of our detections or all of our ML packages. And what we discovered when we did that was vast material differences in outcomes. So if we look at the capture the flag questions, the number that could be answered, with the net flow, that number might be in the mid-20%, which is a failing score. If you're doing an investigation, you're doing a capture the flag, and you can answer 20 % of the questions, you failed. all the way through 40, 50, 60 % for open source data sources. And with CoreLite data being in the mid-90s or higher.

1:00:08In fact, we had the ability to answer all the questions, but it was just a question of timeout. The models would timeout before getting to the answer. And so the answers there were, that result was validating of our original premise in founding the company, but a little bit surprising to me even. I didn't expect the results to be so stark, but they were indeed stark.

1:00:27James Wilson: Yeah, interesting. Interesting. And you mentioned that you were intentionally not baking off other commercial solutions and you were essentially taking a couple of open source varieties of data available plus CoreLite. What did you find or did you explore the idea of, is this an and or an or? Is it a case of this data source or this data source produces a result or do you actually always cumulatively get better results if you've got the IDS logs plus the NetFlow plus the CoreLite available as well? How much of a difference does that coverage make? Yeah, the combination is really important, simply because in general, you won't have all data sources available in all environments at all points in a topology.

1:01:05And for various reasons, logistical and financial. So in the real world, we do combine data sources. And in the research, this is now a platform we can use to answer any question for ourself, including questions about commercial competitors. And so we've done a number of experiments in blending data sources. and it is more powerful to blend. It's better to know that there is some signal, for instance, from NetFlow, often a distant part of the network that you're not monitoring with CoreLite. It's better to know there's something than to not have that visibility at all. So when you combine that little glimmer, that little, it's a little bit of smoke, not fire, with a higher quality of data you might be able to obtain in a small number of locations, then you can get a broader, more powerful narrative about what happened in an environment.

1:01:52James Wilson: Yeah, absolutely. Okay, so so far what I've taken from this conversation is you ran this experiment, you got a huge jump in improvement in the harnesses ability to complete these capture the flag challenges. These capture the flag challenges were based on scenarios that you know with a pretty high degree of confidence are not in the model. So we're not talking about a learned challenge here. but you know to push back a little bit how much of this is interesting right now at this point in time because there's a capability gap or there's a you know training gap you know you've essentially I guess leveraged the fact that there is a training gap here at some point in time that's going to close whether it be because a capability catches up where that training gap is no longer as important or perhaps synthetic training generates the right sort of data to create the pattern matching that might not be there.

1:02:42So I guess what I'm asking here is, this is interesting.

1:02:45James Wilson: Is it interesting right now? Is it interesting long-term? What's the so what that you take away from these results? Yeah. And that's a deep question that has broader implications for AI and human culture generally. I think we're all driving in the fog is the metaphor I like to use. And we don't know exactly, we're driving quickly in the fog. We don't know exactly what the future will bring in terms of AI development. So there may be some of what you describe. On the other hand, I think what we documented pretty persuasively was a cap, a hard cap in inference that is imposed by a given granularity of data.

1:03:24So if you start with very low quality data, what we showed is that you can throw a lot of it at the model, but it's almost a mathematical fact that the model can't answer questions when the answer depends on fields and content that's not available in the underlying data. A lot of people find that hard to accept. They think that there must be some way to turn a lot of straw into gold. If you just take a lot of straw and give it to a fantastic model, you will get gold. And there may be some truth in the fact that the models will improve in some of the ways that you mentioned, but I still think there will be a hard cap on inference set by data quality.

1:04:03And the better we can make the raw material going into agentic systems today, foundationally, the better we can, and this gets to your question of what's the so what of all this. The so what is that data should be a first class citizen. When you're thinking about architecting an agentic SOC, obviously you'll be thinking about skills, about your human capability as well, about orchestration, about hallucination, about metrics, about costs, so many factors will go into that. But data needs to be a first-class citizen in that discussion, because without fantastic data going in, the results of automation will be extremely suboptimal.

1:04:43You get an F when you want an A. And in fact, worse than that, you may amplify the impact of noise and just produce an enormous firehose of false positives or of difficult-to-resolve ambiguities when you could have done something much more skillful and effective in the SOC.

1:05:00James Wilson: Thank you so much for dropping by for this chat. It's been really interesting to look into the results that you got from this study. And I think I would agree with you that this is, it's not one of these white papers that's incredibly dense and hard to read. It's, I think anyone with even just a mild amount of technical knowledge and keen interest in this space would find this quite interesting to see both how you went about running this test and the interesting results that you got. But mate, thank you so much for dropping by for a chat. This has been a lot of fun. It was a pleasure. Thanks so much.

1:05:29That was James Wilson chatting with Greg Bell from Coalite there. Big thanks to him for that. And if you need a network sensor, you know, to find funny stuff on your network, then obviously there's no better choice than Coalite. And big thanks to them for being this week's sponsor. But that is it for this week's show. I do hope you enjoyed it. I'll be back soon with more security news and analysis. But until then, I've been Patrick Gray. Thanks for listening.

1:06:01Thank you.