Risky Business #858 -- The $6.3b laptop

0:05Patrick Gray: Hi everyone and welcome to Risky Business. My name's Patrick Gray. We'll be chatting through the week's security news in just a moment with our regular co-host Mr. James Wilson and also joining us this week is Ciaran Martin, the former head of the NCSC who these days is an Oxford professor and does some work with SANS and stuff as well. So he's going to be our third chair this week, joining us for the news segment in just a moment. This week's show is brought to you by Run Zero, and Run Zero's Todd Beardsley has popped along for a sponsor interview in which he shares a magnificent idea with the audience, which is to harness the post-quantum cryptography hype and actually use it to do some useful things like deprecate EOL devices that you might find in your network, on the grounds that they're not quantum safe.

0:57Patrick Gray: It's a good idea, actually, because as you'll hear, you can use the fears around some of this quantum stuff, which probably aren't going to really significantly materialize to actually deal with some problems that are materializing. So that's a fun chat, and it is coming up after this week's news, which starts now. And we're going to start off by talking about this massive Binance hack that happened last week. I mean, Amberley, who works with us, said it best when this story was breaking, which is it reminded her and I of being in a newsroom when big stuff was happening, right? Because initially it was$3 billion and then it was$4 and then it was$5 and then it was$6.3 and$3 billion frozen and whatever.

1:39Patrick Gray: Absolutely epic hack. Binance, of course, has said that it's going to cover, it's going to be able to cover the losses, which is, I think they're a 90-something billion dollar organization. so they should be able to cover it even if they have to raise money or whatever but it's still going to be a pretty significant hit. But yeah, just wow. Now since then, the FBI and a bunch of CTI companies have attributed the activity to North Korea which I suppose is not so surprising. What is surprising though, and I want to get your thoughts initially on this, James, is it didn't look like a particularly complicated hack, at least to me.

2:15Patrick Gray: It seems like they initially got access to one developer's laptop via a Trojan version of Homebrew or something. And they didn't progress the attack from there, but they learned some interesting things that they later used in the attack. Where it looks like they ultimately, the way they ultimately did this was they got into a second engineer's laptop by offering the engineer a job. you know, typical North Korean job interview thing where they're like, hey, if you want this job, you've got to download and run this project and complete the tasks in it. You know, engineer fell for it. And then things get a bit interesting because the write-up that Binance has done is very, very detailed in some aspects.

2:58Patrick Gray: Like, here's excerpts from the script that it ran on the guy's laptop and whatever. And then it glosses over other stuff like, well, then it, you know, leveraged an internal AI service to get access to XYZ. Anyway, moving on to the next part, you're thinking you skipped a bit there.

3:15James Wilson: Yes, I would like to know a little bit more about that elusive step two that you mentioned there.

3:20Patrick Gray: Yes, step two, question mark, step three, profit, is really like what it reminded me of, yes.

3:24James Wilson: Yeah, 100%. So look, the specifics, yeah, like you said, in June, first developer laptop was compromised with just a typo squad in Homebrew. that didn't get them anything other than what looks like a whole bunch of intel because by the time then in September when they compromised that second engineer with the infectious interview they sort of knew exactly who to go looking for and how to sort of backtrace through internal systems to find this original engineer because that engineer that they compromised in June had seemingly access to container signing and deployment privileges which is ultimately what they used for the attack but yes the big unknown here is how did they go from the second compromise of the second engineer in September with the Contagious Interview, classic DPRK tradecraft, to then see me.

4:10James Wilson: There's just this line where they say an internal AI tool was used to assume the identity of engineer A. Like it's not even clear whether they use that to get back in their laptop. Was that the staging point? Was it just creds?

4:22Patrick Gray: And if I put on my editor hat, it's passive grammar as well, which is deeply annoying. But no, it's absolutely right where they're like, oh, you know, there was an internal AI tool and next thing you know, $6 billion is gone and you're thinking, doesn't sound like a very well-written tool.

4:37James Wilson: Or an exceptionally well-written tool. I mean, imagine the productivity gains that must be there if you can quickly pivot and exfiltrate$6 billion.

4:45Patrick Gray: Yeah, so I think something is deeply embarrassing there. I do wonder why you need$6.3 billion in a hot wallet, though. That does seem like kind of a lot.

4:55James Wilson: Well, there's hot wallets and there's hot wallets. I mean, that's just extreme. I mean, look, outside of the question of what was this AI tool and how glorious is it, there is also those other questions like the blast radius here. Even if this was one, two, three laptops compromised with a bit of a lateral move internally, how does that account for the blast radius of$6.3 billion of these wallets and being able to seemingly exfiltrate this with just the ability to publish a new container?

5:23Patrick Gray: I like that they detected this because large amounts of money started moving around. And that tends to suggest like, yeah. Anyway, look, I think it's interesting that it's developer laptops as well because they said the EDR didn't pick it up. And, you know, developer laptops are like notoriously the most difficult laptops to try to instrument for any kind of EDR because they're always running weird stuff, right? So, yeah. But you would think they would have better controls. You just would think Binance, like what was that other Bybit, right? But Bybit, you know, you would...

5:56James Wilson: That was exquisite in terms of his tradecraft compared to this.

5:59Patrick Gray: It was, but they had also, like, I think the consensus was that they had also messed up and didn't have appropriate controls. And you would think, oh, well, like Coinbase and Binance, like, they're the big boys. You know what I mean? They're going to have better controls. But lol, nope. Turns out, no. Turns out, no. Now, Kieran, you are a bit of a North Korea watcher when it comes to all of this. I mean, what's your take on this? This is like, I don't think this is going to... I think Trump came out and said like, oh, no, I don't know that the North Koreans would do this, which is a lot of fun.

6:29Patrick Gray: But like, I mean, I can't imagine they're going to experience any sort of blowback from this. They're already incredibly isolated. I don't think crypto theft is going to go political, even at this sort of dollar figures. Like, what's your take on this and what it means and what could flow on from this? Firstly, thanks so much for having me. Really brilliant to be on. Love the show. Second, I think you're right. there's not going to be any political blowback their experience they've got an mo for this it's worked repeatedly there's not much to add to what you guys have been saying except let's go a bit big picture it's been quite a good week for traditional banking i mean you're when you're talking about your controls and james is absolutely right how on earth do you get from a handful of developer laptop compromises to 6.3 billion dollars in 20 minutes and try that out in a bank Try that out in a mainstream household name bank and you won't get that.

7:22You can compromise as many developer laptops as you want. But the way money moves around traditional banks means that alarms go off all over the place. They've got such tight regulation. They've had it for decades, but it's been tightened since the great crash. And I think when you look at the stats, I think there's a problem. There is a real problem with crypto. So obviously measurements can change and they're contentious, but roughly speaking since 2016. There have been between 30 and 40,$100 million plus heists from crypto exchanges. The vast majority of them North Korea, not all of them, but the vast majority of them.

7:57They've been two north of a billion by about. And now at Binance, depending on how you measure it, the number of traditional banks anywhere in the world who've been hit for that amount is either zero or one. And you go back to Bangladesh Bank, which was a near miss in Feb 2016. So it's more than 10 years ago. That's$81 million. and I think it does matter because I mean the politics of this are concerning I mean we can look at the hack and there are extraordinary things in it and James and you Pat have gone through them you know the AI tool the job interview and so on but I mean people sometimes I mean a lot of people you know don't like cryptocurrency and they sort of you know have a fairly scornful attitude to people who invest in crypto and say well you deserve what you get but this does matter because this is funding weapons procurement.

8:46We know that from the UN, from two UN reports in 2019 and 2024. Why do the North Korean do this? Why do they specialize in it? Why do they train people to look at the most vulnerable points in the crypto infrastructure? Why do they exercise all that strategic patience? Because they want to buy ICBMs and they want to pay for nukes. So this really, really matters.

9:06Patrick Gray: This is what's really frustrating, right, is because the TTPs are often really admirable and exquisite, but you can't got to hand it to them, you know? Like, this is the problem because it's North Korea where the guy who leads the country occasionally disassembles people with anti-aircraft guns because he doesn't like the way they looked at him and stuff, right? So, you know, not a well-run country, not a country that we should have positive feelings towards. But look, it's interesting that you mentioned the hack of Bangladesh's central bank via the SWIFT network. Someone hacked a SWIFT message.

9:38Patrick Gray: It was North Koreans. They hacked a swift messaging terminal at the Bangladeshi central bank, transferred a bunch of money out. They did manage to get some of that out, but not all of it. They had to take heavy losses on laundering it through casinos in Macau. I think the Chinese government's cracked down on those casinos as a money laundering vector at this point as well. But the point is, I knew people who worked in banking. I knew somebody who worked for a large investment bank that helps some countries run their investment banks. And the changes that were made to SWIFT, stuff actually happened to address that threat.

10:13Patrick Gray: And we did see more attempts at that sort of theft, but it petered out because it's just too hard. Too much of the money is recoverable. And, you know, too many changes were made to the actual, you know, Swift network itself to make that sort of stuff unworkable. But in crypto, it just doesn't seem to be getting better. I think that's exactly right. And if you study, and as you say, I'm a bit of a North Korea watcher. If you study their tradecraft, if you study their targeting over the last decade or so, they've kind of given up on banks. It's just it's not worth it. They can't get out the money anymore.

10:47And I think they got four out of the 35 transactions through from the Swift heist. And then I think they misspelt foundation as fundacion or something like that. And it triggered alerts and so on. So it's a pretty expensive typo back then. But to get back to crypto, I mean, there is a structural problem here. And I say this sort of half jokingly because you have to try and have some levity in these matters. But it's, as you say, it's really, really serious. is authorities all around the world have been telling people for years, you know, don't buy cocaine because it's immoral, you're funding human trafficking and misery and exploitation.

11:18I wonder at what point are people going to say, and I'm half joking, but only half joking, don't invest in crypto because you're buying ICBMs for Kim Jong-un. And it really is something where you mentioned political blowback. I think North Korea for these purposes are untouchable. There's just no way. They're not going to go on holiday to the US or Cyprus or any of these places like the Russian ransomware actors. They're state operatives. It's a prestigious job.

11:44Patrick Gray: It's not like the North Korean government gives a crap if some of these guys get vanned when they're in Malaysia or something. Well, exactly. And what are you going to do to the regime? Are you going to sanction them a bit more because it's pretty tight already? So I think you're going to have to look at improving crypto security, but there's no sign of that. No, there isn't. And I mean, once the money's gone, it's gone. sort of like old school, you know, Mac computers where people would actually throw stuff into the bin and hit delete and then go, oh, hang on, I want to undelete that. You can't, you can't, it's gone.

12:15Patrick Gray: Irreversible. Yeah, and the hack looks pretty basic as you've both illuminated. The money laundering experts seem pretty impressed by the way the money was laundered quickly. Yeah, yeah. So that seems to be the very careful bit of it. Yeah, like the on-chain stuff was pretty cool. They cycled it through pretty rapidly, did some pretty cool obfuscation. Because this is their job, right? Like they are good at this. Now it's, you know, God knows where it's going to wind up. But there's, you know, James has actually done some really great work on looking at how that whole laundering ecosystem works.

12:47Patrick Gray: But yeah, I think, you know, crypto, I've always said it seems mostly used for illicit purposes. Ransomware payments, crime, drugs, tax evasion, sanctions evasion. I don't get the point of it. it's an extremely volatile ecosystem as well, which makes it not so useful as a store of value. So as an investor, I don't get it either. It holds zero appeal, except as like a risk on if it's ripping, like maybe getting on the momentum, but like it doesn't do anything. You know what I mean? Like at least you can turn gold into jewelry, right? So I just, I don't get it. Apart from its highly questionable investment value, its fundamental worthlessness and its terrible security practices and its complete funding of illicit crime all over the world.

13:32There's a lot to be said for it. Yeah, aside from that, it's great.

13:34Patrick Gray: Yeah, so we're going to change the name of this podcast to the Diamond Hands Crypto to the Moon podcast. But look, look, moving on, because we've got to move on. We've got plenty to get through this week. There has actually been a crypto exchange in Brazil had an incident as well where they lost$46 million,$6.5 million frozen. But what's interesting here is that 46 mil is just quaint now. Like, oh, 46 mil, minor incident, right? Like, that's a lot of money. James, it's amazing how desensitized we've become to these sort of things, right?

14:09James Wilson: It truly is. Like, the numbers are one thing, but what really gets me in the sort of the desensitization aspect is when, even when, like, you know, the Binance was at six-something billion dollars in the news, CZ, the former CEO, and then even the current CEO both came out and said, that's cool, we can cover that. Yeah.

14:28Patrick Gray: I mean, let's wait and see what happens, right? Like, let's see if they have any liquidity issues. Because it still seems like a lot of money. Like, I don't know quite how they're going to do that. I think they did. Did they freeze withdrawals for a time?

14:40James Wilson: Briefly, they have also mentioned in Binance's case that they're dipping into things like futures insurance, et cetera. So there's signals that it's hurting them more than just, eh, we've got the cash for this. Well, exactly.

14:51Patrick Gray: But that's, of course, the comms around it are going to be, oh, it's fine. Don't worry. Because they don't want the equivalent of a bank run. But we've also seen an attack against Polymarket. This was cool, actually. Attacker managed to drop a script onto Polymarket via, like, owning a part of their CDN. Like, they snuck a malicious script into something that was being cached by a CDN. Is that about right?

15:13James Wilson: It's about right. The thing that's a bit blurry here is whose CDN was compromised. So Polymarket was using what's called an RUM or real user monitoring tool, a JavaScript package that is essentially a little bit of code that runs on the user's browser, records their clicks and their sort of movement throughout a website so that they can look at that and see where's the friction and whatever else. You understand the patterns. but what happened here was either the um either the polymarket cdn was compromised in a bad javascript bundle was loaded there for this debug bear rum or the other way around that in fact in the debug bear cdn a new package version was dropped i think it was actually that more so from what i can make from this this but either way the malware that was dropped was so specifically tuned and written for specifically for polymarket that whether it was debug bear that was initially compromised or polymarket this is absolutely a hundred percent a very targeted specific attack against polymarket because then what this script did was it went straight into loading up session tokens for polymarket how to act on behalf of that user closing out trades draining funds and just i mean it's like a very very surgical piece of work and they only got 2.7 mil so i reckon

16:28Patrick Gray: these guys should get the 6.3 billion and the people who socially engineered an engineer they should get the 2.7. You know what I mean?

16:37James Wilson: Yeah, it goes in like a sweepstakes fund and we re-divide these profits based on the level of tradecraft.

16:43Patrick Gray: This is like hacker, who is it? Hacker communism? It's a hacker redistribution. I think, you know, we need to set up a committee that determines who gets the illicit loot, basically. Chainalysis had an incident as well. Of course, they're a blockchain forensics company. They said that it was like an agentic attack. I mean, look, we're running down the clock on how long it's going to be before that stops being noteworthy. But apparently it looks like not such a big deal incident.

17:12James Wilson: Is a big deal, not a big deal? Hard to tell. I mean, look, I will say chain analysis, their analysis of on-chain events and tracing these funds is normally so exquisitely good. It's kind of disappointing to see this press release from them using terms like, you know based on this evidence uh you know we we believe this was a uh what's the offensive agentic techniques were used by these attackers but then they don't say what that means what did you see um so yeah so again it's like that's the new equivalent of it well it was a very sophisticated actor um but as to whether it's a big deal look what did they get they got user ids full name full names email addresses and organizational role labels for customers off chain analysis

17:55Patrick Gray: I mean, that's going to be pretty handy for phishing, right?

17:57James Wilson: Very handy for phishing, especially if what you're interested in is what do they now know about what they could have been looking at on the blockchain. So you phish them with, hey, your report's ready, or we've updated this report with a critical material update. Please click here to log in. And that's an easy way to phish their way into whatever chain analysis has provided for them.

18:15Patrick Gray: Yeah, now just closing out our crypto section, Trezor has warned of implanted hardware devices as well. We've seen a few stories. What's the other wallet maker that's been caught up in this?

18:28James Wilson: Oh, the one that ledger. Ledger. Ledger.

18:31Patrick Gray: Yeah. Ledger's had similar issues. I mean, sometimes it's a dodgy reseller or sometimes it's like someone has bought a legit reseller and made them dodgy and whatever. But why these guys have channel partners, I will never be able to figure that out. But you've got a great comment on this one in our news system. And you said, well, you can't keep your crypto in an exchange or in a software wallet or in a hardware wallet. So, you know, what's left. What's really funny about that, though, is that we now have an agent that goes through and replies to our comments if it thinks we've written something incorrect.

19:04Patrick Gray: And the agent, the risky business agent, has written in, Individual multi-sig is established practice, not a verified emerging consensus. Independent seeds, trustworthy distribution channels, separate backups and recovery of the wallet configuration all matter. A supply chain incident does not establish that safe custody is impossible. So we've offended the clanker. And honestly, mate, I think I'm just going to keep my money in the bank.

19:30James Wilson: Well, what's actually really funny about this was that agent correction actually came from a conversation I was having with, I think it was with OpenAI's dot about this story. And I said, look, I don't get it. I can't keep it in an exchange anymore. I can't keep it in a hardware wallet. What do I do? Do I cash it out and stick it under my mattress? And then it churned and thought and thought and thought. And then it came back and said, hmm, the mattress ecosystem is a shorter supply chain. But I don't think we can compare that to the software issues here. So it actually took you seriously.

19:58Patrick Gray: The mattress ecosystem. Yes. Gotta love that. Gotta love that. All right. Now, we're going to move on to a different topic now. Completely different topic. And there's been another wave of attacks against supermarket freezers. This time, another attack affecting supermarkets in Europe. apparently these ones were due to get a patch like today uh which is unfortunate although i think if your solution to these attacks against an iot freezer controller is to just apply the vendor patch instead of applying some sort of access control to them you're going to get owned again anyway right uh because it's i guarantee you there's more bugs in that stuff uh but this is i i hate stories like this when it's about like wasting food i guess it's because i'm like you know younger gen x and i was raised by parents telling me never to waste food you know what i mean like that was a big thing uh for people of my age and i guess because you know i'm my dad grew up on rations after world war ii you know so it's um just a generational thing but i like i i actually find it like kind of upsetting um but kieran i know you wanted to weigh in on this one um you know what do you make of all of these attacks against against uh freezers i think we had a guest recently, I think it was Brad Arkin who said his read was it was either Russia or kids, basically, is the two threat actors he could consider being behind this.

21:17I have no idea. And it's so unusual. We talked about the finance hack and the FBI have attributed it to North Korea really, really quickly by the standards of attributions. I mean, you don't normally come to a full formal attribution within a few days. By contrast, I'm trying to think back. So I was appointed head of what became the National Cybersecurity Center at the end of 2013. So you're getting on for 13 years. And with a handful of exceptions in the early days, I cannot recall a case in that entire period where there is literally no real idea in the public domain as to who's behind this.

21:56Of course, you can speculate. There's no technical evidence that we know of. But and the obvious, the obvious suspects and things like this would tend to be the criminal groups but it's not them because there's no money being made they're destroying the thing in advance rather than trying to extort money to prevent this type of outcome is it a script kitty or whatever well maybe but they don't appear to be looking for the glory that you normally associate with that type of actor so then you turn to nation states but i mean nation states try to achieve some sort of impact and send a signal what signal is being sent and i I will use the possibly Russia word, but in the very most speculative sense, because as somebody who used to run a government agency, there's not a shred of evidence so far.

22:42And you want to be very careful. And I think governments have to be very careful before they start making such accusations, because we just don't we just don't have a clue. And that's really unusual. And I think that it also speaks to something. And I think you rightly raised that there's something psychologically impactful about this. And I say this as a European and talking to lots of other Europeans. I don't know whether it's a generational thing for Gen Xers or whether it's a more wider thing across different cohorts. But there's a sense to which, even though, you know, there's no food shortages anywhere in Europe as a result of these attacks.

23:16But there is this deeply destabilizing aspect to this because people see what's possible. And they didn't really think because it's not happened before. They really didn't think that the supermarket freezers could just be fried and melted by some really basic attack and so on. So I think a lot of people are really quite shaken by it and they're thinking, well, what else happens? And most, I mean, in the UK, for example, which was hit relatively lightly by this, the UK government has got into a bit of controversy because it's not very British. But in the past week, the UK government has basically advised people to start stockpiling.

23:51and it's the first time in my lifetime that the UK government started saying that and last week there was a fair bit of ridicule about this we don't do this we're British etc and there was a bit of debate in the other direction but after this sort of thing people are thinking okay maybe we should be thinking a bit more about national and social resilience because this sort of thing can happen too easily the final point I'd make it comes back to a little bit of the discussion we were having about that sort of the quaint Brazil crypto hack versus the but and the technically very adept polymarket hack versus the massive but pretty clunky Binance hack.

24:26I think as we start worrying about AI-powered attacks and so forth, which we should, we have to remember that there aren't prizes for sophistication. That actually, it really is about impact that we're thinking about. Or as Adam used to put it, it's not dumb if it works, right? Exactly. So, you know, one of the things people will use AI for is just to have really sort of random and difficult impacts. And I think the very final point I'll make about what does this show, I've often wondered if Vault Typhoon were to become realized, right? And I'm not saying this has anything to do with China. It almost certainly hasn't got anything to do with China.

25:09But what would Vault Typhoon feel like if it happened? It would be hundreds of things like this at the same time. you know, little attacks, but cumulatively very, very disruptive, very, very psychologically destabilizing and very expensive. So it is a bit of a shock. I don't like talking about it either, but it's a big, big impact on the European psyche.

25:30Patrick Gray: That's what makes me think that it's not beyond the realm of possibility that Russia is behind this, right? And I've got nothing to suggest that it is, but it is the sort of thing that Tom Uren, our colleague here and author of the Seriously Risky Business podcast, is the sort of thing he would describe as cyber imbuggance, right it's just like throwing sand in the gears is kind of a russian tactic wherever possible just throw sand in the gears do stuff that's upsetting on and on and on and it seems pointless but i think the point is that it's uh it's cumulative the the effects of this sort of thing are cumulative right yeah the only time i can remember being this baffled um was when i was in government and it was tv saint mond and we now think that that was russia and we we don't really know why they pretended to be Islamic State and take out a French overseas TV station, but they may have been practicing, they may have been doing proof of concept, they may have wanted to intimidate the French a bit, but it looks pretty certain that it was them.

26:23I'm not going to say this was Russia because there's no evidence, but there's certainly a possibility that it's a cyber imbuggance by a nation state. I mean, it's probably the theory right now.

26:37Patrick Gray: We're saying the same thing, which is there's nothing to indicate that it is Russia, but you can't rule it out because it is is the sort of thing that Russia would do. Now, look, speaking of weird stuff that could be state-sponsored, but maybe isn't, Amadeus, which is the global reservation system for airlines, turns out there was a threat actor in there for literally years just using stolen creds, stealing thousands upon thousands of itineraries. So we've seen a whole bunch of disclosures from various airlines, Qantas, Japan Airlines, ANA, so on and so on. What's interesting about Amadeus, We've seen one CTI company, I read their blog, and they said, well, you know, this looks like the access patterns look like it's a, you know, crime group or initial access broker.

27:19Patrick Gray: I don't know what they're basing that on, but I can say 100 % that these reservation systems are prime targets for foreign intelligence services. Obviously, companies like Amadeus share that data or are forced to share some of that data with, you know, friendly agencies. but you know agencies that can't get legit access to this stuff are just going to go and steal it so we've got no idea who's behind this one but you know it's we see these systems attacked all the time like booking.com as well it's another one of those ones where you get criminals in there and also nation states and all sorts of stuff James what any thoughts on this one?

27:58James Wilson: Look I just the tell for me here was they'd been in there so long but had never seen to show any interest in the financial or the transactional end of these platforms. And so for me, that was just a further hint that it's more about the data as opposed to thinking about this as a monetization vector.

28:16Patrick Gray: Yeah. Kieran, any vibes off this one? I would agree. Going back, so I'm not, again, I'll try to be as careful as we were in a Russia discussion, but going back through the years, countries like iran have been very interested in passenger data and in who's traveling where and so forth and got absolutely no idea whether this is them or not but you can see why i mean if you're looking at dissidents you're looking at political opponents you're looking at whatever it is and i'm very very struck by james's point that if you're there for ages and you haven't tried to monetize this well what are you doing yeah yeah it seems a little lot seems a little sus we'd say.

28:58Patrick Gray: Now look, soon we're going to have to collect the whole set, James, because we saw a massive data breach from Oracle PeopleSoft instance belonging to the FBI. It's amazing how quickly that one dropped out of the headlines because that is scandalous that that was able to happen. Very sensitive data stolen. And now it looks like a hacktivist group calling itself the Department of Peace. I wonder if they're real or if they're a fake hacktivist group. They've pinched a bunch of DHS data, a bunch of DHS records on stuff and have, what have they, they've given it to distributed denial of secrets. Yeah.

29:34Patrick Gray: I mean, I don't really understand. You know, look, maybe there's some public benefit information in there. I mean, I'd have to wait to see it.

29:42James Wilson: But this was my question is like, for what end, right? So why are there, I was asking, I was trying to research this morning, has this, you know, Department of Peace talked about what they want to achieve with this? what they're aiming to do. And there's nothing other than just saying, oh, this is retaliation for ISIS expansion of powers. I don't see how leaking this is really going to shift the dial or achieve anything meaningful for you there.

30:07Patrick Gray: Yeah, I mean, I understand why Americans are a bit anxious about DHS. And, you know, there's been some reporting about the way Homeland Security investigations is being turned against sort of protest movements and investigating them like they're terrorists and stuff. And it's like it is legitimately concerning. but you know pinching this sort of data and indeed hosting this sort of data feels like you're kind of asking for trouble I mean we're recording this right now as they're counting votes in the United States midterms I have a feeling that is the way to resolve power abuses from organizations like HSI I think if the Democrats take the Senate and the House which they are widely expected to do we are going to see like congressional inquiries into DHS's actions from here to eternity, basically.

30:51Patrick Gray: And I think that's going to do a lot more to move the needle. The power of the ballot box, people. There's a lot to be said for it. Now, Google has coined a new term, LLM jacking. Now, this is the stuff we've been talking about on the show for the last few weeks about how, you know, threat actors are basically stealing tokens, you know, stealing API keys that get them access to tokens and racking up huge bills in people's accounts. I think LLM jacking all well and good but in my mind it has to be clanker jack because that's just like much funnier um but you know there's not much new here that we haven't spoken about but it just it's just interesting seeing that google threat intelligence group or whatever like putting out a report that broadly lines up in in terms of consensus with what our vibes were telling us over the last few

31:40James Wilson: weeks yeah 100 uh it's it's just a really good confirmation that we were we were right in a sense to say, look, follow the money. If you're going to monetize a stolen API key, first of all, sell access to it through these myriad places that are selling cheap inference, but then almost like double dip because then take the transcripts that you're getting out of this cheap resale of LLM access and sell those transcripts off to the model trainers that want it to steal. It's a great business model, if only it weren't illegal. But yeah, I'm no sooner calling it LLM jacking than I would start calling it super intelligence.

32:15James Wilson: It's AI and clanker jacking as far as I'm concerned.

32:19Patrick Gray: Yeah, that's right. Northflank as well. A company called Northflank has had some sort of breach where that looks like you would cluster it in this sort of group of activity. You're giving them an attaboy for the way that they've talked about this because they're not trying to hype it. They're not trying to say, oh my God, it's an incredibly sophisticated distillation thing. They're like, yeah, we had some API keys stolen and customers got charges against them and we've dealt with it.

32:44James Wilson: Yeah, it's kind of sad that the attaboy is good job you didn't lie in your reporting, but that's kind of what it comes down to.

32:51Patrick Gray: Welcome to the media, pal. This is what, welcome to journalism. My God, they didn't lie. We got a real interesting, just look, and this is a side conversation. We're not going to get bogged into it, but there is this really interesting tweet going around that's been getting a lot of attention where someone asked GPT-6 Astra to directly edit the model weights. of QEN 3.5 to improve its ability to do certain mathematical operations. And it did. It actually tuned up the weights a little bit and made it a much better model. People have since replicated this. The reason I don't want to get bogged down in it right now is because there's other people saying, well, there's only so far you can go with this.

33:32Patrick Gray: It's not as significant as people are saying. But what I find really funny about this is the idea that you could basically take an open weights model, get a frontier model and say, make better. Make no mistakes. turn this into a frontier model. And then like what happens to distillation discourse if that happens, right? Like, and you got to imagine that eventually an LLM will be able to build you an LLM. And what then, right?

33:58James Wilson: Yeah, what then? That's what I really take away from this is like take a step back from all the hype in the tweets and just consider for a moment that we went from machine learning that was relatively understandable concepts like gradient descent, et cetera. Then that turned into larger language models. And when that happened, we entered this sort of, I guess, collective belief that, oh, these things are unintrospectable. They're this great big black box of model weights. We don't even know how they work, but it's amazing. But I feel like that assertion of we don't know how they work is actually eroding quite quickly.

34:30James Wilson: And that's the actual materially interesting thing I find here. Whether you look at obliteration, where it's like selective working around of guardrails in a model, that says they're not a black box anymore. You can understand it. But now whether this is some secret sauce built into a model to help and understand how to build other models, whether it's just better mechanistic understanding of how these weights interact, I don't know. But it's super fascinating that we're continuing this trajectory of not only is a large language model understandable and modifiable, it can actually be created within a control environment and almost given like gain of function, to use a sort of similar virology term.

35:05Patrick Gray: Now, just look, before we move on from our AI section this week, I want to plug a podcast I did in the features interview section where I spoke with a guy called Nazar Timoshek, who runs a Ukraine-based cybersecurity consultancy. 50 % in Ukraine, sort of 50 % in the US. So I interviewed him from Lviv. It was crazy. I could hear the air raid sirens at the end of the interview, which was just nuts. but yeah he walked us through how Russian intelligence agencies are now hitting Ukraine and it was a wild chat actually because he's like they're everywhere all at once and it's pretty serious and he had some tips on how people based outside of Ukraine can get ready for this stuff because in his view it is coming so head over to your podcatcher and search for risky business features.

35:55Patrick Gray: Grok joined me for that interview as well so that's an interesting conversation i hope you all enjoy it uh what else have we got here some some vault typhoon uh news usual sort of stuff uh you've described it as there uh there's some research into what you're calling james their champagne tier botnet uh which is uh only used once once an exploit is to be delivered uh walk us through it but please be mindful of the time because we are running out

36:21James Wilson: of time yeah very very much so look i just found it interesting because we know that botnets are a dime a dozen but but the thing about silent train is it's it's kept for very reserved purposes right things like recon and smash and grab and other things that might be noisy they've got other botnets for that where if things get snapped or noticed it doesn't matter but the silent train one it's it's a botnet that's designed for essentially rapid establishment use and tear down of essentially multi-hop way-in or multi-hop network paths for disguising traffic but they keep this one squeaky clean and just bring up a path exactly when an exploit is to be delivered then tear it down straight away.

36:57James Wilson: It's just super interesting to see the level of, I guess, operational concern that goes into this to make sure that there's a separation of concerns here, but also that the top tier of technical ability is reserved just for those exploit delivery techniques that need to be right on the money every time.

37:13Patrick Gray: Yeah, and we've linked through to some work from, well, a write-up on some work from Cisco Talos, who've looked at that. Look again, Kieran, I think it's the case with the Chinese. you know it's interesting right because we don't get to see what the five eyes agencies do really occasionally we might get a glimpse of it but they are restrained enough and good enough that a lot of the stuff that they do is just you know out of our view whereas with the chinese it's it falls into the realm of stuff we can detect and later analyze and it's always really you know well a lot of it's really cool like some of their contractors are dog crap right like they're just spraying you no public exploits at exchange boxes or whatever.

37:50Patrick Gray: But occasionally you see this, again, it's like the North Koreans, like it's a shame you can't hand it to them because they're bad. I think it's a really important report, this Lumen work, because, I mean, Volt Typhoon, I've described it before, the most significant strategic shift in the threat picture against Western countries for a very long time. And the pivot of the Chinese cybersecurity attack system from just spying to preparing disruptive attacks is hugely significant. And every so often, you saw this a little bit after the Obama-G no commercial spying agreement, ha ha ha. You know, they just took a lot more care.

38:30They got stealth here. So they can do stealth. That is one of the things you do see in the Five Eyes agencies. They still make a lot of noise, but they can do stealth. And you know, because of a lot of the noise around AI threats and just the generic concern about advancing capabilities of models and wider access to them, you forget that there's some really, really skilled operators out there who are doing really carefully, strategically orchestrated planning campaigns, and they know what they're doing.

38:59Patrick Gray: Okay, now we've got some, well, I guess you call it volmpocalypse news, just to whiz through. This one is very, very funny for me. The PHP to shell flaw, it turns out it is quite difficult to exploit. James, you make an interesting point here, which is that the thing that is saving us from the non-deterministic agents is the non-determinism of the targets because they're all very different and you would need a custom exploit for every one of them. But funnily enough, people have turned PHP to shell into a Palo Alto Networks device exploit, which is exactly what we saw happen with Log4j when someone turned that into a reliable exploit that would affect Panorama, right, which is like a bit of management software from Palo Alto.

39:47Patrick Gray: So it is just funny when you see these things come along and, well, it's pretty fiddly to exploit, but you can sure as hell make a reliable exploit that targets one company's appliance. And that's what we've seen here. We have seen in the wild exploitation of that GitLab floor that everyone was talking about last week. That is pretty serious. If you've got unpatched GitLab on the internet, you're going to have a bad time. You probably already have had a bad time. We're getting exploitation of the Brotley bug that we spoke about last week. Probably one that we do want to mention real quick is this Linux kernel bug, this critical Linux kernel flaw allowing remote code execution against multipath TCP servers, which initially it looked like there wasn't going to be a massive number of affected or impacted servers on the internet, but now it looks like, well, whoops, yeah, there actually is.

40:40James Wilson: Yeah, the bug is actually in the generic IP stack of Linux here. So the first bit of confusion here is that it's getting headlined as it's an MPTCP vulnerability, but actually MPTCP is what's been used in the proof of concepts and it's because of some of the nature of the way the mptcp implementation works actually allows you to get more let's say more free kicks at trying to actually exploit this thing that's actually deeper down in the ip stack if you look at the i guess the technicality of this bug it is actually something that sits at the ip options layer in the kernel and you could conceivably see that there are ways to exploit this outside of mptcp but the reason that it's sort of going hand in hand with the reporting with mtcp at the moment is two things one um mptcp as i said makes it easy to uh continue to try to exploit this because it's not 100 reliable but also the other complicating fact with mptcp is a lot of folks will be going well we don't use multipath tcp we're fine but if you're using go for example every go version since 1.24 which is a while now uses mptcp enabled listeners by default for any TCP socket.

41:49James Wilson: So there is going to be a lot of exposure to this. And because of the nature of how it works, it does allow for things like exfiltration of data. So that's how Anthropic was able to get a kernel ASLR leak and do a full remote code execution. So this is not just one of these exploits where they're like, yeah, it's memory corruption. And in theory, that could lead to RCE. They've actually shown this one pops end to end.

42:13Patrick Gray: Yeah, that ain't great. So you probably want to patch that one and congratulations by the way on becoming the new world record holder for the person who can say mptcp uh the most times in a two minute period uh congratulations to you on that uh real quick too we've got a android sms of death exploit and this one's funny because it's like just bad badly badly designed software um it is yes i give us the real quick rundown on why why that is bad software.

42:44James Wilson: Okay, well, SMSs, they can come from anyone, right? They're not trusted, right? So do you put the parser for that in your main telephony stack on the phone that's also handling critical things like making calls, emergency calls? No, you should never do that. Parsing untrusted data should be in some tiny little sandbox to the hell process. But sadly, in Android, what's happened here is the thing that decodes something specific to WAF in an SMS message that was vulnerable to a crash. The messages were persisted. It would keep trying to DQ them out of that persistence and run into the same crash.

43:20James Wilson: And so it's like, you know, how do you write software this bad in this day and age?

43:25Patrick Gray: Yeah, yeah. Now, real quick, we've got reports of scanning for another Oracle e-business suite vulnerability, which could get gnarly. That'll be our news next week, I'm guessing. Avanti keeps patching more Connect Secure Zero days. what else have we got here someone published a trojan code claude extension that managed to get a bunch of downloads out of one of the official stores and it planted backdoor ssh keys on those boxes which is like come on guys like that's that's that's not good uh germany has uh said a bunch of its lawmakers iphones were probably uh hacked with spyware this comes on the heels of that Citizen Lab report into the Israeli company that was pushing malware via its advertising network.

44:11Patrick Gray: But we don't know if these two things are connected, but it is a noteworthy report. But yeah, very light on details as usual from Apple. And what Google has disrupt a 1.3 million device botnet. And it's funny because it's from a set-top box manufacturer that's actually an Android OEM. So they're like an official design partner and stuff. And they just ship all this stuff with malware that lets people proxy connections through it and do bad stuff. So yeah, kind of funny. Congratulations, Google, for disrupting that. However, that shouldn't have really happened in the first place. And there's a huge and hilarious spat on Infosec Twitter at the moment where someone from Brileway was using their clankers to do some testing and managed to accidentally hack some guy's company.

44:57Patrick Gray: And the guy is on Twitter saying, what on earth are you doing? And the guy's just like, oh, whoopsie-daisy. Sorry, bro. like it ain't no thing. And it's very funny. So that is a, I've linked through to VX Underground's take on that. But look, we're going to have to wrap it up there. James Wilson, Kieran Martin, thank you so much for joining me to talk through this week's news. Great show. Great to see you both.

45:19James Wilson: Thank you. Thanks, Pat. Incredible week.

45:28Patrick Gray: That was Kieran Martin and James Wilson there with a check of the week's security news. Big thanks to them for that. It is time for this week's sponsor interview now with Todd Beardsley from Run Zero. And yeah, Todd is talking to us about how we should never let a good crisis go to waste. So regular listeners would know I don't spend a lot of time talking about like, we must move to post-quantum cryptography because Q Day is coming and we're all ruined. Because honestly, I think we've got some slightly more pressing things to deal with. And Todd, as you're going to hear, he broadly agrees with that, but says that there's a lot of stuff on your network that is end of life and is not going to be updated with post-quantum cryptographic algorithms, right?

46:14Patrick Gray: So you can use this as an excuse to do a search and destroy on that stuff, and you can sell that to your board. So it's a fun way to make hay out of the anxiety around quantum computing. So here's Todd Beardsley explaining all that. Enjoy. You know, this is becoming top of mind for CISOs because somebody on their board read a scary, scary story about, oh, no, magical computers are coming and they're going to like break all of our encryption forever and we need to upgrade right now. Which is a little bit of an overstatement, but, you know, like any good politician, which a CISO definitely is, you should not let a good crisis go to waste.

46:56And so what I believe is that one could use your PQC mandate, if you happen to get one, you know, you're a CISO, you have a board, you have customers, and take that mandate and apply it to your end of life fleet of junk that is on your network.

47:17Patrick Gray: so so so what you're saying is if you're looking for an excuse to take some old junk out to a field and put a bullet in its head stomp on it off the space tile this pqc might be the pqc flap might be a good way to justify doing that sort of thing to the board yeah and it's not like a bait and switch and it's not you know it's not a shell game of budgets like the money you spend and the effort you spend on getting your end of life house in order, which everyone has one, is money well spent that actually does help solve the coming PQC crisis. Mumble, mumble, mumble, whatever that is. But really, it's solving much more serious and pressing problems, I would think now.

48:05Patrick Gray: Like if I think of the benefit of dealing with like EOL and old junk hanging around at an enterprise, particularly stuff that can be connected to from outside the org, getting rid of that like i'm not thinking i solved a pqc problem i'm thinking but you can justify it by solving a pqc problem is the trick right and so like in in my opinion just like you know i do not blame you at all for not running pqc stories patrick um because this is news of the future like we are not there yet um we're not going to be there for a while your mileage may vary sometime in the next 10 years almost certainly not in the next year almost positively positively not in the next year certainly not in the next two um but we're we're trudging along right science marches on and everything and the the costs for uh you know cracking something useful like keeps going like is forecasted as keep going down and down and down it may never happen there may be like there are reasonable people who are arguing that like it is physically impossible to build a computer that does the things that like a qubit slinging computer can do Right.

49:11But even if nothing happens, if we take this moment in history to like rev up our, you know, our baseline of what is running, what is allowed to run, what do we expect from our cryptography? And that means getting rid of EOL, you know, and either migrating or upgrading or something like that. At the end, oh, no, I've gotten rid of all of my EOL all for nothing.

49:38Patrick Gray: No, no, I'm definitely with you. It's really interesting, though, that I think, what was it, about six months ago, because, look, I read every single story, or at least read the headlines of every single cybersecurity story that gets published, right? It's insane the amount of news that I consume. And I just noticed there was this huge uptick like six months ago about everybody saying, oh, massive, right? It seems to have died off now, which is good. People have calmed down. But why don't you walk us through what a program looks like? Because I mean, I'm guessing, you know, you're here representing RunZero.

50:09Patrick Gray: RunZero does discovery. I'm guessing that RunZero can tell you when something is not quantum safe. Well, when some things are not quantum safe, obviously it's not going to be able to get deep into, you know, every single device that may be present in an organization and give you a full list of every single algorithm that might be present. For that, you need something on board. That's right. That's right. So I'm guessing, though, that, you know, well, you tell us what you think a program to eradicate this stuff should look like. And I guess, you know, tell us how Run Zero could play a part there as well.

50:40Sure. And I do think that, like, EOL is a fine place to start, right? Because if you have old, old end-of-life systems, you know things are legacy, you know things. You can at least say, like, oh, it doesn't do cryptography, therefore not my PQC problem right now, right? You can note that, and you can have it in your asset list. I really think that the name of the game with PQC readiness is building good, dynamic asset libraries of what is actually running on your network. We all know that Microsoft Excel is the number one asset management software on Earth. Hopefully one day we will unseat them.

51:26but they're pretty present in most enterprises. So having that kind of program running goes a long way towards this. Once you've done that, like today, I can tell you like the store and decrypt later problem of post-quantum computing is largely solved with big asterisks, right? Today, right? Because every library and every browser and every component that people use to write modern software is already there, already has the magic in there of being quantum resistant.

52:08Patrick Gray: Yeah, yeah. Quantum resistant, right? But that begs the question, like what sort of stuff, because you're talking about, well, you get to get rid of some legacy stuff, get to get rid of some old horror show EOL crap. What sort of stuff are we talking about there? so that would be like old a lot of times we run into folks are running end-of-life systems and they know it and they're running it not because they're lazy or stupid it's they're running it because they have this one key niche application that only runs on windows 7 right and there's one vendor in game in the game right and they're there they're the only source for it and this happens all the time.

52:47Every enterprise is a special snowflake. Every enterprise has that one bit of niche software that they can't imagine doing business or providing the service that they provide without it. And so it comes down to like either leaning on that vendor and say like, hey guys, you have to get with the program here. The White House told me I have to do this by 2030 sorry like my hands are tied um and um which is the real deadline december 31st 2030 um and just having that conversation and like foisting that on your software vendors um is is really key um i would say then too um for ot stuff for large i think i figured it was going to be ot stuff i'm going to slide into ot right like yeah it's it's touch and go right whether or not a given piece of OT gear even does any cryptography, if it does any modern cryptography, when it comes to things like validating firmware upgrades, or, you know, the normal, which is the real, which is the actual problem.

53:54You know, the spying on things in the side is a problem largely solved by fronting and all that. But the key exchange, so key exchanges solved, but like signing and signing encrypted packages is not, is very much not solved right now. Right. Um, and so if you have some gear that you bought because you believe that you can, you can assert that like, Oh, this firmware is good. It's from the vendor. It actually fixes things. It's not backdoored. Um, if you're relying on, on all that, then you have to get with that vendor and say like, Hey, this thing is supposed to look like it says on the label.

54:31It has a life of 30 years, I can't help but notice that is well into the much more likely period of having, you know, quantum cryptanalysis around. What are we going to do about that in like year 20? Like, how do I upgrade this thing? And having that conversation and pushing the vendor to do that, to at least assert it or offer a discount or something, like, I don't know. There are ways to get there, you know, and like the major vendors are certainly entertaining this, right? I don't think anyone has actually solved it for real. But I know that like Rockwell and Siemens and all them are definitely like focused on this because they sell gear that last 30 years, right?

55:13And that is a pain to upgrade and expensive.

55:16Patrick Gray: Well, I mean, I'd figure ironically enough, you would fix that with an upgrade, right? So you would fix that with by pushing a patch that updates the firmware validation stuff to, you know, something quantum safe. But you have to do it before queue day which is yes yes exactly the exact version of this but i mean in this case you know you've given us examples of enterprise software that is irreplaceable so you can't just yeet that out of an environment nope and iot devices where you're going to have to lean on the vendor so what is the eol stuff then that we can talk about getting actually getting rid of what's the stuff that we can throw into the wood chipper well um you know like i say like eol tends to be there for, you know, either it's there for a reason, like I must run it.

56:01I must run this like EL operating system so I can run this like currently supported somehow application. Or somebody forgot, you know, it's just something that has been running there. Nobody really knows what it does. Right.

56:13Patrick Gray: So it's that category is where this is, this crisis will not go to waste where you can actually identify some stuff. I mean, I remember once like 20 years ago-ish, you know, reading about some Linux server that had been accidentally sealed into a wall with what we'd call drip rock and you guys had called drywall. And there it remained for like a decade until someone was renovating the building. And they're like, huh, look at the uptime. We had an old Sun Solaris machine that someone was helpfully maintaining for like 10 years. That IT had no, this is back when I had a real job in IT. And IT had no idea.

56:50And this was at Westinghouse. So like, you know, I don't think it had nuclear secrets on it, but you never know. Westinghouse had a fair amount of nuclear secrets on them. So yeah, so that kind of hunting for your old legacy stuff that like of questionable value, like, well, let's model what actually happens if things go offline. Like what happens if we pull the kit? Like, can we live without this thing, right? The very traditional, which is something we did during Y2K. You know, we found old things that we couldn't figure out how to update. The company had gone out of business. We knew that the date field was going to be wrong.

57:29Well, what happens if we get rid of it? Like that's, I mean, I spent a lot of my 1998 and 1999 doing that kind of work.

57:37Patrick Gray: Yeah, it's funny because before we got recording, I was chatting about Y2K and about how it turns out. And I said, you're welcome. Yeah, so I saw some reports that I think Dario Amadei was a Y2K doomer at the time, which is very funny and very on brand for him. But I mean, obviously, you know, there would have been issues if people didn't do the work. But like, as I was saying to you, I can't see the board of a major bank going, no, we're not going to authorize those contractors to come and fix that. We don't need computers. We're out of the loan business because we can't calculate past 2000. Exactly.

58:08Patrick Gray: I mean, I think it was always something that was going to get sort of worked out. But I guess, but hang on, hang on. That's the thing, right? There's not as clear a deadline. Like at least with Y2K, you had a clear deadline. I guess what's interesting about the PQC stuff is it's fine until some guy in a lab coat turns up and says, hey, good news, everyone. And then, you know. Well, this is the scenario that I've been trying to study up on and figure out if I believe or not. With Y2K, you had a date. And you knew things would break after this date. Something would break after this date. What a lot of people don't remember is February 29th, 2000 was a weird day as well.

58:49and a lot of things did break on that day because of how leap years work. But with PQC, you don't have a date. You have kind of a tenuous date out in the future and things don't break. Like they just keep doing cryptography and everything seems fine until oopsie doodle, all of your signing keys are now in the world. And by you, I mean everyone. Like you can, like, and I don't think that's what's going to happen. I don't think it's going to be dramatic on its own. I think what will happen is some cyber superpower, you know, pick your favorite, is going to figure it out, is going to have the gear, is going to have like the secret lab.

59:29And then they can be quietly decrypting secrets along the way, like as they pass.

59:34Patrick Gray: You are not going to be able to rent these on AWS for five bucks an hour, right? Like that's not how this starts. So it will be very classed to, you know, your your top five superpowers who can all do the spying for a while until one of them tips their hand and says, like, oh, now we have to, like, fake some firmware keys. Yeah. And we're going to fake the firmware keys in advance of a military operation. Like, that is why you would do it. This is the only thing I can think of that seems reasonable to me of, like, why you would tip your hand. And you'd have to have a really good reason, like an existential reason to do it.

1:00:10Patrick Gray: Well, yeah, I mean, look, that's a whole other conversation. That's a much longer conversation. But look, Todd, we're actually out of time, mate. We're going to wrap it up there. All right. Thank you so much for joining us to talk about the idea of, yeah, never letting a crisis. Never let a good crisis go to waste. That's right. So if you want an excuse to actually go out there and do inventory, you can dress this up as your post-quantum cryptography plan. Easy sell to the board. great to see you again my friend and I'll look forward to chatting to you again soon alright, later Patrick that was Todd Beardsley from RunZero there with this week's sponsor interview, big thanks to him for that and big thanks to RunZero for being a risky business sponsor that is it for this week's show, I do hope you enjoyed it, I'll be back soon with more security news and analysis but until then I've been Patrick Gray, thanks for listening

1:01:09Thank you.