Between Two Nerds: Attribution is dead, long live attribution

0:03Tom Uren: Hello everyone, this is Tom Uren. I'm here with The Grugq for another Between Two Nerds. G'day, Grugq, how are you? G'day, Tom. I'm fine and yourself? I'm very well. This week's edition is brought to you by Push Security. You can find them at pushsecurity.com. They make a browser plugin that protects you from different types of attacks, collects telemetry, protects you from identity theft. pushsecurity.com So, there's this tweet here from Florian Roth. I had a discussion with a friend in Thread Intelligence about AI-driven attacks. And the tweet goes on to talk about how the ability to use AI may erode attribution by making things either more one-off, more bespoke, and so use less.

0:57Tom Uren: or making things so generic that it's impossible to tell them apart. I flung that to you, thought it was potential flutter, and you replied, oh, that reminds me of an article I wrote way back when. This is a topic near and dear to my heart. One of the first frack articles I ever wrote back in 2002 was on anti-forensics, which is very much about the idea of how do you reduce the quantity and quality of evidence that can be used in a forensics analysis. And it's sort of quite deliberately phrased because on the one hand, you want to limit the amount of stuff that's found. On the other hand, you want to make the stuff that's found as useless as possible.

1:43And so in 2004, I expanded on that with another Frack article where I advanced the theory a bit more. And I came up with the idea of, sorry, I didn't invent the idea, but I classified the term for keeping things completely in memory so that they don't, so it was data prevention. So I had this, I'd extended the framework from sort of data destruction, data hiding, to now data prevention, data destruction, data hiding. There's one other one that came up later, which was data chaffing of just creating a lot of fake or just extraneous data. I never liked that one very much. To me, the idea of creating more evidence that you were there just seemed like a bad idea.

2:31Tom Uren: it seems a bit counterproductive in that it seems like it would raise the risk that you would get pinged in the first place and exactly you're better off keeping a low profile this was yeah like it seemed it seemed like a bizarre approach like i can understand the logic which is the way that you you think about it is someone only has 18 hours to do a forensics analysis before they have to do something else, right? And so if you leave so much data for them that it takes them 19 hours to complete, you win. However, the downside is that you're assuming that it will always take them 19 hours to do that, whereas computers are always getting faster and there's always more programs coming out.

3:22Therefore, the ability to sift through that amount of data is always increasing. And so it seems to me like if you're targeting your defenses at what's possible today, if it gets discovered in 18 months.

3:37Tom Uren: Well, it seems like a kind of fundamentally pessimistic approach of we're going to get pinged one day. Whereas I don't think real actors believe that. Either we're going to get in and out and it'll be fine or we'll never get pinged. And so why, you know, why operate on a premise that you don't believe? It's a little bit like fighter pilots where everyone, every fighter pilot knows that he's the best. Like they're completely confident that, you know, these other guys are good, just not as good as me. And I'm pretty sure that hackers sort of have that same mindset of like, yeah, I mean, they're good.

4:17Sure, I'll grant them that, but not up to my level. I think you kind of have to think like that. Otherwise, I'm pretty sure that no one goes out with the intention of getting caught or the expectation of getting caught. That's a blanket statement that has absolutely no exceptions at all.

4:36So a little while after these two articles were out, right, put forward this sort of framework of how, like the theory of denying evidentiary data to an analyst, let all that out. Then in another article in like 2014, this one I just posted on my Tumblr. So it obviously had maybe not quite the same distribution as the frack. But I was pointing out that the problem, like the fundamental problem with relying on data that you recover from an incident is that absolutely all of it is under the control of the attacker.

5:16Tom Uren: Right, right, right. So maybe not in a real sense, but in at least a theoretical sense, it's a choice to leave data behind. Right, right. Is that the sort of fundamental premise? Right. Yeah, that's basically it. The IPs that they used to connect from, they selected those IPs. Now, whether it's because that happened to be random chance or it was deliberate, it was a choice, right? The malware that they use is a choice, right? Like basically their entire TTP, like their tradecraft, their tools, their techniques and their procedures are all deliberate choices that they've made. And they might not be deliberate to deceive, but they could be.

6:01There's no IOC that you can recover that wasn't generated by them in some way that they are in a situation to control it. Right, okay.

6:10Tom Uren: So I guess even in the case of like out-of-band network traffic captures, the IP address, you can't go and remediate that from logs if you're on a particular system, but you at least have some agency beforehand about what IP address. Right. So it's just a choice. I guess it's a trade-off from their perspective of what is worth doing and what we don't care about. Right. And so, in a way, one of the things is that while all of these are choices, they're not free. They cost resources of some sort. So, in a way, the cost of developing a new tool set is what makes them use the same tool sets or to reuse the same tool sets.

7:00but with the LLMs, that cost goes down. So my observation is that attribution by tooling was always parasitic on the cost of developing and maintaining tooling. Right. And once that goes away, then that aspect of identification also goes away. Yeah.

7:24Tom Uren: I kind of feel like parasitic, is that the right word? I feel like it's kind of related to or dependent on or imposed by. Yeah. Parasitic is not the right word, but I like it. So I'm thinking of snake malware and some of the malware where top-end state bodies have invested a huge amount over a long period of time. I think Kaspersky called NSA malware, I think they called it the Death Star of malware, something like that, a huge investment to make it all singing, all dancing, and a plug-in. And it seems like there are reasons why you would choose to do that. So to me, this is the KR strategy approach.

8:11So KR strategy is for child rearing, basically. There's two ways you could do it. One of them is you can invest very, very heavily in one or two, like a very small number of offspring and put all your resources into making sure that those survive to adulthood to reproduce or you could put all your resources into creating as many as possible and then statistically some of them will survive and i think it comes down to that and that the the best approach has always been sort of that invest all your resources in a small number of exquisite tools and rely on those. I think one of the things that's been shown by the war in Ukraine, like the cyber war, is that investing heavily in bespoke tools that are exquisite and really great and all that stuff, like it's a bit of an attritional war even in cyber.

9:05Like all of these things get burned out quite quickly just due to the volume of attacks that you have to do continuously.

9:12Tom Uren: Yeah, I was going to pull you up when you said the best strategy and I was thinking it may be the strategy or the it may even be the best strategy for some players but i don't know that it's the best strategy for all players like i guess no that's that's like that's why it is a strategy in that there's all different ecological niches i guess and it's a choice if it was the best strategy everyone would choose it no one would ever do it the other one right but it's yeah So it can be the best strategy, I guess. So it will match the overall strategies of some players better than others. So if you're incredibly well-resourced and you're doing just these brilliant hacks all the time, maybe you want the absolute best tools and you're just going to rely on the fact that they're never going to get burned.

10:01That they get burned at such a slow rate that you can afford to invest years and years at a time. but on the other hand if um you know you look at like the whisper gate the caddy wiper like all of these things these were not exquisite tools by any means they were uh they're very much like the the soviet mass production of world war ii sort of thing so these were deployed during the beginning of the ukraine invasion there was just loads and loads of these different wipers so one of the i I have to do a tangent. One of the funny things was the, you know how the Soviets were like producing the T-34 in like these massive quantities during World War II?

10:43Tom Uren: That's a tank? The T-34 tank. So during the war, they found ways to reduce the number of steps taken from like 12 ,000 to 6 ,000 to produce this tank. Some of the things that they started skipping were there's these periscopes that you use to look out the tank. And the way that you do the Periscope is you polish some steel so it reflects. If you don't polish it, that's a lot faster.

11:14Tom Uren: Yeah, that sounds like a pain. I mean, that's a lot of time that you save right there, right? Yes. Anyway, I think that's the sort of quality that you're looking at with these sort of wartime tools where it's like, if you don't bother to encrypt, it's actually a lot easier to get the thing out the door. If you don't add support for multiple things, et cetera, et cetera. Yeah, but I guess the whole point of this discussion is that the AI, it affects the cost basis, right? Right, right. It exactly gets back to the same thing. The problem with the Soviet approach is that you save money by just not doing steps.

11:56But now you can save money by just having all of the steps automated. right like the the llm will do all the stuff for you and so you can have the same number of developers as before managing agents and supporting because i think this is the key thing right is that it's very annoying and difficult to develop one piece of malware but then it needs to um it needs to support and continue to run on different versions of the operating system and then it needs to work after different patch levels are out. And then you need to add support for basically like maintaining it over time is more costly than simply developing it.

12:42And the thing is LLMs make even that low cost. Even the maintenance step becomes affordable. So I think all of the reliance and the fact that if you're going to make bespoke tools, you had to reuse them on every customer, on every target, on every victim that you went after. Whereas if you wanted to use generic tools, they weren't very good for what you were trying to do, particularly if you're at the upper end of doing attack stuff. Now I think you can do two things. One is you can take the generic tools and have them as good enough, because a lot of the time you just need good enough. or you can have a bespoke tool per victim.

13:30Tom Uren: Right, yep. So it doesn't necessarily change the strategy. So if you're into death stars, I guess the potential is that you would have just more death stars. You can still follow the same builder. You could have individual death moons.

13:52Tom Uren: yeah so it does and that well i guess that kind of makes sense right and i suppose that would in a way that's compartmentation right between different right exactly however you divvy it up presumably you would compartmentalize everyone in this country or everyone in telcos or whatever whatever right however you construct your compartments you could have just one per each easily enough. Yep. But if you are into the generic approach, you know, just use whatever software, then it's also an enabler for that, right? And I guess that also reduces the ability of the people at the other end to do forensics and attribute because it's just generic, right?

14:40Right. That's sort of like the other approach is reducing the quality of the data that they get back. And one of the ways to do that is to use things that have no authors or that are, for example, what I was using was I used awk to write a backdoor in an awk script. And so if you look at it, it's like, well, this is a publicly released backdoor running on a piece of system software. Like it's not an identifying tool. I used GDB to do things like write into memory and then execute from memory. so you wouldn't have to have like a custom loader that you'd leave behind you could just use gdb which is again a systems tool so it wouldn't be uh like it would be a particularly good signature just say here's something that anyone has access to um that anyone knows how to use you you wrote

15:33Tom Uren: those articles what over a decade ago florian's two decades ago is, I guess, more current, but it's fundamentally the same ideas. But one thing neither of those mention is living off the land, right? Which seems to be the... That's the using generic systems tool. Right. Like that have no signature. But that already has done a lot to remove the ability to attribute, right? Yet still groups that do that get attributed. People write about... I think that's the tradecraft, not the tooling. You are correct. A lot of magic and visibility cloak. I guess AI, well, some number of actors have adopted living off the land for whatever reason.

16:20Tom Uren: Because it's harder to detect. I think, well, for a number of reasons. It's harder to detect. It's harder to attribute. It's harder to block as well. Because if you whitelist a bunch of tools, you still need the systems tools. It's very hard to block those. Right. It just kind of makes sense for a number of reasons. And attribution difficulty, it's hard to know, I guess, for each of those actors that have adopted that technique, how important that is. There could be other reasons that they're adopting it, right? And then AI has come along and it's made it also for actors that don't adopt living off the land.

16:59Tom Uren: It's also making attribution, seems like it will make attribution harder. Right. Right. So I think one of the things with an AI is that because it's trained on all of the public techniques, like here's how you do hacking, they don't necessarily have a signature in the way that if someone has been to the Pyongyang school of hackery, they're going to have learned certain techniques in certain orders. so when you're sort of looking at something done by the North Koreans they're going to have a flavor of how they operate like they like to use these directories to hide and they like to use this sort of lateral traversal approach whereas for an LLM they all basically like the same stuff in the same order so I think that it takes the hands-on keyboard part and makes that now a generic tool Right.

18:00Tom Uren: I was wondering about that. I was talking to James Wilson, who's the technology editor here at Risky Business, and he plays around with LLMs all the time. There was an article or story from Anthropic where they had a number of agents competing to achieve tasks. and they had some limited resource. I can't remember what it was, but basically the agents were fighting with each other and they were kicking each other off that resource and removing access and literally fighting. And he had two, and I thought this was hilarious, two Chinese LLMs that were also sharing a single resource. And he said that they'd worked out a timeshare arrangement between the two.

18:46Tom Uren: so it was like you know you've got the communist and the other and being like yeah if what i need to do is i need to get you signed up today do you have a credit card let's get you into your timeshare we'll get to your slots you'll be able to like come here on vacation you know well i interpreted that as the communist one whereas the capitalist one is oh we've got freeloaders we're going to kick them out and evict them right so i wonder about different llans having different about the thing where they they tested um they tested farms in the same area where one one set of farms farmed wheat and the others found rice yep and the rice were like the rice farmers were communal because you need communities to do rice farming whereas the wheat farmers were individualistic because one person with like a small family can do their own thing so obviously the chinese llms yeah so i do wonder whether llms have different characters and in fact he's spoken about that different alarms do have different characters and when you're prompting them you can prompt them in different ways but i suppose the broader point is that there's so many to choose from that you might you may be able to go up or they're using a particular flavor of lm a very cranky one or a very so i i think right so i think that's the thing is like that doesn't become so much of signature anymore because anyone can use any any right right and i think it'll be not long before everyone is just hacking with llms full stop i think it's i think it's like developing right like three years ago if people are like oh i'm using an agent to develop it's like you're you're producing crack like you're you're you're producing trash and you don't know what anything does you're too dumb to know that it's trash right right this is proof of these days right these days that's just how you develop now like that's just how you do it i think we're probably closer to that that beginning of the three-year curve in this case than we are towards the end yep but i don't think that it's i don't think it won't happen i think that it's absolutely inevitable we'll be at the point where it's like you know when i was your age we used to know how to hack ourselves.

21:05We used to type on the command line, okay, grams.

21:11That's because just like coding, just like vulnerability development, hacking is very much a thing that has a known win state. That's very easy for LLMs to do. A hash prompt is a very good way of knowing that you got root. In Halvor's words, there's no arguing with a hash prompt. Someone could say, yeah, but that's

21:31Tom Uren: theoretical and you go i did it yeah um that's the sort of thing that llms are great at when they could actually see that they've achieved a thing and so like hacking is ripe for llm takeover so i was thinking about what difference this makes in terms of threat intelligence so some of those trends we've spoken about moving to living off the land be going on for a long time. One of the groups I wrote about recently, they didn't really use malware at all. And it was just getting initial access by ringing up people. So voice phishing. And still Google Threat Intelligence wrote, was able to say, here's what we believe is a group.

22:19Tom Uren: I'm presuming it's because they use the same techniques. so they use the same tradecraft even though they don't leave behind artifacts the same way right some of that might be things like well we've got recordings of their calls to help desk and it seems like it's the same people so right i think now you could imagine that they could uh disguise their voice to appear like different people but it seems that it's still the skill of talking someone out or into giving you a password or resetting it or whatever they need to do is still a real human skill and it's still going to be like the same set of target types in the same sort of sequence like they're going to do this uh voice thing then they're going to get some sort of access then they're going to use that access to get something else that they're going to use so like i was just thinking of how lapsus and the you know shiny spiders and all that their techniques leave no traces in terms of bespoke tooling.

23:23They do things like they'd use credential stuffing to gain access. Then they would find Git servers that they would download. Then they would search the Git servers for secrets. Then they would use those secrets to access databases, collect all the databases. Like that was, none of that requires bespoke tooling, right? It's all SSH in, log into that, do a git pull. None of it is custom. It's all very generic. But they're still identifiable because that sequence of steps for the goal that they're doing. And then, of course, they extort it. So that's a little bit of a giveaway.

24:07Tom Uren: Yeah, I guess ironically, or maybe is it ironic? I don't know. the ransom notes seem to be the thing that is distinctive about those organizations and i suppose i mean for them it that is a form of branding right so that's right core to the business you need to associate it with a brand to either demonstrate that you're serious or reliable or whatever threat or yeah whatever you're trying to get across and if you don't have a brand you can't well i was just going to say like nsa is going to be you know they're going to be struggling here because they don't have any way of marking things as their own but my my immediate thought was actually like if you if you look at North Korea let's say North Korea moves to an entirely LLM generic tooling completely like beige generic gray unidentifiable you'll still know it's them because no one else is going after crypto exchanges like that right like they're still like if someone goes after crypto exchanges moves all the money into like the same set of like money mules and all that that they use it's not very difficult to figure out who's behind it like it's probably not mi6 doing something like throwing it out there right so it feels like the for some of the groups for at least some crime groups it seems that like attribution doesn't actually matter because it's inherent in the business model and i guess right clop or and i suppose the reason they have brands is that it is inherent in the business model it's a yeah it's a necessary part of their business so again we're kind of back to the there will be some state organizations where the ability to use llms will be very useful for sort of muddying the waters but for maybe many other groups it doesn't make any difference Yeah.

26:04So I think what it comes down to is if you were relying on tooling and a specific set of, like a set of TTPs, if that was the only way that you could do clustering, what you were actually relying on was that something had to be scarce enough to be reused. So like the cost of developing bespoke malware was so high that it had to be reused all the time. Or that the actions taken were distinct enough that they were different between different actors. Both of those are no longer necessarily true, but that doesn't actually mean that those things are no longer valid, I guess. It moves to a different layer, I think.

26:54There's a different way of it.

26:56Tom Uren: One thing that makes me think of is that it seems like a lot of state threat actors that get pinged and attributed, it's because they're kind of naive to the art of the possible. and so it's they never really took that kind of anti-forensics stuff seriously because they were never really worried about it like so there was no strong motivation to invest in it in the first place right you know we've been so successful for 20 years why bother yeah it's like what difference did it make yeah so i mean i was kind of thinking that it's like the when has any of this actually slowed anyone down right like all the crypto exchanges know that that lazarus is coming for them yep it doesn't really seem to matter all that much um like defense contractors are well aware have been aware for decades what they need to do to be secure like don't go home with someone who's you know 10 times more attractive than you are and came up to you at a bar like that's a threat model and they know about it.

28:06Most guys after a few drinks are,

28:08Tom Uren: that person's 10 times more attractive than I am. That's incredible. It's my lucky day. LLMs do make a lot of things cheaper, particularly on the technical side, but they don't make everything free. I think there's things that still have costs which need to be borne in some way. There's some sort of resource investment. Targeting is very much still like the victimology and all that is still very much going to be a like a factor in determining who did something so if you remember uh olympic destroyer in 2018 that was the malware that was targeted the winter olympics yeah the 2018 olympics in seoul and wherever else it was like kaspersky was the first one out the gate to do like we've done an analysis and we've we've looked at this malware and they did this very clever thing where they adjusted the headers that were used during the compilation to make it appear that it's a north korean group but it's actually not the north koreans it was a false flag could have been anyone though they're flying a bit close to the sun there yeah yeah so but it was very funny that they came out and the first thing they do is they say this is a false flag attack which implies that you know any part of it that's identified could be a false flag.

29:32And they didn't do any attribution, and they focused only on that one specific aspect. Whereas it was amazingly obvious that the country that was being banned from the Olympics for doping and it had a known cyber capability.

29:51So even though the malware was not directly linked to them, there was no, like it didn't need to be TTPs or anything like that. It was just like... Yeah.

30:03Tom Uren: So even... Yeah. I'm assuming it was unique malware set up for that situation, right? And so even though there's no technical artifact that you could tie to any other operation, it's like, well, yeah, who would want to do that? And who is not North Korea? Sudan.

30:26Yeah. So I think there's still avenues which are completely independent of tooling and such. But there's also going to be things where there's human interactions. I think LLMs are still, I don't know if they'll ever be. I assume they will. But they're still a bit slow and they tend to not react like humans do all the time.

30:53Tom Uren: Right. Yeah, I was thinking of the voice phishing we were talking about. where it seems like I can imagine a good person would be still probably a few fold better than an LLM for various reasons but I'm not sure how long that lasts so I guess in terms of the groups that are using phishing of any sort to get initial access it's just that that's a skill that kind of works right like once you found something that works yeah you're just gonna stick with it to me the thing that makes sense is once you have an LLM doing hacking, that means that as a human, you can do more hacking, right? Rather than you can outsource everything, it just means that you can now operate a lot faster and better.

31:37So you can get on something and sort of guide what you're trying to do and let the LLM do all the, you know, typing and whatever for you. And I can see that sort of being similar for phishing, where you would have, or the voice phishing, where you would have a human there sort of sitting in and maybe an LLM goes through and it can give you a bunch of different options of how to take things or it can give you, like if someone says, oh, like you work with Jeff and it'll go like, that's probably Jeff and whatever, you know, based on this database record, here's the stuff, right? Like that seems like it can augment as well and that could be, like that could just enable you to do more.

32:19Like that's how I think like the top tier actors will be using it rather than outsourcing everything that would be enabling them to do more with the same resources that they have right right so i guess the

32:33Tom Uren: conclusion there is the top tier actors will be doing clever careful tradecraft just more of it and the bottom tier actors will be doing more bottom fishing tradecraft just more of it Just more of it. Thanks a lot, Tom. Thanks, Scott.