Srsly Risky Biz: Bring on the AI lawsuits

0:04Patrick Gray: Hey everyone and welcome to another episode of Seriously Risky Business, the podcast we do here at RiskyBizHQ where we chat with Tom Uren all about what he's written about for us this week. G'day Tom, how are you?

0:19Tom Uren: I'm good Pat, how are you?

0:20Patrick Gray: Good. And yeah, Tom writes the weekly Seriously Risky Business newsletter. If you're not subscribed to it, head over to Risky.biz and you will find subscription links there. But the idea behind Seriously Risky Biz is, you know, it's our newsletter that focuses on policy and government stuff, which is why it's very serious business. And, you know, hence the name. Tom, of course, worked for the Australian government for a long time before moving on to ASPE, which is the Australian Strategic Policy Institute, and has worked with us here at Risky Biz for about full time for about five years now, right?

0:52Tom Uren: Goodness. Goodness gracious me. Goodness.

0:55Patrick Gray: Yes, time flies when you're having a good time, as they say. Anyway, this edition of Seriously Risky Biz is brought to you by Spectro Ops, who, of course, make the Bloodhound Attack Path Enumeration Utility or Tool or Platform, whatever you want to call it. It's wicked awesome, is what I call it. So big thanks to them for sponsoring this week's edition of the newsletter and the podcast. So we're going to be chatting about two things that you've analyzed and written about this week, Tom. The first is something that we did cover briefly on the weekly show yesterday, which was some comments from Scott Besant around AI labs getting some sort of, you know, freedom from liability, right?

1:39Patrick Gray: When their products go haywire. So it seems like they want to be able to have endless industrial accidents kind of along the lines of what happened with Hugging Face, but they don't really want to have any of the responsibility that comes with that. Like these AI labs have been making noises about, hey, you know, we're making this technology to drive the economy forward. If you don't want to slow us down, maybe we could use some liability shielding here. But his response to his credit thus far has been lol, LMAO even, which is, I think, the appropriate response. And you agree.

2:12Tom Uren: Yeah, yeah. So my take on kind of reading between the lines is that there have been numerous calls from the AI labs themselves for increased regulation. And my take has always been that they want someone else to tell them what to do. So when something goes wrong, they can then say, well, we were following regulations. We can't be held liable for stuff that someone else told us, the government told us to do. And that's, to me, the underlying subtext of where this is coming from. And if you are an AI lab, I think, or the CEO of an AI lab, it absolutely makes sense. This is something you would at least float.

2:52Tom Uren: And if we could get that, that would be wonderful. I also would like liability from everything I do in my life. It's just that I don't think it's reasonable to ask for it. They're in a slightly different position.

3:04Patrick Gray: I think you mean you would like no liability. for everything that you do in your life, not liability for everything, which is sadly the truth. But I mean, this is the thing, right? Like you look at so much of the discussion around this, I think has been, I don't know, just has missed the point a little. Like when we saw the early stages of the hugging face thing, everyone's like, oh, but you know, are we going to charge the text generators with CFAA violations? It's like, no, because they're machines and they can't commit crimes, right? Like the metaphor I would use is like if I build a machine in my backyard that suddenly goes crazy and starts rolling through my neighbor's backyards and knocking over their houses and stuff, I'm going to be liable for that, as I should be.

3:47Patrick Gray: But unless I've designed that thing to specifically go and do that, there's probably not much of a criminal liability there. We just seem to be landing in a sensible place where the Trump admin is indicating or signaling that it's not going to give these AI labs carte blanche to just do anything they want, like, you know, in light of incidents like the hugging face and whatever. You know, if that happens again, you're going to be liable for them, I think is what they're saying.

4:18Tom Uren: Yeah, yeah. I think there's still genuine questions about if you're using a model for your business purposes and something unexpected happens, who's liable there? I think the sensible answer is well if if you the company who is using it as a third party hasn't thought about it and and got sensible protections then maybe you should be liable well and then there's then there's all

4:40Patrick Gray: of the your laws uh basically which is you know even with not non-ai software they give the software company permission to put the user through a wood chipper if it suits their business purposes right

4:51Tom Uren: so yeah so i think there's a discussion to be had about that kind of extending where exactly who is responsible for what, when. I think it feels to me like a negligence discussion. Like, you know, these things make mistakes.

5:04Patrick Gray: But see, this is my point, Tom, is I feel like, you know, we've got a system for determining where the liability sits in all of this, and it's the court system. You know what I mean? And this is what Besson's saying as well. Like, this is exactly what he's saying, which is like, we're not getting involved here. Like, if you're going to keep having industrial accidents, and that's the term that I use, instead of their corporal euphemism of, like, alignment issues, which they love to say, because it sounds softer than industrial accident. If they're going to keep having these industrial accidents, eventually someone's going to sue them.

5:31Patrick Gray: Like in the case of Hugging Face, it looks like they were able to just shake down OpenAI for a bunch of free tokens and that kind of put it to bed. But that's not going to be every company's response.

5:44Tom Uren: I also think the other part of this story is that when you look at all those incidents, it was clearly not well, maybe not well thought out, not well implemented the controls they had. And so the particular example I looked at is the hugging face incident where the security team at OpenAI noticed that something was wrong. And then they went, oh, something's wrong. Let's just rebuild it.

6:09Patrick Gray: Let's just rebuild it exactly the way it was.

6:12Tom Uren: And let's not investigate a hacking machine and see what's going on. Let's just move on, move on. Nothing to see. And so, you know, from small things, big things grow. So that turned into, that was before it had hacked Hugging Face. And so there was definitely an opportunity for them to revisit their controls and go, oh, hang on, something unexpected is happening. What's going on? Let's figure out what that is. I think that would be demonstrating a kind of good security culture as opposed to, well, not.

6:45Patrick Gray: Yeah, yeah, indeed. So the upshot there is nice try, guys, but you're going to have to be a little bit more careful in future, which is good. I think this is a good result. So let's now talk about the second thing that you have worked on, which is, I think it was last week or maybe even the week before, you wrote about how a Chinese APT group was using AI to muddy attribution. And now you've looked at how a Russian APT group is using AI, but with different objectives, right? So they're going for speed to evade immediate detection, but they don't seem to care as much about evading attribution. So what's interesting is you've sort of brought these two things together and compared how two different threat actors are using AI to tick very different boxes.

7:32Patrick Gray: It's an interesting read.

7:33Tom Uren: Yeah, yeah. So the Chinese approach, when I read about it, it was like, yeah, this is really sensible. I can see someone like NSA or ASD using a similar approach. And it was really to speed development, develop a whole lot of different varieties. And it serves a long-term purpose, which is to enable a long-term collection campaign to keep going. And I guess all of three weeks ago, back then, I would have gone, yeah, this is the way for state actors to do it. And then this report comes out from Anthropic and they talk about a Russian group which is really using AI in a very, very tactical way. It's like our malware has been detected.

8:16Tom Uren: Let's have a workflow, an entirely AI workflow to rejigger it so it doesn't get detected anymore. And it's just let's move on. We don't care about attribution because it's Russia and the targets are often Ukrainian. where you know if you're well yeah you make you make the point here that no one's doing incident

8:34Patrick Gray: response in ukraine it's not like a stone cold whodunit you know what i mean it's like at some target of interest to uh to the russians like you know they're not they're not thinking

8:45Tom Uren: they've been hacked by the dutch you know yeah and it was also really interesting that the they talked about the entire operational workflow basically being managed by ai and the people were not managing the operation. They were managing basically the clawed skills. So they were amending clawed skills to get them to work better. So it was managing the machine that runs the operation rather than managing the operation. And this struck me as this makes perfect sense when you don't care about any particular operation. If you just want scale and bang, bang, bang, bang, bang, let's go and go and go and lots of targets of opportunity, that's perfectly fine now for many state actors that makes no sense at all because getting detected is a problem but but in this particular case it is like the totally makes sense and so i guess it comes down to horses for courses um this also this approach also makes sense i think for a cyber crime actor where there's plenty of fish in the sea you don't care about a particular target you care about getting, or at least some of them care about getting lots of targets.

9:59Patrick Gray: Yeah. I mean, it's funny, right, that what you're alluding to there about certain threat actors not wanting to be, you know, attributed and stuff like this is the sort of mantra of the Five Eyes agencies. They've been an outlier in that regard for quite some time. And I sort of feel like with the adoption of AI by all and sundry, by all of the West's enemies, they're even more of an outlier now. And I just wonder how tenable that approach is. I mean, I think we're eventually going to have to have tiered operations. I mean, we already do have tiered operations, but I think if you want to scale up and keep up with China and Russia, you're going to have to do these sort of shenanigans and you're going to have to get snapped.

10:35Patrick Gray: You're going to have to get attributed. You're going to have to deal with China getting shirty about NSA shells and whatever. I think it's just going to become part of the game in the future. But I mean, now I'm not talking about what's in your newsletter. I'm talking about stuff that's possibly a couple of years out. But I'm curious to know what you

10:52Tom Uren: think about that idea yeah i think um again i think it's horses for courses like i think for a particular target say i don't know pick one out of a hat xi jinping i think you want to be careful around that because your the defenses will be so tight that your opportunities will

11:10Patrick Gray: be very limited you're probably not using repurposed dark web leaked exploit kits on xi jinping you know no dark sword or karuna uh off uh off a torrent for him yeah and i and i think

11:21Tom Uren: it really comes down to what you think your intelligence agencies should do. So the Western intelligence agencies tend to be more focused on what we would describe as, air quotes, legitimate targets, which is kind of narrow in scope.

11:37Patrick Gray: Well, it's not even about legitimacy per se. It's that these operations tend to be scoped around pretty tight objectives to begin with, whereas that's not how the Chinese do it. It's like, you know, you need this piece of information to support this program or this operation, you go out and get it, whereas the Chinese are just like, lol, that database looks nice. Thank you.

11:57Tom Uren: Yeah, and that opportunistic approach or that suck it and see approach, collect it all and sort it out later, I think that lends itself to the, well, we don't care about attribution particularly because we pick up this piece of information, that's great. If we happen to miss out for whatever reason, And well, you know, it's no great loss.

12:20Patrick Gray: To tie this back to the piece that we're actually talking about that you've written, you know, it does feel like we're seeing meaningful shifts in TTPs. Like we're seeing some pretty substantial changes thanks to AI. And I'm expecting that to continue quite a lot. I mean, if I had to pull a key takeaway from what you've written, I think that would be it.

12:44Tom Uren: Yeah, I'm interested in, it seems like these are complementary approaches that are right now mutually exclusive because you can't really rely on AI to be perfect. I kind of wonder what happens as AI gets better, less hallucinations, less what I'd call a mistake. and can you combine these two approaches like have a variety of malware that is also basically run sort of semi-autonomously with oversight rather than hands on keyboards i think

13:15Patrick Gray: that that's the that'd be interesting well i mean i think i think the reason and this is very like very much what you've written about i mean they've clearly designed these campaigns to fulfill their each of their objectives right so one of them is about sort of stealth or avoiding detection in the moment one of them is about dodging clustering attribution things like that I don't think there's any inherent limitation in AI tooling that would prevent you from being able to design a campaign that did both. Like, it's just that they don't need to cover the other side of each of those coins. You know what I mean?

13:47Tom Uren: Right, right. I guess the way I thought about it is you've got a limited amount of, like, you've actually got a limited amount of people doing stuff. And you can get them to use the AI to do one thing or the other thing.

14:01Patrick Gray: But the best thing about AI is it gives you scale. that's what I mean. Like, I think this is where we're starting, I guess is my point, right? I see no reason why you can't have, you know, an APT machine that just craps out basically limitless malware variants that are different enough that they're hard to cluster together. You know, I just think we could do this. We could see this.

14:23Tom Uren: Yeah, I think we can see it too. I think right now, the problem is there's just a bit of lack of trust. And I think they're using not the best of the best models.

14:32Patrick Gray: Rightio. Well, Tom, you're in. Thank you very much for joining me to walk through your work this week. Fascinating stuff as always. Really enjoyed it. And yeah, we'll be chatting again real soon.

14:43Tom Uren: Thanks, Pat.