Sponsored: Should your threat hunt become a detection?

0:03James Wilson: Hey everyone, I'm James Wilson and welcome to this Risky Business sponsored interview with Nebulok's head of detection strategy, Alex Hurtado. In this interview, Alex is going to introduce me to the new Gates framework that Nebulok has just published. Gates sits between the threat hunt that finds something interesting and the process of detection engineering to turn that into a trigger that might catch the same sort of behavior. happening later. It's designed to provide a structured way to answer the relatively simple question of just because I could create a detection for this threat, should I?

0:41James Wilson: Is it going to be worth it? As Alex points out, this isn't just about whether or not the work and the effort to create the detection is actually going to be valuable. It's also about ensuring you don't end up with a bunch of unmanageable detections, brittle detections, low signal to noise, etc. And we also talk about how you can take this Gates framework, the advanced version as Alex calls it, and turn it into a system for the ongoing evaluation and evolution of existing detection mechanisms. So I'll drop you here into the chat where Alex talks about how the framework came to be, working alongside her colleague, Sydney Marone.

1:21James Wilson: Enjoy. In the last year, Sydney and her team, they developed this agentic threat hunting framework that really helps a lot of organizations not have to start anywhere, not having to use any specific tool or software, just literally your command line and your GitHub repo. and it'll help you create sort of this structure, this memory through the command line that takes you that you can use any threat hunting approach like the peak or the Tahiti or even your own processes. But it lets you kind of go through that, the pivot after pivot after pivot, right? Because there's a lot of second, third, fourth order investigations or, you know, pivots into different things to help you answer certain questions that goes on in threat hunting that often is never recorded anywhere, right?

2:13It kind of disappears across, this context disappears across slacks or analyst heads. It's never written down anywhere. So the purpose of that threat hunting framework is one that follows sort of this pattern called the LOC. And we love acronyms here. So the LOC.

2:30James Wilson: Yeah, I've noticed that. So LOC, what's LOC? The L stands for learning. So when you start a hunt, you have to learn and gather context, whether that's from a threat intelligence article, a headline, maybe an alert, maybe an anomaly. And then you go and you form a hypothesis about that intelligence, about a specific behavior, hopefully. And then you go and you check it, cross-check it against your environment with targeted queries. Regardless of what that is, this framework is agnostic to what tools, what sim you use to do so. And then you test that. You check it with specific queries. You keep those records and findings and lessons so that it essentially builds this knowledge base.

3:13And in the future, even as your team maybe goes off and does different things, they churn all of that record of your previous, you know, threat hunts don't leave with them. But the million dollar question is, you know, should that hunt, should whatever queries that you kind of put together for that hunt, could they turn into an active detection that you can deploy and it can kind of look back across all of the data within your organization on an ongoing basis, maybe schedule it every hour or maybe down to five minutes if it's like absolutely critical? And right now, the answer to that question is it kind of depends on whoever's most confident that week.

3:57It's kind of like a gut feeling. No one really knows quite yet. Maybe your senior analyst knows, but maybe your junior analysts, they're like, oh, I'm not quite sure.

4:09James Wilson: Right. So Gates takes what would otherwise be a hallway conversation, a Slack conversation, or just your own internal monologue, I guess, should have really turned this into a detection and actually gives you a structured way to think through it. But let's start with the basics. G-A-T-E-S. What's all this stand for? So the G in Gates is what's called generalizable, even though I kind of wrestled with myself. I'm like, is that really a word? It doesn't sound really nice. It doesn't flow off the tongue that well. But I don't know. I didn't want to say generic. So generalizable, what I mean by that is, is it a repeatable pattern and not a one-off, a behavior that can be used across ransomware groups, right?

4:49It's, so to speak, fails when it's very, very ultra specific, and it won't survive multiple other instances of this.

4:58James Wilson: So like, so if there's like the most minute change in a threat actor's techniques, and the rules no longer applies, it's going to fail that first gate. Is that is that right? Well, pun not intended that first step in the gates framework. That's exactly right. And what detection engineers often refer to David Bianco's pyramid of pain, to kind of explain the difference between behavior-based versus IOC-based detection engineering. Because the idea behind the pyramid of pain is that the more specific that you detect something, the easier it is for an attacker to actually change it, right? Because they could just switch their phone number.

5:32They could switch their IP. They could switch their hash. They could switch their domain, right? So the more like up you go level in the pyramid of pain, the less, the more difficult that you make it for an attacker to operate.

5:46James Wilson: Right. Okay. So the first step is checking that there is some degree of, I guess, repeatability, modularity, or something that's going to make this durable and longer lasting than just one specific use case. What's the rest? Where do we go from there? The next one is additive. So does it fill a coverage gap today, like with respect to your detection engineering program? Or the fail would be like, if there is something that's duplicate, it? Could you merge it into something that you already have? Could you add an additional filter in your logic where you can just make it be truly additive? Yeah.

6:25James Wilson: And maybe to just take us a little bit out of the abstract and into the practical, I think generalizable makes sense. I get it that that's when you're looking for something that is repeatable and usable and not too specific. But additive, this might be a good spot to actually, like, let's go through an example. Like, what would be something that might already exist in your detection space that you could look at and say, okay, I can add to that for a specific example you had in mind? This is going to vary depending on folks' coverage. And if you look across the MITRE ATT &CK framework, there's going to be things that you have detection in place for, things that you don't have.

7:00And if a novel behavior comes on your lap and you don't have it in place today, for example, you can do a security tool disablement right so i'm oftentimes an adversary in order to delete their traces to delete their footsteps they'll try to uh silence your edr there's a ton of edr silencers now we're seeing or they'll just kill the process they'll stop the service they'll even unload its driver and if you don't have anything like that in place like you probably you absolutely probably should do that or uh or if it's like if you have something similar like a lot of folks have detections, like, oh, no logging has been detected from this source during a period of time, they'll say, oh, I have that already.

7:49Like, no, not necessarily. No, not quite, because you're not seeing if it stops, if it kills, I don't know. Like, you have to kind of build the world around it as well.

8:01James Wilson: That makes sense. Okay, so we've got generalizable, we've got additive. Now, the next one is tunable. So I imagine, and this is all about signal to noise. This feels like one of the most crucial pieces because really the ongoing value of any detection engineering work that you do has to have a pretty rock solid high signal, low noise ratio. But I'm curious, like what's the actual thought process to think through this and actually work out is something tunable? The easiest example to use is when you create new users or you delete new users, AWS users. These are very common scripts that people use in the day-to-day.

8:40We have to create new users all the time. But from an adversary perspective, they can also create new users for themselves to do their own operations. So in efforts to understand whether tuning entropy even exists, you have to basically be able to derive what normal is versus what an attacker behavior is. Right now, it's kind of like a yes or no question. But if you can validate this with data, this is going to take you a whole step further because there's different calculations that you can do that can actually tell you your false positives ratios against a rule before you deploy it to production, which is why there's actually an advanced version of the gates that can help you do that.

9:29James Wilson: This is good. I think the right sort of framing is forming in my mind. And I was thinking as you were explaining that, that in a threat hunt, you're going to be pouring over logs and trying to manually determine, does this look right? Have we seen the right, I guess, quantity of users created? Is there some weirdness in the pattern of the usernames or whatever else? That would be the threat hunting side of things. And then of course, you get to the end of the hunt and you say, well, that was great. We found that there was some you know malicious users being created do we and the get that this gates framework we're talking about sits right at that point of answering the question okay the hunt was successful and and resulted in some knowledge but now do we turn this into a detection and if i'm understanding this right the key step here in tunable is actually stepping back and saying you know it's one thing for the human to go and you know pour over all the data and look through all the logs and make their own judgment.

10:18James Wilson: The tunable step is, are the right properties or parameters available to actually dial this in and say there is an automated way to detect the same sort of malicious behavior that would otherwise only be, I guess, intuited by the human when they're looking through the data? Am I following this right so far? Yeah, that's correct. The tunable aspects as well has a lot to do with allow listing. So if you look at an attack and you try to filter out, If you could tell an attack from normal and the way to do that is by filtering out what usually triggers on that behavior from like that's – but that's just benign.

10:59So if someone, let's say, you know, Command EXE or PowerShell is spawned by one of the many drivers that there are out there now, like there's like lists of hundreds and hundreds of RMMs or Lulbans living off the land binaries that spawn PowerShell or that spawn Command EXE. Many times, advanced developers or just legitimate housekeeping operations do these things. So if you could say, okay, it's okay for this list of people to do this behavior or these applications like CrowdStrike and Jamf or other security tools. It's okay for them to do it. They have a certified signing entity that they do this work all the time.

11:49But if anyone else does this, it's not okay. And that makes it tunable.

11:53James Wilson: Right. Okay. So it's all about the parameters that are available to determine automatically in that detection sense what good versus bad is. So if I look through the framework again, generalizable, additive, tunable, these three feel like, these three are like the tests of the framework. When we get to the next two, the exposure tested and sustainable, these feel like it's a bit of a pivot in the framework towards you then have to say, well, you know, can it be bypassed. Is that the next step? And am I right in sensing that there's a bit of a shift in the, I guess, the mental model of the framework now towards, okay, this looks good and maybe is worth being a detection.

12:30James Wilson: Now, just how good can it be? Yes. This one is what is going to bring your detection from just okay to super resilient because we're now putting our adversary emulation hat on and we're making sure that you cover more than the obvious path, right? Are you covering just one path or all of them? Like if you're disabling your EDR tool, you're covering all the three different ways. Maybe you're looking for if the service is stopped, if the driver's unloaded, and if paired with, I haven't seen any logging from this log source. That's three different ways you're covering. And the detection logic would be, you know, if this happens or this or that, right?

13:14So you're covering more, you're covering your bases more than the obvious path. And it's even better in the advanced version if you actually run adversary emulation and run that in like a, hopefully it's a very like safe tested, like test environment, you create real telemetry. And then if that reveals a gap, that gives you real data to work with rather than just throwing it into clod and asking it to build a rule for it.

13:41James Wilson: Right. The last step in the Gates framework is sustainable, right? Can you reliably see it? Is the upkeep worth the risk, right? This is the, I guess, ultimate question of, is this actually going to be worth it going forward? Is there more nuance to it than that? It's also a lot. This hits on something that detection engineers and threat hunters too have to deal with a lot. And it's, can we reliably see it from a telemetry perspective? is the data always going to be there for us to see? Because sometimes it's not. And this is a cost question. This is a leadership question all the time as well because we're always trying to do more with what we have or even less.

14:17And so it's sometimes what we want to test, we require more data, more another log source, maybe network traffic, which is really heavy and expensive. And so if they don't want to invest in buying you this extra data, like data source or more space to put it into your sim, then maybe it's not worth it. But that's where you kind of have to do this. Well, if the risk is high enough, you have to make more of a case for it. And if it's not, maybe the risk is low enough where this is maybe very, very unlikely for this to happen, then that's when you kind of have to just throw the towel.

14:59James Wilson: okay interesting because i when i first read through this i had assumed it was sort of the other end of sustainable which is that you know the alert that it's producing will be useful that goes in the right place it gets seen by someone but it sounds like it might be a bit of that but it's it's also the inputs that are required right if this if you're let's say you're only gathering netflow data today which is you know highly reduced and and relatively cheap you've built this amazing detection but you say well now it requires like a span port and we're going to have to run PCAP and see every single packet and frame going through the network.

15:30James Wilson: And that's a huge increase in data. That sounds like that's the sustainability question that you're asking here. Is that right? It's both telemetry reliability, but it's also maintaining it. If it's too much work to keep this going, if it's like I'm constantly having to add people to an allow list, it's not worth it. It's like there's no way that the amount of work that it requires you to keep this detection healthy is just not worth the squeeze. It's also the maintenance aspect of it. Right. A couple of times as we've been talking through that, I've picked up this really excited little sparkle in your eye when you talk about, yes, and then there's the advanced version of Gates.

16:05James Wilson: So let's go there. What have you layered on top of this to create, I guess, to take this decision-making framework into a more advanced form? Yeah. So the base is you can run it on a napkin, but what's really going to up-level this hunt promotion validation is if you can really back this up with data behind it. Because we all know anything without data is just an opinion, right? Right. And everyone's got opinions. Everyone's got opinions. But this can really help drive home whether you promote a hunt into a detection when it actually proves itself in the wild. So instead of the G being generalizable It does, the generalizable still also works for the advanced, but it helps you demonstrate it by testing it if it's like cross fleet.

16:57Can this detection, can this repeatable behavior work across Windows, Mac OS, and Linux, right? Or what do you need to do to change it accordingly? Do you have, every single time you also start building a behavior, Most detection engineers think about these companion rules that could also kind of work. And it just starts kind of getting your gears turning. So can this be accompanied with its companion rules to generalize this technique even further? So it's cross-fleet and companion rules. The A is for actionable. So a lot of companies and organizations have SOAR strategies. or have a sore maybe that's by way of a product maybe they built it in-house and they probably have you know advanced playbooks automated playbooks that that just trigger off of in the event of certain high risk alerts so can something be done if if this hunt finding came about right if this detection came about is there a validated and automated playbook that can be configured into it can you even do something about it because if the answer is no maybe that's maybe you shouldn't be building a detection out of it because it's like what are you going to do about it then right if it's just informational like okay so what yeah yeah 100 it's it's um super valuable when you've

18:20James Wilson: got a structured way to turn the just because we can should we question into uh less of a hallway or a slack conversation but also but now a really structured way to not only do this at a single point in time but to keep asking those same questions to continue to evolve and i guess drive up the value of detections. Alex, thanks again for spending the time with me. It's been great to chat with you. Thanks so much for having me.