AI Threats & Opportunities in Cyber Security With Material Security Co-Founder Ryan Noon

0:05So this week I'm joined by Ryan Noon. He's the co-founder and chairman of Material Security, the cybersecurity company making cloud-based email a safe place for sensitive data. He previously started Parastructure, which was acquired by Dropbox, where he was an engineering manager prior to starting Material Security. Ryan, welcome to NoPriors. Hey, it's great to be here, man. Always lovely to talk to you. Yeah, it's always fun to chat with you. So one of the reasons I'm excited to be chatting with you today is I feel like you have such a great perspective on both the broader security industry, various tech topics, et cetera, but also specifically how this all starts to tie into AI.

0:38and I know that in Material you were a very fast adopter actually of AI related technologies as the first sort of APIs really came out and you started playing around with them quite early and doing interesting things with them. Do you want to first talk a little bit about how you started Material and then maybe we can touch on how you started getting involved with the AI side of it? Yeah sure. So we started Material I guess 2016-2017 or so. I'd left Dropbox and you know was living in Europe and fell in love with all the election hacking that happened year. You know, that year it was pretty nasty.

1:10Like every random Gmail account kept getting like dumped on the internet. So I had an idea for like, you know, how to protect a Gmail account, you know, just an ordinary personal one in like a fairly novel way. I coded it. It shockingly worked. The Gmail API let you do it. I brought it back home and showed it to some friends. And we realized this is actually a special case of a broader way of thinking. Now, seven years later, it's a, you know, whatever cybersecurity unicorn thing. And we get to work with the coolest companies, you know, in the world by far. And the stuff that you get to do at the scale is just mind blowing.

1:46It's wild to think just where it started and where it's come. And what are the main products that Material focuses on just for the audience? They have a better sense. Yeah. So the broad thesis is basically we've all kind of got these Google and Microsoft accounts. Email is sort of where we started. But since then, we've kind of just went deeper and deeper and deeper into sort of everything that you can use a Gmail account or a Microsoft account for. The bread and butter of the business is selling to companies mid-size and up with these big Google Workspace and Office 365 deployments. The product has a bunch of different modules that are all kind of based around the main things people worry about.

2:29The kind of the first big product that you mentioned in the intro was people have years and years of sensitive information sitting in these accounts. If somebody gets into your Google account, they're just going to download all of your email and go through it later. And your whole life is in there. It's even worse in a corporate environment. And so that product, what it can do actually is finds sensitive stuff that's just sitting around, kind of just sitting in your inbox, your archive, whatever. And then it can basically redact it and then replace it with a clean copy so that if somebody gets in and downloads the whole thing, they don't get anything good.

3:04But then if you happen to need it, like I like having all this information at my fingertips, you can just press a button and do an extra face ID or a touch ID or, you know, more advanced policies and work, but just something that's easy for you, but hard for the attacker. So we started there and then we expanded into fancy phishing. You know, people can send you tricky emails and get you to do things and steal money from you. We expanded into account takeover protection, which is, you know, more of the things that people do after they compromise the account. And, you know, I try to reset all your other accounts and steal your bank account and all of that.

3:38Just the operative concept is defense in depth, which is just, you know, like just assume that the bad guy got in, like, what do they want? You know, like they got over the wall. There should be another wall. and a machine gun. You know, it's like history has all these fairly basic lessons about resiliency that never really always get applied the right way when it comes to computers. Yeah. So it's kind of like, I guess the part of the impetus was the 2016 election where, you know, there is all the things around the Podesta emails and Hillary Clinton and everything else. And the basic idea is somebody is able to hack your account, but it doesn't matter because your email is not accessible to them or the sensitive information that you designate.

4:15Yeah, I mean, it matters. But we used to call the company like seatbelts for email or whatever. It's like, it sucks to crash your car, it really sucks to go through the windshield. Google and Microsoft, you know, have a total duopoly on all of this. And kind of whatever little thing that they missed from a security perspective is, you know, world altering. I mean, there's a headline every couple of months, like every cabinet secretary just got their email hacked because all of the eggs were in Microsoft's basket, you know? And so we kind of just exist to fill the gaps in whatever doors they leave open.

4:51That's, you know, it's very fragile having a duopoly. Duopolies are stable in the market, but very fragile when it comes to security. Yeah, that makes a lot of sense. You were one of the fastest adopters, I feel, in terms of hands-on use of LLMs for security applications. How did you start thinking about the use cases where generative AI would be useful? The second you give a coder a REPL, we will start iterating, basically, right? And chat GPT, if nothing, was not the world's greatest REPL. So, I mean, we just started playing with it. And then we're like, there's a lot of security domain knowledge baked into this thing.

5:24It turns out if you feed, you know, precisely one internet to precisely a million GPUs, it picks up a thing or two about cybersecurity. And so, you know, it's, it's the kind of thing that obviously, like the bad guys are starting to figure out in earnest. And, you know, it's not like you can prevent this stuff and getting democratized. But we just we just, you know, you could do simple things like you could feed it, you know, like a bunch of, you know, raw email headers, anyone who's coded with these things, it's, it's like this weird wetware grafted into the middle of a computer. It's squishy and stochastic and parity, but you have to integration test and model around it.

6:03I think the analogy I used at the time is like Shang Tsung from Mortal Kombat. It has eaten the souls of thousands of security engineers. And so you might as well use it because honestly, there's a lot of just raw operational work that happens in security of just Like we need to, you know, rarify this signal, filter out the noise, and then honestly feed it through a human being who has some experience as to what the bad guys are trying to do. And, you know, it turns out LLMs are fantastic at that. And so that was the first use case that we really kind of productionized. But, you know, beyond that, it's kind of gone crazy.

6:39So there's a lot of engineering you have to do that. So it's kind of amazing because if you look at modern LLMs, they have this mixture to your point of sort of this deep knowledge base, which is the internet and to your point, sort of the souls of security engineers on the internet. And then, you know, it has this sort of chain of thought or sort of reasoning that is very useful to use in certain circumstances. Is there any data that you feel is really missing or a specialized corpus you need to provide or anything else that really helps from a security perspective that you, you know, you need to augment or fine tune or do something with?

7:06honestly like you know i i've seen a lot of you know startups starting from scratch here and and whatever and you know as an engineer like i know when i have headroom and honestly even in like gpt three and a half there was plenty to work with i'm seeing a lot of shovel selling obviously right now in the ai market uh and i'm seeing a lot of like you know i need to pretend that i have a moat so i need to you know fine-tune all this stuff and whatever whatever but yeah no i mean so many things that were very, very, very hard for computers, you know, 18 months ago, are very, very easy for off the shelf models.

7:43So like, I think, you know, maybe chew your food first security industry. Yeah. What do you think are the best application areas then for generative AI and security? Is it pen testing? Is it phishing? Is it something new? Is it some form of like supply chain yeah i mean it's obviously the uh the offensive side is what you're not supposed to talk about too much uh but obviously the bad guys were talking about it and security you know it does have this arms racy sort of aspect to it so like you know we need security lm companies uh because the bad guys exist honestly like the the order zero thing when i keep meeting with founders you hear this there's all these like kind of classic cliches in the cyber security industry like the cybersecurity skills shortage, like America needs, you know, to bring back the draft and make everyone get a security certificate or something.

8:35Okay. Like, you know, that you have like 90 % of a human that you can use for like a penny and a half, right? Okay. Start there. You know? Uh, and so like, there's just basic things like that, but it gets, it gets more interesting, I think from there, but like, let's go to Disney world collectively after we do that. and then we'll come back, you know? Do you see any CISOs actively using LLM tools today? Or is it still kind of early and it's like there's an adoption curve or is it going to just be in the hands of the vendors? Well, I think the best thing about the security industry is that there's also the security cottage industry of like, it's not the fancy security vendor who's, you know, buying the CISO steak and having them drive Ferraris around Vegas every August.

9:22it's like just a strong like security engineer who's just hacking something together. And so some of the best companies that I've seen, you know, are just that. And so you're seeing all these like there are cool projects out there. You know, I, you know, I don't want to name drop too many of my friends on this podcast, but like, you know, just like the stuff that Socket's doing, just like analyzing NPM dependencies, like, you know, even just like stack analysis, like looking for like, you know, hey, you drop sensitive information in the middle of your code base, like that's like such a messy, hard problem as any like computer science can, you know, person can tell you.

9:58And like, these things are pretty good at reading code, you know? So like all sorts of just basic stuff like that is starting to, to pull through. So. What do you think is the biggest risk or cyber threat from this technology? Oh, I mean, like it can be a human and I'm just, I'm just talking about the text models, right? So much of cybersecurity is just text. And there's nasty hacks that are reported where someone's voice was fake very convincingly and they made a phone call and blah, blah, blah. Humans trust humans through computers. That was, I think, the key mistake we made. Yeah, I guess there's a lot of APIs now that do voice cloning like LMNT or 11 or some of these other folks, right?

10:42And so basically, I guess the threat is that somebody voice clones and then they can use it to call you and pretend that they're your bank and ask for permission to do a wire or spoof you on the other side where. It doesn't even have to be that hard. Like as in the standard, like, you know, new employee joins company receives text message claiming to be CEO thing. Like it works at scale, you know, like so like it's the sheer amount of like, you know, you go and you go see these attacks at random bad guys. are sending to people. And they're not even using grammar properly. If all they could do was spellcheck the bad guys and that's all you were using, whatever off-the-shelf open-source LLM for, even that would make a difference materially on cybersecurity policy returns.

11:29How bad do you think this gets at some what time frame? So say it's three years from now and we're at GPT-6 or something. Do you have any predictions in terms of the sort of effective threat level or the capabilities or what might happen then? Yeah, I think we all kind of like wonder about this. I was talking to somebody from the White House who was like trying to figure out how to talk about security and LLMs a little bit. Like think the operative analogy that ended up helping was like Bronze Age versus Iron Age kind of thing. And that like, if you're a tribe or something and you have bronze weapons and your neighbor next door gets iron weapons, then like you're gonna have a bad time.

12:08Like you're gonna need to go and get iron weapons. And so all of this talk about like, you know, we need to, you know, airstrike the data centers and prevent it from being aligned or not aligned or whatever the current term is. Like that's like saying, you know, well, this super high grade carbon steel from space, you know, needs to be restricted. But honestly, like if someone's got iron weapons against your bronze armor, like good night, you know. And so these all on things, it's a step function. And like, you know, forever, often, you know, we used to whine that we only had, you know, 140 characters and not like flying cars, like technology does give you step functions every once in a while.

12:46And like, this is just that, you know, so it doesn't mean that like, you know, we're all doomed now. And I think we getting a, like a sense of the scope of the threat is really hard in cybersecurity, because you could be like, you know, hey, you know, we're a fortune 500. And we left the front door open for a year and no one walked in it. like hackers are fake cybersecurity industry is BS, right? Or you can be some like little no name company and just get run over. And you're like, the barbarians are at the gate. And it's like really hard to know exactly what you're up against. Right. But what's interesting is that like, automation, like it's like the you can be more human.

13:22And you can like one human can now supervise 1000 humans, you know, you don't need a room full of like, jerks trying to hack grandma or whatever, when honestly, like one jerk will now suffice, you know, with a for loop. Yeah, to that point, it feels like there's a few different types of actors in the cybersecurity world, right? To your point, there's sort of individual players, sometimes that's ransomware or sort of financially driven folks. And then there's state based actors, right? And it seems like some of the attacks we had a year or two ago on parts of our more physical infrastructure and supply chain may have been through state based actors.

13:55How do you think about that in the context of these things is that, you know, we must continue to invest in LLMs at scale as a broader national security side of things. Does it modulate your thinking at all? Yeah. I mean, fundamentally, like you have to invest in cybersecurity. Like my, my moral basis for cybersecurity existing is that is essentially like the, the waste heat of all other innovation in, in computing and information, which is like, you know, if a computer is doing something new for you that it wasn't doing last year, then like the utility of that will drive adoption. And then like cleaning up after it for whatever the side effects of that are, uh, is what, you know, essentially cybersecurity, you know, does.

14:38Right. And so we are the, the cleanup crew for all other innovation, uh, which is, you know, it's, it's a, it's a living. It's a, you can, it's an honest living. So whenever innovation happens, like the entire world will adopt it before they realize like, oops, it messes up democracy or like, oops, whatever, you know, like utility drives adoption, not safety, like welcome to earth, you know? And so, uh, so I think like the, on the nation state side, like it, it's, you know, you don't have to even be hyperbolic with like, you know, it's the atom, it's the whatever it's because like, you know, fundamentally like intelligence is now a commodity that we can arms race, you know, like weird, you know, it's, It's not, you know, like atomic power can arms race.

15:21Like, no, like intelligence itself can now go Red Queen. Yeah, that was the original premise under OpenAI, right? The concern was that Google and a few other folks had, you know, real advancements in AI and they were driving most of it. And so OpenAI, I think, originally was meant to be kind of a counterbalance to that. So there wasn't a single player that would effectively dominate all of AI. Or if it was, it'd be under this sort of philanthropic guise, right? And so it's interesting that even in the early days of this stuff, a lot of the emphasis was on this, let's avoid some over aggregation of power within AI.

15:57But if you have a lot of intelligence that is extremely online, you have a ton of power. And the West, I think, is especially vulnerable to this. Open societies, I think, are extra vulnerable when it comes to infosec stuff because we put it all out there. We adopt these systems. We open them up. We let the private sector totally handle them. And we are early adopters of every digital technology, and we are very happy to wave our soft underbelly on the internet as a society. We don't lock it down. How does that differ from totalitarian states from a cybersecurity perspective? You could literally, if you're like North Korea, you're going to say, you're all going to use this Linux distribution, but it doesn't support whatever I want.

16:41I'm sorry, we're an authoritarian state. Oh, well, what if I get phished? Sorry, that's not how bank accounts work in our country. You know, like, it's just like, you can control information, you know, you can't, this usually gets like you through the lens of like, social media disinformation, if you can, you know, regulate and lock down, you know, the entire social media discourse, then like, you know, what election is going to get hacked? And where would it get hacked? You know, but the same thing I think holds true for all of all of cybersecurity. The other interesting, you know, like way of looking at this, that's always kind of baffled me is that, you know, if cyberspace is a space, right, like in like US military terminology, it is a command, just like, you know, North Africa is a command, like cyberspace is a command, like William Gibson, you know, would be proud, right.

17:30But in this space, you are kind of on your own as an American. It's like if I was in, the military protects Americans and guards our borders, what does that even mean with cyberspace? I hope you're hired to see so. Is there anything specific you think the DoD should be doing relative to these sorts of threats right now? Or if you were magically in charge of it, what would you change or what would you do differently? I mean, they do a fantastic job in a lot of levels. Like, you know, it's like, obviously, like we were all had to, the Valley had to deal with like Snowden and everything, you know, 10 years ago and whatever.

18:12And I'm not, I don't need to take a side on that one. But the point is like, we have some pretty incredible people, you know, doing offensive stuff as well in cybersecurity and deterrence works pretty well a lot of the time as well, you know. So when it comes to LLM specifically, I think everyone is still figuring out what the hell is even going on. It's going to take them a while. I think you see DARPA doing really interesting stuff. There are interesting projects out there. But I think, and this is maybe a motif that I see broadly with LLMs, is unless you go super, super deep on this stuff, you kind of see everything through the lens of the popular discourse of ChatGPT.

18:52Like whatever, you know, the, the, the, you know, the New York times or whatever has said about chat GPT or whatever experience you had the first time you used it six months ago, when you were on the free version is how you see everything. And so there'll be like, we need to make sure it doesn't make stuff up. We need to, you know, have it generate, blah, blah, blah. It's, it's all kind of like order zero stuff. I think people have yet to realize that the computer can think in a much more salient way than it ever could before. And so I think people are still playing catch up. Yeah, that makes sense.

19:23Yeah, it feels very underappreciated. Yeah, I feel like in general people are viewing AI as this continuum where it's like it's a CNN, RNN, and now we have Transformers and it's just a straight line. And instead, obviously, it's a big discontinuity in terms of capabilities. And I think most people still don't think about it that way. Or at least I should say many people, particularly outside of tech. And I actually think it's underhyped in all sorts of ways, which may be a different conversation. Shovel selling is overhyped. But I think the thoughtful discourse on what our society will be like in 10 years is probably underhyped.

19:51Yeah, yeah, good point. So one of the big debates that people have in this area is what degree of things will go to incumbents versus startups. And in security, the incumbents are really strong, right? They are very good at buying things and bundling and cross-selling and sort of the traditional enterprise playbook, which parts of tech have sort of forgotten for a while and maybe are coming back to now that we don't have ZERP anymore. How do you think about the things that incumbents will do versus startups? Is there any room for startups right now on the AI security side? I mean, there's always room for startups.

20:22The cynical take here or like the the take I can give that is perhaps most informed and most cynical, whether this is whatever uninformed, informed pessimism versus whatever is is that basically, you know, in the cybersecurity industry, there's some basic economics. Right. There's if you care about this, like there's a great paper that is actually required reading for everyone who's ever joined material, which I've never enforced. but it's called the market for silver bullets, right? Like Ian Grigg wrote it. I think I've sent it to you once. And it's like fundamentally, you know, there's like markets for lemons and whatever, but there's markets for silver bullets, which is that like fundamentally there's the buyer, there's the seller and there's the attacker, you know?

21:03And so like the buyer cannot really be sure of the effectiveness of what they're buying and whatever, whatever. And so you can't really like look at a solution and be sure that it will save you, right? Like, you know, you could buy an insurance policy, And there's a truism that all cybersecurity products are just complex insurance policies or whatever. But the point is that mushiness exists. And so what has resulted in the free market here is these incredible distribution machines. You have, think like Cisco or Palo Alto Networks or even Microsoft and Google to an extent, where they have the Salesforce.

21:44course, they have, you know, the bundle, they have, you know, the big conference with all the glitzy stuff or whatever, right? But they don't really know, like, if you ask the product manager at that company or whatever, like, and they're being honest, like, they don't know what bad guys are going to be doing in five years, any better than anybody else does, right? And they don't know what's going to be effective. So why would they plant seeds from scratch, when they could just go harvest crops that are already growing and, and transplant them into their yard and water them with all these salespeople and all this bundling and all this market power.

22:18Right. And so these, these like paved roads, I think there's just a function of, of the extra, you know, like technological and product uncertainty that is just compensated for that. That risk must be compensated for with extra low market risk, you know? And so that's what you see, you know, like Cisco just bought Splunk, but Splunk buys things. The whole market just works this way. I think I wrote a blog post once where I called it the cybersecurity industrial complex, you know, and it's like their PE firms, you know, dressed up as innovators, blah, blah, blah. I was angry. I used to be very angry.

22:51But fundamentally, this happens. And so that means that we are kind of, you know, entrepreneurs, you know, at their worst. Like there can be new, great cybersecurity companies. There's still creative destruction that happens. You know, some of the best cybersecurity companies, you know, didn't really exist 10 years ago. And that's like, you can still build big ones like VCs, you know, don't stop, you know, like VCs, you know, when it comes to cyber stuff will, will like, you know, just go for base hits constantly, the worst ones, you know, and a lot of the best VCs like never, you know, make bets in cybersecurity, because, you know, at best, you're going to get a$200 million takeout to Palo Alto Networks or whatever, right?

23:26That's the typical outcome. But, you know, you can still build these big companies and, you know, people should still try. But, you know, there's, but that farm system is still active, like no one really knows, like innovation will happen. And if the market's big enough, and, you know, you don't, as a founder, you know, you don't want to stop the game on second base or whatever. And you want to keep going, those opportunities are there. And honestly, like discontinuities breed new companies, you know, and there's entire classes of things that are unnecessary and obsolete now. So much of security is is is emitting logs and alerts and then parsing those logs alerts again and aggregating them.

24:07You know, I spent a lot of time doing data infrastructure and analytics in my life before after my cybersecurity grad degree, but before I started using that degree. and it's just like serializing and deserializing data and parsing some old firewall thing from 20 years ago or whatever and like an LLM can just eat that, like depending on volume and all that stuff. But there's just like a lot of spend, I think is up for grabs as long as people have their expectations in the right place. I guess outside of material, like is there any larger scale security vendors that you've publicly talked about rapidly adopting LLMs?

24:47I know Material's been very fast on it. I mean, obviously, Microsoft had this top-down mandate and had a year on everybody. And so they've been making a lot of noise and marketing it. But, you know, and that's theoretically cool, but I don't know how, I haven't used it personally yet. But you kind of, you probably saw this pattern, which is that like, you know, kind of the growthy companies with the nerdy founders, like immediately started integrating this into the product, right? And then the like, youngish public companies that like, totally still got it, you know, would do like a thinner feature a little bit later, you know, the big fortune 500s are doing science projects, God bless them, you know.