How One Hacked Library Can Take Down Thousands of Companies | Feross Aboukhadijeh (Socket)

22:16Yeah, that's a really great question because we're seeing so many non-technical folks now of Vibe coding and building apps, and it's awesome. I mean, I love to see it. As someone who taught computer science for a number of years, I really love to see more people coding, and I think it's going to help more people get into the field and just empower people. It's awesome. That said, like we've been talking about, developers haven't necessarily done a very good job of assessing risk of third-party code. And so how are we supposed to expect someone in sales or marketing, no offense to those folks, but to understand what this code is doing, right?

22:50And so we need guardrails that are in place to sort of let them loose and to let them have the power of all these tools without having any of the downsides. And so the simple solution is something that we've, again, I feel like I'm just an ad for Socket, but we've been thinking about these problems. And so I do think we have a lot of the best solutions here. There's something called Socket Firewall that I mentioned a moment ago that you can put it in place, and it will make sure that nothing the agent does as it relates to third-party code is unsafe. So the agent can't go out and fetch this untrusted package that we've been talking about.

23:29but it can't go and install this random library that nobody should be using. It's going to guard against that. And that's really powerful because it means you also can let your teams experiment with whatever AI tools they want because we're doing this at kind of the network level. We don't need to be hooked into every new AI tool or have like a approval. You don't need to approve what tools your team is using. You can kind of put this one control in place, and then it doesn't matter what AI they're using. It's all ultimately going to transit the firewall, and we're going to be able to see it and act on it.

23:59Anne Dwane:Great. Okay, great. Another new thing since you started is LLMs. And Zach Stone at Google had asked a little bit about why haven't we seen more bad stuff via prompt injection? And is that something you think about? Yeah, I think attackers always look for the easiest route into an organization, and they're really great at hunting for soft targets. This is one of the things, by the way, about security that makes it fun is that it's this cat-and-mouse game where there's attackers, there's defenders, and when an attacker improves, a defender can respond. And when a defender improves defenses, the attackers respond and go somewhere else where the defenses are lighter.

24:38And so what I think what we're seeing with prompt injection is that it's just not the easiest way into organizations right now because it's sort of finicky to get them to work. It's often unclear how to get data out when you've done a prompt injection. And I think things like the supply chain have just become like why prompt inject one organization when you can take over Axios and have tens of millions of installs. I mean, the attackers that did Axios, they're called Team PCP is the name of the group. And they claim that from, I don't think it was another one of their compromises they did earlier, but they claim just from that one compromise they were able to steal 300 gigabytes of credentials.

25:23And those are compressed credentials. So imagine a text file compressed 300 gigabytes. You're talking an untold number of organizations' tokens and keys. So when you have things like that, I don't think it's going to be like this forever because, as I mentioned, it's really an evolving system. But I just don't think right now that's where the easy opportunities are. Got it.

25:47Anne Dwane:And to that end, if it happens and either Socket notifies someone or they hear that they have had some kind of supply chain attack, what should they do? So the best thing is to not be affected in the first place if you can. So definitely get these tools in place. We already talked about Socket for GitHub and the Socket firewall. There's other things you can do as well. I just want to mention to make sure for completeness people know about real quick. So people should definitely go and set up a minimum release age in their package manager. This tells your package manager to not bring in code that's newer than a certain number of days.

26:20Anne Dwane:What's the threshold would you recommend? I would say at least 24 hours would help a lot. A lot of the things that Socket catches, we catch them in a few minutes. And then we work with GitHub and we work with NPM to get that code taken down to protect the entire community, whether or not they're Socket customers. So if you set your minimum release age to 24 hours or to three days or something like that, you will downstream get the benefits of the work that teams like Socket are doing, even if you're not a customer. But ultimately, that's a pretty good defense, but it's not perfect. There's malware that slips through that lasts longer than three days.

26:54The other thing that's interesting is there's a tension between, you know, when you set that, let's say you set it to seven days, right? You want to be really safe and you want to use any code newer than seven days. Well, now what do you do when there's a vulnerability? And obviously, as we know, Mythos is finding vulnerabilities in a lot of open source code. The other frontier models are doing the same. And so we do need a method to bypass that waiting window and actually update our code to stay safe from vulnerabilities. And so it's this tension, direct tension between, like, as an industry, we've been trying to get people to update faster and faster.

27:25Like we always say, update your iPhone. You know, don't run old software. You're going to get attacked. You're going to be vulnerable. And so after like a decade of trying to get the industry to take this seriously, we've gotten people to realize it's very important to update just in time for supply chain attacks to come around. And the faster you update, the more you're running code that nobody's vetted, nobody's looked at. You don't necessarily want to be running code that was published four hours ago that no one's had a chance to look at, right?

27:47Anne Dwane:Yeah. And the malicious actors are bad. Sometimes they can hang out for 24 hours, and then they release their code or action or whatever. And let's just talk about that. So many of the folks on the call probably are working in organizations where the CEO is saying, lock it down. We've got to get serious about this. But open source is really important. So could you help them navigate those waters? Well, I mean, I'm glad to hear there's CEOs that are saying, lock it down. I've heard it's usually the opposite. We've got to do AI everything, AI everything, right? and pushing people to kind of just go for it and ignore the costs.

28:24So if there's CEOs saying that, that's good news to hear. But no, I definitely think we're seeing more of that. I think it's definitely become a board-level concern in a lot of the conversations we're having where boards are saying, you know, why were we affected by this? What are you going to do to make sure we're not affected by these kinds of things in the future? Or one of their other portfolio companies was affected, and they want to make sure that everybody's protected. So we've been seeing a lot of focus on that even from some CEOs.

28:52Anne Dwane:Got it. Yeah. So we talked a little bit about what startups can do or individual developers. In a large organization, maybe that's been around for a couple years, they were pretty optimized for a SaaS kind of environment. Is there any suggestions you'd have of what they should start, stop, or continue doing besides using Socket and other things? Right. I think the most important thing is, and again, we're talking about supply chain here since that's the topic. There's obviously a lot of other aspects to security, and that would be a longer conversation. But when it relates to supply chain, the number one thing is you need to understand what open source code you're using.

29:27What is part of your supply chain? And by the way, supply chain is actually more expansive than just open source. Right now, open source is the one we're talking about. But even things like the APIs that you use, the various vendors that provide you software, these are all dependencies. Effectively, it's other people's code, but it's your problem. It doesn't really matter. if it's their fault, if ultimately you're the one who has to explain to your customers why, you know, why you were breached or why you had an effect. So you have to think of really, what is all the code that could affect me?

29:55And you need to have like an inventory is what I call it. Just you need to know what that code is, right? And so step one, figure out what you're using, right? Something Socket can help you with. But as a baseline, you need to make sure that all your projects are using a lock file. This is the default in pretty much all package managers today. But what a lock file will let you do is it locks down what versions of open source code your application is using. If you don't have a lock file, you're sort of getting whatever happens to be like the newest thing that's out there. Or there's usually like these loose version ranges where you're saying, I'm okay with anything that's, you know, version 1.2 point whatever.

30:34And so that means that when something is compromised, the next time one of your developers runs an install or does a build, you're just getting that new code. And there's no way for you to know. like it's very time dependent almost like you might get unlucky and and do a deploy you know when something is is malicious or vulnerable so you want to be very intentional and a lock file is just like baseline and everybody should be doing this if they're not like go do that today it's not hard that now creates a definitive inventory of like what you're using and then you need some type of guardrail using that inventory to say if anyone's changing that inventory and bringing in new code we need to be able to decide whether we want to allow it or not And so there's many ways to do that.

31:13Obviously, again, Socket has a way to do that. But you need to know it needs to be an explicit decision, right? It needs to be some type of tool informing whether or not you want to do that. I think that's just step one. And you'd be surprised the number of companies that still don't have lock files. They don't have these kind of basic things in place.

31:33Anne Dwane:Great. Well, let's pivot a little bit and talk about your business. So you're a developer-led growth company, but you also are now doing a lot of enterprise sales. And I think there's 20 ,000 organizations, including Vercel and OpenAI. Almost 30 ,000 now. 30 ,000. OK, good. I can't even keep up. It's definitely inflecting. Can you tell us a little bit about what you've learned about developer-led growth and then the move to enterprise? Yeah. So when we launched Socket, we were just a GitHub app. There was no website. There was no dashboard that you logged into. And it was literally designed to be something that a developer could just install.

32:07and so we've kind of always had this ethos of like we just want to put the tools in the developer hands. There was no billing or pricing or contact sales or you know book a demo like none of this stuff and I think that I mean that came from like my and the original team's background as open source maintainers where we really care a lot about making things that developers want to use and so you know we that's what you kind of do as an open source person is you make things for other developers right and so that's kind of where we how we built the product And that also showed up in the way that we provide our data.

32:38So you can look up any open source package on Socket, and we give you everything we know about it. And this is our crown jewels, really. This is what we spend an inordinate amount of tokens and compute to analyze and to determine. And this is what we sell to our enterprises. But we also provide it for free on our website. And this is great because it means people can come and search for their favorite libraries and see, okay, what does Socket know about it? Does it find anything interesting? You can click through and see, like, the recent malware that we've detected. That makes it really concrete for developers because a lot of times security can be abstract.

33:12You know, it's like this Hollywood movie kind of scary stuff. But, like, if you can actually, you know, click in and see, oh, my goodness, this is a library that I use, like Axios, for example. And you can literally see the code of the malicious version and see, oh, my goodness, like, look at what I can read. I can read the code and see, like, this is what the attacker is trying to do. They're trying to steal this and this and this and this. And you go, oh, my goodness, I'm running these commands every day. And who knows if one of the times I do that, I'm going to bring in some of this code. And they can actually see examples of it.

33:42So that was very helpful to us in the early days to just teach people about this problem that nobody, I would say not nobody, but very few people had as a top of mind problem. And so we were able to raise awareness and really educate people on it. And for a long time, the free version of Socket was all we had. We didn't have any type of billing or upsell or enterprise plan. And then, of course, with our first customers, the first enterprise sales, we're really just selling the free product with support and extra assistance to our design partners. And we sort of built into an enterprise plan and started building newer and newer things that we only put into the paid plans.

34:25But we still kept that really generous free offering for our users.

34:29Anne Dwane:Yeah, that's incredible. And who is the buyer at the organization that you usually work with? Usually it's the security team. So it could be the CISO or the head of application security at a larger company. And more and more, though, we're also starting to see the folks that are in charge of AI rollouts, like AI coding tools or like innovation offices and bigger companies. Like they might be responsible for rolling AI out, but then doing it in a safe way and making sure that doesn't compromise the company. Yeah. And so, you know, they're very interested in how to put guardrails around the AIs.

35:01So they can kind of let their employees run free and, like, use these amazing tools, but to do it in a way that's where the risk is contained.

35:08Anne Dwane:Got it. And what have you learned about moving from founder to CEO now with a sales organization? What have you learned there? Oh, man, it's been a huge journey. Because, you know, I've been a developer my whole life. I've been an engineer, you know, and, you know, very much into the weeds of code and like writing, trying to write good code and almost like craftsman of code. Right. That's kind of how I always thought about myself. And then to see to find my job changing so much, you know, as we've been doing the company, it's been really quite a journey. I like sales now, actually. So I came into this thinking, you know, you know, sales is is is bad.

35:43It's dirty. You know, it's like this, you know, the product should speak for itself. I don't like it when I get cold calls from salespeople. I don't like it when I have to deal with salespeople. That's kind of how I came into it. But really all sales is, is just like if you have a good product, you should want people to use it and you should help them understand why the problem is important and how you can help them. And I think there's a way to do sales that is very true to that engineer spirit where you really are consultative and you'll tell them honestly where the product is. good and where it's not.

36:18And it's really about fit and about being a consultant for them. And so I've just, I love it now because I just get to talk about like the problem and I get to, you know, my passion for the problem comes through when I'm talking to people. And, you know, I think it's important to balance it with the engineering first culture and you don't want to ever have one dominate the other. So there's so many companies I've seen that have just amazing engineering, but they can't figure out how to get the world to care, right? I want to maximize impact of the work we're doing, because I think it's very important.

36:49And I really want open source to have a bright future. And I don't want these attacks and these risks to make people, especially with AI agents coming out, and how easy it is to just write code, right? I don't want open source to die. I think that would be really bad for the world. And so this is a way for me to have that impact. But I've seen so many companies where they have technically excellent product, but they just can't figure out how to get anyone to care. And so that's a way to die, right? The other way to kind of die as a company is to go too far into the sales side and lose track of the fact that the product actually has to be good, right?

37:28You can't just sell. And so I see that happen to some of our competitors. They had a great product, but then they put in a sales CEO that under-invested in engineering, and they just kind of went hard into sales. And that meant that they grew really, really fast, but then they hit a ceiling and they've not been able to get above it because now customers have realized the product kind of hasn't improved in five years now or three years.

37:49Anne Dwane:They've under-delivered versus expectations or something like that. Yeah. I think they're both awesome. Like sales is awesome. Engineering is awesome. And marrying the two and being like true to our engineering spirit, but also not being afraid to do sales and to like actually go and win. It's like this, it's been this nice balance that we've been able to strike at Socket. That was great. Right. You know, George Foreman, who did the George Foreman Grill, which is like one of the best consumer products in the world. He always said, I don't know anything about sales, but I know how to talk passionately about a product I love.

38:19Anne Dwane:And that's kind of what you do. Pretty much. Yeah. Yeah. Well, let's talk a little bit about Socket engineering and development. It seems like your product has really benefited from the improvements in the frontier models. And can you talk a little bit about how you surf that wave and think about that? Yeah, so when we first built Socket, our insight was we think there's going to be a lot more supply chain attacks, and the scanners that focus on the known vulnerabilities are just missing a big part of the picture because they're literally not looking at the code, right? Just to really drive the point home, the way vulnerabilities work is there's this government-run, federal government-run database called the National Vulnerability Database, and it's kind of a clearinghouse for all known software defects.

39:02and people push those reports into there. And then all the vendors do is basically pull down from that database and tell you if you're using any software that's on the list. So it's a very simplistic kind of thing, right? And so we thought, no, if we really actually analyze the code, we're going to find a lot of stuff that people don't know about today, and we're going to find the really interesting threats, like these nation-state attackers, this malicious code, and that this is going to get worse, right? So that was our insight. And the way we were set about solving it was using our experience as maintainers we thought if we see certain signals in the code, then we know that that's a sign that something about this package has changed that makes it riskier to use.

39:42And so the way that we thought about it was, say, you know, you use iPhone or Android, whatever, you know, whatever OS you use, but on your, on your smartphone, when an app updates and it wants to use, let's say, new functionality, like your camera or your contacts or your location, it doesn't just get to use it because it wants to, right? It has to prompt you. Like the OS actually enforces that significant changes in risk have to be approved by the user, right? And unfortunately, open source and the way we code today doesn't have that type of permissions model or capabilities model. Right, yeah.

40:14And so our insight was if we could just find out when that type of change happens in code that we depend on, we would be able to spot a lot of these attacks. And that was true. That worked. But what happened was it was too much data. It was actually overwhelming to most of our users. There were certain really technical teams that really loved to know every single time one of their libraries, you know, oh, it's making a new network request here. Like, that's really interesting. Why is it doing that? And to be honest, that's the kind of stuff that interested me because it might not be a security issue, but it might tell you something interesting about, you know, again, because it's all part of your application.

40:50Right, functionality or whatever. Why is it doing that? Is that going to affect our users? How does that affect the trustworthiness of this code? So it's all interesting stuff. But what we learned was only the really most technical teams who had really solved a lot of their other security problems and had time to spare to look at this stuff found that level of granularity useful. And so what really made Socket useful to a broader set of people was AI models and frontier models. because we were able to take this just amazing set of data that we were producing about libraries and put it into the models and have them synthesize it and really boil it down into something simple that anyone can understand.

41:28So really, you go from all these signals, you go to really almost a Boolean, should I allow my developers to use this library or not? Like, yes or no, right? Really simple. And that's what people really wanted. And before models, there would have been no way for us to do this at the scale that we needed to do it at across every single library in real time. But with models, it's almost like you have this infinitely scalable army of interns. It's actually gotten better than interns now. I'd say they're like PhD level. When we started, we started this with GPT-4. We got early beta access to GPT-4.

42:02And that's kind of where this first started to really work because GPT-4 was just the first model that was like just good enough to really start to do this work at scale. It had a whole bunch of false positives. So we had to include humans in the loop to kind of approve things. But it was very you could see that it was it was going to be a big deal. And yeah. And so it's like now it's obviously gotten way better and better. And it's it's just amazing now we can catch these attacks in like minutes within minutes of that code being out there. Our system is kicked into gear, and it's just pulled apart the code, and it's found all the problems, and it can make the assessment really, really quickly.

42:39Anne Dwane:That's amazing. And you've made the point that it used to be that the kind of the attackers had all the incentive, and the defenders were like saying, hey, I have small resources. How do I defend against all these things? But now with agents, AI agents and the LLMs, actually you can afford to defend more, right? And certainly Socket can afford to scan the environment more. And that's absolutely true. And I think that the thing about security that's challenging for defenders is that the defenders have to be perfect. They have to guard every single way into the organization. Attackers just have to be right once.

43:14Like, they just have to find one weak spot, and they can get in. So it's a very asymmetric problem space. You have to be perfect as a defender or else the attacker gets in. And I think AI, the big question people have been asking is, like, is AI going to help attackers more or is it going to help defenders more? And I think it really depends on the specific part of security that you're talking about. So for things like phishing, I think AI has helped attackers quite a lot more because they can write these amazing phishing lures that are customized to exactly who you are and everything you've ever written or done online and everything about your organization.

43:47There's a lot of, you know, open source intelligence out there that they can use. But for defenders on the supply chain side, I think it's actually been a net benefit to defenders because you have this world where people are just pulling in code and never looking at it. And now you can actually have at least agents looking at it through things like Socket or others.

44:07Anne Dwane:Great. And let's talk a little bit about a question we got relating to private information, like from a healthcare founder. How would you think about partitioning that confidential information? Is there anything unique about that or any advice you'd have? That's a very deep question. I don't know if I can give a super short answer about that, but I think if you're in the healthcare space, I think you probably want to do your first security hires sooner rather than later. There's a lot more regulations around health data. You want to think very carefully about allowing models to access that data and giving them tools that can pull information on various customers.

44:52I think one of the things people need to keep in mind with putting an agent in front of anything like that is that that's where prompt injection really becomes a serious issue. I don't know if you saw recently, Instagram rolled out a support bot that helped people get access to their accounts. if they lost access. And this is important for them to do because Instagram notoriously has really poor support around getting access to accounts. And so they were trying to roll this out to help burn through their support ticket queue a little bit faster. But it turns out it was trivial to convince the bot to just add any email address to anyone's account.

45:33And then you could then go through the password reset flow and reset their account and get into their Instagram account. And so the way to think about these things is if the agent has access to a tool, you should just assume like all your prompts and your instructions that you're giving to the agent can be ignored or can be bypassed through a clever prompting by an attacker. And just assume that those tools are directly accessible by your users and that the agent that's mediating it and trying to sort of be that human, you know, support like intelligence, just assume that that can be tricked, right?

46:01Just like you would for any human in your organization. You know, like you have to assume that even your own support agents at your company could, you know, could be tricked or could be bribed as insider threat. There's a lot of similarities, actually, between the way we think about the AIs and the way we think about the risks of humans in our organizations as well.

46:20Anne Dwane:Amazing. And let's talk a little bit about, people are always curious about how modern engineering organizations are constructed and organized. Is there any learnings for you about building a really AI-native company? That's a good question. We had one founder recently post, he's like, it's not a democracy anymore. It's like, you know, because a lot of people, I think what the sentence was, people were testing a lot of different tools and bi-coding and kind of lost a little bit of the focus on, okay, what are the core metrics and things like that. Yeah. I think that there's definitely a risk that people can go too far with this sort of AI native stuff.

47:01So we've been at Socket, we haven't been pushing it on people in any way. Like I haven't, you know, some CEOs are telling everybody, you know, we got to go all in on it. Token Max.

47:11Anne Dwane:Yeah, exactly. We've sort of been very organic about it, like letting developers experiment with it at their own pace. And it's naturally become something that, you know, most people on the team are using. And I think that's, at least for me, that's the way to go. I don't think doing anything unnatural, you know. But, yeah, I think so many thoughts on this. It's a very big topic. Yeah, we'll do another podcast on engineering works. And we're coming up on time. So is there anything I didn't ask you about or anything that's top of mind for the future of Socket that you want to share? I just think it's such an interesting time to be doing a cybersecurity company and also just any company, really.

47:53What a blessing it is that we get to live in this interesting time and this interesting moment. It's such an exciting time. Every day is different, and the pace is so fast. For all the founders that are on this call, there's so much opportunity. I just feel really lucky to be part of all this.

48:15Anne Dwane:That's great. Well, and thank you for all you do to keep the companies safe and individuals safe. And we're going to post some links, not just to Socket.dev, but also to some of the videos you've posted as you're discovering some of these supply chain attacks in real time and advising people what to do. And it's really fascinating to watch. So it's like reality television. Sometimes these attacks are just like right out of a movie. It's kind of fun to get people want to go and check out our blog at Socket.dev slash blog. Some of these attacks, I mean, they're like every day, right? You can click on it, and you can see it is really very interesting.

48:51Anne Dwane:A thousand a week you're detecting or some crazy number, right? Yeah, yeah, yeah. And not all of them are interesting enough to merit blog posts, because we do see a lot of the same type of thing over and over again. But the ones that we write about are just, they can be very interesting. And do you name them? Because some of them are named after the worm in the movie Dune and others. Do you get to name them, or does somebody else name them? Usually, whoever finds them first can coin a name for them. And so, yeah, we've tried to come up with fun names when we can. Wonderful. That's part of like your – it's like as the discoverer, you get that privilege.

49:24Anne Dwane:Yes, like the stars. Right. Well, for us, thank you so much, really, for all you do, and such a pleasure seeing you today. Yeah, thank you. I appreciate it.

49:34Hey, this has been Kazinoka, co-founder of Village Global. Thanks so much for tuning in to the Village Global podcast, where we go deep on all of the biggest topics in tech. If you enjoyed this conversation, please subscribe to our YouTube channel. You can check us out on Spotify, Apple, wherever you get your podcasts. We'd love to see you for the next one.