Scaling Smart: Strategies for Product Development | Semgrep’s Adam Berman

0:00Product development is so much fun. And I think it makes like every engineer better to have an opportunity to work on a product team or to think of themselves as building products. I talk to a lot to our infrastructure team, too, that like the reason our infrastructure team has been so successful is they also think about build themselves as building products. But for the other engineers of the company that everyone, if you put on your PM hat, if you think about who are my users, how is what I'm building going to bring them value? It's so much fun to build something, get it into someone's hands, see them use it, and then keep on iterating.

0:32See if you can really stretch that value, make things easier, make things better. Are you looking to improve your engineering processes and align your efforts with business goals? Linear B has released the Essential Guide to Software Engineering Intelligence Platforms. This comprehensive guide will walk you through how SEI platforms provide visibility into your engineering operations, improve productivity, and forecast more accurately. Whether you're looking to adopt a new SEI platform or just want to enhance your current data practices, this guide covers everything you need, from evaluating platform capabilities to implementing solutions that drive continuous improvement.

1:07Head to the show notes to get your free copy of The Essential Guide to Software Engineering Intelligence Platforms today and take the first steps towards smarter, data-driven engineering. Everyone, we're back on The Interrupted. I'm your co-host, Connor Bronson, and today I'm joined by Adam Berman. He is the Director of Product Engineering at Semgrep. Adam, welcome to the show. Yeah, great to be here. Thanks for having me. And obviously, great to have you on the show because we have a producer named Adam. So, you know, like clearly there's some synergy happening here. But I think the real reason we're bringing you on is that over the last 18 months, you've had the opportunity to build the second product line at Semgrep.

1:42And this is a unique opportunity that happens at a lot of startups. A lot of new organizations were saying, hey, we're going to start this new line. And throughout that process, you had to decide the shape of how the product development organization at Semgrub functions. And so through research, conversations with our managers, lots of trial and error, I'm sure, you helped evolve this organization. And I think it's a really interesting perspective because usually we talk to leaders who are maybe farther along. They're like, oh, I've built six product lines now. I've done this thing. Or I came into an established organization.

2:12Or we talk to leaders who are just starting who are saying, we're doing this first product. So you're at this interesting transition point. Let's start there. What's the process you took to build and launch this new product. Yeah, when a company goes from a one product company to a two product company, it's this like huge change in what the organization has to be responsible for. When there's one product, every engineer at the company is like singularly focused on the one product, on the one mission, on the one goal. As soon as you move to a two product organization, you start to see the organization needing to function a little bit differently.

2:45You start to think about platforms, you start to think about systems and services that can serve multiple products. And you also start to think about like, okay, who is actually serving the products themselves? How can we make sure that there's like singular ownership, make sure that there's like really core direction for each of the products. And then eventually you start having to start to think about like, okay, how do we make sure these products aren't too siloed as we start to keep focus? You know, you're trying to like do this thing really scrappy at the beginning and then think long-term about the, if this takes off, what does this look like when it's successful?

3:17How do we make sure that like, you know, we don't end up with some product that's totally different, has totally different primitives. We don't want a totally different sales and marketing organization for a second product because it's so different from the first. So that brings up a few interesting questions. The second product line, is it the same ideal customer profile? Is it a related customer profile? How have you approached that? Yeah. And I talked to a lot of other engineering leaders as we were starting to build the second product to think through like, hey, how do you decide? What do you decide to split out into something that is really like cross-functional and seemingly focused?

3:47Where do you draw like the platform approach? So the ideal part here is like, hey, we're still we're a security company. We're focused on static analysis of code. So we're trying to take static analysis and apply it to a second domain. So our initial product looks at the security of your first party code, the code that your developers are writing themselves. This second product, Semgrep supply chain, focused on the supply, your supply chain, the packages that you're pulling in. software supply chain is a huge security concern. There's so many like deprecated things you can bring in. There's so many, like look for Log4j.

4:17There's still people downloading like issues with Log4j into their systems today. Exactly. And a lot of the tools, especially tools from, you know, five or 10 years ago are really focused at visibility, trying to help you see all of the possible vulnerabilities you could be bringing in. But people are bringing in so many vulnerabilities and then using only just a tiny slice of them that we try to help people understand, okay, which packages you're bringing in that have vulnerabilities, but then we use static analysis of your first party code to see, okay, in which of these packages are you using the dangerous functionality, the vulnerable functionality?

4:47We call that reachability analysis. And so we see that we're, the user persona is really similar. It's often the same person or maybe the two different people on the same team, the AppSec team who are using both products. So it's the same economic buyer. It's often the same kind of persona you want to target with marketing. We don't need to think about different sales. We don't need to think about different marketing. And we often even don't need to think about totally separate workflows for onboarding, for access control, for things like that. But we're talking about like two different parts of the AppSec problem.

5:18And so we're thinking about, OK, how do we make sure that we don't just like think about this as an add on to the initial product? How do we make sure we really serve the needs and solve this different problem really completely? So do you treat it as a single AppSec platform or as two distinct products that can be used in tandem? Yeah, and I think that was when we decided, hey, we're going to go build a second product. That was the question was like, do we try to build like two point solutions that people can like deploy separately? Or do you try to think about this as a platform? And what we realized is there is this kind of like better together approach when you're trying to provide a product for the same user or like the same team that you can make it a lot easier for someone to do their job if they don't have to be like really context switching when they're using the tool.

5:59And then eventually you can start building insights from one tool into the other. Try to make it so that like, OK, you're not just prioritizing two separate streams at work. You're really prioritizing the most valuable thing you could do with your time at any given point. So we tried to like really stay focused on, you know, when you're starting building the product from the beginning, you try to stay focused on a single mission, making sure you understand the problem really deeply. What is the pain that AppSec people are feeling? They're feeling the pain of a ton of noise. Okay, now we're going to help you focus on the highest priority thing you could focus on.

6:29But then as you start to build the product, see success, now the question is, and we're, you know, 18 months in, it's like, okay, how do we build these things back together? they're deployed in the same platform. Now, how do we make sure that they can be more useful together? So how did these decisions inform how you then constructed the product organization and belted out to support these two lines? Right. I think one of the traps you can fall into is kind of over-optimizing for the long-term future. At my previous company, Meraki, we would often say to ourselves, like, hey, scale problems are good problems to have.

6:54You get the fortune of having scale problems. Yeah, champagne problems. Exactly. If your product doesn't take off, you don't get scale problems. If you don't have large users who are going to, like, really utilize the full breadth of the product, you don't get scale problems. So scale problems, exactly, are champagne problems. So we want to not over-optimize. We want to focus in the beginning on learning. And we'd frequently use the phrase build to learn. So we talk about, and I'll talk about it in my talk later today, that there's like some trade-offs you're going to make. It's like, do you think about optimizing for the cognitive load of technology?

7:24Or do you think about optimizing for the cognitive load of like business domain? And we decided at the beginning, we don't have to care as much about scale. We don't have to care as much about like building robust systems, because if the product never takes off, no one gets to use our robust system. The system doesn't need to be robust. And so at the beginning, we thought about, okay, we're going to pull the minimum set of people from like a variety of domains or a variety of technical expertise, give us like the breadth within the team to solve problems like okay enough, but really deep dive and say to ourselves over and over, we're going to build to learn, we're going to build to learn, we're going to build to learn.

7:58And knowing eyes wide open to the technical debt, we're going to accrue, We're going to accept that trade-off. We're not going to say, oh, and these systems are going to scale forever because we know that they're not. But we know, hey, if this thing takes off, we're going to take a pause. We're going to understand the directions that things are going to take off because who knows one of these systems is going to need to scale, another might not. We're going to take a step back and say, okay, to achieve the next 18 months of our roadmap to get the product where it needs to be to support the next scale of users, what do we need to rebuild?

8:27What do we need to develop back into the platform? How do we like reintegrate? So it's kind of, you're thinking of it as like a little, people say this a lot, a little startup within a startup, but there's this aspect to it. It's not a startup forever within a startup. At some point, you kind of try to reintegrate some of the systems back into the main platform. So did you take an approach with user feedback where you talked to your current customers to start with and said, hey, we think this is an extension. This is what we're kind of hearing from you that can help you. Let's get some of you on it and kind of start getting that feedback and seeing how it works.

8:56Or how did you get the initial user feedback you needed to make product decisions? Totally. One of the most fun parts about working in product engineering, it's the thing I love so much, is this zero to one helping go from this like kernel of an idea to something that is valuable. But the kernel of the idea isn't really like I have some game changing thing that's going to really change the world. It's like I have an idea that a problem is not well solved. And in order to be able to solve that problem, you need to really have a depth of understanding of the problem. So the first month or two was just talking to whatever customers would talk to us, talking to people who had a lot of breadth and depth in the space.

9:30people who we expected never to be customers, people who are like, you know, just other people who are thinking and have been in this space for 20 or 30 years who would tell us, hey, this is the pain that we see. These are the problems that are unsolved. Trying to get an understanding of like, okay, where is a place you can like land? And from there, we tried to develop like a core group of four or five early partners. And the benefit here is that Semgrep is open source. We're really able to lean on that open source community. Our customers are really willing to give us feedback. Some of them even contribute back to the open source tool, which is great, really cool.

10:03And it means we also have a community Slack with now, I think, 3 ,000 people in it who are constantly talking. And it means that we have access to our customers on a daily, hourly basis. There's this really fun, a lot of people will talk about trying to shrink the iteration feedback loop as quick as possible, make that iteration loop as quick as possible. When you have a Slack thread going with a customer, that feedback loop can be minutes. We would sometimes come up with an idea. We'd build something really quick in an hour or two. We'd Slack a couple of our partners and say, hey, what do you think?

10:34We'd post a screenshot. Maybe we'd want a screen share. They'd say like, hey, I'll hop on in 20 minutes. They'd give us some quick feedback. And then in the same day, we'd be able to take their feedback, make an iteration, put something else up and get another round of feedback. So we're able to move really fast. And that doesn't mean that we were like coming up with brilliant ideas every day. What it means is we were failing really quickly. We were understanding like the edges. We're getting fast iteration on it. Exactly. And so what looked like really fast progress on an idea that like from the outside, you could have just said like, oh, this idea worked really well the first time.

11:02No, no, no, no, no. Let me tell you, we had 30, 40 ideas that were not useful. But the fast iteration loop and customers and partners are really willing to be honest with us, give us feedback because they knew they'd benefit one day from them getting a product that was really useful to them. That made it so that we could use that fast feedback to really shape the direction of the product, find that early landing point pretty quickly. So for other leaders who are listening to this who are maybe about to hit the stage of, hey, I need to go build a second product or I need to build a team to build a second product, what would be the advice you would give them based on your trial and error?

11:37So I think, first of all, from an organization perspective, if you're going to go build a second product, you're going to take resources away from the first product. So you want to have an open and honest conversation with the other leaders of your organization, your boss, with the CEO, whoever, about what kind of pain you're going to put on the rest of the organization. If people aren't engaging there, people are like, it'll be fine. I think you really haven't started the conversation. You want people to sort of agree that things else are going to slow down. I think one of the other things to be careful of is to make sure that your experimentation doesn't put the existing products like stability, reliability, brand at risk.

12:09So you're looking to find ways to really mitigate that. For us, it was making sure that we had developer partners and they were the only ones who were seeing our early versions that were totally okay with something very low quality. and it also meant early on making sure that like the stack that we were developing on was totally isolated from the initial one and that meant at the end of the day we threw away all of that code and we had to go into the mindset of like hey all of the stuff we wrote early it not only shouldn't it be in the product it cannot be in the product but there's a point where you transition from build to learn to like build to build and like go make a product exist and at that point you kept then you can take a step back and say great there are some things we need to make sure are true for stability reasons.

12:49And then you can think more deeply about like what systems to build, how to build them effectively, how to build them robustly. But if you try to do that from the beginning, your iteration time, you're like, you're going to, you are going to throw your initial product in jeopardy in terms of reliability and you're going to go more slowly. So finding ways to separate out your like development stack early can really be useful. It also means as you're trying to find that early feedback, it means like really limiting the scope of who you're trialing with. We made the mistake early on of trying to say like, hey, let's see if we can get 10, 20, 30 people interested.

13:22And they were, but there's different degrees of how interested they are. Can I get a hold of them the same day to get that quick feedback like you mentioned so we can improve the product very rapidly? Or is it going to be, oh yeah, I'll get back to you later this week or next week and it'll slow you down maybe. Exactly. And we can't manage 20 or 30 relationships even if they all wanted to give us feedback really quickly. And so true, not all of them did. And we were able to instead, like we gauge, okay, there's a lot of excitement in this area. We were able to tell them like, hey, great, we'll come back to you when we have an alpha version ready.

13:52But we were able to narrow the pool to four or five folks who could be really good partners, who were really willing to be honest with us, who we could go into something with something kind of scrappy. And that made the relationship really strong. Eventually, these folks ended up being like the strongest partners for us when we went to market. There are like most committed users today. And I think they feel a lot of skin in the game. They feel like we built a product for them. And that's helped us. Like, you know, they sometimes will advocate for us. They'll go talk to some of their friends, their partners, other companies that they think similarly to them.

14:23And say like, hey, this is like, they built a product for us. And I think they built a product for you. And that's helped us get a little bit more traction in a foothold. Are there other insights that you drew from this build to learn phase? When you talk about build to learn, it's something that can be a little trite. People will talk about like the MVP. Yeah. I have probably overused that phrase. Right. And MVP means like a lot of things and kind of nothing. It can be a word that people throw around to just mean like, let's get something out there and, you know, we'll come back to it later. It can also kind of be weaponized to say like, let's cut corners.

14:59And like cutting corners is good at certain stages, but you also want to cut corners sort of tactfully. I think one of the things we benefited from a lot was people on the initial team were pretty experienced as they knew the kinds of corners that you could cut and not pay for too dearly in the future. And the kind of like corners that like maybe are just totally worth avoiding whatsoever. Let's not even dig into this. We'll find a solution for it later. How big was the team that was engineering this second product when you got started? At the very beginning, it was just me. And that was like the phase of we were just talking to I was like had 10 or 15 meetings a day.

15:34with external people who could give me feedback, and I was building something with a Lambda that was totally separated out, and everything looked awful. I'm so glad that doesn't exist today. This is really interesting, because you really took the approach of, we're going to be a startup within a startup. It was almost like, hey, I'm founding this new thing that is building off this open source framework, and it's going to be new and helpful, but it was you, and then I'm assuming you had to bring in some resources from within the company, and also probably hire NetNew Devs to help out. Totally.

16:04And so we benefited from, there'd been a Hack Week project that had some technology that I was able to leverage. There were some other people who were willing, I was like, you know, beg, borrow, and steal, people who like chipped in and they're like 10 or 20 % time to help unblock. And that's awesome. So I don't want to in any way say that I'm like, I want to take credit for this. This is an idea. Also, we look back like, it's an idea the founders had like five or six years beforehand. They're like, hey, this would be really cool. When it's ready. Yeah. Right. But there wasn't traction at the time.

16:27And so they sunsetted it, which is like, okay, what if we dig this up again? And you said, like, think about it as an investment startup within a startup. It's something we literally explicitly did. I talked to the founders and they said, think of us as like your investors. Think of us as your VC. We're going to think about like, OK, your salary is what we're investing in it for a few months to begin with. And one of the possibly good outcomes is that you decide after two months of full time investigation that this is not worth exploring. And that's we've limited our investment and we'll move on to something that could be right.

16:56We didn't have to raise that, you know, series A round for you and really like burn some cash through. Exactly. But the other opportunity is like you decide that there is something here, that there's a market for it, that there's a technology that we have that can make this really useful. And then they can increase their investment and think of it as another capital investment. This is a really cool idea because, I mean, you can really draw these lines, right? So your investigation phase is your families and friends round, right? And then it's like, okay, our seed round now is when we're going into this build to learn phase.

17:23And then maybe the build to build phase is your series A round. It's like, hey, we're actually, we're going for this now. So I like this framing because it helps explain it in a way that I think is very common for anyone who's been involved in the startups. Totally. And it lets us also like kind of view the founders as your board of directors. They're not the people who are saying like literally here's what to do, but they're the people who are kind of steering you. They have also built startups before. In this case, they have literally built the startup you're a part of. So they really know the context really well.

17:52And they were able to guide, give advice, help me like see around corners they'd already seen before. And then as the scope expanded, so the team would have expanded from like a one-person investment to like 50 % of a security researcher, 50 % of a program analysis engineer to really dig into some of the tech. We expanded by the time we were three months away from launch. That was like July of 2022. We launched in October. We had added two more cloud engineers to the team. And the security researcher and the program analysis engineer had gone full time. So we had a four plus me person team, a five person team bringing this to market for the last couple of months.

18:29That's that build to build phase. That's that build to build phase, exactly. And by that time, I had to be able to pitch the other engineers and say like, hey, you're tying some of your career to this too. This isn't just a lark. This isn't something where if five of us tried to all coordinate building to learn at the same time, we were probably not going to do that very effectively. This is something that we feel like there is a lot of conviction that if we build well and we build the right thing, there will be success. and men like, you know, engineers are going to only come join a team if they feel personally bought into the mission, personally bought in and excited.

19:00You never want to be the engineer that's like kind of pushed to some project. And so getting them bought in was kind of a signal to me too. It was like, okay, if I can get other people on the team really bought in that they could spend a couple months on this and make a big impact, that probably means that we're moving in the right direction and that it's also like the right time to do this. And we thought about this as like, great, this is our series A. It also meant that we started like the board of directors also got a little bit wider. It wasn't just the founders. It was now like the other kind of executives at the company were bringing in sales or marketing.

19:27We're building in HR and recruiting in the sense of that other VC. You got to bring another board member. Yeah, exactly. I'm starting to think about this is no longer a short lived experimental team. This is a long lived team. We got to think about the health of the team. There's a long lived product. We got to think about how we're going to market it, how we're going to sell it. And your job as the engineering leader is a little bit like, you know, CEO or CTO is to think about like, OK, where does the business and the technology, How do those things intersect? How can you make sure you're optimizing your organization for the next phase of growth?

19:57So fast forward a year, we're having this conversation at Lead Dev West Coast. You know, it's October 2023. The product's been live for about a year. Yeah. How big is the team working on it now? So it's now two teams. What we've done is we've built like a five, a six person, like full stack team that's like senior and understands the domain and has a lot more like of the technical expertise too, to be able to build some of those systems more effectively. We also realized that there was like one key, like deep technical piece that was very different from the rest, that its own workflows needed staffing because like the work streams were pretty continuous.

20:33And that was security research. We needed folks who could be full time paying attention to CVEs, diving deep into them, understanding what makes them dangerous, writing some grip rules for them. And that workflow was like just very different from how the rest of the engineering team, like product engineering team was working. we tried i think as as much as possible i try to like keep teams together the more you like artificially segment teams the more communication overhead there is and folks want that context because as much as possible if two people are working on the same kind of problem they're on the same team they know how to work together that problem that project is gonna be more successful right um but we kind of got to the point where it felt like we were artificially keeping these teams together and there was pain in the other direction of like everybody trying to keep context on this other stuff, you'd find that like people were kind of tuning out when one set of the group was talking about their work, not because they didn't care, but because there was just too much to pay attention to.