Labs: Everything You Need to Know About Software Engineering Intelligence (SEI)

0:00You're getting pressure to deliver projects on time. You're getting pressure to take the millions of dollars in workforce that you have and ensure they're working as efficiently as possible and then report that back to the board, right? You're getting pressure to make sure that the developer's experience is fantastic because these folks can work anywhere they want in high demand, right? So if you're like the VP and the CTO, I think software engineering intelligence helps you to push one of your initiatives that would have been very difficult to do without data. Gartner just released their market guide showing that software engineering intelligence platforms help engineering leaders significantly improve both team productivity and value delivery.

0:45Through Gartner's in-depth analysis on the critical features of SEI platforms and how they can be used to drive engineering excellence, Linear B was named as a representative vendor and therefore we're giving away a complimentary copy of Gartner's SEI market guide. Head to the link in the show notes to download your complimentary copy and learn how you can unlock the transformative potential of software engineering intelligence for your team. Hey, what's up, everyone? We're back with another Labs episode of Dev Interrupted. I'm your host and Linear B COO and co-founder, Dan Lyons. Today, I'll be joined by my co-host, Connor B.

1:25Great to see you as always, Connor. Likewise, Dan. Always great to be here and to hear about how badly your fantasy baseball team is doing. I know. We were just talking before. We're at five and six. We're battling for a playoff spot. You and I do fantasy football together. I think fantasy football is my number one. Fantasy baseball is more like tie me over until football season starts, but definitely learning a lot about baseball. What I love is that you, not only in your career, but in your personal life, just you love data. You're like, hey, I really want to spend my free time figuring out data around a sport.

2:03In my work life, let's do data around engineering teams. How can I optimize these teams so they perform better? It's almost like you have an obsession here or something. That's actually funny. I think, honestly, probably a lot of our audience has seen Moneyball. Moneyball is what turned me on to fantasy baseball. Checking the data, looking at the box score. Where's the optimization we can find people aren't necessarily seeing so far? Yeah, a little nerdy, but yeah, I like it. Well, and I think that drives into kind of the things that we talk about with developer productivity and developer experience.

2:39What are the workflow challenges that we can find? Or what are the things that people are underrating? Slugging percentage, maybe for an example, for baseball, that people aren't rating as highly as they need to. Our strikeout's a big issue, or our long PR is a big issue. These kind of optimizations. Yeah, I think the cool thing is, let's say that you're observing baseball, or you're observing how your engineering team works, and you might have an intuition or a theory. This happens all the time, both in business and sports. I'm watching a game, and sometimes I'm proven right, and sometimes I'm proven wrong.

3:17I think a player is getting better. actually the underlying data says that's not the case, or vice versa. Something in business is like, oh, I think we might have a problem in this area of the workflow. Actually, the data says we're okay, it's somewhere else. So I think it's also cool to see that, hey, this is what I think is happening. What's the data? Show me that type of thing. So yeah, maybe that's where the fantasy sports addiction comes from. Well, and I know that's the topic for what you want to chat about today, right? which is how are we understanding the problems and then solving them that are happening in software engineering?

3:53Yeah, I think we have a cool topic today. And I'll start out, I think, by just saying earlier this year, Gartner released their debut SEI platform market guide. I'm sure most people on the pod know what's up with Gartner. And I think it's really important because finally, kind of like all this stuff that we've been talking about with data, maybe since the beginning of the pod has come to fruition that it actually has like a formal name and is recognized. And it's this thing, it's called SEI. And there's a lot of love for it. Gartner estimates that adoption of SEI platforms, they're poised to increase about 50 % of engineering teams will have one of these SEI solutions by 2027, compared to only a 5 % increase this year.

4:53So it's like hot topic, picking up. That's where we're going to dive in today. What is up with this SEI stuff? Yeah, so maybe we can start with the basics, Dan, since I know we talk about a lot of these kind of areas, these initiatives on the podcast. But what is software engineering intelligence and what is a software engineering intelligence or SEI platform? Yeah, well, we probably should have said that first, right? SEI, software engineering intelligence, that's what it stands for. But actually, it's kind of evolved since I've been doing this and you've been doing this six, seven, eight years now.

5:32I would say the first incarnation of it, I would describe it something like this. Okay, SEI, it's this data-driven visibility and insights into how engineering teams operate and improve their efficiency. That's maybe how I would talk about it like five years ago. And it's not that it's wrong, but I actually think the space has evolved. We've all evolved with it. And now I would say, actually, it's still about being data-driven and providing visibility and providing insights. But more importantly, it's really how are product delivery organizations transforming in a few different areas, like their on-time project delivery, the allocation and alignment of their costs, developer coaching, efficiency, dev experience.

6:26It's really about transforming these, I would call them more like business value initiatives with data. And that's more so where I think SEI sits today, which is a great evolution from just like data and visibility to like really moving the needle with the business. Yeah, and this lines up with this dual mandate conversation we've been having here for months and almost years now in the podcast of like, how do we both improve operational efficiency within engineering orgs and then align that engineering effort with business goals? And I know a couple of the key capabilities that we're seeing the best software engineering intelligence platforms have are that ability to turn insights into action via automation or other opportunities.

7:12and then that other side of things around how can we understand the investment, the resource allocation that we're making at a team and project level for engineering teams. Yeah, I think you're kind of hitting on some of the, I don't know, I would say like the core features or core capabilities or maybe the best way to describe it is, let's say that you want to run a software engineering intelligence program at your company. So that you want to do that. Okay, you understand the space is heating up. You got to get data driven. Now you want to start an initiative. What do you need to do? So maybe let me describe some of the things that make up SEI.

7:53I think first and foremost, it's around predictable or on-time project delivery. I'll give like the real stories behind that. I guess that's the other thing I like about this SEI space now. I went from just like this. Yeah, measure everything with data to like real life stuff. Realized stuff is, I'm a VP of engineering. The number one question that I get asked is, when is the project going to be done? When's the feature coming out? That's what I'm being asked by my CEO. That's what I'm being asked by the sales team. That's what I'm being asked by the product team. So the first core module of SEI is what I call on-time project delivery.

8:31That's probably my favorite one. Now, there's a bunch of other ones. I'll list them all out. But maybe that's the first one. The other ones are around profitable engineering, resource alignment. Are we investing into the right things? Where's the money going into our projects and people? That's super important. Again, it's tied back to the business. Not just metrics, but it's like CEO comes to you and says, hey, I'm giving you a few million dollars to staff your development team. Where is that money going and what's the outcome for our business? Right? Yeah. So you mentioned this phrase profitable engineering, and I've started to hear that now in the industry lately.

9:15What does that mean to you? And why should engineering leaders be thinking about engineering profitability? First of all, I think it sounds really cool. Profitable engineering. It's catchy, right? Absolutely. Let's do the non-catchy version. I think the non-catchy version is something like, take your project costs, so the cost that it takes to develop every project that you're working on, and do a translation of that cost back to your business folks. again could be CEO could be at anyone on the executive team could be your board and justify that you should be investing in project a and you should be investing let's say 25 percent of your engineering workforce which may be millions of dollars if you're a bigger company right and then how can you translate to say this is how I think it will help the business that's how I think about profitable engineering.

10:18And, Conor, what I would say, it's like so important actually for software engineering intelligence. Because again, if you're just looking at metrics of like cycle time, not saying cycle time is bad or something like that. But if you're just looking at like a cycle time metric, you're pretty far away from talking about how this affects your business. Right. And with these things like on time project delivery, profitable engineering, I think it kind of elevates your career and elevates the engineering team's importance to the business. And that's what I like about it. So why do you think these software engineering intelligence platforms are coming to the forefront now?

10:58Yeah, no, that's an awesome question. I actually thought a lot about it over the last few weeks because that's what I do in my spare time. And I don't think that there's one reason, but there's a collection of reasons. The first thing is, I think there was like this collective consciousness that everyone at a similar time within engineering and also the executive team said, hey, I really think data can help us. Like if we visualize the data of how engineering works, how product engineering works, I think it can help us. And the reason I think it all happened at the same time or similar time is because you saw that in all different areas like sales has data, marketing has data.

11:45They've had it for a long time. But even like HR has data now, people analytics, engagement survey. So that's like one thing that happened. Then the second thing that happened is our ability to collect this data is easier than it's ever been. If you think about Linear B, we're connected into your Git environment. We're connected into your project environment. We're connected into your chat, into Slack. We can see all of your releases. We can see everything with CICD. We can see all the tooling that you're using in the cloud, Sneak. So it's like, not only did everyone say, hey, I think this data would be useful.

12:22We can actually get the data pretty easily now. You can get Linear B up and running with a few clicks of a button. That wasn't possible a long time ago. All right? So I think that's the second reason. Then I think these two other ones are more like what's happening right now. Every company is a software company now. You have companies that sell, I won't say the company's names, but you have companies that sell burgers, cheeseburgers. They're software companies who order your burgers on your mobile app. They're getting more mobile app orders than go up to the counter orders. They have thousands of developers working.

12:59You have companies that sell tractors. Actually, there's a ton of software going into how do you look at all of this equipment and see what's working and what's not working? How do you maintain agriculture using drones and all this insane technology? Farming is like software now. So it's like, I think the third thing is almost every big company is a software company now, and they have recognized that, and they have a large development team. Classic example of this is like Domino's, for example, where it's been this exemplar of like, hey, they became a tech company and transformed to take on this new stage.

13:43But to your point, everyone's doing it now. Software has eaten the world. So I think that's the third reason. And then the fourth reason, I think this is the kicker, I do think the AI movement is helping a lot. We're working with some companies where 25 % or more, 40%, 50 % of the code, specifically pull requests, are raised by bots, not raised up by humans, by bots. Wow. Wow. And people want to measure not only the impact of things like co-pilot, like is this helping, but also how do I improve the developer experience? Because we have so much code coming into our system now that's not necessarily human generated.

14:27Like what are we going to do to improve? And I think if you put those four things together, that's why this space is now mandatory, let's say, or exploding. Do you think that kind of every level of an engineering team can benefit from these software engineering intelligence solutions? Yeah, I do. But I think that they do it in different ways. So the short answer is yes, but it's like different depending on where you work, right, in the org structure. So if I'm a dev versus, you know, maybe like a director level, manager level person versus like the VPE, what are the different ways that those personas are going to benefit from SEI?

15:08So if you're like VPE or CTO, I honestly think the best way that SEI helps you, it's to push one of those initiatives that I just talked about. Like you're getting pressure to deliver projects on time. You're getting pressure to take the millions of dollars in workforce that you have and ensure they're working as efficiently as possible and then report that back to the board. you're getting pressure to make sure that the developer's experience is fantastic because these folks can work anywhere they want in high demand. So if you're the VP and the CTO, I think software engineering intelligence helps you to push one of your initiatives that would have been very difficult to do without data.

15:54Now if I keep going, because I think you asked then about the manager or the director. Yeah, the person who's maybe managing the platform engineering team or managing a team of devs. I think it's one of the hardest jobs in engineering. We have a bunch of, I think, earlier episodes, like being an engineering team leader, being a group manager. I think that's the hardest job, this middle management layer. I think it's that for a lot of industries, but definitely in engineering. You're doing this combination of decision making and managing people, but also delivering on sprint predictability. and you have to be technical, but you also have to have the soft skills.

16:32There's a lot coming at you. And for example, what we're doing at Linear B, you can think about it as like a co-pilot for engineering managers. Deliver these people insights so that they can be, you know, developers are always like complaining, I have a terrible manager. It's because 50 % of the time, they probably do have a terrible manager. It's really hard. But if we can get this data into the hands of each one of these decision makers, a lot of them are new, by the way. I've been, maybe I've been doing this like two years, one year. They can make better decisions, right? So I think like at the management level, it's like day to day decision making with data.

17:10It's supercharged that. Helping coach your devs too. Yeah. Oh yeah, of course. Of course. I'm now responsible for coaching 15 devs or whatever it is, eight to 15 devs, maybe more. And maybe I didn't even mention that as one of the initiatives, but definitely having a developer coaching program that is more data-driven and fair for developers? How do I grow my career? Am I contributing to the team in the right way? Am I contributing to pull request reviews or am I only coding? That's definitely an initiative that a lot of VPs are pushing and it's hard on managers. It's interesting because a software engineering intelligence can really help engineering teams navigate change, whether that's on the personal front where, hey, I'm trying to, improve my career.

17:58I'm trying to level up. I'm trying to move to a new role. Let me get quantitative and qualitative data and then automations and other unblocking opportunities through the platform that can help me to improve as an individual or as a team leader. And that it really helps manage, let's say we acquired a new company. How can we now evaluate how they're integrating with the rest of our engineering or the rest of our product org? I mean, there's a ton of examples of this, we go on, but this ability to better navigate change is so crucial to resiliency for software engineering teams. And not only can it help you scale, but it can also help you navigate a lot of these other challenges.

18:37Well, just like a real life example. Yeah, please. A real life example. We're working with a company right now. And I think everyone listening, if you're not a developer, you can remember what it was like to be a developer. And actually, the career ladder is a little bit vague. like what's the difference between a junior developer and a mid-level developer? Or a mid-level developer and a senior developer? Or a senior developer and a principal developer? In a lot of organizations, it's not defined that well. And sometimes in order to get promoted, it's the type of person that can have a highly articulated conversation with their manager and push for their career and all of that.

19:18Now, the company we're working with, they want to make it more fair. There's all types of developers, right? Not every developer is like an orator or maybe come on a podcast and can like prove their case of why they should be a senior engineer. Now they're saying, hey, with data, we just want to show the capabilities that it takes to be junior, mid, senior and principal. And for them, for example, one of the big things that they're pushing is how do you contribute to helping other teammates? And they're doing that by saying, each week, how often are you providing feedback on poll requests and giving comments and helping others and not just doing your own work?

19:58And they can see that with the data. So now when they have that one-on-one conversation, it's not like you only get promoted from that, but you could say, hey, Connor, I can really see that you are helping a lot. Actually, it picked up from like almost zero, you know, reviews that they're helping with or one per week to like two to three. Can we talk about that? I saw that out of you. That's awesome, man. So it's really like career progression. Really interesting. And it sparks something for me on the platform side of things because you talked a bit earlier about how this space has evolved. You know, it used to just be kind of, oh, let's get some basic engineering metrics so we can understand what's happening to now really providing these coaching opportunities, improving developer experience and improving developer productivity some of these big initiatives i want to talk more about but there is this also other concept that we've talked a lot about on this pod that i think relates to sei and i want to understand how you see them as differentiated or aligned which is value stream management you talked a bit earlier about predictability in the software engineering process and this and trying to bring that predictability and profitability to your software development life cycle how do you see this concept of software engineering intelligence and what platforms that provide SEI can deliver relating to value stream management.

21:10So when we first founded Linear B like six, seven years ago, there was this concept of VSM, value stream management. And there was a big promise there of I'm going to take every piece of value from you're thinking about a piece of value to you have an idea about it to you put it onto paper and I'm going to track it all the way out to the customer getting it. Then I'm going to track it all the way around to renewals and I'm going to therefore manage my value stream and we're going to get better. And unfortunately, what I've seen with a lot of VSM is the actual rollout and implementation of what I just said is extremely difficult.

22:02And therefore, it's an amazing idea in concept, but I haven't seen it actually implemented very well. Now, with software engineering intelligence, if you think of VSM the way that I just described it from like I have an idea all the way to like a customer getting value in renewal, I think what's great about SEI is it's more down to earth. It fits in the middle of that. It's for product delivery teams. It's kind of like in the middle of that whole VSM end to end. And you can roll it out and you can get all the data for it and you can actually improve. So I don't know. I'm not trying to say one's better than the other, but I think like SEI became, I would say, more down to earth and achievable.

22:45And that's the big difference. So maybe SEI platforms provide an opportunity to actually realize the value of VSM and say, OK, how do we implement this? How do we actually put this into practice? I think so. You've mentioned a couple other key initiatives that are common themes or refrains on this show. Developer productivity and developer experience as things that SEI platforms can help teams solve or improve. How do you see SEI driving developer productivity? Well, it's so great because those two things, productivity and experience, go hand in hand. That's what I love about it. Sometimes I think it's just like the way that you're viewing it.

23:25being a developer, at the end of the day, you want to get your code out to prod. You want to. A lot of developers want to see the impact that they make on the business in terms of customers using the cool stuff that they create. And if I just simplify it, it's like, what's preventing me to do that? Where's the toil located where it's more difficult for me to actually get my job done? And with SEI, It actually measures the workflow from coding time to pull request review time to pickup time to deployment time to how long are the tests running. So, therefore, it's kind of like measuring that experience for the engineer.

24:08What are they going through? And when you turn on an SEI platform for an organization, most of the time you will see a bottleneck in that workflow. And if you say to yourself, hey, I'm actually going to use some automation, I think that's the other part of the space, let's use automation in order to fix this workflow problem, your developers will be more productive, but they also have a better experience. So most of the time it goes hand in hand for me. It's really interesting you mentioned automation because I know this is an area where Linear B is taking a different approach to many of the other SEI platforms out there.

24:45turning insights into improvement by actually providing automations built in platform. And it's exciting to see that you can now go into the Linear B platform and kind of just add an automation that relates to, hey, I want to solve this use case. Can you explain a bit more about how you view Linear B as going beyond the basics? Yeah, I just I think, again, it's the evolution of the space like software engineering intelligence. Yeah, the intelligence side of it. some of it is I have an insight and I'm delivering that insight to a manager so they can do their job better. But the other part of that intelligence side is I have an insight, but I'm actually creating an automation that goes into the SDLC and unblocks one of those blocked areas.

25:32So for example, at Linear B, if you go into Linear B now, we have a full marketplace of automations. Some of these automations help new hires onboard more efficiently. Some of them ensure data integrity or the definition of done of a PR. So for example, every PR needs a JIRA ticket or every PR at least needs to have a definition or if it's on the front end, like have a screenshot. So it's like how we work, right? Other times I mentioned, because I just think it's the coolest one, because now we're finding all these bots are creating code. Some of these automations say, hey, let's look at this dependabot and say, how risky is this code change?

26:13And if it's really not risky at all, it's like a minor, minor change, let's not ruin the developer experience, pull them away from coding. Let's not do that. In that situation, if all the tests pass, let's approve it. And yeah, if it's a major version bump, let's pull a developer. So you can kind of think of this automation marketplaces. Yes, step one of SEI was data that shows where the bottlenecks are, but really where we've took the industry is automation to unlock those bottlenecks. And I think that's maybe that's where I believe that the space is going, delivery of intelligence into your SDLC and to your managers.

26:52So unlike something like GitHub Actions, the automations within Linear B's SEI platform are actually helping you both see the insight on the metric side and then directly relate an action to it and kind of roll it out to your org. Yeah, I think the difference is you need both. If you're a platform engineering team or you're the head of developer experience, the first thing that you need to know is where do I focus my time to help all the developers? So when you log into an SEI platform, You log into our platform. It shows you that. Here's where you focus your time. Now that you know where the bottleneck is or where that bad experience is, it's like, what are you going to do about it?

27:32Oh, go check out the automation library. Here's all these automations that other customers are using. Other people wrote, already solved your problem. Load balancing reviews, for example, finding an expert reviewer, creating, automatically creating a JIRA ticket when there's a vulnerability found. There's people that have already done it. So it's like, oh, okay, I can almost just drag and drop and say, this is going to help me. And I think that's the difference. And this relates back to that predictability element you mentioned earlier, as well as dev experience, where it's like, okay, we're helping improve the experience and kind of streamline it so we can better understand where are there actual challenges and when will we actually deliver code?

28:18I think it all relates together. If you think predictability, we usually look at two metrics, like planning accuracy, capacity accuracy. Now we've added in a forecasting, what day will the project be delivered on? We're using a Monte Carlo simulation to do so. But if you look at all of that stuff, it's like what affects the delivery date? Well, the experience of the developer, are there bottlenecks in the process? You can still think of some of your classic door metrics like are they getting pulled away to production because your change failure rate is too high? Are there too many bugs coming into the project unexpectedly, which will lower your planning accuracy?