The Need for Speed | 2025 Engineering Benchmarks

0:00I wasn't going to answer any questions on this pod, BLP, but I'm going to do it because I love you. and here's what I would say. First of all, go read the report, right? It's there for you should know what the standards are, what the benchmarks are. But I think that 2025 is the year of productivity for engineering organizations. So if you're an engineering leader, understand the report, know where you stand against the report. And then for any of the metrics that you wanna get better at, use automation, use AI. You got to be into this stuff now to improve.

0:44Welcome, everyone, to Dev Interrupted. I'm your host, Ben Lloyd Pearson, and I'm joined today by your other host, Andrew Ziegler. Andrew, how's it going? Like, I just want to check in. Like, how are your New Year's resolutions going? Are you still running every day? Oh, yes, yes. You know, I went out last night, but I still woke up this morning, got my run in. It's still going pretty strong for me. What about you, Ben? How are your resolutions? What were they again? What were yours? Yeah, you know, I don't really do resolutions, except I did resolve myself to not get sick too much this year, and that's already gone out the window.

1:18So unfortunately, with all the sickness going around, it's unavoidable. Oh, yeah. I'm always sick this time of year. It's pretty much unavoidable. So you set yourself up for failure with that one. Yeah, exactly. So, you know, before we get into our interview today, I just want to talk a little bit about some stuff that we're seeing happen in the space and how that relates to some of the stuff that we cover here on Dev Interrupted. So, Andrew, what's on your mind right now? What stories have caught your attention? Yeah, something that has been top of mind for me, especially in the last week, is this new memo from Microsoft about its internal changes, where it's combining its dev, div, and AI platform teams internally.

2:00and they're taking the entire developer division and making it focused on AI and it's signaling a huge change within Microsoft and I think this is something that really resonates with large orgs and small ones too all across software engineering right now that we recognize there's a lot of change that has to happen and it has to happen internally with the tools that developers are using so the most standout quote that was for me from the memo on Monday was about they needed to compress 30 years of change into three years. So they're trying to move lightning fast and make a whole lot of innovation happen in a whole lot small amount of time.

2:38So the idea of compressing that is quite fascinating to me. That's, I mean, that is a very stark claim. You know, of course, every leader is going to claim wild things when they take over a new organization. But, you know, for me, it really, you know, resonates with the idea that AI is not going to be this tool that like replaces you. Like if your job was to weave together threads, then like the loom will replace you. But the vast majority of people can provide far more value than that. So I feel like really what we should all be focused on is how can we be more productive, more efficient, do better things with AI rather than like, you know, we've got all these LinkedIn influencers out there quoting improperly, quoting people like Mark Zuckerberg, which I said I wasn't going to talk about today, but Ben, you got to let it go.

3:28You got to let it go, Ben. Yeah. So you've got all these influencers saying that, you know, AI is going to take your job and either they're trying to sell you something or they're probably misinterpreting the words that of somebody that's more important than they are. So, you know, I, I really liked this story because I think it is probably more emblematic of what the future of software engineering is going to look like. I totally agree. And I felt like the memo was very positive in that regard. And the idea of realigning developers around the tasks they need to be focused on now that they can use these kinds of tools to automate away other ones.

4:04And that was really the unlock. They weren't just saying that they needed to move 30 years and three years. They were saying that they could move 30 years and three years. It's an opportunity and it's an opportunity that involves everybody in the process. So what about you, Ben? Any items top of mind for you? Yeah. So, you know, we're recording this episode on Friday, January 17th, just to date this for everyone that's out there. In two short days, there's set to be a potential ban of TikTok here in the US. Of course, by the time this comes out, it'll either happen or not happen. But it sounds like it's going to happen at this point, unless there's some sort of last minute sale of the company.

4:44But, you know, in my perspective, like, bring on the ban. Like, I'm actually kind of okay with this, partly because it's like not actually even a ban. Like, if you actually look into it, really, all it is is they're forcing divesture from a, you know, a company that's owned by a company that is very closely associated with the Chinese Communist Party. The thing is, when you look at how China treats social media, they've banned dozens and potentially even hundreds of American apps within their borders. And over there, it's an actual ban. You can be arrested for using some of these apps. But I think really what was missing from this entire conversation, and this is what I want to get to, because I think whether or not an app gets banned is not going to change the world.

5:34But there really are just deeper data privacy issues in our society, right? Like, we are concerned when a foreign, potentially adversarial government might be misusing personal data of American citizens, but we don't seem to take that same rigorous approach to, you know, privately owned or publicly owned American companies. So, you know, really what I think we're missing is like we should be having this deeper conversation about how do we actually have implement data protection laws that protect all of us and prevent organizations from misusing our personal data and trying to use it to manipulate us.

6:09But we'll find out, you know, before this episode comes out, I guess. Right. I'm sure we'll tell our future selves or our future selves will tell us now. And I agree about the timely topic and something that is, when I think of the TikTok ban, it reminds me of videos I've seen of people in Congress asking questions about how technology works while in the process of creating very large sweeping rules that will impact the usage of that technology, the development of it, the ownership of it. And these are really impactful decisions. And what always stands out to me, and of course, these clips go viral because they capture the most, you know, silly of moments, but is the lack of understanding and the lack of true grasp of how technology is used by people.

6:53And when I hear of something like this, like TikTok ban, you know, is a little more catchy than TikTok divestiture. And so I guess that's why they're calling it that. But it just seems largely unenforceable in the way that many other bills related to technology have gone through that same emotion before it. And what it signals to me every time is just the lack of technical literacy in our world, especially by the people that make the decisions that make the world that we live in. I think there's a part on all of us to do better to understand how technology is impacting the world we live in. Yeah.

7:30And man, it always comes back to the same topic, it seems like. But I believe it was Facebook that went in front of U.S. Congress and had a senator grill them very basic questions about how they make money and quickly devolved into like a basic fundamental conversation about how the Internet functions, you know. And like, how can you expect good regulations being applied consistently in that environment? It reminds me of just very recently when we said, like, would you rather take tech teach tech debt to your grandma? It's kind of that, you know, you need to explain these concepts to grandma because, you know, grandma might be in Congress.

8:07Yeah. Well, man, we need a GPT that educates our congressional representatives on technology. So, cool. Well, those are the stories that we're covering today. But I want to go ahead and hop forward to the guests that we have that we brought on today. In fact, we actually don't really even have a guest because it's just me and Dan talking about something that's really important to me, actually, which is the 2025 Engineering Benchmarks Report from Linear B. And in this conversation that I have with Dan, we're going to talk about some of the biggest takeaways, you know, things about cycle time, PR maturity, PR size, planning accuracy.

8:49We've got lots of really great metrics that we're going to dive into and then share a bunch of new insights that we found this year. So after the break, stick around because you don't want to miss that conversation.

9:03And on January 29th, join Linear B and Luca Rossi of Refactoring for a workshop to learn the three key traits shared by highly successful engineering teams. This workshop gets down to brass tacks with actionable examples. So check the show notes to RSVP Your Spot for January 29th, and we'll see you there. I'm your host, Dan Lines, Linear B COO. So in today's episode, we'll focus on the 2025 Engineering Benchmarks Report from Lanier B. We're going to take a look at what the report found this year and what the most recent data says about engineering teams and what it says about the health of those teams.

9:46So like a lot of amazing information in this report. And to help me walk through all this data, I'll be putting Ben Lloyd Pearson BLP, Linear B's director of developer experience in the hot seat and making him answer all the questions while I get to just relax and chill as a host. So, Ben, welcome to the show. Yeah, thanks. It's glad to be here as a guest, I guess, now instead of the host. So, yeah, I'm just going to chill over a drink, maybe have a beer, ask you a few questions. But no, in all seriousness, I think this 2025 report is super interesting. So, you know, thanks so much for walking us through it.

10:28So to kick us off, can you give us, maybe just give us like a general overview of this year's report? Yeah, if I had to pick like one focal point of this year's report, it's the relationship between developer experience and developer productivity. So specifically I'm referring to with DevX, it's when, you know, it's your team's overall morale, engagement with the organization, their tools, the processes, the environments. and on the dev productivity side, it's a matter of how effectively and efficiently developers can complete their meaningful tasks with minimal waste. So we're hearing more and more from engineering leaders that dev productivity is a major focus area, but you can't actually improve productivity without making DevX a significant part of that strategy.

11:14So if I had one key takeaway from the report, it's that you should be measuring your investments into DevX to ensure that you're minimizing long-term risks. If you're always over-investing in new features, feature enhancements, and you're ignoring, like, keeping the lights on and the maintenance and DevX side of the equation, you may get some short-term gains. But over the long term, that's certainly not going to work out for you. Yeah. No, you know what I think is funny is developer productivity and developer experience relate to each other, right? They're very, I think, highly correlated. And maybe like in 2024 or 2023, I would say DevEx was hot or maybe, hey, let's make sure that we have an amazing experience for our developers, which is important.

12:03But I think what business leaders, even developers themselves, it's like, I want to be productive. and maybe I'd rather be productive because if I know that I'm highly, highly productive and I think, you know, hopefully we'll get into some of this stuff like PR maturity and merge rates and all that kind of stuff. If I know I'm highly, highly productive, I'm highly likely to be having a good experience. Yeah, and I think in particular, you know, knowing how DevX impacts your overall productivity, like when the business, the rest of the business comes to you and say, hey, we need to ship all of these new features on this scheduled timeline.

12:42And you have concerns that maybe you don't have the resources that you need to deliver on that. Like, you know, you need to be able to have that conversation where you can push back to actually make sure that your engineering teams are getting the support they need while you're still delivering the value that your business is expecting. So this year, I think this is like at least the second year, maybe even the third year of the report. and I'm sure we'll attach the report to the show notes and all of that. And there's some webinars that we could point you to and that type of thing. But is there anything new this year about the report versus previous years?

13:19Every year we look for opportunities to add new things to the report. So we've added a few metrics. The first two that I'll cover are related to, that give you more of like a granular analysis of code review bottlenecks. So the first one is approved times. So this is the amount of time from the first comment on a PR to when it's actually approved. And then merge time. So that's the time from the first approval to when the PR is merged. There's also a new metric to help measure upstream conditions that create review cycles. And we call this PR maturity. And this is probably actually one of the more unique additions that we've added this time.

13:55And that is the ratio of changes added to a branch after a PR is opened. So if you see a ton of commits coming into a PR after it's been moved out of the draft stage and is now actually in the ready for review stage, it's how many more changes after that do you actually see on the PR? So I think what our goal here really is to just, you know, both to give you more granular analysis, but also to give us more indicators that can help us sort of comprehensively understand productivity and experience. Yeah. I like that they added these, the approved time and the merge time. I think it's, you know, obviously they're a part of cycle time, but I think it's kind of like every team works a little bit differently.

14:38Sometimes you might want to focus more on approved time or more on merge time. Like for example, let's say, okay, merge time, the time from first approval to when the PR actually gets merged. Like maybe you have some big process or maybe you have some slow process of like, hey, you know, the PR is actually done being reviewed, but it takes a long time to get merged for some reason. I don't know, maybe integration failures or something like that. So I think that granularity kind of probably reflects like being flexible about how teams are actually working. And then the other one that you mentioned, PR maturity, like that one is pretty cool.

15:17the ratio of changes added to a branch after a PR is open. So that's kind of looking at like how many times do I need to go back and forth? And I'm hoping as we move forward here, like you can see so much is happening with AI reviews and like bot-based reviews. I'm hoping PRs are going to get more mature. I guess it seems like I'm like talking about my kids like, oh, these PRs are really, they're growing up. They're getting more mature. No. With AI, like let's say, okay, I'm a developer. I'm developing. I open up a PR. I can have all of this AI stuff run on it before I'm actually even looking for a review.

15:57So it's like I can kind of get it with my like bot buddy to a better state before I'm wasting other people's time. So I'm pretty interested in how that metric goes over time. Yeah, and we'll actually get into it. I have some insights about PR maturity later too. So that's what really excites me about having it is we're actually able to start drawing correlations about it now as well. Okay, so is there anything else new in the report that you would like to highlight to us? This report also includes a whole bunch of new metrics related to project management hygiene. And I know this sounds so boring, but it's actually pretty fascinating.

16:35Or it gives us some very fascinating insights about how engineering teams operate. So we have a few metrics that serve as proxy metrics for project traceability. So these are things like issues that are linked to a parent. So this is like the percent of tasks that are linked to a specific parent, such as like an epic or a story, trying to help you like make sure that things are actually being rolled up into the overall initiative within your organization. We also have benchmarks for branches linked to PM issues. So this is a percent of code branches that contain a reference to a specific PM task.

17:12So again, like helping you roll that up by going all the way from the actual work that's being done in your code repositories and making sure that that's connected all the way up to like an initiative that's happening with your organization. And, you know, what we're trying to do is help teams monitor their development progress and ensure that actual development work is being tied to defined tasks within the organization. Yeah, I think that's pretty cool. Well, it's actually nice to see how this benchmark report has kind of evolved. And similar to PR maturity, I think this report is a little bit more mature because the things that you're talking about, yeah, they might sound boring on the surface.

17:53I don't think they're boring at all. Let's give an example. Let's say that we're giving you a benchmark for cycle time or even those PR merge to whatever. Let's say that you're really, really good at them and you're elite. So you read the report, you find out you're in the elite category. It doesn't mean that you're actually delivering like good business value or good project value. You could be working on something that's like completely irrelevant to the business. And I think with some of this hygiene stuff, we're kind of evolving as an engineering organization to say, yeah, we're certainly not just like code monkeys.

18:29We're not just like a machine that goes out and does stuff super, super fast. were highly essential to the business. So let's give some metrics that show that we're delivering great value, can measure the value, that type of stuff. So yeah, I'm super pumped that this got added into the report. Yeah, and then we have two other new measures for sort of related to this for planning and ownership for work items. So one of our new benchmarks is in-progress issues with estimation. So that's the proportion of ongoing PM tasks that have a time or effort estimate assigned. And then in-progress issues with assignees is also another new benchmark.

19:11So the percent of active PM tasks that have a designated team member responsible for completing them. So again, what we're trying to help here solve is to ensure more predictability, more accountability, and measure whether your organization is able to effectively manage workloads. Because you think about something like time estimations, you know, not every organization t-shirt sizes their work, but it can be very helpful to know when you have large portions of work that are not getting time estimated, because that could indicate, for example, that there may be some predictability risks coming down the pipeline.

19:49I love it. I'll double down on what it said. I think it's like being a pro engineering team or like a highly elite engineering team. Predictability matters a lot. And some of these metrics, I can't wait to see what the benchmarks actually are here. But they're really saying like, yeah, I'm an elite efficiency and quality organization, but I'm also elite at understanding workloads, therefore understanding developer experience, therefore delivering on time to the business. Like that's how you be the best, I would say, professional engineering organization. I'm going to move us now. So we have an insights and takeaways section.

20:29So yeah, Ben, great stuff. Giving us the overview. Really interesting things that have been added. Love that they have all of the project delivery predictability stuff now in the report. So thanks for running that all down with the audience. Let's close the episode on the insights and takeaways engineering leaders can gain from the report. Yeah. So the biggest one that was actually very surprising to me is related to cycle time. And that is specifically longer cycle times correlate with quality risks, specifically around change failure rate. We've long known that there's this correlation between cycle time and efficiency.

21:12Like it's an obvious correlation, but we still have data to validate. It was actually pretty surprising to see the stark difference in metrics like CFR, change failure rate for teams that had faster cycles versus slower cycles. In fact, I think, if I recall, I believe organizations who fell into the elite category of cycle time were like half as likely to fall into the needs focus area for CFR. That's interesting. Additionally, this carried over into deploy time as well. So a longer deploy time also correlates with higher CFR. No, that's really cool. It's like the rich get richer, right? If you're really efficient, then you're also kind of doubling down on quality.

21:53I would have to guess it's probably because if you're that type of organization that can have an elite cycle time, it probably means that you have the right automations in place. Right. You probably also have a lot of automated testing, a lot of integration testing. Maybe you've even started playing into the AI game, AI review game, all of that, maybe even AI test creation. And so I could see kind of just like without even thinking about the data, if you think about maybe like some of the best engineering teams in the world, you think of like Netflix, you think Spotify. Yeah, they probably move super fast and their quality is super high because of that ecosystem or however you want to describe it.

22:38Surrounding the developer is probably really tight. And the other thing is you notice Ben from being a developer. It's like as code is sitting there, it almost gets stale in a sense or like things are moving underneath it. It's like only bad things can happen. The longer code is not merged. Yeah. Yeah. And picking it back up after it sat for a week can be such a huge challenge. So keeping the theme on cycle time. So now that we know that cycle time is both a thing to focus on if you want quality and efficiency. Let's talk about some of the things that impact cycle time. So one of them is a thing that we've thought for quite a while is the case, but it's nice to just have data that validates our theories.

23:22And that is that PR size drives velocity. So smaller PRs reduce pickup time, they reduce merge time, and large PRs sort of do the opposite and actually even require more review modifications typically. We found things like larger PRs wait longer for reviews and have a longer overall cycle time. PRs that wait longer for reviews also wait longer to be merged after approval. Larger PRs are more likely to be heavily modified during the review process. So there's almost this cascading negative effect that is created when your PR size gets too big. Yeah, but again, totally makes sense. Like all the companies that I work with, I always recommend trying to get your PR size into the elite benchmark because you get that double down effect.

24:16One, you're going to be more efficient. You're going to be faster. Two, your quality is going to increase. So it's like any time that you could focus on a metric that does, you get the two for one, right? What else do you got for us here? Yeah. So I mentioned the PR maturity. We found some insights with that. Specifically, it correlates directly with efficiency. Probably not super surprising to our audience, but a higher PR maturity reduces rework and speeds up merge frequencies. So it really kind of encourages better prepared PR submissions. A higher PR maturity also correlates with higher merge frequency and shorter pickup times.

24:55You definitely want to focus on a culture where developers have all of the insights, all of the expectations set before they've sent their PR off to reviews because the more you have to revisit it after it's been opened, the more it just creates complexity issues for your organization. Yeah. I mean, anytime that you're asking someone else to stop what they're doing and like come help with the PR, then you like review it, make the PR change, then come help again. Like it's kind of that opposite double down. It's like the double down effect in a negative way because you're affecting multiple people.

25:30Anything else in the, in here, Ben? Okay. Go for it. Yeah, I just got one last one that is a bit of an example of how not to do it. And that was an insight that we didn't really expect to encounter. We found that teams with poor project management hygiene actually move faster than those that have good hygiene. If you aren't referencing your PRs against a JIRA ticket or project management ticket, you actually probably have a faster cycle time than companies that do that consistently. however you know i would stress that there's probably some issues that you're introducing to the organization if you take this approach related to like misaligned goals technical debt lack of visibility into to where product work is actually happening so you know i'm not going to go out and make the recommendation that we all need to stop updating our jira statuses but i will make one recommendation.

26:28Automate it. Yeah. No, I love that. Automate it for sure. All right. What about the investment profile section of the report? I talked to a lot of engineering leaders about this who want to invest in developer experience and to demonstrate that type of spend to the business. What do you got cooking for us there? Yeah. So this is actually one of my favorite aspects of this report, and I'm really happy that it returns this year. And the name makes it sound very dry and high level. But we hear from a lot of organizations about how hard it is to justify putting investments into DevX. And I legitimately believe that measuring your investment profile is how you do it.