Labs: The Magic of Compound Efficiencies in Engineering

0:00But day to day, if you're not in the loop, and you're not the one that has the ongoing awareness, that's not going to work. And with developers, it's the same. If the VP or a senior manager is looking at reports and looking at metrics saying there are inefficiencies, the problem is here, they could try to move the needle by saying, hey, let's fix this. That's a long process. And if the developers are not aware day-to-day of the same problem in a way that they can also react to, that's not going to work. Have you ever wondered how your dev team ranks in terms of productivity, speed, and business impact?

0:36With Linear B's new engineering benchmarks report, you can find out. The product of comprehensively analyzing the work of almost 2 ,000 dev teams to close to 1 million branches, it's the first ever look at what performance metrics make engineering orgs elite, average, or underperforming. Best of all, if you want your dev team's number to go from average to elite on any of the benchmarks, the report also provides concrete guidance on the behaviors, tools, and processes you need to get there. To explore the report in full, Visit LinearB.io slash benchmarks or click the link in the show notes of this episode.

1:10Welcome to another Labs episode of Dev Interrupted. In these episodes, we dive into the most impactful research and insights about engineering, and most importantly, how to apply them to your organization. I'm your host, Conor Bronston, and today's topic is one the data team at LinearB has a unique authority to speak on. Because they've spent all of their time providing visibility into the development workflow process, they've unearthed parts of the software development pipeline that seem to be taking way too long, and they've experimented with tools and ideas that have helped dev teams program workflows and make these problem areas more efficient.

1:44In this episode, we're going to be talking about what these efficiencies are and the compounding effect they can have on your entire organization when you implement them together. For example, we'll be discussing a way dev teams have cut cycle time by 60%, but only after they implemented another efficiency that previously cut cycle time almost 47%. In short, this episode is all about the power of compounding efficiencies and engineering. For those of you who've read Atomic Habits, you may be familiar with the concept building these habits within teams or your personal life to compound efficiencies.

2:19And we're going to run through these efficiencies so that you can layer them on top of one another to completely change the way your organization works and improve productivity. So walk me through the places we found room for improvements in dev pipelines, ways to create efficiencies, and their overall impact. I've got LinearBees CTO, Yashai Berry with me. Yashai, welcome to the show. Hey, it's great to be here. It's great to have you back. I really enjoyed our last episode we did together around the best programming languages for dev workflow. I'd love to start by diving into where all this research comes from.

2:53Yashai, can you tell us a bit about where all the information that we're to talk about today with Source from? Sure. So yeah, in Linerby, we're fortunate to have access to the workflows and the data coming from thousands of dev teams on our free offering, on our paid offerings. And we have a unique window into understanding what's happening in the PR process, what's happening throughout the development process, where the bottlenecks are. So yeah, we can see millions of PRs, thousands or tens of thousands of developers and thousands of teams, a lot of data that helps us understand how this looks in smaller teams, large orgs, different behaviors, like you mentioned, programming languages.

3:33So a wealth of data in my data team is spending its time picking through that to surface insights about the things we care about. This is about how to improve the developer experience and how to improve productivity by removing those bottlenecks. And there's a great analogy for how you're improving the efficiency for dev teams and something we can all understand, food. If you ask a nutritionist what is most effective as a diet for losing weight, they might tell you that all diets are effective in helping you lose weight because they give you the visibility into what you're eating. They make you think about it and how much you should be eating.

4:10By being conscious of what you're eating and knowing the right amount of food to consume for your activity level, you'll start to move towards your natural balance and you'll see those opportunities for improvement. And I know you've said that this can apply to engineering as well. Can you tell me a bit more about that? A lot of the change that we want to create, and this could be about our personal lives or about the death process, a lot of it is down to human behavior. So changing human behavior is about changing habits. You mentioned the food analogy, like being aware of what I'm eating or not eating, that's a behavior change.

4:45Not like just the awareness, just being not eating automatically, emotionally, whatever. There are tons of reasons why being aware helps me control and helps me improve my behavior. The same thing happens with development and dev teams and their processes. By being aware of the productivity, the inefficiencies, the bottlenecks, by talking about them, by measuring them, by surfacing them, by making them important. And part of what we are striving to improve, that alone starts to move the needle. because if my focus is only on writing the code and I'm not aware of those inefficiencies, those wait times, those idle times, then I may be able to create a great code, but at a level of efficiency that is not optimal.

5:33If I'm starting to think about and live the pain and give it a name, give it a number, like the contact switches, everything that is blocking the team from moving faster, now I can begin to focus and change behaviors around. So awareness is the first level, the first layer of behavior change, and that begins to move the needle. And is this awareness and visibility component mostly crucial for leaders within the org, or do you see it as something that should cascade throughout the org so that every dev has access to? So if you compare this to food, if your nutritionist or consultant consultant had, like, is aware of what you're eating, but you are not, that would probably not move the needle, right?

6:20If you meet every three weeks and look together at a report, then they may have insights and they may say, oh, you're eating too much of that or too, like, you need to change. But day to day, if you're not in the loop and you're not the one that has the ongoing awareness, that's not going to work. And with developers, it's the same. If the VP or a senior manager is looking at reports and looking at metrics saying there are inefficiencies, the problem is here, they could try to move the needle by saying, hey, let's fix this. That's a long process. And if the developers are not aware day to day of the same problem in a way that they can also react to, that's not going to work.

7:03And we've seen that with dev orgs that are focusing just on visibility. Show me the metrics. I'm going to visit the dashboard every three weeks, every month, I'm going to do a retro with my teams. That's nice. That's not enough. That's not really changing the behaviors. So how do you think teams should go about implementing this initial layer of visibility for both leaders and individuals so that they have the right amounts of information? Because I think the concern you're hearing me kind of allude to is, okay, I know most devs do want to know what they're eating, so to speak, and the different impacts that their decisions are having on the broader organization, but also their focus is typically on delivering their project.

7:46It's not necessarily looking elsewhere. How can you start setting up this program so that everyone is getting the visibility they need? So I think it begins with understanding what to measure and what metrics are going to be conductive to improvement, to the culture, and so on. This can go wrong if you're measuring the wrong things. If you're, I don't know, if you're counting lines of code, and we've all seen those public examples, that can go wrong in so many ways. So begin by understanding what to measure. What are the metrics you should be focusing on, which really represent the problem you're trying to solve?

8:18Dora metrics is a good example, a well-known example in our space of a set of metrics that our team focus. They're focused on the process, not on individuals, and are a good place to start. So we have Dora. There is another set of metrics called space, which is also becoming popular. But the key is to measure the process, not people. You're not measuring or stack ranking developers. You're looking at where the process is broken or can be improved. You're measuring and helping the teams. And I think the final step here, and this is where I mentioned, not just a manager looking at metrics, you want to use metrics that also lend to an immediate response.

9:02Something I can do right now to fix a micro change in my behavior that changes those metrics or that problem in a small way for a single pull request or a single piece of work, instead of looking at the metrics and saying, I had a problem last month. So this would be something like breaking PRs down into smaller chunks to enable easier review and cut down on context switch. Right. That's a great example. Small PRs are like a leading indicator. of good Dorometrics and good short cycle times, quick time to delivery. So by measuring the leading indicators, things that you can actually start changing to change the picture, you get a chance of doing a little better next time or a little better in this PR.

9:52If I'm aware that PR size matters, if I'm aware of what is the size of the PRs that I'm creating and the team is creating, and we also talk about what is the desired state. We can set a goal or decide that we want to have our PRs typically lower than, I don't know, 100 lines of code, 200 lines of code, whatever. Now my awareness as a developer of the importance of breaking down PRs is not just a theoretical thing I learned in school or had the manager tell me, this is a live thing. And if I can now, when I'm working on this new item, new bug fix, new ticket, and I've broken down this to two PRs, which are independent and smaller, I've started fixing the problem.

10:35So this is a good metric that lends to immediate behavior change. And in small amounts, this single PR that I broke down into two is not going to change the overall metric for the team over a month, that alone. But every dev doing this every day. Every dev doing this for even some of their PRs and starting to shave off the PRs to become smaller, that is the behavior change. Now, it's not just for the metric. The metric having that number on the wall, if you like, helps move that behavior. But the real reason why the developers will stick to that new behavior is because they see the benefits. Those smaller PRs get reviewed much faster.

11:19They get merged faster. They create less risk in the code base. so I don't have to roll back or fix a bad release. All these things are things that developers feel in their gut. Having the number on one side, having awareness that is important, but also seeing the immediate impact of my small changes. Oh, it's so much fun. My PR got reviewed like this. I'm going to do more of that. That behavior change will now persist. And it's no longer dependent on the metric. So great. Let's put metrics into place. Let's get visibility for both. you know, leadership, but also the entire team, that has a certain impact.

11:57But now it becomes time to say, okay, how do we apply that visibility and knowledge into your actual software development pipeline? And it sounds like code reviews, something we've talked about quite about in the show, is the key area where you think teams can make an impact based on your research. Yes. So first of all, while there are some alternative methods of collaborating as a team to create value through software. There's trunk-based development and other modes. The vast majority of the industry uses pull requests. That has become a standard, carries a lot of benefits. And almost everyone uses PRs or MRs as a way to manage how the team collaborates on creating new code.

12:41Given that, we now have a stage in the development process that is very human-centric and relies on like asynchronous interaction between people. I put up a PR. I asked some people to take a look. They need to know that I'm waiting for them. They need to find time. They're reviewing, giving me some async comments. Back and forth, eventually we'll get to a place where the PR is ready to merge. This is a very different dynamic than the rest of the pipeline. Like when I'm coding for a new PR, typically it's one person or collaboration on, But typically it's just me with a problem with the code. I may have some interaction with the requirements and the product side of things to understand what I need to do.

13:24But it's more like this is creator time. I need to be in the zone. I need to be uninterrupted to be very productive there. But I'm not dependent on other people in a material way. After the PR is merged, typically this is now machine time. There are going to be automatic tests. Sometimes there's a manual testing, a QA process that happens before or after merge. But in many ways, what happens after merge is already automated or semi-automated. And it's, again, less dependent on frequent interaction between humans to get to an agreement. So the PR process, from my suggestion to add to the code dates all the way until it's approved and merged, has a different timescale.

14:10It dominates the time it takes to create value because it is human dependent. We humans do not respond in seconds in our jobs. Like reviewing a PR takes time. It's not a 30 second run of a CI job. It's where dependencies start to come into the process. Yes. And there are dependencies of humans rather than machine or code or automations. So those two specific behaviors cause the PR process to be very important. And if we are spending time there, we are waiting for the review or the response for a review. We are now creating contact switches which are expensive and painful for the developers. I have to respond to comments on a pull request I finished three days ago.

14:56I've already moved on. I'm working on the next thing. I have to reset my brain to what I was doing before and what this review means. And this back and forth can take a while. So that is why the PR process is a very lucrative place to find and remove dependencies in bottlenecks. And also those kinds of improvements really help the developer experience. Everyone wants to get their job done to push value, to fix the bug and move on. and if I have to wait for someone else to review my stuff and eventually to convince them that it's okay and we can merge, that is holding me back. And typically developers like to be in teams that respond fast, that help each other quickly, get things done, small steps through small PRs.

15:45That all creates a feeling of achievement that is ongoing. Yeah, the statistics I'm seeing here in your research are fascinating. I mean, the average cycle time across BNRB's research is a full seven days, but half of that is spent in the PR lifespan. So there's clearly this huge stumbling block here, even though when we break it down, cycle time, in fact, will go down significantly if you cut a PR from, to your point, like 200 lines of code to 100 lines of code. And conversely, it'll double when going the other direction. So I guess my question is, if we know there are these sticking points, this is like a problematic area for dev production, what benefits do you get simply from adding visibility so that teams are aware of this?

16:32Yeah, so if we identify that this is a key area of the process that has a lot of inefficiencies and creates friction, then you want the metrics and your visibility to focus on that. It's not the only thing to measure, but cycle time, one of the important metrics, a good place to start if you're not measuring anything, really gives you visibility into how long does it take me as a team to move value from beginning of coding a new piece of value or fix or whatever, all the way until it's in production. And a large part of that is going to be the PR process. Typically, you want to break cycle time down to its different segments or behaviors, because when I'm coding, the behaviors and the bottlenecks are different than when I'm doing the PR process, waiting for a review, waiting for someone to even take a look.

17:18And eventually when the PR is merged, there is like deployment time, which is, could be automated, could wait on external signals, could be another team's problem. So because of the dynamics inside CycleDimerDifference, you want to segment them out and have different measurements for each part of this, each phase of the process. So measuring that area and getting some detail on the different steps and behaviors is crucial. As I mentioned, also adding some of the leading indicators and PR size is a great leading indicator. We've seen, like you mentioned, smaller PRs will get reviewed faster, will get merged faster, will be deployed faster and safer.

17:56The data is very strong around, as you mentioned, like going from a hundred lines of code to 200 lines of code typically tends to double the cycle time. 200 lines is still pretty small as an average is a great place to be. Moving up the PR size makes everything slow down. So because we know it's such a important leading indicator, let's measure that and let's start behaving, start changing the behavior around that, not just around the final cycle time metrics, which are the result. So that is a good example of measure the important thing, which is the cycle time, and then add measurements of what is going to predict and impact cycle time, like the PR size.

18:39and giving the developers visibility ongoing on the PR side, not just a total retro kind of view. Do I have a problem with my current PR? Is it waiting too long? Is it too large? Is it within the boundaries of what the team wants to have as its best behavior? That allows me as a developer to start shifting my behavior. So a nation of metrics that measure the right problem and ongoing awareness and ability to respond to those metrics, not just to retrospect you. So that first level of visibility where you can break down your cycle time into segments, understand your metrics, and start to respond as a team or as an individual.

19:23When someone does that with Linear B and starts to manage their software development life cycle in that way, what is that first compounding efficiency impact? What do you see in the numbers when people do that? Right. So what we see is that you should be expecting a very sizable change and a very noticeable change in the size of PRs in cycle time. And it's parts like how long does it take to pick up or review a PR? I'm talking about 30, 40, 50 % improvement over across a quarter, maybe four months of using letter B or a similar system to measure. and if the context is provided, like I said, to developers ongoing, not just a retrospect, then we see across the board, cycle time gets slashed by a half.

20:13That's a very typical and very immediate improvement that you see. A lot of that comes from the PR size. Like PR sizes go down by a third and that causes a lot of the improvement because again, smaller PRs get picked up faster. But also awareness to the other, to the actual cycle time and it's segments. If I'm looking at pickup time, which is how we define how long does it take a reviewer to begin reviewing a PR that's waiting for them? This is all about communication, all about how do I let them know that they need to review a PR? Getting the email from GitHub is not good enough. No one sees that.

20:51We used to be able to shout to each other in the room, hey, I just have a new PR. Can I take a look? Or Slack each other. But these are all manual, broken methods. So having some automation, having some solution around the communication between people to help people know that something's waiting for them, that creates stuff like, you know, you can slash pickup time by half. A very difficult case where I can get responses to my PRs very quickly. And if they're small, even more quickly. And that really creates those double digit, 30, 40, 50 % improvements across the board. So as you move from non-formal processes and begin to streamline them for pull requests, things like assigning reviewers, standardizing PR size, like you mentioned, maybe adding context through tooling, how can you attempt to resolve these issues in the PR process?

21:48So yeah, you mentioned tooling, I think, and context is key. So we talked about the metrics and understanding why the behavior change is needed. But additional things like context, I'm going to give some examples, are also crucial to start moving faster. We analyzed some data and we talked with customers and we understood that people that are now reviewing a PR, people that are basically getting pulled in from what they're doing to help you get your job done, besides sometimes being interrupted and sometimes having overload of reviews, on the surface, all of these review requests look the same.

22:28Like someone needs a review. Okay, what is this? Is it urgent? Is it a bug fix, like an urgent bug fix for production? Is it a new feature? How long is it going to take me? Is this a big thing? Is this a small effort? Can I do this in the five minutes I have left before a meeting? or do I need to set aside an hour to do this? Many, many context pieces that are typically not available. You just get the link or an email to a PR. You have to go into it and start looking to understand what's going on. You open the PR, you get this big wall of red and green and you say, oh, I don't have time for this.

23:06This is complex. The more context you're able to push as a developer towards your reviewer, the more upfront work you can save them, the discovery work and the orientation work. So great PRs also describe what they're doing in terms of the code. If you as a developer can spend some time to add context, that will get you a better review and it'll get it faster because the reviewer can make an intelligent choice about when to do it. And am I the right person to do it? And the more context, the better their process will be. So some of that is on the developer to improve how the PR looks and how the PR describes itself.

23:45But also there's tooling that can help you with some of that context. Surface information about the JIRA ticket this is assigned to or related with. Is this a bug? Is this a story? Is this a P0? Tell me how long it's going to take me to review. So we have implemented a machine learning model for estimated review time. And we can tell you when a reviewer gets a request to review a PR, this is going to take you two minutes. This is going to take you 30 minutes. okay now I can make based on the area of the code base and the number lines of code or what inputs yeah it's based on like a multitude of features or parameters like which files what types of files the size of the change area of the code base who is who is the the reviewer and who is the the coder and again we have a great data set to train this with and we see the reviewers respond to those cues and to those that additional context and change their behavior so now when they know this is a small PR and it's a quick win or like this is going to take me two, three minutes, they jump on it quickly.

24:48And when they see this is going to take me 30 minutes or an hour, they allocate time, they schedule this for later. So they can now manage their contact switches better. And this means they respond faster, overall faster, faster for the small PRs for the easy wins. And instead of, oh, everything looks the same, I have to guess or spend my time taking a look and stepping back. So context is very, very important. And it's part of solving the human communication problem, which is inherent to the PR process. So it's about letting people know and about letting them have enough context so they can make a smart decision.

25:29So once teams have this visibility into their metrics, and maybe they've benchmarked it against industry standards with some of the research you've done, and they go into the PR process and they start to fix things. They go, hey, maybe we need to move from 250 lines of code to 150 or 200 to 100 and provide some context with tooling, programmable workflows like GitStream, which I know you have your team experimenting with a lot. What are the effects that you're seeing once those efficiencies are being introduced into the code review process on top of the visibility work that's already been done?

26:01We can talk about three layers. There's the visibility and awareness. Then there's micro-automations of the human process. In Liner B, this lives inside our Worker B. SmartBot lives in Slack and Teams. And this is about nudges for delayed PRs. This is about the additional context that I mentioned, like estimate review time. it's about letting the reviewer know that they have some work to do and letting the coder know that the review is waiting for them to respond to all of these micro-optimizations move you from the 30 % improvement to the 50 % or 60 % improvement that's I would say the first line visibility and some automations the second line you mentioned get streamed that's going even deeper into automation and this is about not just the human interaction problem, but it goes deeper into understanding my process for doing PRs and for like approving code.

27:02Today is pretty uniform. Almost all companies have a policy. If someone has to review like four eyes and we're good. This is very typical. Everything needs to get a review. It doesn't matter what kind of review, as long as it's approved by someone with authority. We're done. Some companies or some orgs with higher regulation or requirements have a blanket policy of I need two reviewers, six eyes. But typically the process is very rigid, very uniform, and cannot take into account the differences between a PR that changes the line in the documentation or replaces an image for a website from a PR that changes a logic, like code logic or touches sensitive areas, changes an API, handles tokens, whatever.

27:49And a lot of that artificial uniformity in the PRs is because of lack of tooling. There's no easy way to make those distinctions and to codify what's the required behavior across different types of code changes. So teams just default to a very blanket approach. And that is not efficient. So GitStream, this is our way of codifying a process and starting to tease apart what actually should be done. What is our desired process as a team or as an org for getting some code approved? And now you can automate things like deciding whether this needs one review or two reviewers, maybe pull in a security expert for this PR because it touches relevant areas in the code or relevant.