Labs: The Evolution of Continuous Merge | DevCycle’s Nik LeBlanc

0:00In today's fast-paced business landscape, the success of your business relies on the success of your engineering team. And the impact on engineering leaders has been clear. We must deliver exceptional software while driving business outcomes. In short, we have a dual mandate. Enter Linear B, the software delivery management platform designed specifically for engineering leaders. With Linear B, you gain the visibility and automation you need to streamline your processes and unlock your team's full potential. Imagine a world where your engineering organization effortlessly manages business outcomes.

0:35Linear B empowers you to accelerate software delivery while ensuring your projects align with your strategic objectives and reporting out to key stakeholders. Gone are the days of disjointed workflows and missed opportunities. With Linear B, you'll harness the power of advanced analytics and real-time insights, allowing you to make data-driven decisions that drive business success. From tracking progress to identifying bottlenecks to optimizing resource allocation, Linear B has you covered. It's time to take control of your software delivery pipeline and unlock new levels of efficiency. Experience the power of a platform designed by and for engineering leaders like you.

1:11Discover the future of software delivery management. Visit LinearBee.io today and start transforming your engineering organization.

1:21Hey everyone, welcome to Dev Interrupted. This is your co-host Connor Bronson and joining me today, I have two incredible folks. I have helping me co-host Ben Lloyd Pearson. Ben, welcome in. Yeah, thank you. Absolutely. So you may have heard Ben on previous episodes, including our recent Labs episode. He is Linear B's Director of Developer Relations. And together, we're talking with Nick Leblah, VP of Engineering at DevCycle. Nick, welcome to the show. Thank you very much. And I heard a rumor this might be your first podcast, Nick. Is that correct? Yeah, I'm terrified. No, you're going to be great.

1:55I hope everyone here who's listening will kind of root Nick on a bit. That's cyclically positive probably for him. Because today's episode is going to be another great Labs episode. And in Labs, as you know, we explore the data, research, and insights that are most impactful to engineering organizations. We won't be doing a data deep dive this week. If you want one of those, check out our recent episode on engineering benchmarks. But we will be exploring engineering team composition and knowledge distribution. Specifically, how they relate to Dora metrics and a concept that Linear B is calling continuous merge and the tool that we built to help push this forward called GitStream.

2:32Ben is our resident expert on continuous merge best practices and GitStream, and Nick has adopted some of that tooling with his team at DevCycle. So we're going to ask him about his experience and how he thinks about some of those concepts as a VP of engineering. But first, we are going to start off by talking about team composition and that knowledge distribution piece we mentioned. So, Nick, let's start with you. You implemented a practice which you admit is a bit controversial on your team and I think maybe speaks to your approach. As I understand it, you are breaking up engineering teams and recombining them to help distribute knowledge and rebalance teams.

3:05Can you elaborate on that? The history of DevCycle, we're also a company called TapLytics that focuses on A-B testing. And before DevCycle came around a couple of years ago, we ran into issues with like, how do we maintain our SDKs? How do we make sure that if there's bugs coming in with our SDKs, and we had many, how do we stay on top of those? And the problem that we had is we had basically one Android developer who was now a product person. He no longer did Android. We had one iOS developer, and then we had our CTO. And our CTO happened to cut his teeth on Objective-C. So that was our team for dealing with bugs on our mobile SDKs.

3:48So there's a big bus problem there, because basically if we were to lose our iOS developer or Android developer, we'd be screwed. We wanted to get away from that problem with DevCycle, so we wanted everyone to be contributing to the SDKs. So that speaks to the cross-functional composition of the teams, and how we wanted to make sure that any team could take on any type of work at any time. The idea of kind of cutting them in half, traditionally teams have been led by like what's known as a team lead. And that's kind of like my background. So I started as a team lead at another company and then came here as a senior and then was a team lead, then director, then VP.

4:32Team leads are kind of hard to find. And there's not a lot of people in my experience that want that responsibility because it's ultimately, you're responsible for the code, you're responsible for the happiness of your team, you're responsible for everything. But you need some kind of presence that's pushing the team forward and making sure that everything's on track. So instead of trying to find new team leads, what we did is we just kind of invested in developers that we already had, gave them an opportunity to try the position, and then split the responsibilities of team lead into two positions.

5:07So we had basically one that is a project lead and one that is a tech lead for each team. So what that means is the project lead is responsible for basically making sure that the project stays on track, that Jira is appropriately updated and reflects the current state, that people on the team are happy. The technical lead is responsible for making sure that the code coming out of that team meets a certain standard. And that worked fairly well. and continues to work fairly well. But the fact remains that these are all people that are new to the position. So they're learning how to operate these teams as they go.

5:47They're learning their own skills, learning their own strengths and weaknesses. And each team is kind of developing their own process for how they operate. We have been running like that for about a year now, and each team had learned a lot. So one team is particularly good at Area A, another team particularly good at Area B, and relationships were forming in these teams that didn't have the opportunity to form across the teams as easily. So we decided to just kind of break it up. So we kind of cut each team in half, and then we split those halves together to form new teams. The point of that was basically to make sure that the leads had an opportunity to work with different leads.

6:30and they had an opportunity to blend what they had learned about what works best for their teams into kind of like a cohesive hold for like how any team can work at this department at this company sorry now the the controversy of this is like it's pretty bad to break up a team when the team is operating well and obviously you're never going to find an ideal time when basically a team's work is over. So it's hard to find that ideal cutoff point where you can say, okay, team A has done their work, team B has done their work, we're going to cut them in half, we're going to make new teams now. So there's always going to be an interruption, there's always going to be chaos, and there was, but it wasn't that bad.

7:12We basically pulled the trigger on a Tuesday, and by Friday everyone was operating as though nothing had happened, which was fantastic to see. And then I I give so much credit to the members of the team for their willingness to do this and their enthusiasm for it. And it just kind of showed how well the members of the department operate together and what they're willing to try to basically ensure that we're operating effectively and that we're learning as we go. I mean, the amount of experimentation that you're doing is actually really fascinating. And it kind of makes me think about, like, you know, there's this meme out there that, like, everyone's becoming a full stack developer these days.

7:56And in some senses, you're almost taking that to an extreme. But it seems to be working in many ways. So, you know, as you sort of experimented with this over time, what are the results that you've seen? Like, how have you adjusted based on, like, you know, I really love the example of, like, deciding you need to actually break up your teams. Like, you know, what have you experimented with on this and how have those results manifested? Yeah, well, one thing that we've certainly noticed is that, let's say, junior developers that might be leaning more front end or more back end, they're gaining a lot more confidence a lot more quickly outside of the stack that they're comfortable with.

8:36There's a couple examples within our company of like junior members that feel comfortable in the front end, but very uncomfortable outside of it. And that comfort level has increased dramatically. And through that, they have more confidence in themselves. So they're being exposed to more technology, essentially. There is kind of a counterpoint that it's hard to develop particular expertise in one part of the stack or the other. So that is certainly a compromise that this structure accepts. But especially for junior developers, I think that breadth is better than a particular focus at this point.

9:12That's a great example. So I'm kind of curious then, have you found any sort of ways to address the lack of expertise? Are you starting to build that internally or maybe hire specifically for it? Let me maybe expand on Ben's question here. How are you leveraging tooling to enable these changes you're making in team structure and composition? Yeah, okay, so that's a good question. We've done some crazy stuff with JIRA. And I know everyone loves to hate JIRA. I'm kind of a fan of it. I've learned to love it. Basically, each team has a board. They call it a focused board. And at any given moment, a team is working on one particular Epic, and that Epic is a feature.

9:56So Epics equal iterations of features around here. And what they do is they basically, on their focus board, they're able to see all of the tickets of the Epic in question, or they're able to just see the Epic in question. and they're also able to see any particular bugs unrelated to epics that have been assigned to their teams. We also have guilds, which is kind of another thing that's been historically difficult to make work well. Guilds can either be an opportunity for discussion about a particular part of the platform or technology, or they can be an opportunity for like-minded people to focus on a particular part of the stack, build up backlogs that are less product focused and more, let's say, technical debt-focused or investigation-focused, and they can try to slot that work into their team's roadmap.

10:45So basically, we have teams established with JIRA, and any project work goes to a team through that. And then we have labels that get associated to guild tickets, and those automatically get assigned to, those appear on the team's boards as well. So we've gone through a lot of experimentation with the boards and how to best represent the work at hand for a team and how to basically prioritize what killed tickets the team wants to pick up and what tickets the team doesn't. And my understanding is you're leveraging continuous merge tooling to enable that more frequent deployment. Can you share a bit about how that is impacting this team structure you've developed?

11:27Yeah. So one thing that we haven't done yet with continuous merge or with GitStream that I'd love to is figure out how to assign PRs to people based on their teams. And I know there's a lot of examples in your docs about how to assign them based on familiarity with the code that's changing or lack of familiarity with the code that's changing. So that's something that we're going to lean into heavily in the very near future. But what we use it for now is essentially to estimate the amount of time that it would take to review the PR and drop those labels on the PRs, merge any depend-a-bot PRs automatically or any PRs that just deal with tests or docs.

12:05I think that's just a prime example in your docs, and it's worked well for us. A lot of our Dora lead time metrics are being affected by dependent bot PRs just sitting around and not being looked at. So it just kind of forced us to act on them. So what do we want to do with these? Do we actually want to merge them, or do we just not want to merge them? So now that kind of took that question away from us, and now it just goes in if it passes tests, which is fantastic. That's pretty much the extent of GitStream at this point. But it was largely like the reason we adopted it is because of door metrics.

12:38We saw blockages in our pipeline and we wanted to smooth them out. And now I want to use it to basically give people opportunity to explore areas of code that they otherwise have not yet. Yeah, that's actually something I've heard from quite a few people that have checked out GitStream in that, you know, they don't really want to bias their processes too much to relying on experts. but they also don't want to bias it too much on trying to push random reviews onto people in an attempt to share knowledge. So the fact that we are able to give you these knobs and switches that let you fine-tune it for your own process, it kind of gives you the best of both worlds, right?

13:17You leverage your experts when you need them, but then share knowledge in all other situations. So yeah, I really love how you mentioned that Dora kind of led you down that path. So, you know, beyond like pickup time and review time, like, is there any other sort of thing that has helped you make that connection between like the value of tracking Dora metrics versus actually like implementing these processes? One thing that I've actually gotten a lot out of Dora is basically something to celebrate with the team. Like, hey, check it out. We're doing great. And it's often hard to pull yourself back and see the big picture and just appreciate your accomplishments.

14:00So by being able to review them daily and say, yeah, we deploy like 10 times a day this week, that's really impressive compared to other companies. And so I just like to make sure that the team is aware of that, appreciates what it means and how our process differs from other companies and allows us to ship this quickly. And obviously gives us insight into anything that's affecting our ability to do so. Yeah, and I think you probably also get, you know, if something isn't going well, you at least get some insight into why so you're not just wondering, like, you know, why does it feel like we're not getting code shipped as often or something like that?

14:38you actually have the real insight you need to understand it. So, I mean, what other issues, improvements, do you think you've seen with your developers more interested in being proactive about trying to find those inefficiencies and improve them? Yeah, yeah, definitely. There's a lot of attention being paid to basically just PRs, how long they sit. It's become an important part of the team's working agreements to discuss how long they're willing to let a PR sit before it gets reviewed. And they push themselves, and they find new ways to be made aware of PRs that do need action. So generally, we would use the GitHub Slack extension.

15:22And then there's always the process of, if you're between tickets, go check out some PRs. But what the teams are doing now, so for one, some of us are using Graphite. Graphite has a really good GitHub integration for Slack that I personally find to be better than GitHub's. So it's really good at informing you of PRs that require your attention and kind of like review cycles on those PRs. What teams are doing now to make sure that PRs don't get missed is within their team Slack channels, whenever a PR is ready, they will post it to the channel and say, this PR is ready. And then if someone is going to take a look at it, they drop the eye emoji on it.

16:01And now everyone knows that this is being looked at. And then if it's approved, then they will drop like a green check mark on it. There's no automations set up that this hooks into at all. It's really just like a visual indicator that the team can use within their channel. And this is one of the processes that kind of bled from one team to another with this, with like the crazy team stuff that we did when we merged the teams or split the teams and made new teams out of them. Because one team was using this really well and another team wasn't. And so now they both are. And it's just really helping just making sure that everyone's aware of the things that they're able to do to help the team.

16:38I'm curious to ask a more conceptual question about where you see these kind of tools, whether it's continuous merge tooling, stuff more focused on incident response moving in the future. Do you see these new concepts reshaping how dev teams are working? Or where do you see opportunities to grow out automation tooling for workflow in the development process? One thing that we're thinking about lately is like Dora measures your basically how quickly you can ship and the level of quality. So like, is it buggy or not? Are you going to break your systems with your code? Obviously, we're a feature flag company.

17:16So we lean on feature flags quite a bit within our own development process so that we can test things out safely and we can resolve any issues very quickly by like disabling a flag, for example. Also, our deployment process is fast, so if anything does go wrong that we can fix quickly, we can deploy it quickly. But what we're focused on lately is it's great if you can push, it's great if you can ship a lot of code, and it's great if you're not shipping bugs. But there's no real, there's no signal there about the value to the user. So you could be shipping high-quality code at a very high pace that users just don't care about.

17:54And so you look like an elite engineering team, but you're not picking up users. You're not giving them a great experience. So that's something that we're trying to heavily focus on now, which is basically like considering developer touchpoints. So any particular feature that we develop, you can imagine that it will export an interface or like, yeah, it'll expose an interface to basically like the API, the dashboard and CLI, for example. And so we consider those to be like developer touchpoints of features. So the features provide value. and then the experience of using the touch points also provide value.

18:29So we don't want a bad CLI experience for people. We don't want a bad API experience for people and we don't want a bad dashboard experience for people. So how do we measure that? How do we make sure that we can still ship quickly, still experiment, but focus on ensuring a good developer experience? The idea of we want people to be going on Twitter or going on threads and saying, holy cow, this CLI is great. or like, holy cow, this platform's amazing. So just unprompted celebrations, basically, of what we're doing. And we see that as a metric that we're looking for. We're also focusing on things like kind of more product-leading metrics, but like how long does it take people to activate?

19:08And so activate is just, it's a metric that we're determining to mean sign up for an account, get an SDK installed, serve a feature. Like how long does that take? How many people actually do it? How short can we make the time between sign up and activation? So these are all the metrics that we're trying to consider now to determine how good a job we're doing at actually serving our users and delivering value to them. Yeah, I'm wondering if maybe you can dive just a bit into the tactical side of it. Do you think this is something that a feature flag type tooling should solve for you? Or is it a gap in the tooling ecosystem out there that is preventing you from tracking that today?

19:51Or have you just not found the right tool yet? What do you think it is? So we know the tools, and we're setting up the tools. So we're using Mixpanel, for example. We're tracking particular points in the user funnels. It's just our focus on that that's changing. So if we truly believe that the CLI needs to be a wonderful experience and the API needs to be a wonderful experience, then we need to make sure that someone's accountable for that And that someone is actually overseeing what it means for that to be a great experience. So who are leaders in that area that we can model ourselves after? How do we compare to them?

20:32We've had a CLI for a while, and until we started measuring the usage of it, we realized that no one was using it even internally. So we put some stats on that, and we started tracking them, and then we've made some changes. and now the uptake has increased dramatically. So basically just trying to pay attention to the usage of these things from not just a product perspective, but engineering too. So they're determining the metrics that they want to be pursuing on their teams. They are establishing the tracking and setting up their own dashboards and going from there. Well, Nick, really appreciate you diving into the unique state of your team, how you're leveraging continuous merge and GitStream to redefine and improve on your Dora metrics and the tooling approach you're taking.

21:18I think it's been a really interesting conversation to dive into some of these pieces and how it's all coming together. I'd love to let our listeners know where they can learn more about you and about DevCycle. Where can they stay in touch? Yeah, devcycle.com. We are on all of the social media stuff. So you can find us on there. Just search for DevCycle. I don't have a lot of interesting stuff online, but you can find me at Nicholas LeBlanc on Twitter. Nothing too exciting and very clever about my handle. threads too or just twitter for threads as well yeah i signed up i think within 30 minutes or something slack wow okay you're gonna have a low number nice yeah and i'm a big chicago bears fan and i just kept i was doing like a bears watch just checking to see when they would set up their account and by the next morning they still hadn't so i wondered if there was like an nfl embargo but like so many other teams had already signed up but the bears took a long time it's kind of embarrassing embarrassing no no I will cut that cut that anyways you can learn more about what DevCycle is doing with continuous merge in their blog which we'll link in the show notes and thank you again for jumping on Nick hey thank you for having me this is a lot of fun after the break Ben and I discussed Nick's thoughts on tooling how to apply continuous merge principles to your team and what the future of GitStream may look like Want to reduce your team's code review time by up to 40 % without touching your budget?

22:45GitString is the new, free dev tool from LinearB that eliminates a key bottleneck in your team's workflow. Pull requests and code reviews. After reviewing the work of over 2 ,000 dev teams, LinearB's engineers and data scientists found that pick-up times for code reviews were lasting 4 to 5 days longer than they should be. The good news is that they found these delays could be eliminated in four key steps. First, by adding context to every pull request, such as estimated time to review. Second, automatically assigning the right reviewer and number of reviewers to pull requests. Third, having co-review automation that auto-request changes.

23:23And finally, by automating PR approvals and merges. To learn more about how GitStream works and to try it out free for your team, please visit gitstream.cm or search for Gitstream in the GitHub marketplace. Ben, I'd love to dive a bit deeper with you on some of the concepts that we heard Nick talk about in our conversation. It was really interesting to hear about how he's using Gitstream. He clearly saw he had a problem and in leveraging Dora metrics to measure it. He realized that Gitstream's programmable workflows and continuous merge capabilities could help us to improve and solve his problem.

23:57What did you think about how he's leveraging this tooling? Yeah, I really love how he described the connection between evaluating Dora metrics and actually taking action on it with tools like GitStream. Because that was really kind of what led up to us creating the tool in the first place. Once we show you the numbers, we want you to have what you need to actually improve those numbers as well. So the fact that their team is so able to just go into the dashboard, get all the metrics that they need about how things went this week or the last month, because it sounds like he doesn't really work with set sprints.

24:36And so they're kind of doing more of a rolling window look at their performance. And then translating what they see or what they surface from their Dora tools into things that they can actually change about how they write code and review code. So talking about building more expertise across his company or unblocking depend-a-bot reviews because they just sit too long when there's really no reason ever for a depend-a-bot PR or very rarely for them to just sit there. So the fact that they were able to go from understanding the metrics to actually taking action on them is something that just really validates everything that we're trying to do here at Linear B.

25:16and I also really appreciate how he's taken a more holistic approach to it. It comes down to how they structure their teams, how they dish out work to each individual, all the way up to how they view the entire process from the initial writing of code and submitting PRs to responding to incidents and making sure that they aren't introducing too many breaking changes. So, yeah, just overall, it's just really great to hear firsthand from somebody who's actually able to be more effective at their job and communicate with their leadership, you know, how their team is performing because of tools like what we're building.

25:57It was, I completely agree, really fascinating to hear from him. it definitely resonated with me and i hear this often with other engineering leaders that our approach is the right one of okay benchmark figure out where your team is get your metrics understand that you use a free metrics tool like like linear bees free or something else to to get those tools compared to industry standard benchmarks like the dora research or our benchmarks report building upon that and then say okay like how can we add in automation, how can we improve workflows and then drive that improvement? And so this idea of a holistic software delivery management approach that says, all right, we're going to understand the problem.