Unpacking DORA’s State of DevOps Report | Nathen Harvey

22:23When you have better documentation in place, documentation unlocks a team's ability to do better trunk-based development. It was something like, and I'm going to take this number from memory so it might be slightly off by a percentage or two, but with good documentation in place, trunk-based development contributed about 12.8x more to your overall performance. The other thing that's important is that trunk-based development gets sort of mediated through continuous delivery. What does all that mean? That sounds like I'm a data analyst now. What it means is that when you're doing trunk-based development and have continuous delivery in place together, they help drive better software delivery performance.

23:02Now, you might argue that it's hard to have trunk-based development without continuous delivery or even continuous delivery without trunk-based development. The truth is it's all kind of messy, and it also helps us see how all of these capabilities build on, reinforce, sometimes tear down each other. Do you think this would be associated perhaps to the quality gains that we've seen in previous research where when a team has high quality, their speed actually improves, even though some folks think that is a little odd? Yeah, that could absolutely be the case. And I think, yeah, you're totally right there that that like traditionally or maybe it's not even traditionally anymore.

23:40Maybe in the old school way, we thought in order to be safe, we have to be otherwise you'd be like, oh, well, if you're if you're going faster, it's harder to be cool. Yes, absolutely. But what we've seen year in and year out, and to a point where it's no longer surprising to me, but I understand it's still surprising to some people that in order to be safe, you have to be fast. And those two things, that speed and stability, they tend to move together with one another. I wonder if that is part of why the documentation piece is so important for trunk-based development, because you're improving the quality of the process, improving the quality of what you're delivering, and then that enables that speed and the productivity.

24:14Yeah, absolutely. That could very well be the case. And this is also one of the challenges and beautiful things that we have here, right? The data can only reveal so much. We can have hypotheses and guess what we see and use our lived experience and our experience with teams to tell stories about that data. But the data can only take us so far. Interesting. Yeah. So we saw some interesting data in our own benchmarking research that we did recently that I think correlates to this. So we saw that cross team collaboration just if we looked at PR data and saw more comments from other teams improved both quality and lowered rework rate.

24:51And so there was actually in our data, there was a positive correlation between enterprises had less of those cross team contextual pieces added to PRs and improving the codery process and therefore had a higher rework rate. Right, right. So I wonder if that's kind of that same approach of like, OK, door is identified. You know, when you have higher quality, which we can imply if there's more activity, more engagement from different teams, you get more quality, you get more perspectives. I wonder if that's part of why we saw that data. Yeah, interesting. And that that data also helps me think about things like loosely coupled architecture, which is another thing that we investigated, have investigated for a number of years.

25:24And I think that there's a there's a like this is another good example of you can ask a question and people will hear it differently. When I say loosely coupled architecture, what I'm really talking about is actually not the architecture of your system, but more so the communication pathways of your team. Is a team able to independently deploy the service or the portion of the service that they're responsible for? And I think the thing that you just described there where across organizations, we aren't seeing a whole lot of PR comments. They may be acting as if they're loosely coupled, but they see their quality going down.

25:57And the truth is they are tightly coupled across those teams, but they behave as if they are loosely coupled. So they aren't participating in that code review process, commenting on those PRs. That to me is like fascinating. I'd love to dig in. Oh, yeah. Happy to happy to talk more about at some point with you. I think the organizational system design is such an important piece. Like we all know it, but I think we all underestimate how much harder things get as you scale. I mean, just look at enterprises who now have platform teams in place because they're saying, hey, look, we need to centralize our ability to have these.

26:27like key infrastructure pieces so that they are coupled across teams. Absolutely. And this organizational structure and how do we keep teams aligned properly, this all comes back to leadership as well. Because, of course, the leaders in our organization, they shape the organizational culture. They have control over the incentives. And look, DevOps was born because we had bad incentives across two different teams. Like if you go back 10 years, 15 years ago, we told developers, move faster. We told operators, keep everything stable. And it's easy for one or the other of them. The operators, and I come from an operational background, I knew how to keep my system stable, except no changes.

27:07But that's no good for either one of us in the long run. It's certainly not good for our customers. And so DevOps was sort of born out of this idea of maybe we could collaborate and align on the same goal of keeping our customers happy, and that's going to help us improve. But we can't get there if our leadership isn't bought in. I think that leadership point is so crucial because it's it can be challenging sometimes to translate the operational metrics that are really important for internal engineering to, you know, the VP even sometimes not let alone getting the C-suite to care about why these efficiency metrics matter as much.

27:42You have to translate it to, oh, I'm helping deliver projects more predictably. My teams are able to, you know, have higher velocity and quality. That rework rate is lowered. We like to talk about like the psychological safety piece and the need for that. And we can like generally opine on that. But until we are able to show and translate that to the business metrics, it can be hard to get buy-in for some of these projects. Yeah, absolutely. Actually, it just came from a talk that was led by Denali Luma, who happens to be a Dora community guide. But she was giving a talk about a bunch of statistical analysis that she's done across code bases.

28:13And she's taken that statistical analysis across code bases and also looked at developer sentiment. and using developer sentiment as like a proxy for the value of some open source projects. And one of the things that she demonstrated through her analysis was that when you have higher variability and slower lead times or slower cycle times, that actually those two things are negatively impacting sort of the developer sentiment. And so when we get to better, like less variability, shorter cycle times, we're increasing the developer sentiment, which is a way of saying that we're increasing the value of this particular product.

28:49Talking about that value is a thing that leaders can understand. They can help internalize that. And frankly, our job as technical practitioners is to build empathy with those in the business, right? And be able to talk to them in their language. And just like we expect them to build up some capacity to do the same for us. It takes effort from both sides, no question. Absolutely. That's super fascinating to hear. And I mean, it aligns to a lot of the things that we're seeing in the data trends. I'm curious, though, let's get back to that benchmarking piece and thinking about, like, how do you define success?

29:22You know, because you mentioned, great, we can we can see these gains. We could start to get an idea of what we do. That's all wonderful. But if you don't know, is, you know, committing is deploying every three days good or is it bad? Like, how should teams go about that approach? Yeah, this is a really interesting space because I think, you know, one of the things also that we do when we look at software delivery performance, as an example, we talk about low performers, medium performers, high performers, and elite performers. Well, I have to admit that it pains me a little bit every year when we do that.

29:52Look, we have these four different clusters. We have to put a name on them. But the truth is, when you put in sort of an evaluative name, like low, medium, high, and elite, everyone immediately wants to be elite. I want you to be elite too. But the elite that I want you to achieve is elite improvement. Not elite as a raw number. I want to find the teams that are making those investments and improving and getting better over time. In fact, the way that I talk about Dora all the time, it's a framework that helps you get better at getting better. This is a journey of continuous improvement. Absolutely.

30:25And I think it's also important to say that like when we show those elite metrics, it may be that elite metric needs to be broken down to say, are you an enterprise? Are you a startup or a scale? But like, you should take a look at that. Like this is benchmarking that I know our team's looking at. I know your team's looking at as well. Absolutely. And like saying like, look, putting a moonshot out there and saying we want to get to elite performance. Great. That's great. But what you need to reward on your journey to elite performance are those continual improvement gains. Right. And so at the end of any year, I'd much rather give a most improved award than highest performance award.

30:56I really appreciate that perspective because, I mean, we've talked a bit about the need for creating good engineering cultures and having a longevity to it. And there's a huge risk when you just say we have to push for X because it can be something that is really toxic to the internal culture can burn people out. It can do the things that are the exact opposite of what builds high performing teams. Exactly. And when you think about improving, like improvement work for better or worse, it's never done. Right. There's always an opportunity to improve. And also as your teams get better at improving, they're also building up their resiliency because I don't know what's going to happen from a market perspective, from a financial perspective.

31:34from so many different things. What I do know is that the world is going to continue to change. And the better you are able and capable of adopting and adapting to change, the better off you're going to be. So how should teams start improving? That's a great question. How do teams start improving? I think the first step in starting any improvement journey is figuring out where you are. So like we have to figure out where are we on this map, right? And I think that using things like the Dora metrics is a great way for you to set a baseline. Here's where we are. And then we have to ask that question of what's keeping us from improving?

32:09And this is where you start to go from the door of metrics into the capabilities that we investigate. And of course, we investigate a ton of technical capabilities, but we also look at process and cultural capabilities. And as a team, through a conversation, try to identify which one of these is my bottleneck and then commit to changing something in that capability. We want to improve that. It's really the scientific approach. We have a hypothesis that code reviews is the thing that's slowing us down. Let's do some work to improve the speed of our code reviews. Now let's retest. Did our baseline numbers improve or did they deteriorate?

32:46Either way, we will have learned something and then we can make further investments. And you basically rinse and repeat that process. That's how you get into that mindset and practice of continuous improvement. And I'll say, if you're someone who's listening, we're going to make this as easy for you as possible. We'll link the report in the show notes. Absolutely. We also have a free Dora dashboard that teams of any size can use free forever. We'll link that as well. If you want to get started, start benchmarking yourself against the Dora parts or understanding, take a peek. And I think it's a great place to start because then you can say, OK, like maybe, you know, you've gotten your free dashboard set up.

33:15You've you've gone and you look at the report. You say, I feel like we're off on code reviews, to your point. Like, we need to figure this out. What would you advise a leader who's starting to see that friction point to do as they kind of implement it? Yes, we want to come back to the data, but how should they go about trying to improve these key areas? Yeah, so that really comes down to the context of your team and which area you want to improve. So if it's code review as an example, maybe you look at the process of code reviews. Do code reviews just sit and wait? Maybe there's something you can do on your team where you almost have an on-call code reviewer, right?

33:47So we have a rotation. This week, I'm on call for code reviews, so I'm not doing any real project work or product work. I'm here to pick up the slack this week and maybe code reviews come to me, So now I prioritize them this week. And next week I hand that off to someone else. Let's try that for three or four weeks and see how do our code review times adjust? And that's, you know, that's just one way that you might look at that. But again, it comes back to the context and sort of the constraints within your organization. Right, maybe your org is more in a pair programming and you need to take that approach.

34:18Or maybe you need to leverage program workflows to do, you know, automation that estimates the time of reviews and how long they'll take to cut down on that context, which is a lot of approaches. Absolutely. A lot of a lot of approaches there. I love that you say pair programming, you know, and pair programming. We know like how do you get faster code reviews? Well, you code review while you write the code. That's pair programming. Right. And so that's a beautiful way to do that. Not everyone is ready or convinced that pair programming is actually going to help them. So, again, that's a good place to experiment then.

34:48Go try it out. Yeah. And I'll just say like we have a whole, you know, within linearity where we look at these like experimentation pieces because I think it's I mean, we're really inspired by the the work that Dora team's doing to say, like, how can we leverage this for improvement? Whether that's our, you know, GitStream programable workflow tooling, evaluating payer programming, or the benchmarking research. It's fascinating to see how the work that Dora has done has really led the industry these last 10 years. How do you view the position of your team and of Google Cloud's role in it as moving the industry forward?

35:21Yeah. So I think from, I'll start from a higher level with Google Cloud. I think Google Cloud's commitment to keeping this research going and keep bringing it out into the world is super important. And I super appreciate also the fact that Google has maintained its commitment to this research being program and platform agnostic. This is not and will never be research into how do you get better on Google Cloud. That's not to say that that research shouldn't be done or isn't being done. And to be clear, you can use Google Cloud to improve these things. Absolutely, you can use Google Cloud to improve these things.

35:53But that's not the remit of this research. And I think it's really important that we maintain that. I think that then second, my team's role, really, I don't know if this came across or not, but I am not the data analyst. I'm not the PhD analyzing all the data. I work with them. But my role and the role of my team is to do things like this, bring the research out to the world and specifically take it to teams to help them go from basically academic research. Beautiful, great. I can get inspired by that. But what does it mean for me? What does it mean for my team? How do we put that research into practice?

36:25And that, I think, is the role of my team. I love it. And I think it also gives you these incredible insights into where the industry is going. So I'd love to ask you, based on the trends that you were observing in this year's report, what do you foresee as the future trajectories in the DevOps space? Yeah. So I think that, you know, first AI, AI, AI. I should have said a non-AI category here. Well, and I think from a Dora perspective, we're really excited about AI primarily because as more teams are adopting it, we're going to find and have better ways to evaluate the efficacy of AI and what is it actually doing from a trend perspective.

37:01I think there's also a lot of interesting things around things like FinOps, if you will, like how much are these things costing us? How do we actually quantify the value of improvement work? How do we quantify the value of friction in our process? How do we get to a place where teams can actually, you know, maybe leverage flexible infrastructure to get better cost out of their systems. And I think one of the fascinating things there is if you can't change your system, it doesn't matter that the financial people or like the finances say, if you ran it on this platform instead of that platform, well, if you can't change, you can't do anything with that.

37:40So change is where all of that starts. So I think there's AI. I think there's that finance. And I really think that we, as a research team, owe it to ourselves and the industry to look again at leadership and the role that leadership plays in driving all of this improvement capability. And I think it's crucial to ask that question as organizations position themselves to adapt to these upcoming shifts and develop practices, particularly around AI and others. Absolutely. Absolutely. It's interesting. We've seen some trends here that really speak to the need for this understanding at a leadership level, too.

38:15I mean, at the Door Community Summit that you and I were both at yesterday, I talked to so many people who were saying, hey, I care about the Door Metrics. I understand their importance to the efficiency of our organization. How can I get my leadership to understand? And having that conversation on how to translate business metrics is really important. I don't want to talk too much about this because this is the whole thing we do, and it's not that what this pod's about. But it's just a crucial conversation for any engineering org to have. And I mean, like one of the things we've seen that it was really interesting, this wasn't statistically significant, so I'm going to preface that.

38:48But like we saw an indicator that, you know, startups and scale ups deploy code 18 % faster than enterprise in our data set. Obviously not as big a data set as Dora, but like these kind of indicators speak to like, OK, is there a leadership problem or is this a tooling problem? What's happening here? There's so many things you can dive into, whether it's on the door side and leaders need to be thinking about this all the time. And that's why that's why I think this report so important is it should give a roadmap to improvement for leaders. Yeah, absolutely. And, you know, when we think about those improvements, we talk about those capabilities, right?

39:18Technical capabilities, process capabilities. I really think of each capability kind of as a dial. You can amp it up, you can lower it down. But here's the thing, if I'm an individual practitioner or, you know, on a small team, there are only so many dials that are within my reach, right? Like, let's say we have an organizational policy about how our change approval process works. I probably can't reach that dial. I can't impact it today. I might make longer term plans to build up how do I actually turn that dial. But today I can impact how frequently am I running tests? What does my continuous integration look like?

39:52So it is this understanding of, are these capabilities within my reach or are they things that I'm going to have to necessarily partner across and up and down the organization in order to impact improvement there? So interestingly, the majority of listeners who are on our show are actually engineering leadership. They're engineering managers, senior managers, directors on up. So I'd love to also hear from you. What do you think that leadership approach should be? Yeah. So I think that leadership approach has to start with listening. and really seeking to understand what are your teams doing? The truth is, as a leader, I can tell you this with a lot of certainty.

40:28Maybe not 100 % because you can't say anything with 100 % certainty, but I will tell this to leaders. Your team knows what hurts. Your team also knows how they can make that pain go away. It's your role, really, to go and listen to them and then create that space and capacity in their roadmap, in their tooling, in their way of working to allow them to eliminate that friction. And I also think from a leadership perspective, you have to remember and be careful with how you use metrics. Metrics can obviously be used for good. They can obviously be used for bad as well. And so I think that understanding that there is no one metric to rule them all, there is no one metric by which you can make decisions.

41:13You have to understand that we are talking about complex systems, complex systems that involve, you know, systems like computer systems and stuff. But within that system, it's us, it's the humans and how we interact with one another. And so I think from a leadership perspective, it really is about listening to your team, making sure that they have the autonomy and the space to make that improvement work. And your role is to support them in learning, support them in making those improvements. Absolutely. If you can give them the support and the space and safety to make those improvements, developer-led improvement is always going to be more sustainable than saying top down to change this number because that will then create all these negative knock-on effects that you mentioned.

41:53And I'm curious, though, because this is obviously so important. Operational efficiency of the engineering team is a crucial part of the dual main-up engineering leaders. The other part, though, is delivering for the business. How do you make the leadership and the rest of the business care about all this willy-nilly stuff or stuff you're doing? Yeah, absolutely. I think, and this advice kind of goes both for the practitioners and for the leaders. We have to learn to speak the language of our partners across the business. So I have to understand, you know, a great example that a lot of us maybe have experience.

42:24We moved from data centers into the cloud. Great. That's awesome. Maybe we didn't think about the financial implications of that. Our CFOs said you moved from CapEx to OpEx. And which X? No, I'm in the cloud. I'm not on X. I'm in the cloud. Like we We don't necessarily have that communication pathway. So I think it is important for both sides of that equation, for the CFOs and for us as on the practitioner side, to be able to speak in their language and build up some empathy there. Absolutely agree. That translation layer is crucial. And metrics like Dora can help give you a first step and you need to take the effort to then apply them and translate them to your leadership.

43:01There's education to be done too. Absolutely. So I'd love to just give you an opportunity to share any closing thoughts, maybe a call to action on where to go to learn more about the Dora report. What do you want our listeners to know? Yeah, absolutely. So it's come up a couple of times here in our podcast today. We were both at the inaugural Dora Community Summit yesterday. Fantastic time. You're doing one next year, right? Yeah. Well, we don't have any solid plans yet, but based on how things went yesterday, I think, yeah, we definitely want to do another one next year, and I hope that we'll see everyone come back and more.

43:33So I think really the truth is that Dora doesn't have the answers. Linear B doesn't have the answers. I don't have the answers. But together, as we work as a community, learning from each other, this is where we're going to get closer to those answers. And it is about learning from the experiences of others, looking at pitfalls that others fell into, helping others avoid them or lift them out of them when they do fall into them. So I think that community and we're here at the DevOps Enterprise Summit, like that's what this is all about, learning from each other. So from a Dora perspective, my call to action is one, go to Dora.dev.

44:06at Dora.dev, you're going to find the latest research. You're going to find capabilities articles, and you're going to find a link over to the Dora community. If you're listening to this, you owe it to yourself to join us on the Dora community so that you can share your experiences and learn from others. Well, you heard it from Nathan. It's been an absolute pleasure here, and we're definitely encouraging our community to join yours. I think it is such a fantastic place to learn about the future and to help shape the future of software development. And thank you so much for coming on, Nathan. Yeah, thank you so much.

44:35It's been a blast. if you want to download the door report we'll absolutely be linking it in the show notes i mentioned we'll also have a free door bash dashboard and the benchmarks because as nathan put it more context that's going to give you more information to understand how to improve nathan thanks again for coming on the show it's been amazing to partner with you on this year's report and we're super excited thanks so much connor