What Engineering Leaders Can Expect In 2024 | Predictions from Ori Keren

0:00Is your engineering team focused on efficiency, but struggling with inaccessible or costly Dora metrics? Insights into the health of your engineering team don't have to be complicated or expensive. That's why Linear B is introducing free Dora metrics for all. Say goodbye to spreadsheets and manual tracking or paying for your Dora metrics. Linear B is giving away a free, comprehensive Dora dashboard packed with essential insights, including all four-key Dora metrics tailored to your team's data, industry-standard benchmarks for gauging performance and setting data-driven goals, plus additional leading metrics, including merge frequency and pull request size.

0:36Empower your team with the metrics they deserve. Sign up for your free Dora dashboard today at linearbee.io slash Dora or follow the link in the show notes. Hey, everyone. Welcome back to Dev Interrupted. I'm your host, Conor Bronsden, and I'm delighted to be joined in the Dev Interrupted Dome by Linearbee co-founder and CEO, Ori Karin. Ori, welcome back to the podcast. It's great to be here and it's great to be in the Dome for the first time. Yeah, it's a fun experience. If you're not watching this on YouTube and you're maybe listening on Spotify, consider checking it out. It's a lot of fun to watch the whole thing happen.

1:08And that's why Ori and I are here to talk about these fascinating topics that we're hearing all about at DevOps Enterprise Summit 2023. And as we head towards the end of the year, teams worldwide are thinking about what are the trends that we need to pay attention to in 2024? What are OKRs for next year looking like? What are our key initiatives? So Ori, given your years, your decades really now of development experience, your years as a founder, your experience about startups, scale ups and enterprises, I wanted to get your take. What are the key predictions that you have about the state of software development and engineering teams in 2024?

1:39Yeah, so I'll start with the topic that is closest to what we do and what I do. around like a developer productivity, I think we're going to see a little bit of a conflicting trend. So one, engineering efficiency or developer productivity, measuring that is here to stay. It's almost like if you look at 2008 and the challenges that sales had there, et cetera, strong sales efficiency technologies evolved out of that. And I believe that from these tough times or tough macroeconomics times, you're going to see engineering efficiency here to stay and sort of like the inheritance of this time. So that's the first thing.

2:21Like companies who didn't start in 2023 are going to start in 2024 and in 2025. It's a no-brainer. It's here to stay for that. Another thing that, another two interesting aspects that related to that is I would start with the first is if in the past you had like companies that say, hey, I'm going to do it myself. So DIY, I'm going to build like a custom solution for that. We're already starting to see less and less. And I think 2024, my prediction is going to, they're going to almost go away. Same things happen when people say, okay, CRM, do we really need a CRM? Do we really need all the opportunities?

3:03We can have it in a spreadsheet or build something of our own. And the technology evolved so much that you're behind if you're not starting. So you're going to see less companies that are choosing to do it themselves and kind of like go with some of the solutions. I would say the last trend that I think we're going to continue to see in engineering efficiency, developer productivity is, there's a saying that in the staff economic times, like the CFO is almost like the new CIO. They're like kind of like making decisions on what information technology do you buy. You got to put like a lot of ROI justification into that.

3:44That thing is not going to go away. You're going to still, in spirit at least, like be in those conversations. And I think that engineering leaders that won't control that data, understand what's happening, run a metrics program, kind of map what's happening in the organization, they're going to have tough times like to control the narrative of why they're operating, you know, in a good way and why there's this justification to maybe increase the team. And the times where I was an engineering leader, I just came to my CEO, hey, I need two or three more people. And I convinced him, you know, by just talking about hunches and not with data are over.

4:23So I think my other prediction is that engineering leaders who will do this transformation, will control the data, will control the narrative and will be able to talk to the CFO, will achieve great things for their organizations and vice versa. What would be your advice to engineering leaders who are trying to have that conversation and trying to be the ones who control the narrative, to your point, so that they're not beholden fully to the board and instead say, hey, look, my data is here. My input is here. I would say, again, it starts with having the data. If you don't have the data, it's all hunches, it's all guesses.

4:57Then there's different types of data, right? So people, you know, we double down on Dora metrics are probably going to be the theme of like this event. But there's other areas where you kind of measure the allocation, how much investment there are going into like each type of investment and each project. This is the type of information that is super interesting for CFO, board, et cetera. Have the data in front of you, be able to start a conversation, initiate a conversation. If you want, it come to you and come to you in the moment that you're at least ready. And so if you initiate a conversation, you can control the narrative or just lead it to where the discussion to where you want it to be.

5:40Yeah, you and I have talked about this before, right? Which is like when you fail to do that, you run into things like the McKinsey framework where it says, OK, here's how you need to measure developer productivity. And, you know, we've all seen many of the critiques of that. If you haven't already listened to Ori and myself and Kelly Vaughn on this podcast critiquing it, definitely go check that out. but to sum it up when you don't consider the impact of your engineering metrics program of your analytics of these decisions you're making at the leadership layer on the culture of your engineering team on the retention of your engineering team on your ability to actually deploy for an engineer when you risk gamification you create these massive I believe you put it as like you create an autoimmune disease within the company so this is a huge risk yeah so that's the other side like the allocation is more like okay I'm a VPE I'm a CTO.

6:28I need to have a data-driven conversation with my financial folks. The other side of it is to try to stay away from those individual metrics, at least in those conversations with a CFO. And my belief is that you need a different system to assess your talent. You use the same system that you kind of like measure your door metrics and your metrics. You kind of create like you said, like autoimmune system, because then people are gaming the metrics and they know they're being measured on that. So they're going to change their behaviors. I would totally recommend like using something different to assess the talent of your team, which probably you should also do and use the metrics like more for the team, team related things.

7:15And this is why I hear you using the phrasing engineering efficiency. A lot of this debate that's evolved has been around developer productivity. And I know you have concerns about that because it lends itself to this focus on individual metrics, which can be very toxic for a team. Yeah, yeah. I think like language creates reality. We know that. So if you call something developer productivity, you use like the single tense and not the plural tense. Like even in sales, you measure the sales efficiency. Even in sales where it's such a more close to like individual sports. Competitive, yeah. Yeah.

7:45You don't talk about sales rep efficiency. You say sales efficiency and you measure like the friction in your process. At the end, you will measure like the individual reps. So, yeah, I don't like the term developer productivity. If we can use dev team efficiency, engineering efficiency, I think that's like the right term to use. And I know that, you know, as we all look at these solutions for developer productivity or engineering efficiency, whichever you want to phrase it as, there is also an issue within companies of tool fatigue, you know, not just the CFO level where they're saying, oh, I'm worried about how much we're spending, but also at the dev level, at the team leader level, and they're saying, okay, what's this new tool?

8:23Why should I start using it? How can the community, the dev tools community, avoid having our like cutting edge tools that should hopefully be improving the lives of engineering leaders actually be a headache? Yeah, so that's a great question. I think there's this term called developer experience. We all heard it. It's gaining momentum. it's still not fully operationalized at this point in time. And I think that what we see from our customer base depends on the size. So if you're like an enterprise customer, you probably have a developer experience team. If you are a smaller one, you probably have like somebody who's like very passionate about it or what we call the AOR, area of responsibility.

9:05So the rise of those developer experience team and IDPs, like internal developer portals, I think they're good. And I predict 2024 for them to go even, you know, to be more rolled out within teams. Those things can help well with what you talked about because it's a central place for people to kind of understand what's stopping me from delivering on my task. If I'm talking about myself as an individual developer, I come in, I want to complete a task scan to end. If I need to learn new tools all day long, here's my productivity. If the test that I need to run, run too long, here's my productivity.

9:46So that's productivity from the eyes of the developer. And developer experience is super, super important. I think 2024 is going to be the year where it's going to be much more operationalized with two things in mind. And our philosophy is always use those two methods. On one hand, there's empiric things you can measuring developer experience. For example, merge frequency. If I'm not, if I'm a developer and I'm not pushing PRs out, forget about how my manager looks at it. From my side, I don't like it because every developer wants to achieve things. I want to come in, complete a task and go to the next task.

10:23There is empiric metrics like flaky tests that are sometimes working, sometimes not working. You want to kick them out because they're not stable. They're like confusing you. There's like the length of the CI. Those are like empiric metrics and we're going to see a lot of companies invest a lot in them. There's also the qualitative side. I see a lot of good movement into that, asking the developers via surveys, where do you think, what is the thing that's blocking you? And if they say tool fatigue, I want a new tool, you should listen to them. So this is a really interesting thing to dig into, given that, you know, if we haven't already said in this episode, AI is here.

11:05It's here to stay. You're not going to get away from engaging with AI tools. We're already seeing some of these trends in our data. We're seeing some of the trends in the Dora research, which we were a partner with for Google. And in that Dora research, we saw that companies that were leveraging AI tooling were starting to see them actually improve developer experience metrics, these satisfaction surveys, because we're seeing improved automation, leveraging programmable workflows, and AI to actually ensure that some of these less fun tasks, some of these frustrating friction points get automated away.

11:40What do you see happening as that trend continues to evolve with AI in the coming years? Yeah, so there's a couple of interesting aspects to that. I think even in the Google report, they're saying, yeah, we're starting to see the trend, but it's still really early. Yeah, early. It's being incorporated. It's going to be much more incorporated in 2024. So that's like a very important prediction. Now, I have like a lot of like ideas around it and a lot of thoughts around it. The first one is, okay, Gen.ai is going to do more impact on how fast code is being generated in 2024 for sure. So you're going to get some fully automated like Gen.AI code going in.

12:20You're going to get some half, you know, like semi-automated like Gen.AI code. And here's the thing. It's like, I keep saying it and say it again. You're going to try to stream more water into the same old narrow rusty pipes. So I foresee in 2024, all of a sudden in the acknowledgement, okay, whoa, like we're generating code. But we thought about like these pipelines and how can we open them up so they can release, you know, reviews faster, CI faster, CD faster. And the keys is not faster in terms of let's throw more machines and more money on the problem. It's being smart. Classify the work that's coming in and then send them on different, you know, routes.

13:06So some of the things can go faster. Some of the things need more human intervention. That's one thing that I think we're going to see in 2024 with Gen.AI. Our customers are already coming to us and saying, we're experimenting with things around Gen.AI. Can you help us figure out what type of impact it does? Of course, you can use the Inevit to measure the impact. Is it cycle time faster in Gen.AI? That's my thoughts around Gen.AI in high level. There's two other things that are interesting. One is the quality of being able to read code is going to become much more important because code is going to be generated faster, like we said, but you need humans to read the code and approve it.

13:52And here's the thing, developers don't like to read code and they like to write code. So having these senior developers that know how to read and analyze the code and how it's going to impact their specific system, And it's a very important quality for the companies to continue to invest in and make sure that they have it because that's going to be a pipeline like opener if they have it. And if not, again, more code is going to sit in the pipe, going to be stuck. My other very philosophical kind of thought that I had yesterday is that, you know how when you had an iPhone, you stopped remembering numbers.

14:31Sadly true. And then you got, I don't know, Google Maps, Waze, whatever you use. We lost the navigation capabilities or at least some of them. And my worry is that since, again, this is like not next year, but two, three years ahead, when more and more dependency is going to be in, hey, I need Gen.AI to help me like generate code. The innovation level will decrease because a lot of good ideas are coming. I'm working with my hands. I'm doing something. And now all of a sudden, oh, this is a great idea. Let's do this. Let's work. And your dependency in something that will help you generate a call to kind of block your innovation.

15:10So I mean, companies like in two, three years from now, we're going to start to think about how do I preserve the knowledge and not lose it and kind of like decrease their dependency in that. Well, that's a little bit more far ahead. That's an interesting prediction because I know you and I have talked about this before about how you believe the best developers are ones who are really incredible and willing to read code. And so I wonder if, to your point, that skill set will become even more important because you need to go beyond just what Gen.ai has done for you. Like, yeah, great, you can now get some of the basics done with Gen.ai, and I'm sure that's going to continue to improve.

15:46But to actually create something innovative and new is going to take potentially more work because of this expectation that so much of it is, in some ways, cookie cutter. Yeah, yeah, definitely. I think, again, And the ability to read code is something that those are going to be the most attractive developers. Like people will look for them. Companies will need to invest resources in training. People like to, okay, how do I increase those capabilities? Are there any new challenges that you see developers facing around another topic you brought up earlier, which is resource allocation? And how do you see engineering leaders needing to mitigate that resource challenge that they may be facing in these choppy economic times to continue to improve productivity?

16:34Yes, we divided it to two interesting things that we're seeing. First of all, there's this topic of remote work. It's not solved. That's like our belief internally, because what we found in our research is that teams that are working, not necessarily co-located and working from, but in the same time zones are much more collaborative and are working better together. So, you know, there's this brave new world where everybody's working, wherever they want from, different time zones, et cetera. I still believe hybrid is somewhere the right way to go and each company needs to find there. But we're seeing that companies that are working more or less in the same time zone are able to break the biggest blockers in engineering efficiency.

17:25And we know that code reviews, that's something we keep on fighting as like one of the main blockers and this year google also like mentioned it as one of the main thing that you need to improve in order it's being done better when you're in the same time zone so i think companies need to kind of say hey this is our strategy it's fine if we still decide to work you know different time zones but let's just acknowledge the prices come up with programs that i kind of uh fix that because again if i'm issuing a PR and I'm waiting three hours for somebody to wake up. It's a productivity killer. So I think companies need to figure out the strategy around remote work, hybrid, time zones, all of that in order to be more successful.

18:14So that's one thing that I kind of see. And the second thing is very simple. We talked about it in the beginning, I'll say it again. You want to be better and face the challenges, you got to start with a metrics program. We talked about it but and metrics program is just the first step and after you do that you gotta make sure you set okrs and have like operational allocate your resources allocate your resources figure out how you change your resource allocation automate automate as much as you can so those are like the things that i think like teams that like will will start doing and will improve uh will give them like a competitive edge yeah so you you mentioned the automation the programmable workflows those piece, that seems like it's a really crucial element of predictable delivery.

18:59And that's really what we're all going for is high quality, predictable delivery that can drive that innovation we talked about earlier. And you mentioned the Dora research this year and now the 2023 Dora report that just came out, which we're very proud to partner with Google on. One of the really awesome insights in there was that teams with faster code reviews perform 50 % better on software delivery. and it was like such a substantial improvement, but it really aligns what we've seen in the research as to where the friction points are in development, which is code reviews are one of those main ones we hear about both in quantitative metrics through our own data and also in the qualitative metrics, whether it's conversations, surveys, et cetera.

19:38Yeah, absolutely. There's this triangle between if you figure out that having small pull requests, it will, you gotta have like small pull requests, so you gotta break your work into small chunks. and then you understand that what it does to you, it increases your merge frequency, which is a very important developer experience metric that will tell you. Because if developers are merging two, three PRs a week, they're happy. We want to complete, I say we, I still see myself as a developer. We want to complete tasks. That's what we want to do. And then on the other side, what it does to you, it keeps them pick up and review and all the handoffs faster.

20:20because here's what's happening and it's a harsh truth. Everybody knows it. If I'm getting a PR, the first thing I'm going to do that I need to review, first thing that I'm going to do, if I have like estimated time to review, great. But if I don't have it, I'm going to assess really quickly. Is this like a 30 minutes thing or can I do it in two minutes? And here's what I'm going to do. It's a very binary like classification. Two, three minutes. Okay, I'm going to take it. 30 minutes, throw it in the queue. I don't know. It's like it's going to... Maybe it's tomorrow morning. tomorrow or in two weeks and that's it that's so when this triangle like works small prs high merge frequency fast pickup and review less like handoffs or even if there are handoffs they're fast and then you get things like out to production and it's still fresh in your head when there's a problem so you can roll out quickly you can roll back quickly that's the magic when when that happens A lot of like the problems are solved.

21:18And that's why like Google like identified it. And we're seeing it with like the leading indicators that lead to like the code we've been talking about. It's like for the last two or three years. And by the way, that's the first blocker in a development pipeline. Once you finish that, there's more now. There's like, why is my CI sometimes failing, sometimes not? Solve that. Why is my CI taking so long? Solve that. Can I release it directly to production? But that's the main like blocker that is heading for activity at this point. And it really aligns to Dora's research, which shows that when you have good quality and speed, those things actually go together.

21:57And what we're seeing here is when you break down that triangle you talked about, these small PRs, quick review cycles. And when we pull these all those things together, we're able to create much faster software delivery pipeline. and that also improves quality because instead of having to look at a massive review or maybe you're missing something or you're lacking context, I agree, I can get this small review quicker. It's a lot easier to figure out the less context switching involved and you're not only improving the speed, but the quality of what you do. And so that kind of brings me to my next question, which is what should engineering leaders top priorities be when it comes to Dora and the insights from Dora as we head into the next year?

22:34Because I think you've alluded to code reviews. What are the other things they should pay attention to? You mean like when they think about the metrics program or when they think about DORA? I'd be curious about both. Yeah, I would say we're seeing this a lot. So I think organizations, they know themselves the best. So they need to start a metrics program. And we're going to have a talk in this session with Syngenta, one of our customers. What I liked about how they operate is they said to themselves, okay, there's DORA, there's other metrics. We're going to choose the things that we believe in.

23:09And that's what we're going to measure. And by the way, once they did that, they went quickly to the next level in the maturity saying, okay, measuring is not enough. How do we create operational cadence that everybody is accountable and everybody's presenting their metrics and they're like sharing them, which is like a very, very important phase. I think once you have those, you get to the next level. Okay, I see problems. How do I fix them? And that's why we love GitStream because it's a great venue to start coding yourself out of the problems and improving them. So my recommendation is start, the start of metrics program, pick the things that are important to you and focus on what you're trying to achieve and how it's aligned best to your culture.

23:58Then I think it's what's going to happen to you is like the third quarter or the fourth quarter syndrome where we say, okay, now I need more. I need allocation use cases. I need like automation. So there's no magic. You just got to start. Like I said in the beginning, it ties really nice to the beginning. Engineering efficiency is here to stay. The gap between those who started and are now in their second or third year and are in advanced use cases and, you know, metrics are stable stakes for them. Between the ones that didn't start yet is growing bigger and the competitive edges, you can really see those customers who run it.

24:37And just by seeing those metrics, they already save like a lot of time. It's hard to compete with them if you're in the same industry with them. Absolutely. And this is why as a company, Linear B has now made Dora metrics free for everyone worldwide with our free dashboard that we've launched. You're probably going to hear an ad for it on this podcast, but it's completely free to download. You can use it for a team of any size. We want to make sure that every team has those table stake metrics you talked about. So you can start to identify the areas of friction for your team to improve because it's so dependent on how your team's set up.

25:09Is it something where it's cross multiple time zones? Maybe you're going to have different challenges than a team that's co-located. They may be harder challenges. They may not be harder challenges. Are you a hybrid team that meets a couple times a year versus one that's in the office a couple days a week? Are you doing more pair programming, less pair programming? So many things, and I'm just rattling off a few, can really affect how your team approaches this. And so it begs a question for me, like we've given a lot of advice for engineering leaders. What about the rest of the C-suite? What about founders in particular?

25:39As a founder yourself and someone who's been around a lot of incredible founders over the years, how should founders think about engineering efficiency in 2024 and beyond? And what else should they be prioritizing as they make these really critical decisions that partner with engineering leaders? I think it boils down to some of the same things. First of all, when you think about your talent, think about what we spoke about before. I don't want founders to say, hey, you know, generating code is easy now with Gen.AI. So I don't need, no, you need like strong developers. You need like the ones that will help you like, you know, authorize the code, read it, approve it.

26:13So that's one thing that I think they should be thinking about it. Another thing when you start your journey, you know, put those metrics and allocation things from the beginning. Because I think if you start a relationship with your board, with your CEO, with your peers, you put this culture in where engineering is accountable like any other department and they can talk about, here's our investments. And it's dramatic because you start like the relationship with your board on the right foot. So I think if founders like thinking about like starting like a company now, yeah, think about Gen.AI, think about strong talent that can still read and generate the important code.