Hunting for a Flywheel | Refactoring’s Luca Rossi

0:00Hey, Andrew, I heard you have been doing some really cool experiments with generative AI. Why don't you tell our audience about what you're up to? I have been in the lab this week, full confession, working on the developer experience for LinearBee's GitStream, where we're using the power of context with generative AI in the code review process. I imagined the process a bit like a restaurant. Bear with me. The model is the chef, and it's skilled and trained and capable of preparing an incredible variety of things, depending on what you give it. And the data, that's like the ingredients. Without good ingredients, even the best chef will struggle.

0:39And when the ingredients are local, as opposed to store-bought, they're often better. But the context is the order from the customer. And the customer is everything. They tell the chef what to prepare, how to season it, if they have any allergies, whether to mix it in or put it on the side. and no matter how great the chef is or the ingredients are, if the order is unclear or missing, the dish doesn't meet your expectations and you're not going to be happy. I think I'm starting to see where you're going with this. So at Linear B, we're focusing on that power of the context to give the customer exactly what they ordered because no one cares if your chef's from an elite academy or if your ingredients are fully organic if you order a steak and you get a pizza.

1:25To that end, LinearBee released the AI Starter Kit for PR reviews. It's a collection of workflow automations that make it easy for engineering teams to start experimenting now with the power of context in their pool reviews. It brings the power of context to your AI models and services along with native GitStream capabilities. Yeah, so this is really cool because we're seeing so many products out there that are emerging to tackle this space. there's definitely a lot of confusion about like what works versus what doesn't and i really like the focus we've been taking with this so far with this experiment is is on responsible adoption so if you're just getting started with ai like you've maybe just bought one or two tools and you have an api that your developers can query or if you're somebody who's actually already purchased a bunch of ai tools and is trying to roll them out responsibly and securely and effectively we're building these tools that are here to help you mature that process.

2:20So where can our audience get their hands on this? Well, anywhere that Google searches or search engine results are served is probably going to point you in the right way. The GitStream documentation is going to have the best starting places for getting started with the AI starter kit, as well as the other features available on GitStream. But if you want someone to walk you through a more in-depth guide or walkthrough based upon your own needs, definitely reach out on the Linear B website. it's really easy to book a demo with an expert and get your hands on some great ideas for how to take this back to your team.

2:52And we'll put the links for this in the show notes as well. Yeah, awesome. So Ben, what have you been reading this week? Last week, we were talking about all these investments that are going into AI. Of course, this was before DeepSeek completely upended the market. But I am proud to say that many of our predictions from then were relatively accurate. You know, some might say we have some foresight here. Another story that has caught my attention for reasons that we'll get into in a moment is IBM. A company that has been around forever at this point, very early to the AI space with Watson, but a company that I think we've all kind of forgotten about for a while, because there just hasn't been a lot of groundbreaking news other than maybe like them buying Red Hat and HashiCorp.

3:38Their Q4 earnings report just came out. Usually these are not interesting at all, but they showed some extremely bullish growth in one area in particular, and that was their AI services. So they announced$5 billion in total sales, and$2 billion of that was in Q4 alone. So a massive, massive quarter over quarter growth. Naturally, there's a huge spike in their stock price, sort of bucking the trends actually of this week of many of the companies in AI. But actually, there's one finer detail that was like, hidden under all of this, and that 80 % of this is actually consulting. So, you know, only about 1 billion was from actual software.

4:20So, you know, getting back to our opening there, where we're talking about experimenting with AI, like, you hire consultants when you're not sure what you're doing, and you're experimenting, you know, so I think that's actually a very good indicator of where the industry as a whole stands today. The question I keep asking myself is, when is that going to change? Like when is software, like purchasing software tools, when is that going to become the main driver of AI adoption? So, but we've got an upcoming guest from IBM, don't we, Andrew? Yes, I'm very excited for our upcoming guest on the show.

4:54We have Dr. Ashwari of WatsonXAI at IBM. is coming on talking about how engineering teams are making those first moves now to adopt AI practices and the things they can do to be more effective in the future with those tools they pick up. And getting a playbook like that from an expert at IBM is invaluable. Like you just said, so much of their market value comes from providing insights and details on how people can implement this. So imagine we got a bite of that for free here on Dev Interrupted for our listeners. I'm really excited to share it in an upcoming episode. Yeah, awesome. You know, and it's really, I think, going to be an awesome story of how legacy companies are reinventing themselves.

5:32You know, everyone is facing this new reality of AI disrupting them. Like, everyone has to be focused on how we're going to reinvent ourselves for this new era. So let's talk about some of that disruption. I think you've got a new story that you wanted to share that gets into some of the events of the last week. I read a great article that kind of summed up some of the disruptions this week. At the beginning of the week, DeepSeek, DeThroning, ChatGPT in the App Store, NVIDIA losing$593 billion in market value in a single day, which is a record one-day loss for any company. And everyone Googled Jevon's Paradox.

6:10And Ben, were you among those Google searches? No, I let my GPTs do my searching for me these days. I don't need Google anymore. Or not a joke. I actually did Google it. So I am definitely one of those millions of people who wanted to learn what it is. And for anyone else out there who hasn't been had been exposed to this yet this week, the Jevons paradox is a paradox where it was observed in the late 1800s where the invention of a more efficient steam engine, you know, some people thought that meant that coal demand would plummet, but the reverse was actually true. So by making a more efficient steam engine, it enabled more people to afford the amount of coal that they needed to run it, which increased demand for coal.

6:56So, you know, and when you really think about it, chips are just rocks, you know, like better efficiency is going to drive more demand for rocks. So I think companies like NVIDIA in particular will probably end up being fine through all of this. But one thing in the article that we're going to link to these in our Substack newsletter and in the show notes is it reinforced a theme we brought up last week, actually, on how there's a bit of an arms race emerging as it relates to this artificial general intelligence that everyone is pursuing. And the companies that I would be concerned if I worked for them are definitely the ones that are focused entirely on AI.

7:34The question I'm asking is like, will these events be a catalyst for VCs to pull back from AI to focus on more efficiency? Or are they going to double down on quality to try to keep leapfrogging the competition? Another story that came out that I was just thinking about that is maybe related to this is, you know, OpenAI just announced a new partnership with the U.S. National Laboratories that is specifically focused on cybersecurity for the U.S. federal government. So like we're already seeing this idea of there being like real national security interests in building this stuff and securing it.

8:08You know, and I think this also kind of highlights an issue that a lot of these Chinese companies are going to have is that, you know, they have to adhere to their local laws. And, you know, companies that really value their security and their privacy and they don't want to hand over potentially competitive intel to foreign agents. It's a pretty risky situation to try to go with something like DeepSeek or any of these other, you know, I think there have been more companies now in the last week that have emerged from China with similar services. And while that threat is definitely there, you know, on the other hand, it also in the environment makes it even more difficult for engineering leaders to justify costly AI spend to maybe more traditional or Western model providers, especially justifying that to non-technical executives who may not fully understand the risks.

8:55They see the cheaper number and they see the savings that they could be gaining while still innovating faster than their competitors. And that's very appealing. So when a cheaper service is sitting right there ready for use, it really kind of calls everything into question about how the strategy should play. And while some industries and organizations could never tolerate those risks for their IP, like you said, handing over their information to, you know, like a foreign entity, nearly every industry is experimenting with AI in some capacity and everyone's risk level is going to be very different.

9:27And even Perplexity, you just mentioned, doing your Google searches for you. They even have a DeepSeq R1 mode now. You can use it there. And DeepSeq is also open source. So you could run that efficient model locally with those benchmark performance. But of course, that does nothing about addressing the censorship concerns related to the LLM, which also hit the news this week about people could get it to say and not say. and all it really comes down to say is that for some businesses it may make sense to pay or use a cheaper service if that means that you can scale faster and turn perhaps your ai wrapper into a fully fledged business with wings that can go vertical yeah and and our producer adam he shared this really awesome article with us and and he does not get enough credit on the show which changes today changes today so thanks adam but it was actually an interview from the ceo of deep seek a few months ago, so before all this hype took off, where one of the things that really stood out to me is they were talking about how China in modern history has been more of like a one to 10 culture when it comes to technology.

10:34They take existing ideas and they 10X that idea. Whereas the Western society has been more of a leader in the zero to one, so going from originating new ideas. And their CEO did really seem invested in this notion of China becoming more innovative rather than copying Western counterparts. And with it being open source, it'll be interesting to see what American companies learn from this because I think Hugging Face is already running their models that has been trained on Hugging Face data at this point. So yeah, I definitely think it's going to lead to more competition. That's true. But when it comes to open sourcing AI, it's only one part of the puzzle.

11:16Another thing that was in the news is going back to legacy companies, traditional enterprises, and how they're adapting. Goldman Sachs, they hired Daniel Marku from Amazon as its global head of artificial intelligence, engineering, and science. And why does this matter? What does this mean? You know, a huge traditional financial, highly regulated company just poached a leading AI leader about being able to implement these practices within their own company. Because Goldman Sachs AI assistant has already rolled out to 10 ,000 of its knowledge workers within its organizations. That's a huge rollout at a big scale.

11:51You know, I mentioned earlier, all industries right now are experimenting with AI. That means you get some really big players in the space outside of maybe traditional tech world, right? And the idea that they're going to roll this out to more of their knowledge workers and have it in all of their hands by the end of the year, they need somebody with that kind of expertise. And I think there's a lot to learn from a traditional enterprise making a move like that. The scale of AI adoption that we're seeing in this story specifically is, I think this is where things fundamentally start to change in how businesses operate.

12:26We have even these traditional companies who are making strategic hires to align themselves with all of this rapid AI development. So new models every week, you really need to have somebody there who can be an expert at how to implement that for your organization. And, you know, I think everyone, every company should have people who are leading their internal AI initiatives like this. I agree. It's about identifying those opportunities now that will have a big impact for tomorrow. And speaking of big impact, you know, last week we had Luca Rossi of Refactoring and Yishai Beery of Linear B, and they sat down to discuss how engineering teams can go beyond academic frameworks to have impact-based practices for their engineering organizations.

13:11And that survey was made possible by our listeners, the Dev Interrupted community, as well as the refactoring community for responding to our survey. So thank you again for lending us your voice, your perspective. It led to some really great insights that I don't want you to miss. So after the break, we're going to bring Luca on the show to tap into some of those insights. And what we're going to do is go hunting for flywheels. We're going to find those things that you can take back to your engineering team to start building momentum for tomorrow. Yeah, I was really excited to record this interview.

13:44So make sure you stick around. Last week, we launched an audience survey to gather feedback on Devon Interrupted to make the show as great as we can for you, our listeners. So far, we've received overwhelming feedback that we should pivot hard to AI content. At least three people have told us this. So we're almost completely convinced that this has to become an AI podcast at this point. And I've also heard that it basically guarantees you endless VC investments. So we'll see. If you don't want us to become an AI podcast, or maybe if you do, now is your last chance to give us that feedback because the survey will be closing very soon.

14:23Andrew, what else do you have to tell them about this survey? Well, you know, Ben is mostly joking, but we are very serious that we want your feedback. We want to hear from you. And I want to take this moment to highlight you, the listener, for joining us on this journey. Your opinions on topics like developer productivity and experience, they really matter to us. And so joined by you, we're rethinking every week what it means to build engineering teams, tackle software challenges, and lead in a world that never stops shipping. And so if you're ready to roll up your sleeves and join our conversation, the stakes are high, but the problems are fascinating.

14:57And we'd love to include you. So in that survey, there's a way for you to raise your hand. And it only takes a minute to complete. So please check it out in the show notes. And thanks in advance for lending us your time.

15:11Welcome back to Dev Interrupted. I'm your host, Andrew Ziegler. And today we have a special episode in store for you. I'm joined by Luca Rossi, the mind behind refactoring and a returning guest on the show, as well as my co-host, Ben Lloyd Pearson. We're diving into the traits and practices that make engineering teams successful based on a survey made possible by you, the Dev Interrupted and Refactoring community members. Luca, welcome back. How have you been? Thank you. Thank you so much for having me. I'm great. I'm so happy to discuss this great work that we did and excited. We're excited too.

15:45So your latest report, it uncovers some surprising insights, at least they were surprising to me when I went through, about engineering teams, from how shipping code impacts happiness to the hidden flywheels in team dynamics that are hiding right in front of us. So let's dig in. Yeah. And I wanted to start with the thing that like really, since I first read this report, I've been thinking about like nonstop. And that's these like three successful traits that you defined for engineering teams and why they matter so much. So in this report, there were like three key items that we found that were consistent among successful engineering teams.

16:24So first, they're well regarded by non-technical leadership. They're happy with their development practices and they're able to ship code quickly. So Luca, let's just dive in those three things to start with. And why are these things so tightly correlated with engineering success? us? That's such a good question, because when we started diving into the data from the survey, we tried to do that without bringing in any kind of bias from our own, because of course, we all have our own opinions about what makes engineering teams successful, you know, when it comes to practices or the type of outcomes.

17:00We really tried to go into the data like eyes wide open. And we found that these traits were, with a wide margin, the most correlated with all kinds or good practices you can think of, which means when it comes to engineering being well regarded by non-technical stakeholders, engineering being happy about their practices, project shipping on time, all these things, you can trace like a gradient that the more these things are good and well considered by engineers, the better the engineering team performs on things like having enough focus time, spending time as it was planned, engineers not needing to wait for others, They're happy with everything they're doing.

17:41So I would love to be able to say these are the successful traits because of this and that. But I can only observe the data and say this is apparently what matters the most or what is correlated the most with success. Yeah, I mean, you know, we hear constantly that like developer experience is like a critical component of being productive, like having a good experience at work and being engaged with things that you do. So, and yeah, I do think it's important though to note that, you know, these are correlations. So it's shipping quickly doesn't necessarily mean that everyone's going to be happier about things.

18:16But, you know, a team that is able to ship quickly and is successful over a long term is probably going to be more likely to be well regarded within their entire organization. So, like that brings me to my next question on this. What are some practices or cultural elements that you think help teams develop these traits? Yeah, this is, again, another great question, because as you briefly said, we cannot know for sure where the, let's say, the direction of the correlation goes, right? So I think that in some cases, engineering, for example, being well-regarded among technical stakeholders is downstream.

18:58The fact that the engineering team has proven themselves worthy of being well-regarded because of the good practices, for example, that you may mention. other times it is maybe also a top-down element that comes from technical survey leadership, you know, technical founders that creates a culture where engineering is well-regarded. So I think there might be different directions that depend on the team, but what we are seeing when it comes to practices that lead to engineering being well-regarded, it's many of the things that you may expect, like actually project ships on time, people not needing to wait for others, engineering investment being allocated in a way that is predictable.

19:43So speed, for example, is very important, but it seems that predictability about outcomes and about where the time goes and where the time is spent plays even a major role when it comes to trust. Yeah. And I've been thinking about that first point a lot, being well-regarded by non-technical leadership. It's like, if you're not well-regarded as an engineering organization, you should dive into those with those other organizations why that's the case. You know, like, you know, marketing is going to launch a new campaign around features that we're going to release. And sales is like promising these features to people that they're speaking to in conversations.

20:20And if your engineering team isn't predictably meeting those expectations, then you're probably not going to be well regarded. So if you have that conversation and you find out that sales is frustrated for this reason or whatever, I feel like that's such a great place for an engineering leader in particular to start that conversation with, how do we make the justification for our engineering organization as a whole to be more effective? Yes. Yes, I completely agree. And I think that this is a conversation that is more and more relevant these days and this time in history where you see companies getting leaner, engineering teams getting more productive with better tooling than ever with AI.

21:01So I think there has been a time where engineering value was kind of taken for granted and teams were ever expanding in big tech and in various departments. But now there is more tight control over what kind of budget goes into engineering teams. And I think this is in many cases even a healthy thing that makes leaders have to come to the table with a good understanding of what's the business value of the things they're doing. And I think it makes engineering organizations mature because they have to go outside of simple their technical castle and go there and be able to speak business, which is a good thing, if you ask me.

21:44Yeah. Yeah. And a lot of organizations aren't experienced or used to doing that. You know, like it's a level of maturity that I think is still very fresh to a lot of people in the engineering side of the equation. You described the data as a gradient, which is something that really stuck with me about how all of these things trend together. And it makes sense because if something is, if the team is well-regarded by non-technical leadership, it's probably because they're shipping quickly. And if they're shipping quickly, it's probably because they're happy with their practices. So they all feed into each other, but it creates a unique chicken and an egg scenario, like a hard problem to solve for the engineering team that doesn't have all of those in place already.

22:25Because if they're not happy with their practices, then they're not able to ship quickly. So they're not able to become well-regarded. And if they're not able to become well-regarded, they can't improve their practices and that way they can ship quickly. So it's an empowerment issue. And how can leaders, you think, how can they get that moving if their team feels stuck in proving themselves through these like chains of traits that we're talking about? Yeah, that is such a tough question to answer. First of all, I want to say that what you just said is completely confirmed by the data. That is, what you see is that successful teams are not like successful at maybe one big trait.

23:06You know, they are successful at everything. So good teams do everything good. So it's like a flywheel, as you mentioned, that success brings more success and of course makes people happier about engineering and that gives engineering teams more leeway and space to have more impact and that creates, you know, more trust. And so I think this is the positive flywheel, which goes also in the other direction. Things don't go well, trust decreases and you get less scope, less impact and da-da-da. So I think when it comes to what to focus on, there is a lot of things that you can work on, a lot of tools to measure pretty much everything these days, but everything takes work.

23:49I mean, when you take one piece, one part of the development process and you want to improve that, it's a lot of work, whatever you focus on, because you have to measure what works, what does not work. You have to embed that into, I don't know, team processes and see if things improve over time, design initiatives to make the thing get better. So I think the safest way is just to focus on one thing at a time, rally your team on that and create momentum on that thing and then go really one by one, knowing that many things are related to each other. So they don't really work in isolation, but you can neither fool yourself by thinking that you can just address all the problems of your team at once.

24:32That's right. Our co-host Dan, he says this a lot, that you should just find one thing and put an automation or a fix in place to make it better. Even if it's not perfect, just making that one small change is really important to getting started. And when it comes to a flywheel, one small change is how you start pushing it the other direction and making it from a negative flywheel into a positive one. Yeah. And I think, I think visibility is such a key aspect to this too. Like, cause you know, I think back to anytime that I've tried to unpack a, like an overly complicated process or something that just isn't working as efficiently as it should be like mapping everything out so that when one of those efficiencies appears, you have a point of discussion.

25:13It's like, here's a pain point we're experiencing today. And here's the data that shows us that we're experiencing that pain. What do we do to minimize that in the future? And, you know, that helps you just one by one, pick out the things that are sort of slowing your team down. And speaking of slowing teams down, you know, we found one thing that was found in this report is that the frequency of shipping code impacts team happiness and performance. The report shows that shipping code multiple times a day makes teams dramatically happier and more effective. So why do you think that shipping frequency is such like a game changer for morale?

25:50Well, I think everyone can relate from our own experience when I mean, as software engineers or for those of us who are managers or CTOs from the time where we were engineers, I think the best times in terms of personal satisfaction and feeling impact was where we are able to do things really fast and push new features and code like nonstop. Maybe you can think about the experience you have with your side projects, right? Where everything feels fast, you ship in minutes, and you can make incredible progress by yourself. So that's the exact feeling that some of the best teams are able to replicate on the full team.