The fundamentals of agent-driven software workflows

26:04But like you said, if the AI cannot find what it needs, it is going to give an answer. It might give like a snippet of code, but it's probably going to be incorrect. Exactly. Now, one thing that I've been talking about, so Linear B, we have an AI code review today. We have a pretty killer AI code review. I talked to our head of AI, who I consider our head of AI, Ofer. And where we're going with this is actually in this context area. So if you think about what Linear B can do, yeah, we provide you probably a lot of our listeners know with all these different metrics. But the way that we get these metrics is we're connected into Jira, we're connected into Slack, We can see all of the incidents.

26:52We can see bugs that went out into production, change failure. We can see documentation. We have this wealth of connectivity. And now actually what we're doing next in the roadmap is going to be while our AI review is processing, if it doesn't have a piece of information that it needs, yeah, of course, it can go get information from other repos. It can get it from documentation. We can see if there's a bug in the area. So we're actually connecting together that missing context, which totally makes sense because that's what you're saying in this first part of the guide is making sure that you have all of that connectivity so the AI can essentially find the answer that it needs.

27:32You're touching on a lot of points that we're going to get into later in this episode, too. So stay tuned for that because, yeah, I really love how this stuff is all kind of connected. But, you know, and as a part of this guide, we went back to recent Dev Interrupted history and we found a lot of guests, former guests that really articulate some of the challenges that we're highlighting in this guide. And for this one, we had Brandon Jung from Tab9 on, who last year discussed with us about this concept of like creating golden repositories, or in other words, a piece of code or a collection of code that you trust as being really high quality and has all the context behind it for why it was constructed that way.

28:12Those types of resources are really valuable for AI when you're starting to get it to make more decision making. And I think what's really important to understand from this guide is that all the recommendations we're making here are both things that will help your human developers today, as you put it, Tan, but also the AI developers that are coming around the corner for you. So you want to do things like establishing data hygiene practices across your organization, maintain high quality training data sets, like establish responsibility within your organization for who is responsible for maintaining high quality data sets and start working towards centralizing that knowledge into a place where you can start feeding it into your AI models.

28:56And the great thing about this is that AI can actually help you do this. So you can start using AI today to start preparing for these changes that we can see around the corner. That golden repo concept is pretty cool, actually. We are adopting it for a lot of workflows. You know, it's kind of been incredible to me how much software engineering has aligned with the content creation workflows we're building here at Dev Interrupted. And the more we have these like idealized sources of information, the more effective our AI workflows can be. And I don't know if it says it in the guide or not. So I'll ask you a question, maybe, and maybe we don't know.

29:34Let's see if we have it in the guide. But is there anything that like qualifies as like a golden, is there any like criteria behind it? Is there, I know it has to do with like a level of trust. We think the code is really, really great, something like that. But I don't know if it gets into it. Is there anything that like qualifies for, okay, it's like meeting the bar of what's needed? Yeah. I mean, the big thing is this varies from organization to organization. What you don't want to do is to feed code that you know has significant difficulties into an AI as a training resource. You know, so as long as you're avoiding that, this is the kind of thing where you can iterate and get better.

30:11Like the more you feed better data into it, the more it can help you create better data for training. So there's definitely like a great benefit to taking an iterative approach to continuously improving whatever data you're feeding into your AI. Yeah, very cool. I'm going to move us on to the next pillar that we have here. So this is around modernizing for agentic workflows. So the second trend that was uncovered in this area of modernizing infrastructure for agentic workflows, it feels like a lot of teams maybe are still struggling with basic like CICD. And now we're talking about like modernizing the infrastructure for agents again here.

30:53What new layers like agentic observability or sandbox testing are required for these more advanced AI systems? Yeah. So you mentioned how LinearB's AI is able to go out and connect with all these various tools to pull additional data and context into the workflows. And for agentic AI to be successful, you're going to need platforms that are designed in that way. Right. You know, a lot of teams are still stuck with like more legacy infrastructure that is potentially stifling automation and scalability and developer velocity. So when you really think about it, agentic AI and AI driven software development are going to require tooling that isn't widespread yet within the most engineering organizations.

31:36So you might have like some outdated or manual ops practices that have little to no automation or maybe there's no self-service capabilities. You probably also have ticket based workflows within your company, like I'm thinking like Jira for project management, ServiceNow. Now, those aren't all going to scale for AI adoption in the way that we use them today. Like if your coding time is going from, let's say, two days down to two hours, what does that mean for a QA process that might take a day or two? Like that now becomes a substantial bottleneck within your workflow. And, you know, there's also just like the challenge of like modern engineering just requires a lot of infrastructure knowledge that developers just aren't always trained for.

32:20Like the technologies are changing constantly. Most developers are trained to write code rather than to think about infrastructure problems. And this all just creates a lot of friction for adopting agentic AI. Actually, you know what? One of the things that I'm seeing, at least with like the customers that I'm working with, and, you know, of course, a lot of this goes back to like the developer's experience. Like, okay, again, you might buy an AI tool, but the developers are the ones that are expected to interact with it, use it. I think like a lot of this comes down to like the accuracy or the quality or the integrity of the suggestions or the code generation that the AI is able to provide.

33:05So, for example, in the first two kind of sections that we talked about, yeah, if there maybe isn't these golden repos or maybe there's the agents having a hard time getting an answer, working its way through the different areas of your workflows and the infrastructure, the quality is not going to be there. Right. And what I've seen is when we're interacting now, we're talking with the developers and maybe it's obvious, but I'll say it. The best AI tools that they like working with are the ones that are giving accurate, high quality suggestions. and that's where their experience is actually improving and their productivity is improving.

33:47Now you have the flip side of that. Hey, I bought an AI tool, maybe my infrastructure or workflows or the context wasn't there and actually it can be pretty annoying to work with because it's giving me suggestions that are incomplete, wrong, that type of thing. So I think it's also like a sensitive time for developer experience and therefore it's like really important to get these first two kind of pillars that the guy talks about right. I don't know if you've seen that at all, Ben. That's what I'm experiencing. Yeah. Like in the field. Absolutely. There's definitely a very strong perception I've seen of developers who are very frustrated with how AI initiatives have been going.

34:29And it's almost always rooted around high volumes of low quality output. And we'll actually dive into this a little bit more later in this episode. But I think back to an episode we had not too long ago with Corio Daniel from MassDriver. And one of the things that he pointed out is that platform engineering is going to become much more important in this era as a way to abstract the complexity of working with AI. You shouldn't be relying on every individual developer to navigate all of these difficulties. like a more centralized platform engineering oriented team, even if it's not a dedicated team, if it's just a group of people who care about it, who have been given authority to act on it.

35:13You want them to be focused on enabling developers to ship code using AI without adding infrastructure or just like burdensome overhead. So the more you're investing into better pipelines, better automation, test coverage, the more you'll enable your developers to accelerate without amplifying risk. So if you already have really reliable pipelines, introducing AI into it helps reduce the risk with that. Yeah, totally makes sense to me. What I like about this guide, I think we only made it to the first two, and we have a few more to go here. But if you're looking at each one of these six pillars of the guide, it essentially is helping you form a strategy.

35:57Exactly. Because again, like we said from the beginning, it's not just like buy an AI tool, turn it on, you're good to go. It's like the best engineering orgs, the best companies actually are looking at all these different factors and saying, this is how I'm going to actually put a strategy together to make, I guess, productivity output and the best experience for our developers. So this is, I like this about the guide. Yeah. And like I said, we're also focusing heavily on stuff that is going to benefit you today too. There's a lot of benefits to adopting these practices now beyond AI. So you're just helping developers today while being prepared for tomorrow.

36:35The next section that we have here, and I hear this actually like a good amount from customers, but there's this area of like governance or control. It's a pretty, I think, critical topic here. And maybe it's even just around trust. So as agents start taking actions, like they're merging PRs or they're triggering a deployment, they're creating more and more code. Governance, that topic is starting to become essential. And especially if you're like leading a company that's in like finance, healthcare, critical, you know, systems, stuff like that, banking, all of that. This third trend is kind of like reframing this.

37:19It's saying using governance as an AI enabler as opposed to a blocker. So why is it important to see governance as an enabler rather than a barrier? You mentioned how AI agents are going to start taking real actions in our workflows. What you want to try to do is shift compliance from being a hurdle that developers have to overcome to one that implements or establishes trust with your agentic AI and makes it more explainable and scalable. So you have all these autonomous agents making decisions, like you said, they're merging PRs, they're identifying and deploying hot fixes, at least they will be soon.

38:01And this naturally is going to raise a lot of concerns around safety, audibility, and accountability. And really what's at the center of this is, you know, LLMs are inherently probabilistic, yet we're often sort of viewing them through like the deterministic lens of software development, even in like, as you said, high stakes environments, like when you're working in banking, there's certain situations where you don't want to have like a probabilistic actor, such as like when determining how much money somebody has in their bank account. But the challenge is that with AI, they're kind of like a black box and it's, it can be very difficult to trace their reasoning or ensure a high level of explainability.

38:40So if you don't have clear audit trails, if you don't have trustworthy data sources, as we mentioned earlier, your teams aren't going to be able to reliably validate the decisions of AI. I think back to an episode we had quite recently with Brooke Hartley-Moy, the CEO of InFactory. And she made a really bold statement where she said that trust is the only thing that is going to matter at the foundational level with your LLMs. If your developers don't trust the outputs of their AI, or if your business doesn't trust the output of it, then you're fundamentally going to have difficulties with your AI strategy.

39:19you. I'm saying the same thing. And what I keep thinking about is that makes sense for AI. It also makes sense for developers, both. Like if you, let's say that you had a situation, you don't trust the output of your developers. Well, then the business isn't going to trust. If you don't trust the output of the AI, also the business is going to trust. So it's actually like a very similar or same paradigm. It's just probably much less trust right now with the AI. But the cool thing that I'm seeing is companies still want to use it and move forward. It's not like governance is, I haven't seen it be like a blocker to try.

39:59But what I will say is most of the companies that I'm working with, there has to be rules in place around the AI. And these rules are always, I'm sure we'll talk about it again with like orchestrating and stuff, but you have to have rule automations that kind of surround the AI around what's allowed to be merged, what's not allowed to be merged. How does your review work? It's kind of like vibe coding, but like the backstop, the review check, the yeah, okay, AI has generated a bunch of stuff, but let's make sure that we still have gates, checks, quality in place so that what does make it to production because we still want to be accelerated, still want to move faster.

40:43And a lot of the customers that I'm working with, they are actually writing rules usually around the PR process to ensure that the code is making it out with like super high quality. Yeah. And you call them gates, but I think it's important to recognize that if you're giving a developer who's been blocked by one of these gates, so to speak, if they just get blocked and they don't have clear instructions on what the next step is or why things are being blocked, then you create frustration, you destroy trust. Whenever you're creating like a governance framework, it should always be centered around enabling developers.

41:19If you're going to block them, tell them what the next step is to get unblocked and help them take that first step. I think that's a really important distinction here when you're thinking about governance. I mentioned Brooke Hartley-Moy's episode. You know, she had a lot of great advice about building better audit trails that are built on trustworthy data sources. I think one thing that's really critical is that you implement smart human loop mechanisms in any AI workflow. So there are certain things that you can hand off to an autonomous system to delegate that task. But then there's always going to be moments, at least for the foreseeable future, where a human will need to interject or be prompted to step in and be an active participant in the decision making.

42:00so that the more fluid those human in the loop moments are the better and as i mentioned you want your governance frameworks to be designed to give developers the easiest path forward and avoid shifting left the responsibility for compliance onto developers without giving them those automated tools to take action so there's this sense that as you're building autonomous systems there needs to be automated guardrails that keep everything going down the right pathway Yeah, it's almost like gates is not the best word. It's almost like you got to smooth out the workflow with automation and make it visible and transparent to the developer on what the rules of the game are.

42:41So there's not like confusion, which, like you said, can be a frustrating experience. Why am I getting blocked versus something else? And all of that, that makes total sense. I think when you're talking to your head of legal, it is a gate, you know, it's a gate to protect compliance issues. That's for sure. Hey, hey, we have gates. We have critical pass. We have control. That's what you're looking for. When you're talking to your developers, it's a guidebook, right? It's a path for doing things the right way. Yeah. All right. Let's move on to the fourth part of the guide. So this is talking about isolated tools versus a more orchestrated system here.

43:24So most AI tools today, they can feel like they're a little bit bolted on to an older, which they are like an older workflow, or maybe a workflow that didn't know that AI was going to arrive so rapidly. And this is the fourth trend that's highlighted in the guide. So it's moving from tools to an orchestrated system. What's the key difference between an AI tool that assists a developer versus a truly orchestrated system where agents communicate with each other? Well, speaking of bolted on AI tools, I feel like the first major wave of this technology was like a chatbot or it was either a chatbot or it was something built into your like predictive typing and your ID.

44:06But, you know, I think we picked chatbots primarily because of convenience And it was just easy to bolt on to existing workflows. And I think years down the road, we'll look back at this period where we're interacting with our AI through chatbots as like a very relatively brief period in the AI adoption journey. But the important thing here to understand is that the future of AI in software development is not about isolated tools. You're not going to be buying a bunch of different AI platforms. It's going to be more about orchestrated systems where agents are reasoning, delegating, and operating across your entire software delivery lifecycle.

44:41So when you think about today's developer tech stacks, most of them lack native support for autonomy or interagent communications. Google recently announced the A2A protocol, which just recently went under the Linux Foundation. This is like the first attempt to create a standardized protocol within the space, but it's still very early on. There's no standard protocol that has emerged for agent communications, which is really forcing teams to rebuild integrations on their own for each use case. So a lot of our existing infrastructure treats tools like isolated functions rather than interconnected workflows.

45:22And the thing we're reiterating a lot here is that your agents are going to be operating across many tools, across many data sources. And until the native integrations are there, everything that you were going to be doing in that space as an engineering leader is going to be more custom and bespoke. There just simply aren't a lot of products out there that have solved this problem yet. Yeah, I think that's the trend on where it's going. I also think that's why I see the demand for, hey, I need one platform that gives me kind of like that, I guess you can say control plan again, but that transparency of show me what all of my agents are doing.

46:04Is there actually bottlenecks or not bottlenecks? Who's talking to who? Do I have the control? because I think piecing it together is tough. Piecing it together is like, it's just, it seems like it would be so hard to manage strategically over a long period of time. So maybe that's why the demand, I'm seeing a rise in that demand on my side from talking to customers. Yeah, yeah, that makes sense. And I think back to an episode we had with Amir Behibani from Memra, where one of the big points that he made was that developers really sort of have to shift from like a carpenter mindset, like building discrete components to more of an architect mindset.

46:46So designing systems, orchestrating higher order systems. And one of the ways you can tackle this or start tackling this problem today is to focus on challenges that are like low hanging fruit within your workflows, things that consume large amounts of time, but also have really strong API integrations and standardized usage patterns. These are going to be the easiest to build like a custom integration layer with today that starts to connect the most critical systems within your agentic development workflows. Be prepared for this to be bespoke for the time being until a more standard and operational platform start to emerge in this space.

47:28In the meantime, you can identify like a lot of these repeatable high leverage tasks with clean APIs and consistent usage patterns. Like one really great example that we've covered here at Dev Interrupted is Google recently published research where they showed how they're now using AI to do 32-bit to 64-bit migrations. Like this isn't a glamorous job. It's something that, you know, I'm sure there are some developers out there that love this kind of work. But I would say most developers don't like get out of bed excited to go migrate a bunch of. But one is like they're paid a lot like it. Other than that.

48:02Yeah. Probably not. But Google has implemented a workflow that uses human loop mechanisms with a few autonomous systems like strung together to handle the bulk of that work now. And in this experiment that we covered, they did, I think it was close to 600 changes and 75 % of that was written by their AI system. And the developers involved with that estimate about a 50 % productivity improvement. So, you know, these are the kinds of things that you can start solving now, even though the platforms to solve this as a product aren't quite there yet. totally makes sense ben actually i wanted to ask you and it will lead to the next section in the guide here but you said something previously that right now or maybe when it started it was kind of like okay humans are interacting with the ai agent maybe asking questions of the agent or getting responses and the section that we're going to talk about here is this trend that you're seeing with like, okay, recalibrating for developer to agent collaboration, right?

49:10Developers are used to using tools, not used to collaborating with agents that like take an initiative. And the first question two part here, I'm sure that creates some friction and the guide talks about that friction. Yeah. But also just wanted to get your opinion. How long do you think that will last where it's like developer to agent versus agent to agent and a developer overseeing a bunch of agents together. Yeah. Well, I think at the core of this is going to be, you know, today, if you think about how most people use AI tooling, they do their work. And then when they need AI's help, they go and they prompt AI.

49:50Imagine a world where that gets completely flipped on its head. Like AI is doing a lot of the legwork. And then when it gets stuck or when it's been told that this is a moment that you need to bring in an expert or, you know, for whatever reason, you might stop the workflow. I cannot proceed. I can't proceed. Like exactly. Yeah, it goes and then prompts the human to intervene, you know, so it's actually like almost a complete reversal from what we're doing today. And, you know, as agents start to take that initiative within developer workflows, it's going to create a lot of challenges around designing natural and low friction collaboration between humans and AI.

50:28And eventually, you know, as you mentioned, it will be more agent to agent. But I think, you know, I don't want to discount the role of the software engineer here. I think the humans in the loop are still going to be valuable for quite some time, if not just forever effectively. But as you mentioned, you know, developers really aren't used to collaborating with agents like as a coworker. I don't want to call them a coworker, but that's kind of how they might start to behave, especially when you start thinking about them taking initiative and going out and maybe they start, they see a customer report for a bug and they go out and fix it and then prompt a human to go see if that actually, you know, fixed it before merging it like that kind of workflow.

51:06And the risk is that if these agents are misaligned with your goals, with the way that you build software, with anything within your team, anytime they're misaligned, you're going to create friction and inefficiency. And if you're asking your developers to override agent decisions, you're increasing cognitive burden. So if you have, you know, getting back to this, this automation and guardrails, like do you have all of this autonomous automation without guardrails? It's a recipe for overload, especially as you think about a world where you have more and more bot generated pull requests. Honestly, for me, it goes back to like also, again, the beginning of what we talked about, the accuracy or the quality of the agent.

51:49because you said, I think, actually, there can be situations where it increases cognitive load because I keep having to correct the work that the agent done. Then you might say to yourself, why not I just do it myself? Right. So it's like it starts with, I think, that foundational thing of like having the good accuracy of the agent by having great contacts and having an infrastructure that supports that great context. Exactly. And I'm reminded of a great episode we had a while back with Tara Hernandez from MongoDB. And, you know, she talked about how important it is to make the golden path the easy path.

52:30So on the probabilistic side, you can like work to change organizational norms to like accept agents as valid contributors. But then there's also like in a deterministic side where you can start automating your guardrails to reduce that mental load. I've already mentioned the risk of shifting AI compliance left onto developers. If you're doing that without actionable automated oversight, that's where you're going to create a lot of those friction points in the collaboration workflows. So the more that you establish clear interaction patterns between their developers and agents and design AI experiences that naturally fit into existing workflows, that's how you're going to build a lot stronger trust with your developers as you roll out your AI initiative and achieve better productivity improvements.

53:18Yeah. So actually, Dan, this brings us to the last and five, like one of the most crucial trends, I think, and that is that feedback loops can make or break AI adoption. And this time I wanted to flip the script. You've been asking me a lot of questions. I actually want to flip this one around and ask you about this because I think it's something that you probably have a lot to share. So why are feedback loops so critical and how do teams create like a signal-rich environment that agents will need to get better over time. Yeah. Okay. I want to talk about what I'm seeing from real-life deployments, strategic AI deployments across like really cool and important companies.

53:56We'll start why feedback loops are important. I think everyone that's in engineering already knows this. It's so that you can get the information, tight little feedback loops so you can iterate quickly on your plan, strategy, and execute. Now, with an AI strategy, what is needed to have a great feedback loop? We talked about it earlier in the episode. You need a platform that provides transparency, in this case, measurable transparency, into what are my AI agents actually doing? What are they producing? So I call that kind of like the first step. It's like, how much are they doing? What are they doing?

54:36Where are they doing it? then as part of that, you need to understand, put the AI agents aside, do I still have bottlenecks in my workflow? Let's say that I got a cursor or a co-pilot. Am I actually producing more work? Yes or no? So you have to have also that telemetry to say, and this is what I see most often, oh, I'm actually not producing more work because I have a bottleneck, for example, in my PR process. I have some agents generating some great code in the IDE, but my business didn't get any faster because I still have a bottleneck, for example. That's having great transparency. And the third part, I think, to that, just starting with the transparency and actually receiving the feedback, is the feedback from the developer.

55:23Do I have a way to engage, see what the developers are experiencing? They can say, is this agent working well for me, not well for me, and so on and so forth. So the first part is end to end, I'll call it like single pain transparency. Now, the second part that I'm seeing to a strategy like this is actually taking action. So not just measuring, but taking action. And usually what I'm seeing customers do is they're asking themselves, am I purchasing the right AI tools? And am I purchasing the right AI tools in the right areas of my SDLC? So for example, I'll use the cursor in Copilot to start with because that's where most people started, like in the IDE code generation.

56:11But am I also purchasing agents that are maybe helping with testing, documentation, where Linear B specializes, AI code review, pushing code from PR to merge? And can I, again, measure each one of those. So the second part of this is actually getting two AI tools in place that will unblock the workflow end to end. And the third thing around it, we already touched on it in this episode, is do I have automated workflow and rules where controlling governance is actually accelerating my workflow? So I do have automation, do I have policies in place that support the tools that I've built. I have the transparency and now I have the automated rules in place to actually accelerate the workflow.

57:03Those are the companies that I see that are doing the best job in their AI initiative for productivity. Yeah, and I want to call out a past episode. That's quite the journey that you're outlining there. And if your organization is still pretty early on to this, it can feel very daunting. Like, you know, let alone like adopting AI, but even just getting the more foundational practices of having visibility into where like engineering bottlenecks are. Like that alone is a challenge that solving that today can be a huge benefit, even without AI. And a lot of organizations out there, I think, are still trying to solve that.

57:40So we had Somya Subramanian on the podcast earlier this year. She shared an amazing story about how she's taken a lot of the knowledge she gained from Google about standardizing metrics, building a foundation of data-driven habits. And she shared a lot of great insights about how to effectively implement that within your engineering team. So if you're deep into the AI, I think we've got a lot of great past episodes. But also if you're brand new to this stuff as well, there's still some great episodes for you to consume from our archive. Awesome stuff. All right. I think that wraps us up. Ben, thanks so much for walking through all of this with us.

58:21This guide is freaking awesome. If you or your team want to dive deeper into these six trends that we just discussed and get the full checklist for preparing your organization for your strategic AI transition, you can find the guide. The guide's called The Six Trends Shaping the Future of AI Development on the LinearB website, so LinearB.io. We'll be sure to put a link into the show notes as well. And of course, Ben, thanks for joining and thanks everyone for listening. We'll see you next week.