AI is the future of the SDLC

0:05Welcome, everyone, to Dev Interrupted. I'm your host, Andrew Ziegler. And I'm your host, Ben Lloyd Pearson. This week, we're talking about how the Pentagon is kickstarting their own kind of Y Combinator and looking for the next Palantir on college campuses. And Reddit's lawsuit against Anthropic for data scraping that's hitting the news. And finally, an engineer's reflection on the AI skeptics in his life that gave us both a good chuckle. Ben, what are we cracking into first? Yeah, let's just start right at the top and talk about what's happening in the defense industry. Yeah, so the Pentagon, the defense sector of the U.S., is launching a major initiative to increase software startups that are incubated for the purpose of serving the military.

0:49So we're talking about startups that are funded for military purposes and incubated in a very similar way that we see startups for consumer businesses, B2B businesses, maybe historically through things like Y Combinator, right? It's an attempt by the government to modernize internal things on old systems. It's actually a theme we've been covering quite a bit here on Dev Interrupted. We talked recently about changes at the FAA and at airports around updating technology there. This was a really interesting shift into like a tech-focused world incubated by the defense sector. So Ben, what did you think?

1:25Yeah, I saw they were offering some startup investments for recent grads that were wanting to launch their own company that supported both the defense and the commercial sector. But I think one thing that's really important to remember whenever stories like this come out is that software engineering as a profession is still pretty young, relatively speaking. You think about other types of engineering, like civil engineering or architecture or material science. All these other types of engineering have a much more storied and established history around them. And, you know, I've previously brought up like how I would love to see far more public investments into building more robust software foundations for society.

2:10And I think we're increasingly heading to a world where software does end up being the ultimate defense in a lot of situations. We're all familiar with the story of how like software has eaten the commercial world. And I think it's going to eat our governance systems as well, just like probably on a slower timeline, you know. And there's been a lot of modernization effort that's happening in the US federal government in the last decade or so. You know, if you are familiar at all with the Air Force, like they went through a really massive digital transformation during the COVID era that ended up then becoming a model for a lot of other organizations within the Department of Defense.

2:48and I've actually met some of the people who have started and worked for some of these more defense-oriented startups. They operate surprisingly similar to like your typical tech company, you know, the same types of people you would see in a Silicon Valley tech company with the same like approach to building software. So I think that's a great thing. I think this is probably going to be a good investment over the long term and I hope we see some cool companies come out of it. Yeah, I think there's a lot of opportunities to innovate, especially within the government. So I'm interested to see what comes of it.

3:17You know, related to that, you talked a bit about modernizing defense infrastructure, that there's like a lot of need for it, right? Well, there was a fun tidbit that came out recently in Japan about how they modernized a part of their own social infrastructure. Maybe that's a fun news bit that kind of resonates with your own interests there, Ben, talking about the opportunities in society to improve things. Because when I read this article, all I could get was immediate flashbacks to living in Japan, where I was for two years after college. And the dreaded experience of having to order like a movie ticket or a concert ticket or buy furniture from the vending machine and the konbini was always so daunting because putting in my address, which was like four sets of numbers and three sets of kanji and really complex code was very hard for me on the machine.

4:04But in Japan, they've just launched a digital address system that crunches all of that hard to understand, hard to input information into a simple seven digit code that folks can use on these types of machines living in Japan to do everyday life things like pay their bills and order concert tickets if you were me in Japan. So this was a fun tidbit that came out. It's a really great effort from a government that typically hasn't done a lot of technological innovation. Japan is actually not really known for having this type of system. So it's really great to see this quality of life improvement for everyone there.

4:38Yeah, I love how it's just like a low hanging fruit to help people out. And I'm sure both foreigners that are visiting and long-term residents, there's probably a lot of value to these types of improvements. For sure. Yeah. So let's talk about something that's very near and dear to my heart, and that's Reddit. What's the story that's going on there? Oh, yeah. So Reddit, we all are familiar with the great field of Reddit. So Reddit has built a$2 billion business on user-generated content that's categorized by any kind of topic you could think of. It's a treasure trove of information for AI. We've seen this already play out over the last several years as foundation models have come into existence, wanting to get their hands on Reddit.

5:19Because Reddit has this naturally organized sets of information that are natural communication between people on highly intricate topics. But ultimately, Reddit is a place where that information and data has gathered. It's people's conversations over, you know, decades even. There's something interesting that has happened where Anthropic apparently has been making a bunch of unauthorized access to Reddit servers to access this information. And this is after a licensing deal fell through, you know, and so Reddit is not getting paid what they think is their fair share. Reddit hasn't really done anything to make that AI, that information AI ready for consumption.

5:57Like there's no cleaning or curating or enrichment that they do on there. And they just hand you all of it. the spam and the bots and the snake eating the tail all in itself so it didn't really write the content it didn't enhance it but now it wants rent from the innovators that want to build on top of it uh it's a really interesting evolving play in the power economy yeah what what is model collapse i mean if anything it would happen through reddit being trained on reddit yeah exactly exactly i've been a Reddit user for well over a decade at this point. And if you're a longtime Reddit user, you may know that it's pretty easy to pile on the hate right now with them because they've kind of made a lot of decisions recently that has upset a lot of their users.

6:42But I want to point out something really important, and that is Reddit has been, it's probably one of the most astroturfed platforms on the internet. And I think this problem gets worse and worse every single year. So if you're like me and you've normalized this practice of turning to Reddit to find advice for local communities and for niche problems and all of that, which historically has been very good, a very valuable platform for that. I think whenever you do that today, there's a pretty good chance that you're now reading bot-generated content because it's been largely unregulated on that platform.

7:17But to get back to this story, Reddit, like many other companies, is being disrupted by all of these new AI tools that are coming out. Think about how there's probably lots of people now that are going to all of these GPT services to ask the types of questions that they might have taken to Reddit in the past. And companies like Reddit may be facing a future where their primary value proposition is all of the data that they have to train those AI models. and you know we mentioned model collapse like i suspect with just how much bot generated data is on reddit that it you know there's always a risk that this data is not as valuable as you might expect and it could actually end up becoming a liability risk if you are actually facing model collapse because your data isn't good enough and i think we should follow this story either way you know it's it's all alleged at this point like anthropic denies scraping this stuff outside of the terms of service.

8:13So maybe they did decide that the data just wasn't valuable enough to continue using. Yeah, I think it'll be an evolving story. So I'll be interested to see how Reddit continues to chase down folks using their data. I'm sure this won't be the last one. Yeah. Yeah. And the last one we have is a bit of a fun one. It's not really news. It's more of an editorial. So what do we have for this, Andrew? Yeah, I love this article. We came across it on Hacker News. This is an article from Thomas Patachek. He's an engineer at Fly.io, and he wrote this really candid article about all my AI skeptic friends are nuts.

8:46And put out several key arguments in his mind about the skeptics in his life and the arguments that they have against AI and using it in coding and his personal responses that are grounded in his experience as someone using these tools. I found it to be a really fun reflection because some of this is obvious in terms of level setting, talking about making sure you're even discussing the same thing, going in, understanding that it's different using chat GPT versus using like an agent in your IDE. There's a lot of like level setting that he does in this article to kind of help folks navigate these conversations.

9:21If you find yourself constantly starting back at the starting point and skeptic conversations, but also he balances it all throughout with humor, the silliness of why would you be against innovation and trying out new things as an engineer and building? I found this one fun to dig into. I recommend everyone give it a read. Yeah. And I like to routinely reflect on where I was about a year and a half ago when I first started really using AI as a part of my day-to-day work and just how far the technology has evolved in that short 18 or so months. There's a lot of overhyped aspects of AI. There's all these claims of artificial general intelligence, AGI, being around the corner.

10:02Those may or may not come soon or ever. It's hard to tell. but you really can't stick your head in the sand and claim that AI is not going to disrupt many aspects of work. It's already happening today. And I keep hearing a lot of comparisons in this article as well to what's happening now to the adoption of the internet and to mobile computing in the past. I think it's a pretty good comparison because I think the transformation we're going through now mimics those periods in many, many ways. And we're going to come out of this with new paradigms in how we build and consume technology. Just think about how different the technology landscape was before everyone had a computer in their pocket and before everyone's computer was connected to this immense data source on the internet.

10:49And the reality is that agentic autonomous workflows are here. And I think now, like today or the next year or so is the time that these tools are really going to prove exactly how much productivity improvements they're going to create. We're kind of in the wild west right now. Like it's very chaotic. There's a lot of things going on. I think we're slowly going to come under more, more and more control over the next couple of years as AI products become a lot more hyper-focused on solving real world problems rather than these like general purpose chatbots that we have today. As much as I love them, it's only one aspect of where all this is going.

11:29Yeah, exactly. And as kind of AI's involvement in the things that we use, it matures. It's also going to fill into those niches and parts of the process that we haven't seen it really resolve yet. It's going to rise. So I'll lift all of those boats once we can figure out how it's going to actually play with all those parts of a process. So Ben, what's coming up next on the pod? Yeah. So this week, Dan Lines and myself sit down to continue our discussion about the future of AI-driven software development. We're going to dive into a lot of really interesting topics, so make sure you stick around after the break.

12:04AI is creeping into every part of the SDLC, but how far have teams really gone? Linear B surveyed over 400 devs, many of them like yourself in the Dev Interrupted audience, and found that 67 % are already using AI to write code. But is that creating opportunities or bottlenecks? Our new DevX guide breaks it all down, including adoption patterns, pitfalls, and the AI collaboration matrix that charts your own team's journey with AI. Check out the guide and maybe even the panel I hosted about it last week with Atlassian, AWS, and ThoughtWorks. I'll drop the link in the show notes. Hey everyone, welcome to part two of our conversation on developer productivity.

12:48Once again, I've got Dan Lines here with me in the studio. And if you haven't listened to part one of the conversation, we covered the shift from engineering intelligence to developer productivity insights and developer experience. So go check it out if you haven't already, because it's full of great insights that really set the stage for today's conversation, where we're going to be more forward looking and think about how AI is impacting the engineering teams that we talk to every week. So, Dan, it's great to have you back for this conversation. Yeah, awesome to be back. Thanks, Ben. Yeah. So AI is all over the place.

13:23We can't seem to get through a single week anymore without discussing this in some level on this show. But it's really hard sometimes to see through the hype to find the actionable success stories. And that's where I wanted to bring you in today, Dan, because we've got some big views here at Linear B about where AI is going. And I want to talk about that. So, you know, everyone's talking about it. Very few people have a concrete plan for the future. What should engineering leaders be doing right now to prepare for AI? Wow. What a big question, Ben. I like to start big. Yeah, you like to start big.

14:03Maybe let's break down where are we all at with AI, where are engineering leaders at with AI, maybe what's on their mind, something like that. So one is most of the engineering leaders that I'm speaking with, I would say they are dabbling, maybe that's the best, or experimenting with AI in one form or another. usually they'll say something to me like i bought some co-pilot licenses maybe everyone for everyone or maybe for a few teams and it almost feels i mean that that's the interesting part they almost it almost feels like they did that because they felt the pressure to do it so that they could say they're doing something with ai i think i think their developers would revolt if they didn't give them something, you know, in this day and age.

15:00Right. But on the flip side of that, it doesn't always fit into like a cohesive strategy, let's say, of what they're doing with AI or why they're doing it. And maybe there's, and of course there's kind of like an assumption, if I do provide all of my developers and their IDEs, it could be cursor, it could be something else, like good things will happen. Yeah. But beyond that, I think there's a lot of room to understand what the next step is. And then I would also say just because we purchase a bunch of these licenses and developers are using it in the IDE, it doesn't mean that it's actually having a positive effect on productivity or the business.

15:55Now, the other thing that I see a little bit is, okay, I want to roll out AI maybe everywhere into my SDLC, but I don't know, do I need control around it? I can't actually roll it out. Maybe I just want to put it on a few services. How do I get the visibility into it? Can I write rules around it? And there's kind of like the control aspects of this as well. So there's kind of a lot to unpack here. Where do you want to take it? Where should we take this? I think what we're kind of getting at here is that AI is clearly having an impact on just about everyone. There is a lot of hype around it. And there are many times where organizations adopt AI, get very hyped up about it, but don't do it in some sort of like structured, coordinated rollout.

16:47Yeah. So then it doesn't live up to the expectations. And there's almost like a backlash where everyone becomes like disappointed in AI. And then, you know, suddenly you're fighting like pessimism around adopting it. And often it's tied to like top-down directives as well that don't align with the bottoms-up need. So maybe we can start with how Linear B is approaching this. Because I think that's probably where we have sort of a unique perspective on this. I think one of the biggest concerns around AI is how do you safely adopt it across your organization? Maybe we can just start there. Like, how can you make sure that the next step you take into AI is going to be done safely?

17:26Yeah. Okay. You need rules around it. I think it's actually pretty simple. So, you know, with Linear B, we have our GitStream technology. And that GitStream technology essentially allows you to programmatically create rules around when AI runs, where it runs, for who does it run, and full monitoring end-to-end. So usually when you think about control, you might think about policy, rules, programmatic control. And that's what LinearB offers. Now, for example, maybe let's give something more concrete than that. let's say that you have deployed cursor or copilot and you see more code is being created more pull requests are being created and therefore there seems to be a little bit of throughput gain now that we have all of this let's call gen assisted code right still have a human there a developer there.

18:30Now it's made it to the next step of the SDLC, which is the pull request and the review process and the merge process. We're stuck a little bit because we have a large amount of code that's now making it there. And it's almost like the pipe was opened on the development side, but the pipes that actually get the code out to production in a controlled and safe way have remained the same width. Now, what Linear B allows you to do is deploy these rules, these policies, and have the control once the pull request is open all the way to merge. And that's doing two things for you. It's actually making kind of that promise of AI and the efficiency come to life.

19:15But also you can start saying things like this. For these services that are low risk, And this is all codable within GitStream. For these services that are low risk, I'm actually going to allow an AI review to run. And if everything passes, what looks good to me? Because, for example, these services, maybe they're more in an incubator state or an experimental state and I need to just run fast. I'm going to let that LGTM to go through. And then I'm actually going to merge that pull request. That's one rule that I can have. Now I might in other services want much more control than that. I might be in a situation where, okay, the PR is opened.

20:01The AI code review is going to run. And what I'm actually going to do with GitStream is still have a human reviewer, of course. but that human reviewer, that developer reviewer is actually gonna be sent all of the information that the Linear B AI code review has already done. It'll say something like this. Hey, Ben, I actually did 80 % of this review and I feel really good about it. I need you to review this 20%. I still need you to review. Okay, now that's a nice level of control. And a third one, and I've seen this kind of like in an in-between state, let's say, Ben, you opened up a PR and some of that was aided by Copilot.

20:48Linear B is going to have a rule in your service area that says, hey, the AI code review has run. But Ben, since you're a senior developer, you're going to decide now if you're going to bring another developer in to review or not. You have the power. So these are a few, I think, kind of like down to earth examples of actually coordinating and controlling where AI is able to run, have more power, have less power. And you can kind of turn the knobs between like efficiency and your level of control. Yeah. The word control, I think is going to be the word of the year, maybe for 2025, you know, and I think it's, it's like, you really think about how the perception of AI, like everyone started with like chat interfaces, predictive typing, like these natural language to code, but that's really only like a tiny portion of the actual like software development process.

21:48And I think many works still view it in that the first way, but in reality is manifesting more like automated workflows. Like even within the IDE, when you look at how IDEs like cursor operate, everything is becoming like an automated workflow now. So, you know, on that thread, Like, where do you see the future of these like agentic AI workflows heading? Yeah. I mean, obviously no one can predict the future. So this is just what I'm seeing and maybe giving a hint without fully being able to say it of what linear B has coming next. But let's say right now we're in the era of code assistant and agents in the IDE, code assistant and agents, like I said, in the PR process and in the deployment process, but there's still developer involvement and there's still a lot of human control.

22:43That's, I think, the era that we're in right now. But quickly, I kind of see two other eras progressing here. The next era would be something like, hey, the agent actually, so your AI agent actually produces all of the code, let's say, and autonomously does the review and makes a decision of whether it can go out to production or not. And maybe in between there, there's still some developer and human checks and balances, but it's more autonomous end to end. And then maybe after that, again, giving hints of where linear B capability set is headed, and it might sound a little bit futuristic, but I think there will be an era where agents are saying something like, hey, Ben, I created a design for you.

23:38I created a Jira story for you. I actually created the code for that story. Does it look good to you? I actually am ready to release it for you. So when you're talking about a workflow, I can see us headed to a point where all areas of that, like SDLC, start being more AI and agent driven end to end. And then the control becomes more important because now you can start controlling, okay, where do I want it to stop and have a human review? And where am I going to allow it to go on its own to actually move as fast as possible? Yeah. And in the past here on Dev Interrupted, we've had some content that shows how a lot of AI adoption can be mapped somewhere between human produced assets versus AI produced assets and human process, human managed process versus AI driven process.

24:37and I know just personally speaking, there have been aspects of my personal workflow that I have like four or five X through AI, but again, it's only like one aspect of my workflow and gradually over time, it feels like more and more aspects, more and more tasks, more and more things that I have to engage with are gonna get wrapped into these AI workflows. At some point, like you really do start to see like these five to 10 X improvements in specific areas. Right. Right. And maybe if you're thinking about how to get like a hundred X improvement or something crazy or 10 X improvement, it's those different aspects of the workflow actually talking to each other, if that makes sense.

25:17Yeah. Again, like someone from the product side saying, hey, I have a design or I have an idea all the way out to, okay, an agent has coded that idea and reviewed that idea and felt safe enough to deploy. Yeah. So this is all very, very future forward thinking, which is really, really great. I love, I love getting to talk about that stuff, but it's also really great to talk about what success is looking like today for organizations. So obviously no one is out there, anyone who's claiming to be a hundred X-ing their developers or getting rid of their entire software development team through AI is, is not telling you, is probably not being honest with you, but we are seeing some improvements.

26:03So I know we've been working with some companies that have started to realize some pretty substantial gains from adopting this like automation driven metrics informed approach to developer productivity, developer experience and adopting AI. So what have we seen so far today? Yeah. Let me, let me just hit you with some concrete numbers that we're seeing from the Linear B community. On average, after deploying AI and automations with GitStream, we see about a 19 % cycle time reduction. That's an average across the customer base. In addition to that, because we have some things that are helping team leaders and managers in the sense of mostly in the iteration flow, like the sprint flow and the retrospective, about five hours saved per manager as well every sprint.

27:01These are just kind of like hard pieces of data. Now, even more so than that, Ben, I pulled a few examples. I won't say the name of the company that we're working with, but more describe. Just three that came to mind. One is we have a company that saves 71 days. so 71 days you can do the math of our saves so 71 days of saving just by deploying the get stream ai code review in the pr summary if you add up all the minutes of time saved from automating the summary of the pr time saved from reviewing the pr doesn't mean a human reviewer can't be involved as well. Okay. That's a control part. Add that all up.

27:53You get a 71 days saving. That's 71 days across. How long of a time? Okay. So that's going to be over. Let me just check the data here. That's going to be within a six month time period for this example. Yeah. I love the real time data analysis. Yeah, yeah, yeah. I have it up here running live. Now I have another customer that saved 41 days of work with GitStream, and they've done this more so on safely merging 2 ,500 pull requests that were created by bots. So you think about like a renovate bot, a dependa bot, think about bot created work, 2 ,500 PRs that have been merged without human review.

28:43Okay. Now that's coming within a five month period of time. Yeah. I mean, even if it only takes you 30 seconds to click those buttons, the green buttons to merge that, I mean, you still have to imagine the time savings from that. Yeah. So you get, even if it's taking you 31 seconds, but you're also removing yourself from what you were doing. Yeah. Doing that, you know, merge, then having to come back to what you're doing as well. Let me put the time on my calendar every week to click green buttons for debate. Right. And so that's with the GitStream safe merge. And the last one that I have here is I have another company that's combining those two use cases.

29:27So I have one where they've rolled out the GitStream AI code review. They've rolled out the GitStream AI PR summary. And then they've rolled out the safe merging rules. And again, the safe merging rules means you have to pass the criteria. The AI review has to pass. The test has to pass. It has to be in a low risk area. And they were able to save 75 days of work in that same time period. So when we think about kind of like some hardcore stats or what it means to be successful, we said this in part one of the episode as well. It's like how many days or how many hours of work have you returned back to the engineering organization?