Agents get their own AOL, Andrew gets published, and vibe coding is actually good?

15:41Ben:I do agree with him that in team environments where typically you would have maybe weeks or several cadences of planning and discussions before you ever touch a prototype, a lot of that is going the way of the Dodo. The prototype comes out a lot faster now. It can be easier to throw together a prototype from a meeting transcript even. And so the ability to iterate rapidly is, it probably cuts that kind of meeting cadence down to literally just figuring out what the problem is. That said, though, it's like his methodology is really great for also getting out like a lot of possible solutions and then like picking which one might be best for different solutions, which could serve better for different types of problems you're trying to solve where you want to throw a lot of stuff at it.

16:25Ben:And they talked about how they applied this approach with like building their own coding features. Open source recently has become, or Warp has recently become open source as part of its development. And so they're kind of like restructuring their whole development process to allow people to come in and quickly ideate and ship and deliver. It's a really interesting shift to see because I think more folks are going to be doing this, this build than align. But then I also do think that for very specific domain specific problem like the one that I solved in the research paper that I used my preparation methodology for.

17:03Ben:It required me spending a lot of time explaining stuff to the agents that they wouldn't have had a way to know. It lived outside coding space. It wasn't something they were quick and grabby and good at because I was teaching them how to make an educational platform. Whereas in Zach's world, it's like he's using agents to build an agentic orchestration system. They're speaking their own language and so they can probably build in it probably more fluently then we can brainstorm what they think they need. Yeah.

17:29Andrew:Yeah. And, you know, and we're seeing this sort of exact thing play out with content as well. Like here at Dev Interrupted, you know, when we have new ideas, sometimes it's easiest to start with a final draft rather than like an outline or an overview of what we want to cover. You know, we establish the high level of what we want to cover in the content. And then we just go straight to a finished product and adjust from there. Yeah. But beyond that, you know, we've seen other examples here at Linear B where, like, for example, someone on the marketing team uses AI to build a prototype of something that we want to deliver to our customers.

18:03Andrew:And then the dev team works backwards from that prototype to implement it within our platform. So, you know, even outside of software engineering, the function itself, you know, there's a lot of opportunity for non-engineering roles to do the build aspect of this. And yeah, our engineers, they're also starting to work this way in some cases too, where sometimes, and this is not for everything, but sometimes it makes sense to spin up a prototype as one of the first steps for a new capability that we want to build. So, again, so that the organization can have that tangible thing in our hands to give feedback on.

18:39Andrew:You know, instead of starting from the same PRD doc, we're operating on the same platform and able to give very explicit feedback about where we see opportunities for improvement. You know, and that approach in the past consumed tons of time, so it wouldn't have been practical, but I can definitely see it becoming our default approach more and more, particularly when we're building like substantial new features, you know, something that we've never done in the past or something that is a brand new capability. It definitely does make sense more and more to just go straight to that final product and sort of reverse engineer it for lack of a better phrase.

19:18Yep.

19:19Andrew:All right. Let's talk about the technical interview. Andrew. Has AI killed it off? Is that another thing that has been wiped out in the AI era?

19:26Ben:It is definitely getting wiped out. You know, it's funny that we keep saying all these things that are getting wiped out, going the way of the dodo, because even just last week, we had, you know, Brian Bischoff on the show talking about, you know, software stack and how things are constantly dying week after week. And what we're supposed to mean by that, and what we mean by that is disruption. And disruption is certainly coming for the technical interview world. It's already shaken up how folks prepare for getting hired in a tech role. But an interesting part about this article is it points out how obviously startups are moving way faster at this than enterprises.

20:01Ben:But you might be surprised that the interview process at fame companies have largely unchanged since AI has hit the scene. But what's happened is that 67 % of startups report that AI has changed their interview process in some way. So either interviews are handled where there's like a coding process that's done like with a screen share or there's a live coding project or they ask them about the tools they use and kind of shadow their process. Understanding how employees use AI is critical now in hiring them. And this is a really interesting expansion because I think always for technical interviews, like the puzzle-based approach of doing like the leap code challenges and then hopping in and doing like the coding challenge was always flawed.

20:46Ben:We were always talking about how that was flawed. It doesn't quite highlight problem-solving skills. It doesn't properly showcase your ability to collaborate and code on a larger scale, which is what you would be doing in your role. But more importantly, it stresses the importance of just like memorizing a bunch of like edge case situations in code and knowing how to handle them, which in the old world was knowledge that you needed in your organization because if it wasn't in a human's head, you couldn't ask an agent. But now because that knowledge is so accessible, it's less of something that you need to screen for in a candidate.

21:23Ben:What you need to screen for are problem solving skills, the ability to break a problem down into smaller parts, understanding where to start, having taste in AI skills, knowing when to throw something away and when to iterate. There's so many little gotchas in working with AI that can either waste a lot of time or make you super productive. And the value between them is massive. And so that's what employers now are evaluating is how fluent is this individual at using these tools? How well can they use it to enact their will? But more importantly, how well can they collaborate with others with it and working on like brand new problems?

22:00Ben:And we're going to be seeing an explosion of AI skilling and uptraining that I think is going to shadow the world of like LeapCode before it, especially because engineering itself is going to become a more accessible field for folks that traditionally weren't, you know, studying those quiz questions over and over again.

22:17Andrew:Yeah, and we'll include a link to this LeadDev article in the notes. It's chocked full of a lot of really great comments and quotes from industry professionals and how the skills they're evaluating for really haven't changed. It's just the process for doing it that has changed because of AI. You still need engineers who can explore problems, understand and validate code and communicate well. And additionally, system design and judgment of technical trade-offs are increasingly important in AI-driven companies. You need people who can build and manage complex AI workflows and then quickly pass judgment on what's best.

23:00Andrew:Like you said, we've highlighted a lot of the issues with interview processes in the past. Some people today say don't use AI at all. Even I think Anthropic really discourages the use of AI in early stages of their interview process. But I think – and maybe you can justify that in some situations, particularly for like if it's a low-level technical interview for like a junior engineer who's looking at their first role. You might want to evaluate their basic competency with their tech stack and do it in an environment where they can't just go straight to AI and ask questions. But, you know, when people interview at Linear B, you know, we tell them use whatever AI tools you want, you know, get the best outputs that you can with them and leverage it for all the advantages that it gives you.

23:48Andrew:And, you know, we want to test you on whether or not you can understand the value of those outputs and why they were built the way they were built. You know, those are the skills that we really need. And it's skills that AI can't easily replicate or fake, you know. Yeah. I think these trends are going to continue to shift. You know, I think interviewing in particular is going to get is very, very much being disrupted significantly by AI. And, you know, maybe we'll even get to a point where AI starts to run more of the interview process itself. You know, let the person use AI, but they'll be challenged by an AI agent that sort of forces them to walk you through all of those higher order skills that you want to screen someone for.

24:31Andrew:Or you might find it flips differently.

24:33Ben:You might find that the process becomes really deeply human, like what Anthropic is doing. I think that that could be another direction it goes as well. It would be interesting to see kind of how it shapes up. But I think the other really interesting part of this too that stood out to me was as the engineering interview process gets less technical and more communication collaboration focused, other interviews around them are becoming more technical because we're all meeting in this new middle. and so there's this interesting happening interesting thing happening with like you know you're applying for a pm a pm role right in the days of the past like in no world were you probably going to be asked to like do some sort of technical assessment or put together a technical project you might be tasked to put together like a product launch or a pitch or a positioning thing but you probably were not tasked as much with doing something technical but now that's an expectation for most PMM roles.

25:27Ben:And so you're going to get quizzed and trained on like how fluent are you? How well can you go from that positioning plan to having an actionable thing? How can you use AI to convert that into action? And so on the flip side, you're also going to see all these roles around engineering get increasingly more technical in how they're screened.

25:44Andrew:Yeah, almost like these roles outside of engineering are becoming more engineering led. I can think particularly like product managers. You know, that's a role where I think there's an increasing demand to have technical competency and the ability to use AI to build things end to end, you know, because you might be that product manager that is tasked with building prototypes of products, you know, before they've even had any development work on them.

26:12Ben:So yeah, I think that expectation is really, really high now. It's like if you're a PMM and you have an idea, it's almost the expectation that you can create some initial prototype. Going back to even what you were talking about with Zach earlier, because you were aligned on what you needed and it was faster to make a prototype to talk about than it was to have meeting cadence about it. That's the real unlock. And PMMs who go into these hiring process, engineers too, who go into this hiring process understanding that are going to stand out like headed over tails above everyone else. And so if you're applying for those roles, keep that in mind.

26:47Ben:I'm sure you're already thinking about how can you showcase your technical skills. So Ben, what's this last one about?

26:52Andrew:Yeah, our last story today is the last mile where enterprise AI dies. And this is an article from Andre Savin. It's a really great breakdown of a survey from McKinsey, where they surveyed 10 ,000 engineering or executives. And this survey revealed that while 88 % of organizations are deploying AI, less than 20 % are seeing significant bottom line impact from that. And 86 % of leaders feel unprepared to adopt AI in their day-to-day operations. So some pretty stark numbers there. And the core problem seems to be this quote unquote, last mile gap, where AI technical capabilities need to be integrated with actual business operations.

27:39Andrew:You know, companies might have hundreds of pilots and high adoption rates and all these experiments going on, but they're not seeing the downstream productivity gains, because most of those benefits have been trapped within individual workflows, rather than organizational improvements. And Savine argues in this article that organizations have over-invested in AI technologies while under-investing in the production layer. So that's things like governance, verification, redesigning their operations, as we've been talking about today a little bit. And McKinsey's recommendation is that companies should be spending about five times on people versus what they're spending on technology.

28:22Andrew:So spend five times more on people and processes than AI technology. And the result of poor last mile planning is that AI value is primarily being captured through layoffs rather than redeployment. You know, and I think we're seeing this with all these companies who are coming out and announcing layoffs. I mean, we have Atlassian, we have AWS, we have WiseTech Global cutting 30 % of their workflows block. The list kind of goes on. A lot of these companies that are explicitly tying reduction in forces to their AI transformation. And it kind of seems like this pattern is going to continue in the short term.

29:00Andrew:And we've been covering this a lot here at DI, this awkward transition from AI experimentation to AI impact. And it is a very difficult chasm for many organizations to cross. And even for more nimble organizations, I think the success of doing this has been pretty mixed. Sometimes experiments turn into something that has bottom line impacts, but other times the productivity gains are isolated to a really small component of a larger system. And I'm definitely sensing this universal pressure to do more with AI, but not everyone's stopping to evaluate whether that work is actually translating into business gains.

29:40Andrew:And the thing that I just keep coming back to time and time again mentally is this statement that came out of last year's Dora report. And that is that upstream productivity gains are being lost to downstream chaos. And that describes that last mile problem. And I think the unfortunate reality, as this article argues, is that it's leading to more layoffs rather than reinvestment into the organization. So, Andrew, what did you think about this article?

30:09Ben:So it's a really great summary of what I think is going on here. And, you know, I do think that it's an interesting situation where companies find themselves wanting to rapidly buy and adopt these tools. And then they struggle with a workforce that isn't quite skilled up on how to use them. They're still in these growing pains of how, you know, the new normal, new level operation is going to be. And in all of that thrash, that chaos, as the Dora report calls it, you know, I don't think any of us have a really clear grasp yet. on what that virtuous cycle of employee and AI automation all working together and shipping things.

30:44Ben:Like there are companies that are AI native, you know, with one employee and they have like a whole bunch of agents or something that are exceptions to this. They kind of have that thing figured out. They're the companies of the future. But for companies that are inheriting this, that already preexisted with a lot of already enterprise, you know, applications and process and people, it's a huge challenge to come in and like unfossilize that is kind of what it feels like and then start to move around the pieces. Enterprise's strengths is that they've taken a bunch of things over time. They've cemented them together into this incredibly robust foundation that has lots of layers of management and process and tooling and skills.

31:28Ben:And by like turning it to stone and making it as hard and as big and broad and as great as possible, that's what becomes the really sturdy platform that other companies and themselves could build on top of. And so that, you know, being that strongly aligned and being that rigid was their strength. But in today's market, moving really fast, being nimble and resourceful is what is being rewarded. And enterprise companies want to be part of those gains. They have so much to bring to those environments, but they kind of lack that celerity. They don't have the ability to move as quickly or as fast.

32:05Ben:And so unfortunately, you get in a situation where like the employees in their minds, they feel stuck. And so the option is either I can like rapidly upskill them or I can restructure my headcount and bring in new people who are thinking this way or get rid of folks that I don't think that we need their role anymore. And so you get these massive disruptions and enterprises that don't happen other places. And that thrash burns trust. It causes burnout. And it really creates this death spiral of, oh, we're not going to use AI well, or I don't want to be part of this. That is, I think, plaguing a lot of these big companies right now.

32:45Ben:And you just see it in headlines like this week after week. But ultimately, it comes down to employees not perceiving the same benefits from AI that their employers are. It's a cognitive disconnect between what employers see as ultimately numbers on a spreadsheet that they're optimizing for, which is their job. That's like their stakeholders, they're reporting to a board, they have some fiduciary responsibility. They kind of are in this very like high level position where they're so far removed from the problems that they lack the language and the ability to break through and really kind of convey that with employees.

33:22Ben:And then on the flip side, employees feel like a number in that spreadsheet and they don't feel like they're supported in their upskilling but more importantly uh and i've felt this before as well it's oftentimes like no good deed goes unpunished the better and the more efficient and the more elaborate you can get with your ai workflows the more will get asked of you to do more of that to teach others how to do it and then that becomes a really big burden for your AI-enabled folks within your org. They burn out. They don't have the resources. They need to upskill everybody. And it creates also like an us versus them dichotomy within your own employees because you have folks that are getting lots of gains from AI.

34:06Ben:Maybe their process or the job they do can just be really largely done by AI. But maybe you have folks that are like maybe like in design and that's not their world. Their tools are still very immature. And so no matter how good and how AI forward that designer is, they're not going to be able to match maybe some of the level that the engineer can get, right? So that disparity as a baseline is really discouraging for a lot of folks. And I think that's the biggest challenge that we have internally is creating an environment where you're not making a pressure cooker, but rather you're having open, honest conversations.

34:43Ben:And when you are finding things that work, you don't don't go on it right like if you find if you find that little flickering candle within your org don't snuff it out like there's a way to um do this well and listen with your employees and that starts by building trust and having open conversations yeah you know a lot of organizations

35:02Andrew:hear about these like fabled product managers you know getting back to our our conversation on the interviews and how they're changing everyone wants that fabled product manager that is like the 10x person that can do it all they can go end to end with with coming up with the idea for a new capability implementing it with ai building all the assets to enable your organization and and go market it to the to the to the outside world um they want those like superhuman people that are just operating on top of ai agents that can build all these things that used to take a team to build. But the challenge is if your organization is not set up to facilitate a single individual doing all of that work, you are just creating, you know, maybe some people are savvy enough, they can navigate all the enterprise complexities and find a way to operate that within that.

35:53Andrew:But your typical worker probably can't, they're probably struggling to understand how to translate those individual improvements into things that the rest of the company benefits from. So yeah, Yeah, we've been covering this a lot. You know, we talked about the messy middle of AI development last week, and we've been talking about token maxing leaderboards. This is a common thread that I think we're going to keep touching on in the coming weeks. So, Andrew, what are your agents up to this week?

36:22Ben:Well, my agents, I've been on site at AI Council. So there are a few really cool data science booths that were here. In particular, I've been learning from them from their different handouts. What I love to do when I go to these events now is to, like, walk through the expo floor and do like spoken word to my agents. I'll be like everything I see, like I'll let them know and I'll grab some like brochures. I'll take pictures for them. I'll do a little like expo hall exploration and then my agents will go and do some research and be like, oh yeah, this might be relevant to you or this looks cool.

36:50Ben:But the tools here are really amazing. In particular, like I really loved like all the DuckDB stuff here, Hex as well. You got Databricks. This is some really big, really smart data companies that are leading the charge. And so I've been having a really great time learning from them and passing that on to my agents while I've been here on site at AI Council.

37:09Andrew:What about you? Yeah, speaking of passing things on to my agents. So, you know, we covered a few weeks ago, the Carpathie method of using agents to ingest raw sources and build its own wiki goal to have it to help your AI be informed about the topics that you want to focus on in that moment. So, yeah, I've really been embracing, you know, letting my agents build content maps around like token maxing, for example, just so I can get a picture of who who's saying what about it out in the space, what companies are doing certain things. And it's been really cool. Yeah, I've I've blown a few of my Claude Max sessions completely on just having to do deep dives into some of the stuff that's in my my data sources.

37:58Andrew:But yeah, it's pretty cool because I'm using Obsidian for this, as I think a lot of people are today. And it's really neat to see how I can use AI to sort of take all this disparate information and structure it in a way that helps me take action on it a lot more quickly. And I'm using it for content, for understanding the projects I have going on, for helping me navigate work with people on my team. It's actually a really cool method. And I do want to point out that, you know, when we covered this on the show, it was I think it was just a tweet that sort of went viral. But now he has a whole markdown file that explains how it explains to your A.I.

38:37Andrew:how to to do this system. So it's been a pretty fun journey this last week.

38:42Ben:Yeah. If you all are listening, if you have not experimented with keeping your knowledge graph or your own knowledge system with A.I., it's a huge, huge unlock. If you're in a situation where you don't keep logs and journals of your thought process and what interests you and you don't accumulate those things over time, you're going to have a hard time hill climbing in the future. Right now, we're all experimenting and throwing things at the wall, but eventually, there will be a providence that needs to be established. And those that have spent this time cultivating and building up this corpus of this is who I am, this is what matters to me, or this is what's interesting to me, even in a one-off way.

39:18Ben:This creates a knowledge graph that kind of takes what's interesting to you and captures it in a navigable format for an AI to operate on. This becomes your context fluency layer between you and your AI. And you need one of these in the future. Maybe you don't need one now, but you should always be investing in your future. So this is a good shout out, and it's great to hear that you're building that.

39:41Andrew:Yeah. Sweet. All right. Well, that's the Friday Deploy presented by Linear B. Make sure you give us a like on whatever platform you're listening to us on or watching us on. Thumbs up, subscribe, all the great things. It really does help this show out and help us get more attention on these challenges that we're bringing to the industry. So thanks for joining us this week. We'll see you next. See you next time.

40:12Andrew:AI is everywhere in software engineering, but most teams still can't prove its impact. That's where the APEX framework comes in. APEX is a new operating model for engineering productivity, designed to measure AI where it actually matters, at the pull request level. It connects AI activity to delivery outcomes, not just tool usage. APEX is built on four pillars, with AI leverage, predictability, efficiency, and developer experience. Apex helps you increase throughput without sacrificing delivery confidence or burning out your team. Because speed without predictability creates chaos and faster coding often shifts bottlenecks downstream.

40:49Andrew:If you want to operationalize AI the right way, Linear D and Apex gives you the system and the cadence to do it. Download the guide and start measuring what matters.