Why engineering leadership matters more than ever | Manoj Mohan

16 Dec 2025 · 49 min

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Dev Interrupted Podcast Episode Summary

Episode Title

Why Engineering Leadership Matters More Than Ever | Manoj Mohan

Episode Overview In this episode of Dev Interrupted, hosts Andrew Zigler, Ben Lloyd Pearson, and guest Manoj Mohan explore the evolving role of engineering leadership in the context of AI advancements. Manoj argues that while AI may change how code is generated, it will not replace the need for skilled engineering leaders who can provide critical judgment and strategic oversight.

---

Key Themes and Discussions

The Role of AI in Engineering

  • AI as an Intern, Not a Replacement: Manoj posits that AI should be viewed as a tool to assist engineers rather than replace them. The focus should be on how AI can take over mundane tasks, allowing engineers to focus on more meaningful work.
  • Historical Perspective: Drawing parallels with the Industrial Revolution, Manoj suggests that technological advancements often create more opportunities rather than eliminate jobs. AI has the potential to generate new roles and industries similar to past technological shifts.

The Importance of Engineering Leadership

  • Critical Decision Making: As AI takes on routine tasks, the demand for high-level judgment and creative problem-solving by engineering leaders will increase.
  • Framework for Success: Manoj emphasizes the necessity for leaders to start with understanding customer pain points rather than jumping straight to models and AI solutions.

Framework for Managing AI in Enterprises

  • The "3GF" Framework:
  • Ground: Ensure AI recommendations are based on credible sources and provide transparency.
  • Guard: Implement privacy and security measures to protect sensitive data.
  • Govern: Establish evaluation metrics and ongoing assessments to maintain AI trustworthiness and reliability.

Skills and Practices for Engineering Leaders

  • Continuous Learning: Manoj advocates for a culture of incremental learning and development among engineers, emphasizing the importance of staying updated in a rapidly evolving technological landscape.
  • Identifying Productivity Risks: Leaders should assess where their teams spend excessive time on low-value tasks and explore automation solutions to enhance productivity.

Importance of Naming and Tool Naming Conventions

  • Cognitive Load on Developers: The episode discusses the challenges developers face with obscure tool names, which can create confusion and cognitive fatigue. The need for clearer naming conventions is emphasized to facilitate easier collaboration and understanding.

---

Industry News Highlights

  • Federal AI Regulations: An executive order establishing a national regulatory framework for AI aims to unify regulations across states.
  • Linux Foundation Initiatives: New projects aimed at promoting collaboration and standardization in AI development have been launched.
  • Critique of "Scale is All You Need": Industry experts are reevaluating the effectiveness of simply scaling AI models, noting that true advancements require more nuanced approaches.

---

Key Takeaways

  • AI is a Catalyst for Change: While AI will alter job roles, it will also create new opportunities for engineers to innovate and lead.
  • Leadership is Crucial: Effective engineering leadership is vital for navigating the challenges posed by AI and for leveraging it effectively within organizations.
  • Emphasize Problem-Centric Approaches: Solutions should stem from a thorough understanding of customer needs rather than a preoccupation with technology itself.

---

Guest Information

  • Manoj Mohan: Experienced engineering executive with a focus on AI, productivity, and the future of engineering roles. He shares insights through his [LinkedIn](https://www.linkedin.com/in/manojmohan/) and [Substack](https://thesystemsmindset.substack.com/).

---

Episode Conclusion The conversation wraps up with a reiteration of the importance of engineering leadership in an AI-dominated landscape. The hosts encourage listeners to embrace learning and collaboration as they navigate the future of technology.

---

This summary encapsulates the core discussions from the episode while offering insights into the evolving landscape of engineering leadership amid advancements in AI technology.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:05Welcome to Dev Interrupted. I'm your host, Andrew Ziegler. And I'm your host, Ben Lloyd Pearson. This week, I'm sitting down with the seasoned executive, Manoj Mahan, to discuss why AI should be viewed as an intern rather than a replacement for engineers. He shares his optimistic view of this new industrial revolution and breaks down his framework for deploying responsible AI at scale. First, we have some great news stories to cover today, starting with federal AI regulations, AI agents at the Linux Foundation, why scale is all you need is dead, and why whimsical tool names confuse everyone. Andrew, let's just start at the top with these new AI regulations.

0:44Yeah, so the United States president signed an executive order establishing a single national regulatory framework for artificial intelligence. And the goal of this is to override state-level regulations that can cause patchwork interruptions for companies that are expanding their AI domain into those areas. We're talking about creating an AI litigation task force to challenge state AI laws that go in against this executive order and doing so by withholding billions of dollars in broadband program funding. Of course, this type of broad action would require legislative action, but this is the work of many large tech companies as well to ensure that their road to an AI rollout is smooth and paved and not going to be stopped at every state along the way in terms of legislations and roll out confusion.

1:32So this is a really interesting order from the White House in regards to how the power makers that may be are shaping AI legislation within our country and enabling all of us to work with this tool. But Ben, what do you think about this type of broad sweeping executive order? Yeah, well, I've been expecting something like this to come along for a while now. We're kind of in a Wild West phase with AI right now. That reminds me a lot of the early days of the web. And over time, the web centralized around or consolidated around centralized providers and had more regulatory frameworks that were implemented on it.

2:13So I think it's totally reasonable to expect that AI will go through a similar process. And I also think that it makes a lot of sense to regulate this at the national level rather than having like a patchwork of state laws. You know, if you think about it, AI is trained on data that is created by people all over the world. The engineers who build AI tools live all over the world. The data centers that train the models and provide them as a service are also located all over the world. So when you're in this world where all the aspects of what goes into providing AI as a product or a platform is distributed across many geographies, it makes perfect sense that the federal government would regulate it under something like the Interstate Commerce Clause.

2:54But with that said, it seems like so far the approach of the executive branch of the US federal government is to lessen restrictions on AI rather than actually creating some sort of regulatory framework. I'm kind of personally not really a big fan of this, like calling the global competition some sort of race to something, because a race implies that there's some sort of end state that we're trying to reach for, like getting to the moon first, for example. And I don't really think that's like how AI is developing. You know, it's more like an ongoing progression that continuously improves at an incremental rate and just gets better and better and more robust.

3:31And, you know, my opinion is that we should really be more focused on raising the tide so that all boats are lifted. You know, that would be things like, you know, publicly available training sets. Like what if everyone in the world could build powerful AI models rather than having being forced to have centralized providers because they're the only ones that have access to the data that makes it possible. And I know we have lots of non-American listeners out there, but I think it's really important to cover because a lot of the legal frameworks that get established for things like technology in the US often get adopted in parts of the other world too.

4:06And it's kind of challenging to cover American politics right now because things change so dramatically week to week. But this is certainly something that we'll be keeping our eyes on as it develops because AI regulation will impact everyone that uses it, including software engineering teams. Indeed. All right. So let's talk about what's happening over the Linux Foundation, Andrew. Yeah. So the Linux Foundation, they've launched the Agentic AI Foundation or the AIF, was founding project contributions from Anthropic, Block, and OpenAI. And these are foundational AI projects, things like Model Context Protocol, Goose and Agents.md that are establishing the basis and the open source frameworks by which many developers and tools are adopting and advancing with these types of agentic coding frameworks and things that are available to us.

4:56And this foundation is also backed by Amazon, Google, Microsoft, Cloudflare, Bloomberg, you name it, a big player in the AI space. They probably have something to do with the new AAIF because this is all hands on deck to stabilize and secure and lay out a bright future for the frameworks and the protocols that will power AI and make it capable of scaling. This is the same exact type of coming together that we see earlier in like the Web 2.0 era and the cloud era of coming together to provide best-in-class standards that are open source for teams to build and roll out their computing systems on.

5:36The same thing is now happening with our AI tools. This is a pretty interesting announcement, especially, you know, I've worked a lot with all of these tools listed here. So to see them all come together into one cohort makes me really excited about what's next for them. Yeah, yeah. I love the Linux Foundation. Really cool to see a lot of companies that are, they kind of feel like a bunch of friends of the show, right, Andrew? Like, we're very familiar with a lot of these companies. But there's a lot of really cool projects that are being added to the Linux Foundation as a part of this. Like, Andrew, I know you're a huge fan of Goose.

6:07Biggest fan of Goose. You're right that these are all friends of the show. And so it makes me happy to see this kind of success for them. Yeah, yeah. I also really like what I see with agents.md, which is a project from OpenAI to provide guidance to AI agents on how to interact with repositories. And this one stood out to me because, you know, my personal approach to building AI workflows has been less focused on the prompts that I give to my models. You know, we've all heard this phrase prompt engineering. I actually think that's the wrong focus area. like context is the most important thing.

6:39So I think in a, yeah, I think in a, in a perfect world, your prompts are actually very simple. Like even as simple as like just a few sentences, but they should enable your agents to access all of the appropriate context. They need to make the right decisions. So, you know, the, the organizations that are being successful right now, like they're building a corpus that provides that context to their agents. And I think everyone should be focused on this. So it's, it's really cool to see the Linux foundation playing a role because this is how our agents are going to get more capable of responding to a wide range of situations.

7:12I'm definitely going to be keeping an eye on what's happening with this. And, you know, Andrew, who knows? Maybe we'll even try to attend the MCP Dev Summit next April. You know, that sounds like a pretty cool event. But also, yes, I definitely want to keep bumping into all things MCP and Goose and AgentsMD, of course, out in the wilds. I think you're spot on about the importance of context engineering and really what you have to come to bear with every AI project that you work on and bringing in your best practices. And that's what these tools, these frameworks are all about, establishing a really repeatable, successful framework for you to get the most out of your tools.

7:47So really exciting news, excited to follow this one further. And jumping into our next article here, we're talking about scale is all you need is dead. Yes, the movement behind this idea that you can just keep going and going and going on model size, compute and training data to just get bigger, better, smarter models. Well, this theory is really starting to not hold water for many leading AI scientists. In fact, at the recent Neuro-IPS 2025 conference, which is the largest AI conference in the world, many scientists are publicly acknowledging that scaling large language models is insufficient for achieving what we call generalized intelligence.

8:29And that challenges the previous dominant scale is all you need approach. And so you have industry studies from research at MIT, McKinsey, and BCG, also reporting that, you know, over 90 % of companies aren't realizing significant ROI improvements from their investments in generative AI. And maybe that's just yet, but maybe that will be forever. These are all questions that hang over everyone right now, I think, in using and adopting these tools, especially as consumptions and costs go up. And this really dramatically changes the drumbeat by which AI companies that are scaling, especially foundational models, can argue that, you know, getting more resources, more time, more compute, more power, more training data is ultimately going to yield a bigger, better product.

9:13In fact, many have called out that a lot of the work now is upon us and all of the stages in between. I would fine tuning is securing reducing the size and costs of the models. There's a lot of work to be done. So, Ben, what do you think about this article from Gary Marcus? Yeah, well, first of all, Mr. Marcus, you know, he lamented how difficult it can be being a contrarian on issues about this, you know, and I get it. Like, you see. But I'm sure you could relate with that, Ben, you know. Yeah, what's that saying? No, actually, I am. I am happy to call myself a contrarian. But, you know, the issue is you see all these glaring issues with society around you.

9:52And most people don't agree with you. And then by the time everyone catches up to your perspective, you've already moved on to the next challenge that you're contrary about. But what I really like, you referenced it, but I really like the part that references how many or why many AI initiatives fail. And I think it really boils down to this. Like if your AI initiative is focused on AI adoption rather than solving some sort of discrete problem, it's far less likely to be successful. In fact, the overwhelming majority of those types of initiatives fail. And it really, this article really reiterates a theme we've been hitting on constantly here at Dev Interrupted.

10:29You know, I'm looking forward into 2026. And I think that's the year that this incremental improvement from the frontier models begins to exponentially shrink. And at the end of the day, like when you really think about what an LLM does, they're really just like a facsimile of certain aspects of human intelligence. Specifically, our ability to combine words into a coherent whole. I believe that they're incredibly powerful, but I think we're officially reaching the state where most of the gains at this point are going to come from the application of existing technology rather than further incremental improvements.

11:04I think this race to have the biggest, most powerful model, like that's where the real bubble is happening right now. You know, and I and personally, like just as I've invested more time and effort into using and learning about LLMs and GPTs, I have found myself believing less and less that AGI will result from this technology. Like it really does feel like the gap gets wider the more you understand it. So great article. I really enjoyed reading this one. Yeah, and I like how you called out that the, you know, in 2026 being the year that the improvements for the models, they make them shrink, they make them smaller, more efficient.

11:40I think it even says something that we talked about this article instead of GPT 5.2. You know, it speaks to what we're all focused on right now. It's not yet another foundational model rollout and the transformative numbers on the benchmarks. We're getting to really, you know, a point where we're barely eking out changes on that type of leaderboard. and there's bigger things to focus on. It's a really great summary. Interested to see where this one goes. But I want to wrap things up jumping into our last article of the day. This is a really fun one about, you know, developers, software engineers losing the plot when it comes to naming their tools.

12:17And I read this article and instantly related with it in all of the good and bad ways. I saw myself in this article and I did not like it. But then I emerged from the other side being like there is a better way. So this article, it highlights a trend, you know, if you're a dev, you might be familiar with having very wacky or obscure tool names, library names that become kind of abstract and unrelated to like what you're coding. And the author of this article, he cites examples like Viper for configuration management, Cobra for CLI, or Melody for WebSocket connections. Like maybe the person who named it that, they had an idea or a clear analogy, but it doesn't translate when others pick it up and read it.

12:59And noting that these names that they have, they don't have a descriptive connection to their actual purpose. And this actually introduces a lot of cognitive fatigue to developers because now, you know, you're going through your project dependencies and you just have this big long list of random nouns and you're like, what is CASBEN or async with a K? Like you don't understand what they relate to. And this cognitive cost, you know, it can definitely cause issues with actually sharing and using a project, making sure you're using the right dependencies. I loved that ultimately this really called out that the naming schemes, they have to be a little more formulaic.

13:41You need to have a system. And this is something that I've kind of evolved with as well, because before, like my own project naming system, I would want to find that clever name. If you think I'm lying, go look at my GitHub profile and you will laugh at some of the repo names that I do have on there. And sometimes that paralysis would reach a point where like, I wouldn't even start writing code or start the Git repo until I had come up with that clever name. There's like a whole new level of paralysis, I think. So, Ben, what did you think about this article and all of the wacky names that we're giving our Git projects these days?

14:17Well, I have to think of my own personal example of wacky project names where a developer came up with a just a crazy name for an open source project that ended up getting our company associated with Ritual Sacrifice. by all the crazy internet conspiracy theories. So I definitely resonate with poor naming decisions being a problem from time to time. But I feel like this article is really particularly relevant in the AI era, even though it's nothing to do with AI. But, you know, think about it. Like we're all out here vibe coding now. Sometimes I don't really even know if a project that I've started with AI is ever going to have legs.

14:55So naming it is kind of like a wasted effort. Like I don't want to give up a good name on something that may not be used for very long. You know, I think sometimes it's like best to just keep the name like extremely simple and obvious to what you're trying to do with it. Oh my gosh, you went like the total opposite direction. You went like with like your new coding projects. It's like you're a pioneer back in the days. It's like my baby has to make it through the winter. It has to make it to its first birthday before we give it my grandfather's name. Like I love that that's like where you're at with these projects because that's how quickly they come up.

15:29and how you use them. But, you know, I think that there's like so many ways you're going to take it and apply it. If you get a systematic way, it actually just totally dissolves as a problem and it actually starts helping you. So now I'm like, I like the name stuff in a domain namespace kind of way. So like, let's say I have a bunch of projects for like agents, right? You might have your marketing agent, your copy agent. And instead of naming them that way, I would be like agent hyphen marketing, agent hyphen copy. That way, all of the agents live next to each other in the folder. And this is nice for me for scanning and scrolling.

16:02Makes collecting up my projects really easy. Makes naming them even easier. But the extra unlock here is that now they're nice and semantically organized for an LLM to understand too. So now when an LLM can pull up my list of projects, it knows all of the similar namespace things. Things that are working on the same kind of project or maybe employ the same type of framework. They live side by side in the folder. That actually is a huge unlock. So definitely always be thinking about not only is this confusing a developer, but is this confusing my LLM? Yeah. And, you know, and this reminds me of an episode we had quite a while back, actually, with Luis Alejandro Vega from Bloomberg LP about creating a culture of sharing internal developer tools.

16:45So as a part of their workflows, they help developers name and brand projects that they want to create and share with the rest of the company. And they come up with some really clever names. Like, I think my favorite one that I remember, they called it Logger, which does exactly what it sounds like. It pushes stuff to logs. And they had this fun little lumberjack mascot for it. So, like, you know, you can still have a very standard name, but have fun with it as well if that's what you want. So if you haven't listened to that episode, I definitely encourage you to go back and check it out. What a fun roundup.

17:18All I'll say at the end here is save the fun names for, like, your devices. You know, just ask my producer what all of my headphones are named after. That's where I put all the wacky fun stuff these days. But this was a great article. If you have a fun naming scheme for your projects or if you feel particular at this, I want to hear about it. I'm like a project organization nerd, so don't be shy. And now that we've gone through our news roundup, stick around because after the break is my conversation with Minoj. Engineering leaders are doubling down on AI, but how do you actually measure its impact?

17:51With Linear B's new co-pilot and cursor dashboards, you finally can. Linear B brings all of your AI coding assistant metrics into one place. Adoption, engagement, suggestions, acceptance rates, and connects them directly to delivery outcomes. See how deeply AI is integrated into your SDLC, quantify productivity gains, and understand where trust is growing. You'll turn AI data into real engineering insights. Visit Linear B to try it free and start measuring what matters. Today we're sitting down with Manoj Mohan, a seasoned engineering executive with experience leading large-scale software data and AI transformations at companies like Intuit, Meta, and Apple.

18:39And Manoj is passionate about empowering engineers and simplifying complex infrastructure, while also maintaining a focus on responsible AI and developer experience. And he's wildly optimistic about what the future of engineering holds. And we're really excited to sit down with him today. So Manoj, welcome to the show. Thank you, Andrew. Thank you for having me. And thank you for that warm introduction. Look forward to our discussion today. Yeah, likewise. So, you know, we're here at the Engineering Leadership Conference, And you're here on a panel today talking about AI is here to stay. Can you tell us a little bit more about that?

19:14Yeah, yeah. First of all, super excited to be here at the ELC annual conference. It's super energizing to be here and be part of the Linear B team and everything. So the conversation we are having in the panel discussion today is how are engineering organizations going to evolve in a post-AI era? Yeah. So that's the panel discussion we are having. I know it's kind of top of the mind concern for engineering community and things like that. So that's the conversation. And I'm happy to spread some light if you want me to. Yeah, let's park there for a minute and learn some more about that. So AI is here to stay.

19:53We talk about that a lot on the show. And it opens up a lot of opportunities for teams. And it shifts everyone into a new paradigm, a new way of thinking. So what challenges and opportunities do you see right now for engineering leaders? Yeah. So let me start with the forest view, right? So the common misconception that I hear from several folks is as AI is starting to generate more and more of the code, engineering leadership and engineers are becoming less relevant, right? That seems to be the resonating theme that's concerning to most folks. And in my opinion, I think engineering leadership is more critical than ever before as of today.

20:39And why do I say that? The reason for that is the onus of making a right judgment call, the onus of creativity, the onus of investing in right big bets based on criticality, all of this has to come from engineering, the rockstar engineers and the rockstar engineering leadership, right? And maybe an analogy that I can state here would be back in the 1800s when industrial revolution was happening. Everybody was freaking out that these big machines are going to take away all the jobs, are going to eliminate all the jobs. But if you look back at history, what happened is just the opposite of it. These machines actually spun up so many new industries, be it aircrafts, be it transportation, be it railways, that in turn created millions and millions of new jobs, right?

21:37So every transformation that we have seen historically always looks like it's an end or it's a doom. But if you give it its own test of time, it's going to translate into newer opportunities, the ability to do more meaningful work if you just let it carve out its path. So that's how I see it. I, you know, I sense a lot of optimism with how AI is taking over all of the mundane work. Yeah. Right. And one other analogy I could use is I see AI having the ability to generate code more along the lines of intern, having an intern in an organization. Right. Now, just because an engineering organization is constrained on capacity and say, if you feed in 100 new interns to that engineering organization, do you think it's going to magically solve all of those capacity constraints?

22:37Absolutely not. That's not going to happen. If anything were to happen, it is going to choke the organization even more. It's going to create collaboration bottlenecks. It's going to create innovation bottlenecks. It's going to create dependency bottlenecks and so on. Right. So I don't think the solution is that AI writing code is going to eliminate engineers. What is going to happen is it is AI is doing all of the mundane work and it's going to free up engineers to be able to do more meaningful work. Right. It gives them more space to innovate, to be creative, and to go after bigger, bolder bets so that we do more meaningful work in the longer run.

23:24So that's the optimism that I would love for everybody to think and thrive upon. Yeah. I like how you use the analogy of being back in the industrial revolution and factories being introduced. There's been a lot of transformative things that have transformed industries that we've talked a little bit about before and Dev Interrupted. That's a great example. Another one is like when the Macintosh came out and traditional artists kind of uprosed against the new computer-aided graphics. And the same thing for architects with doing drafting on a computer versus doing it traditionally on like a drafting table in a room full of architects.

24:00So in each of those changes, like you said, it opened up a new world. People had to step through a threshold. it's a bit of an uncertain time of what's going to be on the other side of the store but ultimately it's more jobs and more opportunities and more ways for humans to have a higher impact on the work that they do and it reminds me a bit of like earlier this year when everyone for the first time googled jevon's paradox to learn about remember when deep sea came out and the low inference cost caused the open ai to dunk down for a day because everyone thought oh if ai is so cheap that you can have it in all these different ways.

24:37And it's like, why does a company like OpenAI have its moat? But then what everyone realized is that when AI access became more available, it increased the demand for it even more. And so what happened is the consumption then went up even higher instead of then, oh, the costs that you or the benefit or the profit you gained from that going down. And that Jevons paradox applies also to engineers and engineering jobs. When everyone is an engineer, when everyone is building software, when it's at everyone's fingertips and grandma can vibe code, the need for engineering knowledge and engineering expertise is higher than ever before.

25:17Absolutely. And now all sorts of industries and companies and places where maybe before they had, maybe they had a technologist in-house. They're not a tech company. They had a technologist though, but now everyone within that company is a technologist in their own right. And that's like a really exciting time, but it's also a time of a lot of work for people like you and I, you know, to talk about it here at Devinterrupted, but also to take the stuff that we've learned, our foundational knowledge as engineers, and help train the world for this new era that we're in. Yeah. Yeah, absolutely. Very well said.

25:49You know, and every such transformation, there is a sense of skepticism. And for folks who don't understand the granular details that are happening, that skepticism just explodes into paranoia and things like that. Right. You know, the only way to approach any such skepticism is to drive more and more clarity, right? Clarity of, you know, of inculcating some knowledge within yourself and also trying to, you know, break down the barrier and, you know, create some clarity with, you know, with what is at stake and things like that. Yeah. So if you're an engineering leader right now, what can you do to build that clarity within your team to get everyone aligned?

Read the full transcript

26:31Great question. So I would suggest a couple of different things. And some of these things have worked in the companies I've worked with in the past. So one, if you were to carve out all of your productivity or contributions in a typical week or 40 hours of work, you try and identify where you end up losing the maximum amount of time, but that is the least on productivity. It could be something like sending out meeting notes after meetings might be taxing for engineers and engineering leaders. It could be putting together, documenting a design or vetting out a design architecture. Or it could be being on incident calls when you have an issue that's impacting multiple customers in production.

27:21Yeah. So you look at what are those time consuming activities in decreasing order of where your value add is minimal. So identify the toil. Yeah. Identify the tail or identify where you could amplify your productivity. Yeah. And then look at how do you kind of solve for it either through a process or through automation or with AI or with agentic workflows or whatever. Right. I wouldn't prescribe AI as the only means to improve your productivity. It is definitely one of the most significant means because it enables better automation. Yeah. So looking through along those lines has helped immensely.

28:03One example to quote is, you know, my team of engineers used to be stressed out, like, and this used to come out from the top of the mind conversations. They used to be stressed out about the on-call process. and with the on-call process, they would have to go through the stress and the burn and the churn of trying to figure out where is the root cause of the issue. And this would span sometimes, two hours, sometimes three hours, because our mean time to detect would vary from 90 minutes to 180 minutes and so on. So that used to be a very stressful thing for the engineers. So what we did is we said we are going to try and solve for it by investing in an experimental troubleshooting bot.

28:51And that troubleshooting bot will help identify the initial few nuggets that will give a starter for the engineer. So earlier, the engineer is already on the stress of being on a customer incident call. But now with the troubleshooting bot, they come to the incident call, but they have nuggets of information that they can look into. Right. So that was a huge morale boost for engineers. And that was like a that was the big pain for them. The big pain point. Right. Because it's like starting out that incident investigation from nothing is really hard. You know, you're grabbing through some logs or trying to piece together some information that you have.

29:33And so starting with a best possible first guess from an assistant is a great way of reducing the toil and stress. And we obviously we didn't get it right on first attempt or first iteration. And our strike rate of success of what it recommended was low to begin with. I think it was 12 percent or something. Yeah. But iterating through and learning, giving it feedback, we were able to notch it up several, you know, several points higher. and now it's kind of become a default standard for every engineer on the instant cost. Yeah, it has like the same or maybe a little more accuracy than a human giving their first guess someone with domain knowledge.

30:12And I love that you pointed out that you had to iterate on it, right? You built the frame of the workflow. You got it saying stuff when something wrong would happen, but it didn't get all the pieces right. But as you used it more and more, you found, oh, it's kind of falling into this trap or, oh, it's making this wrong assumption. and then you can equip it with more and more knowledge. And in doing so, you kind of incrementally build this tool that then you get to keep around and it actually has a lot of long-term value. And I'm wondering, when you iterate through that process, how do you maintain the quality and make sure that you don't take steps backwards?

30:48Like, did you all use evals, for example? Yeah, yeah, I was about to say. Okay, amazing. Let's talk about evals. How would you use that? You know, I'll give a couple of flavors to it. So first and foremost, for anything that you leverage AI in an enterprise world, right? Enterprise world is kind of scary because there's compliance, there is legal, there is security. There's a scary word here, for sure. There's tons of things you've got to put together. But I call it as the three GF factors, like the three great factors. Anytime you think about AI for enterprise, I call it the 3G of approach. What it means is, one, whatever AI enablement you're doing, you got to ground it.

31:31That's the first G. You got to guard it and you got to govern it. So when I say, when I talk about grounded, grounded is you got to give the citation of what sources of truth did AI refer to in order to come up with that best recommendation. Yeah. Think of it as the system being transparent about what it is coming back with. Observability. Yeah, some sort of observability. And that also enables your users to gain confidence on, you know, okay, this is the reason why AI is giving me this particular answer, right? But the lack of that transparency will mean that you're going to lose the trust of users for every false recommendation.

32:16And then gaining that trust back is going to be a crazy, impossible journey. it, right? So that's the grounding part. The second part is guard it. So guard it is privacy by design, right? So you're going to have tons of information, be it sensitive data that you need to mask out. You might have information that should not make its way to the model and things like that. So you got to have those guard rails or guarding principles with a privacy first design approach. That's number two. And the third one is governance, right? And governance is where mostly evals comes into effect because, you know, everything that AI does in terms of coding capabilities, I treat it like an intern, right?

32:59So anytime you give work to an intern, like you're going to have to measure it as precisely, as accurately as possible, right? So you have to look at data drift metrics. You have to look at model drift metrics. You have to look at fairness metrics. You have to look at all sorts of critical metrics that will tell you or that will be the validation mechanism for you to say that, look, for this being the problem statement, for this being the outcome from AI, this is absolutely the right approach. Right. So the three G's I have explained, it is the grounded, guarded, govern it. And the three Fs are what are the outcomes.

33:41It is fast, faithful, and fair outcomes, right? So you take care of the three Gs and you get the three Fs. I love that. And that's exactly what you want. You want a fair outcome that is rationalized for your product use cases across the cohort of users. You want it to be fast, fast and accurate. And, you know, so along those lines. So that's how I look at it. So in the enterprise world, if you take care of the 3GF, I think you're better suited to increase the odds of adoption success with the outcomes. I love that framework. And, you know, you talked about the Fs and you talked about the Gs, but now I want to talk about the Ss.

34:25And so I want to talk about skills, the skills that are important to developers right now, to engineers. And I'm wondering, what are the things that you think that engineering leaders should be screening for and understanding from their teams as they build them in order to have teams that are really ready to grapple with everything AI can do? Yeah, great question. The principle that I do for myself and for everyone in my team is, you know, small incremental learnings done consistently day over day will lead you to more, you know, being more knowledgeable, being more empowered to leverage the latest and greatest tool.

35:06And the reason I state that along those lines is the pace at which the technology, AI, the stack is evolving is freezing fast, right? You think you're an expert on day one and you could be outdated by day 30. Yeah. So that's why I think hold dear to your principles, learn incrementally and apply the patterns, right? And leverage those patterns on, okay, what is a use case where I should leverage retrieval augmented generation, right? Where do I leverage vector data stores? Where should I leverage embedding? When do I invest in an in-house foundational model that's custom to your organization versus when can I leverage an open AI foundational model?

36:00So having a lay of the land, a high-level understanding of the forest view, and being able to map it out to the relevant use cases, the relevant pain points or problems and how you would tackle it, I think is a great start. and that is also too much to chew for all of us. The way I would approach it is consistently look in small pockets and try to kind of get a feel for it, get a sense for it, build a trust. And there are tons of opportunities. I'm not going to outline all the courses, Coursera, YouTube, Google. Yeah, there are plenty of them. You could lose track of it. So I'm not going to list it out.

36:39But in fact, in many cases, the best thing you could do is just get your hands in it and mess with it and do it more than just read about it. You know, it's rich. I love that you called out a daily practice, a daily incremental practice, because I also think that is the key to getting better at these tools is just do a little bit every day, but also then reflect on your usage and what you learned from it. And I also love to partner with the LLM to write artifacts of the process along the way, markdown documents of what we talked about, what we aligned on, what was the spec, what were the tasks, what was the framework?

37:10And then getting all of that as like something you check into Git, even. It's become a really important part of my daily process. So that daily practitioner view is really important. And one more nugget I would say is, every week or weekend, I look at meetups, right? I look at Luma. Yeah. What are people meeting about and talking about? There are at least two or three hackathons happening over weekends on weekdays. I did one last week. I know there are all the time now. And we did one here at ELC. Yeah. So there are tons of hackathons happening. And if you feel like you need that initial starter pitch, right, and you want to join forces with someone else, hackathons are a great place.

37:52Meetups are a great place. You should just expose yourself. And even as a beginner, you get that kickstart and then you could just, you know, drive it on your own with a bunch of agents running on your laptop all day. Yeah. Yeah, exactly. And so on that same question, Mendoz, are you vibe coding? I am vibe coding. Yeah. So what's that like for you? What's your process like? I take a couple of different approaches. First one is I try and get OpenAI and Gemini and Anthropic to write comprehensive prompts for a high level problem statement. Yeah. So I, you know, I trick it with, you know, a Lyra-based, you know, comprehensive prom specialist.

38:35I treat it like an expert. I give it a high-level problem statement. I go to Figma. So you do a big thought exercise first. Yeah, I do a big thought exercise. I kind of, you know, I know because I know I'm not, my thought process is not complete, but I know what that end outcome needs to look like. So I try to kind of paint that forest view, not at the 100 ,000 feet level at which I'm thinking. I try to break it down to 10 ,000 feet level. And all of these tools, like I take nuggets of good things that it comes up with in its comprehensive prompt, and I put together all of it collectively to build one of the best prompts possible.

39:13And then I run it on deep research mode, you know, to get the next level of breakdown. And then I modularize it. And then those modules I do with Cloud CLI. I like Cloud CLI. That's my favorite tool of the time. So I give it small incremental modules to develop. And, you know, and it tries to develop all of it. I consolidate all of that into a Git repo. And then I try deploying it on Vercel and things like that. So that's been my approach. And just rapidly building and trying it. Yeah, yeah. And again, I, you know, I don't think I'm doing enough justice because I'm most of the time I'm running one agent or one problem statement on my laptop.

39:55You feel lazy for not running a whole bunch of them in parallel. I should be running like 10 Claude agents and 10 Cursor agents and 10 Anthropic agents on 10 different projects in parallel because, you know, that's what that's the age we are living in. So I would love to get there. So I feel like I'm, you know, I'm lazy to do that yet. Well, I mean, in the world we're in and the kind of role that you're putting yourself in, in your process, you know, you are the top down person with the vision and ultimately you're using the AI agents as a team. Right. And so what it does is it moves all of the importance of the work that used to be before, like, oh, you know, get the POC down, get something up as fast as possible.

40:36Now, a lot of that work actually is understanding really clearly what you want and then being able to put it into words. And then using that incrementally, like you said, to find the best parts of the process. And then once you have it in a modular way, capture it and bring it in and then use that as a reference to create that next modular piece. And you are able to piece it together slowly over time. But all of that builds out of your intent, your ability to express what you want in natural language. And so has that shifted the kind of skills that you've been building and using as an engineer?

41:10Yeah. And part of it is also there's some history to it, which is I still treat the AI code gen capabilities like an intern. I don't think it has earned the trust to become a junior engineer. You got to review all the code and you need to understand what it does. So every time I have tried giving it a forest view macro multi-dependency level problem statement, it would falter big time. And once it starts to falter, if I go and tell it even as precisely as possible, hey, you know what? This dependency between this particular module and this module is not working. I know it's a bug. I want you to fix it.

41:55And I'll tell it three times, four times, five times. Fix it, fix it. It still wouldn't get it. So that's why I still, you know, hold on to my theory that I still treat it like an intern. I would love to see it graduate to more of a junior engineer someday. And then maybe I would give it more of a forest view problem statement and expect it to kind of go after solving for it. I'm curious to you, from your perspective, as an engineering leader, how do you take that practice and what you align all of your engineers to do in this AI engineering world? How do you convey the impact and the adoption of that back to, like, your non-technical leaders and the folks that, you know, they hear from the boardroom-level conversations?

42:37We need more AI adoption. We need more throughput. How do you balance those conversations? That way you don't burn out your devs, but you still take advantage of opportunities in the market. Yeah. So I have been somewhat fortunate to be associated with organizations where the leadership understands that AI is not a magic wand that can solve all the problems, you know, with a magic bullet. Right. So that's that's been my blessing because because that has avoided a lot of undue pressure. But I can see that pressure build up in several other enterprises. So the principles I try and abide by, even for startups that I advise and all that is, one, do not start with AI.

43:24Do not start with the model. Start with your pain point or product problem statement that you absolutely have to solve to create a mesmerizing experience for your customers. Yeah, start with the problem. So you are in the business of solving for your customers, creating a compelling value proposition. Now, what is that? And what are the North Star metrics that are truly going to move that experience for your customer? If you have clarity on that aspect of it, then break down that North Star metric into more granular quantifiable metrics and then look at which of those metrics are you going to solve for in a more meaningful manner.

44:13For example, let's say you have done all the funnel analysis around your product analytics. You know that a significant fraction of customers are dropping off in a particular portion of the workflow. Maybe that workflow is too hard for the customer to understand. Maybe the integration or the data glitches are preventing the customer from being able to get to the next step. Right. So is there value add with embedding a chatbot that understands the context of that particular step within the workflow and identifying the top three recurring reasons? Are you able to surface three default options for the user saying, hey, are you stuck on this particular step?

44:55Are you running into this particular error? If so, click here. And if you could create that magical experience, then you have automatically created a value proposition. Right. So so that's been my philosophy. Stay centered around your core pain point and then look at avenues with AI or without AI and how you can move the needle for your customers in a compelling way. So you find the problem that your customers have and you solve it and you use AI as a tool, as a catalyst to make that change and to solve that problem. Not just using AI for AI's sake. You know, when you have a hammer, everything looks like a nail is kind of the scenario we fall into.

45:37And I really like how you highlighted the user intent of what they are coming to you for. As a company, as a software provider, as an organization, you're solving something for somebody. And it's important to understand what that solution is and what it makes valuable to that person. That way you can build that better experience for them. But there's also something to be said about understanding intuitively their intent of what they're trying to accomplish when they come to you and they use your products. And before AI, it can be hard to understand intent. Like you said, like you look at the adoption or the usage of a tool or a workflow and you see at some point there's a big drop off and maybe you have a lot of theories and you run a lot of tests and you figure things out.

46:21But there could also that in this world be an opportunity for AI to be a part of understanding the user's intent and then using that to keep them embedded within the system, keep them from dropping off. We sat down with Lake Dye recently who put this beautifully about how intent was solved for with search, how the user searches for A, but they really mean C, and we serve them B, and then they click on D. But ultimately, D was what they wanted, but that was really far from what they said. So you had to really neatly understand their intent to step their way there. And I think AI is going to enable a lot of tools and software to have that same delightful experience that search has had for us.

47:07And I think that's really exciting. Yeah, yeah, absolutely. Absolutely. I think all of these tools, all of these are options we got to leverage, but the attempt should not be to leverage them for anything and everything that you're trying to do, right? That decision making, that ability to invest the right tool in the right area is where the engineer brings the best of value. AI is absolutely here to do all of your mundane code generation stuff. And it is essentially to free up your engineer to go after the bigger, bolder, more compelling, more meaningful work items so that we are able to serve our customers, our stakeholders, and improve the ecosystem for the better.

47:53Wow. I mean, this has been like an amazing deep dive on how you think about using this as a force multiplier, but also how you lead teams and are educating them. And everyone is upskilling right now. And I really like your optimist perspective on how teams are going to be utilizing these systems. And, you know, you're giving a panel today at ELC. And so there will be more to follow from you as well. But I'm curious for our listeners and those that are tuned in today for our conversation, where can they go to learn more about Minoge and what you're working on? Yeah, so I post a newsletter about all of the stuff that I'm fascinated with.

48:27So most of the stuff I deal with is AI and engineering leadership and productivity gains and hacks and all of it. So that newsletter on LinkedIn, I also have a Substack newsletter. Well, then us too. We'll plug you in with our community. We'll get those links in our show notes. That way folks can go and check that out. and then I just really want to thank you again for sitting down with us on the Dev Interrupted Dome. It's been really great chatting with you. Yeah, and thank you. Thank you. The pleasure is all mine and I've always been a big fan of your podcast and all the amazing guests and the topics you cover.

49:02So please continue the great work. Thank you so much. It's great to sit down with the community members. So thanks again and thanks for tuning in to Dev Interrupted.

49:18you

From the publisher

The common narrative suggests AI will make engineering leadership obsolete, but history - and the Industrial Revolution - suggests the opposite is true. Engineering executive Manoj Mohan joins the show live from ELC to argue that as code generation costs drop, the demand for high-level judgment and strategic oversight will only skyrocket. He breaks down why leaders must stop starting with models and start with customer pain points, utilizing his "3GF" framework to manage the risks

LinearB: Measure the impact of GitHub Copilot and Cursor

Follow the show:

Follow the hosts:

Follow today's guest(s):

OFFERS

  • Start Free Trial: Get started with LinearB's AI productivity platform for free.
  • Book a Demo: Learn how you can ship faster, improve DevEx, and lead with confidence in the AI era.

LEARN ABOUT LINEARB

  • AI Code Reviews: Automate reviews to catch bugs, security risks, and performance issues before they hit production.
  • AI & Productivity Insights: Go beyond DORA with AI-powered recommendations and dashboards to measure and improve performance.
  • AI-Powered Workflow Automations: Use AI-generated PR descriptions, smart routing, and other automations to reduce developer toil.
  • MCP Server: Interact with your engineering data using natural language to build custom reports and get answers on the fly.

More from Dev Interrupted

All 208 episodes
Why engineering leadership matters more than everDev Interrupted · 49 min
Listen in VO