How leaders win over their team’s biggest AI skeptics | Superhuman’s Loic Houssier

28 Oct 2025 · 46 min

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Dev Interrupted Podcast Episode Notes

Episode Title

How leaders win over their team’s biggest AI skeptics | Superhuman’s Loic Houssier

Episode Summary In this episode, hosts Andrew Zigler and Ben Lloyd Pearson interview Loic Houssier, Head of Engineering at Superhuman. They discuss strategies for fostering organic AI adoption in an environment typically resistant to new technologies. Loic shares insights on overcoming skepticism, leveraging respected team members, and measuring productivity gains through AI implementation.

---

Key Themes and Concepts

  1. Organic AI Adoption
  2. Skepticism in Engineering Teams: Many engineers, especially senior ones, are cynical about new technologies, viewing them as buzzwords rather than practical tools.
  3. Empowerment Over Mandates: Rather than enforcing top-down directives, Loic emphasizes the importance of using an internal champion (the Chief Architect) to demonstrate AI's value authentically.
  1. Strategies for Overcoming Cynicism
  2. Internal Champions: Using a highly respected engineer to validate AI tools helped shift the team's perception from skepticism to acceptance.
  3. Pragmatic Measurement: By blending qualitative feedback with quantitative data, teams can assess AI's impact more effectively.
  1. Unexpected Productivity Gains
  2. Faster Ramp-Up Times: New engineers can get up to speed quickly with AI tools that simplify the onboarding process.
  3. Creating Internal Tools: Teams are able to build tools rapidly using AI, saving significant time compared to traditional methods.
  1. AI Guild
  2. Empowering All Levels: The creation of an AI Guild allows engineers of varying seniority to explore AI tools, share best practices, and collaborate on solutions.
  3. Facilitated Learning: The Guild acts as a knowledge-sharing platform where team members can learn from each other’s experiences with AI.
  1. Measuring AI Impact
  2. Qualitative vs. Quantitative Metrics: Loic discusses the balance between trusting engineers' qualitative feedback and gathering quantitative metrics to prove AI's usefulness.
  3. PR Labeling System: Engineers are encouraged to label PRs (Pull Requests) based on whether AI tools were beneficial, enabling tracking of AI's impact on productivity.
  1. Communication and Cultural Shifts
  2. Trust and Transparency: Creating a culture of trust allows for honest discussions about AI's benefits and limitations.
  3. Addressing Psychological Barriers: Effective communication strategies are crucial for overcoming fears and misconceptions surrounding AI.

---

Pivotal Moments and Insights

  • Real-World Example: Loic shares a specific instance where AI transformed a time-consuming compliance task into a quick, efficient process, reducing a potential two-day effort to just 90 minutes.
  • Collaborative Environment: Encouraging collaboration through the AI Guild ensures diverse perspectives and fosters a culture of innovation and experimentation.
  • Adoption Metrics: Loic highlights the importance of measuring the number of PRs per engineer as an indicator of productivity improvements linked to AI adoption, while understanding the context behind these metrics.

---

Takeaway Points

  • Empowerment is Key: Leaders should focus on empowering their teams rather than imposing mandates to encourage technology adoption.
  • Continuous Learning: Establishing guilds or councils for technology adoption fosters a culture of learning and continuous improvement.
  • Effective Communication: Addressing fears and misconceptions surrounding AI through transparent and supportive communication helps in reducing resistance.

---

Additional Resources

  • [AI Productivity Guide for Engineering Leaders](https://linearb.io/resources/ai-productivity-guide-for-engineering-leaders?utm_source=Substack&utm_medium=referral&utm_campaign=202509-ai-productivity-on-demand)
  • [Superhuman](https://superhuman.com/)
  • [Amazon Brain Drain Article](https://www.theregister.com/2025/10/20/aws_outage_amazon_brain_drain_corey_quinn/)
  • [Field Guide to AI Slop](https://www.ignorance.ai/p/the-field-guide-to-ai-slop)

---

Host and Guest Links

  • Hosts: [Ben Lloyd Pearson](https://www.linkedin.com/in/benlloydpearson/) | [Andrew Zigler](https://www.linkedin.com/in/andrewzigler/)
  • Guest: [Loïc Houssier](https://www.linkedin.com/in/houssier/)

---

Conclusion The episode provides valuable insights for engineering leaders seeking to increase AI adoption within their teams. By fostering a culture of trust, leveraging internal champions, and measuring impacts effectively, leaders can turn skepticism into enthusiastic acceptance of AI technologies.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:05Welcome to Dev Interrupted. I'm your host, Andrew Ziegler. And I'm your host, Ben Lloyd Pearson. This week, I'm sitting down with superhuman CTO Loic Hossier, who joins the pod to discuss how he overcame skepticism towards AI by empowering his most cynical developers instead of issuing top-down mandates. He'll detail his team's biggest and unexpected productivity gains along the way, which were achieved by a balanced mix of high-trust qualitative feedback and quantitative adoption data. But before we dive into this interview, let's cover this week's news. We have an accusation of brain drain at Amazon, Claude Skills Are Awesome, How to Say No, and a field guide to AI Slop.

0:47Andrew, where are we starting this week? Well, you know I love a little bit of gossip. What's the accusation? Oh, you must be talking about the article you and I read in The Register by Corey Quinn about the Amazon brain drain phenomenon and how that might be at play behind that AWS outage just this past week. It was quite interesting. It gave us a little bit of context on the outage and where it sits for AWS and its history of outages and what it means for its downstream customers. But it was also interesting to compare this with, in the Pragmatic Engineer, there was a deep dive on why this failure happened at AWS, ultimately caused by DynamoDB's DNS.

1:26And so once again, it's always DNS that seems to knock down the world, whether it's on AWS's systems or Cloudflare's. But for me, this took down my beloved Asana. Ben what did it take down for you yeah well for for me it was really just all the things that our podcast depends on that uh went down I was like thankfully I still had most of my AI tools I guess but I know our producer Adam was was panicking a little bit because you know some of the tools we use weren't available to get our episode ready for last weeks yeah nothing was a nice last minute fire drill yeah no I think everyone felt a little bit of the pressure downstream from the outage and it really kind of shines a light on how much of the world and the internet that we use every day runs on aws systems i love that you mentioned this on it because they also had the saddest and most brilliant error message i think of any tool that we use uh calling this some difficulties i was like yes this is somewhat difficult i think at least but you know i mean first of all i think you know the thing to really take note of is that an aws outage makes national news.

2:33Like that is actually generally like a positive thing for AWS when you think about it, because they are a company that's known for being pretty reliable. I mean, a lot of the internet runs on them at this point. So like the fact that this is a huge deal is actually something that is kind of a testament to what it's become. I kind of went down the rabbit hole a little bit, and I read about some former AWS people and including one, Justin Garrison, who wrote an article back in like either late 23 or 24, where he made this claim that he suspected there'd be a major AWS outage. He called out the year 2024.

3:12But then he said, no amount of multi-region redundancy will protect you. And that was actually the case for this outage, which is really interesting. I've read a lot of articles that sort of summarize what happened here. And from my understanding is that, you know, there was this outage in US East one, but there's a lot of this critical infrastructure for all of the AWS regions that only exists within US East one. And when that went down, like DynamoDB, sometimes the impact of outages can actually resonate out to people who either don't even host in US East one or have multi-tenant redundancy or multi-region redundancy.

3:52So there's been, there's a lot of opinionated takes out there on like what happened And like, holy cow. So, I mean, unfortunately, we all survived, it seems like. And we got through our period of some difficulties. And now I think life is back to normal. So hopefully there's not another one for a while. Yeah, no more outages for a little while, please. What about these Claude Skills? You know, what came across our desk on these? Yeah, so we found this article from Simon Willison titled, Claude Skills are awesome and maybe even a bigger deal than MCP. So Anthropic is out with this new product they're calling Claude Skills.

4:25It's a way in a nutshell of just generating folders that contain Markdown instructions and some scripts and additional resources that enable Claude to perform specialized tasks. So like, for example, creating a slide deck within Google Drive, just as one example, is using like these really lightweight files and like primarily Markdown and YAML. And, you know, there's a general comparison, I think, that you can draw to this versus MCP. But the difference being that skills are kind of this really like simple and like efficient and extensible way of using plain markdown instead of like a complex protocol.

5:03So, you know, I really love this article. And one of the things that really stood out to me is how a data journalist can really benefit from a tool like cloud code. Like they listed a lot of examples of how a tool like cloud code can reduce the effort to gather and interpret data like practically to near zero. which is this is something i've been doing a lot recently uh it's like it's not vibe coding it's like vibe scripting i think i have data in one location in one format and i need to move that data to another location and put it into a different format and really all you have to do is like provide the schema and the context behind what you're doing and the gpt is kind of do everything else so you know like mcp is a great solution to this you know you like build an interface that a GPT can connect to and you can tell it how to use that interface.

5:52But they just aren't really portable. Someone needs to build it. You got to maintain it. You got to like structure your API calls to work the way you want them to. But skills like on the other hand just need Markdown files and they're a very simple way to produce them and then replicate them elsewhere. You know Markdown has been this long staple of software development but I've also been like recently urging a lot of my non-technical friends to learn it too because it really is becoming that like universal human to gpt translation layer for a variety of reasons and there's a little bit of yaml mixed in the clod code design which kind of makes it a dual edge sword you know like yaml's very flexible but can get go off the rails pretty easily clod is like one of my favorite two favorite gpt friends right now so i will almost certainly be checking this out when i have the chance yeah Yeah, no, this is a great recap on CloudSkills.

6:45If you haven't played with CloudSkills yet, they're very powerful. It's a really interesting way of working with tools and actually kind of like collecting prompts together for a usage. I watched last week a deep dive video by Claire Vo on creating a CloudSkills. She actually got into like the schema and looked at the docs and put one together on a video. And it was really informative to watch someone build with it. So if you yourself are tinkering with CloudSkills, you know, I'd personally love to learn from you as well. So feel free to share the things that you're building in CloudSkills. As for me personally, I'm really excited about these hitting the scene.

7:20And I'm going to be honest about it. I feel like everyone's kind of catching up to how I've been using tools like Cursor already in this kind of agentic way. And there was a quote in this article by Simon that really stood out to me, saying that, quote, anything you can achieve by typing commands into a computer is something that can now be automated by cloud code. It's best described as a general agent. Skills make this a whole lot more obvious and explicit. And I completely agree. I've already been using these agentic coding tools as a much more general purpose automation and building primitive on my computer for a number of months.

7:56It's how I created our AI code review benchmark, which literally was a collection of scripts with lightweight markdown files on when and how to use them that lived inside folders in a project because I didn't want something as heavyweight as MCP because as you rightly called out, then you have to code it. And sure, I could vibe code it in a second, but then it's still something I have to lug around. So the skills, I think, are really great. They're going to scale. And I'm excited to see this get adopted by more folks because the more people that benefit and use this, the more we can settle this into like a protocol, into something that's really standardized across folks.

8:33and the more that we can all collectively benefit from working in this way. So I'm excited to see what kind of advancements we make next in tool usage with Cloud. Yeah, in a way, it's almost like it's a tool that takes your agentic approach to building things and sort of expands it beyond the Git repo, you know, because a lot of that stuff is still sort of constrained within Git. And, you know, there's a whole lot, there's a whole much bigger world of tools out there that these GPTs need to connect to. It also really smartly addresses a token consumption problem that's heavy with MCP servers that have a lot of tools and have very descriptive tools.

9:10They can be very heavy on MCP usage or on token usage, but rather using them this way, much more lightweight. I think we're going to be seeing a lot of optimizing techniques in the near future of like, okay, we have these tools. Now, how do we make them really token efficient? What's our next story, Andrew? The next story is an article by Amy Vora entitled How to Say No. And we love this article. It's great general advice for anybody about how to protect your time and work on the things that matter to you most. And this article comes to you with a list of techniques that Amy's used herself to protect her time and be able to focus on things that matter most to her.

9:47There were a few that stood out to me. One of them was my favorite was the use the old shopping trick, she called it, where if you want to buy something, it's in your cart, maybe it's expensive and it's going to be a big purchase, right? Don't hit buy immediately. Wait a day and come back. And if you really if you really still want to hit the buy button the next day, then, you know, you should get it for yourself or whatever reason you're getting it. Doing that same thing with actually saying yes to a task is advice that I could personally take. I'm very eager about jumping on tasks as soon as they come across my plate.

10:19I think a lot can be said about letting that task sit for a day and then asking yourself, do you still want to do that task? Another one that really stood out to me was gathering data about what breaks when you don't do things. So a lot of times we do things out of habit. I'm very routine-driven myself, so I have a lot of recurring tasks and things that I constantly do. And sometimes they come across my plate, and I'm like, do I really need to do that right now? And her advice here is, you know, say no to as many things as you can and then only do the things that break and only you can fix. Yeah.

10:53So the point that really stands out to me, first of all, those are two great ones. I especially love the first one about waiting 24 hours. I actually had a co-worker say that that's how they treat all executive requests. Like anyone from an executive level, when they ask you to do something, wait 24 hours to respond to them, which is bold. But if you can pull it off, I guess it's a good move. But the point that really stood out to me is to, from this list of tips is to celebrate the things that you're doing. You know, like if you're someone who always says yes to everything, like you're probably just going to be heads down trying to get all the stuff done rather than like focusing on reflecting or celebrating things from the past.

11:32And I think like deliberately setting aside time to reflect rather than to produce is a really good practice to just ensure that you're always prioritizing the right things. Because that's really what saying no comes down to is establishing priorities. Right. And now with AI, it's easier than ever to say yes to everything because you can just say, well, we'll just vibe code this in this afternoon and see how far we get kind of thing. So I think dedicating time to reevaluate your priorities and, you know, celebrate what you've accomplished is a good way to counter that that sort of pressure. And this article is not specific to engineering, but it's a really quick read.

12:12And I think there's tips in there that everyone could benefit in a leadership role. Totally agree. Jumping into our next article, you know, we kind of covered this really interesting piece by Charlie Waugh. And it's a field guide to AI slop. This is our second week covering Charlie's writing. I love following his stuff on Substack, so we'll be sure to include his newsletter in the roundup below. But Ben, why don't you give us a walkthrough of how people identify AI slop in the wild? Yeah, so there's basically a bunch of tips and tricks in this article on how to identify what I would call poorly constructed AI workflows.

12:49And first of all, I just want to clear the air. We use AI in some capacity for practically everything that we produce here at DI now. Whether it's like helping us devise content strategies, understanding new concepts and trends, actually writing and drafting articles, or even just like suggesting titles for an episode. You know, we're actively adopting it everywhere that it makes sense for us to do because it's just letting us increase our leverage. Like there's a lot of toil that comes with producing something like this show. And it really just lets us leverage ourselves a lot better. But with that said, man, you have to have really, really tight guardrails on all of this, or without that, that's how you become obvious AI slop.

13:32And I think the point that I love the most out of this was they shared a chart about the long-term rise in the use of m-dashes on Reddit. It's shocking to me to see how substantial that is, but also not surprising to see it happening. because you know personally i've just had i've long believed that conspiracy that like reddit is like overtaken by bots and there's there's really no reason for m dash usage to increase other than because there's more ai being used for them there's no linguistic revolution happening around the use of m dashes but there was a lot of examples like if writing has a lot of parallelism like it's not just x it's also y which i've seen that everywhere within whenever use AI.

14:15There's a list of emojis. And I kind of have the suspicion that like emojis are a way for GPTs to share context or information dense context. So there's actually like meaning behind the emoji that like other GPTs actually like get that a human just looks cringe to us. But really what it comes down to is like GPTs are really great at saying the right thing, but not so great at making like definitive arguments. You know, there's not like a real opinion behind a lot of the stuff that comes out from AI. But, you know, the most consistent advice I give to everyone when they're using AI is like, if your output is bad, go back and fix your inputs.

14:51Like don't try to fix the issues downstream. You know, so if it's producing bad writing or bad code, the problem is not the code itself. It's the things, it's the system that created that code in the first place in the context in prompts that it was giving. So, you know, writing code or writing articles or social media posts, it's disposable now because it's trivial to generate those data artifacts. And good code writing requires a really deep context and an awareness of what your desired output is. And if you spend your time focused on building that context, rather than trying to like tweak the prompts or the output of that context, that's when GPTs become really effective.

15:36whether you're writing a blog article or code, you know, and they're really powerful in the right hands when they're operated this way. But if you don't keep it tight reins on them, things can get sloppy very quickly. That's the big unlock here. And you should always be iterating on those inputs as opposed to those outputs. Now, I think it's really tempting to pick up outputs and make them better just in the systems that we're used to working in. But we are living in a new world where you have to focus on what comes upstream to create that content for you. I also really, I like how you called out that they use emojis almost like pictograms.

16:09I've totally picked up on this as well. And obviously the GPTs, I love picking for like the same like 10 or 12 that they, the rocket emoji. Yeah. Like there's ones that are now just like dead and just as dead as an M dash because it's like emojis have killed it. And like the rocket emoji is like r.i.p rocket emoji you did not make it this year and so but but coming coming out of it I think like there is a lot of semantic meaning behind emojis a lot of times when you encode them in text I think of like github emojis you know it's like you can describe them there's one word descriptors and you do sometimes get these inserting cases where it combines them or makes them dense and when it combines emojis it makes me think of almost like an ai kanji like It's taking two symbols and it's putting the next ruler and it's making a new symbol.

16:57That's how language works in the East. It's actually quite fascinating to see the LLM do it. You know what I think it is? It's a way of communicating tone. It's using emojis to communicate tone. Yeah, it's like putting on a mask so it seems all nice and friendly. Like, oh, I'm going to put an emoji list on this so I'm not an asshole. But I also, that was interesting how this article, it's like an AI slot bestiary. the way that some of us has given labels. I think that's the best thing about those articles. Charlie's equipped us with really well-scoped terminology for referring to the phenomenon in the wild.

17:31We've already had like terms for some of it, but he gives us some really great ones like snappy triads. Like you do see this all the time, these snappy triads of like fast, efficient, reliable, or think bigger, act bolder, move faster. Like things move in threes in the AI world. And another one he called out is unearned profundity, as in, you know, like a serious narrative shifting, like mic drop moment that comes out of nowhere, totally unearned in the posts. I think these are really common in some of the funniest LinkedIn posts I've read recently, where it's like everything changed or something shifted.

18:07It's like a movie trailer. And so I really appreciate this AI slot bestiary. And I'm going to be using a lot of these terms. I was like, what's the coding equivalent of this AI slot bestiary? and I was thinking of our friend, the friend of our show, Brigida Buckler, about like her musings that she often writes and how it's like, here's what I've seen AI doing the last few months, you know, and I think. Yes, it's like observing it. It's a very similar effect. Yeah, it's like observing it in the wilds, like taking field notes. That's why I always appreciate a memo from ThoughtWorks and we cover them here all the time.

18:44Yeah, absolutely. Awesome. Well, that's the news that we have for today. After the break is my conversation with Loic Hossier of Superhuman. Stick around. Are you struggling to prove the impact of AI in your engineering org? Linear B's new AI productivity guide gives leaders a structured framework to track adoption and tie results to outcomes that matter. Throughput, quality, and real ROI. Inside, you'll get tactical advice on measuring adoption with developer surveys and AI acceptance metrics. Plus, five proven workflow automations to cut dev toil across the SDLC. Don't settle for vanity metrics.

19:23Discover how to drive real AI productivity. Download your free copy today. My guest today is Loic Hossier, an engineering executive with 20 years of leadership experience at startups, scale-ups, and global enterprises. He currently leads engineering at Superhuman, the most productive email app ever made. He specializes in helping high-growth companies scale with speed and discipline, bringing a culture of excellence and rigor to how teams ship, scale, and make decisions. Loic has also held senior engineering roles at Product Board, First Base, and DocuSign, where he led global teams through platform expansion and merger and acquisitions.

20:03Loic, welcome to the show. Thank you. Thanks for having me. Excited. Yeah, awesome. I've heard that you have a team that may be skeptical of AI. So if you have a team that's skeptical of AI, what's the first small-scale experiment or step that you would recommend for a team to sort of build trust and demonstrate the value of AI without disrupting core workflows? Ah, that's a good question. We were in that situation at the beginning of 2025 at Superhuman. and I think the thing that clicked with my team, mostly senior people, long tenure, they have their flows, they know how to work. Yeah. They look like, they're pretty cynical with everything that is like a bit budsy.

20:49Very typical engineers. I mean, you know what I'm talking about, right? And I think where we've been smart, we've used our most senior and respected engineer, which is like a chief architect, can go like with his stack everywhere, like the typical crazy guy that everyone looks up to. And he was one of those cynical. And we discussed together and said, you know what? We do like a deep dive. Because he did a deep dive six months ago, and it was not ready. It was not there. It was not good enough for the level of quality that we were looking for. And we moved fast. So it was more like something that was slowing people down rather than anything.

21:26And in Jan, he did another try. He's like, oh, my God. That's different now. at least I see the potential at least with the right board rails that can work and he started to use it on a daily basis being intentional putting that into his flow and he's like a, I mean I don't like this term like 10x engineer but like it's typically someone that you would qualify like this if you have to so he slowed down a bit to learn a new flow to learn like how this is impacting his own ways and everything and basically did a quick talk internally to say hey people that's crazy and i think like creating this sense of oh damn if mike is looking at it seriously now there's something to it so that was the first thing like creating this sense of like oh damn it's not like just buzz anymore it's not just like vibe coding stories that you hear on twitter and anything and everything like it's it's real it's real sure it's not perfect and everything but that was a kicker we've seen you know both here dev interrupted but linear be our sponsor company like we've seen a very similar effect where we're product decisions that we maybe tried to make like a year ago um with the given technology like it just didn't you know we've taken too much effort to build like something with ai or it just didn't work the way we wanted it to and then um you know and then stuff we're doing today it's like it actually takes a fraction of the effort that we expected because the technology has advanced so much that suddenly now things that we didn't think would be possible, like we're suddenly doing like with relative ease, you know?

23:03Yeah. One thing that is interesting on that topic is like, uh, we try to measure the impact all the time and between a qualitative and quantitative, uh, I would say metrics. The one thing that everyone is aligned on is that if there's a velocity gain, it's basically on two things. One is like the time to ramp up on the new stack. You're not in that domain. You need to collaborate with a new team. you don't know the stack, you don't know the libraries, you don't know the structure and everything, it's like brand new. Damn, like the ramp up now is so easy. You can just like talk to your code, understand the entry point, but why they do it this way, not that way.

23:39Like all like cursor code code and everything, like they will basically bring you up to speed super fast. So like this domain expertise that you don't have, that will take a couple of days to be fully set up, fully understand the way the new team is working, one hour. And you get like a nice picture. So that's the first one that we've seen. the other piece uh you were talking about like the the time to build becomes like different based on the stacks and everything what we've seen is like all the tools the side tools that you're always like oh damn if i had like one day i would build that tool that will save me like hours and hours and hours it's not production code but like something that you would do on the side maybe it's uh i don't know some sort of like a csv parser on that you don't have the time to do and everything and now two prompts 20 minutes and you have something that is working you never had the time to do so all those small additions overall contribute to a great like improvement of the velocity yes and a phrase i'm hearing more and more is like disposable code so even if even if that problem is disposable you only solve it now and then you just throw the code away for the rest of time it's like it only took me two prompts so why not do it anyways yeah yeah and it's saving so much time globally and you don't care.

24:50It's kind of like drawing on the back of the napkin. It's like code on the back of the napkin and it just works. And I'm wondering because another thing that I hear a lot as I speak to a lot of engineering leaders is there's almost this relationship between internal AI initiatives and building AI products. So if you don't have that sort of internal built competency with AI, If you don't know how to use AI in your day-to-day work to be more productive yourself, whether it's writing code or onboarding to a new stack or you haven't internalized a practice, you may struggle to actually adopt AI as a product.

25:32If you were trying to build AI into your product, and that's something that Superhuman has been doing a lot recently. Do you feel that sense as well? That's an interesting one. Are they interconnected to some extent, especially in the... That's a good question. I'm not sure. I'm not sure because the approach is slightly different. As you said, on one end, it's like a productivity tool and how you improve your flow, which is slightly different that, okay, I use an NLM and an OpenAI API or based on inference, whatever, to do the job that I need for the code. I think you can be very cynical for your own tooling because our people are craftsmans or craftwomans.

26:20So they love the craft. So they are very specific in the tools and tweaking their own workflows compared to, hey, I need to do it for my users. I know the way to surface that type of information. I need to use an NLM. So I feel it's slightly correlated. it. And I don't know if one is influencing the other. I guess seeing the result of an NLM in production for our users getting better and better is an incentive to maybe reconsider your own flow and how you can leverage some of that in your flow. But the problem is like it's different tools. Even if like the backbone is still an NLM, your flow depends on the way it is surfaced like in the way you work.

27:02So it's not because OpenAI or whatever APIs are working well for your use case implemented in your code, that your IDE use that, I would say, the right way. And yeah, I don't know what you think. Hey, I'm not the one being interviewed. Well, so you touched briefly on measuring the impact of AI. So I wonder if maybe we can dive into that a little bit more because that's something we're hearing a lot. Like every engineering leader we talk to, this comes up. How do we make sure that we're investing in the right tools and that those tools are actually having a positive impact on our organization and that we can prove that that positive impact has happened?

27:46So how does Superhuman approach that challenge? That's an interesting one. In Q1 in 2025, the first approach was like, you know what, let's not measure, let's get adoption. Yeah. Because everything is changing. Everything is different. Like if you're a mobile engineer working on Kotlin or you're a front-end engineer working on React, the models might be different. Your ideas are different. Your tools are different. Yeah. And the maturity of such tools might also be different. So you cannot expect to have the same level of adoption or the same level of productivity gain based on the different teams.

Read the full transcript

28:24So that's the first point. So we opened the flow. We said, you know what? Free for all. You take three subscriptions to whatever tools you want and everything will pay. And the only thing that we were asking for is to consolidate the use cases where it was working. So we compiled a list of like, hey, here, not mature, doesn't work. That, damn, that was cool. Here, promising. Maybe we should try again in three months. So that was the aspect to drive the adoption. The adoption at that time was mostly measured in terms of how many subscriptions people have, do they consume their tokens, whatever.

29:00And it was great. adoption was there. And then you mentioned how do you prove this is working? Because at some point my CFO was like, hey, I would say, you guys, they're spending quite some money. Is it useful? So we started to be a bit more precise. It's still juggling between qualitative and quantitative approach. We trust our engineers. They work hard. So if they tell me yes, I think I'm winning 20 % productivity gain over the course of a month. we trust them very qualitative uh but we wanted some quantitative data yeah so uh basically we started to flag our prs uh with some specific labels yeah like yes i use the eye and in that case no impact or like i spent so much time trying to get i would say the pr with the app of ai if i have done it myself it would have been like so much better uh so we were starting to like categorized the work pretty fast.

30:00We're seeing that 90 % of the PRs were AI helpful, qualified, and with an adoption roughly said around like 70%. So 70 % of the PR were flagged. Sometimes you just don't need, like you fix the CSS or fix, like you don't need AI. So like it's, I think it's, you don't need to measure more. So like all of a sudden we had some sort of like a proof that one, it's used, and two, it's useful and again we rely on the trust of our people uh if they say it's useful and they're pretty like vocal so if they say no it's shit yeah pardon my french uh if they say it's not great uh we will trust them as well so clearly that was like a very solid signal um now in terms of productivity uh it's multifactorial so ai is one piece but there's also like all the change you do in an organization you secure your team and everything and uh the one kpi we're looking at at superhuman is the number of PR per engineer per week, on average, not at the individual level.

31:02So that's why we normalize per engineer. And we've seen the trend going up. Even as we grow, even as we are getting acquired over the summer, we've seen this trend of basically the raw throughput being better, while the, I would say not the average, but the median PR size staying the same. So there's a lot of things we can say about this metric. But at least it's one that is the most directionally right about the performance of a team in terms of throughput. And we've seen the progress. And we do believe that AI was a major part of that. Yeah. The PR throughput, it's a tricky one because it's probably not the best one to set goals around.

31:47because if you tell people you have AI, make more PRs, you'll get more PRs. I'm transparent on it. I'm transparent on it. So like I have some, I have my OKRs and I say, guys, ideally we want to be there. Do I will use that as a performance metric like for the performance cycles and everything? No. But just know that this is my way. So I would say show to the board, show to the rest of the exec team how we work better. Because one, I don't want to measure too much things because then you start more measuring than working. Let's have one thing that is simple. Everyone understands. You understand that it is, okay, that's roughly said a lagging indicator of our throughput and our performance.

32:31But at the end of the day, I don't care. You are, I don't know, you're a backend engineer and you're working on removing doing a huge migration from one framework to the other. You will have a massive PR and you spend two weeks on it and everything. I mean, I won't complain. I won't complain. So it's still a metric. It's still an objective, but like a loosely held so that they understand what's behind it, which is much more important. And if you're in a high trust environment, you know, and you know that it's, it can at least tell you something has changed, you know, within your organization. and it may not be something you track forever you know but it could be something that as you adopt a new tool you can say we can see that something has changed about our throughput understand the ramifications of that and try to understand if there's any sort of negative impact from it and with every metric the metric in itself is useless it's like the discussion that the metric is triggering that would be interesting so like if you see something like maybe one team not taking off as part of this like constant uh i would say improvement then you can ask questions you can begin you can talk with the tech lead and like what do you think what's your perspective right everyone is like beneficiating from that why not here what's what's the problem and and maybe it's contextual maybe it's uh oh because like this month we were working on this freaking project it was like really tricky so a lot of thoughts small amount of card uh makes sense makes sense but you have these discussions so and that's the most valuable part of any metric here at devon erupting we really evangelize this idea of like you know metrics metrics lead to action you know if you're just measuring and you're not doing anything with that information there's really no point in measuring so the goal should always be like have it lead to something that makes improvement you know so as long as that's that's the outcome like that's what really no exactly and you mentioned something important like that the culture of trust yeah like i was like direct i said hey guys i need a kpi i'm freaking kpi um for my uh i would say for my exec team um come up like i discuss with my leads we think this one is like okay ish nothing is perfect understand that this is what it is basically don't game it uh i don't need to have like amazing results if it's not related to what we do okay so let's be honest with each other and everything and when you trust people that like, okay, we understand what you're trying to do.

34:59We do our best and boom, that works. But trust. Trust. Yeah. So I want to talk about some of the barriers to adopting AI within an organization. So just beyond technical implementation, you know, because that's something that we kind of discuss endlessly on this show. So what do you think are some of the like effective communication tactics that you've used to like address particularly like psychological barriers around AI and around fear in particular. You mentioned that you had that lead developer that changed the perception around AI within the organization. Beyond that, what have you done just to help communicate the value of AI within your organization and help overcome the non-technical side of AI?

35:47Yeah. One of the aspects that we've decided very early on is to separate the different stacks. We understand things are different based on the way you work because different workflows, different like ideas and everything. So just saying like, hey, don't compare yourself. Don't, whatever. Like we won't do that. So we built guilds. So we have this concept of guild. We have a front-end guild. We have a back-end guild, blah, blah, blah. And we decided to create like an AI guild. Okay. The purpose of the guild was to gather like the best practices of each pod, understanding like how they work, what's working, what's not working.

36:20And also like being evangelist at the same time. It proved to be a really good way to have everybody knowing that this guild existed. One person on their part was there. So everyone was aware of everything that we would do. So for example, I had to modify a bit our compliance approval process for new tools. Because I just wanted, like you want the tool today, you need to have it today. You don't want to wait one week because compliance, because security and everything. We still want to be safe because we manipulate some, I would say, pretty critical data, but we wanted this to be like the P0 for the compliance team.

36:59So they knew that new tool for AI and productivity, you stop everything that you do, and you validate it. So those are the things that were helpful. And the guild was basically saying and basically sharing this information that, hey, all the roadblocks from a compliance financial standpoint are removed. you want to try something try something oh and you know uh who said yesterday i was on the uh guild meeting that team is doing x y and z sounds cool oh yeah i will try it this week and and i think that works a lot so for example on my side so i'm a cto vpn whatever you want to call it i totally stepped away i didn't want it to be like top down like oh we need to use ai i said hey guy ai seems cool free for all like my kpi is like throughput if it's not helpful don't do it if it's helpful please i would say don't be smart and i totally delegated the influence to that guild led by this chief architect that was highly respected and i think that it worked a lot like i was focusing on what i do best which is like working with the rest of the organization making sure that there's like no like red tips and everything and they had like the in-depth understanding of what's working what's not working while i would have been maybe a bit more like removed from the code and using some like, of course, it works.

38:23I've seen a Twitter, also a tweet yesterday or something like this. Yeah. Yeah. We actually have a similar concept. We call it the AI Council, which I love because it's almost like a Jedi Council, you know? And I think what I really love about this approach is it kind of reflects like what's becoming like a traditional approach at this point of like the DevX team where it's like like a broadly focused team that sort of has to overarch the entire organization in many ways. And it's more focused on like broad developer challenges, like tooling and best practices and training, support, enablement.

39:00It's like seeing the forest, but then also being able to like when they have to like dive in and like evaluate the individual trees to like help like the front end team, understand why this model is hallucinating like crazy when they use certain libraries and stuff. Because I think this is something that a lot of organizations, like 2025 has been the year that a lot of organizations have really learned one really powerful lesson. And that is an AI tool. If you just throw an AI tool out there, it's probably going to fall flat on its face in many situations. It'll work sometimes, but there'll be a lot of times where it just doesn't have the context that it needs to be successful.

39:38And a lot of what an AI guild or an AI council can help is identifying those situations where it does have the context out of the box and you can just deploy it and just throw it out there and be like, here it is, developers, go use it now. But then also, more importantly, finding the situations where it doesn't have all of the context and tools and resources that it needs and then helping those teams build those resources so that you can then deploy ai in that situation you know yeah and um also we've seen it like this so our ai console equivalents so for one it's it's not a jedi console in a sense that we didn't like selected like the most senior jedi like to work on it but well what i actually love about that team is it it's all ranks yes exactly it's like we have junior people who really got into ai and like please join us and like like you're helping guide yeah the most senior people that are accompanying i i think like the the most senior people comes with that sarcastic curiosity uh about things so they will try things like to really understand if it's working and not i would say basically i would say supreme victory like as soon as they have like a line of code that is generated but more junior people have still this brain plasticity yeah they know the tools are changing every week so like they will try the tool every week compared to someone that is maybe like um i would say more senior you try it once not working and confirmation bias and you i mean you don't look at it for like a month or two so i think that blend was working uh working well you need people also that are pretty energetic so some if those people are too introvert uh too much like talking to the rest of the organization and everything might be uh might be a challenge so So we have a good blend of people out in that console, and it's working well.

41:32And they are the ones who are facing like, oh, we have this cool config file for cursor, and you're using our context and everything. And all of a sudden, okay, can we do it? It's personal. We put it in the repo, and everyone is using it and everything. So those practices start to raise from this console. And yeah, it's cool. Yeah. And I think one more point on the levels of seniority. Like junior, you know, whether it's developers or any role within the company, junior level people versus senior level people adopt AI in very different ways from what we've seen. Like a junior level developer, generally speaking, they often are using it for more things, more like code generation or maybe learning onboarding to a new tech stack.

42:17Whereas a senior developer, you know, they're typically they already know how to write code really efficiently. So they may maybe don't use AI for that as much, but they might use it for ideating in the early planning stages. You know, so it's really important to get all of those perspectives on your AI counsel or guild. No, totally. Totally. And you're right. Like the ideation part is probably like the way like most people are using, like most senior people are using it. Like, hey, I see this. That's a problem I'm trying to fix. My strategy is to do X, Y, and Z. What do you think? challenge that.

42:54And they use basically tools like this to make sure like, okay, this is thinking out of the box this time. That's interesting. I might dig in. And it gives you also more confidence that you're on the right track. Yeah. So I just got one more question for you. So do you have any specific examples of workflows or daily tasks within the superhuman engineering team that were you seeing the introduction of AI lead to some sort of non-obvious improvement to either speed or quality? I can mention one very personal. We were acquired over the summer by Grammarly. And in any due deal, there's a lot of computation of data that you need to do.

43:35Which, by the way, I want to mention, it's like my two favorite tools in the world coming together. Lovely, lovely. So hopefully you'll get the best of both worlds. And during this due diligence, you need to compute a bunch of data. Like you crunch. And in one case, it was like, hey, at least all the open source libraries that you're using, tell me all the license they are, MIT, LGPL, whatever. Tell me who's the licensor. I was like, what the f*** is the licensor? Sorry for your audience. And then like, oh, that's the name that is officially on the license and everything. I was like, holy cow.

44:09I would go into all the GitHub repo and find. I said, hey, Cloud Code, I have this small problem. I have this CSV with all the list of the open source libraries. We already tracked the GitHub repos of those. Find, create a script that we find the readme or the license.txt or whatever file that is containing the license. In that document, find the licensor. Oh my God. 90 % of it was matched. Wow. Something that would have, I would have brute force it. I mean, there's no chance. Like a due diligence, you don't have the time, whatever. and you cannot just involve all your team to distribute the work and everything.

44:51And it took me, I guess, one hour and a half vibe coding through it. And so that was a good example. Something where I would have spent two days and nights during the due diligence. As you said, disposable code. I don't even know where this script is now. And it's done. So that's a good example. Believe it or not, I used to work in open source compliance And so this use case, I would want to rip my eyes out rather than trying to do that manually. So having AI for something like that is pretty incredible. Yeah, because all the tools that are doing those, I would say SaaS, and to understand the open source libraries that you're using and everything, they don't go to who is the licensor.

45:34So you have this nice file and everything, and you believe that you're covered for your due deal. And all of a sudden, you have this because that is coming. And I imagine that data has got to be so messy. There's no way that data is clean. It's not structured. it's not last. So it was fun. It was fun. But it was a good usage of AI. Yeah. Well, awesome. Well, it's been really great having you on our show, Loic. Thank you so much for coming out today. Thanks for the good time. To all of our listeners, thank you for tuning in today. If you're not subscribed to our Substack, head over to devinterrupted.substack.com.

46:01Click the subscribe button. Give us a review on wherever you listen to us for podcasts. And tune in to us next week. We'll see you then.

46:18We'll see you next time.

From the publisher

Forget top-down mandates. How do you foster organic AI adoption on a skeptical, high-performing engineering team? Loic Houssier, Head of Engineering at Superhuman, joins us to share how he did just that. He explains his strategy for overcoming cynicism, which involved leveraging a highly respected internal champion, the Chief Architect, to re-evaluate the tools and prove their potential was no longer just buzz.

Discover his team's biggest, unexpected productivity gains: dramatically faster ramp-up times on new codebases and the rapid creation of valuable internal tools, rather than just raw code generation speed. Loic details Superhuman's pragmatic, high-trust approach to measurement, blending qualitative feedback with simple signals like PR labels. He explains how they fostered this adoption with a bottom-up "AI Guild," empowering engineers of all seniorities without a heavy-handed mandate. He also reveals a stunning real-world example where AI turned a potentially days-long compliance task into a 90-minute win. This episode is a practical playbook for building genuine, bottom-up AI adoption.

Get the guide: AI productivity guide for engineering leaders

Follow the hosts:

Follow today's guest(s):

Referenced in today's show:

Support the show:

Offers:

More from Dev Interrupted

All 208 episodes
How leaders win over their team’s biggest AI skepticsDev Interrupted · 46 min
Listen in VO