How to measure AI developer productivity in 2025 | Nicole Forsgren

24:40What do you find are the most like, I don't know, two or three most common improvements companies need to make? I will say, you know, kind of echo that process. There's almost always a process that can be improved and that can be approved, improved without a lot of engineering lift or a lot of engineering headcount. Right. Most large companies in particular have something that is several, several steps. It's the way it is because it's the way it is, but that's no longer the way it is. Right. And even small companies, sometimes it's just a little too YOLO and you don't know what it is and you're kind of chasing everyone around.

25:12So if you can create a very lightweight process, that can also be helpful. That can be one of the best places to start, especially if you have limited exposure to the whole rest of the org. right? Sometimes just a team process can help. I will say from a business leader standpoint, a lot of what you can do is provide structure and support for this organizational change. Communicate what you're doing, communicate what the priorities are, communicate why this is important, celebrate wins. Because if folks try to do this just like a one-off side, fully isolated project, it's really challenging to get some good momentum and get people to care and to get them stay involved, right?

25:56Because it feels like just another interim internal project that isn't going to matter or that isn't going to get celebrated, but it has these huge upside potential returns for the business. It's interesting what I'm hearing here is nothing about tools or technologies. It's not like move to this cloud. It's not like install this new deployment system, its processes, and people, and org and morale. Yeah. Now, there will be technical pieces that are very important, right? Especially now with AI, right? We're rethinking how build and test systems work. We're rethinking feedback to users so that it's very, very customized in terms of what is shared and when it is shared.

26:40There are a lot of technical pieces that are involved, but that's not the only thing, right? It's necessary, but not sufficient. And that doesn't have to be the place that you start. I'm going to ask you, I have a hard question I want to ask you that I thought of as you were talking. I feel like this is the question that most founders and heads think about. And the question is just like, how do I know if my edge team is moving fast enough, if they can move faster, if they're just not performing as well as they can? What are just maybe smells, signs that tell you, yeah, my team should be moving faster versus like, this is just the way it works.

27:14This is as fast as they can move. Most teams can move faster, right? So, and also, given what we know about cognitive load, not all speed gains are necessarily good, right?

27:30Or the upside is going to be kind of limited, right? Once you hit kind of a certain point. Most people are not even near that point. I don't know a single team, frankly. But how do you know? You know if you're always hearing about bills breaking, flaky tests, overly long processes, if you have to request a new system or if you need to provision a new environment or if it's really, really hard to switch tasks or switch projects. right so if someone has an opportunity to go work in another part of an org and they don't for reasons that are unclear and like not political and anyone says anything about the system that's usually a pretty good smell uh that there's friction somewhere because once you finally figure out your system and you're able to get work done you don't the switching costs can often be really really high to go anywhere else and so sometimes people will do that but But I've worked with companies where switching orgs within the company, you had to basically pay the same tax as a new hire because the systems were so different and they were so full of friction and it was so difficult to do so many things.

28:48I love the first part of your answer, especially, which is you can always move faster. I think every founder is going to love hearing that. To your point, though, there's diminishing returns over time. Yeah. And you don't know about the quality, right? So I think that's the other side is that you can always move faster, but faster for what? Are we making the right business decisions? And I think that's especially where PMs come in. We can ship trash faster every single day. We need strategy and really smart decisions to know what to ship, what to experiment with, what features we want to do in what order and what rollout, right?

29:22The strategy is the core piece. And then think about speeding that up. If we don't have the other pieces in place, I mean, garbage in, garbage out. I want to follow that thread. But before I do that, just to mirror back what you shared. So signs that your team, there's a lot of low hanging fruit to improve the productivity of your team is builds are always breaking. There's flaky tests that are constantly incorrect, false positives. It's hard to context switch between different projects. The system, you just hear people talking about the system is just really hard to work with. Is that roughly right?

29:52Cool. Okay. So going back to the point you just made, there's a sense that AI is making teams so much faster because it's writing all this code for them. You're going to have all these asynchronous agents, engineers working for you. It feels like a core part of your message is that's just a one part of engineering work. There's so much more, including figuring out what to build, alignment internally. Maybe just speak to just like, there is a lot of opportunity to improve engineering performance, productivity, but there's so many other elements that are not improved through AI. Yes. Or could be in the future, right?

30:25Like, I think there are a lot of ways that we can pull in AI tools to help us refine our strategy, refine our message, think about the experimentation methods or, you know, targets of experimentation or think about our total addressable market, right? But we need to have that strategy and plan fairly well aligned, right? Or at least have like two or three alternatives that you want to test because now the engineering can go, or at least the prototyping especially, can go much, much faster, right? We can throw out prototypes. We can run A-B tests and experiments the customer are facing, right? Assuming that we have the infrastructure in place, which allows us to learn and progress much faster before, right?

31:09Some places it used to take months to get something through production, to do A-B testing and get feedback. We can do this in a day or two, definitely under a week. But we want to make sure that we're building and testing the right things. Are we partnering with the right phone? Do we have the data that we need? And I will say AI can actually be a pretty good partner there if you have a good conversation with it and then also check with your experts. What type of data should I be looking at? What type of instrumentation do I need? What type of analysis can I do? because then you can also go to your data science team and say, like, I'm planning on doing this.

31:44I'd like to, because let's not just yell OAB tests, right? Because that can be, it's a shame to do a large test and end up disrupting users or disrupting customers or breaking privacy or security protocols and also end up with data that's unusable, right? Because you just can't get the signal that you're looking for. But now I'm also seeing people kind of accelerate that into a few days versus a few weeks. And so they can kind of start those key stakeholder discussions from a much more informed, kind of filled out space. Today's episode is brought to you by Coda. I personally use Coda every single day to manage my podcast and also to manage my community.

32:24It's where I put the questions that I plan to ask every guest that's coming on the podcast. It's where I put my community resources. It's how I manage my workflows. Here's how Coda can help you. Imagine starting a project at work and your vision is clear. You know exactly who's doing what and where to find the data that you need to do your part. In fact, you don't have to waste time searching for anything because everything your team needs from project trackers and OKRs to documents and spreadsheets lives in one tab, all in Coda. With Coda's collaborative all-in-one workspace, you get the flexibility of docs, the structure of spreadsheets, the power of applications, and the intelligence of AI, all in one easy-to-organize tab.

33:04Like I mentioned earlier, I use Coda every single day, and more than 50 ,000 teams trust Coda to keep them more aligned and focused. If you're a startup team looking to increase alignment and agility, Coda can help you move from planning to execution in record time. To try it for yourself, go to coda.io slash Lenny today and get six months free of the team plan for startups. That's coda.io slash Lenny to get started for free and get six months of the team plan. Coda.io slash Lenny. I love that you work with a bunch of different companies and a bunch of different types of businesses. I think very few people get to see inside a lot of different places.

33:43What kind of gains are you just seeing in terms of increased productivity with AI? Like how real, like how big of a gain have you seen? I'd say it's real. And I would also say we don't have great measures for it yet. We're still trying to figure out what to measure and what that looks like. One of the best is going to be velocity, right? All the way through the system. How quickly can you get a feature or a product or something through the system so that you can then experiment and test, right? Either from like idea to like final end or even kind of a feature and a piece through the system so we can test.

34:15That's really good. Now that's also hard to tie back directly to like a particular AI tool in the hands of a particular developer. But there are some other things that we can look at and we can see. And that I've seen is, again, this kind of rapid prototyping. I hate lines of code, but I'm going to use lines of code. We do see, I know I worked with some folks who had kind of a whole set of companies they were looking at, and they found that AI was generating significantly more code for the people who were using it regularly. But then they also found that for folks who were like, you know, regular users of AI coding environments, AI IDEs, the tool kind of gave them more code.

35:07And then the engineers themselves, the increase was double what the coding agent had given them. And so one, I'd say probably kind of a secondary or knock on or just a smell, right, is it can unblock you. It can speed up the work that you would already do, right? I know sometimes when I work, it's like the first few minutes, it's hard for me to start. But once I get started, I'm there. And so they're really good at unblocking and unlocking that. Something I've seen people on Twitter sharing is how good OpenAI Codex, especially, is at finding really gnarly bugs. And I think it was Karpathy that shared he was like so stuck on a bug and no AI tool could figure it out.

35:46And then the latest version of Codex spent like an hour or something looking into it and found it for him. Yeah, I'm hearing incredible things like that, right? Well, and even also, you know, writing unit tests and spinning up unit tests and creating documentation and cleaning up documentation because I know now people are like, oh, well, we have agents. I don't need to read the docs because there's the code there. turns out agents rely on good data, right? Because it's all about how they've been trained or how they've been grounded. And better data gives you better outcomes. And some of that data includes documentation and comments.

36:23And the better documentation, the better comments you have, the better performance you're going to get out of your AI tools. And AI can help you write that documentation. I've been working with Devin a little bit, and it's really good at that stuff. Yeah. Okay. Let's talk about this framework, this book. So you're publishing a book called Frictionless, which sounds like a dream. How do you create a dev team that's frictionless? It's called Frictionless, Seven Steps to Move Barriers, Unlock Value, and Outpace Your Competition in the Age of AI. There's a seven-step process to this. Walk us through this.

36:54Maybe give us just context on this book, what it was meant for, what problem it solves, and then the seven steps. So I will say I also wrote this with Abhinoda, who has just of DX. He has incredible experience in the space, right? He's worked with hundreds of companies. And so it was kind of nice bouncing ideas off of him. And, you know, also thanks to all of the engineering leads and devx leads and CTOs and engineers that we talked to, to kind of make sure that we, our smells were right. Right. And so who is this book for? Let me actually take a tangent on Abhi and DX, since you mentioned him.

37:29This is super interesting, and I think it connects so directly with this conversation. So Abhi started this company called DX, which is such a great name for a company around developer experience. They just sold the company for a billion dollars to Atlassian. It's a very high multiple on their AR. It, to me, shows exactly why this conversation is so valuable, just how much value companies are putting into improving developer experience. Atlassian would spend a billion dollars on this like a early stage startup that was doing really well and people left it but it was like early stage-ish a billion dollars uh and now and the idea is they have all these companies working using Jera and all their products uh they're all trying to figure out how do we measure productivity uh it's worth a lot of money to them so and I know you were an early advisor to them too so yeah it just shows us how important this is yeah well and I think it also shows us how much value you can get out of this, right?

38:23Like there's so much low hanging fruit. There's so much unlocked potential and it's hard to know where to start a lot of times. I've been at large companies that have a lot of expertise and a lot of really, really smart people. But if you haven't kind of been in this space and thinking about it this way, it's hard to know where to start or it's easy to make simple mistakes up front that mean like you kind of need to start over later. So I guess which kind of also brings us back to, you know, who is this book for? It's for anyone that cares about DevX, right? So definitely technology leaders, anyone who's trying to kick off a DevX program, or is working on a DevX improvement program.

39:04I think it's particularly relevant for PMs. Because if you're PMing something that involves software, building and creating software, improving DevEx will only help your team. And also you have key skills and insights and instincts that are so important to DevEx that many times I will say I've seen engineering teams just miss. Okay. What's the framework? What are the steps? Where do people start? So the book goes through a seven-step process and then also kind of provide some key kind of principles at the end. Step one is to start the journey, right? So assuming you're kicking off, you can start the journey.

39:50And this involves what we have already talked about, right? Go talk to people, have a listening tour, synthesize what you learn, visualize the workflow and tools, right? Like get a handle on kind of what the current state is. Step two is to get a quick win, right? So start small, get a quick win, pick the right projects, share out what you've done. Step three is using data to optimize the work, right? So kind of establish some of your data foundation, find the data that's there, start collecting new data, use some surveys for some really fast insights. It may include example surveys. Step four then is to decide strategy priority.

40:32Once you have some data, then you need to know of all the things that are potentially broken and you've already gotten your quick win of all the things that are left, what should I do next? And so we walk through some evaluation frameworks there. Step five is to sell your strategy. Once you've decided, now you have to kind of convince everyone else, right? So now you want to get feedback. You want to share why this is the right strategy right now. Step six is to drive change at your scale. So here we address folks that have local scope of control, right? If you're starting on just a dev team, you want to do it yourself, kind of grassroots effort or global scope of control, right?

41:07If you're, you know, the VP of developer experience or something like, there are some things that you can leverage for a top down. And then how do you drive change when you're kind of somewhere in the middle? Because you can leverage both types of strategies. And then step seven is to evaluate your progress and show value and then kind of loop back around. And I will say that we wrote this so that you could kind of jump into any step wherever you are right now, right? Like if you're kicking off a team or an initiative, you'll probably want to start at step one. You should definitely start at step one.

41:39If you're joining an existing initiative, you could jump into picking the priority or implementing the changes. So those are – so there's the seven steps. There are a few practices that we also recommend. So thinking about resourcing it, change management, making technology sustainable, and then also bringing a PM lens to this, right? How can we think about developer experience as a product? And how do we think about the metrics that we have as a product? Awesome. Okay, I have questions. Point people to the book real quick. What's the URL? How did they get it? When does it come out? Yeah, developerexperiencebook.com.

42:21So right now you can sign up for the mailing list. We'll let you know when it's out on Pre-order. And we'll also be sharing pieces of the workbook. So we've got almost 100-page workbook that goes along with the book. And then it should be out by end of year. Okay. So one piece of this is just this term developer experience feels very intentional in that it's not developer productivity, developer work. It's how do we make developer experiences better at our company, which includes they get more done, but also they're happier, things like that. So I think that's an important element of this, right?

42:55Yeah. Yeah, absolutely. Because again, it's not just about productivity, right? We talked about this from kind of the frame and the lens of we need to be building the right thing and you want to be productive, but you also want to be thinking about, and this is what engineers are also just really incredibly good at, give them a problem and don't tell them how to solve it. And then they can solve it better, right? They have the freedom, they have the innovation, they have the creativity so that they can solve this problem. If it's only about productivity, then it's just like lines of code or never PRs or whatever, right?

43:26But we really want to talk about value and how do we unlock value and how do we get value faster? And that involves, yes, making them more productive and removing friction because then they have the flow and the cognitive load and the things that we kind of talked about. Awesome. Okay. And then say someone wants to start this team. What does it usually look like? At Airbnb, I remember this team forming and it was just like an engineer or two getting it started and taking charge. What do you recommend as the pilot team? And then what does it look like as it grows? So there are a few ways to do this, right?

43:58So if you're doing it yourself, you could do it with a couple engineers, maybe a PM or a PGM or a TPM to kind of help communicate because really comms plans are just so important here. on a small scale, right? What we want to do is look for those quick wins, look for things that you can do at small scale. Are there, you know, some folks call them things like paper cuts. Are there small things that you can do to help people see the value and feel the benefit themselves, right? How can a developer's work get better? How can their day-to-day work get better? Kind of build momentum from there. If you're working from a top-down structure and you have the remit, you still want some quick wins, but those quick wins can look a little more global in scale because you have the infrastructure or the backing to make different types of changes that aren't only local.

44:56So an example of a small local change could be just cleaning up your tests and your test suites, right? Any team could do that. Any team could do that. More global scale might be changing an organization-like process that is just overly cumbersome or throwing some resourcing into cleaning up the provisioning environment. What kind of impact have you seen from teams like this forming on the engineering teams at their companies? I'll say I've seen a huge impact, right? For smaller companies, hundreds of thousands of dollars for large companies in the billions. Well, also, we need to learn how to communicate that, right?

45:34What does the math look like? Many times we can look at saving time. We can look at saving costs. We can look at a lot of different things. We can look at speed to value, speed to market. We can look at risk reduction. But the gains really are there. I will mention that it tends to follow something like J-curve, right? So like you'll have a couple of quick wins and it'll look like a big, big win. And then you'll hit a kind of a little divot where suddenly the really obvious projects, the low-hanging fruit are handled. So now we need to do a little bit of work, right? We might need to build out a little bit more infrastructure.

46:05We might need to build out a little more telemetry so that we can capture the things we want to capture. And then once we get that done, then we start to see those benefits really compound. So going back to that measurement number, what do you recommend? How do people find these numbers? Because I think that's so much of the power of this is like we saved a million dollars doing this. What do you look at to figure that out? You know, I think there are a few different things to keep in mind, right? Who is our key audience? And we usually have a few key audiences, right? We really want to be able to speak to developers because they're the ones that are going to be using the systems.

46:40They'll be partnering with you on either building them or at least providing feedback about what you're doing. And so for them, we often want to frame this in terms of things they care about. So time savings, right? If something gets faster, they can save time. They don't spend time doing setup when they don't need to anymore. related to that is reduced toil, right? So compliance and security are super important. Also, many times it requires several, several manual steps that I don't say they're not value add. They are not value add from an individual human perspective, right? If we can automate as much as possible, that's great.

47:19And, you know, improved focus time. So that's from the developer side of you. Leadership often cares about, they care about those things, right? But they always care more about other things. So we could talk about usually costs in dollars, right? So can we accelerate revenue? What does our time to value look like? What is our velocity? How quickly can we get feedback from customers? And for folks and organizations that are in really competitive environments, that can be really compelling because it's all about speed. We could talk about saving money, right? So here we can look at maybe quantifying savings.

47:56So, you know, one example is test and build, if we can clean up a test and build suite, to a developer, they really want to hear about time saved and more reliable systems, right? There's less toil because they don't have to keep rerunning tests or kind of go clean up test suites. From the business perspective, cleaning up a test and a build suite can be cloud cost savings because all of those tests are running somewhere on a cloud. And if they always fail or if it's just kind of a waste of spend, And that can be useful, right? Recovering some capacity, right? We can always talk about time and productivity gains, right?

48:32So how much equivalent developer time are we losing on things that are not necessarily value add, right? And then sometimes we can correlate to business outcomes. And correlate is usually the best we can do here. But there can be some pretty compelling correlations in terms of speeding up time to value and increase market share, for example. So let me pull that to red and come back to this. What I think is the biggest question people have right now with AI and productivity, which, and I don't, I don't think anyone has the answer yet, but I'm curious to get your take of just what should people do today?

49:07What's the best approach to understanding what impact AI tools are having on their productivity? Because they're spending lots of money on there. Like, I don't know, what are we getting out of this? And I guess things are moving faster, but I don't know. So if someone had to just like, okay, here's what I should probably try to do. What would be your best advice here for measuring the impact of AI tools on productivity? I would say it depends. And in part, it depends on what your leadership chain really cares about, right? Like we're usually pretty good at like figuring out what matters to developers, and we could communicate that to them.

49:41But if we're trying to just identify two or three data points to really kind of focus on, because when we're first starting with data, sometimes it can be challenging. What do they care about? Think about the messaging you've been hearing. Have they been talking about market share, right? Losing market share or competitiveness in the marketplace. If that's it, focus on speed. Think about ways that you can capture metrics for speed from like feature to production or feature to customer or feature to experiment and what that feedback loop looks like. If they're talking about

50:16profit margin all the time, right? Now, we always talk about money, right? Because this is business. But if that seems to be an overarching narrative, look for ways that you can save money and then translate that into recovered and recouped headcount cost, right? Or sometimes you'll kind of like reinvent, change a process, and then you no longer need as many vendors, right? So So reductions in vendor spend can also help there. And I say also it depends because sometimes something will, they'll say something, right? Like leadership will say something and it kind of comes up as a theme. If you can solve a problem that they have or it's like something that they're focused on, if you can slightly reframe it even, right?

50:58Like if they're calling everything developer productivity, go ahead and call it productivity. If they're calling it velocity and velocity is what matters to them, think about how to frame this in terms of velocity. If they're talking about transformation or disruption, right? How does this help with the disruption? Because then it will resonate with them. We don't want to make them work to understand what it is that we're doing and the value that we provide. That is such good advice. So just to reflect back, the advice here is if your company is trying to figure out what sort of impact are AI tools having on our company versus just like what does the company care about most?

51:32What do leaders care about most? Could be market share. Could be profit margin. could be a velocity. We need higher velocity or we need a transform, transformation. So your advice there is like figure that out based on words and phrases you're hearing, then figure out ways to measure that, ways to measure market share growing, profit margin increasing. So it could be, I love these examples, like time from feature idea to production or to experiment. So maybe start tracking that. If it's margin, it's like money saved by fewer tests failing or some vendor you don't have to pay for, things like that.

52:07And then velocity. Velocity, I imagine that's where things like Dora come in of just like speed of engineering shipping or what would you think about there for velocity? I would say it's actually, you know, one of those, I would pick as broad a swath as you can. So if you can go from idea to customer or idea to experiment, how long does that take? How long does it typically take and how long can it take and does it take now with improved use of AI tooling and reduction in friction, right? And that's where I will say, we talk about this a little bit in the book, you know, how do we deal with attribution challenges?

52:40So like, what was responsible for this? Was it the DevEx or was it AI? Go ahead and disclose that, right? Say, yes, we rolled out AI tools. We also had this effort in DevEx. They partnered very closely together. Both of them probably contributed to this, right? Like if we had AI tools without the DevX improvements, we probably would have had some improvements, but not nearly as much, right? If people were starting to do this today, say they're just like, I want to start measuring developer experience, are there like a two or three metrics everybody basically needs that you should just start measuring ASAP?

53:10If you're just starting today, and if you have nothing at all, talk to people, obviously. After that, I would do surveys. Because surveys can give you a nice kind of overall view of the landscape quickly so that you know where the big kind of challenges are. And I say that because if you're just starting, you might not have instrumentation through your system, all the metrics. And if you do already, it might not be what you think you want, right? Metrics that were designed without purpose, questionable. Metrics that were designed for another purpose, they might work for what you want, but they might not.

53:45So we can't just assume we have them. So that's one reason I like surveys. and we include an example in the book, you can just ask a few questions, right? How satisfied are you? What are the biggest barriers to your productivity or what are the biggest challenges to getting work done? And let them pick, you know, maybe either from a set of tools or maybe like a set of processes and then say, like, let them pick three, just three. Of those three, how often does this affect you, right? Is this hourly? Is this daily? Is this weekly? Is this quarterly? Right? Because sometimes it hits you every single day and you're just mad about it.

54:23Sometimes it only hits you once a quarter because it's end of quarter, but it's so onerous, right? And then kind of open text, right? Like, is there anything else we should know? That can give you incredible signal because by making folks prioritize the top three things, if you let them pick everything, It makes the data super, super messy. But three things and how often you can just come up with a score or a weighted score if you want. And then go kind of dig into where should that data be? What data do we need? But also then you've got at least some kind of baseline, right? It'll be a subjective baseline.

55:01But now you'll know what the biggest challenges are. I love how all this just comes back to starting by talking to people and asking them these things, which is very similar to product management and just building great products is have you talked to your customers and everyone thinks they're doing this but most people are not doing this enough yeah and i will say like one thing that's challenging when you start when you think about getting data right so interviews your data and that's important surveys are a little more quantified right because we can we can turn into counts but that's where we also want to be careful right a lot of folks go to write a survey question and they'll say something like were the build and test system slow or complicated in the last week?

55:39You're asking four different questions there. If someone answers yes, was it the build? Was it the test? Was it slow? Or was it like flaky or complicated or something, right? So it can be really difficult to untangle what the signal is you're actually getting there. And so it is worth time chatting with someone who's familiar with survey design, having a conversation with Claude or Gemini or ChatGPT around here are the survey questions or can you propose them? And then make sure you take a couple of rounds. Is this a good survey question? What questions can I answer from the data that I get? What problems could I solve?

56:19If you can't answer a question with data, don't get it. And you have example surveys in your book for folks that want to just copy and paste and not have to think about this much. Example surveys, A lot of example questions, we even recommend what the format, what the flow should look like, how long it should be, how long it should not be. One thing that I was reading is that you don't love happiness surveys specifically, asking engineers how happy they are. Is that true? If so, why is that? I don't. Now, I will say I don't love a happiness survey because there are too many things that contribute to happiness.

56:54Happiness is a lot, right? So happiness is work. Happiness is family. Happiness is hobbies. Happiness is weekends. Happiness, there are so many things that contribute to happiness. Now, that doesn't mean I don't care about happiness. I think happiness surveys are not particularly useful here. What can be helpful is satisfaction. And people are like, what's the same thing? It's not. Because you can ask, are you satisfied with this tool, right? And then ask some follow-up questions. Now, those two are related because the more satisfied you are with your job and your tools and the work and your team, it contributes to happiness.

57:30And I used to joke, remember the old commercials like Happy Cows Make Happy Cheese? How to California, that was the best. Happy devs make happy code. They write better programs. They do better work. They're better team members and collaborators. but but capturing and trying to directly influence happiness like that's that's that's not what we're here for right and it's just it's too challenging it's too all-encompassing satisfaction can give us some signal in a totally different direction uh in terms of just tools you see people using are there any that just like oh yeah this one's really commonly uh great for people this is just like a tool people are finding a lot of success with like there's the common ones copilot cursor I don't know.

58:15Is there anything that stands out that you want to share? Just like, hey, you should check this tool out. People seem to love it. I think the use, right? Copilot, Cursor, Gemini. Cloud Code. Yep, Cloud Code. I love Cloud Code. I have a whole post coming on waste use Cloud Code for non-engineering use cases. It's so interesting. For example, Cloud Code, find ways to clean up storage on my laptop. and it just tells you here's a bunch of files it's just like chat gpt running on your computer and you could do all kinds of crazy stuff on your computer for you like a little mini mini god well i'm gonna do that now this is great it's so good it's yeah that's why i'm writing this uh i had a dan shippers on the podcast and he said clock code is the most underrated ai tool out there because people don't realize what it's capable of it's not just for coding and that's i'm trying to explore more and more okay is there anything else that you think would be valuable for people to hear, to help people improve their developer experience, help them adapt to this new world of AI and engineering that we haven't covered?

59:22I think something that's important to think about in general is to bring a product mindset to any type of DevX improvements that are happening, and also the metrics that we kind of collect and capture. And by that, I mean, we want to identify a problem, right? Make sure we're solving a problem for a set of users. We want to think about creating MVPs and experiments and get fast feedback, do some rapid iteration. We want to have a strategy. We want to know who our addressable market is. We want to know what success is. We want to basically have a go-to-market function. We need to have comms. We need to get continuous feedback from our customers.

1:00:04We want to keep improving. And at some point, we want to think about, you know, sunsetting something, right? Is it in maintenance mode? Is it sunsetting? And I think that's important in general. But I think it's extra important now because when we have AI tools, we're using AI tools, we're embedding AI into our products. Things are changing so rapidly that it can be really important to take like half a beat and say, okay, what's the problem I'm trying to solve right here? Is this metric that we've had for the last 10 years still important? Or should this be sunset? Because it's not really important anymore.

1:00:37It's not driving the types of decisions and actions that I need. Before we get to our exciting lightning round, I want to take us to AI Corner, which is a recurring segment on this podcast. Is there some way that you found a use for an AI tool in your life, in your work, that you think might be fun to share, that you think might be useful to other people? So I have been kind of working on some home design and like like redecorating rooms and stuff um i'm working with a designer because i know what i like but i don't know how to get there i'm not good at this but i've really been loving chat gpt and gemini especially to render pictures for me right so i can give it the floor plan i can give it one shot of the room that's like definitely not what it's supposed to look like and then i can give it pictures of a couple different things and i can just tell it change the walls or change the furniture layout or change something and it helps me and it's relatively quick um it helps me kind of visualize the things again i know what i like but i don't know how to get there so i know if i like it or not which is probably a very random use but it's fun for now my wife does exactly the same thing she's sending me constantly here's what this rug will look like in her living room here's this water feature it's so good and it keeps getting better it's just like yeah that's exactly our house with this new rug yeah and all you do is just upload these two photos and just like well how would this look yeah i've been impressed a couple times i mean definitely the machines are listening to us um it's given me a mock-up of of a room or something and then it throws in a dog bed because i have i have dogs i'm like i did not tell you to do that but yeah that's probably the color and style of a dog bed that i should have in this room speaking of that have you tried this use case ask chatjpt uh generate an image of what you think my house looks like based on everything you know about me i have it because it is memory and remembers everything you've talked about and it's hilarious you got to do it okay that's that's on my to-do list there we go uh bonus use case nicole with that we restart very exciting lightning round i've got five questions for you you're awesome let's go what are two or three books that you find yourself recommending most to other people outlive by uh peter atia is fantastic another one that's maybe related.

1:02:49I hurt my back. So like, it's not great. Back Mechanic by Stuart McGill is incredible. So shout out to anyone who has hurt lower back. It's for a layperson to read through and like figure out how to fix lower back problems. It's kind of a random one. I will say I love how big things get done. I can't pronounce the names. I think one's the Scandinavian one is, but it kind of dissects really large projects through recent-ish history and where they failed and why. And I think it's really interesting for us to think about, especially in this AI moment where basically all of our, at least software systems are going to be changing.

1:03:29So how do we think about approaching what is essentially going to be a very large project? And then, sorry, I'm going to throw in a bonus one, The Undoing Project by Michael Lewis. matt velozo recommended it to me and it's so good yes uh i read gasped at the last sentence oh of the book it was not yeah i was not i read that and i i do not remember that last sentence oh man okay cool uh next question do you have a favorite movie or tv show you recently watched and enjoyed i will say i watched love is blind if i gotta like shut down at the end of the day love is blind is fun there's a new season out yeah i'm very excited um and shrinking shrinking Have you seen Shrinking?

1:04:07No, I think I started The Therapists and yeah, I gave it a shot. It's cute. Okay, okay. Sweet. Is there a product you've recently discovered that you really love? Could be an app. Could be some kitchen gadgets and clothing. Yeah, the Ninja Creamy is a big favorite. Did you say this last time? I don't know. Somebody said this and I still remember it. It's like you make ice cream and stuff with it, right? Yeah, you can basically freeze a protein shake and then it turns it into ice cream oh man which is delicious um another one is a jura coffee maker i'd love good coffee and i'm not great at making it so i can just push the button it'll give me anything i want including like lattes cappuccinos or anything so that's kind of sweet okay uh do you have sugar and caffeine i just need to power through the day there's the there's engineering productivity 101 one yeah oh man okay two more questions do you have a favorite life motto that you often find useful in work or in life and come back to in various ways yeah i think one that's come up a couple times it's not like a verbatim thing it's i think it's more the vibe but like hindsight is 2020 but it's also really dumb right i think if we made the best decision we could at the time with the information that we had available like that it is what it is right if If you make a bad decision because you made a bad decision and you knew better and you had the information, not great.

1:05:34But I don't think we give ourselves or other people enough grace because we always end up finding more information out later. Hear, hear. Final question. I was going to ask you something else, but as we were preparing for this, you shared that you have a new role at Google. Maybe just talk about that. What you're up to there, why you joined Google, anything folks you know. Sure. So I am Senior Director of Developer Intelligence in Core Developer. So it's super exciting and super fun because of all of these things we've been talking about, right? It's like focused on Google and all their properties and their kind of underlying infrastructure.

1:06:09How can we improve developer experience, developer productivity, velocity, all of these things we've been talking about? And because I'm kind of the numbers person, right? How do we want to think about measuring it? How does measurement change? How do feedback loops change? How can we improve the experience throughout and then kind of drive that change through an organization in ways that are meaningful and impactful and faster than they've been before? Nice job, Google. Getting Nicole. What a win. I need to get some more Google stock ASAP. Okay. To follow the questions, where can folks find you online and find your book online if they want to dig deeper?

1:06:44And how can listeners be useful to you? So online, you can find the book at developerexperiencebook.com. I'm at NicoleFV.com and LinkedIn. Occasionally, sometimes it's a mess. I try to wade through all of the noise I get there. To be useful, sign up for the book and the workbooks. The workbooks are free. I'd love to get any kind of feedback on what works, what doesn't. I always love hearing those kind of stories. Nicole, thank you so much for being here. Thanks for having me, Lenny. My pleasure. Thanks again. Bye, everyone.