Building a culture of excellence | David Singleton (CTO of Stripe)

25:00But I wasn't stuck for very long because something that the team that striked it done was we made the error messages coming out of our API Link to the piece of the docs that told you how to solve the problem genius. That is a really Good example of being ridiculous in the craft. So it is It's pretty uncommon. It's certainly very uncommon back then for companies to do this like it error messages on your API Who's ever gonna look at it turns out developers while they're building and if they're running into problems they will look at them. It's actually a really high -states moment because it's a time that matters.

25:34So that is an example of where we'd be meticulous in the product. And the way that we kind of figure out where to be meticulous, like where to really sweat the details and go up on beyond, is again, we try to be as users first, if we possibly can. So we have a process that is widely used across product teams that even like, folks building stuff internally for stripes to use, which we call friction logging. The way this works is you need to kind of put yourself in a particular user's shoes. Like it's actually important to have a very clear idea of who is the person I am kind of modeling the friction for right now.

26:08So for instance, I might be integrating the StripeBilling API and I might put myself in the shoes of an engineer at, let's say, at LassieN, which is one of the world's largest SaaS platforms and using actively the stripelling for automating all the revenues today. I might put myself in the mindset of that person, I might have met them recently and kind of understood a little bit better than otherwise, doing this particular thing. And then I will go through the process of actually using the product, starting off in the dashboard, hopping over to the particular page in the docs that I need to follow, starting to write code and so forth in order to do my integration, and make really careful and meticulous notes of what the experience is.

26:52So, stream of consciousness notes, but paying particular attention to the places where I ran into friction that I think that particular user on mental modeling would find difficult. And then those are the places where we will tend to go and invest a tremendous amount in really thinking, how could we solve this problem and then putting a lot of attention into detail into making it right. And by the way, the example I gave you around the error messages, there's actually more code in the jobs that start the Stripe API to handle those edge cases than in the actual main flow. And I think that's quite remarkable.

27:28Most people wouldn't do that, but it turns out not only was it something I was impressed with, but when I talk to Stripe users, this is very frequently something they tell me and delights them about the product. It's a reason that they can adopt it more quickly and it's a reason that they keep adopting other Stripe products because they know it's going to be easier than doing anything else. So it really matters. So the point is like we are intentional about where the places where being particularly meticulous matters and then when we are we try to do it in quite a systematic way. It's great business as well.

27:57So for instance We actually just earlier today put a blog post up that explains we've been shooting our both what we call our payment element, which is our kind of More batteries included UI for putting payment integrations directly on your website. You can style it to fit with the rest of your page, but it has a lot of powerful features, and we also have a set of hosted services called Start Check -out. The point here is, we've gone and scarred a whole bunch of check -out flows on the internet over the course of many years, and we look out for all of the little broken edges, and we've been working hard in these products to tune those.

28:36And then, not doing just stuff there, we've also been really obsessing about what are the things that we can do in the flows that we've already built that will, you know, move latency or remove a click or whatever. And what we find, we've recently measured the difference between, so with a bunch of actual users who've migrated from a fairly vanilla integration where they, you know, built their own checkout flow to one of these surfaces, to our payment element or to strike checkout. It turns out that it increases the average user's revenue by 10 .5%. And that is huge, right? So this is an industry where little changes that you might go about making, or even like big changes that you pour blood, sweat, and tears into will typically deliver uplift in that you measure in basis points.

29:23That's hundreds of a percentage point. And this set of small changes come up in over time, letters up to 10 .5%. It's huge. And the point is, you only get there by knowing that this is a thing that might ultimately matter and being meticulous in every single step along the way, and that it compines to a dramatic impact in the end. So I think that particular set of experiences is one where this operating principle has really mattered. Thinking about those kind of flows as what led us to build link, which is our product for making one click checkout available. And that has had a tremendous impact, both on convenience for customers of our users and also on their conversion rate and so forth.

30:04So it really matters. And then one final other area, if I may, I think Stripe is relatively well known for putting a lot of Karen attention into our website, a lot of our marketing or product explanation pages. And I think for a business like ours, it's not immediately obvious that it makes a tremendous amount of sense to be super meticulous in building those experiences. But we are. And again, I think it really matters for us and for our users. So the way that we got into this mode was we recognized that we are building a lot of infrastructure. So we build what we call our global payments and treasure network.

30:45It is literally our product infrastructure for moving money into stride, holding balances and behalf of our users, moving money out, all of the automation between moving money between different businesses and all of the kind of regulatory stuff we have to handle. All handle in this big payments engine. But as a user, it's very hard for you to come in and inspect that and say, was it good? I mean, hopefully you won't talk too much about other type users, but it's very hard to know from the outside. So if we put a lot of care for detail and attention into the experiences that we used to explain the value and the power of that, it actually helps our users understand exactly how it is that we are trying to operate for them.

31:24And that's where all the meticulousness that we put into our site have come from. There are lots of features like, you know, we have a spinning globe on our page that shows you what payment methods you can use in different countries. We have this big animated wave on our homepage. And you know, in general, Stripe is kind of known for these, you know, great internet moments when we launch products. And so we did it because we wanted our users to be able to see the care and attention that you were taking. It's turned out to have this wonderful side benefit to some extent because those experiences tend to get shared a lot I go on Twitter and LinkedIn and elsewhere because they're worth looking at just because they're beautiful And they you know often push the web platform forward and that then means that folks are likely folks who themselves want to push Internet business forward are likely to know about stripe and no other capabilities.

32:11So meticulousness plays with impact in so many different ways at stripe and I enjoy working that way in many different areas. I have a few follow up questions along those lines, but before we do that, I want to double click on this friction logging process. Just understand how you actually do that. So maybe two questions there. Is this like a regular task people have on their plate? Like say, there's executives or just PMs in general. And is there like a template that you share just like you talked about? Who are you? What company are you working for? What problem are you trying to solve? How do you actually kind of tactically operationally that?

32:49Yeah, great question. So it's a it's a practice that is quite valuable Generically there are many there are many places where you can apply friction logging in order to Really kind of shine a light on what is the most effective place to invest time and effort? So we use the practice across lots of different functions and in lots of different ways It's right, but it is the case that almost every product team has someone, it's often the PM, someone is the engineering manager who has a regular repeating loop of going through the end -to -end flow of the product and writing a friction log. For years and years I have gone through the process of onboarding as a user to strike once a month, writing a friction log, and then tagging in the right people across the company that might need to take action on some of the things that we're observing.

33:37And one of the reasons that that kind of process of taking a big step back is useful is at this stage we have many people. We have thousands of engineers who are working in parallel and while everyone cares a lot about being meticulous, paying attention to detail, getting the details right, can have fun with experiences that diverge and going through this process of looking at it all together on a regular rhythm with friction logging really helps us maintain a cohesive whole while we all operate in parallel. And so therefore, senior leaders, executives, I guess, of our larger areas will often do this as a practice themselves in order to kind of make sure that everything that they're responsible for is coming together well.

34:17So it happens kind of recursively at different layers. So your question is there a template? So relatively simple process. So yes, there is a template. And in fact, there's a striped up data article that we put into the show notes that has the template. But it literally is, say what you want to do. Be very explicit about the user that you are trying to have a mental model for, because that helps you make the right choices as you go through the flow. And then really just keep a very clear assume of consciousness log of what you're finding as you go. By the way, also really important, praise the stuff that's good.

34:49So I'll often set these docs around to a lot of folks inside of the company. It's a great opportunity to recognize great work done. So if I ran into that error message that it's like this is our presentation. It's a great example where I'll be like, this is awesome, nice work. Being a PM that has received emails from the CEOs of companies with all the problems they ran into in their training book, it's often like, oh God, I have like so many goals I got to hit and I have this timeline and I have these things I'm working on. What is the culture like slash? How do you give team space to do these things that are just like, we're just gonna make the experience better, how do you actually do that?

35:23So that PMs aren't stressed out when they get these things. It starts with our operating principles. So because we have operating principles of users first and being meticulous in our craft, we actually tend all of our ways of building planets tend to be wired quite well around this idea that we're going to reserve enough time to really make the experience good. Maybe the most pronounced version of that is not even the issues we identify in friction logs. Maybe we can talk about how we invest in reliability and so forth, but things go wrong. And one of the things that's very important to me is that we are a learning organization.

35:58We need to understand why they went wrong and then take action to stop the same thing going wrong in future. And so that means that we identify instant remediation. That's what we call those. And we prioritize those carefully and most of them, the ones that matter actually get prioritized before other work on the roadmap, which means that as a PM, when you're building your roadmap and your plans, you do have to think about, well, approximately how much bandwidth with, I need to reserve in this area in order to address polish stuff that might come up through friction logging and also to take care of those kind of operational things as well.

36:30It varies by team and varies by stage of product. So there is no stripe kind of like we will reserve 50 % of our time to do this stuff that wouldn't make any sense. But we do encourage and ask every team to think hard about how much should they be setting aside for those kind of activities. Does that make sense? Absolutely. I was going to ask you if there's a percentage and basically trust the teams to set aside the right amount of time. That's right. Awesome. Okay. So sort of along the same lines, I actually asked Shrias Dochi what I should ask you. Oh cool. No, I was going to do this. And what he said is I should ask you how you did UX reviews, pre -shipping right after you joined Stripe.

37:11And he just said, you were very good at this. And so what does that like and how should people learn from your experience? The way that I did UX reviews very early on in my time, it's tried the way that every group at Stripe does these reviews is employing some of the techniques we talked about already. So that active friction logging, it's very often done asynchronously. So someone will sit down and reserve an afternoon and go through the product and make particular notes. And that's your previous fault. We also find it very valuable to go through that process together. So we will bring together both the team that built the product that we're looking at, but also a lot of their cross -functional partners, so things like the support folks that are going to be handling or kind of building the process to handle questions from users about it, executives in the org that they're a part of to do these walkthroughs where we're taking exactly the same approach as I described with friction logging.

38:07So imagine we're a particular user and experiencing the product together. And then just discussing very openly, we actually, the way that we do this today is we have a log of issues that we want to talk about that anyone can type into while the PM usually is walking us through the flow. And then we'll get together at the end and actually talk about each issue and, you know, does this need to be addressed? What do we really think about it? So that's the approach both that I see early on and that we take in general at scale today. There's a lot of benefit to doing this together. So someone wants to describe this to me as if you were a chef, you're going to taste the soup and you're going to talk to your kitchen staff about what went into the soup when it was good and wasn't so good.

38:49If you run a movie studio, you're going to sit down and watch a lot of movies together. And so actually kind of looking at product together, I find extremely energizing and also really valuable in actually teaching some of the kind of bar for craft and values or how we want the product to interoperate for our users at scale across the company. And building on that, there's something that we do occasionally, and I've done occasionally it's called Walk the Store, where we'll actually do that process of like looking at the product together with the whole company. So we have a weekly meeting called Friday FireSide, where you don't have to attend, but majority of the company does attend.

39:31And we've done a few series where we'll actually to go through some really interesting, critical product flows together and talk about how this is reflecting various priorities that we have and shifts that we're trying to make, but in particular, with the user experience and a particular user at the core. And that turns out to have a tremendous amount of benefit and value in helping everyone have a shared language for talking about things and get on the same page about everything. So yeah, those are some of the techniques that the Trace might have been thinking of. But it's amazing how many directions you all come at to create this high quality and product.

40:08Walking the store, friction logging, these entire company meetings, looking at the product, UX reviews, this is how it's possible to build something so high quality, right? It's not just wave this value and we're going to be building great stuff. It's like if they come at it from so many places. Yeah, that's an interesting observation. I think almost anything that you talk about as a value, I mean, it being a value matters, but you need to have a practice behind that. That means that the value becomes real for everyone. So we very frequently find that for any product development team, so long as they are, they identify the right metrics that actually represent the experience that they want to deliver for their users.

40:49And then get together frequently, like predictably, to look at how the team is doing against those metrics, but then everything else just happens naturally because you understand what you're trying to move and every little micro decision you're making within your own time prioritization and then in terms of the actual work you're doing, we'll ladder up to that. And so what I said, it is like you have to have the predictable and regular thing that ladders up to the value you're trying to deliver in order to make it happen. Coming back to the UX review, just a question there. Presenting to the CTOs often very stressful in a review like that.

41:24What advice would you give pms and designers and just leads of a team for how to prepare for a meeting like that, whether it's Stripe or anywhere? I personally try to be as friendly and unscary as possible, but I know that no matter how much I do that, these can't be high stakes meetings. at Stripe, and I think it's probably most from most companies, but at Stripe, the main answer here is put your users hat on, and if you understand your user well and what they're seeking to get out of the experience, and anchor any questions you get asked or levels you may feel, like, that was an out of the blue question back to, well, here's remember what we're trying to do for our users, that usually makes any meeting like that go really well.

42:10So that would be my main piece of advice. So I mean, a risk that exists in any company as you get to run a bigger team or whatever is that individual contributors, individuals will have very small amount of time with you in general. So there's always a risk that you might make a fairly unimportant remark and it will be taken out of context to be for important. So I personally also try very hard to anchor, feedback I'm giving in what's the user experience for trying to deliver, does this actually matter? Recognizing that that is a risk. And yeah, it definitely takes constant practice. You've touched on a couple of these things, but just if someone's trying to get better at building product, either a PM designer engineer, what kind of advice do you often give for helping people just build better product.

43:02Almost always goes back to things we've already talked about here. So remember, the very top I talked about iterating very closely with users. If you have a mechanism to listen to users, get something in their hands quite quickly and then get their feedback on it to run it back through a feedback loop, you're very unlikely to go raw. Especially if you put a lot of thought into these being the right users to whose needs you wanna focus on because they lack up to your strategy. It's actually very hard to go wrong if you have that feedback loop working really well. So if you don't have that feedback loop, and actually it's remarkable, as I talked to folks across the industry and so forth, it's remarkable how frequently that feedback loop doesn't exist in product development cycle.

43:43So if you don't have that, go figure out how to create it. And if you do have it, but you can't get something into users hands very rapidly from you having the idea that this might matter to getting their feedback on it, figure out how to make that go faster as well. And it's right we focus a lot on all of our developer tooling and infrastructure on making it possible for changes to get delivered continuous into production so that we can get them in front of users very rapidly. And that's really important. This episode is brought to you by BrainTrust where the world's most innovative companies go to find talent fast so that they innovate faster.

44:17Let's be honest, it's a lot of work to build a company. And if you want to stay ahead of the game, you need to be able to hire the right talent quickly and confidently. BrainTrust is the first decentralized talent network where you can find higher and manage high quality contractors in engineering, design, and product for a fraction of the cost of agencies. BrainTrust charges a flat rate of only 10%, unlike agency fees of up to 70%, so you can make your budget go four times further. Plus they're the only network that takes 0 % of what the talent makes, so they're able to attract and retain the world's best tech talent.

44:52Take it from DoorDash, AirBeamB, Plad, and hundreds of other high -growth startups that have shaved their hiring process for months to weeks at less than a quarter of the cost by hiring through BrainTrust Network of 20 ,000 high -quality, vetted candidates ready to work. Whether you're looking to fill in gaps, upskill your staff, or build a team for that dream project that finally got funded, contact BrainTrust, you'll get matched with three candidates in just 48 hours. Visit usebraintrust .com slash Lenny or find them in my show notes for today's episode. That's usebraintrust .com slash Lenny for when you need talent yesterday.

45:28Something that I've heard about you is that you get into the weeds. You get really deep as a CTO of a company at the scale of Stripe. I came across this term, Engineer Recations, which I think connects to this. Can you talk about that and how you think about just getting in the weeds? First of all, in terms of how much getting in the weeds matters, at Stripe we find that it's very important for engineering managers in particular, but really all managers, to really have a very detailed understanding of whether their teams are on the right track and whether they're getting stuck in order to make sure that we make the most progress in unit time for our users.

46:04So we find that, again, the nature of the problems we're solving really reward domain expertise, like assimilating at the understanding. and we ask all of our managers to be kind of the editors in chief of their teams plans. And I think the only way to do that is to get the right look for yourself to understand what is actually going on in the ground. No, it would be really damaging if I was only doing work, you know, the IC engineering level every day, like that doesn't ladder up, someone who's able to like help guide the team or whatever. So it's important to do judiciously, but done on the right cadence, I think it's very, very valuable.

46:39So, engineering Cation is something that I personally do. So, it's obviously a portmanteau of engineer and vacation, but it's not a vacation. The way we get to the name of the trailer, or at least the way I think about it, is what you do on engineering Cation is I'll clear, and other people abstract it as well. I'll clear several days in a row, three or four days. Actually, join a team, pick up a small task, hopefully a small feature that we can get all the way from start to finish, like in production, and do that. going through the exact experience the team has. So you get to understand the developer tools, the build infrastructure, how you get stuff reviewed, how good is the documentation?

47:15How long did I have to wait in order to actually see my thing live in order to start that feedback that I talked about with users? So you're actually going and joining a team and doing some work. While one does that, it's really valuable and important to keep a friction log because then what you want to do is right up the experience, It's both as an aid memoir for yourself because it then helps you go and for all of the next years' worth of conversations I might be involved in around making trade -offs and helping set parties, that context really helps. I actually reread these reports periodically.

47:49It's also really valuable to actually share the work with the team, which then demonstrates what I understand what it's actually like. Here's how we're going to make sure that we're carrying that information through the departing. That's what you do. The name is, it's often the hardest thing about these is finding the time to clear your calendar for them many days. So the reason I think about it as like a vacation is, of course people go on vacation. By the way, I work very hard to use all my vacation days every year. I think that taking a break and recharging your creative juices is valuable.

48:18So the point is like when you're on vacation, the world goes on and it's all fine. So I treat it like I'll decline every meeting in order to clear my schedule and have a maker schedule in order to do that. And so I've done this many times at Stripe. I advise engineering managers at Stripe that they should do an engineer occasion sometime in their first quarter to six months. The company gives you a tremendous amount of context for what your teams are actually experiencing. And then for folks who do it once a year on an ongoing basis, and it provides a tremendous amount of context. That is insane.

48:50I've never heard of that process. How do you stay knowledgeable and up to date on the infrastructure and the code You have to code and all of a sudden. I think it's so fast. The process is literally to help you do that. I guess maybe something I left out and explaining it is, we'll often identify a buddy on the team who's gonna help you through. So for what it's worth, let's try it. We write a lot of code in Ruby. We write a lot of code in other languages, like Java, TypeScript as well, but our core infrastructure is mostly implemented in corporate infrastructure, most of the internet implemented in Ruby.

49:24When I started, I had never written a line of Ruby in my life. And I knew I wanted to do this thing and actually done something like it before joining. And I was really scared. Like, what kind of, like, how much credibility might I lose if I show up and I can't write a line of Ruby, but my first buddy was a guy called X -Shade, who's still trying to do great stuff. And he helped me learn Ruby during these three days. Yeah, they rid me a little bit, but ultimately everyone really appreciated that I've done it. Imagine being the engineer has to review your PR. I tell them this is an important part of the process.

49:58Please don't treat me any different to anyone else. And obviously, to make this work well, you have to set the expectation that you're not going to like take me action if something isn't exactly what you want it to be. It's a moral kind of longitudinal process. Is there something that you found in this process that comes to mind that was really surprising or interesting or just kind of lasting in the past that in a couple years? One of the most interesting things that I think I've run into is there are a lot of places that as we scale, we know that it makes a lot of sense to invest in automation.

50:28There were a bunch of places in our development process where while we've done that, we were working hard to help each other out and actually saying, hey, please come talk to this other team if you want to use this thing. If you're working, for instance, at a different time zone from someone else, it's often the case that actually it would be much better just to document it well and automate it and then maybe make the consultation available async. And so running into those kind of pieces of friction, and then having a good conversation about it has really helped. But one good thing about Stripe, I think, is that kind of change doesn't depend on me doing anything.

51:02So we really care about making Stripe a place where engineers can do some of the best work of their careers. And that means being enabled with good tools and good developer productivity. So we invest a lot into developer productivity. that we have a team who work on our dev tools. And they actually run exactly the same product development process that I just described for our external users internally. So, understand your users really well. Talk to them a lot. So for instance, we do a monthly survey. We operate enough skill in our where we can ping enough people once a month that we get a statistically significant sample of the entire organization, but I probably to get everyone to respond to every individual a new response by once every six months.

51:43And we asked very directly, what's the experience you're having? And then we use that to prioritize the roadmap of our developer productivity team. We also measure everything. So we know where people are getting stock. And when we compare both free responses and the data, it helps us invest in the best places to make everyone else more productive. OK, that's exactly where I was hoping to go next. And let me set this question up. So I saw a tweet of yours recently where I have some notes that's here I'm going to read, that you deploy changes to your core API 16 .4 times a day on average. Your uptime has been 99 .99 % and 1 in 10 people in the world have transacted with a business -powered by Stripe at this point.

52:25And so my question is just what does it take to achieve something like this, especially in terms of tooling and culture and everything else you're just talking about? Well, I think it's worth saying that Stripe, we are trying to do something relatively unique, which is what we power for the businesses that build and striper users, it's really business critical to them. We're literally talking about money coming into your business to help you run your operations or make it even possible for you to run your operations and maybe pay your employees and so forth. All the revenue automation that we do on billing subscriptions are financial automation products helping to close your books like this stuff matters.

53:03You cannot care about it. And we also operate, strike today, we move as much money as all of e -commerce was when strike got started. So we operate very significant scale. And so business critical, significant scale, we just have a tremendous duty to our users to get this right and be extremely reliable and available for them. So we do think about that a lot. Now, there's one way to be very reliable, which is to try to change things as infrequently as possible. Never change anything, then you have many fewer opportunities for things to break. But we don't take that approach. The needs of our users are evolving so rapidly.

53:42The number of ways that we want to serve them and can serve them is evolving rapidly enough that it matters a lot that we can operate, constantly changing environment. Hopefully I've explained why that matters. That feedback that the users only possible if you can actually get something in their hands. So we choose to design the way we work to hold those two things true at the same time. And it does take a lot of care and attention and it takes a lot of systems. So maybe you just kind of build this up. First of all, there's a lot in our development process and the ways that we take changes into our product that makes this possible.

54:20One of those is we really care a lot about having really good test suites. So we believe in automated testing. We don't have manual testers because manual testers couldn't possibly cover the vast array of API endpoints and configurations that we offer, but automated test can. So we work hard to have a lot of automated test coverage. And then every single change that an engineer produces gets run through this battery of tests before it can even possibly go towards production. And then we work very hard as changes end up in production to put them through increasingly realistic and then more broadly exposed environments.

54:59So we have a bunch of staging environments where we'll send a battery of more realistic end -to -end tests. Then finally, when something actually goes to production, it starts at very small percentages of the traffic and then wraps up to the whole. So we can detect problems before they become huge problems. There's a number of things that we had to do to make that all possible. So for instance, every change that Stripe Engineer submits once its master's test it actually ends up in production automatically over the course of the next 45 minutes or so. And that, I don't think there are a lot of financial services companies at least not until maybe the last few years that have operated that way.

55:36And so that takes a certain in -mind shift, you actually have to assume that that's going to happen and put the right systems and processes in place. And then the other thing that's important is to recognize that we just, we have to of SASS about getting very systematically good at addressing anything that can go wrong. So I mentioned earlier, it's really important to me that we are a continuously learning organization and something else that matters of course is things will go wrong. You know, sometimes there is a downstream partner where something breaks. Other times there is, you know, a particular kind of network audit sort of our control or whatever.

56:12So things will go wrong. So it's important that we have the right systems in place that minimize the the damage that any individual thing going wrong can cause. And we work hard on that. We've redundant systems in a bunch of places. We think hard about how something that breaks for one user wouldn't carry over into other users. And then we actually very actively work hard to put things right when they are rough. So that's called instant response at most companies, including Stripe. I actually think Stripe is very, very good at instant response. We've got very good tools for both declaring incidents and then pulling the right people together to put them right.

56:45But we don't stop there. We really obsess about reviewing them carefully and identifying not only like what would have stopped this thing happening But how could we prevent this whole class of issues in future and as I said earlier we prioritized working on that stuff Ahead of anything else on the roadmap that's because of remember what is we're trying to do how important it is and how if we don't do that Well, we can't move quickly for our users So that's how we do it But I don't want to I want to kind of come across here and sign like we've got it all figured out of course all of this is always entirely in flux.

57:16We're always anxious to figure out how we can make it go better. For instance, in recent years, we realized that we could get a lot of benefit from what we call chaos testing. That's like injecting errors and making sure that the system's responded such way that it doesn't cause any impact on users. So it's constantly evolving and we really enjoy learning from other companies and learning from our users as well. But it's something we care about as much about and think very rigorously and systematically. the only one. Did you say that it takes only 45 minutes from pushing code to it going into production?

57:45Typically, yes. So we, that battery of tests that I described that gets run on every change. That gets run in parallel to when you send it for review by another human, so it's run once typically takes about 15 minutes. And then once the changes merge into our code base, we run that same test suite one more time, so another 15 minutes. And typically takes about 30 minutes for the systems to automatically deploy in production. That's why we run. And think about establishing that tight feedback loop with users. That means you can get feedback from a user in the morning. You can figure out how to address it.

58:19And you can actually fit something back in their hands by the end of the day. And that loop running inside of 24 hours I think is pretty important. That's incredible. A remember times that are being de -worked took hours for tests to finish because there's a lot of them. And they fix that over time. but I'm used to more on that scale. It takes constant effort to hold those numbers, by the way, are important. Like those are about the right numbers, I think, for a company operating our kind of scale. Of course, as you add more stuff, you add more tests. So we've had to work on, we've had to invent mechanisms to make it run more in parallel.

58:53You know, I have a thing called selective test execution where we figure out for the, not the battery that runs before it goes into production, that we run everything and just run it in more machines to make it parallelize. But for the changes that we run for individual changes, we actually have systems that fear all, which tests are the ones that might be material for this particular thing. And that's been a real source of innovation. In fact, our distributed change and test environment that's right is the biggest distributed system that we actually run. And so running that is, you know, requires just as much energy and effort and rigor and craft as everything else.

59:29What are some of the things you've done that have had the most impact on developer productivity. We've touched on a couple of these already. That auto -deployed mechanism definitely had a very significant individual impact. So before we introduce that, so that sometimes within the last five years, each deployed to production required an engineer to actually babysit it, to watch it as it went on, it makes sure all the charts look good. And it was a bit of a kind of game of relapse because you always wish that your change would get bundled in with someone else's so that they were paying attention to it for you.

1:00:01So literally being able to be like fingers on keyboard and something else while your change is going live and it's automatically monitored, that was a big improvement in our productivity. Also like very small things, this is just like optimizing check -out. Very small changes really compound in a big way. So for instance, by eight months ago, we made it possible, we got a, we used a tool for code review. It's just quite popular as GitHub enterprise, but we built a bunch of our own experiences and controls around that. We have little tick box that you can take on your pool request. That's what you call an individual change that an engineer produces.

1:00:36That is auto merge, which means once the reviewer says, this is good, I don't need to come back and look at it again. The system should just take over. And that just cuts out one whole kind of human distraction step. And that laddered up to a big change in productivity as well. So these very small changes can, if you're deliberate with them, ladder up to a big impact. I'm going to go in a different direction now. AI, very hot right now. Yeah, has AI impacted the way you all build product yet? And if not, where do you think it starts to go in the next few years? Yeah, we're very excited about AI.

1:01:09Now just take one step back. Stripe has been using machine learning and advanced machine learning techniques at the heart of our products since the very early days. Radar is our solutions for payment fraud. And it has used, I mean, is the very core of our product and has used machine learning since we introduced it. We also, if you think about the nature of Stripe, we operate an environment where we have to do a lot of work to catch bad actors in the system, fraudsters, or fraudulent businesses. So we've been employing a lot of machine learning techniques there again for years, and we couldn't operate the company without them.

1:01:44So the difference between ML and AI, I think, when we think about AI today, we're mostly talking about the applications of large language models, but those are really based on this new technology called Transformers. It was a paper that came out of Google a while back, but four years ago, five years ago. And we put transformer technology into those systems at some point more than a year ago. Now, it's proved to have a very profound impact in our ability to solve those problems for our users, which is great. But we are also excited about the applications of large language models. I mean, there's really two dimensions to this.

1:02:17Number one, we feel privileged to be able to help serve and power a lot of businesses getting started in this space. One big difference between AI startups and others is it's actually quite expensive to operate an AI startup in terms of the compute resources like running this stuff in these GPU machines is quite expensive. So typically these companies need to have a monetization model from day one and I'm really excited to say like most of the AI on these is running on -stripe and we're working very hard to make sure we serve all their needs. Open AI, for instance, is using us for monetizing a chat GPT plus and all their other products, but they don't just stop there.

1:02:58We're actually helping power all of their subscriptions and revenue tracking and financial operations around the business. And the point about that is, we've been putting very large engineering teams on this stuff for years, and that means, you know, anyone who wants to build a reconciliation product and the subscriptions business model. If you want to let yourself use a bit, you know, big engineering teams on them. But if you can use Stripe instead, you can actually focus those really precious engineering resources on keeping up with the rate of, you know, breakneck innovation in the space. So we're excited about that.

1:03:32But we're also really excited about the applications of these AI techniques on our own ability to serve our users better. And so, for instance, at Sessions, we'll be talking about a few of these. We started working with OpenAI when they had the data. It was a private data of GPT -4 available at the beginning of this year. The first thing we thought of was we have a lot of documentation on Stripe. We put a lot of care and attention to making it good. But if you want to achieve something with your Stripe integration, you're typically going to spend a lot of time reading docs and synthesizing them as an end user.

1:04:06We realized we can have GPT -4 read all of our docs, store them as embeddings, and then answer questions for developers. And so we now have that available in an early stage release inside of our documentation. It's turning out to be pretty valuable. We've also been able to apply these techniques to other areas of our product. So something that we're going to show an early version of at sessions is we have a really powerful product as part of our revenue and financial automation suite, which we call Sigma. It allows you to write SQL queries across all of your Stripe data. So you can like interrogate your scripted to understand your business in really fine grain detail, you know Which country has the fastest going sales which segment in which country has you know particular attributes to it Very powerful product, but it's one that is Potentially challenging for non -developers to adopt like you do have to know how to write SQL queries Unless you just use the ones in the menu in order to get the full value of the product well It turns out with large language models We can apply this technology where you can ask questions in natural language and are engine will actually write the SQL query for you.

1:05:12And we've had to do a lot of work to kind of tune that to make sure that it's reliable and understandable, and it's going really well. We also see great applications in actually applying these technologies to make us more efficient internally, answer users questions faster, help each other out more quickly. And so we're doing those things as well. One concrete thing that we did, which I'm pretty excited about, is not long after we saw a chat GPT come out, We realized it would be really awesome if we could apply that to many use cases inside of Stripe. But as you can imagine, a lot of the data that we are dealing with in a daily basis is very sensitive to our business or users, and we care a lot and we have a lot of governance around this.

1:05:56But we wanted to make that technology bill. We couldn't say, hey Stripe, just go and use TrackEpt. So we stood up an integration to GPT4 and an internally UI to use models like that. But here's the thing that I want to kind of relate to all of your viewers. We find that we built presets into that. So when you work with large language models, you want to write a prompt that then helps get the model into a state where it's going to be able to solve the particular problem at hand. And we find that writing prompts is something that's accessible to folks across lots of different job families, not just engineers, folks in our marketing team, folks in our user support team who've been able to use this as well.

1:06:32But you put a lot of care and attention to getting a prompt that works really well. We not make it possible for those to be shared across the company. So I can grab the preset to help me write a blog post with the right kind of tone. And I find it extremely valuable in helping me and many other people across the company. So I think there's a lot of innovation in high companies. Server users and operators is possible right now. and we are working hard to employ all of that on behalf of our users. What about as engineers, do you, are you pro -copylet? Is there some along those lines you find useful?

1:07:09Where are you of? How do you think about that? We're excited about copilot. We've made copilot available to our engineers in Charlie. We've had a fairly rigorous trial to make sure that we felt good about it's actual impact on our ability to write code, and the code was actually generating, and find that it was very effective for us. Both in productivity, but also just in helping our engineers feel like they could direct their energy towards some of the bigger problems, like how should the stuff fit together rather than some of the micro problems? And so because we had such great feedback in that, we've made it available very broadly internally.

1:07:51and we'll be very excited to see where that space develops over time. Do you have a sense of the productivity gain that say, imagine you have a lot of 10x engineers, the gains that an amazing engineer gets out of a co -pilot versus, you know, newer engineer? It's honestly too early to tell because we've only recently made it available at scale. What we're typically seeing is that the kind of tasks that it accelerates are, remember I mentioned how much energy we put into having comprehensive test suites. It's actually using Copa to generate your test cases, turns out to be extremely valuable. There's a reason I'm on a boilerplate that is repeated code and concepts that goes into writing good tests.

1:08:33But you also have to reason very carefully about is this actually testing as I go out of one. So Copa takes a lot of that boilerplate generation out of the way, so you're actually thinking about the stuff that really matters. And I think that is going to have a profound impact on quality of code that we write as well. But it's a little too early to say because we literally just hold it by what kind of actual measurable impact is having. I was talking to an engineer and he's just like, I like the active writing code and it makes me sad that this is doing this for me. So he turns it off. But I think over time people will be like, yeah, let's try and never really enjoy this part of it.

1:09:07I mean, I'll speak from personal experience. I love writing code. I really love writing code. It's pretty much what I do for fun. I also love running code with co -pilot because as I said, it means that I can focus on the parts that really matter. And I think that's been a fairly consistent experience across Stripe as well. Awesome. So you run a massive engineering team. I don't know if you share the numbers of engineers, but I know there are many. What are, and this is a broad question, just what are some lessons you've learned about managing people and or managing engineers, whichever direction you want to go?

1:09:40Wow, big question. There are so many different directions we could take that. So I'll share a few observations. They don't necessarily all have a particular theme to them. I think, as I've been responsible for bigger and bigger teams over time, one of the things that just repeatedly learn is I personally will not be involved in really any of the decisions that matter and happen. There are thousands of decisions that an organization of any skill is making every single day. Every individual engineer or PM is making hundreds of small decisions and some big ones every day that affect the general trajectory.

1:10:16So the most important thing is to really focus on hiring the right people. And that means hiring people that you can trust with a tremendous amount of autonomy. If I try to get involved in lots of decisions, everything will grind to a halt. So how do you do that? It is important to be rigorous. We've described already what our hiring process is. I really lean into getting to know folks in very well as we're hiring them. By the way, something I didn't mention earlier that is, I think, important here is we pay a lot of attention at strike during the hiring process to references. This is typically later in the process, but if you put someone through an interview process, you probably spent what, like, eight hours total with them.

1:10:56If you talk to people that actually have worked with them before, you're probably benefiting from thousands of hours of direct experience. So like we take this very seriously and we try to ask you know, smart questions that will help get real signal out of that. And I've certainly found when it comes to hiring folks that then go on to really, you know, make a transformative impact that references are often the time that you get the best conviction on the hire or not. So hire the right people and then you have to trust them. Well trust is interesting because it's actually quite challenging to trust people that you've just hired, right?

1:11:28Well, you've got no stake really on you yet. So what I find is you do actually just have to be generous in giving trust and assuming trust by default, the beginning. But then you do need to, you know, hold people accountable enough that they are proving that they, you know, they can't handle that trust. So, you know, sometimes that means hiring someone who you think quite plausibly can end up in a very big role into a smaller role to begin with. Other times you just have the big role you need to fill in which case, It's important to be quite conscious of it really trusting and delegating to them with you know, love of support so I'm always working hard to Delegate sometimes like a little bit more than uncomfortable with because that's the only way to like really operate at significant scale I think something else.

1:12:14I'm sorry. This is really a smorgasbord. It's more like a perfect and something else I've realized as the organizations like work with and being responsible for have grown is that you have to be extremely conscious about managing your own time. There are hundreds or thousands of people who, like, quite legitimately might want to get some time with you. And, you know, if you just let your, the places that you spend your time and energy be controlled by the stuff that comes into your inbox with the stuff that gets, you know, put on your calendar, that's going to be random. So it's quite likely that that's not lettering up to the biggest impact.

1:12:48So I personally have been running a look for, I don't know, maybe like 10 years where Sunday evenings usually starts in Sunday afternoon, I read a lot of what happened last week and then Sunday evenings actually write a list for myself of you ask me a question sort of this podcast which was like what would what would like be great success for this podcast from your perspective. I basically think about that for my week every Sunday night and make a list of if we got these things done this week that is a good week and obviously we need to be dynamic like things will surprise you through the week but honestly then that that drives a lot of just where I decide to spend my time.

1:13:23And I try to encourage everyone to think that way through the organization. And I think it does let her up to more impact over time. Think the final thing I'll say here is, I mentioned how important operating principles are to the culture. I think for any manager and any leaders, that includes ICPMs as well. But for any manager, any leader, like how you show up really does set the culture that is around you. So I try very hard to show up very consistently every day, even though there will very frequently be things that are going wrong and are bad and are hard. It's actually very important to show up quite consistently, certainly, you know, in line with all the values that we have and the culture that we want to set in order to really kind of model that at scale.

1:14:09And, you know, different days that can be easier and harder. So the final thing is I think it's quite important to manage your own energy. There are some tasks that I do that are maybe not the most important thing, but I know that I get a lot of joy and energy from them and then that carries over into the other stuff that that also needs to get done. Just two more questions before we get to our very exciting lightning round. One is just I feel like Stripe is incredibly good at planning and organizing, advertising, I know every inside, it's always not as beautiful as it may seem on the outside, but what have you learned, what a stripe learned about planning and advertising that you think other companies might be missing?

1:14:50Yeah, it was funny you say that. I would say planning inside of stripe does not get the highest net promoter score that you might imagine. I do think it is actually quite effective. And by the way, the reason I think that planning, the way we do planning it stripe doesn't always get the best internal wrap is we have grown very rapidly. And that means that almost every time we come back around to planning, we actually, and this is kind of a theme strike, we think about a lot of the ways that we work internally from first principles. So we will tend not to, by default, just pick a system off of the shelf that some other company has run or that we've used before will think hard, well, what, considering what we're doing for our users, how should we do this?

1:15:33And when we're thinking even first principles, we'll often go out and talk to a lot of other companies and try to learn from their experiences. Actually, something I've really appreciated and enjoyed at Stripe. A lot of the learning I've done here has come not only from folks that are at the company, but also talking to a bunch of folks that we have the privilege to work closely with because their business is building on Stripe or a similar stage of scale to us. A good example of this is Amazon. Amazon is a Stripe user because we work with them. In my first couple of weeks at Stripe, I was actually able to meet with Charlie Bell, who then ran operations at Stripe.

1:16:05And a lot of the things that, sorry, you then ran operations in Amazon, a lot of the things that we actually know I do at Stripe have come from learning from that experience. But it wasn't just, oh yeah, we'll take the thing because Amazon did it. It's up to him and lots of other companies are to learn. So the point is like we care about learning from others and then we apply it. So bringing this back to planning, because we tend to think a lot from first principles and we've been growing rapidly, the right planning process for us as a company has actually changed quite dramatically and quite significantly over the years.

1:16:33And so, you know, we do planning in a relatively deep way once a year and then in a slightly lighter way halfway through the year So if you imagine you do someone once a year and you're going rapidly, you're like, oh, what I would even say exactly Are you doing? And so I think, you know, for anyone who has the privilege of running the same planning process again and again You can really like, iterate it and tune it. We've had to make more deliberate and kind of more sweeping changes as we've as we've gone along But there are a couple of things at the core of how we do this. So one is it's really important to focus on who are the users that we are seeking to serve for any given product area that we're working in.

1:17:11And kind of holding their needs front and center in your plans is important. Again, it's right to do something that is, it's not a new conventional, but it's like not, it's not the most common practice in the industry, which is we work very hard to serve companies from the very, very small. The folks are literally just starting their company on Stripe with Stripe Atlas all the way up to multinational Fortune 10, Fortune 5 companies, Fortune 1 companies, I guess. There's a very different set of user needs across those businesses. So for any given area, it's very important for each team to be clear about who are they serving.

1:17:47And then we run a kind of inverted W process. So we typically have teams surface, just, well, their immediate thoughts are on the most important things to do. We'll then have a group of product leaders get together and try to synthesize the most important parts of that into a kind of draft overall company strategy, and take that back down to teams to figure out, well, how does, if that's what we're making, I think we should that tweak my plans at all, and we bring it back up for synthesis, and then back down for everyone to really distribute with a lot of context within that orgs. And we find that that our parents still very, very effective.

1:18:24I haven't mentioned this yet, but I actually use Stripe for my newsletter. I check the app many times a day. So I'm probably in the fortune one, one million somewhere in that. Do you have any feedback? Genuinely, I'd love to hear your Stripe feedback. It's, you know, it's great. Like the app tells me that I'm, that I'm, that I'm, that I'm, that I'm, I don't tell us this great. Like what, what bugs you? Well, it's interesting for churn, for cohort retention, which I care a lot about for the newsletter. I actually use a different tool that feels like gives me better, a table of per cohort, how many people are still around.

1:18:53So there's an opportunity to have better cohort retention metrics. Cool. Okay, maybe we can email about that. Quite genuinely at Stripe, anytime we talk to Stripe user, we are always looking for feedback. That goes for any occasion. We often bring users into the beginning of our Friday fireset, I already mentioned, and we're always asking them for their feedback, and we really, really want to talk to them. Well, I wish I had more. It's works great. Final question, Stripe Sessions coming up, it's going to happen before this comes out. What should folks be watching for? What's next for Stripe? We've touched on a few of the things that we're sharing broadly for the first time at Stripe Sessions and this call already.

1:19:33Per our development process, these have been in the hands of some Stripe users for significant lot of time and we've really crafted them with them, but I'm super excited about a number of the things that we're going to talk about next week. One is I've mentioned our revenue and financial automation suite. We've really been crafting the features there for some time and really a nice example actually of how building with closely with users really helps build things at scale. We've said the features that we work with companies like Atlassian and CloudFlare to enable their business on strike billing.

1:20:06So they have relatively complex models. For instance, you might sign a deal where it has a discount in the first year, and then there's another product that's being used in the second year, and then something else happens in the third year. You can not model all of that in Stripe Billing, and we're going to be showing how you can do that in the dashboard at Stripe Sessions next week, so it's now going to general availability for all Stripe users. So it's giving all of our users from the very small to the very large, the same power that any, you know, the world's best sauce platform might have.

1:20:32Excited about that. It's also the case that we didn't talk about it in this call. Stripe Connect is our product for platforms at marketplaces and If I may actually take one second it's a really good example of how Focusing on the needs of our users in one area kind of took us into a really valuable space to solve for many users in other areas So the ideas behind Stripe Connect came from working with companies like lift and Shopify in the very early days they were using Stripe for Pay -Ins, but we realized all these companies are operating these multi -sided marketplaces and there's a ton of heavy lifting you have to do to make that work really well.

1:21:09You have to have a bunch of regulatory licenses in place, but also there's a lot that you need to do to actually make the money move and kind of a coin for it in the right way. So connect is our product for platforms and marketplaces and one of the things that we've done over the course of the last year or so is actually taken a lot of the features that we'd built in the Stripe dashboard to help folks manage things like gathering the right documents from their users or handling refunds and so forth. And we've actually made them available as embeddable UI components that are also completely customizable, which these platforms can take and put inside their own dashboard.

1:21:45So they can present a lot of the power that Stripe is enabling to their customers without having to do a ton of engineering work themselves. And generating engineering leverage for customers, which is very frequently the main thing that we can do to help their businesses. We're going to show a bunch of that stuff next week, which I'm excited to write. Finally, we touched on it already, but it'll be great to show some of the innovation we've had around AI and large language models. Again, I think it's going to make a big impact over the coming years. Well, with that, we've reached our very exciting lightning round.

1:22:13First question, what are two or three books that you recommended most to other people? So most means that you have to integrate under the curve. So I'd say high output management by Andy Grove is Definitely the book that I've recommended the most it was a book that really opened my eyes to how to get the most out of teams You know over the book courses my career. So definitely check that if you haven't read it already Recently I so we talked about being meticulous in our craft. I've really enjoyed a build by Tony Fidel Tony Worked on the iPod and then the iPhone and then was the founder of nest and he puts a tremendous amount of thought into his users and how to build really great crafted experiences.

1:22:53I had the privilege of working with them very briefly at Google, but I felt the book was excellent in terms of helping provide some practical pointers there. And then I would be remiss if I did not mention Claire Hughes -Johnson's book, Snelly and People. Even though it only just came out, I'd recommend there's a lot. There's actually some stuff in here where I'm sharing some techniques that we've used with Stripe. So you might enjoy that. And then if I may share my comment. You've got it, too. It supports my laptop on here. I hope you read it. I've had her on the podcast. I've read, you know, skimmed it as much as I can because I'm not building a business right now.

1:23:27And she was in the top 10 bestseller list on the Wall Street Journal I saw. That's right, that's right. It has a ton of very practical advice. So I recommend that one. What's a favorite recent movie or TV show? Okay, I'm gonna cheat here because I'm gonna interpret that as like what's the best video content you've seen recently. Sure. I have fallen in love with YouTube for learning about the really fast -moving AI space. So it has been remarkably valuable and Andrew Carpathi has a bunch of really good seminars, but not just that across the whole spectrum. I think there's just so much gold on YouTube if you want to learn in new skills these days.

1:24:06Is that the channel you recommend? Andrew Carpathi's channel. His channel is awesome if you want to learn about about AI and he's the X Tesla AI, I believe recently I've gone to open AI but was doing this kind of on his own for a few months and has produced a bunch of great YouTube content Great pick most people pick white Lotus or something like that and you go with a AI YouTube channel. I love it What's a favorite interview question you like to ask? Okay, well actually you will find some recommended interview questions from me in this book One I really like actually, because it sounds like a softball and then it's not, is I often ask people, especially for leadership hiring, which leader that they've worked with, they admire most and why.

1:24:48So it sounds like a softball, right? But it actually tells you a lot about what this person really cares about in leadership. Sometimes I'll follow up and ask, like, how does that manifest in your own leadership style? But then, and this I think is really quite telling, I always ask folks, I'm going to spoil my entry question if I reveal it in the podcast, but I always ask folks. Okay, so imagine you were their manager. Tell me the performance review or the development feedback you give them to help them be more effective. And I think everyone always has things that they could improve. Certainly myself included.

1:25:21And so, folks ability to think critically about someone that they kind of admire or lie in eyes. And how they could actually be more effective. I find quite telling. What are some favorite products you recently discovered that you love? The products that I have recently discovered and definitely love is mid -journey. Also building the business side of their business on strike. But so for books, I don't know, mid -journey is an AI tool for generating images using stable diffusion. But it's really pretty awesome. I've been using it a lot with my daughter, so we'll come up with stories and we'll generate beautiful looking images with mid -journey, and then she'll drop them into books and she'll write the prose.

1:26:00The reason it, I think it's pretty cool is I was very surprised and skeptical at its UI to begin with because its UI is discord. They drop you into discord and you're in a channel where you have to prompt the artificial intelligence. And I find it very confusing at the beginning. It's like, this is not the right interface for this tool, but actually it's very smart because you learn from other people on discord how they're prompting the AI in order to get the kind of results that they're looking for. And I find that made it possible for me to get a lot more power and value out of the tool. So I definitely subscribed to Mid -Jerney and had a lot of fun playing with it.

1:26:36What's your favorite image of the creative if there's one that comes to mind? We made a cover for a book that my daughter is writing. My daughter is nine years old, by the way. So we're talking here, good children's stories. She made an image of a wolf wearing a purple velvet cloak sitting in front of a campfire with a like shack in a forest and a nebula behind it's beautiful and we actually built that with by combining two images the wolf and the shack and actually getting the wolf out of one image and putting it onto the other we use the new like Apple image segmentation company and base thing which worked great so it's also fun combining these AI tools but that one has been pretty good.

1:27:17What a world. Two final questions what's something relatively minor you've changed in your product development process that has had a tremendous impact on your ability to execute as a team. So we talked a bit about this already auto -deployes and that auto -merge feature but probably the most profound thing is we put a little button in every single developer tool. It's right it is an emoji of an octopus that is crying. If you click it it makes it also just type in what's going wrong and then we have our developer productivity team read all of those and use them to prioritize what they're up to.

1:27:50So just like frictionless problem recording turned back to be really valuable. We call those paper cuts. Crying octopus, I love that. Yeah. Who else from Stripe or former Stripe should I have on this podcast is my final lightning round question. I see how this works. I think you should have on Emily Sands is our chief economist and leads our information work, something that's data science, but also the teams that build all of or a lot of our internal tools. And I think Emily just has great frameworks for thinking about how to translate what's really going on for users into great sense of metrics that you can then use to kind of get the right action happening as I described before that's super important.

1:28:30And the second person would be Michelle Boo. So Michelle joined Stripe as an engineer in the very, very early days and has been with us for obviously a long time. She's really our principal product design architect in terms of the actual abstractions that we use to model our users' businesses on Stripe. I think she has very deep insight into how to think about getting those things right. And I think you would enjoy talking. David, I so appreciate you making time for this, especially during this hectic period ahead of sessions. Two final questions. Working folks finding online if they want to reach out, maybe learn more about what you're up to and then how can listeners be useful to you?

1:29:07Cool. Okay. Well, I am at DPS on Twitter. That is definitely the best place to get hold of me. But you can also check out my personal blog at blog .singleton .io And useful. Honestly, please send me stripe feedback. You can do that on Twitter. You can discover my email address on my blog as well. I would love to hear how we could serve you better. David, thank you again for being here. Thank you. It was a lot of fun, Lily. Same. Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast.

1:29:50You can find all past episodes or learn more about the show at Lenny'sPodcast .com. See you in the next episode.