Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team)

25:01I really understand those folks. So one kind of theme I'm hearing here so far is you guys are basically just like mining for gold, just like looking for like a vein in some mine. And once you hit it, you're like, oh, this worked. You just go crazy on just testing all kinds of things to see how far you can take that one, the light thread. Yeah, and I think, you know, I shared that that how this idea came partially from, you know, a monetization win that they had. I think there's a lot of, you know, because Duelingo runs, you know, I don't know what percentile we're in, but it's got to be a very high percentile of per capita experiments run for a company, based on company size.

25:45We're just constantly learning so many things and there's a really great cross sharing of modernization say, hey, this thing worked for us. Is that something you can use? I'd say it's rare that we go into something where it's just like, let's just try something. Typically, we have some insight because of all of these experiments that we've run that, hey, if we do this, I don't know, it's one here or it's worked here, it's driven this user engagement. that if we sort of massage that and make it trying this scenario or a different screen, we come in at least with the strong hypothesis of this will work.

26:17A lot of times we do look and we're like, a lot of the apps that we look at actually are games themselves. So it's like, all right, you're playing Royal Match or there's the new Pokemon trading card game that I'm spending way too much time on. You look at these games and see what they're doing and it's really good fodder for what we can do. But a lot of times you're least going with a strong hypothesis based on what you've seen work elsewhere. Got it. So just to double down on that point, it's not just random experiments. It's, here's a hypothesis. We're fairly confident or has a chance to be true.

26:47Let's try it. It's not just, let's just try everything. And I think that like how strongly you feel about the hypothesis directly ties to how hard that experiment is. Like with copy, for instance, we've actually set up really good infrastructure for copy testing. And I'm of the opinion that companies should run. As long as you have the user base to do it, like copy test constantly. Like the amount of copy tests that we've had that is just like, that have won and you, I don't know, you just, you just try things and you figure out what wins is, is definitely lead genit to it. And if shipped like massive wedds from little copy changes, is there an example of that?

27:19Or just like the impact going back to that goal screen, we used to say continue are like standard CTAs continue. And we changed that to commit to my goal. And it was like a massive win. Even just, you know, and again, it was like, okay, users tapping on that, what are we asking them to do, commit to the goal? What is that gonna lead them to do, commit, not, you know, turn? Just that little copy change, that one time right there led to huge wins. And in copy changes are so cheap. Like it's just, you translate us, you know, for us, you know, we have a lot of, you know, users all over the world, the UI languages.

27:54So, but like just come up with a bunch of ideas, translate some strings. This is one where, you know, the feedback that you'll typically get from Luis. So all of our changes at Dueling Go go through product review that are reviewed by Luis. So Luis reviews every single change that we propose or every experiment that we run. Tibaly with coffee is just like, I don't know, test it. There's nothing better than be told by Luis. I don't think this is going to win, but sure, if you want to. And a lot of times, you know, today's credit, he's right. And a lot of times, you know, our intuition was right.

28:26But it's just so cheap to do it. I think when the lift is smaller, it's great to have hypothesis, but you don't need to beat it up too much. On this thread, I didn't realize Luis reviews everything you're planning to change. Maybe the answer to the question I was going to ask, which is one of the criticisms of running a product and company this way of just experimenting constantly with all these micro -improvements and changes, is it can lead to something like a monstrosity of a product to an experienced citizen consistent and cohesive. And just like that often happens, is the solution to that having the founder basically review all the changes.

29:01Is there anything else you all do to avoid in moving in a becoming far bill or whatever? It's a good example to it. Yeah, our product review structure, where we've got our head of product design, one of the lead, the product management leaders, and then Luis and PR. And because they see everything, it, they have a really high product bar. and so that helps. I think over time we just as PMs have to look at okay where is our feature headed and so you know we do this with the streak at least on a quarterly basis to look at okay well what what have we learned you know how is our streak developed and how do we imagine this going in your future I don't think you know it's it's easy to do you know and end in some sort of awful local maxima if you're not constantly looking at your roadmap and think and you know you You don't have a clear strategy.

29:51For me, it's like, if you have a clear strategy for where your feature is going, hopefully all of those A, B tests are not just done to get some cheap gain. They're done with a long -term goal in mind. I do think though, and we do this every now and then with the streak, eventually you just hit a local maxima and you say, you talked about lunch and neutral experiments. This is a great example where it's like, all right, cool. Now we need to throw a bunch of this stuff out. Based on all of the learnings, can we reset this real estate? can we reset this UI, reset this feature in such a way that is just as good as what we have now, but is way more plain or simple that we can, again, start to layer on it.

30:27Those are really hard experiments to get win -off, to win obviously because they are so optimized, but they're really important to do. Otherwise, yeah, you just end up with a kitchen sink of a feature. One other tidbit I just want to mention, something that in advantage you all have with that other companies don't have is people want to learn, like they want to learn the language. And so getting pushed to come back to an app for something that they want to do is not an advantage a lot of products have. So anything you want to have there of just like this is why doing go might be a little different from what you're working on.

31:02I mean that is definitely a benefit if I had an app that this is actually why I think a lot of mobile games do streaks differently because you know to say that you played a mobile game and somebody who plays a lot of mobile games, you know, that you've done it for 3 ,000 days in a row. Like, I don't know, maybe that hits a little bit different than you've learned Spanish for 3 ,000 days in a row. I think the comparison set is much larger though than a lot of companies give them credit for, you know, themselves credit for. And I think that there's, I mean, there's a lot of ways the companies think about their, their very few companies I imagine out there are the world saying, oh, we don't do some degree of good for our users.

31:40Even if it's like a game, it's like, I don't know, you're like making Can you're given somebody a moment away from the craziness of their lives? And so I do think, though, that is contingent on companies who are gonna figure out if a streak works for them to figure out how can you frame the streak in such a way that a user does feel good about it. And it's easier for a do -a -lingo, but I think there's creative ways to phrase this for users. The other thing, maybe just on that, that I'll say, is the streak works really well for do -a -lingo because with language learning, it's really hard to see day -to -day progress in becoming more fluent.

32:14Like, and fluency is not even the right word. It's like becoming better at Spanish or whatever. It is a year's long process for someone to get better at a language. I mean, Duelingo makes it easier, but you still got to put in thousands of hours if you're going to reach C1 or C2 fluency. And that is really hard to track on a day -to -day. And so the streak works really well for us because we might not be able to tell you, you know, 0 .0, you know, 1 % more Spanish, but we can show you, hey, like you've gotten your streak a little bit higher. And so I think this works particularly well when you're at app that is doing something that's going to be send surf felt over a longer term to help contextualize that progress in a way that makes more sense or at least feels more tangible to a user.

33:02Great. That was a great context. Empowering to a lot of companies that aren't necessarily doing language learning. Okay, so I took us on a long tangent away from lessons and Experiments you ran along the journey of iterating on the streak. So a few things you've shared so far as just like it started being like started with this XP idea and then went to like a lesson Then you iterated on ways to make it simpler maybe harder yet it's street goals where you commit to like I'm gonna hit a certain goal of streak What else what else have you found that has worked didn't work lessons learns? You know, again, sticking with this one to seven days streak, you know, it is, the idea of a streak, particularly probably to this audience, is like, obvious.

33:41Like, oh, it's a streak, it goes, it just counts so many days. We've realized over time that a lot of users do not understand how a streak works. And it can be as big as, you know, as small as, well, I don't understand how streak freezes work or I don't, like my mom the other day was talking to you about, she's like, oh my, I like didn't use Duolingo and I come back, you know, streak's still there. So there's certain elements of the feature that we can do better at explaining. But even like what a streak is, it's tracking how many days that I've used the app. The more that we can make the feature easily comprehensible to users, the more attentive it is.

34:19And we've run a number of experiments to do this. You talked about early easy copy changes that we made. So this is my first win, experiment win. When I joined Duolingo was, when you start a streak, we use, we have a little copy at the bottom of the screen that just, I don't know, it's kind of like flavor copy. We use it to celebrate you or give you context. And I ran some tests that just like tried to and eat words, explain what a streak was. That was it. And it was a massive win because it really required, like it really dumbed down here is exactly how the streak worked. And it really helped users just understand what they needed to do.

34:59And I think this is something that's like you kind of constantly got to remind yourself, particularly if you work in tech and you're building cool tech features, but your user base is not a bunch of tech workers to like think about, all right, who is my audience? And for us, it's not just like tech workers, not people in America, people all over the world of all ages, of all cultures. And so making sure that your feature is even something as simple as the street because understandable is critical. What was the actual copy? Do you remember? It was, I think it's still in the app. Started day to extend your streak, but misaday and it resets something like that.

35:38That makes sense to me. Very clear. Yeah, good words. I love that. Okay, and then when you say massive win, by the way, just to give people a reference point, What do you think? What does that look like? What is a massive win in this scale? Well, and it's funny. I mean, this was four years ago, but I think it was like in the order of magnitude of over 10 ,000 DAUs for us. And actually, maybe this small bit of context. So, Duelingo really cares about the metric curve, current user retention rate. And actually, our first ever Duelingo post with Lenny was the newsletter that our former head of product wrote, Jorge.

36:11Still with single most popular newsletter post of all time in my newsletter across 300. I would highly recommend that if you are interested in the most, give it a read. You know, to summarize, basically, what we found is that if we wanted to drive DAU's and do a link of cares, you know, our growth nor star is DAU's, the metric that is most effective, you know, where a percentage change in that metric is most effective at driving DAU's is current user attention rate. This is just users who are not newer resurrected. getting them to come back tomorrow. And so most of the work that our teams do, our retention based teams do, is focused on Kerr.

36:47And so the retention team that I lead focused on Kerr, just so happens the streak is the best feature at driving Kerr. And so this experiment was the biggest Kerr win that we had had, it was like a top three Kerr win anyway for us. Just this little copy, right? And as I say, like death copy a thousand different ways. is sometimes it's not the big beautiful feature that's going to drive the huge gain. Huge gain sometimes is just something simple as a few words. I have love this. OK, so when you said 10 ,000 years, I think that references you guys measure in incremental impact and absolute numbers of new daily users you're going to drive with attributed to that experiment.

37:28So it's out there. Yeah, and we do both. I mean, we'll also look at like a lot of times I'll look at for retention day one versus day seven versus day 14. A lot of what I'm looking for is for us to have a better day 14, excuse me, better day 14 impact than better day one impact because it means that users are retaining better over time. This is particularly for users that would see a future multiple times. I just like absolute DAU numbers because as long as you're controlling for different biases, like a recency bias or a novelty bias, it's a really easy way to just have an absolute comparison.

38:04You start to look at percentage changes and then it's influenced by who your treatment. How many users saw the experiment? But at least an absolute number is easier in my mind to start comparing. Again, they're a doubt and they're pitfalls with it, but we find that that's a pretty useful way. That lesson comes up a lot on this podcast and that approach to experiment. So you're a good company. Quick tangent, if there's not an answer to this, no problem. coming back to the idea of just like experimenting like crazy and not creating a product that nobody wants to use anymore long term. Is there an example of an experiment that was positive that you all decided no we don't actually think this is what we want in the product the end of not shipping?

38:46You know retention doesn't only work on the streak although you know you would you would think we do most of our work is the streak. We've touched a lot of different surfaces over the years and there was one experiment that we launched that you know we talked about XP earlier. in the lesson, the only UI elements are a progress bar at the top, and then how many hearts you have, right? So we keep it really simple, and this very much speaks to the design, you know, philosophy of dualingo, which is simpler, UI is better. And we decided, hey, let's add XP in there. And so let's show your XP ticking up as you're going, you know, through a lesson, that's going to make the user feel good, it's going to give, you know, show you what you've earned, you're going to be less likely to quit, all of these good reasons to do it.

39:26And then you finish the lesson and then we'll show you've collected all this X -P. And at one, the hypothesis was a good one. But we realized that I remember having this conversation with Louise is like, cool, this is our most important screen in the app. It is our lesson is where users learn and the focus here is on learning. And now you've added this other thing up there to distract, you know, that could be distracting for users. And I think the question, we talked about roadmaps and strategies here, the question that that he had for me and I didn't have a good answer for it at the time was like, so what else are you going to do with us?

39:58Like, what's your iteration ideas? Where is this going to go? Is this going to make the, you know, lesson experience more gamified or, and what we realized is that like, honestly, it was kind of just a, you know, an easy engagement with idea, but we had touched our most, you know, sacred space in the app to do that. And so that was the case where it's like, yeah, it was a nice win, but we had added that UI element. And, you know, at least at the time, it was less clear what we would do with it. And we realized that like long term, it was just going to get in the way. And we'd rather for simplicity's sake, pause that, you know, shut it down and keep the lesson, you know, to be a little bit of learning sanctuary where it was.

40:40Now it's funny. Nowadays, I think we actually have enough XP -based mechanics and fun things that we can do that like, I think the beliefs about the unless an experience have changed. So something like that could work, but at the time, didn't it feel good to keep that around? That was an awesome example. Hopefully we have time to talk about how the team operates, whereby mine goes as like, oh, but you have all these PMs and teams that want to show impact and the performance reviews and all that stuff and you're shipping something. They're like, oh, we did it, we did it. So I want to chat about that later, but let's keep going on things you've learned and things that didn't work on the journey.

41:15The other thing that I'll call out with, the streak, it's like we have the image of the streak is this flame, right? And we have the streak flame and it's very much corridor iconography. It's important to acknowledge that's a metaphor for a retention mechanic, like the idea of keeping a flame lit. And again, if you think we've established the flame as for a lot of users as sort of their understanding of a streak, which is great, but there's a lot of people in different cultural context and different stages of life where the idea of keeping a flame lit to show your commitment to something makes less sense.

41:53We did some UXR in India many years ago and this was something that just did not resonate at all there, which was a really interesting learning. And that's something that's like, again, depending upon what your user base is, the more global UXR you can do to understand how users are actually understanding and experience your feature the better because you just, again, encounter insights like this. And so, So even our screen design, we used to have a flame. It was mostly this flame that would light up every day. But again, it was like an indication of a metaphor for a mechanic. And when we redesigned it, we did this, Kurt, one of our animators, did this awesome odometer animation where it's like your number would tick every day.

42:32It looked good, but from a product perspective, what was cool is we actually focused the design on the screen to show your number going up. And then I would say, seven day streak, eight day streak. And I think that as you're thinking about designing around a streak, don't get too bought up or caught up into like, what is this, the beautiful story that you're trying to tell at the expense of it being a really comprehensible feature. And so as you're thinking about product design, making that product design a clear distillation of this is what we're actually tracking, form should follow a function here.

43:08It was a learning press, and you'll see that now a lot of places where we're showing streak, we're really leading with the number, not necessarily the flame. That's a theme that I'm hearing again, and again is clarity. Don't obsess with making it too clever and don't ever think it just like clarity has a big impact. Clarity also does enough to come at the expensive delight, and this is something where you hit a milestone stone and duo gets, you know, it becomes, we call them a Phoenix duo and he becomes awesome and you know, lights on fire. And I think there are things that you can do to still make the experience really exciting and delightful and celebratory.

43:48And I would not lose that, but just don't do it at the expense. And I think it's also about figuring out for what you, you know, you can get away with doing more of this for users who are deeper into their streak experience. Then users who are starting where it's like your goal for the one day street accusers just to make sure do they understand how this feature works. I mean, even something again, just like another random experiment at the bottom of the street screen, we have a calendar. And over the years, it just looks more and more calendar -like. And that is simply because we find that the more we make it look like a calendar, days on top, you know, little circles that check, like the more we make it look like calendar, the more that people realize, hey, this is a daily mechanic.

44:27And so think about the screen holistically, but every single thing that you're doing on the screen, how can you use it to communicate? What is the point of this feature? How does it work? This episode is brought to you by Coda. I use Coda every day to coordinate my podcasting and newsletter workflows From collecting questions for guests to storing all my research, to managing my newsletter content calendar Coda is my go -to app and has been for years Coda combines the best of documents, spreadsheets, and apps to help you get more done. And Coda can help your team to stay aligned and ship faster by managing your planning cycle in just one location, set and measure OCRs with full visibility across teams and stakeholders, map dependencies, create progress visualizations, and identify risk areas.

45:12You can also access hundreds of pressure tested templates for everything from roadmap strategy to final decision making frameworks. See for yourself by companies like DoorDash, Figma, and Qualtrics. Run on Coda. Take advantage of this special, limited time offer just for start -ups. Head over to coda .io slash Lenny. And sign up to get 6 free months of the team plan. That's coda .io slash Lenny to sign up. And get 6 months of the team plan. Coda .io slash Lenny. I imagine one of the biggest wins was just giving people flexibility along the journey, and it like streaks freezes all these things.

45:50Is that a big vein of opportunity discovered? It is, it is great. Actually, I'm gonna show you the one of the thoughtful gifts that anybody has ever given to me. This is our dualingo serenity, our streaks serenity prayer, my co -lead Antonio. It's like an image, right? It's like, it's amazing. And so it says, Luis grant me the serenity to accept the flexibility I need, the courage to reach perfection when I can and the wisdom to celebrate regardless. And that actually is like kind of our strategy with the streak. So this show and tell by the way that was great. Yeah, I will, I guess for podcast listeners, we'll have to get an image somewhere.

46:26This idea of flexibility versus perfection and then regardless celebration is core to how we think about the streak. Because for the streak for us, it's very much a bend not break. If you're going to miss a day, I'd rather you come back having missed that day to an intact streak. But if you don't have to miss it a much rather you don't and much rather you come back and use the app every day. So that thing on flexibility though, that's almost certainly been the biggest, you know, from a mechanic perspective, the biggest DAU driver. You know, one of the earliest experiments we ran was going from you still only be able to have one streak freeze and then we let you have either two or three.

47:07So we tested two different arms. It was a none, again, another huge DAU win. This actually is funny. It was something that, and this is again, a call back to that growth model posed from Jorge. It actually was really bad for Kerr because we were basically saying, hey, you can take a day off. You know, that's okay, but it was really good for, this is going to be like Alphabet soup, IWAR, an active weekly active user return rate. So basically users who had taken a day off, we were getting them to come back more at higher rates. And so it made up for our losses and Kerr. But effectively what this meant is that, you know, why two streak freezes work better than one was, I don't know, sometimes people just need a lot more flexibility than one day.

47:49But one of the, again, the really interesting inside of this experiment was that three streak freezes was actually no better than two streak freezes. And there were two competing things here. And I think this is important if you're going to build a streak, you know, to figure out what your flexibility mechanic is. We are getting more users to return after longer times away to an intact streak. But if you start taking three days off from any habit, it's just going to be less likely that you return even four days later. And so we had these competing things where more users might be returning to a streak.

48:20A lot of users were also just not coming back. We were training them to take more time off. So that flexibility, what's the right amount of... I mean, again, this is another area we've run hundreds of experiments on what is the right number of flexibility. What is the right amount of flexibility? and we are constantly surprised here, we don't. I still don't have the answer for at every point in your streak, journey how much flexibility you need. One thing that I can say with certainty though is give more flexibility when a user is starting their streak. Again, one of our biggest streak -freeze experiment, I feel like I'm constantly saying this, one of our biggest wins, but they all were really, really big.

48:59One of our biggest streak -freeze wins was when you start a new streak, we give you two streak -freezes. And again, it's so funny to think back. It's like, how are we not doing this to begin with? But at the time, the streak freeze was kind of an overly gamified mechanic. You had to buy them with gems. That's our in -app currency, because we wanted the whole idea of, to feel like it was really something you earned that there was a little bit of pain to getting that streak freeze. And so we tested though, what if we just give users when they start off their streak to streak, streak freezes? And holy smokes to that one.

49:32And it's sort of obvious now, you know, in retrospect, but if you have a one or a two or a three day streak, it's really easy just to let it die and restart. You know, again, you need to get to seven days. Well, we've seen the data for it to really lock in. And so giving users more flexibility so it's harder to lose their streak initially. And then conversely, and this is what we keep learning. Eventually, once people get along streaks, you kind of don't want to give them as much flexibility. you know, because there's a lot of times where, yeah, users don't, I'm like this, I've got like a 400 day streak.

50:07Note that that is a lot less time than I've been at Duolingo. I have lost in restarted streaks a lot of times at my time at Duolingo. But you start getting along streaks and you really care about this feature. You really care about your streak. And most people, as long as you're not like backpacking through, I don't know, you know, the backcountry of Utah. you'll be in a place where you can get service. And so figuring out, who is the user that actually could use Duolingo and not conditioning them to start taking days off that they didn't otherwise need to do is important to figure out sort of where that line is for your future.

50:46This is fascinating. You can also buy a street right, like with money. That's a feature, right? Yeah, you can buy. It's funny. This is also something that you buy. Yeah, you can buy a streak freeze in the way it works is you can buy gems and then you can use those gems To buy a streak and this is something we wrestle with all we're actively working on experiment right now That's having a small hit revenue, but it's a really nice one for attention And I think it's actually worth thinking about from day one as you're building a streak Do you see this more as a modernization feature and do you see this more as a retention feature?

51:20What's the role of modernization and this what's the role of retention? And I think for us, it started out much more organically. And so we have a lot of modernization hooks that again, as the retention PM, I would love to get rid of. But again, it's sort of part of how the streak works right now. And so we always have this tension of, hey, if we start to make it harder to buy streak freezes, then fewer people buy them, by gems to buy them. And so there's this more convoluted series of impacts that happen. Yeah. Now, I love that people wanting to buy streak freezes is like the ultimate sign of how much streaks matter.

51:56Yeah, it's streak free. So the other big one that we've recently demonetized or introduced a free option for is getting back a loss streak. So we used to lose the streak. We had a feature back in the day called streak repair. We'd give you your streak back. You had to pay gems. But we found that worked way better was a feature called earn back. And this is basically where you would have to do a certain number of lessons, as long you came back with a window soon after losing your streak. Do a few lessons and we just give you your streak back. And it was such a retention winner. And again, what we thought about was it feels like you've earned it so much more when you've done, like you deserve to have your streak back.

52:40We haven't cheapened the streak because you've done something. And in this sense, this idea of cheapening the streak is something like from a philosophical philosophy of the streak. From a philosophical level, we wrestle with all the time of, cool, we're giving out more streak freezes. At what point do we cross the line and users start to realize their streak means nothing? Now, everything that we've seen users are totally cool with using streak freezes and still thinking about the streak as this meaningful thing. but my co -lead Antonio who made that awesome cross stitch for me, she is sort of the keeper for us of the sanctity of the streak.

53:15And a lot of times as we, and I think this is just really important to have is you're thinking about building your streak. You can almost always get engagement wins up to a certain point by just cheapening the streak, making it easier to extend letting users have more flexibility. But you kind of got to hold the line at some point. It's not clear where that line is. And once you talk about one way doors or two way doors, there's a point where you go too far and it's a one way door. And all of a sudden those users, those 9 million users on one year's streaks don't care about their streak anymore.

53:44And that is, I don't know, again, retention PM perspective. That'd be an extinction level event for us. I don't want all of these users to stop caring about their streak. And so to have somebody who is invested in the sanctity of the streak, and you know, for us, it's Antonia and Luis. He's very good about this. Is really important just so you make sure you don't go too far. That's an awesome insight. So to protect and personal notifications, I think are another example. This in general for companies, like how much is too much? Because everyone's just like, let's just send another push, it's fine.

54:15Just one more. And so your solution to that is a person is like the keeper and the gate, almost the gatekeeper plus the founder of how far is too far. That it's good if you can have that. I think push notifications are also easier because there's a lot of things you can do around. All right, we'll set a budget cap for how many notifications we'll send. You know, you can create a policy. Yeah, a policy is, but I think with a lot of things, it's hard, at least for the streak, it's harder to create policies for in the same way. A lot of it has to be done based on feel. And so you kind of just got to use your best judgment at times.

54:49Sweet. Okay. Any other, maybe like one or two more lessons from this journey of what streaks has become today. I mean, you mentioned notifications. I've mentioned this a few times. What, it's funny. You tend to think of, like, exactly as we say, ah, you can just always send another notification. It's gonna be some win and at some point, you know, it'll be a bad experience, but it's tough to see that. There's actually a notification that we, so we send two notifications related to your core streak each day. The first is a practice reminder. We send it. This is actually an interesting insight. 23 and a half hours after you practice the day before.

55:20Whoa, that is a lot. That is a lot. Three and a half, okay. So basically, if you practice at noon today, we'll send two and 1130 AM tomorrow. And we have done it because it's like assuming they are free in that time the day before, maybe they'll be free the same time. And we actually moved, we used to let users set this practice reminder time and, you know, our thinking was cool, you're going to say seven PM. That's what I really want to extend my streak each day. And then you know what? I said, this is somebody with two kids. life gets in the way. Life always gets in the way, and when you think you're going to practice, you'll change, your life will change, whatever.

55:56And what we realized is the best indicator of when you should practice was when did you practice the day before. We could almost certainly get more detailed. We have tried a bunch of ways to have much more complex logic, and what always wins is 23 .5 hours. That's so interesting. Reveal behavior versus... Yeah, exactly. So, again, we send this practice to our MITRE 23 .5 hours later. The other thing that we'll do though is we'll send it, we'll call a streak saver. And this is at 10 o 'clock at night, if you have not extended your streak, we'll send you a message saying, hey, you know, like it's your last chance.

56:27This is it if you don't extend your streak. And you would think that like, that's kind of spammy. That's kind of annoying to get a notification from an app at 10 p .m. But what we found is because people care about their streak. Their streak is this good thing that they attach positive emotions to, that they don't really want to lose. That notification reminding them, Hey, come back in. Like, people see this, buy in large as a positive notification and not a negative notification. Obviously, it serves our purposes well of getting users to come back and not lose their streaks. But again, I think if you can think about your notification strategy related to, what is the feature that it's tied to?

57:02How do users perceive that? You can almost certainly get a, not get away with more, but you know, you could be thoughtful about notification load and Windows and notifications. And again, for us, this like late night message it's also highly impactful, super good. You know, it's actually something that could be perceived as spammy, but a lot of our users really do. I get somebody who it's often late at night to come and I work here and I'm like, oh, forgot to do my, you know, I was thinking about doing it all day. You know, here it is, 11, 15, and I saw it done that message is really powerful.

57:36Yeah, it has saved me many times. I totally know that message. And I love that it's like a late night message from an app very rarely do you actually are happy about that. And I love that this actually is a good example. We're like, it's really funny. All of the stories that you hear about people extending their street, if you look around the Dueling Go party, where it's like 11, 13, 11, 45, all the Dueling Go employees that are doing their lesson at the last minute, you always see these pictures of people in the club doing, are like at a concert doing Dueling Go. And yeah, because it's like otherwise that you're gonna use a street freezer.

58:12God forbid evil is your streak. It's so funny. Okay, anything else. And if there's like more than one more, definitely share. But any of their really interesting lessons or wrong turns or insights. You know, I talked about streak freezes and we've done a lot with streak freezes, but I think if you're gonna make flexibility a thing, it's probably also useful thinking about how do you celebrate perfection. And so we have a feature that we have, it is the simplest thing in the world. It's called perfect streak. And it's just if you don't use the streak freeze for a few days, is we make your streak look gold and we make your little progress bar on the calendar.

58:45Just look a little bit nicer. There's no reward for doing it. You don't get anything other than this nice little indication and it is awesome. It is a simple feature. It is ultra and not complex and it is really powerful. Not only for getting users to, you know, as a bit of a reward, they'd be, hey, you know, get to seven days without using a streak freeze and, you know, your streak becomes perfect. But it's also a really nice indication of users who aren't using street freezes. Here is the thing that if you don't use the street freeze, which again, Candle, I would love for you never to use the street freeze.

59:21If you don't use the street freeze, your streak will stay perfect. It's funny, we actually just, we're constantly responding to bug reports about the street. It is, I swear to God, we have the best infrastructure around this feature because it is so important. And we had an employee who lost her, like a four month perfect streak. And it was a big deal for her. Cause she, you know, cause she like did her lesson as she was crossing international deadline. There was like a bunch of stuff going on that was like, it was just kind of weird in her back end. But like, people start to care about perfection as much as they do their streak.

59:57And you know, for that person, it was, it was a big deal when they lost their perfect streak. And so, you know, this is just an example of, look, if you're gonna, if you're gonna go after flexibility, which is good. Finding a way to pull users back into perfection is a really important counterweight to have. What I'm imagining is you guys need like an Amazon style chat bot that just gives you the streak back. It's just like, okay, here you go. We have very much, so we have, if people lose their streak, you know, there's ways to get in contact with us, but we've actually thought about that where it's like, okay, we should just like build a self -service feature and if you're, if we think that your excuse is good enough, whatever, We'll just yeah, yeah, yeah, yeah, because again, it's for us ship it back.

1:00:36I'm much rather you be on you know Be honest week then you know have lost it particularly of it, right? But it also can't feel that easy I love this I also love this point about just the power of the Animation and user experience having impact that's really interesting is that something you find off and just like make like Celebrating and making it feel really Amazing without like copy or you know or like any feature is just like holy you're awesome This is another thing where it's like when users care about the feature using not only animation, haptics, sound effects, using as funny we don't have sound effects on the streak, this will probably something we'll look at in the not too distant future, but like haptics are like something we have done a lot of testing on.

1:01:18Like the phone vibrating. Yeah exactly. You know, like they're being a really cool haptic pattern as you extend your streak, all of this stuff wins. And it's cool because I think it wins. There's a few reasons. One of it just makes you feel good, right? You know, you get some cool moment in your streak and we celebrate, you know, we celebrate, you're in this visual way, your phone's buzzing, it just like feels awesome. The other thing it does is it like causes you to pause on that screen and I think there's this desire as you think through a lot of, you know, as PM's think through like, oh, how can I get users to do this funnel as painlessly as possible?

1:01:50You know, I talked about good and fortunately, there's a lot of times where I don't, I want you to stop. I want you to stop and like land on this screen. You gotta be careful not to do this for too many screens, right? But like for the one big ones, sometimes I just want you to pause there and enjoy the moment. Because if I can get you to enjoy the moment more, you're going to care more about your streak, and you're going to be coming back tomorrow. And so animations that are cool and that causes you to like really soak it in, haptics, you know, that feel good. All of that comes together to make you really focus on that moment.

1:02:19All of that just gets used as more connected to their streak. So animation in the right times works well, and it's something we've had when quite a lot. Who designs the haptic stuff? Is there like a haptic designer? For the longest time, it would be a product designer. Or it initially started as like the engineer would be like, all right, you know, would cobble together haptics based on what they felt good. Then it became a product design role where they would kind of use their best judgment. We actually just recently required an animation studio, Hobbes, and Detroit. And now they are the sort of keepers of, you know, they're do a lot of motion design where a cappedx very close to that.

1:03:00And so they do a lot of that. I do remember trying to hire for a while, a haptics like contractor, like haptics design. And it was the saddest hiring I've ever done. Because it was just like, I don't know, like it was such a specific, like I just went through a lot of people who, you know, it's just a really tricky space of like kind of sound effects, kind of motion design sort of technical. Yeah. So it's not such a unique role. Very unique specific skill set. Right, and there's very few apps that really need this deeply. So you're almost creating this person. Yeah. That is fascinating. I don't want, I'm gonna, that's like a whole podcast on its own.

1:03:37By the way, I was gonna say as you're talking about this, I love that it's a win to celebrate people that don't lose their streak. Like you introduce this way to make it flexible, and that's a big win. And then you could go the opposite direction of if you don't use this feature. You also feel even better. Well, and it's funny, you talked about the danger of feature bloat. We talked about the danger of feature bloat. This is actually something I'm constantly thinking about with this. We have the streak, but then we also have the perfect streak. And we can count how many weeks you've had a perfect streak.

1:04:05All of a sudden, we have two streak numbers that are competing with each other. It's funny, we actually don't introduce the concept of a perfect streak until after you've hit seven days. and some of this is just because the cognitive load of additional streak features, a lot of our like cool -ish streak features, you got to get on a long enough streak. And not to say we haven't tested it because we have, because we've dealt everything, introducing these features earlier, but what we've found is that pretty universally, they lose when we introduce too many things, too many concepts to users, too early in the experience.

1:04:37It's just hard for them to manage. Okay, sweet. I know that we can go down this track for hours and hours. There's endless learnings about all the things you all have done along the journey. I'm going to shift to talking about how your team operates. So there's a lot of threads you touch on of just how a team can do this so well. Shift 600 experiments, as you said. Continue to find opportunity. What are some maybe lessons or advice you'd have for folks that are like, oh, wow, I want to work more like this from your team's experience. How do you use your team to operate that folks can learn? Yeah, maybe just a little bit of context.

1:05:15So we're, Dueling Go cares a lot about metrics. So most of our teams are metric based teams. So we do the most work with streak, but you know, the metric, what we really care about at the end of the day is Kerr. And Andy, I use, because we see that DAUs hit Kerr. And so when you can be really laser focused on my goal, each quarter is to make this metric go up. I think it's much easier to make sure that you're working on the highest ROI thing. I think when you think more about like, oh, I want to make this feature better. I think it's easier to get lost in what better means and how you think about better.

1:05:49And so I do think that like having a really strong degree of focus, you know, as a team on, you know, what is the metric that I'm caring about and how is that directing my efforts is versus feature oriented. So basically your teams are structured around a metric slash a goal slash outcome versus we own this feature. Or this one. Retention owns streak, I guess. But that's only because we've seen streak drive curve better than any feature, but we are not I mean we have this you know IAP hook with our streak freeze purchases. There are other teams that work on you know That can and have worked on the streak because it's not ours to say no no We do all the iterations here.

1:06:31We just know that it drives our metric better and the same way that like leaderboards We have a team that focuses on how much time you spend. We want users spend more time on duolingo, so they'll learn more. Leaderboards are the best vector for doing it. So that team does a lot of leaderboards work, but every now and then I have an idea that I think we highly retentive, and I will go in and I will pitch to them, and then we'll do some change to the leaderboard to make it more attentive. But I do think having that clear metric of we're trying to drive curve, not we're trying to just make this feature better, helps at least make sure, give the team clear marching orders and that focus I think is really good for prioritizing backlog.

1:07:07This is a really important point. This is the same way Airbnb worked when I worked there for a long time. Here's a goal that we want your team to be responsible for. You can work on any product you need to hit this goal. As you said, often various products are most connected to what you're doing, but what you're describing is like, even though teams, I imagine you own it from a bug perspective, and you're the shepherd of this part of the feature, because it hits your goal, helps your girl most. But any other team can come in and be like, Jack, so we need to work on some streak stuff to help with learning, you know, like go for it.

1:07:40Does that just detangent there? Do they like work really closely with your team? If they wanna do some work in the code, is it like, how does that work? It looks like it's pretty good. If you're, again, this is where I say, they're a soft ownership, we're not against teams doing things to the streak, but if we're gonna do something, given, you know, we probably have multiple quarters worth of a roadmap around the stress, say probably we do. Multiple quarters of roadmap for what we can do to this streak. If other teams want to come and mess with it, okay, we got to just figure out how is that going to work with, you know, what what our plans for the streak were.

1:08:13How do we make sure a lot of time some teams are coming in thinking, hey, let's do this to the streak. They're like in context that we might have. And so there's as much of like a much simpler version of what we're doing now a bit of a knowledge sharing of saying, all right, well, this is what we think about the streak is what we've seen work hasn't worked. How does that influence some of the hypothesis that you have? And so I think getting that really making sure the juices work this way. Good old faster and product management work right there. Cool. What else is interesting about how you all operate and how you all work to achieve the sort of success?

1:08:41Again, my team lead runs is the Antonio runs like the most really process. If you're going to run this many experience, you have to be really process oriented and really thoughtful about which experiments are going to run when, how is that going to set up the next one, we, you know, there's heavy -jera automation, I think sometimes the Atlassian sweet makes my eyes bleed, but like there's a lot of times where that degree of process helps the team unblock engineers and make them move really fast. And so making sure that you have really good process around how are you going to run so many experiments, it's worth investing in.

1:09:17Can you follow that thread? Actually, just what do you, when you say that, what does that that look like, what are some elements of that process to make this work efficiently? All the way down to, like, really detailed roadmaps around, all right, we're running, this experiment is based on the results of this experiment, or might, you know, hook into an element of this feature. How do we make sure that we're lining up implementation on this so that as soon as this thing runs and we're ready to go, we can start rolling out the next one. You know, I hate features just sitting around and us not, again, continuing that thread.

1:09:48So it's not just thinking about what's our engineering bandwidth, but also what's the design bandwidth to make sure that we have the next iteration of this feature ready to go. You know, we're planning months out as we think about these feature iterations, even small ones, you know, feature iterations because again, you just, you lose, when you lose cycles, you know, not pushing on a feature, it's just sort of lost opportunity. And so everything from being thoughtful about engineering road maps to design road maps to product road maps, all of that needs to come together in a system. So essentially mapping dependencies across function, and you're saying in JIRA, you can do this.

1:10:25You can do a lot of it in JIRA. There is a non -zero amount of Google Docs that we have that sometimes does things. Yep. A little bit. I don't know. Sometimes it just looks a little bit nicer. It's a little more flexible. but Jira is our, it is where the mother load of process is. Great. Okay, what else? Again, another thing that I'll just say is like, we really resist the urge to do the big V1. And I think this is, I shared the street goal example where a lot of times when we're exploring something, we will say, okay, well, that's cool. How do we strip away a bunch of stuff and figure out what our core hypothesis is?

1:11:05and then just ship that thing first as a V1. Because it's easy, and I've found this time and time again. It's easy to add things to features that make them win. Like, I've worked in retention engagement long enough. I can add enough things to poll and bells to add and whistles and to make something win. But there's a lot of times where it's like cool. Well, they don't want because all the whistles you add in, not because of what your core hypothesis was. And a lot of times if you can just really simplify what the feature is, It's also much easier to ship. It's easier to design. You're not designing for a whole system.

1:11:38You're designing for something much simpler. And so getting everybody to think that way allows us to end up shipping faster, shipping simpler, designing faster, getting faster approval, getting insight, and then doing what I talked about with Street Goal, being able to run iteration after iteration after iteration, add these things iteratively. And then, you not only by doing this are you able to move faster, but you get confidence at each step of the way that, hey, my series of hypotheses is actually born out. Or if it's not, cool, then we're going to drop that part of the feature and then just ship what actually matters.

1:12:17If I can try to summarize kind of the broad lesson so far that I'm hearing and maybe you would have shared this, but I'm just thinking like if I were to try to design a company need to operate that we operate. You essentially map all the levers that drive the business. So you have kind of this mapping of all the metrics that drive up to growth and daily active users. You curve, end it up being the biggest specific metric to move to drive growth long -term. So there's like imagining a tree of all the opportunities you could work on. You found like this is what is most connected to our growth long -term.

1:12:56You basically just start mining, I don't know if mining is the right metaphor, but just like looking for things that move that specific metric, you just like look and poke and explore. And then once you find one, you just go real deep on trying a lot of different. You come up with a hypothesis and a strategy of here is how we think we can do this and how we can move this and then you just try a bunch of stuff. There's also this element of the rest of development quote, there's always money in the banana stand, comes to mind where it's just like keep working on see there's more, there's going to be more opportunity.

1:13:28And when I joined Duelingo, the PM that I took over for Anton who used to lead the retention team, I remember saying, dude, the streak like it just counts up. You guys have been testing on for years. How much more work can we do on the streak? and he was like Jackson, you child. He didn't say that. Exactly this, but this is how I felt it. Like Jackson, you child, we're not even 30 % of the way optimized. And four years later, I say that with such conviction, we are so far away from, I mean, we've made a ton of strides, but we are still so far away. And every quarter where we ship a ton of wins and improvements to the streak it just continues to prove to me that there is so much more to be done.

1:14:11So I think your framing of it is, and I would say there's a lot of thought that goes into, again, I talk about the strength of the hypothesis that you have, you have to have as you start to build out larger future strategy. I do think it's really important to not just do a bunch of random stuff, but do it with intent, with a goal in mind. Otherwise, you do end up in these local you know, local maximas, but yeah, I mean, there's still a bunch of stuff that we haven't tried. I think we have high confidence in working out, and so we'll keep doing that. Are there any other say lasting lessons from this journey that if somewhere to try to operate this way, build streaks into their product, anything you'd recommend?

1:14:55Yeah, I really do think it starts with, you know, streaks are an engagement hack. You know, you can make your app more attentive. I'm almost positive, almost every app out there can make it more attentive. It is loss of version. That is, again, armchair psychologist Jackson, like it's just a human thing that works on humans. But if your app is not something that users want to use every day or whatever, you know, cadence you want your app to work on, and it's going to be, you're only going to get so much from that streak. And honestly, it's probably going to distract you from what really should matter, which is making your app something that people want to use every day.

1:15:36And so if you start focusing on the streak, but you haven't made that a enjoyable experiment, or experience, you're just going to waste a lot of time, honestly. And so I think making sure that you have your core loop of your app figured out that it is giving value to users is something that they want to come back to every day. That really sets the stage for something to layer a streak on. So resist the temptation if you haven't, you know, if you don't think you've reached that point to go too hard down the, you know, the path of streak. That's a really good point. Just like a streak is not going to solve your problems if people don't actually care about the core value you're providing.

1:16:09No, and honestly, it'll probably cause more problems if what you end up focusing on is how do I make the street highly engaging, but your app is on, I mean, you're just, you're, you're wasting time that could otherwise be, you know, better spent on solving more or critical problems. So that's one learning. You know, the other thing that I'll say is, we met with one of our board members, Bing Gordon a few months ago, rather. And he had this comment where he was just like, the reason why users care about your streak so much is because you care about your streak so much, you being dual -ingo. Like the reason why users care about our streak so much, is dual -ingo cares about the streak so much.

1:16:47And we're like, what do you mean? Well, after every session, you see a big streak screen and it's animated cooler than almost any other screen in the app and then sometimes you see some other screens and there's all these other feet, like you don't let a user forget it, you talk about them and messages and so I think it's worth thinking about, look if you're going to build a streak and then you're going to cond it off into the corner of your app, you know where users aren't going to see it, like they're probably not going to care about it as much and which might be fine because there might be other lovers that you think are more important to pull on.

1:17:17But there's a reason why, you know, we focus as much on the streak, you know, as we do and that's because we want it to be top of mine for users. And it's not by accident, then, that users start to care about. And so I think just as you're thinking about building the streak, making sure that you're giving it the visibility it deserves, if you wanted to have the kind of impact the Duolingo has, you know, it's sort of an important hierarchy principle to think about as you design things. That's such a good point. Like you look for cues to the app of, tell me what I should pay attention to, what's important.

1:17:47And if you're just like fire explosions, you made a streak. Oh, maybe I should pay attention to this feature. and then the push notifications obviously encourage you there too. Anything else along those lines? Maybe final thing is, look, we ran so many tests on our dual -ingo streak to figure out what worked. I mean, we have a philosophy of dual -ingo of tests at first. You know, we are a lot of times willing to test things. I really think that if you're going to try to introduce a streak or you want to improve on the streak you have, like don't get too caught up in the philosophy of everything.

1:18:24I mean, make sure your hypotheses feel like they're good, but my recommendation is just try things. And this is again, you said it earlier, it's like, this is most human psychology is anything. And as soon as that becomes the case, you kind of just got to understand what users respond to. And the easiest way to do that is to stop spending time batting around ideas in a conference room and just try some stuff. So huge recommendation to like, if you're going to invest in a street, try and figure out what works through testing with users rather than trying to get it perfect on the first try. Say someone's listening and they're like, should we do street because it's worth doing?

1:19:01What's your take on just like the chances that a street feature would be helpful to another consumer? I am well known for saying in the company that I think every street, every team, every app could benefit from a street. Now, how you implement it is very different. And I think you got to like, what is your user's use case? Like, if they're going to come use, I don't know, tax software. OK, you know, not only say this, tax software to be a hard one, but maybe it's all about, you know, you need users to come back every day during the tax season, or how many times, I don't know. Now that I say this out loud, times you upload your paper.

1:19:37Yeah, that is hard use case. But the vast majority of companies, means I think have a good idea of like, all right, here is my ideal use case. I want users to come here three times a month. That would be ideal. Our four times a month, you can build a streak to work. Peloton has weekly streaks because the idea of doing a Peloton workout every single day is hard for a lot of, it was hard for this user during COVID. It's just like every now and then you get on the Peloton. That was great. But the idea of a weekly streak was something that I could keep up. And so I think figure out what your usage pattern is as a user and then build your streak around it, but you know as long as you're not, you know, like a release, again, the the tax example is probably a good counterfactual, but as long as you have some degree of frequency in your in your use, I think almost anything can have a can have a streak.

1:20:29So doing it again, a $14 billion dollar company, this feature, possibly the most contributing factor other than the core product to that level of success and market cap. And it's hard to imagine another just feature of a product that has had this much impact on growth and revenue and building this sort of business. So I love that we spent this much time on it, the mother load, the mother load of advice and insights. So thank you again for putting it. Very fun. With that we reached our very setting lightning round. Are you ready? I'm ready. First question. What are two or three books you recommended most to other people?

1:21:10All right. I'll start with a guide to Midwest conversation. So I'm based in Kansas City. I'm a proud Midwesterner and us Midwesterners talk in a certain way. I think you hear about Minnesota So, to nice, but we tend not to say what we mean, and it is a very funny primer into what Midwesterners actually mean when they say what they say. So, highly recommend reading that. I like that they give that to people. Just like, here's what I might be telling you, which you may not read. My wife is German, and I gave it to her. I can see German. It's going to be the opposite of that, in a very different way.

1:21:45Another book, this is a good one, Fate is the Hunters. This is a really cool book. It's a memoir of one of the early commercial airline pilots. And it is wild to hear the stories about what flying was back in the day. I'm former management consultant. I flew every week for almost six years. And I never once had to worry about, am I going to make it to the other end of this flight alive? That was not the case back then. And so some of the stories about what it used to be like to be a pilot on some of these planes before modern aviation technology is fascinating. it makes you really appreciate what we have.

1:22:22It feels good to read a book like that being a software PM or engineer whatever like how different that life is. Hard hardware is hard. It's not have to design. Okay, next unless there's any other books you're going to share. Okay, great. What's a favorite recent movie or TV show you really enjoyed? So I have two kids. I watch a lot of Bluey. It's really good. It brings me no shortage of joy. But adult show that I or show not meant for four year olds that I have watched. I just finished the latest season of Emily and Paris. Man, wonderful. I realize it's not the highest brow of television, but just like beautiful people and beautiful cities solving problems that are not, you know, earth shattering.

1:23:05Sometimes it is nice to just tune out. Also, I'm learning French undo lingo, So, slight plug for the app, I can understand a lot of the French that is being spoken. And there is no better joy than having invested as much time as I have in French and actually being able to use it. So, huge fan of them. That is so funny. What a fun Venn diagram of interests. My mother -in -law loves Emily in Paris. I saw someone tweeting about, like, what these are Shiaan? How is she still in Paris? Yeah, you bet it's just non -ask question. There's a lot of questions to this show that are left unasked. The only favorite product you've recently discovered did that you really like other than the one.

1:23:43Last week I went to Home Depot and I bought a new ladder and ladder innovations. You don't think of often, but you can make one of the legs go a little bit further than the other leg and as somebody like myself who has a house that is built on a slight slope. Every time I go up in my ladder, I take my life in my hands. But with this ladder, I'm always even, I cleaned my gutters twice last week just because of how awesome this ladder has, how much this ladder has changed my life. So ladder innovation, I don't think it gets talked enough about it. So I'm happy to give it the spotlight it deserves.

1:24:18I appreciate you doing that. It's the first ladder recommendation we've had on the podcast. Two more questions. You have a favorite life motto. They really find useful and worker in life. They share with folks. This probably will not be much of a surprise, based on how I've talked about our willingness to test things. But you miss 100 % of the shots that you don't take of a big fan of just trying things even if your possibility of success is not 100 because you learn a lot along the way. So final question. Do you have any fun traditions at Duolingo, amongst either the PM team or the company in general that might be delightful to share?

1:24:52We have way too many traditions to count. I will share the weird tradition that we do at every retention standup. And this started during the pandemic. We obviously used to stand up in person and then when we went remote. We did this thing where whoever's last person to go would count down three to one. And then we'd all try to clap at the same time, which was kind of fun and dorky, but we fell in love with it. And four years later, we're still doing it. Recently we've added, we all say yeehaw in unison afterwards. I can't tell you why, but trying to synchronize a clap via Zoom and then all shouting yeehaw.

1:25:35I did this in a phone booth the other day and after I came out someone told me you know that those aren't as Soundproof as you think but you know and you get a good opportunity to give a e -ha you kind of can't pass up on it So I love these little things like they're so you know They sound so minor, but they're such important elements of team culture and tradition and so important for pms To find ways to just have fun and do something ridiculous I will say it took a while to get people behind shouting e -ha But now that we have people doing it, you can't take it away from there. We all love it.

1:26:09I call their all hands for a while, y 'all hands. Feel free to steal that. You get it. You get it, Lenny. Yeah, I get it. Jackson, this is incredible. I feel like people are going to listen to this with notebooks and just like, okay, here's a bunch of ideas we should try with whatever working on, whether it's streaks or not. Thank you so much for being here. Two final questions. Working folks find you online if they want to reach out, learn more, learn more about do a link, go and know your hiring product managers to share more there. And finally, how can listeners be useful to you? Yes. You can find me on LinkedIn.

1:26:40That is where most of my online social media is. So Jackson, Shuttleworth, and then how people can be useful to me. Yes. As you said, we are hiring. We're actually hiring for my team. Are you interested in thinking about streaks as much as we do? We might be the right home for you. So please, you can plan our website. We're also hiring for a number of other product management roles in there, all as thrilling as this work is. Then I'm always interested about how other companies have implemented streaks in what they've learned. What I'd say is if you're a company whose implemented a streak, maybe in a different way, the Duolingo has, or you found a whole ton of success, and another vector, another element of the feature that we didn't talk about today, I would love to know more.

1:27:26I used to catalog basically every streak I found out there. And as it's become more of a popular feature, it's just been hard to keep up on. So, you know, if you have interesting streak and sites to send my way, I would love to hear them. I love that collection of all the best ways of doing streaks. Jackson, I just want to say congrats to your team and you for having so much impact. This is like the dream of a lot of pms and teams is to see this much impact and continue to ship wins. and so congrats. Nice work. Thank you very much. And with that, Jackson, thanks so much for being here. Yeah, thank you, Lenny.

1:28:02This is a lot of fun. Same. Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or a leaving review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lennyspotcast .com. See you in the next episode.