Why Duolingo Worked (with Luis von Ahn, CEO)

22:42But in our particular case, what happened was this. We launched. Because we weren't making any money, we also couldn't spend money on marketing. We couldn't performance marketing because performance marketing is this thing where you can pay Google 30 bucks and then they send you a user. But to me, that seemed really dumb to spend 30 bucks to get a user when we were not going to make any money off of that user. You want to be making more than $30 from that user. Yes, or whatever it is, you want to just do more than that. And so we couldn't spend money on marketing. We weren't making any money. We had a bunch of engineers.

23:19So what were we going to work on? And the only thing we decided to work on was teaching better and increasing user retention. That's it because that's the only things we could work on. So we spent from 2012 to 2017 basically optimizing retention rate for users. And we optimized it a lot. I mean, for example, we went day one retention. So that's the retention of people when they show up, what fraction of them come back the second day or the next day. When we first launched that number was 13%. As in you lose 87 % of your users the day that they... Well, they may come back another day. 87 % happened to not come back the day right after.

24:04It could be that they come back later. But that's what we saw. We ran, who knows how many AB tests over this period of time. But by 2017 that number was probably about 50%. And maybe misremembering exactly the numbers. But we were able to drive all of our retention numbers by a lot. Because that's the only thing we could work on. Had we not done that, we probably would have spent all our time optimizing our ad pipeline and optimizing all kinds of weird stuff that ends up making you money. But I don't think it helps you grow that much. And I think we were in retrospect. This was a very smart move.

24:35I'm at the time. I don't think we were thinking how smart this was. We just kind of did it. It gave you a durable asset. I mean, the fact that you now have a product that ensures that you have fully maximized the potential for a user to stick around. It means the user base you've built and the machine by which you eventually do make money is far more durable. That's right. And I believe that. Now, of course, I believe all of this in retrospect at the time. None of this. We were just like, I don't know, make a thing that people like. It was also worth you raising money at a half a billion dollar evaluations.

25:06So this was when money was free in some sense. It's just like here. Have more money. But it worked. Okay. So you joke that you weren't doing any marketing in some sense. That's true. In some sense, it's not. You have the green owl. I have had a great brand forever. How did a couple PhDs from Carnegie Mellon of all places come up with a Cuddly Green owl and build a, I don't think of CMU as wonderful as an institution as it is, as like the top design and marketing place in the world. They happen to have a really good design program. But look, this evolved over time. We did know one thing. We did want a mascot early on.

25:43We were like, look, a mascot. Somehow this will help us. We didn't know exactly how, but we did want a mascot. Eventually, we chose an owl because of in Western cultures, at least, a symbol of knowledge. They ended up being green because my co -founder hates the color green and I thought it would be funny if it was green. So that's why it was green. And now your whole office looked behind you. The first version of the owl was not very Cuddly or I mean, it was kind of not great looking honestly. Over time, it evolved. We hired some amazing illustrators. We hired this guy who still worked for us.

26:15Greg Hartman, who's in our head of art now. He just made an amazingly beautiful owl over time that evolved. And so that's one thing. We just made a really good asset that looks good. And then the other thing that happened is it started getting a personality. But the personality was a little bit accidental. The first way when she gets the personalities, the owl was actually not in the app at all at first. The only place where people saw it was the app icon, which is kind of a very close up version of the owl's face. And then also when you get a notification, it has the app icon so it looks like it comes from the owl.

26:49So the first place where the personality for the owl came up was basically what we said in the notifications. And we started saying some weird stuff in the notifications. And I think people started giving it a little bit of personality because of that. Then another thing happened with the notifications. At some point, we decided that we can't be super spammy. So we're going to stop sending notifications after five days of inactivity. So if you don't use the app for five days, we'll stop. Then it occurred to us, well, we're going to stop sending a notification. We may as well tell you. Right. Get the credit for it.

27:18Yeah. So people know we're good guys. This is what we thought. So that last notification on the fifth day, we wrote it. It's just said, these notifications don't seem to be working. We're going to stop sending them for now. That's what it said. Well, we didn't know this. We didn't expect this. It turns out this got people to come back a lot because they thought that Duelingo was giving up on them. It's like your mother. It's kind of like guilt -tripping you. Because of that, though, it started giving rise to all kinds of memes on the internet about how the owl was obsessed with getting you to come back and he would kill you if you didn't come back.

27:53But that was the community. We didn't do that. After that, we realized that we could push on those memes. And so we started making fun of ourselves with those memes. And that has worked really well. And at some point, we got the idea of getting an owl suit, a suit that somebody could wear. At first, we thought it was going to be good for recruiting events, like going to try to hire people at universities and the owl suit would show up. Turned out, you could start putting somebody in an owl suit to do weird stuff in TikTok videos. And that gets a lot of people to watch it. But that was a little bit accidentally discovered.

28:26And in fact, the first person who started doing this was a very young woman, Zarya, who, when she proposed that, I thought, this is such a dumb idea. I was obviously completely wrong. It turned out this has given us a lot of prominence in the world. Have you ever tried to quantify? I mean, this is like a core pillar of dual -ingo business strategy at this point. Have you ever tried to quantify the business impact of having all this earned media? Yeah. I mean, we have. Our belief is that about 15 % of our users come from that. On any given day, roughly 15 % of our new users come from the owl for lack of a better word.

29:03Some sort of owl thing. The majority of them come from people telling their friends about it, but about 15 % come from this weird stuff. And just to multiply some other numbers together that you've talked about, if it really is 15%, and you have 100 million users, what's the value of 15 million users? Well, if it's actually at 30 bucks a pop would be the paid cost to acquire them. That's what $450 million. Well, you're only taking the active ones. $30 on Google to get a user period. A user that may not stick around. So it's more than that. This is probably when I don't know the exact number, but it's hundreds of millions of dollars of funds.

29:42It's the most valuable Allen history. I don't know if that's true, but it's been really good for us. Fascinating. All right, listeners, we want to thank a new friend of the show, plaid. The name is likely very familiar to you after our recent ACQ 2 episode. Odds are you've used plaid before without even maybe realizing it. If you've ever linked your bank account to apps like Robin Hood, Venmo or chime, you're one of the millions of people like one in every two Americans who've already used plaid. I feel like I've grown up in the tech industry alongside plaid. There are so many modern experiences that are powered by them.

30:19And at its core, plaid isn't just about making it easier to connect to your bank. It ends up being the backbone for thousands of companies building faster, safer, and more seamless financial experiences. So whether it's reducing fraud, speeding up onboarding, or turning old school banking processes into something that feels instant and effortless, plaid is making it happen. So last year plaid rolled out some powerful tools. Think cashflow data for better credit decisions, anti -fraud tech with AI, and analytics for bank payments. And this year they've leveled up again with major updates across all three of those product lines.

30:53Yep. There are even helping businesses manage things like direct billing for your subscriptions. So the bottom line is plaid is making it easier for companies to build smarter, safer, and more personalized financial experiences that just work. If you're building financial tools or infrastructure, plaid's data analytics can give you a serious edge, whether it's fighting fraud, underwriting smarter, or managing payments more efficiently. So if you want to learn more about how plaid created one of the biggest networks in financial services today, listen to our recent ACQ2 episode with plaid's founder and CEO, Zach Perei.

31:26And our thanks to plaid. Okay, I'm going to keep coming back to my like, what's a heuristic type of questions? A startup comes to you with the idea that they're going to run a similar strategy. Any advice for when it could work or not, or is this pure lightning in a bottle, I have no idea when it will work? Some stuff I think you can reuse. There's a few pillars in our strategy. One thing that we haven't talked about that I think accounts for probably have the value of the company is our obsession with AB testing. We have AB tested our way to where we are now, and we have probably run, I'm going to make this up probably about 2000 AB tests per year of all kinds of things.

32:07And so we had just built the company around our entire product organization. That's not just product managers, engineers, designers. The whole system is built around shipping AB tests as fast as possible. That has worked really well for us, and I would say, you should do that everywhere. If you have a consumer product, I mean, you shouldn't be AB testing stuff. If you have like a B2B, Sasting, whatever, do something different. But for a consumer product that has a lot of users, you can really AB test your way to improve almost any metric you can. That you should do. The other thing that I would give advice to is hiring really good people that you should continue doing.

32:44In terms of the specific strategy of having a premium thing that you don't monetize for a while, I'll tell you one of the reasons why that may not work anymore. We have found ourselves, but we have seen it all over the place. We were fortunate that we put out an app in 2012. The app market was not yet set in stone. It's pretty close to set in stone these days. It's not 100%. But if you look at what are the top 50 apps globally for the year this year versus last year, and you compare, they're pretty similar. Maybe one or two changed, but they're pretty similar. It's similar to the year before and similar to the year before.

33:22At this point, it's pretty hard to enter. I don't think that the strategy that we employed in 2012 would work today because the app market is pretty set in stone. It's just kind of is. Yep. What about the strategy of something clever earning you hundreds of millions of dollars of sort of free earned media? It's great if you can do it. Of course, a lot of cleverness in this that many employees at Duelinggo did, but I think we were lucky in a few ways. I think the fact that we're an education company gave us license to do some stuff that many of the companies may not have been able to do because it's kind of hard to say that language learning is bad for you.

34:03So that means that we can do risk -a -stuff on TikTok. But if a company that is trying to do like, I don't know, like some sort of fintech company or something, I think it's much easier to accuse them of stuff. And I think that helped with us that just come on, everybody. We're the good guys. I think we were able to get away with a lot of stuff because it's hard to say bad things about language learning. Right. If I'm coming at you with you could argue it's a fair interest rate, or you could argue it's a predatory interest rate. If I have a mascot that's sitting there with a knife saying, use our app more, it's not going to go well.

34:34Exactly. I think we benefited a lot from the fact that ultimately we're doing it for education so people are okay to take some of this. Right. Fascinating. By the way, you know another place when we talk to other companies, every company wants to talk to us, not because they learn from us, but because they're like, oh you guys, yeah you guys are kind of fun. This is good. Nobody thinks of us as competitive. Well that's great for us. And it's because almost every time we talk to whoever, fancy person at random companies, my daughter loves you. You know I think we've benefited a lot from that. Which is great.

35:05How many other 10 billion dollar companies could that be said for? Like if you're a 10 billion dollar company, almost you'd think like de facto you're going to have people who are your mortal enemies. Again, because everybody's like, ah green owl. Yeah. Yeah. So it's an interesting comment on competitors. At this point you probably don't have a whole lot of meetings about legacy language learning companies and saying, how can we acquire users versus them? Who do you think about as competitors for your users? I mean the way we see it is we compete for time. And so our competitors are Instagram, TikTok, et cetera.

35:44That's that's who we compete for time. And by the way, we're losing. We're we have much less time spent on our app than TikTok has or Instagram, et cetera. But when we look at our users and we see that our users maybe stop using dual lingers, something we ask them, you know, why'd you stop? The most common answer is like, yeah, you know what I'm spending more time on pick your random social media site. I'm spending more time on Instagram. That's it. The competitors is whatever takes your time. All right. Well, you planted a seed of something that we talked about for like one minute in this podcast.

36:13But every person I talked to who's ever interacted with dual linggo former employees and other podcasts you've been on is all about experimentation, AB tests being data driven. There has been lots of inks build on this for anyone who wants to read excellent blog posts about the, you know, the AI behind your push notifications and how all those systems work. One high level question for you. How do you as a leader balance a cultural tenant being data and experimentation driven versus instinct driven? We're actually weirdly both. I'll tell you where this comes from. We realized early on that pure experimentation can lead you to really bad places.

36:59If you kind of just don't think about it at all, you're just going to optimize some metric. I can lead you to really bad places. And one of the best stories that I heard about that, and we repeat this a lot of dual linggo is something that we learned from the founder of Groupon, who happens to be from Pittsburgh. I mean, we're located in Pittsburgh. He happens to be from Pittsburgh. Yes. We're huge, de -script users. Yeah, yeah. This is great guy. He told us the story, which I'm going to tell it. And at this point, it probably has changed a little bit because I probably remember now what is in the law of dual linggo as opposed to exactly what story he told us.

37:31But you know, he told us a story about emails that Groupon set per day. So Groupon basically, we got a deal and the ideas at the beginning, you could only, they could only send one email per day with one deal. He was strict on it. It's like only send one email per day because more than one email is spamming. At some point, some enterprising product manager shows up and says, what about if we send two? Andrew Mason says, no way. Like no, we're only going to send one. And then this product manager gave this amazing argument against it. We're not trying to launch two. We just want to know, are you against knowledge?

38:07And of course, nobody's against knowledge. So, so, you know, he's so he said, I'm using this. Fine. Fine. Fine. And so they tested two. Guess what happened when they tested two, two is way better than one. So they started sending two. And then like, wow, if two is better, test three turns out three is better than two and four is better than three, et cetera. And they maybe tested their way to like, I don't know what number seven eight or something until the channel stopped working because people marketed a spam and then it died. The channel died. But so this is the thing about A, B testing. You have to, you have to really do the common sense thing above the A, B test.

38:49And so what we do at Duelingo is we have, we have this two step process. I mean, the first one is before you even A, B test something. And by the way, here we are against knowledge. There's certain things we just don't want to know. And it's because it's too tempting to do it. So the first step is before we do an A, B test, we have this process called product review, which I assume happens in many other companies. But in this case, every single change that goes to the AppGets product review and product review is basically a set of leaders in the company that includes me. There's about five people.

39:18And we look at every change and we have to agree for it to go out. We stop a lot of things before going out to prevent this type of stuff. Here we use intuition and just common sense. We're like, no, nobody wants to receive 32 ,000 notifications. That's just dumb. Even if the A, B test says so. I think that has helped us quite a bit. Do you have an example of a way in which you've been anti knowledge in the past, something you decided you didn't want to know? Oh, yeah. I mean, pretty recently, somebody wanted to know what happens if you put an ad that takes over the whole screen when somebody opens the app.

40:00Like a product manager wanted to know that. And we're like, yep, but we don't. Particular with monetization, it's so tempting because what happens with monetization is you run the experiment. And then it says, it's just something like if you launch this experiment, it will make you $50 million a year. And you look at it and you're like, that's nice. And then the finance team figures out that this happened like somebody from the finance team looks at this experiment. Suddenly, the CFO is in my conference room or something. And he's like, yo, $50 million bucks a year, you got to launch that. We try not to know the answer to many of these monetization experiments.

40:34When you wear your CEO or your head of product hat, these things are sort of obvious. And what's underneath it is this implicit assumption that there is a future dollar value associated with treating the user with respect and having a feel good association with your product that it's basically like all of the potential future dollars we could earn are jeopardized by doing this. It's we're pulling forward revenue if you do some. But it's very hard to quantify, but it's very hard to quantify. Complicated, expected value calculation on it. Have you ever tried to quantify that brand value or future value to?

41:11We haven't no, we haven't what we just have is a group of I'm saying five, but it's on the order of five leaders that have a very good gut feeling about no, this is this too much or no, this is not good. And I think their gut feeling is quite good, I would say. We run into this problem sometimes, you know, we do run an experiment and where we have to make a trade off like, for example, we're in some experiment because we know if we do this, we lose 10 ,000 users but make this much more money or whatever. We've never known how to make these these trade offs. So generally what we do is we go for users as in if something decreases the number of active users that we have, unless it's going to make us 500 million dollars, we're really not going to launch it.

41:56So just take a laugh, I need a bit joked about using that line on me. This applies to any business, like for us, if you told Ben and me, okay, go objective function, maximize the value of acquired, we could come up in 10 minutes with a whole list of things that we know exactly what we would do and it would make an obscene amount of welcome to this daily episode of acquired. Yes, it would, but for the next few months and not for the next few years. I'm not even sure that that like, that might be the value maximizing thing to do, but we would never do it because it would just kill our hearts, you know, like, yeah, that that also matters a lot.

42:34Yeah, it's funny though, your heart, you can boil a frog quite well and product managers at Duelinggo have gotten really good at boiling my heart slowly.

42:47Things that I think I was not okay with five years ago, I think I'm a lot more okay with because they just kind of push the envelope every slippery slopes they get you. But do you end up feeling like, okay, you know what, they had a point, like I no longer... Yeah, sometimes I've definitely been wrong about stuff. I mean, that's for sure, but I still claim in the long term, it really is better to do what's best for the user. And I don't mean what the user's asking for because the user's asked for weird shit. I mean, what you think is better for the user. I think I think that in the long term, I'm pretty sure that's that's value maximizing.

43:19I think this is a pretty specific thing to founder led companies, which is we will do what I personally feel in my gut is best for the users. And like that, just carrying the day every time that does not happen in a professional management setting. It may not. What I'm happy about is that I would say five years ago, it was solely my gut. And I think at this point, there's a group of leaders, a gem or had a product, semi -archief design officer, that who's gut somehow is like identical to mine at this point. Maybe 1 % different, but I feel pretty good about the fact that if I was gut, this is pretty good.

44:03So we've talked a little bit about this idea that you have a bottom of funnel governor on what goes out. And we've talked about that there's a mid funnel thing that's the AB test. And I think you've said in the past, the latest number is 50 50 50 50. Okay. It is eerily close to 50 50 as in like it really is 50 50 is not 48 52. I can't explain one of those where I'm just like really 50 50 come on people has to be something different, but now it's pretty close so that gets to this question. Why is it that over time your collective gut has gotten better about what experiments will succeed? The way has gotten better is because we get the results of the experiments.

44:43Our gut is kind of a learning machine. We're learning we're like, yeah, that type of stuff. At this point, we have years of experience that people that have been doing product review have been doing it for years now. We have years of experience knowing this type of change does that type of thing and we're pretty good at it. We're not 100 % but we're pretty good at it. Not only do you know which direction things are going to go, but you also know which ones have more impact. So for example, we know that the session and screen. So that is the screen that you get after doing a lesson matters a lot in terms of getting you to come back the next day.

45:17But we know that other screens like a screen in the middle of a lesson almost matters not at all in terms of getting you to come back the next day. So we've just kind of learned what what screens matter and what parts of the app matter for certain metrics. We just have a really good gut on that. Hmm. Do you have to treat big experiments differently than little experiments where your gut may not have as much training data in it? Yeah. And there's some experiments that are a lot more complicated than others. Some stuff takes us a long time. Well, first of all, now we also have pretty sophisticated data signs that tells us for how long to run certain experiments.

45:52We also have a pretty good gut feeling about this particular experiment should run for a pretty long time. Most experiments we run by the way we're running for two weeks. We have enough data in two weeks to know where to go. But there are some experiments that we run for about six months because we know that you're going to see a very big difference at the beginning than later because people get used to the thing. There's a very heavy statistics component to the core competency of your company. Where did that come from? I mean, the first many employees were basically computer nurse from Carnegie Mellon.

46:22I think that's where that came from. We understood things like statistical significance coming in. Yeah, makes sense. Does it feel related to your crowdsourcing research or does it feel like you you brought to what on who was a super key hire and then, you know, made that a pillar of the company? I don't know if it's exactly my crowdsourcing research, but it's generally the first few people that worked at the company. We had a pretty good grasp of statistics. What I will tell you though is that it's funny. This is true for most of the things. I don't think this is unique to do a link. For most things, you know, we had a pretty good grasp of statistics.

46:56That's good. But the level of sophistication that we do analysis today compared to that is insane. I mean, we now have people with PhDs and statistics who actually honest to God know what they're doing. As opposed to we had, quote unquote, a pretty good grasp of statistics early on when we were figuring out how to teach things, you know, we kind of had an idea of how to teach things. By now, you know, we've hired dozens of people with PhDs in second language acquisition and the level of sophistication that they have in terms of learning outcomes and stuff like that compared to what we were doing at the beginning is night and day.

47:32So we had some expertise for a lot of things, but today we just we've become a lot more sophisticated about most things, including the statistical analysis of our experiments. Knowing what I know today, I'm pretty sure that experiments from eight years ago, many of them we analyze wrong. What do we know? We're like, yeah, we look at averages, we look at statistical significance. We were not looking at the stuff that people look at today. In fact, I don't even understand what they're doing. So just very smart people come and tell me do this. I'm like, sounds good. So all this to me and I bet to many listeners sounds like the perfect to you.

48:04I'm like, AI, you are a perfectly suited company to AI, maybe just at the highest level. What's going on? What's changing? How is it impacting everything we just talked about? You are right. We are very well suited for AI. I mean, we've been the whole time. You know, AI, of course, means something a little different today than it did 10 years ago, but AI's been around for a while. But since 10 years ago, AI meant everything we were just talking about. Yes, statistics. You know, since we launched, we were very clear that what we wanted to do is we wanted to have a computer teach you a language, not another human teacher language.

48:42Another human can teach you a language very well. That works. No problem. But having another human teacher language is expensive. We wanted to have a computer teach you a language. So because of that, we're very well suited. We need to make a computer that is as good as a human teacher. And so we've been working on doing that as much as we can. Over the last couple of years, what has changed is these large language models. And again, I emphasize that the second L is language. We are a language learning company. Large language models are particularly good with language. And because of that, we've been using them a lot.

49:16I mean, so I'll tell you the two big things where we where we're seeing large language models be used. One is, it used to be the case that a lot of the data that we created for courses was half handmade. So what that I meant, we made tools, but usually a lot of the stuff was humans doing things. And then, you know, maybe maybe the computers would fix it up or whatever, but it's kind of half handmade. We've gotten to the point where most it's not 100 % but most of the stuff that that we're creating data for language learning is made by an AI. There's a couple of good things about that. First of all, it saves us money.

49:49So it's a lot cheaper than doing elements. But the most interesting thing that has happened is we can now create data so much faster. And it's not that we can go fast. Okay, that's nice. It's that that allows you to do things that you weren't going to do before. So for example, about five years ago, somebody pitched a feature on Duolingo, which was a type of lesson where it's kind of like a mini, call it like a two minute podcast where you basically learn how to listen in the language that you're learning. It's like, it looks like a little animated thing. We call it dual radio now. But it was like a thing where you could watch two minutes of something and listen to it and you're learning how to listen.

50:24And I was told that creating the data was going to take five years. And as soon as I was told that I was like, no, no, this feature red light here. Do not proceed. I do not want anybody working on five years of things. Something that I don't even know is going to work a very similar feature. Now we can create the data for this in a couple months. And so now I'm like, yeah, sure. Why not? What is it about today that causes that incredible thing? Well, all of that data is done by a computer as opposed to having it done by hand. I mean, for it to make a difference in the Duolingo language learning app, we needed to make hours of content, like many, many hours of content.

51:02And specifically in this case, you're talking about like animated graphics, not just animated graphics, the audio of it. Like actually, you know, what you're going to say, you have to make it interesting. So you have to make something that people had to listen that was interesting. It said, you have to do all that. It was super slow to make hours of content. And it was going to take five years to make content. And not only that, you have to make it in all the languages that we teach, we teach 40 languages. And we have to make hours of content. So that was going to take an inordinate amount of time.

51:29But now, you know, with an LLM, you can do it pretty fast. And what we do is we do it. Then we kind of spot check it. And if we don't like it, do it again. We're very happy with that that you can basically do things that before we're just, it were not impossible to do, but practically impossible to like, why would you do that? So that's an important place where we use AI. The other big one is just conversational practice. We're working on this feature that some of our users already have. We haven't fully really launched it to everybody. But it is basically where you can have a conversation with one of the dual and go animated characters.

52:03And it looks like you're having a video call with an animated character. And it's a really good way to practice a language and actual conversation. That's something we just couldn't do before. Yeah, I've heard Mark Zuckerberg say about Lama use cases a few times or maybe it's about meta AI use cases that it's for people to practice conversations that they don't want to have yet in real life. But they need to sort of feel like they're having the conversation and get some authentic responses. Well, so that in the case of language learning, what's amazing about it is, look, you want to learn a language you have to be able to practice conversation.

52:33You just have to do that. And people don't want to do it. In fact, we've done insane amounts of user studies where people, they may tell you, they want to practice a language with another person. But when it comes down to it, they don't want to do it. There's some small fraction of extreme extroverts who like it. But the majority of people would just rather not talk to another person in a language that they're not very good at. But the beautiful thing here is that you can just talk to an animated character and you really don't care because you don't think they're judging you. Even though technically they're really judging you.

53:02So that's mainly what the thing is doing. But you just kind of don't think about that. Well, hey, that's just absolutely incredible. And AI and data also opens up the other direction for you to like you have an incredible amount of data within do a lingo that can be used and fed back into LLM's your own I would assume and others that would just be incredibly incredibly valuable. Does that open up a whole another new business dimension for you all? Maybe at the moment, that's not something we're doing. We're using the data for ourselves more than one billion exercises are solved by our users every single day.

53:39So we have the data and basically we're watching I know tens of millions of people learn a language every single day. We're using it to teach better. Today, we know that do a lingo is about as good as a classroom in terms of learning a language, but not as not yet as good as a good human tutor, like a one -on -one, good human tutor. The problem is those those are like $50 an hour, like a good one. We're not as good as that. It's hard to break the future, but I'm assuming that if you give us three years, we'll be about as good as a human tutor. Well, and it requires making an hour. I mean, if you're doing three minutes at a time, that's a structural advantage where you're just able to address people that will never, ever make the hour.

54:20Yes, and you can't have a paid human tutor that is there for three minutes and then another three minutes later. And then this... They'd have to be sitting in the stall next to you, just waiting around. Well, it's cool. I mean, it's funny. You say three years. I wonder if it's fast because you've got two tracks of progress happening that are accelerating. You've got the foundation models of LLMs themselves, which are getting better. And then you've got your data which you're getting, you know, what did you say? 10 million solved exercises a day or two? No, a billion per day. Sorry, a billion per day, right?

54:53So the quality of both of those are accelerating. So I'm really bad at predicting the future. How long things are going to take? Well, humans are and you're a human. So yeah, just this very hard. It's a matter of a small number of years to get as good as a human tutor. You're in this education space. Are there any education companies that you look up to as shiny examples of people who've built great companies or do you mostly look outside your sector for inspiration? It's mostly outside. Education has been hard because you want to start an education company. You're going to a lot of problems. First of all, for some reason, most education companies have decided to monetize by charging the school systems, which is fine, but we just can't look up to those companies because we're so different.

55:42We're a consumer company. We've having the past look at those, but this is very little to learn from somebody who has spent a lot of time making a sales force for K to 12 schools. I'm glad they're doing it. Good for them. It may work. I don't know, but it's not much for us to learn. So ultimately, for us, where we learn because we are a direct consumer, we learn more from games, companies like Spotify, Netflix, any of the meta apps. That's where we learn the most, but it's because we are a direct consumer. There's just not very many direct consumer scaled education companies. Well, there are, but they're all nonprofits with brands like CMU or Harvard or Stanford.

56:22But yes, they are there, but they're not apps and they're not touching 100 million people. They're touching a few thousand people. But yes, it's kind of interesting to think about. They started as every institution starts as this niche thing using the latest and greatest technology available. Then over time, you build a durable brand and you try not to jeopardize that brand in any way. And those words that I just said could apply to do a lingo and you're just like 75 years earlier on your journey than these other institutions. Like, is the goal to become a accreditation bearing institution where people feel like when you've graduated from it, it means something?

57:04I would really like that. We are doing that for language. We're trying to do that for language. We have a standardized English test. We're doing it for English mainly because that's where it matters. Of this test called the dual in going English test. And the idea is that this is a test that you can go and take it. You can take it online and it gives you a score and it tells you how good your English is. And that is actually valuable. For example, it is now accepted by most universities in the US. If you're a foreign student that wants to come to the US, you have to take an English test 98 out of the top 100 US universities according to US News and World Report Accept our test.

57:38We haven't quite connected the test and the app, but we want to. For some languages, it's already there. We're giving you an estimated test score. So if you just use the app, we will be like, oh, my test score is this much. And then you can go and certify the score. You know, what I would love to do is to be able to own the score for languages. So right now, if you ask somebody how much French do you know, the answer is usually something to the effect of, I took four years of high school French or intermediate, but that doesn't really tell you what we would love to get to the point is if when you ask somebody how much French they know, they'll say, oh, I'm a dual English 65.

58:14We're not there yet. Right now, people tell you they're using dual in going, they may tell you they're dual English streak, but that's something that we want to get to. And I think that'll be really valuable. You're right. I mean, maybe in 75 years, we'll just have a really stodgy stone buildings and a 30 billion dollar endowment. Well, it's funny. You took, I mean, like, what would the market cap of Stanford be if it were a, you know, publicly traded corporation? Like, it would be quite large. It would. And you know, we have this board member, Bing Gordon, who's who's amazing. And he'll happily tell you.

58:43He'll say market cap of a university is twice its endowment. Oh, wow. That feels like a lot multiple to me. I don't know where he pulled that from. But this is why he'll tell you. I don't know. I don't know where he pulled that from. But he'll tell you. So Stanford's probably, you know, I don't know what they're endowment is 30 billion or something. 40 billion or 50. I guess. Yeah. Okay. So it's a 80 billion dollar company. This is such an interesting thought experiment because the mission of education is so unbelievably important. But it has historically, at least, you know, last 20, 30 years, been so hard to invest in education technology companies because they so rarely are using modern business models, the technology that all the other consumer companies are using, the distribution methods that all the other consumer companies are using, they're instead sort of trying to snap to how do I sell into the education supply chain?

59:33And you just haven't like you might be the one example of this education technology company that said, I am not playing in that existing system. And that is how you invent something new. It varies on the day. But we're certainly the most valuable at tech company. We may be the most valuable education company. Some days Pearson is more valuable than us. Some days we're more valuable than them. But and what's interesting is we're doing it with just one app. And if you look at something like Pearson, they sell gazillion books and stuff like that. I mean, we feel pretty good about that that we really are actually able to teach better.

1:00:02But you know, we feel pretty good about that. All right. I want to close with a question on headwinds and tailwinds. What's the most insane tailwind that just makes things easy for you running dual lingo? And then other side of it, what is a headwind that you think you are just always going to have to deal with? And it's a thorn in your side. But that's all right. AI is certainly a big tailwind for us right now. I think there is a growing proportion of people who want to do useful things online. And I think we get tailwind from that. I think if you use tooling over 30 minutes, you don't feel the same as if you use Instagram for 30 minutes where you're like, my God, I just lost 30 minutes of my life.

1:00:43I do think that's a real tailwind for us. In terms of headwinds, this is more for me as a person. Employee problems are tough. You know, we hire the top of the talent pool. I'll use the word entitlement is there. A stuff for me. I grew up in a country where literally for two, three hours a day, there was no electricity. There are some people here who are like, why do we only have two flavors of coconut water?

1:01:16That's always hard. It's funny to connect that with something you were saying earlier. Part of why that exists is the XYZ Elite Educational Institutions of these days and their brands and their brand caption one. But if they were just a duo, Lingo score of how good a product manager you are, that would level the playing field a lot better. Yeah, exactly. We could get this people that weren't fed with a silver spoon as much. To go back to your comment of you grew up in an area where there were multiple hours of the day read and electricity. How much do you feel like growing up in Guatemala? I think you went to one of the sort of rare private English speaking schools to have the opportunity to learn English.

1:01:59How much of that formative experience for you contributed to you found into Lingo? It's a lot. I mean, first of all, in a place like Guatemala language learning is life changing. That was a big deal. But I think the other thing is I was fortunate. I received a rich person's education. Even though I didn't grow up rich, I was fortunate that my mother spent basically all her money on sending me to this fancy school. So I went to school with very rich kids. It's not like we were poor. We were not. But we were middle class in Guatemala, which probably for US standards is poor. But in Guatemala, we were middle class.

1:02:30I really saw the difference because where I lived in the neighborhood where I lived, the kids didn't go to the school that I went to. I could see how much more I was learning. I mean, it was pretty clear that I was learning, I don't know, twice as much as they were. That was a big thing when starting Lingo. We wanted to do something that would give you know, access to education to everybody. So it's a big thing. Me growing up in Guatemala had a lot to do with starting Lingo. Fascinating. Well, that's a great place to leave it. Luis, thank you so much for joining us. Ben David, thank you. Great questions.

1:03:03And Lissair's, we'll see you next time. We'll see you next time.