EP 87: Luis von Ahn (CEO, Duolingo) on The Marketing Formula that Propelled Duolingo to 500M Users

30:25How did you hear about us? And we did a lot of number tension there. And we figured out there's really a very high correlation. People actually answer pretty truthfully there. It's not 100%, but it's pretty truthfully. And we use that data and we're now able to really figure out where things come from. And that really works. But at first, all we knew is that a lot of people were watching our stuff. Um, and we weren't paying for it, which, which was great. I mean, people were watching our stuff and we weren't paying for it. How did you, I mean, this is something innately a founder can do, I guess, but you're, you're taking a chance by making your brand and the icon of the owl, uh, into a meme even more than just the internet's already doing.

31:10Right. And so how did you think about like, clearly you must have a good sense of humor to find this funny yourself, but there's a risk associated with that, right? You're a learning app. You don't want to be taken as a joke necessarily. There was a risk and we discussed it. There was a risk and we discussed it, but in the end, you know, it was a risk, but it wasn't, it didn't seem like a existential risk at first. It's like, well, I just published that video. Like, what's the problem? And, you know, it kept working and we kept doing it more and more. I think that was, we kind of boiled the frog into it.

31:44I mean, it just didn't seem like that big of a risk at first. And I did learn something pretty big here. We hired a person who was straight out of college. And Cami hired this person, a woman named Zaria, who was amazing. I remember her hiring meeting. Cami was trying to explain that we should hire Zaria. And, you know, I saw her resume. I saw the whole thing. And I'm like, why are we hiring this person? I don't understand. And Cammy's like, trust me. Look, you have to hire some young people because they understand young people. You don't anymore, please. And just trust me on this. I'm like, okay, Cammy.

32:30And Zaria came on. And pretty quickly she said, you know what? You know where young people are? And TikTok. I'm like, my God, TikTok. Fucking people dancing. I don't understand it. and uh it's like look let me make this video they made a video they showed it to me i'm like i i don't even understand what this is and i have no idea what this is in fact it's not going to work because nobody's going to understand this this is what i told them but they said well will you let us and share carry on i don't think it's going to work anyways they posted a couple of videos and they went crazy viral. Like, yeah, I don't know how many.

33:09I lost track. 20 million views on crazy viral video. And I still didn't understand why it was even funny. I just did not get it. And one of the things that I understood, which would be a major shift, you know, most of my time at Duolingo, and even before Duolingo, I spent designing products. When you design a product, it is important that every single one of your users understand what you're doing. It needs to be understandable. And I was applying that lens to our marketing. I thought, well, everything we put out has to be at least understandable by the world. This thing you put out makes no sense to me.

33:42How? In fact, most people won't understand it. And what I learned that with marketing, that's okay. Especially with things like social media that targeted to the people, you know, somehow TikTok targeted it to the people that understood it. And that was fine. And so, you know, that worked out. And we started really leaning on TikTok quite a bit. And then we started getting worried that we were too reliant on TikTok, at least for marketing. So we started figuring out how to get better at that. We tried the same thing on YouTube Shorts and it works on YouTube as well. So by now, our TikTok and our YouTube provide about the same number of new users to Duolingo, which is awesome.

34:27The other thing is we started replicating the same formula in multiple countries. So now we have accounts, TikTok accounts and YouTube accounts on, you know, in Germany, in Japan, in Brazil, in the Spanish speaking world, in a bunch of different countries in Vietnam. And they've all succeeded and they all use the same formula. And so, you know, it's worked out really well. but the formula, you know, there's a number of things, but I think the biggest thing about the formula is if you look at our TikTok stuff and our marketing stuff, it does, it's not selling to you. It's just funny crack. And like, there's no button that says download now or buy now or anything like that.

35:13It's just, we now have built a personality for the owl and people love it. And I think that that really gets a lot of people, you know, it's basically free marketing. We don't we don't pay for that it's not true that we don't pay for any marketing but that that we don't pay was there a session in which you needed to come up with what the personality of of the owl was it's just it's iterative it's it's evolved and um it really didn't mean i would say the first glimmer of a personality came this was many years ago from the passive aggressive notification and this one was one i added and it was it was as follows and but i didn't think it was going to to be this big or anything.

35:54It was, um, it was because, you know, we've always stopped sending notifications. We stopped using Duolingo. So we, you know, if you don't use Duolingo, we stop after like, it used to be about five days. We're like, okay, after five days, you know, whatever you've given up, we're not going to spam you forever. And it occurred to me, you know what, if we're going to stop, we may as well tell people that we're stopping. Why not? So the last notification on the fifth day that we started sending was these reminders don't seem to be working. We're going to stop sending them for now or something to that effect.

36:26And it turned out this we didn't expect. It turned out that that really got people to come back because they, you know, it's this passive aggressive thing that they, you know, they felt bad that Duolingo was giving up on them. And that is that is the first glimmer of personality that was just like, oh, this owl really wants me to be there and he's going to be passive aggressive. And that's I think that's kind of what gave rise to a lot of the memes. but I mean, you know, we didn't. And, and so the personality of the owl has really evolved over time. And it's been kind of co-created by a lot of different Duolingo employees, but also by the community because they've made a lot of this stuff.

37:04And so there, there hasn't been, no, what we have had meetings about, especially for some of the videos, you know, with a lot of the videos, we, you know, we are learning brand after all. And so we don't want to do things that are lewd but with a lot of the videos we are um we're unhinged and so um we do have a system in place for not posting things that we're going to regret later the system has failed a few times um but we do have a whole approval mechanism uh for not posting really crazy crap What do you do when it feels it's crossed the line and something just offended people in a way that you didn't expect?

37:50Do you just take it down and move on? We usually take it. I mean, that hasn't happened too many times. We do take it down, by the way, I don't know, good or bad. Usually the most offended are the bilingual employees. um so we the times we've taken stuff down is more often than not is because somebody in the company or a group of people in the company are like how could you do that um and so that's that's what has happened but you know but there was only one time that where there was a major major backlash and publicly which was when we commented on um the you know i couldn't have told you this was going to be that bad there was a our account our tiktok account left a comment on a video about the johnny depp amber heard thing and the comment said something to the effect of like you guys think amber heard is on tiktok or something like i think that was the comment And there was a huge backlash because somebody posted saying that Duolingo does not care about spousal violence or something like that.

39:06And there was a huge backlash to the point where our social media team had to make their personal accounts private because they were getting death threats. Wow. So that's when we learned to stay away from... You know, there's certain things we just stay away from. Even if our comment is, you know, I don't think that comment was particularly incendiary. But, you know, there's some things we just stay away from. We had one one time. We have a wool joke on TikTok and Instagram reels and YouTube shorts. And we had one one time that was, it was when the market had corrected. And it was VCs six months ago versus VCs today.

39:51And the joke was VCs six months ago were spend as much as you want. Like, are you are you really burning as much cash as you can? We can give you more money, blah, blah, blah. Then VCs today were, are you sure we need a marketing team? Is that something? And the joke was that the VCs were flippant in their advice, right? Right. People took it as we were making a joke about layoffs of a marketing department. Right. And it was the joke was about us and VCs, but I understood where they were coming from. So we got a big that was a big outrage. And, you know, it was something that we ended up. I think we apologized or something.

40:37But you just never if you're going to take chances at these lines, inevitably, you're going to touch one and you're not sure. I stand by that was just a good joke. And we were making fun of VCs and not founders. It's impossible to tell. It's impossible to tell what somebody's going to, you know, and herd mentality takes it, you know, they kind of go against you. We haven't had a, that was the last time. This was maybe like a year and a half ago or whatever. We haven't had a big problem since then. And I think it's important, by the way, if you're running a marketing team like ours, which I think is a major home run machine, to be okay with it.

41:13So, you know, what I did when that happened, I mean, they thought I was going to fire a few of them. And I actually walked to the desk of, in fact, the person that posted it, and I said, it's completely okay. To stand out is to risk being polarizing in some way. And so inevitably, there's going to be some people that don't find it funny. And inevitably, there'll be some percentage of people that are offended. And it's hard to say where it's going to come from. So that's great that you went over it and said that. Yeah, the checks and balances are important in trying to get enough diverse eyes on the things so that you can understand where the totality of your audience maybe is going to react to it.

41:51But it's the cost of doing business in some ways that inevitably you're going to get hit with some of these things. Yeah, we're going to continue. I mean, and by the way, we've had a number of things that we've had to pull. I mean, the most recent one, we had a hilarious video that the marketing team just made. It was hilarious. that we were going to post close to Halloween. We had gotten a famous person in there. It was great. But it had kidnappings. And the Israel-Palestine conflict erupted right around that time. And so we didn't post it. We were so sad that we couldn't. I mean, obviously, there was no way we should have posted that.

42:32But in other times, this would have been an amazing video. What about being scrappy on the brand marketing side? I'd heard a story about you guys just hanging around, waiting for billboard inventory to pop up, and then being able to take advantage of it. Can you tell that story as well as other examples of just sort of being scrappy about opportunities that present themselves? Because historically, our marketing budget has been pretty small. I mean, it's still pretty small. Our marketing budget now is about$50 million a year, which for a company with our revenues, it's about a fifth of what it should be.

43:10You know, we are very scrappy. I mean, there's this story with the billboards. And the billboard was going to be a funny billboard. This was a billboard not necessarily so much for marketing Duolingo, but it was for getting employees. So we just put a billboard in the 101 in San Francisco that said something or other like, own a home, work in tech, move to Pittsburgh. And it just said Duolingo or something like that. Yeah, I mean, we only got that because we got a really good deal. We just hung around for a long time until we got a really good deal. But we used that same idea in a lot of ways.

43:52By the way, that is exactly what we do with our performance marketing. So we do performance marketing. We don't do very much. We do, you know, I don't know, on the order of 10 million bucks a year on performance marketing, a little more than that, but not much more than that. And that's what we do. And what we do with performance marketing is we noticed we were in a really good position, which is we don't particularly care which country our users are coming from. We don't particularly care at what time we acquire them. We don't need to acquire them. There are some companies that need to acquire them for a specific date or for something.

44:29We just don't care. And so one of the things we do with our performance marketing is we just have a system that basically looks at the whole world. And so if, for example, right now ads are too expensive in Italy, we just don't do anything in Italy. We move all our budget to France or whatever. If it's too expensive in France, we move it to Vietnam or something. So we're always shifting budget a lot between countries because we're in this great position where we operate in every single country in the world or have users in every single country in the world. And we are agnostic as to where they come from.

45:04And so that's the type of stuff we do. We're very scrappy to get budget. I heard when you were starting Duolingo, you had no experience in teaching languages. languages and so you read a bunch of books and they all contradicted each other or they they had mixed messages on how to learn how to teach i i don't know if that's true but if so like how did you go about figuring out what the core nucleus is of teaching language and how to start we didn't i didn't know how to teach languages i mean i knew i knew spanish because that's my native language but i had never taught spanish um severin my co-founder knew german because that was his native language and he'd never taught german and so yeah we we read a bunch of books we you know there were some commonalities about what all the books said how to best teach a language and we kind of took those unfortunately the books didn't agree with each other they all had their own method and they're all you know um so we took the commonalities and then we did what we thought made a lot of sense by the way we made a bunch of dumb mistakes that only computer scientists would make so for example this was a this was done um and we thought hey this this this kind of approach to deconstruct everything language is just words so all we do is teach you words and how to use them and if you know all the words and how to use them you'll learn the language done and so we're lucky though so that's what we're going to do and so the first we are one of the first versions of duolingo had just we just were teaching you a list of words ordered by frequency so you know what the first word the most frequent word is the teach you the first then the second most common word whatever the hell that is teach you that word etc turns out that's a very bad way of teaching um and uh you know it's also not true that because you've learned enough words to cover the first like 30 words or whatever cover something like 20 percent of language it's not true you know 20 percent of language by knowing the first 30 words that is just absolutely not true.

47:06But that's kind of how we thought about it. So this is the type of stuff that we were doing. Over time, we learned that we could run A-B tests to figure out how to teach better. And so that we started getting better at that. So we started kind of figuring some stuff out ourselves. But then we got significantly more pro when we hired our first person with a PhD in second language acquisition um that was a we should have done that earlier um yeah we got a lot more pro uh and and that that has really helped and now what's interesting is when we got these people with phd's from second language acquisition it's not like they know how to teach with an app they didn't because they know how to teach in a classroom setting um but we knew how to teach with an app now we had our own wonky ways of teaching with an app and they had their own very professional ways of teaching in classroom settings.

48:01What has been really amazing is kind of, you know, they've learned how to take everything they know. And some of it applies to an app. Some of it doesn't. They learn how to make that kind of closer to the app. And we with the app and with the A-B testing and everything have learned how to go more closer to them. And so at this point, you know, I can say that compared to 10 years ago, Duolingo teaches way better. But we've understood that actually there's just all these skills that you need to learn and that they don't carry to each other. Like, for example, you can be amazing at reading a language and have no idea how to speak.

48:37And we just didn't understand that at first. So now, you know, we try to teach you all the different skills that are required to learn a language. And, you know, we understand that the way to teach better is at first you really need to teach things about how to, you know, how to talk about concretely about things and greetings and stuff like that. These are not the most popular or the most kind of high frequency words at all, but they are very useful to get started. And so that's, I mean, we've learned that. In the early days when it was just you and Severin, I think we talked about this a little bit before we started recording, but neither of you actually retained as users of the app necessarily in trying to learn new languages.

49:22Was there a single product unlock or an aha that you realized about how to retain users or teach people through it? Or was it just a continuous process of these little tests to get incrementally better every day? So what happened with, this is before we launched Duolingo. You know, I made the first version of the Spanish course. And Severin made the first version of the German course. And we were going to learn each other's language. which I didn't know any German and he didn't know any Spanish and we were going to learn each other's language. And we ran into this problem that neither of us wanted to do it.

49:57I mean, we would come into the office and we would say, hey, did you do your Spanish lesson? And you'd be like, no, man, so, so boring. And I thought the same thing. It was so boring to learn German. And that's when we realized that the hardest thing about learning a language by yourself is staying motivated. And so that's when we started like adding, you know, adding all kinds of game mechanics, like progress bar at the top. At the end of a lesson, we give you points. The lessons should be short. There's all this stuff. And this is before we launched. So we weren't A-B testing anything. But we added a bunch of stuff so that at least we could swallow taking a lesson.

50:40I mean, it's like, okay, well, at least it doesn't feel like a complete chore. it was yeah kind of fun and that's you know we launched with that um and so by the time we launched it was kind of fun it wasn't as fun as it is today but it was kind of fun we had done no ap testing it was just at least with us we had made it you know kind of engaging and but but the the learning remained which was just um making us engaging as possible and so that continued What did you think this could be when you were starting the company? You had been independently successful selling reCAPTCHA to Google. And so you were able to be a little bit more mission driven in your pursuits, I think, which my understanding was that was an important consideration in the early days.

51:30Did you think that this would be a commercial potential public company? Or was it, hey, this is just a good service, and maybe it's a good business along the way? I never thought this would be a public company. I thought, you know, I thought we could do something that would help teach people, mainly English. I thought that that was going to be there. I mean, I didn't know how big it was going to get. When we were launching, Rosetta Stone was like huge. We thought at the time it was huge. It's not even clear that we thought we were going to get bigger than them. We just thought, well, we're free.

52:08And we're at least going to let people, you know, learn that can't afford to pay. And in recess, it was super expensive. It was like 500 bucks or something. Like, okay, well, we're going to let people learn that, you know, when they don't have to pay us 500 bucks. Before we launched, I thought if we ever get to about 100 ,000 monthly active users, that would be success. That's what I thought. And, you know, that's funny. I mean, we get way more than that new users per day now. There's a great YouTube video. I think it's around the launch of Duolingo, whatever, 14 years ago or 12 years ago or whatever it is.

52:51But I believe it's on that video you make reference to this concept of the cap on human coordination that historically had been around 100 ,000 people. I think that was true of the pyramids, Panama Canal, getting a man on the moon. That's been when physical objects were involved. That was the most we could scale to as humanity. Can you talk a little bit about that and how you thought that as an influence of starting Duolingo? We always wanted Duolingo to be free. And it was one potential way to monetize it. We ended up not monetizing this way, but there was one potential way of monetizing it, which was, look, maybe we can get a lot of people learning a language on Duolingo.

53:36And at the same time, while they're learning, they're helping us translate stuff. Because at the time, computers were just not as good. Today, they're essentially perfect. But at the time, they weren't as good as they are today at translating stuff. So we thought we could, rather than charging people to learn a language or using ads, we thought we could use their work to help translate stuff on the web. And then we could make money from that. So, for example, the idea was going to be something like, well, if you write a news piece in English, like if you're CNN and you write a news piece in English, you can send it to us.

54:08And our users, while they're learning English, they could help translate it into Spanish because they're native Spanish speakers. And that's part of their learning. And then once it gets translated by our users, we send it back to CNN and then we charge them for the translation or something like that. We thought that was potentially a good business model because no money was involved, at least for the users. Turns out this is a very bad business model. So we ended up not doing that. But that was one of the original ideas with Duolingo to try to make money while keeping it free. It turns out a much better way to make money while keeping it free is to use a freemium model, which is you can learn as much as you want, but you may have to see some ads.

54:45You don't want to see ads, you can pay us. Turns out that works really well. That was not clear back then that this would work so well. Yeah. So what year was it when... So Layla Sturdy from Capge invested and told you, hey, you got to figure out how to make some money here at some point, right? What year was that? She invested in 2015, I believe. At that point, Duolingo, she invested a valuation of about half a billion dollars. With roughly how many users? Zero revenue, plus or minus. Zero revenue. We must have had. Yeah, I don't remember anymore. Or call it, I'm going to make up a number, call it single digit, maybe one, two million daily active users.

55:30One, two million daily. Daily actives. And if you looked at monthly actives, it was probably, call it, I don't know, six, seven million monthly actives, something like that. So a rich valuation, certainly for those times. The good news is we're growing a lot. so we had just you know we the previous year we had like 10x so it looked like we're growing a lot which was good and but you know she said to me look um great product great team um yummy money and uh yeah she took me out to drinks after my third drink she said look at you're not gonna find uh she's used exactly this word you're not gonna find a bigger fool than Google.

56:20This is it. We invested. We like you. This is great. We believe in you, but you're not going to find a... You're not going to go get a valuation hike from anybody else. You're not going to do that unless you make some money. So get it going. And then I got to start some other types of... Well, and so now at this point, you have a very mission-driven company that are very aligned with your democratizing access to learning languages and all that. And so you come back to them and say, hey, you're a little hungover after a couple drinks with Layla, you start noodling it, and you say, hey, we've got to figure this out.

56:59I assume not all of the company was excited to start showing ads or charging subscriptions. Is that a fair assumption? Yeah, no, they really were not they were in fact um entirely against it um most everybody that joined duolingo joined because of the mission to really deliver education to everyone um and so you know i said to everyone we gotta we gotta you know somehow pay for this whole operation and what everybody you know first question everybody said is why well you gotta pay for your salary and listen you've been paying our salary this whole time what's the what's the problem you know that's when i kind of told them kind of what Layla had said.

57:38You can't raise venture capital forever.

57:45And, you know, people finally were like, okay, okay, fine. And I thought this took like months for people were like, fine, you got to make money. How are we going to make money? By the way, when you have that, I mean, a lot of our companies have dealt with different versions of this, in particular now trying to get people back into office or whatever it is, right? Different social issues or the mission-driven nature of a company and trying to corral it in some way, shape, or form. I think as we're recording this, there's this large business called OpenAI that's maybe going through some elements of this.

58:18How did you actually go about building consensus around getting people on board with this? It sounds like you didn't do it by edict and say, hey, listen, this is the way - No, no, it had to be consensus. New England was generally a consensus-driven company. It had to, I mean, the way that works, we were, I don't know how many employees we were, I can call it 150, maybe keep or take. You know, there was clearly a spectrum of how much people cared and also a spectrum of how influential they were. And, you know, I spent time with each individual who either cared a lot or was very influential, like one-on-one.

58:58And some of them were easy to convince. There were a few stragglers who were just like really resisted the idea um but you know it took it took a period i don't know how long it was a couple of months of you know people being you know upset at this and then when they started seeing the mocks of how they became even more upset when they started the first time somebody saw a build of doing with ads in it people were like what is this. That's awful. And it's because, yeah, it's true. I mean, the ads, the truth is the Duolingo product experience is quite polished. And then you put an ad, which is looked like, sorry, but most of these ads that you play on apps look like crap.

59:40Yeah. Um, so yeah, it, it, it took, it, it took quite a bit to, to figure that out to, to, sorry for people to get used to it. Um, but you know, the explanation that really resonated most with everybody is this, if we make money, we can use it to really speed up our own mission. And it was a hundred percent true. As soon as we started making money, we started growing the employees a lot. Instead of having like four separate product pipelines. Now we have, I don't know how many product pipelines we have. We have like 50 separate, you know, kind of lanes where people are just improving the product.

1:00:19And that's all because we, you know, we make revenue now. How did you think about the line between subscription, which my understanding is that drives is a very small percentage of the users, but a high percentage of revenue, and then advertising, which is the opposite. When you were thinking about this philosophically, how did you come up with where the lines of demarcation were? We didn't have too much philosophical. It was a lot of detesting. The only thing that we decided to do is we're going to put an ad on the free pop that we're going to do at the end of the lesson we would put one ad that you know after five seconds is skippable we're going to do that and we decided we were going to do and then we added an edict that said we're not going to degrade the free user experience further than that that is it we're going to be graded by putting one ad at the end of the lesson don't degrade more um we didn't know whether ads or subscription was going to give us you know that much more money in retrospect.

1:01:21I think subscription businesses are a lot more well understood now than they were, at least digital subscriptions like that for apps, than they were a while ago. In retrospect, this would have been obvious, but at the time, it was not clear to us that ads or subscriptions, which one was going to be the killer one. But after running it for about a few months, it became clear that subscription was so much of a better business. um and so yes up to a a give or take you know eight percent of our monthly active users are paid subscribers and 92 percent use it for free yet for a revenue give or take 80 percent comes from the subscription and you guys philosophically will not charge people for content right like that's uh you don't want to become rosetta stone and gatekeep content too much correct we don't charge people for content.

1:02:13The only things that are okay for us to charge for are things that we have to pay for. So for example, um, we now have a tier in Duolingo, which is Duolingo max, which has some features that have, um, large language model usage. You have to pay for that because we don't have our own large language model. So we have to pay for that. And because we have a variable cost on it, we do put it behind, um, um, you know, you have to pay for it. Um, because otherwise that would be a major money loser for us there's a rule i've heard you guys have that if uh if five people or less can build something as well you won't charge for it is that is that still true and how philosophically do you think about that this is you know it's one of those internal issues called the five germans and a dog there's five germans and a dog can build it uh why are what why can because you know the reason we decided not to do it is because that you know that's how you that's how honestly that's just how we beat rosetta stone i mean we weren't german but we just made a thing that was free and it was not made by 300 people it was made by a very small number of people and very quickly we took them over in terms of usage not in terms of revenue because we're making any revenue but in terms of usage we took them over very quickly um and so yeah, we're going to get disrupted.

1:03:37If we build something and start charging for it, that can be built so easily. Some company that has a small amount of funding, I mean, for five people, you need five million bucks of funding or less than that. They can build it and eat your business. So that's been the rule. We're getting less worried about that these days. That was more of a worry a few years ago when our dominance was not as big. I mean, at this point, it's just we have a lot of moats and it's become pretty hard to compete with us. So we're a little less worried about that. But that was a worry. That was a pretty major worry, I'd say, you know, six years ago.

1:04:14So now you're actually branching outside of languages. How do you think about doing that? And then what's the hardest part? Is it getting the brand? Is it the content? Is it rebuilding elements of the product? What's the hardest part of doing that? It's creating the whole learning experience. um you know we're branching out to math and music um there's some differences in how you teach math and music to languages and so we're creating the learning experience that's to me that's the hardest part what's the most interesting difference of how someone would learn a language versus math or music well math needs more explanation than languages um it's also the exercises and there's a lot more graphical elements to it it's just a lot of math i mean some of it is just numbers but a lot of it, especially if you're going to teach geometry and stuff like that.

1:05:02It's a lot more graphical stuff. And you have to start really... There's a lot of things you have to figure out that we've now just figured out with languages. We figured out, for example, with languages, I'll tell you. It took us many years to figure this out. And in retrospect, it's obvious, but if you're going to teach a language, you should align yourself with this thing called the Common European Framework of References, called the CEFR. Just do that. Don't deviate. There's no such thing for math for math there's something like it's called like the common core but it's not clear to us that that the common core is as good as uh cefr and so that's the type of stuff that we just don't know and and also for math um we're not entirely sure who our audience is there's our audience we want but there's an audience that we may end up getting that is not what we want what we want is everyone we want to do for math what we did for languages which is you know 80 percent of our users were not learning a language before Duolingo.

1:06:00We want it to be the case that anybody who's 25 years old or 30 years old or however, or they're eight years old, they just use Duolingo to learn math and they get better at it. We want that to be true. It may end up being that the only people we get are the people who are forced to learn math because we just can't make math fun enough. But what we would like is to get to everyone. And with math, this is a real open question. Can we get it so that your average 35-year-old decides to spend 5 to 10 minutes a day getting better at math? Can we do that? I don't know. That is what the product team is tasked with.

1:06:38And we have a really good product team, but I don't know if they'll succeed. Because it's interesting, and you alluded to this, of getting to spend 5 to 10 minutes a day. but you don't view your competition as Rosetta Stone or a private language tutor or whoever. It's like, how do you think about your competition and what you're going up against for that individual? Competition is time. And, you know, if you want to be more precise, it's time on the phone. So a real, you know, when we talk to our users and we ask them, you know, the ones who like stop using Duolingo, would you stop i don't know we've done i'm sure they exist but i don't know we've done any user research where they've told us they've gone to a competitor like a like a language learning competitor that's not what people say the most common thing that people say well the first thing they say is i haven't stopped which they're just lying about but once they once you convince them that yes you know that they have stopped they'll tell you well to be honest i'm just spending more time on TikTok or Instagram or whatever the hell it is.

1:07:43So our real competition is, is, you know, the apps, games, uh, whatever it is, the, the things that take up your time on your phone. Is there an optimistic point about humans and their desire to learn embedded in that, that they're willing to waste time on TikTok or Instagram, which presumably I think most to that is uh or spend it learning a language if the language experience is comparable or enjoyable or easy like how do you how do you think about the i believe that obviously we haven't gotten there i mean obviously tiktok has more uses than duolingo but i do believe that um as long as you're close to as fun as tiktok or as engaging or however you want to call it addictive engaging however you want to call it as long as you're close i think we have a good good shot because afterwards the good news about doodling was afterwards you don't feel like you wasted your time you at the very least got somewhat better at spanish um whereas if you spend 30 minutes you know scrolling on instagram no you may learn a couple of things but really mostly time wastage um and so i we think that as long as we're as close as long as we're close to how engaging that is we're in a good spot and and again i think we've gotten somewhat close but we're not there uh you know we have i don't know the latest numbers we released something like 24 million daily active users we don't have 500 million daily active users so we got to get there yeah There's room to grow.

1:09:24In terms of artificial intelligence, you all have been fairly early in terms of using this. Can you talk about your original desire to maybe build some of this stuff internally and then where you all are in the evolution of using tooling and what that's going to mean from a user experience standpoint? Yeah, for artificial intelligence, you know, we've always, from the beginning for Duolingo, the idea was we're not going to use humans to teach you. there's a reason for that uh because humans are expensive um you know if we want to teach literally whatever hundreds of millions of people you gotta find a lot of teachers and they're expensive and you gotta pay for them and we're just not going to use them basically we're gonna teach you with with a computer um so that has always been the idea and if you're going to do it with a computer you know you got to make it so the computer can teach as well as a one-on-one human tutor and so we've been working really hard to use ai to teach as well as a one-on-one human tutor um there was a major shift of course in the last year call it because of large language models i and i don't think almost anybody else could have told you five years ago that large language models were going to be this good um turns out they're extremely good so you know immediately when they came out, we started applying some of our AI efforts into that.

1:10:52That's not the only place where we've used AI. We use AI for a lot of different things. But at least for conversational practice, this is the thing we were never able to do. Without large language models, we just could not build a very good conversational partner. But today, we can build a really good conversational partner with a large language model. How has going public made you all operate differently? Has it been a good experience? What would you recommend for people maybe thinking about it? For us, I think it's been an excellent experience. For me, it has been a really good experience. I think there's a couple of things.

1:11:30I heard this from Rich Barton, who started Zillow and a number for their amazing companies. He said, look, going public is like joining the major leagues. Everything just operates at a higher level. And I believe that. I mean, just everything started operating at a higher level. You just are forced to do all these things that you didn't have to do before, so you don't do them. It's like when you get no visitors in your house, you're not really forced to clean. But if you're going to have visitors every day, you keep a pretty clean house. It's the same thing. Everything just, we operate a lot better, I think.

1:12:04For me in particular, I've been very lucky that we have a really good CFO and finance team. So I haven't had, my job hasn't changed all that much because a lot of the public company stuff, it's done by our finance team. And other than just regular, doing regular earnings calls, it hasn't changed all that much for me in particular. But I think I've had a fortunate experience for a number of reasons. But I think probably the biggest reason why we've had a fortunate experience is that we got very good at forecasting. And we don't overpromise. And I think we've historically haven't. We haven't just been one of these companies that completely overpromises.

1:12:52And so I think that has helped. Why do you think there haven't been more successful consumer companies? As we look today, you all are at your all-time high, despite having gone public. Everyone else that has gone public in the 2021 vintage or whatever are definitely below their all-time high. I would say there aren't that many businesses that have been created between$5 and$20 billion,$25 billion in equity value within consumer. Do you think there's some innate cap that you either get super big, like a Facebook or a Google or a Snapchat, or it feels like there's just not a lot that exists at that level versus enterprise software.

1:13:41There's a ton. Do you have a perspective on that? I mean, we ask ourselves this question that there does seem to be something to once you get kind of big enough, you just immediately turn into like a Facebook. There does seem to be a little bit of that. I think another thing that has happened, certainly for a lot of kind of the smaller consumer things, I think a lot of them were fooled by performance marketing. I think we were very fortunate not because we were so smart. It's not that we were so smart. It's just because we were not monetizing, we didn't do performance marketing. But performance marketing is such a drug.

1:14:24It's such an addiction. and once you're in it, you can't stop it. And the problem with it is that really cuts into your margins to the point where it's very hard to ever become very profitable when you're relying on a lot of these consumer apps that are not the Instagrams of the world. If you look at where their users are coming from, it's overwhelming with performance marketing. When you look at like 80 % of their users are acquired or something like that, you're going to have a shitty margin. That's that. And so then, then yeah, investors are going to think you're a shitty business too, because you're a shitty margin.

1:14:59Fortunately for us, we never did rely on performance marketing. And so our margins are really good because we don't do that. And I think that's it. I think, I think in retrospect, I can understand that again, it's not because we were so smart back then, but in retrospect, I can understand, look, the truth of the matter with performance marketing is that companies like Facebook or like Meta or, or Google are excellent at extracting all the value for that. Like, you know, if there is more value to extract it, there's more margin to extract, they'll take it. Because that's what they're good at. And so I think that's another big reason, I think.

1:15:38Interesting. So Duolingo today has approximately how many people? LinkedIn says 1 ,700. Does that sound about right? No, that's a lot of, I don't know why that is. There's some contractors there. Who knows what else randomness there is. We have about 750 employees. 750. Okay. I had heard you or Severin, your co-founder, have been or were in every hiring meeting. Is that still the case? Both of us. Both of you. All 750 people you've met with. Well, we've hired more than that because we've also lost some people. Yeah, I guess. And by the way, not only that, we've also turned down a lot of people.

1:16:20So yes, we've been in the hiring meetings of thousands of people. How do you make that work time-wise? And how is that a principle? Every week, we just do it every week. There's a meeting every week that lasts about half an hour where they present us. The hiring manager comes and presents the case for the person that they're getting to catch. For semi-roles, this goes super fast, particularly for more junior roles. Sometimes these meetings take a long time. And we may go through a lot of back and forth. but you know it works it's not like i mean again it's not like the majority of the people that apply for duolingo get rejected much earlier than that the only ones that get to us are the ones that where everything looks pretty good and so they get to us so it's not that many per week it's got it so it's not that you get you don't meet every employee before they join but you sign off we used to not we used to meet every employee that was not that that was a while ago i mean and That probably stopped five years ago.

1:17:22At this point, it's really just the case gets made to us. Yeah. By the way, this is very important. And it's because of this. Everybody else but us, and it's not because we're special people. I'll just tell you what it is. Everybody else but us has a true incentive to fill the role. The person from the talent acquisition team just wants to fill the role because they're being paid to fill the role. The hiring manager has a hole in their organization, so they need to fill the role. So everybody just has an incentive. So what happens is they're just like, yeah, this person is good enough. And so they come in with an incentive to hire, whereas we have an incentive to hold the line.

1:17:57And by the way, yeah, again, it's not because we're such great individuals with such great, it's just a matter of incentives because we do the same thing. Because I've seen when Severin wants to hire somebody, he also has this problem. And when I want to hire somebody for my team, I also lower the bar. And so we try to, you know, it's simply, it's not because we are such great judges of talent or whatever. It's simply, it's a matter of incentives. And that's why we keep doing it. So it's kind of human nature. If it's yours, then you see the pain of having to restart it. And so that's exactly right.

1:18:30I mean, when I'm hiring for my executive team, I'm just like, my God, I've been looking for six months. Look, this person is pretty good. They're pretty good. They're pretty good. And then if I show it to somebody who's way more impartial, they're like, really? Come on. And so I think that's what it is. It's simply that. Beyond just being a good person, what's a non-negotiable for someone joining Duolingo? And maybe to the good person example, if you could also tell the car ride of what you all do. If people hear this, maybe they'll know now to be nice to... Well, we do a bunch of things. And, you know, we do that, you know, for us, it's important that people are kind.

1:19:14And one of the places where that shows up a lot is how you treat people who are lower in the organizational structure than you are. Everybody's pretty nice to me when they meet me and they want to work for Duolingo. But, you know, we definitely see how people treat some, and this is usually for more senior roles, where we look at, you know, how they treat much more junior people. And, you know, I mean, this is one of, you know, I've told this story before where, you know, we were looking for a CFO for a long time. And one person had everything look good. Everything look good. but um they did not treat their driver well and the driver is a person that has worked with us you know not it's not a duolingo employee but it's the same you know we have a few drivers that we work with but you send people to the airport you know to pick them up and you know we asked them was this person good like actually no they were pretty pretty nasty to me and we didn't hire them and that was it was painful for me not to hire them because i wanted to hire this person because I have been looking.

1:20:24This is an example. I had been looking for a long time. At that time, I probably had been looking for like nine months. And this was a very smart person, very competent, but I guess they were not nice to our client. And my sense is, if that happens, they're probably not going to be very nice to a level one finance associate or whatever. Beyond that, beyond being kind to people, are there certain traits that you look for that are non-negotiables? or is every role kind of specific beyond that? It depends a lot on the role. No, it's not the role, but more the function. So, you know, in the case of design, you know, we really look at people's portfolio and we really care about polish.

1:21:09You know, some design organizations are okay that some of the designers don't have as much polish as others. For us, every designer that we hire, we want to have polish. So we look at their portfolio a lot. It just kind of depends on the function. but i would say company-wide you know we want people to be kind and obviously really good at their craft um that that's just you know kind of obviously so there's one other thing that we do look for it's it's it's which which we try um we really want people who are um in one way or another exceptional and and we we look for that quite a bit and exceptional can mean different things i I mean, in some cases, you know, exceptional is the usual kind of tech company definition of exceptional, which is like went to Stanford, got a 4.0, et cetera, et cetera.

1:21:55And that's pretty exceptional. Honestly, very few people were able to get a 4.0 from Stanford. So that's pretty exceptional. But we look for a broader definition of exceptional. I mean, first person in the family to go to college, it's pretty exceptional. comes from a country, from a very poor country, actually grew up in a very poor country and somehow managed to, you know, go to the university in France. That's pretty exceptional. So stuff like that, you know, we try to make it so that everybody we hire has something exceptional. What have you learned about, I guess, both leading and managing over the course of running Duolingo?

1:22:39Managing, definitely I've learned how to not become, how to not be a micromanager. I actually believe that if you're a small company, you know, under 20 employees, it really is actually the thing for you to be a micromanager. Like, why not? After a certain number, you just can't be. And then after you start hiring very senior people, you piss them off. So I've learned not to be a micromanager. You know, there's a number of other things. I've learned that ultimately when there's a fuck up, not to point fingers to anybody but yourself. And we actually have made that pretty good part of the culture of doing it, especially in some functions where, you know, there's fuck ups happening every day.

1:23:23And we've gotten pretty good at, you know, it's just like, okay, well, keep going. and as a leader i think it's really important to be very very ready to blame yourself um i think a lot of leaders i've seen it with you know even with people who have in over the years report to me were they really trying not to get blamed for something like no no somebody else did it i think people respect it a lot more when you say look ultimately i'm responsible for it and you By that, I'm responsible for all the Duolingo fuck-ups. And we've had hundreds, maybe thousands. It's probably safe to describe yourself as an accidental entrepreneur.

1:24:10Is that fair? I did not grow up thinking I would be an entrepreneur. And then Captcha, then ReCaptcha, then Duolingo, I guess. When ReCaptcha sold, how many people did you have? Not a lot. A dozen. got it what what what part of the entrepreneurial journey uh do you innately enjoy the most i don't know if it's the entrepreneurial journey that i enjoy it's more and i enjoy doing things that have impact that part i enjoy so i enjoy i'm enjoying a lot what i'm doing i know with duolingo but it's because i'm still allowed to touch the product um the day i'm no longer allowed to touch the product, my job satisfaction will go down quite significantly.

1:24:57So I like that. I mean, I like tinkering and I like being able to have a lot of say in what goes into the end product of a consumer product. I've heard you say that working out has been a life-changing thing for you. When did that start and how has it actually changed your life? I mean, it started about right around when I started Duolingo. I just think it was a streak. I decided to have a workout streak. And I haven't lost it, actually. It's been, you know, however many years. 13, give or take years. Every day or every? Every day, every day, every day. I've not lost it. I did cheat a couple of times as follows.

1:25:45These flights to Japan. There's a problem with the date there. By working out twice the day before and, uh, you know, something like that. But, but yeah, I haven't lost my streak of working out every single day. Um, what's the most bare bone thing you've done to, uh, to make it work? Uh, uh, yeah. The New York times seven minute workout. Yeah. Got it. Um, look, because some, because you can always say you can not go running, but sometimes it's like horrible weather. You're in some random hotel that has no gym or whatever. And all you got is you. And yeah, you do that seven minute workout a few times and that's all you do.

1:26:29How has it changed you? Is it, is it the energy from it? Is it the energy? You know, I used to get sick quite often, which is funny, which I know I am sick, but I get sick once every like two years. I used to get sick very often before that. That's one thing. I think the other thing is it really allows me to regulate my own emotions quite a bit. But I think I, you know, I'm generally a pretty emotional guy, but I think that my wife says I'm Latin American, Latin American men. But I think, you know, it allows me to regulate my emotions, I think. Someone that maybe struggles with that or wants to get going, is there any advice you would have to them on like how to just start making progress?

1:27:17Is it just get a streak going in a similar way? Yeah, yeah, yeah, yeah, yeah. And by the way, that's what matters the most. Just get a streak going with anything. I mean, this is, this, by the way, it's the same with Duolingo. Look, a lot of times people are like, no, no, no, there's a, there's a better method for learning a language. And it's like, look, ultimately that's like saying the treadmill is better than the elliptical. It probably is. But you know what matters 95 % is that you do something. That's, you know, did you do it with a treadmill, with an elliptical, or one of those weird climbing machines or whatever.

1:27:48Sure, probably there are some differences. But between not having done anything and having done even the dumbest one of these, it's better to do the dumbest one of these. I mean, and that's the thing with Duolingo. We're optimized for keeping you going. Is there another habit or anything you've picked up on maximizing productivity and balancing your work-life balance and all of that stuff? I mean, I'm pretty efficient at most everything I do. And I think that's, you know, a lot of people comment on that. I do most everything very fast. I eat very fast. So I don't know if it's a habit that I pick.

1:28:21For some reason, I'm just very fast at most everything I do. I'm the same way eating, by the way. It's always been a problem. People think that I've been to jail. Yeah. People think that I've been to jail because I eat so fast. I eat so fast. But the truth is, I love food, but I don't savor it in my mouth for five hours. Yeah. That's funny. Anyway, one of the things I'd be curious about is ReCapture was sold in, what was it, 2008, 9, 10? Nine. Nine, okay. And at that point in time, I don't think it's ever been disclosed, but you didn't raise any outside money and sold for, you said, tens of millions of dollars.

1:29:04And now Duolingo is an enormous success. I'm sure we can look up in the S1 how, you know, your ownership at time of IPO. But safe to say you've made more money than you can probably spend between both of these things, unless you want to go buy a sports team or something. But I assume, I don't know if that's in the cards for you. Depending on the sports team. Depending on the sports team. Well, in that case, then there's a lot of stuff. There's a lot more that you can keep going to buy the sports team. What would be the sports team of choice if you could? I mean, I don't know. I like soccer. Yeah, okay.

1:29:43I'm not that crazy. I mean, I like soccer, but I'm not. It's not going to be the Steelers or one of the Pittsburgh teams. Okay. Well, the Pirates would be cheap, but. Yeah, that's right. That's right. How has that most changed your life or how you view the world, having that level of success? asked, it seems like you still live in Pittsburgh, presumably a similar-ish life. But is there any way that it's changed to the positive versus the negative on stuff? Yeah. I mean, you don't worry about some of the stuff that you used to wear. I mean, I remember when I was a graduate student, you worry about like, are you going to have enough money to pay the rent and stuff like that?

1:30:27Those worries entirely go away. And I think that's quite a nice thing. I mean, I just don't. And not only that, the other thing is just not having to spend any mental energy on like, um you know jude does this toothpaste cost more or less than that other toothpaste etc just just i you know i basically learned just don't think about money unless again unless you're going to go doing some crazy thing like by a sports team it kind of just doesn't matter and you know at least i don't spend the mental energy on that now for me i'm you know it's not like i go around spending crazy amounts of money um so it hasn't changed that much how i live my life.

1:31:06I mean, I have, I have like three pairs of jeans. That's it. It's kind of what I have. Um, yeah, it hasn't changed that much. I mean, I did, you know, right now I'm in New York, I'm in New York and, you know, I did buy a pretty expensive house here, but other than that, I haven't done all that much. I've heard you say something about like to truly understand something is to be able to explain it simply or some philosophy like that. Can you espouse or expound on that a little bit uh for for why you end up from my phd advisor manuel blum i mean it is an amazing thing i think and it's hard to explain unless you've actually felt it um once you actually understand something you can give an amazing clear simple explanation of it um and i used to think that i just didn't know that um and you know i learned that from my advisor and it was as follows it was early on in my phd you know i had to explain a scientific paper to him and he said go read this paper explain it to me because i haven't read it i would come you know i came in for a one-hour meeting and i said like the first statement and he said i don't understand and i'm like how can you understand it's just a simple a simplest statement like i could thank you i just don't get it why did you say that why did you start with that what i don't get it and i spent the whole hour and i couldn't even go after past the first line of the paper um i came back the next week and it was the same thing it took me six months to finally get to a point where he said okay i understand meeting him weekly it turned out he was he was pretending the whole time actually that paper he was a real expert in um but it took me really six months and it is true i mean the explanation that i had at the The end of the six months was just because I really understood it.

1:33:05It was just crystal clear. And I could explain something that at first I was planning on taking like 40 minutes to explain it to him. I was able to explain the idea in like a minute. There's like, no, this is the main idea of the paper, blah. And yeah, but that only happens once you really understand something. And that's, I mean, that's the genius of my PhD advisor. He really understands things well. I think today there's more people learning languages in the U.S. through Duolingo than the education system combined. Do I have that fact right? Correct. There are more people learning languages on Duolingo in the U.S.

1:33:40than there are people learning languages across all U.S. high schools combined. But I heard you say recently that you're not anti the U.S. education system. And in fact, maybe pro of it. Having learned so much about how people learn and built a business around it, at least for languages and now increasingly music and math, if you could wave a magic wand, is there anything from a teaching standpoint or how we go about the educational system in the U.S. that you would change if there was one thing that stuck out? is very different from what we do at Duolingo. But I mean, what matters the most in the educational system are the teachers.

1:34:27This is by far what matters the most. And there are some excellent teachers. But there's also quite a high variance. And, you know, what I would waive the most is, or what I would waive or want to is make sure that every teacher that is out there is amazing. That's pretty hard to do, given a system that doesn't pay teachers very well, given a system where, you know, there's essentially the equivalent of tenure in some of these places where, you know, you basically can work and not do all that much because of teacher unions. So there's a lot of stuff that is working against that. But the thing, you know, you want better education, better teachers.

1:35:07That's, you know, again, very different from Duolingo. Duolingo, we don't use humans, but the truth of the matter is that for the foreseeable future, humans are going to be needed for, you know, of the educational system. Can you tell the story of Bill Gates? So Bill Gates called you once to try to personally recruit you to join Microsoft. Can you give that backstory? I picked it up in something I was researching, and I was like, well, I have to get that story or a little bit more detail into it. Clearly, it didn't work. You never went to go work at Microsoft, right? Yeah. And so when I graduated from my PhD, there's this one thing that happens when you're kind of the winner takes hold kind of thing.

1:35:46I was the kind of most sought after candidate after I graduated from my PhD. Every university wanted me to become a professor there. And usually when you graduate from a PhD in computer science, you go become a professor in computer science at our university. But there's another path, which is you could also join Microsoft Research. And I was the most sought after candidate. You know, every university, every computer science department wanted to hire me. Microsoft Research wanted to hire me, too. and Microsoft Research, I guess, at the time, I don't know, they had a thing where the person they wanted to hire the most that year, they would get Bill Gates to call you and personally pitch you for as long as you wanted to.

1:36:27So I ended up talking to him for like an hour. Mainly not because I had, you know, I love a lot of the stuff that they've done at Microsoft and certainly Microsoft Research, but I wanted to become a professor. So I had no desire, but I wanted to talk to Bill Gates for an hour. So I was happy to do that. Yeah, well, it's good. Listen, if Bill Gates ever calls me, I will gladly take his call and I'll spend an hour talking to him. He's such a smart guy. He's crazy when you enter a room and you're like, he's clearly smarter than me. Yeah, you think that with him. Because I've been in a room multiple times with him for different things.

1:37:01Every time you're like, yeah, that guy's smarter than me. Do you remember what you talked to him about or what questions you asked him? At that time? No, I don't. Too much. But I've talked to him about a number of things. I mean, he's, you know, he's a, he, we've talked about doing one stuff a number of times again. He's such a smart guy. Yeah. Now, uh, last one, I'll let you hop. What's the, what's the story of you setting up a honeypot website to catch people that were cheating on your, uh, on your tests when you were a professor? Yes. I was a professor of computer science. Um, and, um, uh, it wasn't tests.

1:37:35I, you know, we were, I was assigning, um, you know, math problems. It was basically discrete math, but it was math problems. And we had this rule, which was a dumb rule, but it was a rule that I inherited from the guy who taught the class before me. That was, you can't use, you can't Google for solutions. And so we tried every time that we never, because there's only so many really awesome math problems. And, you know, we tried to disguise them a little bit, but, you know, we thought probably people are Googling for solutions. And then it occurred to me, well, I'll teach them not to do that. And so I made up a problem.

1:38:11I specifically made up a problem. And then I made a website for that problem. There's a fully unique problem in a website that had the problem, etc. And I own the website. And what happened is I assigned the problem. And I said, of course, as usual, you can't Google or anything. But if you Googled that problem, you got to my website, and I could track who you were on the website. And so, um, to about 30 % of the class, this was a class with 300 kids. So about a hundred of them fell on the little website and, um, I found them all. And then, um, the next lecture, you know, I didn't say anything.

1:38:51I didn't contact them. The next lecture, I just went in, I stood in front of the class. I projected my screen. I typed, you know, the query to get to that problem. I got to the website and everybody's just like, okay. And then I said, I did a, you know, to, to know who owns a website, you just do who is and the name of the website. And I typed in, in the prompt there, who is the name of that website. And then it said that it was owned by me. And then you see everybody's jaw just drop. And then I'm, then I, then I, then I, you know, I'm using, this is using Linux. I'm showing them, you know, the, the server logs and just everybody's IP address.

1:39:29And I'm like, Ooh, and everybody was just, they were, um, They were peeing their pants. And so. What'd you do with it? Did you, did they confess or what happened? I mean, I knew who they were. So I just gave them a zero, all of them in that assignment. I said, look, I'm not going to ruin your life. I'm just going to give you a zero in that assignment. You can still get an A in the class, but. I was not interested in ruining anybody's life. I just wanted to have them not do that anymore. Yeah. Well, it's a good lesson. Well, thank you for doing this. This is fun. I hope, hopefully you feel better powering through here.

1:40:03Uh, yeah. So thanks for doing this. It's a pleasure. And, uh, yeah, this is great. Thank you. Same. Pleasure is mine. Thank you.