Sal Khan is hopeful that AI won't destroy education

23:55You can't be looking around and doing shady things. And then that video gets peer reviewed by people you don't know to just make sure you're not doing shady things. And then, yes, if you've got 90 % on that unit, we say, you know, unit one of calculus. You can now begin your tutoring journey and there's some training that you go there. And when we were doing that, University of Chicago reached out and said, hey, everything's up in the air with the pandemic. Could we use your certification for college admissions? And we and we said, yes, you can. And then MIT and Columbia and Caltech. And now there's a list on the on the website of 40 plus universities, Yale, Brown.

24:34You can name them. They all said, hey, you could use schoolhouse certifications as a way to prove your mastery. People are like, how are we going to do assessments in the future with AI and everything? And it's like, Salcon's already figured out. Well, yeah, AI is going to add a whole other layer. I mean, I think we're going to be able to do some nice simulation-based assessments and things like that. And a fun example, I just met this young woman from Afghanistan like three weeks ago. And she, when obviously she could not go to school going up in Afghanistan, Khan Academy was her school. Taliban take over Kabul.

25:12Her family become refugees in Pakistan. She's still not in school, but she's been learning all this time. She wants to go to MIT. She applies. MIT is really impressed with her application, but she has no diploma, no formal transcript, no SAT scores, nothing, no AP scores. And they're like, can you go on Schoolhouse and validate yourself? And she did. And she got in based on that. I just met her. She was at this Y Combinator, you know, for the top AI engineers in the country type of thing out here in Silicon Valley. And my wife and I took her out to a little meal. and I was like, yes, this is the certification vision of Schoolhouse.

25:51It's pretty powerful to hear stories like that. You know, most recently, and this is where some of the controversy started coming, I think it's good controversy. We went to the same college admissions folks and we said, look, this certification and they weren't just interested in the certification. They were also interested in the kids who tutored because you could imagine if you certified yourself in calculus and then tutored calculus. Now, this is like, this is like one of the, like, Like, who incentivizes people to actually do the tutoring? Because you're not, isn't like paid. But like I can say as a person who tutored kids when I was a kid, like that's so powerful in just knowing the material better.

26:27If not also putting that on your little kiddo resume, whatever they make them do these days. If you're one of these colleges and, you know, there's a lot of cynicism around people gaming it, et cetera. But as someone who, if young Hank ran 100 sessions tutoring chemistry and he has a 4.8 out of 5 rating and there are these quotes saying, I learned more from Hank than I did in my school or he makes me excited about chemistry. Yeah, if I'm one of these universities, like, I want Hank on my campus. That's the kid I want. But we went to these universities and said, what else could be of value? And they said, well, this is probably too hard, but what if we could have, if we could give students practice in having dialogue about hard subjects?

27:10We don't want them to water down their passions, but we want them to be able to have constructive dialogue because everyone's living in their bubble now, geographic bubbles, socioeconomic bubbles, social media bubbles. And so that's when we launched, only a couple of months ago, this dialogues platform where most people would get anxiety just as I'm about to describe this. We make young people fill out surveys on tough issues, immigration, Israel-Palestine, affirmative action, gun control. I could go down the list. The stuff that we are afraid to talk about at dinner parties. And kids fill out a survey.

27:43We pair them with kids with the opposite viewpoint, and they have a conversation. And then after that conversation, it usually goes about 50 minutes, they fill out a questionnaire on, do you feel that the other person was, you don't have to convince them, but do you feel like the other person heard you? Do you think they can represent your point of view? Can you represent their point of view? Is there anything, any feedback you want to give to that other person? It could be constructive or it can be positive feedback. You do as many sessions as you want. And then on your transcript, it just says how many sessions you participated in.

28:17It doesn't say what your point of view was. And you can highlight anything you choose to highlight. You know, hey, Hank said that Sal was, you know, he really opened your mind to a different viewpoint that you never took seriously before. Whatever it is, I could put that on my transcript. And if I want to, I could share that with many of these same universities. The controversy is there was a New York Times op-ed. I was like, oh, this is just another thing to fake in college admissions. And, you know, Nate Silver, you know, tweeted, this is affirmative action for boring kids. And all of a sudden, I'm like, no, how can it be boring?

28:48How can it be boring? Like, it's very not boring to be willing to have an open conversation about these topics, but being able to do it constructively. And, you know, what we've seen, these kids aren't holding back. They're holding their positions, but they're doing it in respectful ways. All I have to say about this is affirmative action for boring kids is that's a tweet. You know, that's like such a tweet. And I'm so tired of tweets. Like, I mean, I'm a tweeter. Don't get me wrong. I post right here. I post, I post way too much. I want to live that Salkan life. Get off, get off of these postings, but I post, but man, is that a post that's like pure, like I'm going to have the, the take that everybody's going to feel good about, even if it makes no goddamn sense.

29:36Oh man. Uh, well that's, that's fascinating and does make me sweaty. Um, that's, that's not a choice that I think I would have made. That seems it's bold, which yeah. Uh, it's, you just call yourself brave. No, I'm saying the students are brave. I actually haven't, I've observed some of these. Well, it's a little brave on me on my part too, but it is. I think it's brave on your part. Yeah. I'm saying it's the opposite of boring. These students are brave to do that. I haven't been able to do it because yeah, I don't want, if I were to get on dialogues, the kids will say, oh, that's Sal Khan. And did you know that Sal Khan has this view about the Middle East?

30:11Did you know that Sal Khan has this view on immigration? I'm like, nope, not going to be constructive. So I can't participate. That's probably not ideal. Yeah. Yeah. I probably won't be on there either. But I think this is a, I think it's a brave decision for an organization to make, which leads me to a decoder question. How do you make decisions? How does Salcon make decisions? How do, how does Salcon make decisions? I always try to remind myself and the other people who are inputs into decisions, like what we're trying to solve for to begin with. It is amazing how many times in life, you know, there's a friend who's a very successful venture capitalist and I, I've made fun of him in the past, but he has this like hack that always makes him look brilliant.

30:46You go about half an hour into any meeting and he will say, hey, hold on, hold on a second, everyone. Let's take a step back. What are we solving for? And he looks brilliant. And everyone's like, oh yeah, we lost track of what we're solving for. And it's true. You know, 30 minutes in the meeting, most people are getting into the weeds. They're not. And you're like, no, what we're solving for is how do we get more students to participate in society or how do we get, so it's important to have those true norths. And then, yes, I try to listen to people who will have, who are ideally as close to the ground as possible on what they have to say.

31:23And then, you know, try to make a call that both makes sense intellectually, hopefully takes the path of less cynicism and more like, no, maybe the world can work this way. Like, why don't we just try. And, you know, people say gut, sometimes it's like, that's a lazy way of doing things. But, you know, your gut has a ton of neurons. Like our gut, as you know, would be a fairly smart animal if it just wanted... I don't think the gut's actually making the call here. But the gut is really, you know, we have these neural nets, we have these, you know, hundreds of billions of neurons that are doing things subconsciously all the time.

31:59And at some point, they're just giving your conscious mind a memo saying, yeah, no, I think this is the right decision. And that's at the end of the day, if you have a good true north, if your gut has paid off in the past and you're surrounded by people who are advising you for the right reasons and you make a call and, you know, it is also important to have folks around you that are willing to disagree and commit like, hey, once we make this call, let's try our best. Let's not grumble and be passive aggressive after that. So maybe you can walk me through that process a little bit because I want to talk about you, about Khan Academy is starting to use AI tools.

32:29I got a call from you a couple of years ago. I remember pacing around my basement on this phone call where you tell me about you going into AI tools. And I was like, man, I would not make this call right now. I do not know where this is going. I don't know what the public opinion around it is going to be. I know it's going to be complicated and weird. I don't know where it's headed. I don't know how powerful these tools are. And I was like, but more power to them for heading straight headfirst into it. This was very early. This was like right after GPT-4 launched. How did you make that particular decision?

33:06I remember our call. I don't remember exactly when that call was in the whole timeline, but it was actually, we made the decision well before GPT-4 launched. Yeah, yeah, yeah. But this was after. Yeah, so I probably couldn't talk to you. It was probably as soon as we were out of NDA with OpenAI, I probably called you and said, hey, maybe there's something here. But it was summer of 2022. so almost exactly, well, roughly three years ago. And I got a call from Sam Altman and Greg Brockman. They said they're working on their new model. They'd want to show it to us because they thought maybe there's something interesting about Khan Academy showing positive social uses of it.

33:43I was curious. What they showed us was GPT-4. This was well before ChatGPT. This was about six months before ChatGPT existed. And ChatGPT, for those who remember, it wasn't even launched on GPT-4. It was launched on GPT-3.5. it blew my mind the way that you would. And, and honestly, even when they showed it to us, they hadn't fully appreciated, like it had trained and they, they ran a demo with us and they said, do you want access? And I remember it was a Friday and they gave us access myself and our chief academic officer. And I was slacking with Greg Brockman. And I said, Hey, does this work in other languages?

34:15And he wrote, I don't think so. And then I just, I barely speak Bengali. and I tried to speak and it spoke back, not only did it speak back to me in Bengali, it wrote in Bengali, which I can't read. But then I said, can you transliterate that into English text? And I was like, wow, it can speak Bengali. I took a screenshot and I sent it to the OpenAI folks and they're like, yeah, after you asked, we checked. It looks like it can speak every language. So there was like, oh.

34:40See, this is why I would have said no. I'd be like, you guys don't even know what language is it to speak? There's all this emergent. It's fascinating. I have this other friend who's a professor at Stanford and, you know, he's trying to understand. Apparently, like, you could go from one language to another as like a rigid transformation in a multidimensional space. And that's why it can talk about things in Bengali that it never got trained on in Bengali. Anyway, there's a whole, there's some deep linear algebra there. But this was very early on. And, and, you know, this was another thing I didn't realize that we were doing a year after that, after we had launched.

35:11Well, I immediately, like that weekend, I couldn't tell anyone about it. We had signed an NDA, but I was like up all night. I said, write the Declaration of Independence in the tone of Donald Trump. And it did it. George III's a loser. We're going to have the best country. It was actually a pretty good one. George III was a loser. But then when I said, hey, you are an empathetic tutor, I think the exact prompt I said is you are Robin Williams in Dead Poets Society, and you are going to Socratically tutor me not... Because I knew this could be used for cheating, et cetera, too. But it was able to do that.

35:40And even though it made errors and hallucinations, I was like, there's something real here. And so, well, we got most of the Khan Academy team under the non-disclosure agreement so that we could see it. And about a month into that, and this is still months before the rest of the world even knew that this was coming, our team was surfacing all the fears, the cheating, the safety, the privacy, the hallucinations, the math errors. But I was pretty convicted that like, look, this is, yes, those are real fears and errors. Like we need to turn those into features, but this is going to be so powerful and it's getting better so fast that if we don't really lean into this with our mindset and our mission focused mindset, other people are, and they're not going to care about whether it's cheating.

36:26They're not going to care about whether it's good for kids. They're not going to, you know, and honestly, this stuff's going to be so scary for people that they're going to need hopefully someone that they can trust that's built up trust over time. So yeah, that's, that's when we lead in. And then, um, I mean, there's interesting chat GPT to talk about emergent phenomena. The day that that came out, November 30th, 2022, I slacked the OpenAI team. I was like, wait, we're under a non-disclosure agreement. We're not supposed to announce any of this stuff until March. What did you guys just release?

36:53And they said, we didn't release anything. We just put a chat interface on an old model that's been out for like seven months and the world exploded. It's interesting. Some kids in a garage or at a college campus could have done that. And now, you know, have chat GPT. No one did it. No one did it. But so even that was a shock that you just put a chat interface on these existing models and it makes people think about it differently. Wild. How do you actually turn chat GPT or any LLM or GPT into something that is useful for a student? So you have to work in math that works. You have to try to obviate hallucinations.

37:37You have to put walls around it so that it doesn't do things you don't want to do or like be too sort of friendly with students in whatever way it has to be. It has to look and feel like a tool. How do you actually like functionally do that? What do you do to it? Yeah. And look, it's evolving because the models are changing and just people's expectations around these things are changing so fast. But some of our original thoughts I think are still true where there's some just basic prompting you can do. And I say basic prompting, but when you want something that can robustly work for millions of folks, the prompting has to be quite careful and you have to create all sorts of systems to test, et cetera.

38:14But to say, look, this is going to be Socratic. Use these techniques. You're not going to give answers, but nudge students forward. There's a lot more to it, but that's the gist of it. You're going to want to have some level of transparency and oversight, especially if you're talking about under 18 students. And transparency buys you a lot. It doesn't just buy you safety. If a student's saying, hey, I want to learn how to build a bomb, I want to hurt myself. Not only having teachers or parents being able to see it, but then having another agent that can see that and actively flag it is important.

38:49You know, we avoid hallucinations from the get-go. The models themselves have gotten a lot better over the last two, three years. But you anchor it on content that you know is good. And we have a lot of content on Conicom. Do you have like a big like context window that has a bunch of true facts in it or something? No, we don't. Well, there's two ways that you can do it. One is if you're using our tutor, which we're calling Conmigo, on Khan Academy content, on a Khan Academy article, Khan Academy video, it has that context in it. or if you're getting help on an exercise, it knows like we've passed it the solution to the exercise.

39:23So it won't hallucinate or hopefully make math errors or very infrequently make math errors in those cases. So that's one, anchor it on things it knows. You can also, you know, these called vector databases, you could put a lot of content in these databases where based on the conversation that you're having with the AI, it can find which pieces of content are closest to this conversation and then throw those things into the context window. So you don't have to throw everything in the context window. Google now does have some models that you can have like a million tokens, roughly a million characters in the context window.

40:00So you can actually throw a lot. Your average book is like 40 ,000 words. So if, well, actually it's not even characters. A token is like a, call it two thirds of a word. So you can now throw a lot into context windows too. But that also has cost issues, et cetera. But the other guardrails, Yeah, I mean, the transparency, there's a lot that we do around the math to just double, triple check the math, just hacks on. The AI calls another AI to go through the reasoning that can come up with every possible way that the student might approach it, then compare the student's answer to that. So there's things that we've been playing with that have improved the AI's ability to do there.

40:36So if I'm using Conmigo and I get the question wrong, it might have already predicted that wrong answer and know that I got it wrong, like how I got it wrong. And what you've touched on is actually one of the hardest things. The AI models are actually pretty good at math now. They weren't good at math two and a half years ago, but they are now. Right. But now they're like, they call math out. That's actually what they do. They're making like Python calls behind the scenes to actually do the computation. And the places where you still see the most errors is when the AI is evaluating the student.

41:07So let's say the correct answer is one third and the student has put in 0.33. That's not quite right, right? They should be 0.3 repeating. A good tutor would say, OK, are you sure that's the full answer? Are you sure it's just 0.33? How would you express that as a fraction? Something like that. And so, yes, now what we have going on behind the scenes is the AI, even before looking at the student's response, is saying, like, what are reasonable responses here? and then compare that, or what are reasonable things a student might have done? Or when the student gets an answer, what are ways that the student might have gotten that answer?

41:38And then compare it. And this is constantly evolving. As the models get better, we're having to do less of that. But there is still a lot of that going on behind the scenes. We need to take another quick break. We'll be right back.

41:58Support for the show comes from Rippling. If you're a business owner, here's the truth. SaaS promised to make work easier. But now the average company is buried by hundreds of apps that silo your teams, slow you down, and simply don't work together. That's not SaaS. That's SAD. Software as a disservice. That's why you need Rippling. Rippling is the unified platform for global HR, payroll, IT, and finance. They've helped millions replace their mess of cobbled-together tools with one system designed to give leaders clarity, speed, and control. By uniting your employees, teams, and departments in one system, Rippling removes the bottlenecks, busywork, and silos your software created.

42:41Automated, perfectly in sync, and seriously simple to use, Rippling gives your company one source of truth for your people, their data, and everything they touch. With Rippling, you can run your entire HR, IT, and finance operations as one. or pick and choose the products that best fill the gaps in your software stack. And right now you can get six months free when you go to rippling.com slash decoder. Learn more at r-i-p-p-l-i-n-g dot com slash decoder. That's rippling.com slash decoder for six months free. Terms and conditions apply.

43:24We're back with Khan Academy CEO, Sal Khan. Right before the break, Sal and I were discussing Khan Migo, Khan Academy's AI-powered teaching assistant, and how he's thinking about the right ways to use AI in education. But now I wanted to focus on some of the more fuzzy aspects of learning, like how what a teacher provides is not just a textbook and instructions, but also a motivational relationship and whether AI can even remotely replicate that. One thing I've noticed, so like people always say like, hey, you've helped me, like you're the reason I got a five on my NP exam or whatever. And, you know, I'm always like, I think probably you had something to do with it is what I say to them.

44:07But what I have learned is that there's a big component of teaching that isn't teaching. It's motivational. It's like the coach in the room who you have an obligation to and who will be disappointed if you don't do the thing that they ask you to do. It's a human being, and it's easier to feel obligation to human beings. Do you think that LLMs can play a motivational role? Is that part of this? Yeah, I think over time. And they're already playing a little – we've even implemented a little bit of that. Even for myself, I had to give some commencement addresses, and I took my first draft, and I got an AI to give me feedback, and it made me feel good.

44:49Like, oh, yeah. I'm more confident now. It sure does do that. It does, yes. If nothing else, it will make you feel good about your writing. I could have put a pretty trashy, a pretty bad speech in it. Phil probably would have given me positive feedback. But what you're touching on is absolutely right. Even, you know, a lot of folks know Khan Academy started with me tutoring family members 20 years ago. And when I really think about it, yes, I was explaining certain concepts to them. But a lot of what I was doing is exactly what you're describing. I'd be like, hey, where are you? How come you didn't do the thing I told you?

45:19Hey, look, you got to be a little more confident with how you answer these questions. Let's lean into the problem, whatever it might be. And so our realization, you know, the first version of Khan Mego, it just kind of sat there and waited to be asked. And then it would help you as you needed it. And I say that's analogous to a tutor walking into a classroom and say, hey, kids, I'm here in the back. If you need me, come get me. But I'm here. I'm going to be reading a novel, whatever. That's not good enough. You want something that holds them accountable, et cetera. So this new version we're launching, we're piloting it in the fall, is the AI is front and center.

45:50You come. It'll say, welcome back, Hank. Hey, it's been a couple of days. We're falling a little bit behind our goals. Are you ready to get started on this next task? This is what your teacher wants you to work at. And once you're done with that, I have some ideas for you to work on. And then once you go into it, it's constantly like, hey, look, you got that wrong. Not a big deal. I think it's a good idea for your review why you got that wrong. And there might be some game mechanics. If you review it, I'll give you some points and things like that. But that's the future. And yes, if we fast forward a few years, we can all imagine having AI.

46:21A real human being is always going to be better, but a real human being is not always available. That can be even a better accountability holder for us. You use a phrase for this. It's not the best tool, but the best available tool, something like that. Did I make up that's a you phrase? That's not a me phrase. I mean, I've said stuff like if I had to pick between amazing teacher and amazing technology, I'd pick amazing teacher. Hopefully you can have both. Yeah, I mean, I've also heard you say that replacing humans in education would be a disaster. And I think that a lot of teachers and parents and students would agree with you.

46:59How do you think we avoid that fate? People are complicated and they're expensive. And so it feels like lots of people would like for all the work to be done by things that are simpler and don't complain. How do we avoid this fate? One, I always like to point out to people like the economics of our education system. Most people would argue that the teacher role has the most direct impact on the student of everything that you're spending money on. But if you look at a lot of places, California spends around$25 ,000 per student per year with like 25, 30 student teacher ratios. New York City spends a lot of East Coast school districts spend 30, 35 ,000.

47:37And so depending on how you account for it, there's about six, seven, eight hundred thousand dollars per class of 25 or 30 kids. They're not paying the teacher that the fully loaded cost of a teacher with benefits, even a senior teacher pension and everything. You know, maybe two hundred thousand if you put all of that in there. And it's usually a lot less. So a lot of the costs of education are going into layers of other stuff. You know, some of that stuff is needed. Some things. Yeah, buildings as part of it. You know, yeah. If you had to cost cut, that's not the way to do it. Now, I think AI might help in some of the back office stuff.

48:12You might be able to automate the registrar's office. You might be able to automate, you know, other functions in the office of a school, and that might save money. But we've always said, even before AI, that our goal is to raise the ceiling for hopefully you already have access to a reasonably good classroom and a great teacher. But it's still hard for that teacher to personalize for 25, 30, 35 kids. We'll give them the tools to do it. But we also want to raise the floor. And, you know, I talk about this young woman in Afghanistan. She did not have access to a teacher, but she was unusually motivated, and it would have been sad if she didn't have something.

48:46And so that's where we did raise the floor for her. We, you know, we hear stories of there's a young girl in a Mongolian orphanage who used Khan Academy. There's kids in rural America who don't have a physics or calculus class within hundreds of miles of where they live. of kids in inner cities who don't have a calculus or physics or chemistry class at their school or an advanced one. And so that's where we, where we raised the floor. But the ideal, you know, I always point out about what, about 150, 200 years ago when textbooks started to become a thing, a lot of teachers were afraid that there was going to replace them because teachers thought that, Oh, I'm, I'm the source of the knowledge.

49:19Why would anyone come to me if they can read that? There's a whole book here. There's a whole book here with exercises and everything. It's all, but now teachers can't imagine teaching without a textbook because Because they're like, of course they need me still, but they need their practice. It's good to have another resource. And I think that's going to be the same thing with AI, where a teacher who's leaning into the tools, you know, a teacher who refused to use a textbook 50 years ago is going to have trouble. And a teacher in 50 years who's refusing to use AI might not be able to be all they can be.

49:48But if they're using it thoughtfully and the AI is helping them form better human connections with the students, make more engaging, interactive lessons, more personalized lessons, support certain students while they can support other students and they can tag team, they can act as their teaching assistant. Yeah, it's going to become invaluable for them. So I've heard you say that the incentives of this sector, this education sector, which is the second biggest industry in America, are strange. I think you used the word strange. And then the person you were talking to did not ask a deeper, further question.

50:20And I was like, oh, my God, do I want to know all the ways that Sal Khan thinks the incentive structure of education are strange? Tell me, tell me, what are some of the strange incentives? Yeah, and look, this goes back to the nonprofit question. You know, I don't think you need to make everything in the world a nonprofit. There's certain things that the private sector does very well. I'm, in my heart, a capitalist. I believe the capitalist system generally works. And there's certain things you want government to do where the private sector wouldn't naturally do it or wouldn't have the ability to coordinate to do it.

50:52There's certain areas where markets aren't working. They aren't doing what's efficient or what's not aligned with our values. And maybe for political reasons or just government is too slow or bureaucratic to take advantage. And that's where the nonprofit sector matters. And education and I would say another major sector, healthcare, are two of the areas where the beneficiary, in the case of education, it's the student, is the beneficiary. The decision maker in education, it's typically the decision maker is usually like the school district. Sometimes it's the teacher. And the payer, and the payer is in education usually the taxpayer, they're three different entities.

51:29They're three different groups. And you actually see a very similar thing in healthcare. And even if it was rational, we still also have this value that, you know, just as if someone is bleeding and they're dropped at the emergency room, you don't want to say, hey, let me see your insurance. You want to treat them. Unfortunately, I think that might have happened sometimes. But, you know, you want to treat them first. Similarly, if a young person in our society wants to learn, I think most of us feel that it should not be based on how much money their family has, whether or not they should get a high-quality education.

51:58So that value system and the fact that these three agents all have different incentives, I think is what has led to education not always having the most engaging, the most effective. You know, a lot of the people in the district office, very well-meaning people, but there are certain guidelines, regulations they have to follow. Some salesperson from a big publisher comes and tells them a good story. They adopt it. The kids hate it, but too bad. You know, before we built our district offering, I couldn't tell you how many times we would talk to like a chief academic officer or superintendent of a district.

52:32And I would say, hey, why don't you all use Khan Academy, look at our efficacy studies? And they're like, oh, no, no, we believe you. My daughter swears by Khan Academy. It got my nephew through calculus. It even got me through statistics in grad school. But, you know, we have this vendor we adopted last year. and, you know, they're on some state list that you're not on. And, you know, we already wrote them a$5 million check. So I think we have to use them. I'm like, there's no sense. I'm like, would you use those with your own child? Would you use them yourself? And they're like, no, not really.

53:10Oh my God. Yeah. That is what I've identified. And my solution to that problem was simply to not engage with it. But what we have both done, I think, and to our credit, is that our first customer was the student, and that's how we got into these places. Like, we didn't get in from the top down. We got in from the bottom up. And I've seen several organizations, like for-profit companies, too, to get into this business that way. And that's the way to do it if you actually want to help, whereas if your customer is the superintendent, then you're going to have a pretty different set of structures.

53:44Can I ask you a question that is a real pink green question here? So once upon a time, I came across this like large database of videos that had been scraped to use in sort of a database that anybody could have access to for training AI. and the only YouTube channel that I found that had more videos in it than the ones that I, that my company complexly makes was Khan Academy, which indicated to me that these, that there wasn't just like grab every video. It was grab videos that have good information that we can trust and that that is more valuable. Does that make sense to you that that is why we got grabbed more than the average video?

54:26I think so. Yeah. I mean, I don't know about the database and all that. But yeah, I'd like to, I'd like to believe, I mean, you know, even from the, from the, you know, I've, even in the early stages of Khan Academy, even as a nonprofit, you know, there was definitely a bias, especially in Silicon Valley to be more of a platform than to be like focused on, let's say the artisanal quality of whatever you're trying to create. And I would always point out, it's like, you don't need a thousand exploitations of L 'Hopital's rule. You know, one good one might go a long way, but maybe four or five max.

54:58you know, with different takes on it. And it's actually sometimes been hard for me to make people believe that because it goes against like, well, that won't scale. I was like, no, it kind of does scale because L 'Hopital's rule isn't going away. And it's not like we're creating, you know, some new site or something where you have to scale. Yeah, it scales the same way a textbook does, you know, you just have to, except a lot easier because making another copy is basically instantaneous. You know, and then we have interchangeable parts. If L 'Hopital's rule does get updated, We can just update that part.

55:26So given that, and my question is like, how do you feel as a creator of content about LLM's training on the stuff that we make? Do you think that that's just sort of they can learn the way anybody else can learn? Or is there like a difference in value offered by different content that should be compensated differently? You know, if I thought Khan Academy could get a meaningful check for this, I would love to take it. But generally speaking, in a world where let's say I don't have a choice. I mean, maybe I do. But let's say there's a world where I would rather our content is used for training. If a model in two or three years can create a stock tile video and, you know, draw diagrams that are helping people and, you know, that's going to be net good for society, I think.

56:18I'm generally supportive of it. on the intellectual debate around, you know, can we, should we just view an AI as just a super smart savant that like sees a lot of material and now can paint in someone's style or write in someone's style or create a video in someone's style? Or should we view it as like, no, it's, it's IP theft. You know, the courts are going to decide this, but I've been kind of a little bit more on the savant side because as you, as we know, our videos aren't directly encoded anywhere in these models. These models are just learning. They literally are learning to create and create associations of something similar.

56:56And, you know, there have been people who could paint in the style of Leonardo da Vinci or whatever. This is a similar thing, although, you know, on steroids. But what about you? Are you like, I don't know? I'm like, I don't know. What I'm like is I'd like these things to make their way through the court so that we can all know, so that we can know what the legal situation is, regardless of kind of how I feel. because there is sort of a law, intellectual property question here. You talked, I want to kind of finish off with this. So you talked at the beginning or somewhere in the middle here about content creation for artificial intelligence.

57:32Of course, already these things are creating content. Like what an LLM exports is content. It could, maybe it's just for you, but it's still stuff. But are you thinking about, like, will there someday be an ability for me to like say my, you know, I, I need to know about Lagrangians and, uh, and Sal Khan just sort of like pops up and starts tutoring me on Lagrangians with a piece of content that had not previously existed. And like, you're doing the thing and the, the chalk, the digital chalkboard is happening and I'm hearing your voice. Is that like a, is that a future? Are you going to never die is kind of what I'm asking.

58:09That's my, you know, we'll see. Uh, but the Mr. Mr. Selden. Well, that's it. No, don't get me started. I mean, you know, when you hear someone like Demis Hassabis talk about, you know, all diseases will be cured in the next 10 years, it gives those of us who are about to be 50 a little bit of hope. I think it's only prudent that I and other content creators and Khan Academy prepare for that reality. It's likely, you know, as best as we can guess, it's likely to happen. It's probably going to happen, I would guess, of really robust quality. It'll probably be three to five years when something like that will happen.

58:46You know, my vanity likes to think it'll take longer. It might. We'll see. It might be next month. Same. But, you know, and then, you know, our content can still be in some vault someplace. It's like, this is what people did before AI took over. This was the source. It'll hopefully still have some almost like primary document value to it. Like, oh, you know, Hank is like a real person. And he talked about having an upset stomach that day. And, you know, AI won't do that kind of thing. Who knows? I think we're going to give the AI some pretty upset stomachs. They won't have stomachs, but something inside of them will feel.

59:21It'll talk about its stomach. It'll talk about what it just ate or what Hank just ate. But yeah, it's coming. And I think we have to prepare for it. And we are building tools like the new version of Conmigo and the teacher tools. We are having it so teachers can co-create content with the AI, including practice, and administer it through the Khan Academy platform like Khan Academy content and then get insights back. So we're trying to prepare for that reality. This is wild. This is wild. I don't know. My take on like what the way that on what I make and the way I make it is I think that it would be I think it would be hard.

59:54And I also think that people like connect to people. I think that they're like like if people don't know that it's not a real person, there will be a betrayal when they find out that it isn't. And if they do know that it's not a real person, they won't feel the same way about it. That's not to say it couldn't be a useful tool, of course. But I think that there's going to be one of the hardest things to make into AI will be like a pop idol, you know? People want like a real thing to, not that that's us, but there's an element of it. Certain niche audiences we can. Yeah. Every once in a while, somebody will be like, I just really like a smart man who teaches me things.

1:00:36All right. Weird way to end it, Sal. I remain very impressed by you, and I'm glad that you got to spend a little time chatting with you today about all of the crazy, crazy ways that you are having a big impact on the world. So thank you so much for taking the time. No, thanks for having me, Hank.

1:00:55I'd like to thank Saul Kahn for taking the time to speak with me, and thank you for tuning in. I hope you enjoyed it. If you'd like to let us know what you thought about the show or what else you'd like us to cover, drop us a line. You can email us at Decoder at The Verge. The team really does read every email. Or you can find me on Blue Sky, Threads, or, of course, YouTube. Decoder also has a TikTok and an Instagram. Check those out at atdecoderpod. They're a lot of fun. If you like Decoder, please share it with your friends and subscribe wherever you get your podcasts. Decoder is a production of The Verge and is part of the Vox Media Podcast Network.

1:01:27The show is produced by Kate Cox and Nick Statt and is edited by Ursa Wright. And the Decoder music is by Breakmaster Cylinder. See you next time.

1:02:06To...

1:02:15Now try! The different Tic Tacs from mint and fresh to fruit and sweet. Tic Tac. Refresh your good vibes.