The AI Tutor For Every Child and the Next Frontier of Education, From Khan Academy’s Creator Sal Khan

22:02And it gets bootlegged and it gets in the hands of 200 ,000 orphan girls who live in barges. And then they essentially just take over. And so I've always used Young Ladies Illustrated Primer to our team at Khan Academy. It's like, this is what we're hoping to build in the long run. And I never thought we were going to fully build it. I thought we were going to be able to approximate it. But already, Khan Migo can do some things that are maybe even beyond what the Young Ladies Illustrated Primer did, where you can talk to Don Quixote in any language that you want. You can get into debates with it, et cetera, et cetera.

22:41And I think in the next three to five years, almost everything that Neil Stevenson imagined, I think he wrote the book in 1994, I think is actually going to be a reality, which I didn't think was going to happen in my lifetime. How does that impact the structure of the actual school? So say that you think ahead five or 10 years and you're in the context of an advanced K through 12, maybe it's con labs, maybe it's something else. What happens or what changes or how do you interact with the tutor versus the classroom environment. And I know it's hard to predict the future, but I'm sure you've thought a lot about this stuff.

23:10I think big picture at a place like Khan Lab School, where, you know, even before generative AI, we've been, and at Khan World School, actually, we just documented, we were almost embarrassed to say the results. Steve Levitt of Freakonomics fame, he actually looked at it himself because he was like, is this real? And he's like, oh, it's real. I mean, these kids are learning about three to four times faster, not 3 % faster or 30 % faster, three to four times faster. And at Khan Lab School, we've consistently seen at least 1.5 to two grade levels in a year. And this is before generative AI. So I think in some of these settings, we haven't had the same issues that in the traditional school system, kids are getting about 0.7 grade levels per year.

23:55Now, I think if you go to more fluent areas, areas where parents are college educated and they can afford more supports for their kids. You're probably seeing at least a grade level or maybe closer to what we're seeing at ConLab School or like 1.5, two grade levels a year. But I think in environments where basic fluency has been less of an issue, you're now going to free up more time for teachers. So they're going to spend less time doing things like grading and lesson. Most teachers spend about half their time doing that kind of stuff. So that gives them more energy for themselves, but also more energy to just be with the students, which I think is really powerful.

24:30Students are going to feel a lot more supported. And I think if you fast forward three to five years, kids are going to feel very supported in that kind of that basic fluency and that core academic. But I think it's going to be like the magic school bus. Like you're going to be able to put on your like Apple visors or whatever that you're going to be able to like go on with the teacher and like, you know, jump into the circulatory system or time machine yourself to ancient Greece and have a debate with Socrates. and you're going to be like, I think we're going to have, and you're going to have, you know, one of the things we do at Con Lab School is just try to create more time and space for students' passions.

25:06And so you're going to have a world where a middle school student is going to be able to create like studio quality techno music or, you know, a Lord of the Rings quality, you know, epic, epic science fiction movie or fantasy movie. So I actually think it's going to be very exciting. And I think if you go into, let's call it more mainstream classrooms, I think we're going to go a lot further towards solving the, just the core fluency issue and not just in math, but in writing as well, because that's the other thing. The main problem with writing, it's very resource intensive. It's hard to get feedback.

25:36It's very expensive for the teacher to give feedback for 40 kids. It's mind numbing, frankly, kids have to wait a week to get that feedback. Then they may or may not get a chance to iterate on it. And so most people aren't getting much practice. And that's why you're seeing a big problem with writing on top of the problem with math. So I feel confident. I mean, already this year in Newark, New Jersey, which, you know, is not a special case school district. The North Ward had just started using Khan Academy pretty intensively in November. This is pre-generative AI. As of June, 70 % of those 6 ,000 kids use Khan Academy at a level that's associated with pretty profound efficacy gains.

26:14We're still waiting on those test scores, but I think we're going to show for the first time in like U.S. history or recent history, a pretty large acceleration of a large urban school district in math. And that's pre-generative AI. So I'm feeling actually very optimistic about that. And then, yes, especially for those students whose families might not get an opportunity to, you know, put them in AI summer camps like we do with our kids or, you know, take trips or, you know, I think AI is going to bring the world to them in really powerful ways. They're going to be able to practice their debating skills.

26:46They're going to be able to connect things to other ideas. They're going to be able to create pretty profound things. They're going to be able to also go on that magic school bus. And it's not going to be something that requires a ton of resources. So I'm overall hopeful. You know, there's always forces of friction and cynicism and et cetera, et cetera. But because teachers have a lot to gain here, it's going to save them a lot of time and energy. I think you're going to see a faster uptake of this than you see of other solutions. There's the old saying that the future is here is just not equally distributed.

27:16and with things where you have sort of strong structure in place, like the education system, is this a five-year transformation, a 20-year transformation? How long will it take for schools to adopt this despite all the incentives to do so? Yeah, I've learned not to be too aggressive in my predictions, but I think it will be safe to say that in five years, almost all teachers will be using generative AI for lesson planning, grading, progress reports, rubric creation. I think that might happen by late next year, honestly. So that's going to happen fast. So I think in a world where almost all teachers are already using the tool, I think having a generative AI tutor supporting the student, I think that's for sure going to be mainstream in three to five years.

28:05I think because ChatGPT introduced the emergency around cheating. Like, it's not like it's a nice to have, it's like a problem. It's like broke the education system. People need a solution. And so I think that has to be fixed. And I think it's actually going to be fixed with generative AI as well. That has to come into play in the next year or two. So, you know, five years in the future, you might, when you just visit a classroom, you might not superficially see a lot of differences. But I think when you start double clicking on it and you realize that, wow, teachers have a lot more time now for student-facing things.

28:38A lot fewer students are stuck. There's a lot more space for differentiation now. I think that will be transformative, and I think you're going to see it in student engagement, motivation, and in things like test scores. So in this novel, The Diamond Age, one of the things that happens is because these tutors are so powerful and they have memory, right? Some of the things you're working on and they're engaging because they have narratives and the dragons that's teaching you calculus and physics with your environment. There's a lot of engagement with that and much less interest in some cases in sort of like group schooling settings.

29:13Like, can you talk a little bit about how you think about that? Because you also started Schoolhouse, Schoolhouse.world, with our mutual friend Shashir, founder of CODA. And, you know, this is like small groups online and believing that there's still the importance of this. Can you talk about sort of how those pieces fit together and what the motivation was there? Yeah, absolutely. And just a little bit more detail on Schoolhouse, we started during the pandemic. That was a moment where Khan Academy's usage went up 3x. And it was clear that it's great if people are using Khan Academy, but everyone was isolated and they needed more human support as well.

29:48And I'm always thinking about, OK, how can you give something at scale that used to feel really expensive? And tutoring, live human tutoring, for the most part, is expensive. But I was like, well, what if you could leverage volunteerism? And so it was a bit of a, you know, optimistic idea that there would probably be a lot of people who want to do, you know, what I did with Nadia, that a lot of people would actually love to do that. And so we set up Schoolhouse.world as a way, you know, it's another nonprofit, a sister nonprofit to Khan Academy. Its mission is to connect the world through learning.

30:17And the whole idea is, is that you get free tutoring. And who gives the tutoring? It's volunteers. And a lot of those volunteers could be near peers. They could be high school students, college students, but some of them also are teachers who teach at a fancy private school and want to give back. They might be a retired professor, et cetera, et cetera. It's gone well. I mean, it's definitely at a smaller scale than Khan Academy. Schoolhouse is on the order of about 10 ,000 folks per month, but it's doing some pretty powerful things. Already a large fraction of the kids who take any given SAT are already using Schoolhouse to get live human tutoring.

30:51You know, Khan Academy has been the official practice for kind of the asynchronous tutoring aspect of the SAT or the individual practice. But now you can join cohorts and become part of that. And, you know, I think the power of Schoolhouse is as powerful as it is to get help academically. I think that social connection, especially social connection across borders, et cetera, et cetera, hugely powerful. Same thing with Khan Lab School, Khan World School. We think it's all about the human to human connection. Khan World School, which is an online high school, but its anchor is a Socratic seminar where kids are getting together almost every day and they're debating things.

31:28Now, it's not five hours on Zoom. We think that's wrong. That's mind numbing. That was what was wrong with pandemic schooling. Like that's literally going to ruin kids eyes and like ruin them. But, you know, an hour, two hours. But you also don't want completely asynchronous. You want some interaction. And at Khan Lab School, even though people think it's Khan Academy, it's downstairs from that. Of course, yes, we do use Khan Academy. But if anyone visits, you're going to see far more collaboration and interaction amongst these students and far more talking amongst the students than you will see pretty much at any other school.

31:59And I've become a believer more and more and more. Yes, it is important to improve test scores, et cetera, et cetera. And we are doing that. But that social, emotional, that character development, that ability to collaborate with other folks, that's arguably the more important thing. And I almost didn't do it by design out the gate, but one of the things I'm proudest about at all of these projects we're doing is that element, is that the collaborative element, the connected element. You know, whenever we bring a new faculty to ConLab School, they're always like, it's weird how collaborative these kids are.

32:31Even when you get into high school, even in Silicon Valley in high school, where a lot of other schools get quite competitive, secretly quite competitive, but everyone knows it. That's not happening at KLS. The kids are genuinely collaborative. I think part of that is more time and space to pursue your passions. I think a lot of part of it is mastery learning. It isn't about, you know, elbowing out your peer. It's about, hey, everyone here should learn it. And I think a lot of it's about the peer support, which we're trying to scale at Schoolhouse, but has always been part of ConLab School. How did the relationship with OpenAI and GPT-4 start and the origins of Conmigo?

33:04Was that something you came up with and you went to OpenAI? Did they come to you? Like, I'm a little bit curious about the origins of both the product and the relationship. Yeah, you know, this story is going to be just another one of the data points behind the benevolent alien theory. It was about a year ago, end of summer of 2022. I got an email from Greg Brockman and Sam Altman, who I was familiar with, I didn't know super well, saying, hey, we're working on our newest model. We'd love to talk to you about it. And I was skeptical whether it would have any real strategic bearing on what we do at Khan Academy.

33:41But just as someone who's followed the industry and has always been fascinated by this, you know, I knew I had been following GPT-2, GPT-3, DALI. So I was like, oh, are they talking about GPT-4? That's kind of cool if I get to be on the early, get to see what's happening there. So we had that meeting and they hadn't even finished the first training run of GPT-4. But they said, you know, we think it's going to be done in about two weeks. And we think this is going to be the model that really wakes up people to the power of generative AI. We want two reasons why they were talking to us. The first was we want to launch with some social positive use cases.

34:18To us, the most obvious ones are education and health care. And y 'all are the first call we're making. Like, and we want to do it with organizations that people trust and are going to like do it in a thoughtful way. And so as far as I know, we were literally the first people they called. And I said, well, you know, I'm up for looking at it. And the second reason why they want to chat with us is it turns out that Bill Gates, when he saw GPT-3, he's like, oh, this is cool. But we all know GPT-3 really didn't have a good handle on knowledge. And he told the OpenAI team, he's like, I'll be impressed if this could pass the AP biology exam.

34:53He literally said that. And so I think part of the reason they also reached out to us is like, y 'all have a lot of AP biology items. Could we test this? And we're like, yeah, sure. I was a little bit of like, what's in it for us? But like, yeah, sure. That's kind of, I mean, I knew what was in it for me. I just wanted to see what this was. So about two or three weeks later, they said, oh, you know, can we give you a demo? I was like, yeah, sure. So it was me, my chief learning officer, we get in and they had a AP bio question and they said, Sal, what's the answer to this? And I'm like, okay, it was about osmosis.

35:22I'm like, it's C. And they said, all right, let's see what it said, you know, it said it's C. I'm like, oh, that's, that's maybe it's like, I was like, ask it to explain why it's C. And I know everyone's used to chat GPT and even GPT-4 now, but this was before anyone had seen any of this stuff. And I asked it to explain and it explained, it explained it quite well. And then I said, explain why the other choices aren't right. Explain that quite well. I said, and that's when I started to get the goosebumps. And I said, create five more questions like this one. It did it. And that's when I'm like, okay, this, this is a game changer.

35:54And then they're like, would you like access? I'm like, yep, please give us access. So that weekend, I literally couldn't sleep. And myself, our chief technology, our chief learning officer, we were literally slacking each other at like four in the morning with like things that we had gotten it to do that like seemed like science fiction. The most notable was taking on personas, like acting as a tutor and being able to engage in a Socratic dialogue. And we also were discovering some math errors and stuff like that. We were surfacing that to open it. It turns out that they did have some mistakes in their training data, which we, you know, so that was our service to humanity above and beyond the core kind of work.

36:28They realized like, oh yeah, there's some math. But then we started to say like, look, if we can, if we can figure out ways to put right guardrails around this, make sure there's safety, security, we can mitigate the math errors. We can mitigate the hallucinations. This is a game changer. So that's when we started working. ChatGPT comes out a few months later. I Slack Greg Brockman. I'm like, well, Hey, I thought we were under ND. Like, what are y 'all launching? and like, no, this is just on GPT 3.5, but everyone's, you know, seems to be taken by this, even though what we're working on for was so much better.

36:57I was bummed at first, but I was like, oh, in hindsight, it was a good thing because it threw out this very imperfect thing that was not designed for education. The education world took it the hardest because kids are going to cheat now, et cetera, et cetera. And then when we launched with GPT-4 on March 15th of 2023, it almost allowed us to directly address all of the concerns that people were already having around chat GPT. And so that's, yeah, that's how we got to where we got to. That's really cool. How did you end up thinking about safety? You mentioned, for example, you wanted to ensure there was safety around the tool and probably certain answers or certain questions couldn't necessarily be asked.

37:31Like, how did you end up approaching that? So step one, we, as soon as we saw it, we're like, oh, this could be used to cheat. And so like, okay, how do you help without cheating? And so that's when we said we started putting guardrails in the system prompts, so to speak, where it says, don't give the answer. Make sure the student does the bulk of the work. It's okay to help them, but primarily ask leading questions. You know, we have a lot. We've tested a lot of prompts to just get to that sweet spot, and we're constantly tweaking it. So that's the first guardrail. Second guardrail was, you know, this is a new environment, and we don't know where kids are going to take this.

38:09So let's make it transparent to other stakeholders what's going on. So we said, OK, we're going to record every interaction with the AI and make it accessible to parents and teachers if the student is under 18. And, you know, we can debate that for, you know, there's good arguments why older students, there might be some constructive stuff that they could use the AI for that's probably not good to share with parents. But anyway, that's it. We took the most conservative stance out the gate, especially for under 18 there. And then the next one is, you know, we have a second AI that's moderating the conversations.

38:39And if it looks like it's going into an unhealthy place, not only will it kind of shut down that threat of the conversation, but it will actively notify parents and teachers. And so that's those are and then we are trying this best to do digital literacy as well to students to recognize, like, look, here's what you could use generative AI for. Here's where you should be skeptical of generative AI, et cetera. And we are feeling increasingly confident that that combination, and also none of the interactions, and this is to credit to OpenAI, none of the interactions are being used to train the AI.

39:13And arguably, there could be benefit in the future for that to happen. But because this is all new frontier and it's all very sensitive, we and OpenAI just said, yeah, let's just keep this as kosher as possible. Let's just make this as safe as possible. If you think about, you know, going beyond Khan Academy, you've also spent 15 years generally thinking about education. Do you have a prediction for what happens to like university with all of these technologies? Yeah, I think I don't think generative AI dramatically changes what was already going to happen to universities. I think it's not news to anyone that the return on investment for for university, especially if you have to take on debt, is is is mixed depending on on who you are and what you do with it.

39:56obviously we have a student debt crisis. And even if we were able to forgive some, it's not going to solve it for the next generation of students. And so universities are bloated with cost. They're not particularly good at either informing or preparing students. In certain cases, they are. In other cases, they're not, of preparing students for what they need. I think you're going to have a world where the very elite universities, people join those. Yes, it's almost like you're joining a cast, you're joining a club. So I think they're going to be fine. I think whether or not they can justify their ROI, they're going to be fine.

40:34I think now there's some interesting things that are happening around legacy admissions, et cetera, et cetera. So that'll be interesting. I think the community colleges at the other end of the spectrum, they're very flexible and they are very good at thinking about what are the needs? Like, okay, are there industry certifications that could be useful? Okay, we can support students for that. Are there other types of, you know, mini badges that could help someone get a job? We can do that. So I think the community colleges that are low cost and nimble and are able to adapt to the changing environment, I actually think they're going to do fine too.

41:08I think that vast middle of universities, especially the ones that are charging, you know, 60,$70 ,000 when you include room and board per year. And a lot of the kids are graduating with debt. That is very hard to even cancel with bankruptcy. and they're not getting the pay to be able to, you know, I tell everyone, you know, all this student debt stuff, they should be holding the universities accountable. Like, where did that money go? Like, who got that money, right? Someone lent it to the student and then that went someplace. It went to the university. They took the money. Thank you very much.

41:39And now they're just kind of hiding when you have trillion dollars or$2 trillion of debt and it's ruining these kids' lives. I think if you just said, you know what, the universities are accountable for 10 % of that. Their tune will change overnight. Overnight, they will start informing students much more about the career outlook. If you were to like, yeah, not everyone's going to become a museum curator. Let me just tell you that right now. By the way, the kid who you heard about who just became a museum curator majoring in this major, it turns out that their grandmother built the museum. It's a hard job to get.

42:15So don't think that this is going to lead there. Just be informed. And I think the universities, if they were on the hook for even a small fraction of that debt, would almost like not let you major in something unless they felt like you were going to pay back the debt. So I think you're going to, there's a reckoning there. The university costs can't get bloated forever. I think some of what's going on around admissions, the scrutiny, I think it's healthy. And I'm a big advocate of a competency-based world where it's not how long you sat in a chair or how long you threw Frisbees and went to frat parties.

42:47It's what do you know and what do you not know? And so we are working and I know others are working to create competency signals, credentials that if you do it, employers will recognize it. And it doesn't matter if you did it when you were 16 or when you were 36, you're not qualified for that job. I think that's where employers want to be. And I think it's not going to, quote, disrupt college, but it's going to give college an alternative, which I think is very healthy. I think one of the big shifts that's coming as well is just the type of skills that are going to be relevant in a more and more AI related world.

43:20So if you think ahead 10 years, 20 years, 30 years, et cetera, are there specific things that, for example, you're encouraging your kids to learn or that you think that people should really be focused on as they think ahead for their own children, you know, in terms of the skill sets that will be relevant in a generative AI world? A lot. And I both have like preschool, kindergarten aged kids. So these are active decisions. No, it's a good, good, good question. It is something I think a lot about. But I think there's at least two metaphors that can help us think about that problem. One metaphor could be, we can go back to the 19th century.

43:52And I'm not just going to talk about, a lot of people made the metaphor of steam engines and horses and all of that. But I actually think the camera is the best metaphor because being an artist was a real thing. It was really a technical field. You were a portrait painter. And the best artist, you would study for years to be as accurate to reality as you could. And they could look at how the light moves and all of that. And all of a sudden, a camera comes out. And all of a sudden, artists start saying, okay, that's the end of art. Like, this thing can capture reality better than, you know, anyone can.

44:23But then very quickly, people realize that, no, in some ways, this liberates the artist. That this allows us to think about, I mean, it's not a coincidence that the Impressionist movement kind of coincided with the advent of the camera. People started saying, no, it's not about capturing reality. Art should be about the impression. Art should be about the feeling it conjures. And it actually all of a sudden led to an explosion of what art is. It allowed art to come out of that trap of just painting nobility and these grand scenes and start getting into things that evoke and really challenge us.

44:53I think the same analogy, people are saying, oh, crap, this thing can write pretty well. This thing can code pretty well. This thing can create movies, create images pretty well. Well, so what that tells me is that it kind of liberates the creator to move beyond that. Someone who can elevate and integrate and manage these tools. The other metaphor I would say is, imagine if, just as we're saying with Conmigo, every teacher now has a teaching assistant. So now every coder is not going to have an army of coding assistants. Every writer has a writing assistant. So people are going to have to move into the managerial ranks quite quickly and figure out, okay, instead of being a coder, I'm now an architect.

45:32Instead of being a writer, I'm now an editor. And in order to do that, you have to know those skills really, really, really well, arguably better than previous generations. So I think it's even more important that people get not just adequate, but excellent at writing, communicating, coding. I think creativity, because there's so many outlets, someone who's creative, who leverages these tools is going to be like unstoppable. They're going to have like godlike power. And that's good. I know there's, you know, there's these debates with the Screen Actors Guild, and it's a very sensitive issue. But the way I think it, just as YouTube, there's a lot of talent that would not have been discovered without YouTube.

46:08You would not know about Justin Bieber. You would not know about me. Yes, I just put myself in the same category, very self-aggrandizing. But, you know, think about how many movies we've all seen that had like$100 million budget and were horrible. Like horrible waste of time movies for$100 million. You're now going to have a ton of people be able to create movies with similar special effects and acting and screenplays and everything. And a lot of them are going to be horrible. In fact, even more of them are going to be horrible. But every now and then we're going to discover the Justin Bieber of moviemaking.

46:37And that's an amazing movie. And that's some kid, he or she did it on a budget of$1 ,000, not$100 million. So how do we foster that? How do we give more time and space for that? But I think if you have kids, it's kind of very traditional advice. They should just get really good at reading, writing, math. I've never been a subscriber of like, oh, there's Google. You can search stuff. You don't have to learn anything anymore. Nope. The more that you have rapid access to knowledge in your head, the more that you are fluent with your mathematics, that you have information and concepts at the tip of your fingers, these tools will accelerate you more than anyone else.

47:11And the kid that has to get the calculator for three times seven or has to go to Google to figure out when World War II was, the world is going to pass them by. Amazing. I think that's a really great note in terms of an optimistic view of, you know, it's the same basic skills. It'll be important 20 years from now. And kids really just need to learn the basics. And you've provided such a great platform to achieve that by. So thank you for all the work you've been doing. Oh, thanks for having me.