#154 Sal Khan: Revolutionizing Education with AI

0:00It's interesting, especially in poorer parts of the world, including in poorer parts of the United States, people are skeptical of things that are free. They think there must be a catch. And that's the irony here because affluent families, they have high usage of Khan Academy. They know that it's better than what they could pay for. And then you have poor families that don't fully appreciate that. So we need to, and obviously we're working with anyone who we can to build that awareness. Khan Academy already exists in 50 plus languages to different degrees of fidelity. For example, in Spanish or Brazilian Portuguese or Hindi, it's almost as good as it is in English.

0:36And Generative AI adds an interesting dimension to this. AI might be the most important new computer technology ever. It's storming every industry and literally billions of dollars are being invested. So buckle up. The problem is that AI needs a lot of speed and processing power. So how do you compete without costs spiraling out of control? It's time to upgrade to the next generation of the cloud. Oracle Cloud Infrastructure, or OCI. OCI is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds, offers one consistent price instead of variable regional pricing, and of course, nobody does data better than Oracle.

1:29So now you can train your AI models at twice the speed and less than half the cost of other clouds. If you want to do more and spend less, like Uber, 8x8, and Databricks Mosaic, take a free test drive of OCI at oracle.com slash Eye on AI. That's oracle.com slash Eye on AI. Hi, I'm Craig Smith, and this is Eye on AI. In this episode, I talk to Sal Khan, founder and CEO of Khan Academy. We talk about how artificial intelligence and generative AI can transform education. Sal discusses Khan Academy's new AI-powered tutoring tool called Khan Amigo, powered by GPT-4, which provides personalized instruction and assistance to students.

2:31We talked about the challenges and opportunities of implementing AI in education, including issues around bias, transparency, and cheating. Sal shared his vision for using AI to provide high-quality, personalized education to all students globally and how it can supplement teachers instead of replacing them. I hope you find the conversation as fascinating as I did. I'm Sal Khan. I'm the founder and CEO of Khan Academy. We're a not-for-profit with a mission of providing a free world-class education for anyone, anywhere. And where did you go to school and how did you start Khan Academy? Oh, it depends how far you want to go back where I went to school.

3:19I was born and raised in the New Orleans area. I eventually ended up doing my undergrad at MIT. And I think that's a little bit connected to, I initially thought I wanted to be an AI researcher back in the day. But I kept falling into these education. I kept gravitating towards education. I ended up going and working in tech, not in education. And then I go to business school at Harvard Business School. And then I found myself as an analyst at a hedge fund. And I always used to tell folks when I was at the hedge fund, hey, I'm just going to do this long enough so that one day I could start a school on my own terms and be like the Dumbledore figure.

3:55And it was around that time, 2004, I was a year out of business school. I had just gotten married. I had family visiting me from New Orleans. And it just came out of conversation that my 12-year-old cousin Nadia was having trouble in math. I offered to tutor her remotely. She agreed. Long story short, it helped. She got caught up with her class, even a little ahead of her class. At that point, I become what I call a tiger cousin. I called up her school. I said, I really think Nadia Rehman should be able to retake that placement exam from last year. They said, who are you? I said, I'm her cousin.

4:25And they led her. And the same Nadia who was put in a remedial track was then put into advanced track. I started tutoring her younger brothers. Word spreads in my family. Free tutoring is going on. Before I know it. I'm tutoring 10, 15 cousins. And I had a day job. I was an analyst at a hedge fund. And then it was around 2005 that my background was in software. I said, oh, I see some common patterns with my cousins. They need more practice. They had some gaps in their learning that was keeping them from advancing. So I started writing software for them that would let them practice at their own pace, let me as their teacher or their tutor keep track of what they were doing.

5:00And I called it Khan Academy. The domain name was available and it was, it's kind of my family project. And I did have even then slightly delusional aspirations that one day it could turn into a real institution that could educate a lot of folks. In 2006, a friend suggested I make videos to supplement that software. I initially thought it was a bad idea. I said, that's frivolous. You know, YouTube, you can't do anything serious on that. But then I gave it a shot and And that took on a life of its own. And then by 2009, I frankly had trouble focusing on my day job. And we set up, at the time I set up Khan Academy as a not-for-profit, mission-free world-class education for anyone, anywhere.

5:40Quit my day job. It was a stressful year. But by 2010, we had our first philanthropic support to become a real organization. Yeah. And you have a bricks and mortar school somewhere, don't you? Yeah. Yeah, so this is a sister side project to the core of Khan Academy. I wrote a book in 2012 called The One World Schoolhouse. The first third of the book is how did the education system just get to the way it is today? The middle third of the book was what I just outlined. How did I fall into this and this Khan Academy journey? And the last third of the book was, well, given the tools we now have and what the world likely needs, What could education look like if you think about it from first principles and ideas of personalization?

6:26You don't have to have every student work in lockstep. If you need more time to learn something, great. If you've already learned it, move on. Related to this idea is this notion of instead of seat time, hey, I need you to sit in a chair and learn a year of algebra, two years of algebra. What matters is whether you know it or not. It might take you two weeks. It might take you two years. So whether you want to call that competency-based versus seat time or mastery learning. mastery learning is a related idea that if you haven't learned it well yet, keep working on it. You should have that opportunity.

6:56And there were other ideas, how can you use tools so you can free up more time for human interaction, et cetera, et cetera. One thing to write about all this whole other thing to implement it. Also, I had a selfish motivation in 2014, my oldest child, who's now 14, but he was entering kindergarten. And I was already doing the rounds, telling everyone about personalization and mastery and how all of these tools could be used. And I didn't want to be a hypocrite that I'm saying all this stuff, but my own kids aren't experiencing it. And so we started Khan Lab School. It's a different organization than Khan Academy, but I obviously helped start it and I'm the chair of that school.

7:28But the whole idea is Khan Academy is already reaching hundreds of millions and one day billions of folks, and that's great. Hopefully we can have a lot of positive impact through that. but it would be a shame if even though that exists, that we haven't been able to reimagine what schools could look like. And so the hope is Khan Lab School. It is a K-12 school out here in Silicon Valley. It's only 300 students, but if we can be very transparent and show ways to do it that are, you know, aren't ridiculously resource intensive, that can make a more enriching experience for everyone. So be it. We also have another project.

8:06This is a newer one, Khan World School. We started with Arizona State University. We're like, Can we scale this idea even further? Have an online high school and now middle school that can embody some of these same principles. Yeah. How much AI? Well, first of all, what did you study at MIT undergraduate? And then I wanted to ask how much AI is in the platform at this point? I was a computer science. I majored in computer science and I majored in math. and some of my freshman advisor was Henry Patrick Winston, who's one of the fathers of AI. I took the Marvin Minsky course, who was obviously one of the titans of AI back in the day.

8:48In terms of Khan Academy, we've, until this last year, we had dabbled a bit in AI to do things like recommendations for the next best activity, et cetera. When OpenAI reached out to us a little over a year ago, this was summer of 2022, and showed us GPT-4 and we were seeing it before anyone else saw this. And this was months before chat GPT comes out. Chat GPT is not even using GPT-4. It's really just a chat interface on GPT-3, GPT-3.5. But that was the first time we said, wow, you know, a lot of ways, everything that Khan Academy is about is scaling what I was originally doing for my cousins.

9:24And this technology bears promise to take it that much further, go that much richer. So, you know, we signed the non-disclosure agreement with OpenAI and we went heads down to say, all right, is there a way to do this so we can capture the upsides, a tutor for every student, a teaching assistant for every teacher, and also mitigate the downsides, you know, the cheating risk, the bias, the transparency, hallucinations. A lot of these models out the box aren't that good at math. How do we address that? And that's what we've been working on. And we launched in March of 2023 at the same day that GPT-4 was launched, and it's called Conmigo.

10:00And I think when people use it on Khan Academy, they see that it's dramatically different than what a lot of folks even come to expect from GPT-4. It really has a Socratic tutor element to it. It is surprisingly better at math than a lot of people expect. We've done a lot of work on that. We've done a lot of work to minimize hallucinations, and we have a whole series of guardrails to help protect young people, young learners. Yeah. And how do you guard against hallucinations, which for me is the real challenge of using LLMs for education? Do you tie it to a vector database, which seems to be the most popular solution?

10:43We do all of the above. We definitely have a vector database. That's this whole idea that you can put documents into a form that when the AI is having a conversation with you, it can say like, what are documents that are very close in the vector space to this conversation and then slurp those out, summarize them, and then use those as your fact base. So we're doing that. You know, we put very simple guardrails, like it's, we use a vector database actually for Conmigo to provide internal links within Khan Academy. Links are one of those places where if you let it make up the links, it'll make up links.

11:15So it doesn't provide links outside. It's also in the prompting, it anchors usually on Khan Academy content, either article, video, or exercise. So that also helps minimize things like hallucinations. And then it's at this stage, I think in the next year or two, you're gonna see the underlying models get a lot better at both the hallucinations and providing references. But I also think there's just a digital literacy aspect to it. Making folks know that, look, If there's something that is being said that you're going to index very heavily on, you have to double check it. And honestly, that's true even before generative AI.

11:49That's true even if a teacher tells you something in a classroom or you read something on the Internet. Yeah, I'm, as I said, particularly interested in AI tutors. So is Canamigo essentially an AI tutor? I haven't used it. I've talked to people that have. and let me explain what I mean by AI tutor and you can tell me whether you guys are doing this. So there's a company that I'm familiar with, I've written about in Korea. There's also in Yoshua Bengio's, one of his students has a company called Corbit. I don't know if you're familiar with them k-o-r-b-i-t out of montreal it's an ai tutor and both of these guys uh these companies say they have uh a you know a knowledge tracing algorithm that that tracks uh the the student knowledge as the student interacts with the system they have the recommendation engine that then recommends content of video or text or audio to the student to fill in gaps in the student's knowledge.

13:05And then they, which I thought was brilliant, the Korean group has a score prediction algorithms. And so if you're working on, they focus on test prep, but it would work equally in coursework, it predicts your final score. And if you follow the recommendations that the system's giving you based on the knowledge tracing algorithms, your predicted score goes up until at the end they meet. So you get the score that it says you're going to get, and it's an incentive to follow the recommendations. That, to me, sounded like a game changer, but I don't see it anywhere on the market. I mean, these guys are having, other than test prep, they're having difficulty getting into schools and that sort of thing.

14:06I guess, one, does that kind of a system make sense? Is Khan Academy building something similar? And two, why is there such resistance to these solutions, which really do promise personalized education for anyone with an internet connection? Yeah. You know, what you just described where a system is intelligently recommending the next best thing for a student to work on based on past performance, and he's even able to do some inference and prediction and even tell, say the student or teachers or parents, hey, and this is a way that you can help also support what's next best for the student. In many ways, that is what Khan Academy 1.0 is.

14:49That's kind of what we've been working on for over a decade now. I think what we are now talking about is taking it a step further, which is feeding all of that into a generative AI model with a lot happening at the application layer where it can converse with you, where it really feels like you're, and it's, it can dive in deeper. If you're watching a video, just as if, if you would ask a teacher a question, you can ask it a question. If you say, why do I need to learn this? And it'll say like, you know, well, what do you care about? And then you say, well, I care about this. And it remembers it.

15:24So it's starting to get into almost more of the, the EQ side of a, of what a great tutor would be. Yes, it can help you on block. It can recommend, but it can also emote. And I know that's a strange thing to say, but it can also empathize. It'll remember interactions. It can have a long form relationship with the user. And also it's connected to the teacher and their access is something of a teaching assistant where it can report back to the teacher or the parent or both. What's the student's been up to, what they've been up to, collaborate with the teacher to modify lesson plans, create even potentially help grading, things like that.

16:02So that the teacher feels even, sorry, the student feels even more supported. I actually have never sensed resistance to these types of things. I think it's more about traditional ed tech. And I'd put Khan Academy in this category is you make a tool and you go to a teacher and we've had hundreds of thousands of teachers use it in classrooms. But if you go to a new teacher who is doing their thing and you say, look, we have the efficacy studies. If students do this, they're accelerating by whatever, 30, 40 % and here's all your tools. Most teachers, when they see that, if they think you're being genuine and the efficacy studies look legit, they're like, oh, this is great.

16:39I definitely want to do this. I care about, I want to accelerate, but they're already overworked. They already have a million things to do. And now you're expecting them to learn one more tool and figure out how to, and space in the classroom and all of that. And so I think that's historically been the hard part. What's interesting about this next wave, especially with generative AI, teachers spend 10, 15 hours a week doing things like lesson planning, grading papers, writing progress reports, just helping them, having the generative AI help them with that. I think we can get that 15 hours down to one hour, two hours.

17:12So it's going to create natural, it's going to create less time and energy scarcity for the teacher, which is a natural incentive for them to do it. And then obviously if it can help the student too, and they can more personalize their questions and there's healthy reporting and safeguards going back and forth. We just started talking about this only a few months ago. And when I tell our team like, Hey, why aren't we having more school districts using it? They're like, it's us. Like we're, we're, we're pushing people away right now. People are actually very eager to, to use the, to use Conmigo.

17:43Yeah. Uh, and, and the thing I've heard, I, and I've spoken to teachers, I've written sporadically about this since I think 2019, my first article, I did an article for the times. Um, the, The teachers are eager, and I've spoken to teachers that use Connecotomy because they can use it for free. And this leads to a question about the business model. But if you want a school system, a district to adopt it, you know, it gets tied up in, you know, budgeting and teachers unions and all these different things that make it very complicated. So Khan Academy has been successful in penetrating that market because it's free and the teachers can grab it directly.

18:39But what's the business model? I mean, how can you keep developing and keep it affordable? Yeah, and it's getting more and more nuanced. I think historically, you're absolutely right. Our mission is free world-class education for anyone, anywhere. It's not free to produce Khan Academy. So we've had to raise, you know, our budget now is pushing$70 million a year. That's a lot of money. It gives me some anxiety having to, you know, work with the team to raise that every year. But I remind folks, that's the budget of a large high school in the United States. And we're reaching a fraction of humanity and hopefully a large fraction eventually.

19:17But there's a few things we've realized over the last few years. If you rewind about six years ago, we had this, all these efficacy studies about how Khan Academy can help if students put even 30 to 60 minutes a week in. We had a lot of teachers, we call them grassroots teachers who were just using it on their own, hundreds of thousands. And then we would ask ourselves, well, why aren't districts using us systematically? Why don't they use this as like a core resource in what they do? And so we started talking to leadership at school districts and almost every conversation would start off with one of them saying, oh, first of all, thank you.

19:49my niece or nephew or my son or daughter, or even myself going through graduate school, you know, Khan Academy really helped in some way, shape or form. And they would then say, oh, and we believe your efficacy studies. I mean, my niece, you know, swears by it, et cetera. But if you want us to use it systematically, you have to give us support, training, integration with our rostering systems, district level dashboards. And that was the first time in Khan Academy's journey. This was five, six years ago where we said, wow, that's a lot of boots on the ground, so to speak. That's a lot of enterprise level.

20:25But if we're really serious about moving the dial, especially for the kids who need it the most, we should do those things. But at the same time, those things are also very expensive on an incremental basis to go send someone there to do the training and look at the IT systems. So we told the school districts, we're like, look, our whole goal is to just improve education generally, support you, but we're going to have to charge you for that. So we created what we call our district offering. It's$10 per student per year for all of that bespoke support and dashboards for the district to use it well.

21:01You fast forward to this past year and we launched Conmigo. Conmigo adds another element to it, which is the computation costs of especially frontier models like GPT-4, generative AI. It's very expensive. Initially, our estimates were looking something like 10 or 15 dollars per user per month just for the computation you know there's a reason why chat gpt pro is 20 that's the one that uses gpt4 is 20 a month um and so coming out of the gate we just we just were very ultra transparent with school districts we're like we we're building this thing we believe the cost is going to come down but if you want to use this right now we just want to cover the checks that we have to write to open ai which is writing them to microsoft which is essentially writing them to NVIDIA.

21:45And so we already have about 50 ,000 districts, sorry, 50 ,000 users in districts, not 50 ,000 districts, 50 ,000 students and teachers already this school year using Conmigo in that district setting where the district is resourcing it in this way. And there's even a, you know, the state of Indiana is actually done, is doing a pretty big project with a bunch of folks using Conmigo and Con4Districts. The good news is we think we're going to be able to bring down that dramatically. I think by the end of this year, it's going to be the computation costs are going to be on the order of 10 or$15 per year.

22:23We've been doing a lot of efficiencies on our end on how we use that computation. And then there is a pretty heavy support and training, especially with generative AI. You want to almost have a white glove service for the teachers and for the administrators. So that's probably also just the cost of that is probably another, let's call it$10 per user per year type of thing. I mean, it's mostly focused on the teachers and the administrators though. And, but, you know, I think it's gonna get us to a place where in places like the United States, this will be, I think much more affordable in the 20 to$30 per year range, maybe a little bit more.

22:59And there's something also interesting where if you go into the developing world, especially the developing world that's on the other side of the planet, and we have a dedicated instance of GPT-4, it's not being utilized that much at three in the morning, California time. That could be really interesting to utilize that and offer it at a much, much more accessible rate. You know, as I was saying, we're a nonprofit and our mission says free. We're just trying to make it as accessible as possible. So, you know, think about what we can do in India or the Philippines or places like that, which we're already thinking about.

23:30Yeah. On the, on where it's going on, and I'll, I'm sorry, I didn't play around with Conamigo before the call, but I will.

23:44This system that I described and that Conamigo is building toward, recently, again, with generative AI, real-time avatars have made a huge leap. And I always thought that that was the missing piece in AI tutors because it's one thing for, and I've visited schools that are using different kinds of technology and you have these kids like slumped in their, in their chair and they're, you know, tapping on the keyboard and looking at their phone and paying attention to their friends. But if you had a face of someone that's reacting to you and talking to you as opposed to just lines of text streaming across the screen, I think it would be a lot more engaging.

24:40Have you thought about that? Yes. And I have no doubt that that's going to come. I don't think it's going to come immediately. I think we're probably looking at probably the three-year timeframe for that to become really well done, well implemented. And I think it's, I don't think that's actually a substitute. I think that's just going to be part of the mix. As we know, kids, even with real human beings, sometimes chat is their optimal mode. There's sometimes it feels like a lower threshold, easier to ask questions, but we're already working on being able to have voice conversations. And yes, I think it's, in the not too far future, you're going to be able to have a video conference with your AI tutor and see its emotions.

25:16And it will be able to see your emotions too, which will be pretty powerful. Yeah. So the adding generative AI capabilities is kind of where you are and Conamigo is the focus right now or where is the company building? Yeah, it's all of the above. Our core, our vision, free world-class education for anyone, anywhere. We've always said that means we want to have all the core academic material from pre-K through the core of college available in all of the world's major languages to be usable in a self-paced mastery way, exercises, videos, articles, teacher tools, and now generative AI tutors and teaching assistants.

26:07So we're continuing to develop our core content. We are continuing to build out teacher and district tools. But yes, generative AI is definitely an ever increasing area of focus for us. Yeah. Do you partner with different schools or with different sort of MOOCs like platforms like edX or Coursera. You know, I talked to Andrew periodically about this or is your content of your building your content yourself? Our content, we've historically we've done that in house, but we're definitely talking and working with all of the major AI providers, folks like OpenAI and Google and others, to just make sure that we have access to the state of the art and then figure out how to apply it in thoughtful ways.