Rapid Response: Meet your kid’s new AI tutor, w/Khan Academy founder and CEO Sal Khan

0:00Scaling a business in the US can feel like a maze. Each state has its own payroll, benefits, and compliance rules, pulling your focus away from growth, which is why founders use Deal. Deal is the professional employer organization, or PEO, that gives you a dedicated HR expert, plus Fortune 500-level benefits for your team. The National Association of PEOs says businesses can grow twice as fast if they use one. So, if you're scaling, make it simple with Deal. Go to deel.com. slash MOS and get up to three months free. If you're in private capital, you understand that your next great deal starts with who you know.

0:41That's why over 3 ,000 firms trust Affinity, the CRM platform purpose-built for private capital. Affinity's relationship intelligence reveals your team's strongest paths to founders, investors, and decision makers using insights It's hidden in emails, calendars, and meetings. With powerful AI tools like Affinity NoteTaker and Deal Assist, you'll spend less time managing data and more time closing the right deals. Learn more at affinity.co slash marketplace. Sam Altman and Greg Brockman of OpenAI reached out to me and said, hey, we have a model coming up. Even though it had issues with math, it had issues with hallucinations, it was able to really, not just pretend, but in some substantive ways, really act like a strong tutor would do.

1:34Used well, this could get us much closer to this ideal of free world-class education for anyone, anywhere, or being able to scale up the type of tutoring that I was doing for my 12-year-old cousin back in 2004. That's Sal Khan, founder of Khan Academy, an online learning platform that's made educational videos accessible to millions of people across the globe. Khan Academy has gone all in on AI through an AI assistant called Khan Migo that's already being field tested by thousands of students and teachers. Sal has recast assumptions about education through free online videos, thousands of which he's personally hosted.

2:13Now he's advancing a tech revolution using AI that may have even more impact on how we learn, how we teach, and the concept of education for people of any age, anywhere. This is Rapid Response. Let's get to it.

2:40I'm Bob Safian, and I'm here with Sal Khan, the founder of Khan Academy. Sal, thanks for joining us. Thanks for having me, Bob. So Khan Academy now serves more than 150 million learners around the world. Many of us almost take it for granted that anyone with web access can get expert instruction for free. But you argue there's an even bigger revolution coming in education, courtesy of AI, which you write about in a new book, Brave New Words. What is making you so excited? It really does, to some degree, go back to how Khan Academy got started. It started really with me tutoring a cousin who needed help remotely.

3:24She was in New Orleans, which is where I'm born and raised. And I was in Boston at the time. And it worked out for her. She went from being in a remedial math class to being in a more accelerated math track. Word spreads in my family, free tutoring is going on. And I'm now tutoring 10 or 15 cousins. My day job, I was an analyst at a hedge fund. This is back in 2004, 2005. And I started writing software for them to help scale that kind of personalization that you can do when you're a tutor. And then I started making videos for them. And then it became clear that people who are not my cousins were using those videos.

3:56And if you think about that arc from back in 2004, 2005, all the way to some of the numbers you just mentioned, I think it's actually close to 160 million or 170 million now registered users. We're in 50 plus languages. We're being used in almost every country in the world. If students are able to put 20 minutes a day for three or four days a week, they're accelerating 30, 40, 50 percent relative to their peers. The whole arc of it really is how do we use technology to scale the type of personalization that you're able to do and the type of engagement you're able to do when you're able to have a one-on-one tutor.

4:32What AI does is allows us to go much further in that dream. There's interactions that folks have with our AI tutor called Conmigo that are really indistinguishable from when I used to text message my cousins back in 2004. And it can do a lot of things that I wasn't able to do with my cousins, emulate a historical figure, simulations, assess you in much broader ways than we've been able to do with computers, be a teaching assistant for teachers, not just report to them what students are up to, but also help them develop lesson plans, grade papers, write progress reports, all of this administrative work that isn't really directly student-facing that teachers need support on.

5:15When you started Khan Academy, when you started posting all these videos, there were some folks who were kind of threatened by it because it was such a different way of accessing information and education. And there's a lot of that fear around AI also. I mean, a lot of the initial reactions to ChatGPT were like banning it, that it's like a tool for cheating. Yeah, there's a lot of fears around AI and a lot of them really become noticeable in the education space. Even before generative AI, especially in the education lane, I've always made the argument that technology should be a means for liberating the classroom.

5:56So instead of making it, you know, someone at the front of the classroom's lecturing on and the kids are trying to stay awake, which is actually a very inhuman experience. You're not allowed to communicate with each other. You're not allowed to play. That's not natural. And if you can go from that because now you're liberated by, well, the kids can get the mini explanation at their own time and pace. Now we can use class time for a Socratic dialogue. Now we can use class time to do a game, to do a simulation or to do independent practice. The teachers can support people in a more personalized way.

6:27All of a sudden you've leveraged technology to make things more human. You fast forward to this AI world, I think it becomes even more relevant. I think teaching is one of the safest professions in a generative AI world. And in fact, it's going to be enhanced and more value is going to accrue to any profession that really anchors on the human element, on the human connections. And the AI is going to be there to streamline a lot of the less human things. Imagine being a seventh grade English teacher. You have five classrooms of 30 students, 150 students. You've just assigned a book report on the great Gatsby to all of them.

7:01It's going to take you about 20 hours to get through those papers. If you could take that 20 hours and turn it into two hours, That's a huge win for the teacher, especially at a time of record burnout. But it's a huge win for the students because now the teacher has more energy to do some other things or maybe to assign more writing for the students so they get more practice and more feedback. So I think this is used well as long as we put pedagogy first. It can actually unlock the human element. Yeah, I mean, you have to trust that what that AI is doing in grading those papers is appropriate.

7:33And I know you have aspirations for what AI can do that you were introduced to, as I understand it, by having kind of an early look at playing with chat GPT. That's right. You know, we've dabbled with pre-generative forms of AI over the years at Khan Academy to do things like recommending videos or recommending the right next exercise. And I, just as a nerd, had been paying attention to GPT-2, GPT-3 and all of that. And it's fascinating, super cool. But I didn't think it was ready for primetime. And when Sam Altman and Greg Brockman of OpenAI reached out to me and some of our leaders and said, hey, we have a model coming up.

8:13This was summer of 2022. This was about five or six months before ChatGPT came out. And what they were showing us, by the way, was more advanced than what the world would see in November of 2022 at ChatGPT. What we saw with GPT-4 back in that summer, even though it had issues with math, it had of issues with hallucinations, making up facts, it was able to really not just pretend, but in some substantive ways, really act like a strong tutor would do. And that's when we said, look, there's a lot of risks here and we have to mitigate those risks, put some guardrails around this, especially if it's going to be used in an academic setting.

8:49But used well, this could get us much closer to this ideal of free world-class education for anyone, anywhere, or being able to scale up the type of tutoring that I was doing for my 12-year-old cousin back in 2004. Sure. You see this, you start playing with it, and you realize, like, we've got to change our whole business or add this to our business. Is that where Comigo comes from? That's right. And, you know, I was definitely all in on this. But you could imagine the debates that we had in the organization. How do we handle cheating? How do we handle hallucinations? How do we handle math errors?

9:22How do we handle a situation if a student has an inappropriate conversation with the AI? So we and I encouraged our team, we shouldn't ignore these risks. These are real risks. But we should write them down and we should turn them into features that mitigate the risks. But keep moving forward on ways that we can create value. So we started doing that in the fall of 2022. This work of creating a tutor on Khan Academy for every student, a teaching assistant for every teacher. As you mentioned, it eventually got named Khan Migo. And we were able to launch it in March of 2023, the same day that GPT-4 was launched because it is based on GPT-4.

10:02You now have thousands of users who are engaging, teaching, learning through Conmigo. And I imagine you're getting data and information back about what works best and what doesn't of these. Yeah, we've learned a ton. It's been a little over a year that Conmigo has been out and it's evolving very quickly, as you can imagine. But from the student side, we've gotten a lot of feedback just saying, hey, it's amazing for me to ask, why does this topic matter? Being able to connect it to things happening in life. I mean, it really does feel like a tutor for a lot of them. But we also learned that it's not equally helping all students.

10:43There is a I would say about 20 percent of students that immediately just run with it. They get what this is and they're off to the races. They're asking good questions and they can't get enough of it. But I would say a lot of other students, they're actually, unfortunately, not used to being curious in the same way or being allowed to ask anything they want to ask. But the more we talk to teachers, we realize that this isn't an AI issue. This is actually an issue in the education system that the students really don't even know how to communicate in many cases. They don't know how to even communicate what they don't know.

11:15And so the teachers are telling us, even though some of these students might struggle typing into an AI, hey, can you explain the distributive property or what happened to that negative sign? Even though they're having trouble articulating that, that skill of articulation is arguably much more important than the skill of knowing how to do some algebra or know how to do some physics. Yeah, I mean, they have to learn a new way to learn. And I guess teachers at the same time have to learn a new way to teach with these tools, which is not dissimilar from what happened with Khan Academy. You called this flipping the classroom, right?

11:51Yeah, well, you know, back in the day with Khan Academy, I was writing the software, I was making videos, and I started getting letters from teachers out there saying, hey, you already gave a pretty good mini lesson on photosynthesis or factoring polynomials. I, as a biology or as an algebra teacher, I don't want to use my valuable class time anymore to re-give that lecture. I'm just going to have the kids do what used to be classwork, essentially a lecture, do that on their own time at home. And then I, the teacher, want to use our class time to do what used to be homework, the problem solving, because then students are supported.

12:21I, as a teacher, now have a much better visibility when they're doing problems in the classroom on what their actual understanding is. In a traditional classroom, if I'm just lecturing And every now and then I say, hey, Bob, what do you think? I'm not really getting a sense of how well students know it. You fast forward to this AI world. In some ways, this AI will make traditional workflows for teachers or traditional processes more efficient. But I think it's going to open up the aperture of what a teacher can do. You can imagine this world where we're about to teach the Underground Railroad and the teacher, and we have seen teachers do this in the past year, say, hey, we have a special guest and get our AI simulation of Harriet Tubman on the overhead projector.

13:03That's a really cool way to get kids immersed into a subject in ways that they wouldn't have before. Before you assign a paper, even before ChatGPT, God knows how the kids wrote that paper. They could have paid their friend, their sister. There's websites that'll do your paper for$5 an hour. And then they'll submit and the teacher has no idea what happened. And all they see is the final output. They grade it. Now the students could do their essay with Conmigo. Conmigo would act as a coach. The student would do the work. But then when they submit it, Conmigo cannot just give the teacher the final output, but actually the whole process.

13:38We spent four hours on this. Here's the whole transcript of us working together. But if Bob goes to chat GPT or his older sister and copies and pastes it in, Conmigo tells the teacher, this is shady. I don't know where Bob got this essay from. So I think it'll mitigate fears and problems that existed well before AI. We've gotten very clear feedback from a lot of students, especially high school and college students, that especially these days, they feel very afraid to test out ideas. One, they might they think people will judge them, think that they're not smart, or they might have said something the wrong way and people are going to be offended.

14:12And so to be able to do it in a safe environment with the AI and oftentimes take both sides of an issue and test out arguments has given them a lot of confidence to then go into with real people and have those same conversations. Now, on the other hand, it's going to give me confidence. I'm going to share all these things with it. But then it's going to report back on how my learning is going to my teacher, which might feel a little creepy. Like it's almost like you're spying, like I'm working with it as if it's private, but it's not really private. Yeah, and you're pointing out a very real tension that we are having debates inside of our organization on.

14:48If I was a student in a classroom, yes, my teacher could say, hey, what has Sal been up to? and Conmigo would say, yeah, you know, he's been doing a little bit of calculus. And we had a really great conversation about supernovas. And my teacher said, tell us more. What was Sal asking? So that will be transparent to the teacher. And if I said something potentially very intimate, like, hey, I want to harm myself or I'm feeling really down, Conmigo does actively notify the teacher. So our debates are, hey, is that a good or a bad thing? We're erring on the side of safety. We probably for sure have to do it for under 13 users.

15:22But might it be better? And we don't know fully the answer to this. And to some degree, we're going to do it based on what the districts feel comfortable with and what parents feel comfortable with. What's most extraordinary to me listening to all this is how quickly Saul and his team at Khan Academy have sprung into action to put generative AI to use and to test it in the real world when so many organizations are only tiptoeing their way toward the future. After the break, I'll ask Sal more about the guardrails needed around AI, as well as how he sees it impacting other industries. Stay with us.

16:01Real leaders don't back down when the stakes are high. They innovate, they push forward, and then they take the stage at the Masters of Scale Summit. Join us in San Francisco, October 7th to 9th, to hear from the CEO of the New York Times, scientists using cutting-edge technology to find cures, the leader of crypto powerhouse Coinbase, a retired four-star general, and many, many more. Apply now at mastersofscale.com slash apply25. That's mastersofscale.com slash apply25. If you're ready to take your startup from idea to impact, then AWS is your launchpad. From data storage to machine learning to secure app hosting, AWS gives you the tech, trusted by the world's fastest scaling startups.

16:51And here's the best part. Join AWS Activate today and you could score up to$100 ,000 in AWS credits tailored to your stage and network. Go to aws.amazon.com slash activate and start building. Expanding your business in the US can feel like a maze. Every state has its own payroll, benefits, and compliance rules, which can pull your focus away from growth. That's why founders use Deal. Deal is the Professional Employer Organization, or PEO, that gives you a dedicated HR expert plus Fortune 500-level benefits for your team. The National Association of PEOs says businesses can grow twice as fast if they use one.

17:35So if you're scaling, make it simple with Deal. Go to deel.com slash MOS and get up to three months free. Before the break, we heard Khan Academy founder Sal Khan talk about how generative AI is transforming education. Now, he gives a lesson in how to navigate AI guardrails and explains what he means by educated bravery. Let's jump back in. There's so much scrutiny right now of certain areas in education, you know, whether you're talking about critical race theory or whatever, that you have to think about your guardrails in really complicated ways because different school districts have different rules about those things.

18:25Oh, absolutely. And the good news is, Khan, we've tested it hard because obviously the last thing we want is some kid in some school district, you know, all of a sudden it looks like they're getting politically indoctrinated by the AI in some way and they take a screenshot and not a good look for anyone. And even more importantly, it's not good pedagogy. We test it hard to make sure it's not showing a political bias. I'll give an example. I had a reporter, they were skeptical about this. So we went into Conmigo and we took on the persona of someone who's very pro-gun control. And we told Conmigo, you know, I can't believe people still think the Second Amendment makes sense.

19:05There's all this gun violence. It's got to go away. 90 % of American classrooms, half of that 90%, if you're in a blue state, the teacher will say, yes, I 100 % agree with you. I can't believe that there's all these other people who want to keep it. And if you're in a red part of the country, they'd probably say, how dare you say that? This is such an important freedom and it keeps us from fighting off tyranny and blah, blah, blah. But Conmigo did neither of those. Conmigo says, look, before we go into the present, why do you think the founders included the Second Amendment? So it put it on me.

19:34It pushed my critical thinking. And I said, oh, well, you know, that was right after our independence and Great Britain didn't want us to have arms. So we needed it then, but it makes no sense now. And Conmigo said, hey, I think you're generally right about why historians think we have the Second Amendment. But before we judge it, why do you think it's persisted? And so once again, it forced me to think in terms of critical thinking as opposed to, which we're seeing all too often, people in their own echo chambers of other people saying, amen, yeah, that other half of the country is crazy, which is not constructive.

20:06Reading your book, like reminded me of a question that I sometimes ask myself, even when I look at some of the videos and the information that Khan Academy does, this question about sort of what is important knowledge in the future, right? Like, is it grasp of information? What are we preparing kids to do and what's going to make them most valuable in the work world? Like, how do you think about all of that shifting? Yeah. You know, even when search first came out 20 or 25 years ago, there were a lot of people and people have still made this argument. Oh, you could search for anything now. Kids don't have to learn as many facts or as much knowledge.

20:49And now you just have to know how to search. And now with ChatGPT or other AI tools, the argument becomes even stronger. Hey, this can draft your essay for you. It can kind of give you a business plan, whatever. The reality is it is going to be important to know how to use those tools. But in order to really be excellent at using those tools, you have to have a lot of knowledge yourself. I have seen the difference between how I can search when I have content knowledge versus when I see sometimes people who don't have the content knowledge. They oftentimes don't even find what they're looking for because they don't have enough context to search for the right words or to be able to evaluate the different sources they're getting and which one is more likely to be legitimate.

21:29And the same thing is going to happen with AI. So I say all of it's as important as ever or more important than ever to learn the fundamentals, the reading, writing, arithmetic, so to speak, and the content knowledge. You know, I'm not one of these people who think memorization is a dirty word. I think it needs to be more than just memorization, but I think it is very valuable to have a lot of content in your head. You're going to be a better tool user. And the changes that you see coming in education from AI, do you think they're going to be any less revolutionary in other areas? I think, well, education, obviously, this is what I think about all the time.

22:10So I think this is probably the most revolutionary. But I talk about tangential areas to education, assessment, job placement, interviews. I think all of those are going to be, you're going to see some really interesting changes over the coming five to 10 years. I always give the example, New York City Public Schools spends$40 ,000 per student per year on education. The average class size is 25 students. So 25 times 40 ,000 is a million dollars. I guarantee you that million dollars is not going to the teacher and it's not going to the facilities. And, you know, at best, that might be 20 % of that.

22:44The rest of that 80 % is layers of administration, paperwork, et cetera, et cetera. If we can make, you know, things like the registrar's office way more productive, then, you know, we can get people out of the registrar's office and turn them into teaching assistants or turn them into teachers. So then you'll have more supports for the students. You can name any industry. I cited the book that McKinsey, I believe, did an analysis and they said roofers would be the least affected. And then someone interviewed roofers. They're like, oh, no, chat GPT has transformed my life. I use it for my invoices.

23:13I use it for my pricing. I use it for my marketing. And it's like, OK, even the roofers have been transformed by this technology. You end the book with this phrase about existential risk or existential opportunity that find ways to make it an existential opportunity. Yeah. Yeah, and the phrase I use is educated bravery. And the spirit of it is, you know, you could be brave without being educated, but you might do some very stupid things if you're not well informed about why you're moving forward. Or you can be very informed and very educated, but not very brave. In fact, educated people are some of the best people at figuring out all the reasons not to do something.

23:52And the best is to have a healthy tension between the two. When we started seeing this technology and seeing all the opportunity, it was also very obvious to us that there's a lot of potential risks here. And I think there are, in fact, even inside of Khan Academy, there were folks who were like, hey, we should just not deal with this stuff. We should just ignore it. And ignoring it has more risks in my mind than taking action. Let's not ignore them. Those are real risks. Let's try to mitigate them. Let's turn them into features. But let's move forward because there's so much opportunity here.

24:22And if we think about it on a higher level, people like to philosophize and should it be regulated, et cetera, et cetera. and I'm very clear about it. This is not a flip of a coin. We are all actors in this game. And you could try to slow it down, but that's not going to do anything to the bad actors. The bad actors who are trying to manipulate our elections, who are trying to put out deep fakes, who are trying to commit fraud using this technology, they're not going to slow down. They're going to keep running ahead. And if the good folks slow down, we're going to have less tools to police them.