Back to school: How AI is transforming the classroom, with Sal Khan

0:00Starting a business comes with its share of ups and downs, which is why staying true to your vision is essential. a non-negotiable for Romeo and Milka Bregali, Capital One business customers and co-owners of Ra's plant-based restaurant in New York. Romeo and Milka took a leap of faith when starting their own restaurant, gutting an empty space and building it from the ground up, every pipe, every wall, every detail. But building from scratch came with a heavy financial burden, which is when they turned to their Capital One business card. With the flexibility of the card's no preset spending limit, they were able to spend more and earn more rewards while bringing their vision to life.

0:36Today, Raz's success is proof that with passion and the right support, it's possible to make your dreams a reality. Learn more at CapitalOne.com slash business cards.

0:51I'm 100 % against AI replacing human teachers. I think human teachers are hopefully one of the safest professions in this world. My view of technology has always been it should liberate the human teacher to be able to do more human things, which is forming connections with students. And that the tools, whether it's AI or other technology, should really just be about helping that teacher make it more personalized, make it more engaging, etc. Sal Khan is a parent, entrepreneur, and one of the leading thinkers when it comes to how AI is shaping education. His company, Khan Academy, was an early innovator in online education, and they now support over 150 million learners in 190 countries.

1:39Sal is not only thinking about how AI will impact educators. He comes at this topic holistically. How can AI restructure education to make it better and accessible to all? Our kids are heading back to school to start a new year, and AI is top of mind for lots of us. So on this episode, Sal breaks down how AI is shaping education. We'll talk about AI tutors, cheating, and what an AI-forward classroom will look like in the future. But before we get into all of that, we wanted to take a temperature check on how some parents are feeling about AI. So we rallied a few parents on our team here at Pioneers of AI to find out.

2:22My name is Leigh Tall, and I'm the head of podcasts here at Wait What? and I have 11-year-old twins, so they're about to start sixth grade. So for one of my kids' fifth grade project, he wanted to design a video game that would have an educational component where the user would have to answer questions about Egyptian civilization. And we ended up using this platform called Bloxels or Bloxels. He pretty quickly was figuring out kind of how to use it, but he was hitting some roadblocks and getting frustrated. So I said, let's go to ChatGPT and ask it to help us come up with a roadmap of how you would design a game using that platform step by step.

3:06So that was like a really great example, I think, of being able to use it in a fun and interactive way. You know, my other son, he's more of an AI skeptic, which is funny. At 11 years old, you know, he says, AI is just going to take over the world and humanity, and that's just really dangerous. And so he has more trepidation for sure. My name is Jordan Smart. I'm a producer at Pioneers of AI and my daughter is seven months old. It would be funny though if I just said, yeah, she used ChatGPT today to figure out how to warm her bottle. But of course that did not happen. So since she is not able to use AI as much as maybe a kid older than her would be able to, A lot of it is just me thinking about how I will introduce the technology to her in the future.

3:57I want her to be curious. I want her to be inquisitive and use AI to supercharge and augment that practice instead of being a kind of crutch to take the shortcut, to not develop the, I think, that really essential foundational desire to just know what something is. I'm Eric. I run marketing and audience here. We have a four and a half year old daughter who kind of speaks and often acts like a much older kid. And we may have gotten sort of lucky in some ways that she is not old enough yet to be using it. And in five years, if I'm enough of an optimist about it, in five years, it'll be five years better than it is right now, right?

4:44Now we're talking about ChatGPT5 and all the things it can do. There are lots of things that are improved. It really is much better. I do wonder, in three years, is that going to be exactly the same? Are we going to see the hockey stick growth? Is it, what are the types of things that she is going to be encountering? Because the reality is, it just may not be in the same galaxy by that point. But parents aren't just thinking about how their kids use AI. They are also invested in how teachers are implementing AI in the classroom. Eric used to work as a teacher. That was before ChatGPT, but he can imagine it saving teachers a lot of time on busy work.

5:26You know, I think one of the problems with teaching in public schools is classroom management and grading papers and administrative tests and things that people don't feel like they want to do. But I think there are lots of people who want to interact with kids and, you know, have a healthy, positive impact on a kid's life. And if those are the things that you can focus on, if your focus is social, emotional, on real, like, mushy, human-only skills. And I think there are more people who would be willing to do it if you eliminate some of the hardships of teaching because of the augmentation of AI.

6:00I haven't taught grade school, but I teach a college course. have been doing it for many years, and grading can definitely be so tedious for teachers. It's probably, like, discouraging if you're thinking about how can you, like, read the soul and heart that the students put into these papers? But yeah, I guess anything that can help lift the burden, if you can create a rubric that, you know, is checking for very specific things, or let's say, I don't know, maybe the LLM is good for checking grammar or something like that, But if they're co-grading in some sense, I wouldn't be against that. If you have a class of like 30 kids, 20 kids in a college course, how do you reasonably grade these papers?

6:45You know, correct me if I'm wrong, I know some of my college professors use teacher assistants, TAs, to grade papers. So that's kind of using a human to augment what you're doing when you're grading. Am I just now swapping out a student with an AI? I definitely understand teachers seeing this as a way to just kind of lessen the load. We're taking on these types of theory and practice questions around AI and education on this episode of Pioneers of AI with Sal Khan. And I promise no pop quiz at the end. I'm Rana El-Khalyubi, and this is Pioneers of AI. A podcast taking you behind the scenes of the AI revolution.

7:37Hi, Sal. Welcome to Pioneers of AI. I'm so excited for this conversation. Thanks for having me, Rana. So you and I share the MIT connection. You did your undergraduate there, and I am curious what was that experience like? Like, you know, MIT, I find, I don't know your experience, people tend to either love it or let's call it not love it. And I loved it. I was one of the people that loved it. I found like I was a kid in a candy shop. All the other people were so interesting and, you know, inspiring and it made you better. And, you know, and I think it also builds confidence to be in an environment where like the person sitting next to you might be on the Russian math team or, you know, and I'm just this kid from Louisiana.

8:19But you're like, OK, over time, OK, we're no different, really. So yeah, for me, it was a wonderful experience. My wife, I met at MIT. How cool. Actually, I gave a commencement address at MIT back in 2012. And I spoke how many couples came out of the place. And I made a strange joke that maybe it was a DARPA-funded breeding project. OK. I loved my time at MIT, too. I did my postdoc there and it was transformative. It's how I started my company and I just love it. You know, I come from a family that highly, highly values education, but I'm realizing that there's a growing narrative contesting the value of a college degree.

9:02So I don't know. I'm curious what you think about that, especially in the age of AI where you can do all sorts of things with AI and, you know, do you really need formal education? Yeah, look, I think there's a lot to that critique. You know, the cost of a four-year degree from a private institution without financial aid is the cost of a pretty nice house in most of the country. And especially when it's not clear that it's even – we're seeing underemployed folks, even in degrees that a few years ago were super sought after like computer science. We're seeing kids having trouble getting jobs now.

9:39I think people do have to ask the question. Students should be asking, what is the return on this investment? And sometimes I get pushback from folks saying, well, you know, but there's so much more about higher education and you're sitting at the quad. I mean, all these things that I loved about MIT. And I was like, yeah, that's true. But most people already aren't going to that experience. You know, they're commuter students. They're not throwing Frisbees at the quad and joining fraternities and sororities and whatever else. And so I do think if we can't give people at least a path towards economic mobility, then I think we're not serving them.

10:12So I do think it needs some reinventing. Yeah, we will come back to what that future of education looks like in a second. But before we do that, I am intrigued and I'm super inspired by the kind of OG story of Khan Academy. And actually, both my kids use Khan Academy a lot. And especially my daughter, who struggled with math when she was younger. She's fine. But she, you know, it wasn't her favorite subject. She was, I would say, nudged by her mom to use Khan Academy a lot. And she reminds me of Nadia in your story of the origin story of Khan Academy. So I'd love for you to share that story. Yeah.

10:52My oldest was very easy to get on Khan Academy. And he found math, I think, more straightforward. But I've had to do some nudging with my younger ones. So it's happening in my house. I think the nudging is even harder when you're like, why do we have to use it? It's your software platform. That's why you're making me use it. I'm like, oh, no, no, no, other people use it too. Yeah, just like hundreds of millions of kids use it too, right? Yeah. You know, usually that type of flex doesn't resonate with your kids. Yeah. You know, 2004, after MIT, I had gone, worked in tech, then I went back to business school.

11:24And then I was still in Boston after business school working at a small hedge fund. And I got married, 2004, a year out of business school. And Nadia, who at the time was 12 years old, she was having trouble in math. I offered to tutor her when she went back to New Orleans. So I started tutoring her remotely, slowly but surely. Honestly, it was a lot of confidence issues she was having. Had to deprogram that. She got caught up a little ahead of her class. At that point, I became what I call a tiger cousin. I called up her school. I really think Nadia should be able to retake the placement exam from last year.

11:57She's, they said, who are you? I said, I'm her cousin. And they let her. And the same Nadia that was struggling became one of the stronger math students in her class and was able to place into an advanced track. Word spreads in my family, free tutoring is going on. Before I know it, I'm tutoring five, 10 cousins every day. But with a background in software, I started writing this very simple software to give them practice. This is back in 2005, 2006. And so that was the first Khan Academy. I got the domain name, it was a family project. And then a friend suggested, well, I was showing this off at a dinner party.

12:29The host of the dinner party said, this is cool, but how are you scaling up your lessons? I said, it's hard, just getting logistically, getting on the phone. And I feel like I'm repeating a lot of the same lessons sometimes. And he suggested, why don't you record your lessons as videos and upload them onto YouTube for your family? And I said, that's a horrible idea. YouTube is for cats playing piano, but I gave it a shot anyway. And yeah, I mean, that kind of took on a life of its own. There were some people who offered some venture funding for it. It was tempting, but the second or third conversations aren't what I wanted to do.

13:02There was just all these people sending me letters from all over the world. Most of them didn't have a penny to their name. So I said, no, I, this feels like a, you know, I'll do the hedge fund thing. That's how I'll make my money, but let's, let's make this a nonprofit with a mission, free world-class education for anyone anywhere. But then a year later I said, maybe I won't make money through the hedge fund thing. Maybe I'll, you know, if I can just make enough money to have my family can have an upper middle class lifestyle, I'd be happy to devote myself to this thing. And that's when I quit my day job.

13:34And, you know, the first year was tough. But by 2010, we had our first real funding from Gates Foundation, Google, others to become a real organization. Yeah. You know, my daughter just graduated college and she is very mission driven and impact oriented. And so I want to ask you this. And she's kind of thinking through her next steps. What was it like to leave your cushy hedge fund job and start Khan Academy and especially structure it as a nonprofit to have the opportunity to, yeah, to be values driven? What was that like? And what's your advice to her and other, you know, other people her age?

14:11I get this question a lot from young people. And you can imagine young people are the most idealistic. They want to change the world in the best way. And I give them kind of what I think is reasonably practical advice of like, unless you have something that you're ready to go off to the races with, like you've already seen whether it's for profit or nonprofit, there's clear product market fit. You have a clear source of funding. Yeah, if you're Mark Zuckerberg in your early days of Facebook, run with that. But most people aren't. Most people have, especially if they're socially motivated, they're like, well, I just want to really help this issue.

14:44And maybe I start a nonprofit or I've created a prototype. And I was like, that's awesome. Keep working on it, but don't necessarily jump in with both feet yet because that first job really starts to open and close opportunities for you. I've seen this with other members of my family where they tried to go a little bit of a nontraditional route out of that first job. And if it doesn't work, it's very hard to switch back into a traditional, let's call it corporate track. So I know this might sound ironic coming from me, but look, I did follow. My first job was at Oracle. And then I worked at some venture-funded startups where I was able to get a salary.

15:21Then I go to business school. Then I work at a hedge fund. The hedge fund was a small hedge fund. It was my boss and me. But it was still, I was getting a good salary. And so I was able to save up some money. You know, some of those, in fact, the business skills that I learned at Oracle and the hedge fund and the MBA, I use all the time. I try to run Khan Academy as the best hybrid of a for-profit and non-profit. It is a non-profit, but we try to have all the best practices of being agile and innovative and being really conscientious with our resources the way a for-profit would be. We're going to take a short break.

16:01When we come back, how Sal became one of the first people to try out GPT-4 and how that revelation led to a huge pivot for Khan Academy. Stay with us.

16:29If you've spent any time building AI products or leading technical teams, you know this. Transformation doesn't fail because of ideas. It fails because teams can't move together. Enter Atlassian's Teamwork Collection. It has planning in Jira, documentation in Confluence, video updates in Loom, and now AI agents in Rovo, which connects the dots across your work so nothing gets lost. It's one AI-powered teamwork platform designed for how modern teams actually build. Learn more at Atlassian.com slash TeamChanger. That's A-T-L-A-S-S-I-A-N dot com slash teamchanger

17:17So I just finished reading your book, Brave New Words How AI Will Revolutionize Education and Why That's a Good Thing So let's switch to AI And I want to take us back to the summer of 2022 You got an email from Sam Altman and Greg Brockman Basically wanting to show you GPT-4 Tell us about that meeting and how that was an inflection point for Khan Academy. Yeah, when I got the email, we get probably 10 emails like this every day. And most of them would go nowhere. But, you know, Sam Altman and Greg Brockman were more, you know, I knew of them. I knew they were legit. And we got on a Google meet.

17:58And they demoed it for myself and our chief academic officer. and they put up an AP biology question and they said, Sal, what's the answer? And it was something to do with osmosis. I was like, it was like C, osmosis. And they're like, okay. And then the AI said, yeah, the answer is C. I was like, oh, this is interesting. I said, ask it to explain the answer. And it did. And then I said, ask it to explain why the other choices aren't correct. It did. And it did perfectly. And I started getting chills. And they said, would you like access? I was like, yes, please. So then you, I guess, packaged all that together in Conmigo, which is Khan Academy's AI tutor.

18:34What intrigued me the most is how you thought about some design principles as you put Conmigo together. So can you talk about some of these, like the Socratic questioning, as opposed to just giving you answers and the memory function, which I thought was really interesting. So yeah, talk about some of these design decisions. Yeah. Well, you know, you could imagine as soon as I had access, I was like, okay, this changes everything. We started having all the debates inside of Khan Academy that, frankly, the world is still having, which is, okay, this is cool, but what about hallucinations? What about math errors?

19:05What about safety? What about cheating? And the team was almost split at 50-50. Everyone thought it was cool. Half of the team, and I was probably more on that half, was just like, look, this is too important to not go all in. The other half was like, hold on a second. But what I told that half is like, look, 100%, your fears are valid. Let's write them all down and more. Let's brainstorm all the ways that this could go wrong. And let's turn these into features. Let's turn these into guardrails. And so cheating is an issue. Well, let's make sure at least our AI does not cheat. We're going to make sure that it's a Socratic tutor.

19:42It's going to ask questions, nudge kids in the right direction, but not give the answer. We're worried about kids doing shady things with the AI, unhealthy things. Let's give some oversight. Let's make sure it's transparent to parents and teachers where they can talk to the AI as if, you know, same way if you had a tutor, a human tutor, it's great for the parents and the teachers to talk to the tutors. Like, what have y 'all been talking about? And the tutor might say, well, I'm a little worried about Rana. She seems a little bit down. That's good. You know, if a student talked to a real human being and said, hey, you know, I'm really, I want to hurt myself, that real human being should say, well, here's a helpline.

20:18And by the way, I have to tell your parents this. Like, this is something I can't. Our AI should have that same property. It should have a moderation filter that actively notifies parents or teachers if something is off. On the hallucinations, well, what if we can anchor it on our content, which we have a lot of? And we saw that dramatically, it's almost eliminated hallucinations. on math errors. You know, now there's a lot of these reasoning models, which are essentially, we had to build it from scratch on the first version of Khan Migo, where the Khan Migo had to reason through the math, where it's like, what are possible things that the student might say?

20:54And how do they compare to the real answer? And we saw pretty dramatic improvements in math. And then we also recognized it wasn't perfect. It still could hallucinate. It still could make math errors. And so we have to educate the students and the teachers about it. And once again, this isn't a new phenomenon. Even using Google, you can get misinformation. It's not like everything on the internet was already like a completely vetted source. So it's really important to educate people that like, look, we think Conmigo is by far the most legitimate AI in terms of tutoring and not making hallucinations, but it's still plausible.

21:28So you need to validate things, you know, don't rest only on what the AI is saying, double check it. So that was our, and it continues to be our approach. And, you know, for the most part, I think it's worked well. Yeah. You know, I spent most of my career building artificial emotional intelligence. And yeah, learning is a very social and emotional experience, right? Students can be anxious or frustrated or excited or bored. And the best teacher is going to clue into that, into these sets of nonverbal signals and then adapt the learning experience accordingly. How does Canmigo do that? And does it do that?

22:05Like, do you agree that, you know, these AI tutors need to have emotional intelligence? And what does that look like? The more the better. But I won't make a claim that it's definitely not there yet. We're realizing that a great tutor doesn't just wait to be asked questions. A great tutor is proactive, motivates, guides. And so this next version that we're launching is a much more proactive Canmigo that acts as a concierge, acts as a guide, holds students a little bit more accountable, gives teachers a little bit more actionable information instead of just giving the raw data. So that's definitely where we're going.

22:42But to be truly emotionally intelligent, you have to be able to see facial expression. You have to hear a tone. The demos are already capable of that. I think for it to be truly productized and useful, we're still a few years away from that. But I also want to be clear, I still, in any world, I don't view this as a replacement for the real human in the room. You know, everything we're doing is assuming that you have that teacher, ideally. Now, we also build for, if you're a young woman in Afghanistan, you have no school, okay, how can we help you there? But ideally, you do have a teacher and this thing can act as a teaching assistant and tutor for you and the teacher.

23:21And then it hopefully lets the teacher do even more. So if the AI can sense that Rana is a little discouraged, you know, maybe eventually it can see your face. And it says, well, you know, just, she looks a little unsure. Why don't you go talk to Rana, Sal, teacher Sal? That's going to be way more powerful than if like AI tells Rana, like, I'm giving you a virtual hug, you know, like, no, go tell your teacher to say, no, go, go talk to Rana, look into her eyes and say, you believe in her. And I know today's hard. What's going on? Yeah. You know, so my company, Affectiva, we basically developed facial expression recognition technology.

23:58back in the day. But there's a privacy concern, right? Like, I'm curious, do you see a world where we would have the camera on during these learning sessions for the AI to clue into these nonverbals? Yeah, my general principle is, if there's, I mean, this is kind of how we thought about the first phase of Conmigo even, if there's something that can help people learn and get motivated and engaged, like, we shouldn't run away from it just because it also might introduce some tough issues. So, So yes, a camera looking at kids, it could get very creepy, very fast, but done safely, it could be very useful.

24:35So no, I imagine in the next five years, it's going to, I think you have to be very careful who has that data, what they're doing with it, what can be used, you know, training, et cetera. But if you can have a trusted party that that information is purely being used to make the learning experience better, then yeah, I think it's absolutely a legitimate thing to be doing. You could imagine putting a low-cost camera or several in a classroom already and seeing everyone's affect and then using that to tell the teacher, hey, you might want to try this, or you might want to arrange the chairs in this way, or go walk over to that student.