#242 Dylan Arena: The AI Education Revolution: How AI is Changing the Way We Learn

0:00AI should be used to augment human relationships. When people used to ask me in 2019 how I thought AI could be most helpful in education, I immediately talked about speech recognition and computer vision. Because I fried my arms as a coder, I was really aware of how much easier it can be to just say what you want to say instead of having to type it. Because to me, it was physically painful to type it, right? But for young kids, it's also much easier for them to express themselves verbally than to write. Now, there's a real richness in how young learners can express themselves with spoken language that they can't yet translate into written language.

0:32So there's a lot of good that can come from considerate applications of speech recognition to the classroom. Even if you think it's a bit overhyped, AI is suddenly everywhere from self-driving cars to molecular medicine to business efficiency. If it's not in your industry yet, it's coming fast. But AI needs a lot of speed and computing power. So how do you compete without costs spiraling out of control? time to upgrade to the next generation of the cloud, Oracle Cloud Infrastructure, or OCI. OCI is a blazing fast and secure platform for your infrastructure, database, application development, plus all your AI machine learning workloads.

1:17OCI costs 50 % less for compute and 80 % less for networking, so you're saving a pile of money. Thousands of businesses have already upgraded to OCI, including MGM Resorts, Specialized Bikes, and Fireworks AI. Right now, Oracle is offering to cut your current cloud bill in half if you move to OCI. This is for new U.S. customers with minimum financial commitment. See if your company qualifies for the special offer at oracle.com slash IonAI. That's IonAI all run together, E-Y-E-O-N-A-I. So go to oracle.com slash IonAI to see if your company qualifies for this special offer. I am excited about the idea of a freewheeling conversation instead of a Q &A.

2:14I'm excited to see where it takes us. By way of background, I studied symbolic systems and philosophy in undergrad at Stanford. Symbolic systems is introduced in a it's computer science, psychology, linguistics, and philosophy, and then mastered in philosophy. I was a software developer for a number of years at Apple, sorry at Oracle, but I fried my hands too much coding. So I have to use a weird keyboard. I use a lot of voice to text. And in fact, that was my first real engagement with machine learning with the technologies that are the you know, forerunners of today's Gen AI. Then, because I fried my arms and I realized software development was not a career path for me, I went back to grad school at Stanford and I got a master's in statistics and a PhD in learning sciences and technology design.

3:03I was focusing on game-based learning and next generation assessment and I was doing a lot of experimental research, so I wanted to get the master's in statistics so that I would understand the tools that I was using to test my hypotheses about learning. The goal of learning sciences broadly is to understand learning environments, learning contexts. And one sort of subfield within learning sciences is about creating productive learning experiences. That work carried me to Kidaptive, to co-founding Kidaptive, which was a startup that originally was focused on creating really engaging learning experiences for preschoolers on the iPad.

3:42The iPad had come out in 2010 and a bunch of parents were like, whoa, this interface is so intuitive that my two-year-old is like doing great stuff with it, you know? And so a whole bunch of junk food showed up on the iPad and the kids were doing all kinds of things and, you know, their attention was all over the place. And we thought, what if we could give them a really cool, enriching learning experience that was as engaging as a Pixar film, but also helped build skills. Not the expedient ABC123 skills that some people think is like, oh, I got to make sure my kid is doing this, but the broader, the rich, holistic set of skills that you want in a well-rounded early learning ecosystem.

4:25So we built what I thought were pretty cool experiences. It was called Leo's Pad. Parents loved it. Kids loved it. App stores loved it. But everybody wanted it to be free because this was like 2011, 2012 and, you know, apps are free, especially if you got a two-year-old and, you know, they, they pick up one piece of junk food and they play with it for a week and then they, they delete it and they go to some other thing. You don't want to pay for that. You just want, you just want to give them more of the free stuff, more of the free stuff, even though there's no nutritional value there. So we didn't, we didn't want to do any of the things that startups did to monetize back then um like advertising to toddlers or harvesting data in some way or like making their goldfish die and then they would pay 99 cents to like you know buy another revive their goldfish you remember that period where kids were spending like hundreds of dollars on their parents credit cards through perhaps through games um we didn't want to do any of that so we pivoted and we started doing support for other big operations that had created learning experiences that generated learning relevant data but didn't really know how to make meaning from those data because our technology platform allowed us to pull learning relevant data from a variety of contexts and make them all commensurable and contribute to a single holistic longitudinal learner profile.

5:39So we did that for another eight years. So 10 years total, two years trying to do that sort of App Store thing and then eight years doing what was essentially B2B SaaS, business to business software as a service. And also, in a sense, really high level professional services. We had a team of learning scientists and psychometricians and data scientists who would work with people who had created learning experiences to understand the data that were coming out of those learning experiences so that they could make the learning experiences personalized or generate rich analytics about the learning experiences, either for the product developers or for the parents or for the teachers or for the learners themselves, or generate great recommendations about what to do next.

6:22at some point i recognized that number one doing that kind of deep careful work with one partner and another partner and another partner leads to years-long relationships that are great and deeply collaborative but it's not the kind of hockey stick growth that a venture-backed silicon valley startup is supposed to achieve um so they i kept getting pressure to like create a plug and play thing that you could just pop into any learning environment and of course it'll give you the right data. And I said, but that'll never work because if you don't understand the context in which the data are being generated, you don't know what inferences you could make from the data.

6:56So instead, what we needed was to find a home with an organization that was so big that it had its own ecosystem so that the technology and the patterns that we had established for taking data from multiple different programs and making them all commensurable could bear fruit. And that led us, in short order, to McGraw Hill. Sean Ryan in the school business unit had already been thinking about the idea of a universal student record, a longitudinal profile of a learner that takes in data from a bunch of learning experiences across time and across contexts and distills the insights from all of those experiences to inform the educator and the learner about where they are and where they can go next.

7:39so that was a match made in heaven we came aboard at mcgraw hill i am now after a couple hops along the way in mcgraw hill the chief data science and ai officer but i'm still by disposition a learning scientist so i'm interested in data science and ai only insofar as they support learning and and um i mean how would you just generally backing up how do you see the promise of ai and education i know when we spoke you were very negative on the idea of a personal tutor and and you know students having a relationship with the screen and i'm negative on some aspects of it absolutely um i should say first i see tremendous potential for positive applications of generative AI and AI more broadly in education.

8:35The specific reservations I have are twofold. One, there's a set of people who have tried to replace humans with machines in teaching for over 100 years. Sidney Pressy's teaching machine was debuted in 1924, I want to say. And all along that time you can see repeated places where people thought oh this is the technological innovation that's finally gonna let us create something that's just as good as a teacher so we can just have the kids engage with that instead of with a human I think all of those efforts run the real risk of extracting from the educational process from the learning process the most crucial element of it which is the sociality of it the human connectedness when I was in grad school, I did a study in which participants are navigating a maze and at certain points in the maze, they get tips like, hey, turn left or don't go that way.

9:37And we experimentally manipulated whether they believed that the tips were coming from a human, the experimenter, or from a machine. In reality, they were always coming from the machine and they were always programmed to do the same for people in every condition. But the people who believed that the tips were coming from a person, learned the way out of the maze faster. And that tapped into a general effect that's been found in lots of contexts, which is if we think that something is social, in other words, that we're engaging with other people, we're more dialed in. We attend more, we care more, we put more effort in.

10:13Genitive AI can leverage this intentional stance, which are the philosophers called the tendency of the brain to interpret things in the world as if they're motivated by social actors. Because generally I can create situations in which it's quacking like a duck and it's walking like a duck and it's got feathers that look duckish. What I worry about there is the counterfeit nature of those duck-like things. when I believe that I'm engaging with a person, I invest in a certain way socially, psychologically, emotionally, that the system can't fully back up the way a person could. It's totally fine to engage with a system that behaves in person-like ways as long as I understand it's a tool and I can use it as such and treat it as such.

11:07Everything is great. We have seen already a number of high profile examples of individuals who have formed parasocial relationships with personified AI and had tragic outcomes from that. But in 2018, I want to say, there was some kid who developed an AI girlfriend relationship in which somehow or another she convinced him that it was a great idea for him to take a crossbow and try to break into the palace and assassinate Queen Elizabeth. He did not succeed. More recently and more tragically, a 14-year-old boy committed suicide. and it was later discovered that his AI pseudo girlfriend had been encouraging and reassuring about it.

11:55Those are just two examples that come to mind of this general problem of forming a relationship with something that you lean on as if it's a person, but it doesn't really have the support there to act as a person would. That said, again, that doesn't mean I'm against the idea of leveraging AI and tutoring. I think it's a fantastic idea if done right and done safely. I just want to make sure it is done right and safely. Does that make sense? Yeah. Yeah. I mean, that's interesting because I've been wanting to write something about that, about relationships with personified AI or whatever you, AI avatars.

12:36And I don't know if I've had this conversation with you, but, you know, humans invest things with human-like qualities. I mean, your Roomba that's riding around in your, you know, if it gets stuck and starts beeping, there's a certain sense of, you know, you feel sorry for it if you've left it. Yeah, exactly. Or the dogs that Sony had out for a long time and people formed relationships, particularly elderly people. And when they stopped supporting them, there was grief. I mean, people held funerals. It was a big deal. And, you know, I've been looking at Replica, which I don't know which companies were involved in the incidents you spoke about.

13:43And there's a filmmaker recently who has a film called Eternal You, and he explored this idea of after-death avatars, so you can speak to your loved ones after death. And there's a very dramatic example of a Korean mother who lost a child, and she's wearing virtual reality glasses and haptic gloves, and it's heart-wrenching. And the point of that is, one of the points of the documentary is, well, what happens? I mean, this can't be free. So are you going to charge a woman for access to her dead daughter? I mean, there's a clear ethical cloud hanging over that. And as you said, even if it's a tutor, if you develop a relationship, an emotional relationship with the tutor, should you really have to pay to maintain that relationship?

15:11Anyway, it does raise all those questions. That said, there is a role, as I was saying earlier, that I use, you know, ChatGPT's latest, OpenAI's latest model. I use perplexity, and I've learned a tremendous amount just asking questions about things I'm interested in and following rabbit holes. You know, there's a term you don't understand. You have that explained and it goes on and on. So how do you see it? So I agree. I mean, AI isn't there to replace teachers, but it can alleviate a lot of the drudgery of teaching. and it can also give students a way to study outside the classroom and, you know, to augment their learning.

16:18So tell me what McGraw, well, first of all, tell me how you feel AI should be used and then let's talk about what McGraw-Hill is doing. Sure. Sure. And I want to say, as you mentioned, there are these uses. And I have a colleague whom I respect immensely who has done research on the therapeutic uses of personified AI to help people who are in distress in various ways. There's definitely potential for good there. My only point is that there's also potential for really bad outcomes. So I think if you do that follow-up work, it'll be totally worthwhile because there's a lot of nuance and complexity in that issue.

16:53how I think AI should be used. I think AI and McGraw-Hill and I are aligned on this, should be used to augment human relationships. When people used to ask me in 2019 how I thought AI could be most helpful in education, I immediately talked about speech recognition and computer vision. Because I fried my arms as a coder, I was really aware of how much easier it can be to just say what you want to say instead of having to type it because to me it was physically painful to type it right but for young kids it's also much easier for them to express themselves verbally than to write now that doesn't mean they shouldn't write writing is an important skill that they need to develop but uh there's a real richness in how young learners can express themselves with spoken language that they can't yet translate into written language so there's a lot of good that can come from considerate applications of speech recognition to the classroom.

17:52Similarly with computer vision. You know, there's a lot of kids who do a lot of work in print in K5, even though there's a lot of focus on digital stuff these days, and it should be that way. That's good. But if I'm a teacher and I got a bunch of worksheets, wouldn't it be great if I could just put them in a stack and have a thing scan them? And then the machine learning algorithm and we'll identify six categories of major misconceptions that seem to be there. And one of those misconceptions is held by a plurality of the class. So, hey, maybe tomorrow focus on that. You know, just make sure you shore up that gap in kids understanding.

18:25Those are, in my mind, great examples of the use of AI to ease the logistic, administrative, and computational burden on the teacher and help the teacher direct their attention. Because that's the scarce resource in the classroom, right? the teacher's attention. So if you can help the teacher direct their attention where it's going to be most powerful, fantastic. You can put the teacher in a position to have that really rich conversation with their learners, then you're augmenting those human relationships that are at the core of learning. That's how I think AI should be used in education. And that's the kind of work we're doing at McGraw-Hill to use all kinds of AI, but Gen AI in particular in education.

19:03You mentioned we have two recently released Gen AI features within our existing products. One of them in the higher ed space is called AI Reader. And the idea there is kids have always had a bunch of stuff to read in college and kids have never really done all the reading. Apparently, I haven't been in college recently, but apparently it's gotten worse recently. It's gotten more like, oh my gosh, how could I possibly read this 300 page book and this 50 page chapter in econ and this text, whatever, in bio. So problem to be solved. College students have a tremendous reading load that they cannot consistently meet.

19:44And you have that problem that we all know. You're looking at a wall of text and you read a paragraph and then you realize you didn't get it. You're like rereading like two or three times and it's just not quite clicking for you. So to address that problem, we talked to university professors and instructors and we worked with them to come up with this idea. Like, what if you could just highlight that text and say, could you just explain this a different way? That idea was the core of what has become AI Reader. And the goal there, again, is not technology for technology's sake. It's there's a learning experience we want you to have.

20:15We want you to wrestle with text and build meaning in your own mind from that text. Just like you said, you use perplexity to be like, wait, explain this to me differently. You use these tools to give it to you in a different way to make it a little more accessible and line up with your existing knowledge structures so you can build and add to your knowledge structures. That's the basis of the cognitive theory of constructivism, which is a theory about learning. We learn by constructing knowledge in our own heads. It's not, there's knowledge here and I pour it here and now it's here. It's like, no, no, no.

20:46You're showing them a model, but they have to build it for themselves. We have to construct it in our own heads. AI Reader is a tool to help students construct meaning in their own heads by giving them more avenues in to a body of text. Quiz me on this. Explain it to me like I'm a kid. Simplify it. You know, all of these things. In 612, we had a different learning challenge that we were trying to solve, and AI was a way to help solve it. Kids in middle and high school are often engaging in digital experiences in which they're required to write short, free-form responses. Now, just like I talked earlier about there being a stack of worksheets, and a teacher could plausibly read through them all, but not realistically.

21:27if I've got 30 kids in each of five sections doing this, you know, five times a class period, that's a lot of, you know, 200 word little blurbs for me to read through and try to provide meaningful feedback on. And sometimes that's not even the main goal of the assignment. It's not primarily a writing assignment. It's just an assignment that has a writing component in it. If I'm walking the classroom like teachers often do, and I look over a kid's shoulder to see what they're working on, and I read that they've written a little blurb, I might offer them feedback right then. But it's opportunistic and sporadic.

22:01So we wanted to give teachers a way to help provide that kind of feedback to all the students if they want it. That's the feature that we call writing assistant. And we're currently in beta in two of our programs with writing assistant. And what it does is simply if I'm a student and I'm writing a short form response, I can click a button that says, give me feedback on my draft. And I engage in a couple of conversational turns with a tool that will either, if I don't know how to get started, it'll give me like a little, you know, push out of the gate. Like, hey, just try, you know, the prompt is asking you to do this.

22:35You've read this story. Now it wants you to respond to this particular question. What do you think? And then the kid will write some kind of draft. And then the program will just meet you where you are. There's a rubric behind the scenes, right, to evaluate the quality in the draft and wherever it is, the program's goal is just to try to get you to the next one up. Again, in a couple of conversations, you're not going to spend half an hour like crafting and rewrite. It's just a quick touch to do that little bit of support that might help you improve your writing in the moment. In each of those two cases, AI reader and writing assistant, there's a clear learning experience that we want to drive and there's a way that we can leverage AI to support yeah um it was something uh the the ai reader it it occurred me uh and i've wondered about this i mean uh you know i do interact a lot with with these chatbots for my own education uh and i i for example i watch a lot of youtube videos um uh i watch you know series on on tv and and what amazes me is that i'll come to something two years after i know i watched it and i don't remember any of it and uh you know there's there's and i don't know the science but there is something about passive uh engagement and active engagement that that affects the learning process and with reading i'm guilty of this you know i'll get a somebody will send me a report and i just loaded into chat gbt and ask for a summary uh and it's very use useful for short-term tasks because okay i understand what the report's about i see the main features i can respond to it but uh you know a week later i won't remember the report or the summary or anything so how do you address that.

24:57And our students, do you see that as a growing problem? You said that they're reading less and less, but maybe that's one of the reasons is they're just reading summaries that are AI generated and it just doesn't penetrate to long-term memory. Our sponsor for this episode is Xtreme Networks, the company radically improving customer experiences with AI-powered automation for networking. Extreme is driving the convergence of AI networking and security to transform the way businesses connect and protect their networks to deliver faster performance, stronger security, and a seamless user experience.

25:52Visit extremenetworks.com to learn more. That's extremenetworks, all run together, dot com to learn more. There are certainly learning risks of anything that makes the learning task easier. Learning, in a sense, must be effortful. I talked earlier about the constructivism theory of learning. building a new knowledge structure is tough taking in new knowledge and incorporating it into existing understandings of the world is is work and our brains don't like to do work so we try to offload that work and we have forever because it's calorically efficient to avoid work when we can and brains are really calorically expensive the risks of not taking things in fully if they're just you know cliff notes or quick summaries uh are real we got to weigh those risks against the risks of not taking an excited to the reading at all and we also have to

27:03engage our learners in the metacognitive task of of owning their own learning and their own learning goals there's a trope you know is this going to be on the test right why do kids ask that. They ask that because they need to, in some sense, manage their intake. You know, like if you're telling me a bunch of interesting stuff because you're excited about history, but it's not actually on the test, I'm going to tune out. I'm holding myself accountable to learn the stuff that I know I'm going to be measured on. And in some sense, we roll our eyes because we're like, no, but we want kids to be hungry about learning everything.

27:34But also, you know, they're rational actors, right? So they're going to make sure they're focused on what you need to be focused on. If your goal in taking a report that somebody sent and feeding in a chat to get a quick summary is simply to have enough of an understanding of the report to be able to respond coherently, then you've achieved that goal. And to your point, you didn't encode it deeply or elaborately enough for that to stay years later or even months later. Similarly, the TV show that I watched a couple of seasons ago, I remember being good, but I'm not exactly sure what happened. If a learner is engaged in a psych course or an econ course or a finance course, and they really want to learn this concept, they know they're going to be tested on it, and it's for their major, so they know it's going to be instrumentally useful for them going forward.

28:22We want to help them encode that information. The more richly we can help them encode it, the more likely they will be to remember it later. so there is absolutely a balance to be struck you're right and one of my biggest fears as a learning scientist supporting the implementation of ai in various areas is how do we avoid this leading to a diminishment of human capacity the two examples of that that haunt me are the fact that when i was a kid i had memorized all my friends phone numbers and my family's phone numbers everybody's phone numbers and i don't have anybody's phone numbers to remember us anymore i I know my own and my wife's.

28:58I don't even know my son's one. Because there's now a tool, and I have done the sort of cognitive offloading of that task to that tool. And brains are great at not remembering stuff we don't need to. So I don't even remember that anymore. I don't remember it. It's in the tool. Similarly, we as a society have gotten less good at wayfinding because GPS is now ubiquitous. So we're not as good at dealing with maps or at remembering roots. We just plug in a destination and we turn when it tells us to turn. Those are both areas in which we had a capability that has diminished. And we could argue about whether on balance we're better or worse for it.

29:36Where we enter the domain of education that we got to be really mindful of whether we're giving students a tool that will increase their capabilities or decrease them in ways that we think are damaging. The introduction of the calculator was an opportunity for society to grapple with this. Calculators became ubiquitous and efficient enough and small enough in the 70s to begin introducing the classrooms. And there was a big furor over whether this was going to be the end of mathematics education as we knew it. We ultimately landed in a good place. We do not let students use calculators when we're trying to teach them arithmetic because they have to understand the concepts and they have to develop a fluency with those concepts.

30:13We do allow students to use calculators in advanced trigonometry, calculus, those kinds of courses. And in that case, we're actually enriching their learning because there are things that you can explore in terms of the behavior of functions, for instance, with a graphing calculator that you could not feasibly have done in the 1950s where you're plotting all the points by hand. So we're actually allowing kids to engage more deeply with rich conceptual material because of the affordances of this tool. So are calculators good or bad for math is not a reasonable question to ask. It's how calculators be good and bad for math.

30:49We're now in a similar situation with various applications of Gen AI. The most obvious one is these tools are great at writing. But does that mean we should let kids just use them instead of learning how to write themselves? And what about the deep relation, the intrinsic relationship between writing and thinking? I don't know who it was. Maybe it was a bit consigned. Someone said, how can I know what I think till I see what I say? And that's a great encapsulation of this relationship between our language and our thought. If we simply let kids go boop, boop, boop, boop, boop, and then there's an essay that comes out, they haven't done that work.

31:25They haven't grappled with those ideas. And they definitely have not learned as much as they would have if they had persisted through all those drafts and revisions and feedback and gotten it to a point where they could express their thoughts in that way themselves. That would be bad. On the other hand, there are kids with language challenges or kids who really have rich ideas but aren't great at expressing them yet. How can we use the technology to support those use cases without falling onto the side where we're actually diminishing an essential capacity for thinking? That's the challenge that we got to meet every time we consider a new application of AI, Gen.AI or other for education.