The future of learning

0:00This is Stanford Engineering's The Future of Everything, and I'm your host, Russ Altman. Since we started this show eight years ago, it's become an archive of amazing and impactful work by my Stanford colleagues. Research is not something that just happens in the lab, and as you'll hear on this show, the research at Stanford can impact areas like health, technology, law, and business, and many other topics that can affect everyday life. We hope you'll tune in to learn more about how research has the potential to help your life and to help the lives of people you care about in your family and your community.

0:32Candace Thille:In that computer interface, we can observe the learner. And we can observe the learner, just as you said, like Netflix does, like Amazon does. They're trying to do it to understand you better as a consumer. Yes, very different motives. But we're trying to do it to understand you better, you individually, as a human learner.

0:59This is Stanford Engineering's The Future of Everything, and I'm your host, Russ Altman. You know, if you've listened to this show more than once, or even just once, and you liked it, go ahead and follow us. That'll help make sure that you hear about every episode and never miss anything about the future of everything. Today, Candice Till will tell us that research about learning and practice of teaching and learning need to be tightly linked. If they're separate, you might not get the best results, but if you can integrate them, then you will get a cycle of improved teaching and improved learning.

1:30It's the future of learning. Today, we're continuing our feature, The Future in a Minute, with Candice. At the end of our interview, I will ask her a few rapid questions, and she will give us beautiful short answers. Before we get started, please remember to hit follow. If you listen to us once, if you've listened to us twice, no matter how many times you probably are, I hope, are enjoying us, go ahead and follow. We'll keep you up to date.

2:00So we all know that learning is important. It's important for productivity and your functioning in the world. But you know what? Learning also makes people happy. Researchers in learning learn a lot about how to do the best teaching and how to facilitate learning, especially in adults. However, there's a problem. Sometimes the research that is done in the academic controlled setting doesn't always transfer well into the real world teaching and learning schools. Well, it turns out that there are ways now where we can integrate the research about learning with the practice of learning to have a tighter feedback loop and to increase the relevance of the research to the actual practice of learning.

2:44This means we have researchers who are producing better and more robust results, and we're having learners who are doing better, learning faster and better. We'll hear about that from Candice Till, who's a professor of education at Stanford University. She's an expert in media and technologies in the context of learning, and she will tell us how learning knowledge and research has evolved and how AI, at the end, is impacting her ability to build effective learning environments. Candice, to start out, why did you decide to focus your career and your research on the science of learning?

3:22Candace Thille:Oh, because learning and the capacity to learn and having agency in your learning has pretty much every kind of outcome positive correlation that you can imagine. And I wanted people everywhere to be able to learn what they wanted to learn to build the knowledge and skills they needed to do what they are trying to do in the world. Great, great. And so kind of a little bit out of order, can you tell me what is the science of learning and what are the current big challenges to the field? Oh, the science of learning is, first off, it's a really nascent science, unlike physics. It's not like we know all of this stuff and it's just a matter of getting everyone to know what we know.

4:07Candace Thille:It is, so the science of learning is studying actually how people learn. that moment of where your knowledge and capability changes from one moment to the next. And it's an interdisciplinary field. People come at the science of learning from obviously from education, but also psychology, communication, computer science is huge, neuroscience, all of those are involved in really understanding the processes of human learning. And you work at a school of education. Is the science of learning kind of integrated into the curriculum for educators? Not for all educators. At our school at Stanford, we have a particular one of our graduate programs, as well as a master's program, is called Learning Science, Technology and Design.

5:01Candace Thille:And that is a real focus on both the science of human learning and also how you use emerging technologies and a design science to take what we know about human learning and use it to design effective educational experiences. And so I hope this is a fair question, but when you look at how teaching and learning happens in the world, and I think your expertise is adult learners, and I think that includes college. We're going to call college students adults for the purposes of this discussion. For the purposes of this discussion. And they're over age 18. Great. When you look at the way we instruct them, are we making the best use of our knowledge of learning science, or do you see a little gap there?

5:44Candace Thille:Oh, there's a gap there. And everyone will always talk about the gap that you're pointing at, which is we know a lot about how people learn, but very little of what we know actually influences practice. But I want to talk about this as a bi-directional gap. There are a lot of people, and I know most of us have experienced them, that are amazing teachers. And some of them are formal teachers. Some of them are just people who help us learn things. And they actually have a lot of wisdom and knowledge about how to support someone to learn something. And the other gap is all of that knowledge, all of that skill doesn't find its way back into our science.

6:26Candace Thille:So I call it a bi-directional gap. And my life's work is really shifting that relationship between the science of human learning and teaching practice, whether that practice is done by a human or by a machine. Great. And thank you for that last comment, because I wanted to get to the fact that you have embraced technology as both to do this research in your field, but also as a potential way to deliver teaching and learning, as you just described. And that's not obvious to me. For example, I could imagine a learning specialist saying technology is not really part of this. We're talking about humans inspiring other humans.

7:08So how do you think about technology for education and learning?

7:12Candace Thille:I think about technology as providing infrastructure for human learning in the sense of, if you think about it, when you're either a learner or when you're a teacher, you're making minute by minute decisions. and your decisions are trying to answer the question, if I'm the teacher, for this learner who's in this particular state that they're in and we're trying to help them to get here, what's the most effective thing I can do for that learner to help them move from where they are to where they're trying to get to? Now, in order to, we call those instructional differentiation decisions. Oh, good, that sounds very technical.

7:53Candace Thille:That does not really science-y? Now, in order to make a good decision, really, we would need to consider features about that human learner. We'd also need to consider features about the thing they're trying to learn. And really importantly, features about the context in which that learning is occurring or in which that knowledge will be applied. Now, if you think about it, trying to think about what do I know about the learner? What do I know about what they're trying to learn? What do I know about the context we're in right now? And putting all of those variables together and thinking about, given that, what's the best thing for me to help this learner?

8:32Candace Thille:That is way beyond our human capacity for managing teachers. But it's not beyond our algorithms. So I'm not talking about using algorithms to replace teachers, but to support that process of human decision making for both the learners and for the teachers and for people designing learning assets and for, you know, my favorite group, learning researchers. Great. Great. So thank you for that. And that allows me then to ask you about some of your historical work, which the comments you just made to help us understand why you would do this. So one thing that you did, I think even before you came to Stanford was called, I think, the Open Learning Initiative.

9:14So now that'll put some meat on the bones. Tell me, what was that? How did it work? Why did it exist?

9:21Candace Thille:Okay. Thanks. The Open Learning Initiative started in 2002. So from education and AI terms, a long time ago. Definitely pre-AI era. Well, pre-generative AI. We were actually using AI even then, and I'll talk about that in a minute. So what happened was the William and Flora Hewlett Foundation had just funded OpenCourseWare, which was MIT's project to make all of their course materials openly available online. And they were going around asking other universities to make OpenCourseWares. At Carnegie Mellon, we were really happy about the open idea. But the challenge was to really support someone to learn something.

10:06Candace Thille:You need more than just course material. You need instruction. So building on the decades of research we had done in cognitive tutoring and other learning sciences, our pitch was to create a smaller number of courses, but that they were designed to support humans to learn using what we knew about the science of learning. And then I started just collecting the data from the learner actions because I wanted to know, was there evidence that what we were designing was actually helping people to learn something? And then about a year after we started the Open Learning Initiative, we got one of the big NSF Science of Learning grants, where then the OLI courseware became a research environment where we were collecting those data, not just to understand where the things we were designing, helping people learn, but also to then do experimentation in different strategies to see what actually helped learners.

11:08So can you paint me a picture of what a learner experienced as part of the online learning initiative? Like what was their learning experience? I'm taking that they weren't just sitting in a classroom with a sage on the stage lecturing at them.

11:21Candace Thille:No, no. One of the things we knew back then is that learning is, as one of our articles said, learning is not a spectator sport. Right, right. That what you need to do to learn is you learn by the process of trying something and then getting feedback. And that feedback can be feedback from, oh, you got hit in the head. That's a kind of feedback by the ball when you threw it wrong or something. But another kind of feedback is timely and targeted feedback that can understand the move you made and make an inference about why you made that move and then give you explicit feedback about, oh, well, it looks like you did that because you were thinking this.

12:07Candace Thille:Part of that was right, but this is where you might have gone off track. And then most importantly, the opportunity to practice again once you've gotten the feedback. So the OLI courseware had small amounts of text, no video at that point in time because we're talking 2002, no lectures, but small amounts of text and then lots of interactive activities where the students could try something, get feedback, get hints if they needed, if they weren't sure what to do. Was it a mostly, so it was an online experience? It was a computational experience? Completely. It was completely online. Completely online.

12:50And did it also involve any other students or was it kind of a one-on-one tutoring type setup?

12:56Candace Thille:This was a one-on-one tutoring type setup. And that was in line with the goal of the Hewlett Foundation at the time. Gotcha. Because they were trying to make, quote unquote, a high quality post-secondary education available to anyone in the world that had a good internet connection. Great. So how did OLI or the Online Learning Initiative, how did that evolve over time? I know it did. Oh, yeah, it totally evolved over time. So I should say the way that we developed the courseware in OLI was not just I thought, oh, here's a great course. Let's build it. A key ingredient of the OLI project was collaboration, interdisciplinary collaboration.

13:40Candace Thille:So we brought people that were experts in their field together with learning researchers, together with UX designers, user experience, together with software engineers, and together collectively designing the environment. The other thing that was really kind of cool about it is faculty from different disciplines, like we were building courses in biology and in chemistry and in statistics and in logic and in economics, all these different faculty coming together, talking with each other about the challenges they were having and the solutions they were coming up with. So it started this incredible interdisciplinary dialogue across different fields of what we think of different fields of higher education.

14:30Candace Thille:But everybody focused on how do we support the learners to get these concepts? Were the outcomes good? The outcomes were great, actually. Probably the most the study we were constantly doing studies, but probably the most famous study we did was called the accelerated learning study. And that was, again, my program officer at the Hewlett Foundation, Mike Smith, said, Candice, you're telling me all this stuff about using what we know about human learning to design environments. How do you know it's working? How do you know it's better? And I'm like, you know, I tried to explain how educational research is challenging.

15:11Candace Thille:It's hard to get good effect sizes, et cetera. And he's like, okay, well, show me they can learn faster. I said, faster? He said, yeah. So we did a study where we took introductory statistics course students and randomly assigned them to condition. In the traditional condition, they took the traditional statistics course, which is 15 weeks in length and four class meetings a week. In the OLI condition, they completed the same course, but in eight weeks with two class meetings a week. So, you know, so half the last time, a quarter of the instructor contact hours. And then we used measures of both, they all took the same midterms and final.

16:04Candace Thille:So we had those measures, but we also used another measure developed by a different research group measuring knowledge of introductory statistics as pre-test and post-test. We also sampled both conditions during the study to make sure that the people in the OLI condition weren't just spending a lot more time in those eight weeks on statistics per day or per week. And the results were that the students in the OLI condition on the midterms and finals performed as well or better than the students in the traditional condition. And they performed 18 points better than the students in the traditional condition on the external measure of statistics knowledge.

16:49Candace Thille:A funny thing about that I want to share is that the faculty member that was doing the two meetings a week with the students, He, before the study, he's like, Candice, this isn't going, I'll do it, but it's not going to work. And it's not going to work because my students learn in relationship with me. And if I'm only seeing them for eight weeks and for twice a week, I'll never get that, that won't happen. And after the study, he said to me, this is the best experience I've had teaching statistics in my 15 years of teaching statistics. And I asked him why. And he said, well, for the first time coming into class, the students knew what they knew and what they didn't know.

17:38Candace Thille:I knew what the students knew and what they didn't know. But probably most importantly, the students knew that I knew what they knew and didn't know. So we could spend that, you know, 50 minutes we had together focused on authentically, this is where you are. This is how I can support you. Really great story. And so that is very important for everyone to know because it shows that really this is incredibly important to get out into the world. Now, I know you've done many things since. But one of the things that's fun to hear about is you did a brief stop or not so brief stop in the tech industry at Amazon.

18:20How did that happen? How was that? How does it inform your work today?

18:25Candace Thille:Okay. So I went from Carnegie Mellon to Stanford and I was at Stanford for about four or five years continuing this work. And I got a call to come up and do some consulting for Amazon because they had a particular workplace learning problem they wanted to solve. So I came up and consulted to them. And I left and thought, okay, they're not going to do what I said, so that's fine. And I sent it to the VP who had invited me. And she said, well, if you want to do something different at Amazon, you write this thing called a PRFAQ. And that's a press release of when the intervention that you've designed is released into the world, what will be written about it.

19:07Candace Thille:And then the FAQ is all of these different questions. The first being, who's the customer? What's the customer problem? And then the rest of the questions are, how do you think your solution is going to actually, what's your evidence? So I wrote one of those about my proposal and they decided they wanted to do it. But they said, but this isn't a consulting gig. You need to come up here to Seattle and lead this team. So what my team did was our goal was to innovate and scale workplace learning for Amazon. So the idea was, I mean, Amazon at the time had 1.6 million employees and a need to rapidly upskill people in complex capabilities.

19:57Candace Thille:And they recognized that the old mechanisms for doing that just weren't cutting it. And they also recognized that they were looking at their growth trajectory and didn't think that higher education was going to be able to supply them with enough talent to meet that growth trajectory. So they wanted to develop an infrastructure and a system for rapidly developing complex capability. And that perfectly aligned with my research. And also, as you can imagine, I'd spent the last 15 years trying to build the technical infrastructure using NSF funding and private foundation funding and graduate students.

20:38Candace Thille:And here was a well-resourced company that said, we want you to build what you want to build. So come here and we will give you the resources to do it. My hope was, and I said this to them, that my real passion is for not-for-profit higher education. So my hope is once we build it and demonstrate it works, that you will give it away. they chose not to do that. But, you know, that was my hope. But you still have the knowledge and the takeaways from that experience. Absolutely. So then that's why I took a leave of absence from Stanford. And I actually ended up having to resign from Stanford because only two year leave of absence.

21:24Candace Thille:But then fortunately, my colleagues and Stanford wanted me back. So I did get rehired. And so now I'm building an extending what we built for the public good. This is the Future of Everything with Russ Altman. We'll have more with Candice Till next.

21:56Welcome back to the Future of Everything. I'm Russ Altman. I'm speaking with Candice Till from Stanford University. In the last segment, we learned about learning science, how to think about learning, and what are some of the challenges in making sure that learning environments for learners are effective and work in the real world. In this segment, we're going to talk a little bit more about this relationship between research and practice and how to optimize it. We're also going to talk about how AI is creating great new opportunities for much richer environments for people to learn in. Candice, in our conversation, you made a reference to the relationship between research on learning and then turning it into practice.

22:36And I feel like there's still there's still some conversation to have about that. So what is your current way to think about? I don't know if it's attention, but it's certainly two different activities. Learning how students, learning how people learn and then translate that, translating that into effective teaching and learning strategies. So where are we with all that? Okay.

22:55Candace Thille:So I'm going to first push on that assumption that that's a linear process. That's what I'm trying to change. That our notion is we researchers do our research, we create our causal claims, and then we throw it over to practitioners and say, implement this and people will learn better. And when practitioners then take our research and try and implement it and it doesn't work. And then we say, oh, that's because you didn't implement it with fidelity. You didn't do it exactly the way I designed it. And then practitioners reasonably say, nah, you designed it with all these artificial constraints that don't exist in the real world.

23:41Candace Thille:So that causal chain that you created, that exists for your system, it doesn't exist in the real world. Yes. So what we need is, rather than this linear model, to engage practitioners as collaborative researchers. And what I mean by that is everybody who tries to teach someone something, you can think about that as you're testing a hypothesis. You think, okay, here's my learner. I think if I do this, it will help them learn what I'm hoping they'll learn. And if you think about that as generating an observation, and then we have an infrastructure that we can collect all of those observations, then our expertise about teaching and learning can be not just from these scientifically controlled studies, not just from our own personal intuition or observation, but from being able to see the observations essentially that everybody's making, which then allows us also to include a lot more diversity and a lot more voices in our science.

24:49It sounds to me both totally reasonable, what you just said, and a lot more complicated to orchestrate.