Best of: The future of language learning

0:00Michael Frank:Hey everyone, it's Russ Altman, your host of The Future of Everything. You know, as kids go back to school, we are re-releasing my conversation with Michael Frank on the future of language learning. Michael studies the differences between how AI learns language and how children learn language, and it's a world of difference. AI takes tons of data, data centers, and it's extremely expensive and it takes a long time and you have to expose it to tons of language. Kids listen, watch, and in very short order, have a very good understanding and mastery of language. What's the gap? Why is there such a big difference?

0:36Michael Frank:Michael's goal is to build a better scientific understanding of language learning, both to improve kids' learning of language and to build better AI systems that can learn things more quickly. So, no matter why you're interested in language acquisition, from kids to AI systems, this one is probably worth a re-listen. Thanks. Before I get started, please remember to follow the podcast if you aren't doing so already. Hit the bell icon if you're listening on Spotify. This ensures that you'll get alerted to all the new episodes and will never miss the future of anything.

1:14Michael Frank:You know, we've all seen the miracle of how children learn languages. As babies, they start out, they babble, they do random things, random noises, but they Then they start to acquire a few words. They begin to put them together into sentences. And before you know it, they're fluid. They can speak. They can interact. Not only that, they can understand cultural cues, social cues. They even can infer unstated things that are implied by the things that people say to them. Well, Michael Frank is a professor of biology at Stanford University and an expert on how children acquire language. He studies not only how they learn the words, but he also studies how they use language to understand the world, the social interactions in the world, and basically how the world works.

2:05Michael Frank:So, Mike, you study children and how they learn language. This is particularly interesting these days when we're seeing AI systems also learning language. Do they have anything to do with one another? Absolutely, Russ. Yeah, so this is fascinating. It's an amazing moment to be in cognitive science of language because we're seeing for the first time other agents that are at least able to produce grammatical sentences of English. And sometimes they even seem to make sense and really be meaningful in some way. So there's been a long tradition of trying to use artificial intelligence and the precursors to the current systems to try to understand kids' learning with the idea that these could be real scientific models that we could study in detail.

2:49and a lot of that history was really cool but the models we were looking at were not very capable so suddenly when something like chat gpt explodes onto the scene all of a sudden we're thinking wow we could really learn a lot from what it means to just kind of learn english learn enough natural language from observing enough data so that's of course there's still big gaps yeah so

3:09Michael Frank:you know i have to confess this right at the very start i have a 16 month old granddaughter and so i am watching her in real time learn language. And I also know a little bit about large language models. And one of the things I know, and I think everybody knows, is they've been exposed to a ton of data. And I watched my granddaughter and the way she's learning language is not because of being exposed to a ton of data, although it's an extremely rich environment that she's in. So tell me how you as a kind of professional in this area, how do you try to learn things when the two learning methods seem to be very different.

3:46Well, so it creates a really exciting scientific question. We think of this as the data gap between human learners and artificial intelligence. And the big question is, what is responsible for that data gap? So what's the first of all, you want to quantify the size of the gap. So something like GPT-3, which is now several generations back, was exposed to 500 billion words. and you're 16 month old, has probably heard 15 million words. Wow. Yeah, so it's more data than you might think. Probably mostly from me, I'm sorry to say, but that's a whole different. So yeah, a million words a month is a lot of words actually.

4:26Does that include repeat words? Oh yeah, so that's being immersed in this environment where people are talking to you or around you and that's counting kind of everything. Probably the amount that's actually something you could really learn from as a baby as opposed to your parents talking about the credit card bill is pretty different. But then you mentioned kind of what are the big differences, right? The language that your granddaughter is hearing is social. It comes in this kind of complex context where there's something happening, something being communicated about. It's grounded in the world around her.

5:05It's got all kinds of extra multimodal, meaning like lots of different sensory modalities information. She's got vision. She's got touch alongside the sound. So there's lots more richness to what she's experiencing in those millions of words than what ChatGPT gets from its many hundreds of billions.

5:23Michael Frank:Great. OK, so I want to step back because, you know, you were doing this way before ChatGPT came in. And I want to make sure we look at some of the things you're doing. One of your approaches involves lots of data, you know, big data approaches. And so could you tell us about some of the resources that you've set up and how you use them to learn about language acquisition? Absolutely. So despite the fact that, you know, language learning you're talking about is a big data problem, a lot of the earlier approaches to studying language were very bespoke. It was studying one child, maybe somebody keeping a diary or doing an experimental study of 15 kids at the nursery school.

6:00And one of the things we've learned is that that kind of small data approach doesn't really work because of the diversity of outcomes for kids even in one culture, and then the vast diversity of the languages and cultures that kids are learning in around the world. So what I've tried to do is really build up databases that give us kind of some rich quantitative information about what language learning looks like around the world. One of those, our flagship is called Word Bank, where we've got maybe close to 100 ,000 kids worth of data from something like 50 languages or dialects around the world.

6:35And what we've got actually is what parents say about them. So we've got parents checking off my child knows ball and dog and cup, but not alligator. And so for each child, we've got this kind of holistic assessment by the parents, sometimes many times, sometimes just one time of what they know, what they're saying and what the parent thinks they understand. And that allows us to look at variation in all sorts of really interesting ways.

7:02Michael Frank:And the parents are good reporters in your experience. Like there's not a lot of, I don't know, projection of their hopes and dreams of the child onto your data set. Well, so this is part of the craft of doing good research in this area. You don't say, hey, how brilliant is your child? Check, very brilliant, exceedingly brilliant, or off the charts and also adorable.

7:26What we do is we ask questions about observed behaviors that are happening right now, not retrospectively. And we do assume there are some biases involved here. So there are certain things we have to take with a grain of salt. But in general, people don't have a specific bias that their child does or doesn't say alligator. You know, they either remember that happening or they don't. And so maybe any individual word is a little noisy. Maybe we're not sure if they actually said, you know, cat or not. But averaging over thousands of kids, we can actually get quite a detailed measure and we can learn, you know, unsurprisingly, that cat is learned before alligator.

8:04Michael Frank:And now you mentioned, I think you mentioned that it's almost 100 ,000 kids. And I think you said multiple languages. So are there differences? Are there easy languages and hard languages to learn? Or for a kid, is it all just like, bring it on. I'm going to learn whatever we're talking here. You know, the amazing thing, I think this is really a discovery, you know, not just by me, but by language acquisition in general, is that kids really can learn, you know, the whole variety of human languages. And that's for an interesting reason. It's because languages are evolved to be learned by kids. So a language can't survive as a language if it's not learnable.

8:38So, but, you know, largely the thing that pops out of our research is how consistent the process of learning is across very different languages. Now, of course, the details are different. The words are different. The grammar is different. But the general factors that influence language learning around the globe appear to be quite similar.

8:56Michael Frank:Very interesting. Now, I don't know if this is science or just something I read in People magazine. But is there a time when the babbling, like, so my 16-month-old granddaughter is babbling shit. There are a few words, but it's mostly about, you know, blah, blah, blah, blah. Do those diverge quickly when this child is destined to learn different languages? Or is the babbling all kind of sound the same? And like, when do they start sounding like the language that they're ultimately going to speak? Yeah, they actually really do kind of start along that path. So my father-in-law is a speech pathologist, and he loves to say that what babies are doing is playing the instrument, right?

9:37The way when you pick up a guitar, you could just kind of go strum, strum, strum, strum, strum, and you get this kind of the open strings. You don't get music, but you get something that sounds guitar-like. And babies are kind of exploring their vocal instrument or for sign language acquiring babies, their manual instrument in kind of the same way. So you see these language-like elements start to emerge as they figure out, oh, that works. Oh, that sounds kind of like what I'm hearing or looks kind of like what I'm seeing in the case of the signs.

10:04Michael Frank:Great. So that actually makes really good sense. And I love that analogy to instruments. So in addition to WordBank, I'm wondering, you made a reference to the fact that when the human kids are learning language, it's a multimodal experience. They're seeing things. They're hearing things. They're learning. And I know you've studied this, the social cues. Do you have any ability to collect that kind of data? I could imagine it would both be useful for you. And it might give the hint to the AI guys about how they might be able to be more efficient in their learning. So what's the status of our ability to study these multimodal elements of language learning?

10:42Well, creating those kinds of data sets is one of the biggest challenges right now for us developmentalists. So actually about 10 years ago, we started creating a resource that's become very useful. It's called SACAM. So the idea was that we had these for the first time little cameras that we'd put on kids' heads. and we tried it a bunch in the lab and it was fun. And then a couple of parents and I got together. I actually didn't have a kid at that time. It's how long ago it was. And they said, we're going to try this two hours a week, you know, every week from six months until, you know, the kid stops wearing it sometime around two and a half, three.

11:18And we created this corpus of videos from the child's own perspective. And critically, the videos, these parents were developmental psychologists and committed to their craft and were willing to let these sometimes somewhat unflattering videos out in not in public, but in a research repository for other researchers to use. And as a result, there's really exciting research being done with these videos.

11:43Michael Frank:That is really exciting. So have we gotten to the point where we can like get some preliminary results from that? Or is it still very much in the infrastructure building phase? So some super cool results are actually coming out really soon doing machine learning on one kids experience from this corpus. So you can really get a deep dive into what you can learn from just a little bit of one child's life. And I can see that it was, it must've been like tracking their head movements. So you can really see where they're focusing. But you know, this was still by machine learning standards, pretty small data.

12:11So we collected 400 hours and that's like about eight or 10 % of one child's waking hours for one year. And so of course, you know, these greedy AI folks. So one of my collaborators, wonderful guy named Dan Yamens came to me and said, you know, this was great. And I'm sure you, it was very hard for you to collect, but could you just do something like 10 times as much? So that's really what we're doing now. We've got this study in progress called the baby view study, which is essentially a baby GoPro. We, we, we hooked up a GoPro rig for very small kids and we're sending these things out, you know, all over California and all over the U S and having parents put them on their kids once or twice or even three times a week and gathering these data that really reflect what the child's experience is like.

12:58And so that's going to create, hopefully, you know, once it's done, an even more useful resource, both for scientists who want to characterize what do kids see, and also for AI folks who want to train models that are a bit more baby-like than chat GPT.

13:12Michael Frank:That leads me actually to the next set of questions I wanted to ask, which is like, when you're getting this data, you're going to see what they're looking at and who they're taking cues from. And I know you've looked at, you know, language is not just language, it's a social tool. And you've looked at the social development of children. Can you give us some examples of how language acquisition kind of dovetails with the acquisition of social skills and being able to pick up social skills? I actually think this is very important also because I've heard that people are worried about cell phones and the potential that they're stunting some development of basic human capabilities in terms of social interactions.

13:50Michael Frank:So I don't know if that comes up in your work, but is there anything there to say? So just starting out with what social learning looks like. I mean, if you think of the baby as kind of a code breaker, this is kind of funny, but think about your granddaughter there sitting on the floor and the parents are talking about the credit card bill. How do you break that code? You've never heard of a credit card bill. You can't see one. It's not there. They're talking really fast. But if you think about now her parents talking to her in a kind of a grounded social interaction, there's toys in front of us.

14:22They're giving kind of short, repetitive phrases that are really grounded in something that she's interested in, the toy she's playing with right now. And, you know, that's part of maybe a routine or a game where she can kind of figure out the rules. And they use all these rich cues, like looking at the thing together, they're pointing to it. All of that is going to create a situation where she can break that code. You can figure out what the name of something is. And maybe once you know the name, you can figure out what that word means, like, you know, kick the ball, right? So then you're saying, kick the ball.

14:53Yeah, good job. Kick the ball with your foot. So you get this set of cues that can help you kind of progressively break into more and more of this code when, especially when it's grounded in these kinds of social interactions that follow the child's lead and give them all those social clues and contextual clues.

15:12Michael Frank:Now, I think you've also done some, what looked like to me, fascinating work that the children actually evaluate how reliable the person who they're interacting with. I'm guessing that mom is rock solid, but other people, it's like, I don't know. I don't know how much I'm going to believe what they're saying. So what have you found about the kid's ability to kind of, I'll use the word decode, like the reliability of someone they're interacting with and how much they can really latch on to the words that they're learning. Any surprises there? Yeah. So, I mean, I don't think we think that kids are going around being super skeptical of all the people around.

15:49I'm not going to learn from you. But at the same time, it is really interesting and useful generally as a social agent who's trying to learn about the world to figure out who to learn from. You want to take lessons with the expert in some sense. And that can just mean which of your peers is better at this skill or more reliable. And so developmental psychologists often use that idea to figure out what kids know and how actively they're seeking out information. So we did this one study where we looked at how kids processed uncertainty. And we'd put out a ball and another toy, whatever, like a box or something.

16:29We'd say, OK, look at the ball. And the kid goes, OK, here's the ball. And then you say, and where's the blicket? And the kid looks at you like, huh? Which one's the blicket? You got to tell me here. And so we actually were studying that huh response to try to figure out if the kids were actively tracking and looking to their social partner for more clarifying information and what they would do. And, you know, of course, what they do at different ages changes. Maybe the littler kids are just puzzled. The bigger kids may actually ask a question or point to it or make a guess and kind of try to get some affirmations.

17:00So you can really see these active learning strategies where the kids are helping figure out that code by their own actions.

17:08Michael Frank:You know, just going back to my situation, you know, as the grandfather, I just spent a lot of time just staring at her. And it's very clear that her mother and her big brother are two incredibly important sources of information that she tracks noticeably more than everybody else in the room. And so this kind of all makes sense. This is The Future of Everything with Russ Altman. More with Michael Frank next.

17:47Michael Frank:welcome back to the future of everything i'm russ altman and i'm speaking with professor michael frank from stanford university in the last segment michael told us about some of the ways that he is gathering data in order to learn across multiple cultures and geographies how children learn language in this segment he's going to tell us about an exciting project where he collaborates with people all over the world to increase the size of the database and the ability to quickly test hypotheses about what's universal and what's specific about childhood language acquisition. Mike, another project you have is called Many Babies, which I love the name.

18:27Michael Frank:What are the goals of Many Babies and how is it kind of advancing your research? We're just generally interested in trying to understand how child development varies across different cultures and communities and individuals. And, you know, if your traditional strategy is studying 16 kids at a nursery school in Palo Alto, you don't get a lot of traction on that kind of topic. So what we did, you know, starting almost now, you know, seven, eight, nine years ago is sit down with a group of folks interested in this larger scale science and say, how could we distribute the same experiment around the world?

19:02How could we get not just a couple of babies, but many babies? Many babies. Gotcha. So we ended up with this consortium of researchers all over the world. And we decide on something that's really critical, something where we really have a lot of uncertainty about exactly what babies know or what they can do. And then we create a consensus protocol and experiment. Often it's as simple as playing babies infant directed speech, like that high squeaky, like, hi, look at me. Do you see this?

19:32Michael Frank:and then is that what we call it that's great infant directed speech i spend a lot of time doing that oh yeah they used to call it mother ease but you know sometimes dads and granddads are experts so so um our first many baby study was actually studying the global preference of babies for infant directed speech and the world around babies like it when you talk high and squeaky to them it catches their attention uh they focus longer even you know uh if in their culture, they're not hearing a lot of infant directed speech, they still like it, even if the infant directed speech is in English, and they're not learning English, still sounds kind of appealing to their perceptual system, it draws their attention.

20:11Michael Frank:So this is fascinating, because there's a whole bunch there. But in terms of the organization, you've basically created a worldwide network where hypotheses can be tested, you all agree to do it. And then it also sounds very kind of just in the sense of justice, because you're doing it not just in an English centric way, but you're learning much more general principles about human language acquisition. So it sounds great. Who supports this kind of work? Well, you know, to be honest, this is actually one of the projects that I've had the hardest time finding financial support for, you know, and this is maybe a sad fact about the way that science funding works, right?

20:47When you say, okay, I've got a consortium of, you know, top universities in the US that's going to study something. Let's bring it to the National Institutes of Health or the National Science Foundation. But if you say I want to do, you know, I want to give$3 ,000 to a lab in, you know, in Uganda to redo this study, it gets a lot harder to find somebody who's willing to send that money, even though it's a tiny amount of money and it makes a big difference to that lab to buy them the monitors and the speakers. This kind of global consortium work where you flexibly send a little bit of money to incentivize a collaboration, it's actually very tough to do.

21:21But so much of our work has been volunteer. which says a lot about the, I think, the enthusiasm globally for this kind of research.