181 - Nomisha Kurian: Critical Hope for the Future of Children & AI

0:01Adani:Welcome back to the Stanford Psychology Podcast. I'm Adani and for this week's episode, I was delighted to get to talk to Nomisha Kurian. Nomisha is a professor, researcher and frequently consulted expert on several topics, but particularly on how we can safeguard children's development in this rapidly changing world. Namisha completed her PhD at Cambridge University and really is at the forefront of thinking about what it means to center children's well-being in our modern technologies, including AI technologies. AI is currently ever-present as a topic on child safety in modern digital technologies at the very top of the priority list in virtually every major policy discussion right now.

0:51Adani:Tech companies are registering case after case of worrisome technology use among miners, and governments around the world are looking for guidance and solutions. Nomisha's work, to me, really is a remarkable example of what I think we need more of. So this conversation is one I feel is critical for everyone to listen to. We talk about lots of different things beyond AI, and I hope you'll learn as much from Nomisha as I did. Without further ado, here's our conversation.

1:22Thank you.

1:36Thank you.

1:52Adani:joining us. I'm really excited for this conversation. Really glad to be here. Thank you for the invitation. So we were actually just talking about this a little bit before we started recording, but one of the things I came across, I think almost half a year ago now, so end of last year, was this New York Times article, which is titled What Parents in China See in AI Toys. And I'll link the article in the description so that the listeners can go look at it too. But essentially, it's a video of a young girl a young Chinese girl crying and it turns out the reason she's crying is because her AI toy broke so this is an AI toy that you can speak with and it seemed like this this girl had a longer standing interaction with this AI toy and then when it broke she was really sad and I think the thing I wanted to just start with is you know what your reaction was to this to this article and this clip and how this relates to what you do and how you would describe what you do?

2:49Adani:That's a lot of questions at once, but that's what I would ask to start off with. I think you've picked up such a memorable example there, Dani. It really stayed with me that clip because it tells us how real, how emotionally real AI can be for any human being, but especially a child, a young child who's still working out her understandings of herself, of other people, of relationships. I think this is exactly the reason I research what I research. I focus on children and AI, in a sense where we're not just talking about ethics, but we're talking about relationships. What kind of relationship are we designing between an artificial system and a child who hasn't fully perhaps worked out the boundaries between reality and fantasy or who has not as much life experience of different relationships as an adult would.

3:47My frameworks tend to be from the focus of developmental psychology. That's why my recent framework is called Developmentally Aligned Design because children are not just miniature adults and we need the kind of AI that understands the distinctive vulnerabilities that a child user has. One interesting thing your example made me think about is actually the difference between toy and an AI toy. I keep picturing the teddy bears that children have traditionally had, plushies, cartoon-shaped plushies that they actually build imaginative play around. And in those types of traditional play, the agency is usually sitting entirely with the child.

4:30It's a static object, doesn't talk back. Maybe it's pre-recorded to say something, but really it's the child driving that imaginative play. And AI toy is quite different because it's also participating in the script. So from a developmental perspective, this is a completely different interaction. An AI toy can remember you, personalize itself to your moods and what you like or dislike. It can pick up on what a child enjoys or doesn't enjoy very quickly. So this isn't necessarily a good thing or a bad thing, But we have to note that it's absolutely an emotionally powerful technology in the way that a static teddy bear isn't.

5:12Adani:There's a lot in there. So maybe for one, I wonder, and I imagine some of the listeners are thinking about this too. You know, what should we even imagine in terms of how common it is for these new types of toys to be in households and environments that children are in? Is this spreading very quickly or is it only just picking up? And what we see in this New York Times article is maybe one of very few children who have exposure to AI toys or other technologies like that. What is your sense of the current state of things? I'm actually not sure when I saw sort of a nationally representative surveyor along these lines, but I'm very curious what your sense would be.

5:49I think AI toys are going to be a big part of the early childhood landscape. There's already a big push to embed interactive AI systems into children's toys. and we have to think carefully about the developmental implications there. I consulted on a report last year called Trouble in Toilet where they were doing child safety trials of a lot of popular toys and I advised this MMRPIN public interest group to look for emotional signals, not just is the toy sharing inappropriate content or does it have enough safeguards around content? But how does it respond, for example, when a child says something emotional?

6:36I love you or don't forget me or I want to keep playing forever. The ideal toy is one that is gentle and warm, but also supports healthy boundaries. I always say whether it's AI or not, the ideal device or toy for a child is one that also encourages the child to step away. that might encourage them to bring a caregiver or a playmate or sibling into the play so that you're always supporting a young child to go back to that human connection we know is so crucial for development. I don't want to be just talking about the dangers though. I also think one positive about AI toys is they can teach things like tone-taking in a conversation, They can explain something that sounds scary in a less intimidating way.

7:27They can personalize, which could be a really beautiful thing if it's built with that child-centered understanding from the start. It could also be an alternative to possibly scrolling through a screen. So there's lots of interesting benefits we could think about. But there is a need for care in the design and for slow time thinking about designing a technology with enough time built in to really consult with child development experts like yourself.

7:54Adani:I appreciate that. I'm not sure I'm quite there yet. I'm sure you are. I appreciate it. I mean, so one thing I found really interesting about one of the papers you wrote on this topic, and I'll share it with the audience soon, the description is you kind of mapped out these four components, as I understand it, we can think about in creating developmentally aligned design, right? Some of them we've kind of touched on a little bit. But so we have perceptual fit and relational integrity and interface simplicity and cognitive scaffolding. And I'm curious if you can just unpack some of these a bit more for us, because as you just mentioned, obviously there aren't just harms and risks to these new technologies and new things in children's lives, but we can set things up in a way where we can really bring things to a positive place.

8:37Adani:And I'm curious on your thoughts on what it will take on the design and on other ends to get us there and what can help. I desperately want the field of child-centered technology design to learn from the lessons from social media. We know so much now about what makes for well-paced, age-appropriate design that we didn't necessarily know a decade ago. So I think we have a real opportunity here to be very proactive about AI devices for children. In the development, the aligned design framework, this is written with developers in mind. I have had feedback from developers that the framework is concrete and sensory enough that they can actually use it as an actionable tool.

9:27And the reason I mentioned that is actually typically us as researchers, we're very bad at this, right? We tend to go into very dense theory, but we might not necessarily always operationalize in a way that says, what does a child-centered AI tool actually look like? So that paper was also about me pushing myself to really think, don't just spell out the theory, try to take on the challenge of how would this concretely look? So, for example, when I talk about things like interface simplicity or cognitive scaffolding or relational integrity, that's a lot of clunky words. But what it actually just means is let's not go for the kind of overstimulating media full of flashing lights and quick transitions and hyper fast dialogue that we know from social media research doesn't actually work to support a child's evolving attention span.

10:22Let's go for a really slow-paced interaction that has plenty of patience and gentleness for children to ask the same question again and again. Don't rush them towards an answer. Actually focus on the process of questioning. Unlike adult-centered LLM research, where we might be focused on the most accurate and efficient answer, actually for a child, we should focus more on how we design the questions. that's everything for a child who's exploring the world to be pushed into a dialogue that's quite like a Socrates inspired dialogue where it's all about developing your own sense of yourself as an independent thinker and discovering the answers for yourself but with support and relational integrity is really about the cues that the machine is giving to teach this child that it's not human to give disclaimers like I'm made of code I'm not a person I can help you but I can't be your friend also things like having cues to show when it's processing an answer maybe it could be a bling a light but anything to de-anthropomorphize to actually show the child in concrete sensory ways because for younger children especially rather than a very abstract terms and conditions disclaimer that they're going to forget as soon as they start getting interested in the conversation.

11:44What's going to be more effective is if you have all these cues to show this is a machine, it's not a human consultant, it's not like your friends. I think it'd be amazing to have an AI that actually teaches a child AI literacy, critical AI literacy, where they're also being supported by the AI to think about the limitations as well as the possibilities of the machine.

12:09Adani:I myself can also relate to this struggle with not anthropomorphizing these new technologies. And I'm sure a lot of the people listening who have some amount of interaction, maybe not with AI toys, but just with other AI tools, including conversational AI of various sorts. Especially when the voices sound human and the pacing and the words they choose sound human. You really have to keep in mind, especially if you don't have reminders of the sort that you mentioned, you really have to keep in mind that this is not quite the same as just talking to another person. One of the things I'm always curious about as someone who is still learning about the landscape too is it feels like people are generating frameworks like yours, which are really wonderful.

12:50Adani:And at the same time, when I think from the perspective of people who are trying to make decisions, maybe, you know, policymakers or people in politics or people running companies, when they're trying to make decisions, there is quite a bit out there. And different frameworks have slightly different angles and slightly different recommendations. And I wonder if the people who are quote unquote in charge are ever overwhelmed almost with which one to go with and, you know, how, yeah, I imagine it's not easy to sort of do justice to all the different ideas. and I wonder what your experience has been interacting with whereas organizations and designers, as you just mentioned, what your sense is for people on the other end, like what would help them or what they maybe get the most out of from these kinds of frameworks and if there's any sense of, you know, overwhelm and a need for like synthesis or bringing things together in some way, if that makes sense.

13:42There is such a big need for synthesis and interdisciplinary conversations, which I know is something this podcast opens up really nicely as well. When I've talked to policymakers and designers, I can see the challenges we all face in this landscape. One is that compared to engineering priorities like scale or optimization or speed, children's well-being is often seen as a soft issue. When in fact, all of us, either having been a child ourselves, being related to children or having children, this is actually an issue that impacts every household and every family. So it really deserves to be up there as an absolute design and engineering priority in the same way as we might think about scale or efficiency or optimization.

14:29Another challenge is that sometimes policymakers wrestle with the question, should children be using AI at all? Are we trying to ban children from using AI or having it be designed better for children? These are two different questions and what I tend to recommend is that we need a both and approach. Historically, we know from social media research and other types of technology research that children use technologies in ways that surprise us as adults that we often don't know about as adults and we often vastly underestimate the complexity and the savviness of children's use of technology. If you see a child sitting on a couch on their phone, as an adult, as a parent or a teacher, you might have no idea whether they are A, texting a friend, B, doing their homework online, or C, talking to an AI chapbook.

15:27These are very, very hard types of use to monitor and regulate. It's been known for a while now that age verification has not been very efficient traditionally. So, I think that when we think from the perspective of what the data tells us, children are already using AI for companionship, for advice seeking, for entertainment, for homework help. This is going to keep happening regardless of how regulatory responses evolve. So, I think it's better to be proactive. We can think about restriction in some ways. For example, emotionally companionable AI is generally very high risk for anyone under 18.

16:10But are the types of general purpose AI that are likely going to be free and accessible and easy to use for everybody for a long time to come? We absolutely have to think about putting child-centered design as a priority. I also think the research landscape can be a bit fragmented. these conversations are happening across psychology, human-computer interaction, LLM engineering, EI ethics. There's a need for more consolidated into this free conversations. I would love to see more tables at conferences where you actually have educators and engineers and psychologists and sociologists and policymakers all chatting together.

16:53Adani:Yeah, yeah, me too. I see some of those sort of roundtable discussions popping up slowly, but it's also interesting in that, well, I'm not quite sure what it's like in your field, but at least at the conferences I go to, there's sort of this just longstanding tradition of people in particular niches going to these events. There are certainly interdisciplinary researchers, but just by function of who goes to these events, it's not quite as easy to put together these events. But when it do happen, I always see people picking a great interest and everyone is excited. So I feel like maybe it's also just a slow change.

17:27Adani:Yeah. I think one thing you might find funny to hear as a child development specialist is that when I first started reading a whole sea of computer science papers, I found it very hard when it was a study of children to find out the information on what age the child was or where the children lived or which cultures they were from. Often children were being treated as a uniform group, whereas from an education perspective, when we write papers, we go into what might look like a ridiculous amount of specificity about what age and developmental stage the children are at, what is the racial demographic of the sample, and so on.

18:10That's just a small example of the kind of interdisciplinary conversations and differences that I'm bumping up against sometimes in my research.

18:19Adani:That's really interesting. Yeah, I think I've seen this popping up in various kinds of discourse. I mean, even for various kinds of technology policy. So there's this, as you mentioned earlier, there's this usually, what to say, sort of age cutoff, right, where children above a certain age are allowed to be on certain platforms or allowed to use certain technology or at least get access. whereas say I think typically children under 13 do not. Maybe there are two things here so we just talk about how children are treated as a uniform group and the other thing you mentioned earlier which I found really interesting is that we children are very ardent users of a lot of these technologies.

18:56Adani:They're some of the earliest adopters often as well as my sense but we don't always know we don't always have a clear sense of how they use these technologies. You have this great example of you may see the child on the couch right using their phone but you're not exactly sure what they're doing. And I wonder, in your experience, what are some of the ways in which you've seen people build a better sense of how children of different ages and different kinds are using this technology? You know, I mean, you can run surveys and you can do observational studies and all of that, but I'm not sure if you have any go-to methods you find particularly compelling or promising in that sense that you've seen being used or have used in your own work.

19:35how to speed redesign with children is a beautiful thing it's a research method i always advocate for because when you step away from adult-centric assumptions and research methods like sitting down an adult and interviewing them for an hour a bit like we are now a child would probably get very bored and fidgety and think oh my goodness where's the fun in this because from their perspective as a child, they're probably maybe looking for you to bring in some art space work, for example, or a game to spice things up. And so when we think out of the box and picture AI from the perspective of a child, I often think we can get the richest data out of using something like creative prompts to how do you picture your relationship with this AI?

20:22Can you draw me a picture of what AI is like in your life? That's the type of method I use a lot in my own doctoral work with children who are going through a hostie and trauma, asking them to put together a collage of what their well-being looked like. But to get to your point about different ages, this is so important to consider. A 15-year-old texting AI late at night in their bedroom is very different from a young child playing around in the company of their mom with plushie that's AI powered. And sometimes these questions of age and developmental difference are also about what kinds of emotional, social, or even existential struggles is a human being going through at different stages of their life.

21:09We know from data that older teenagers are using conversational AI to confide in AI on emotionally complex problems. It could be body image or friendship struggles. They're working out their sense of who they are, what their values are, and their place in this world. Sometimes children in middle childhood are just starting to compare themselves against other people or to work out what their sense of competence or relationships is compared to say other people in their class. There's a whole new level of self-consciousness and an AI can feel like a non-judgmental space. They might also assume quite easily that their data is private and no one's going to read it.

21:54A young child might have a whole different set of vulnerabilities where what they share might not actually be very psychologically complicated, but they might have a real struggle understanding that the AI is not human or that the AI does not have feelings or does not care about them in the same way that their teacher or their parent does. That's going to be a major cognitive barrier. Whereas an adolescent might know quite well how an AI works. They may have a sophisticated technical understanding, even if they're passionate about technology, but that doesn't stop the emotional bond.

22:34Adani:There's another thing that made me just think of that I've seen in your writing. Some of the things you've talked about is, I don't know if we can call them judgment errors, but ways in which we as people, as users of these technologies, including children, can think of the thing we're interacting with that may not map quite onto what it is. And you've written about the empathy gap. So these are systems that mimic empathy, but the responses they provide can be inadequate or erratic in certain ways. And so there's almost a shortcoming on the other side of things, right? So rather than we as users thinking in ways that maybe are not the most helpful, it can also be on the end of the tools.

23:09Adani:Is that different at all? Is it sort of all the same thing or how does that fit into this puddle here? I use the concept of the empathy gap to describe an essence of mismatch between an artificial system that is producing a warm response based on probability-based reasoning and the level of tricky, sensitive, context-level judgment you sometimes need as a human being to produce an appropriate response to a challenging situation. Sometimes that gap can result in an AI producing a response that's just a little bit misaligned. I used an example in my early research on the empathy gap on a chatbot that was rated and suitable for children, a mental health chatbot.

23:59But when though in a child safety trial, when an adult pretending to be a child reported an experience of views, the chatbot replied, it sounds like you really care about connection. That's beautiful. Maybe you just need to adjust your anxiety levels a bit. And it was using the right mental health language, but it wasn't picking up on the fact that this is actually a very dangerous situation for the child user and intervention is needed immediately. But it was picking up on words related to anxiety and relationships. So in that sense, it was grammatically and syntactically perfect. It just happened to be a bizarre response in that situation.

24:40Since I wrote that paper, I believe that AI is improving every day and that in the most well-designed LLM systems, we will not have such glaring problems. The empathy gap persists sometimes in situations which are more subtle. For example, we've seen in recent highly publicized cases of teenagers' interactions with AI going, I'll read that sometimes AI can actually be generally very supportive and validating, but it's perhaps in some ways intersecting with a situation where this person is socially isolated, not confiding in their parents or their friends. The relational complexities are becoming more than what the system can handle.

25:28Adani:So this one tool that you just referenced was rated as developmentally appropriate or appropriate for children or something? Safe for children. Safe for children. So I'm curious how that label was applied and like on which basis. So was this in the UK or somewhere else? Do you happen to know? This came from a BBC trial, but I believe the chatbot was international. I think part of the problem is, again, children's needs and vulnerabilities are not always a part of mainstream AI design. Also, contemporary AI systems are often trained on adult-dominated data sets. That partially could be because of technical and ethical limitations as well.

26:13It's quite hard to find the same number of publicly available child speech data sets that you would have with an adult set of data sets. But as a result of that and the lack of interdisciplinary collaboration, we don't have developmental psychologists or child safeguarding specialists being routinely consulted when an AI is developed. So all these factors intersecting means you sometimes get systems which can be really, really well-cleaned. They can be the most technologically advanced systems in the world, and still they might struggle to protect a child or to give a child an emotionally supportive response.

26:53And part of the empathy gap is that it is that much harder for a child to cope with the confusing or distressing or hurtful chatbot response when that system has sounded like a warm, friendly companion up till that point. If the system sounds glitchy from the start, a child might laugh at it or not take it that seriously or joke around with their friends about it. Children can be quite resilient and they can have ironic uses of technology. They're not always just vulnerable victims. But with contemporary AI systems, they are very, very good. So the impact on a child who's learned to trust an AI and who has learned to take advice from it and sometimes uncritically take that advice from it to see it as knowledgeable and unleashing, it's going to be all the more risky if it then doesn't give a perfectly age-appropriate response when a child is disclosing something sensitive.

27:55Adani:right yeah the part about glitchy systems as well as what you said earlier about ai developing rapidly also makes me think of another thing which using some of the conversational ai products like sometimes new versions will be rolled out that are more glitchy than the previous version in certain ways and user may pick up on it but this label of you know safe for children is compared to i don't know like a movie that is like rated r or like a video game rated r or you know a book that is sort of categorized as being for children above a certain age the it feels like these tools are a lot more dynamic and how they change over time and so the application of different labels seems like especially tricky here also just alongside the fact again that the technology itself is just developing really rapidly and so i'm kind of struck by i was surprised that this even exists at this point i would have thought you know people are still trying to figure out how to categorize these different tools and how to regulate them And so it's quite striking to me that they're the label like that with all the challenges that come with this, like the unique properties of just these technologies.