161 - Yuan Chang (YC) Leong: Emotional arousal & dynamic brain connectivity

0:00Welcome back to the Stanford Psychology Podcast. I'm Su, and today I'm so happy to share my conversation with Professor Y.C. Leong, a great mentor who I had a chance to work alongside. Dr. Leong is an assistant professor of psychology at the University of Chicago. He is the director of Computational, Affective, and Social Neuroscience Lab, which is a part of the Department of Psychology, a member of the Institute of Mind and Biology and the Neuroscience Institute, and an affiliate of the Data Science Institute. His research explores the neural and computational mechanisms underlying how goals, beliefs, and emotions influence human cognition, with a focus on why people interpret and respond to identical situations in different ways.

0:40In today's episode, we discuss what someone sees intellectual radar these days, alongside with his recent paper, Dynamic Brain Connectivity Predicts Emotional Arousal During Naturalistic Movie Watching, in which they show that we can decode arousal with open-movie fMRI datasets. sets. So without further ado, here's our conversation.

1:23Thank you so much for joining me today, YC. I'm really happy to be hosting you today. First of all, I would like to start off by equipping our audience with you and your work, a little more. You're at the University of Chicago, directing the Computational, Effective, and Social Neuroscience Laboratory. However, I know that you recently went under a name change from the Motivation and Cognition Lab. Sometimes words are just words, but does the new name signal a shift in research focus or priorities? Should we go back to how you started and where you are right now? That's a funny question. I think that words are often more than words, especially names.

2:04When you choose the name of a lab, you're also trying to express an identity. So when I started the lab, I sort of saw myself as someone who studied motivated cognition, which is the phenomenon of how goals and beliefs steer thinking towards desired conclusions. And to a degree, that's still true. That's still what I study. But the thing about starting a lab is that things kind of evolve and you get trainees and trainees have interests and those interests become intertwined with mine. And so after a while, it just felt like the name Motivation and Cognition Neuroscience app didn't really encompass everyone's interests anymore.

2:41So I decided to adopt a name that I felt was more encompassing of what everyone in the lab did. So, you know, I'll give a few examples of the stuff that we've been working on recently. One is how emotions affect how we remember and understand narratives, which is related to the paper we'll be talking about today. A graduate student in the lab, Jaden Park, just started a project where we're using functional near-infurbite spectroscopy to scan couples while they argue to try to understand how features of the conversation or argument map onto brain activity and the extent to which those things predict memory and relationship quality.

3:17and on a more macro scale we're also still interested in the divergent interpretations of political messages which Sue you worked on when you were in the lab if I were to put you know everything into a single phrase that ties the lab's work together I would say it would be emotions in the social context using computational methods hence the name computational affective and social neuroscience lab or casenelle okay that ties everything together and i feel like the name chain it has a good drink fit too so i like it um can we say that you talk about these research projects and how you started with your own research and then came to to the current point right now but i see a common theme it's like i i feel like it's these naturalistic uh data sets You mentioned couples arguing, for example, the political information, movies, and also the idea of people interpreting and responding to identical situations in different ways.

4:25It's just like people basically interpreting something and that's realistic response and then reacting to it. Is that like a common theme in your projects? What should we take away from that? Yeah, I would argue that that's a common theme. So I think there are two parts of that that you alluded to. One is this idea of seeing the same thing and getting something different out of it. And the second is the use of naturalistic methods. I would say that both are important parts of the lab. The first part, which is this idea of how is it that we experience and inhabit the same world, we see the same thing, but we can get something completely different out of it.

5:03That's been a core theme in the lab. Ever since I started the lab, I would argue that that actually stretches all the way back into graduate school, that a lot of my work is premised on this idea that how we interpret and respond to any given situation is not just a function of the objective information out there in the world, but also of what we as individuals bring to the table. We have goals and desires, fears and anxieties, beliefs that we hold dear. and I'm interested in understanding how these factors affect how we interpret the world and why is it that we can see the same thing, be in the same situation, but take away something very different.

5:43This, what I would call human subjectivity, is part of what makes the study of people interesting and intriguing for me. One way to put it is if everyone was kind of the same, I feel like studying people would be a lot less interesting, but at least for me. I find that part of what's interesting to me about psychology and neuroscience, understanding how these patterns of brain activity give rise to these different subjective interpretations. So I would say that that has been a theme that runs through the lab from visual perception, sort of like how we literally see different things depending on what we want to see, to more abstract phenomena, such as political polarization, where we read the same news article.

6:24It's not as if we're reading different words, we're seeing the same words, but what they mean, means something very different to people. And that's part of the phenomenon that I'm interested in. And yes, I do use naturalistic methods. It's one of the methods that I use. It's not the only method. I do still take advantage of controlled experiments because I think that's just something really nice about having clean variables to be able to build computational models on. But at the same time, I've always had this pull and push of, is my experiment too clean and artificial and controlled? Should I inject some naturalism.

6:59But then when I do naturalistic tasks, I then start to feel like, ah, I'm losing control and I kind of want to take back some of the control. So the lab does both controlled experiments and naturalistic experiences, test paradigms. And I think it will continue to do both with the idea that on one hand, you want to run experiments with a lot of ability to experimentally manipulate and control variables to also putting people in situations that are naturalistic, rich, dynamic, and that I'm more resembling a real-world context. And I try to do both in the lab. And also, there's this, as you mentioned, nice balance of you can start with one and then you can follow up with the other.

7:39Starting more naturalistically and then with more constraints, you can just follow up. I can see there's a nice dynamic there too. Following up on that, actually, the work we will be discussing today is going to be a great example of how we integrate these psychological theories you just mentioned and some of the methods we just described. Your paper together with your mentee Jin Kei, Dynamic Brain Connectivity Predicts Emotional Arousal During Naturalistic Movie Watching. Building on the fundamental idea that emotional experiences can be described along dimensions like balance and arousal, your study aimed to understand how these dimensions are represented in the brain and whether these representations are consistent across different individuals and diverse situations.

8:26Could you start by explaining the significance of studying emotional representation in this way, particularly in the context of naturalistic experiences like watching movies? That's a great question. For a long time, cognitive neuroscience and affective neuroscience included has a tendency to use brief decontextualized stimuli such as words, pictures, or sounds. Part of the reason was the limitations of the analytical tools we had at that time that kind of restricted us in terms of the type of stimuli that we tend to use. And as a result, we know a lot about how the brain responds to such stimuli, such as like a photo of a cute cat or a bloody hen, and how emotional responses to these stimuli or images or sounds or words reflected in the brain.

9:16What we know a lot less about is the kind of emotional responses to contextualize and dynamic stimuli. For example, in this paper that we'll be talking about today, we use movies as an example stimuli. If you think about a scene of Sherlock Holmes picking up a piece of paper, that scene in itself has no meaning in the sense that you don't know how arousing that scene is unless you also know what that piece of paper means. It could be an important clue. You wouldn't know whether this scene is positive or negative unless you think about how it relates to the plot. So there's nothing in terms of its visual characteristics, why that would be arousing or positive or negative.

9:59Its affective meaning is derived from its relationship to the context. So that's one aspect of the importance of these dynamic, rich, contextual situations. So the second thing I mentioned, which was already in my response to this, was it's dynamic, right? That emotions, our experience of emotions tend to fluctuate over time along with the ebb and flow of what's going on. So when you use punctate stimuli, like images of cats and surgery, those are the type of stimuli that people tend to use. They fluctuate from one image to another image without things tying them together, which isn't really how we experience a lot of our affective experiences in real life.

10:40it tends to ebb and flow at a slower timescale as kind of like the plot evolves, as we gain more information. And I think that is another thing that is often captured using something like a movie where our emotions ebb and flow with the plot. I want to be careful and not overstate the claim that we should all be studying movies and it's the only naturalistic experience out there. You know, if you really think about it, a cute cat is also a naturalistic experience, right? Pictures of stuff is literally what Pinterest boards are where you would scroll and then look at an emotional response. And so it's still important to understand how to bring response to those things.

11:16But what I'm arguing is that we now have an opportunity to also consider another aspect of emotion, one that's more contextual and more dynamic. And that is what I argue is part of the contribution of this paper by using movies as a structured way of eliciting these emotions across people. and using that to study how the fluctuations and change in emotion are encoded or represented in the brain. So you say that more so than neutralistic, it's more about dynamic contextual information that is present in the movies. I see. You talk about dynamic context, but that is not the only dynamic pattern we see in the paper.

12:00You highlight that the generalizable arousal representation involves dynamic functional connectivity patterns between large-scale brain networks, too. What does it mean for arousal to be encoded in the interactions between these brain networks? And how does this perspective complement or differ from focusing on activity in specific brain regions? We talked about the stimuli a little bit, but your tip also talks about, yes, these stimuli, they're dynamic in context, but they also have dynamic connectivity patterns with different brain regions too. What can you tell about that? Yeah, no, that's a great question.

12:37The way that we're thinking about it is that information about the world isn't just represented in the activity patterns of a particular brain area, but also in how those brain areas interact with other brain areas. And that's what dynamic functional connectivity tries to capture. So functional connectivity or functional correlations are essentially a measure of the statistical dependence of different brain areas with one another over time. And this is dynamic functional connectivity in the sense that we actually look at how these patterns change and evolve over time. So it's not the overall connectivity between brain regions, but also how these connectivity kind of fluctuates.

13:21depending on the stimuli. Of course, we're not the first to look at dynamic connectivity. It's a concept that's been in cognitive and affect neuroscience for a long time. But very often, in terms of thinking about how emotional representations are encoded in the brain, a lot of the past studies have focused primarily on individual brain areas, such as the amygdala and the insula, for example, and maybe the interactions between a small number of brain regions and how they might be capturing information about emotions. But in our case, what we were really interested in too is not just the specific connections between a small set of regions, but rather the interactions across the brain in a whole brain manner.

14:07And this idea is, I would say, an evolving or growing perspective in neuroscience. For example, by Louis Pessoa and many others who starting to think information and how the brain functions can't really be cleanly distinguished into this brain area does this and this brain area does that. The function of the brain area needs to be considered in relation with its connections to the other brain areas, to networks of brain areas, and how they function in tandem to encode information. So that's one aspect and one inspiration of this work, where we want to move beyond the role of particular brain regions, but actually look at patterns across brain regions and how they might actually also contain information about stimuli, about experiences, if you will.

14:52And the second thing I will mention is that this is also inspired by work in systems and theoretical neuroscience, suggesting that one of the things that arousal in particular does is that through the release of the neuromodulator norepinephrine, it actually causes different brain networks and brain errors to be more synchronize with one another. So we have this evidence from animal work and theoretical work. And so we had a hypothesis going in that this would actually be reflected in patterns of large scale functional connectivity as well. So one of the benefits of fMRI is that we are able to measure the whole brain at once.

15:30And that's a tool that many other neuroimaging tools currently lack. For example, apnea, which I mentioned earlier, you only measure activity from the scalp close to the surface of the brain. And with fMRI, however, you can actually capture the entire brain at once. So we can actually not just look at specific brain areas, but also at how these brain areas are talking to one another, if you will, and how that might actually contain information. I will say one last thing, which has to do with the nature of the stimuli. I think that one of the reasons why dynamic connectivity, again, there are many reasons, but one of the reasons why I think it hasn't been used as much to study, for example, emotional responses to images and sounds and words, is that connectivity is computed over time, right?

16:18Like it itself requires dynamics for us to calculate a measure for it. So granted, there are ways to identify connectivity patterns that are related to a punctate brief stimuli. At least from my perspective, this seems to be a very natural way of studying movie data because movie themselves are evolving over time and we have a measure that is also evolving over time so we have the stimuli that matches the method and i think that's part of the motivation of why we use this dynamic connectivity as the measure of interest so like with the directed attentions of the participants you can actually have this corrected time series, right?

16:59You don't have to worry about, oh, are they paying attention to this part? Are they thinking about that? Yes. They're actually flowing together with the movie's progression. I see. I kind of want to go back to valence and arousal specifically. In one of our earlier written communications, you mentioned that there's an implicit assumption about many prominent theories of emotion or phenomena that states arousal is inherently ambiguous and susceptible to misattribution. But the neural base of this assumption has remained largely untested. How might your finding of a generalizable neural representation for arousal provide a neural-based explanation for why arousal might be inherently ambiguous or susceptible to misattribution across different contexts?

17:48Right, that's a great question. I think that it's helpful to provide a little bit of context in terms of the background that I'm drawing on here, which is that classic theory of emotion, often described as a two-factor theory of emotion, has this idea that how our emotional responses to a given situation is often a cognitive construction of our arousal response. So we feel a particular level of arousal and we try to explain it in terms of what else is happening and create, if you will, a story of what's going on. So I think a classic example of this phenomenon is that in a very classic study by Dutton and Aaron, experimenters found that men who cross a scary elevated bridge were more likely to find a woman experimenter at the other end of the bridge attractive compared to men who walk across a sturdy, low bridge.

18:38And the interpretation of this finding has been that the men who crossed the scary bridge misattributed their arousal due to the fear to romantic attraction. So even though they were aroused likely because they had just crossed the very scary bridge, because there was a woman experiment in front of them, they misattributed this arousal to romantic attraction to the experimenter. So this is a very classic study in social psychology highlighting this phenomenon of misattribution of arousal. This concept rests on the assumption that arousal states are confusable across context and situation, right?

19:17For this to work, your brain must confuse or is able to confuse arousal related to a particular situation to that of another situation. In other words, arousal might be represented in the same way in different contexts. The assumption of many of these studies is that arousal states are confusable across situations. But this idea, to our knowledge at least, hasn't really been very much explored in the brain. So we know that arousal is associated with similar physiological responses like pupil dilation, increased sweating, faster heart rates. And we know that arousal triggers these similar peripheral nervous system responses.

19:59what we don't know is whether arousal is associated with similar brain states across situations. So what we show in this study is that we are able to decode arousal from a similar pattern of brain activity across different contexts. Your brain, when it's aroused because of one particular scene in Sherlock and in another scene in a different movie, that arousal is represented in the brain in a similar manner. This could potentially explain why arousal states could be confusable across situations because the brain is encoding and representing them in similar ways. That to us is why we think that this could provide a neural basis for why arousal is inherently ambiguous because the brain is encoding it in a similar way and as a result is susceptible to misattribution across contexts.

20:49I remembered that bridge example word by word from your social psychology class that I took. That was a clear flashback for me. Yeah, we talked about the central finding of your paper, but you did not find similar evidence for emotional valence. Why do you think emotional arousal but not valence appears to have this generalizable neural signature across different movies and participants? The first thing I would say is that now findings are difficult to interpret, so I don't want to over-interpret them. But in the paper, we give a couple of speculations. The first is that maybe valence is not represented in a generalizable manner.

21:31What we mean by that is the positivity in one situation may not be represented in the brain similarly as the positivity in another situation. To give a more concrete example, the joy of winning an Olympic medal may not be the same as the joy or seeing a friend. In both cases, they could be very intense joy, but the joy has different meanings. So as a result, the brain might not actually be encoding them in a similar manner. And there is some mixed evidence in the literature. So some studies have found that, oh, you could decode a valence representation. In other studies, for example, in a method analysis by Kristen Lindquist, what she showed was that, you know, if you look across all studies, there isn't really a clear pattern that emerges in terms of which area is encoding valence or representing valence information.

22:24So there's other work that has also failed to decode valence. So we're not the first group to say, hey, you know, we're having trouble finding a valence representation in the brain. Having said that, there are groups who have found it. So I want to be careful in terms of not to overstate the claim, because I think it's also possible that we just couldn't managed to find it in our study, right? And there are many reasons for this. For one, we were looking at representations that were consistent across people, right? So in our paper, what we really tested for is the extent to which a model that is learned on one individual can generalize to another individual.

22:59And maybe valence representations are more individual specific, that maybe your brain is representing valence in a similar way within yourself, but it's not the same as how my brain is representing it. Relatedly, it's also possible that the valence representation is not encoded in dynamic functional connectivity, because that's the main thing that we looked at, right? Maybe that is more represented in activity, or maybe it's more represented in specific patterns of brain activity, which is not kind of what we explored here. And that could be another reason why we weren't able to find the valence representation.

23:33That's all to say, as I've mentioned that now results are difficult to interpret, and we can only speculate on reasons why we might not have been able to find the valence representation. But I do think it's a very interesting question as to whether valence is a generalizable construct. But I think that it's not a question that we can really answer with this particular study. Going back to the arousal topic, you mentioned that arousal is measured using various methods in psychology and it's not always clear how they relate. While this study specifically focused on self-reported emotional arousal, how does your work fit into this broader landscape of arousal measurement and what potential avenues does it offer for future research linking different types of measures of arousal to brain dynamics?

24:25Yeah, you're raising a very important point that we haven't really touched on throughout this conversation, which is that the way that we've measured arousal is through self-report. So we ask people how aroused they were when they watch these movie episodes or TV episodes. And in the field in psychology and affective science, the researchers have been measuring arousal in many different ways. And arousal is such a central construct in psychology. It's almost like everywhere and everything has a level of arousal in it. But it's worth mentioning that depending on the researcher, depending on the tools that they have at their disposal, different people often measure arousal in different ways.

25:06So some people measure physiological arousal, such as the dilations of your pupils. So the fluctuations and how large your pupils get, that has been taken to be a measure of autonomic arousal. Same with the skin conductance, the extent to which sort of like you're sweating, essentially, is a measure of arousal. Your heart rate, in terms of how quickly your heart is beating, how variable that is, has also been measured as a measure of arousal. There are also other neural measures, such as particular frequencies and EEG, has also been taken as a measure of arousal. So in the field, we have all these different measures of arousal.

25:42And when we write papers, we say arousal. But is that the same thing? And right now, I would say that the evidence is unclear whether they actually refer the same thing. In fact, there was a recent review paper of a theoretical piece, I think written by Clarence Smith, who basically made the argument like, oh, should we be focusing on arousal? Is arousal like a thing in itself that we should really be basing our theories on? If we can't really know that it's referring to the same thing, like, are we confusing each other if we use the term arousal for all of these different psychological phenomenon?

26:20And one of the ways that I think that the current work can provide at least one perspective to this theoretical question is that I think that we can think of how we might be able to map different forms of arousal onto the brain into a common brain space to try to understand how different measures of arousal might be related to one another. I'll make it a little bit more concrete. So we have all these different measure of arousal. We can build connectome predictive models of them. So basically, this is what we did in the paper. We can try to map each of these arousal measures onto the brain, right?

27:00And we can then ask, to what extent are these models producing the same prediction? That's the first thing we can ask, right? So in that sense, what we're getting is how often do these arousal measures, based on their mapping to brain activity, co-vary with one another? So that's one thing that we can ask. The second thing we can ask is to look at the patterns in themselves, right? To what extent are these arousal measures correlated with the same connectivity patterns in the brain? And that actually gives us a quantifiable and empirical way of saying arousal as measured by pubertal dilation is such and such similar to arousal as measured using self-report.

27:41So by mapping all of these arousal measures into a common brain space, I think gives us a way to try to understand all of these different arousal measures and their contribution to cognition. You can almost think of this as a common denominator that you can put all of these different arousal measures onto the same reference book that you can then read out from the patterns of brain activity. Once we have that, we can also use these models to try to understand how they affect behavior, right? So given any brain data, if we have these different models of arousal, what we can do is to decode different types of arousal from brain data and see the extent to which they are related to the behavior.