Briefing Chat: When to trust eyewitness memory – according to science

0:00Business is booming, but you know what really helped? Zero. The accounting platform for small businesses. I used to think unpaid bills were just part of it. Ugh. But batch payments? Wow. Automated accounts payable? Even better. All in one place? That's a big one. And I can keep track from my phone while I get the job done. No bills in sight. What a life. Do right by yourself and your business and search XERO for your free trial. Conditions apply. Do you keep hearing podcast ads, like this one for example, but always wonder how you actually get involved with them for your own brand or organization?

0:38Well, it's easier than you think. We're Acast, and we give you the platform to do it all yourself. Browse thousands of popular podcasts, choose the shows that match your perfect audience, set your budget, and launch. And if you want a hand, our podcast specialists are there to help you launch with confidence. This is podcast advertising without barriers. Get started at acast.com slash advertise.

1:07Hello everyone and welcome to the Nature Briefing Chat podcast. This is the show where we chat through some of the stories that we've been reading about in the Nature Briefing, which is a weekly email newsletter from Nature. And today your usual hosts are away, but you've got me, I'm Sharmini Bundel. We've also got Lizzie Gibney. Hi, Lizzie. Hi, thank you for including me today, Sharmini. Let's dive straight in. And I'm going to start us off this week. This is a nature feature and it's all about eyewitness memory and kind of improving the way the legal system deals with eyewitness memory using science.

1:40The main thing I know is that we are so incredibly fallible. I always think of that. There's that meme of, you know, people playing basketball and there's a gorilla that you just don't see unless you're told about it. And that has really freaked me out ever since I saw that. They give you a job like count how many throws there are in the basketball team. And because you're focused on that, you don't notice a person in a gorilla suit walking through the basketball match. But basically my brain isn't doing the things I think it's doing is what I took from that. So I would be quite worried myself as an eyewitness.

2:09For a long time now, there has been this sort of understanding that it can be really unreliable. And one particular big thing that happened was DNA evidence became available and people went back and looked at previous cases where they still had evidence that they could take DNA from. and overturned loads of convictions. And so there's the Innocence Project in the US which does this. And some of their data shows that when they're looking at the kind of thing that contributed to some of these false convictions that later got overturned, a huge number of the big proportion of them involved eyewitness misidentification.

2:47So then how do you factor that in then if you've got a fresh case coming up? You know, how do you direct a jury into like how much to trust eyewitness evidence if we know that it's failed so often in the past? Well, there has been lots of changes to how things are done in order to sort of try and improve the accuracy of various things. So, for example, police lineups is a big one because that's when you're bringing someone in and say, do you remember what the suspect looked like? Is it any of these people? And there are lots of things that you can kind of do wrong with the police lineups that might skew the results you get.

3:23Like, let's say I'm a witness and there's a police officer saying, do any of these people look suspicious to you? If that police officer knows who the suspect is, I can pick up subtle cues or that can in some way influence things. It's not a blinded study, basically. Exactly. They know the answer already. Even if they don't mean to, they can be subtly kind of hinting towards that person. Yeah, it's a bit of a mire trying to figure out how much we can trust these things. But what this feature is about is about the role that confidence plays. So if you were on a jury and someone is like, no, no, honestly, I definitely know it was this person.

4:03I definitely remember. How much would that sway you in sort of judging that eyewitness testimony? I think it would sway me a lot but then actually probably mainly what it tells you is more about that person than about necessarily what they saw. So in general, people have dismissed confidence and there's been results that basically previously showed that how confident a witness is, is kind of irrelevant to how likely they were in chapter or not. However, this has more recently been potentially overturned. So this is the sort of new science that they're talking about in this feature, which is basically saying actually confidence can be in certain situations really relevant.

4:42And one of the examples just given in this feature as a sort of case study is there was a witness who saw a crime and described two white men with long hair. And she did a police lineup and did not identify any of those people. One of them was the suspect that the police were, you know, pretty confident could be involved. And by the time it got to court, this witness told the jury she was over 100 % sure that that was the guy she'd seen. And the key thing here is the difference between the confidence when they're in court sometime later and at the moment where they're first sort of being presented with a lineup, say.

5:27And so what some of the researchers mentioned in this feature are looking at is what the immediate confidence was. And when they're there in the lineup and you ask them how long it takes them to make that decision. Right. Either they're like, yeah, it's that one. Or like the longer it takes them to go, let me think. Like that correlates with actually how accurate that is. The quicker they do it, the more accurate. Yeah, exactly. But the sort of key finding, and which was a bit controversial because it kind of went against the prevailing wisdom, is that in this particular trial, when the people initially in sort of very ideal circumstances were really confident that their identifications were correct, they almost always were.

6:10They were like 97 % correct. Whereas when they weren't confident, then it was kind of a bit over 50-50 whether or not it was right or not. So one important thing to take away from that is what is the confidence in that immediate time after? But then the other element in here, which is quite interesting, is in that case we talked about, the witness, the first time she saw that line up, the witness didn't pick out the suspect at all. and generally at the moment if you have a lineup and the witness doesn't get the person or like maybe picks the wrong person that is not given much significance and so there's some suggestion that actually that's really important to know that could be in fact negative evidence yeah well that's like a null result yeah exactly isn't it is the kind of thing that people say oh this this hypothesis didn't bear out.

7:08So we're not going to publish that. We're just going to leave that in the drawer. So there's also interesting ways in which perhaps this process could be improved. Actually getting that out there in the field, as it were, applying it to the judicial system, that is easier said than done. So what would you want to do? You'd want to know when you were having this eyewitness testimony, you'd want to know how sure they were in the immediate aftermath or how long it took them to decide that would be important information for a jury to know how credible that. And like the data suggests that that is relevant.

7:42The other spanner in the works I was just thinking about in all of these studies, presumably their data point on whether that person did the crime or not is what the trial found? No, at least some of these studies, they've set it up. Okay. This isn't like observational data. Okay. No, this is at least in some of these cases based on getting volunteers to like fake crimes and then asking people like, have you seen this person before? And then they say, oh, so no. So yes, it is useful to be able to know. Yeah, you know for sure. Yeah, absolutely. But really interesting stuff. And just all this stuff about memory and what we think we remember and how our memories can be swayed is just fascinating.

8:19Fascinating and kind of terrifying. Yeah, we'll put a link to that feature in the show notes. Well, that did remind me a little bit, actually, because this whole idea of the evidence that ends up being used versus that which isn't is quite tied to what I'm going to talk about, which is the first registered report that Nature has published. So Nature had an editorial on this and also published the journal paper. And so a registered report is when you have a study and you submit for peer review just the plan. You submit your research plan and your methodology, like the statistical tests that you're going to do.

8:54And in certain cases, you're like the hypothesis that your testing and then that goes to peer review and what happens is the journal then either says yes we think this is good enough and the peer reviewers say this is a very worthy question let's do it and the journal commits to publishing your results whatever they are so yeah this has been such a problem for such a long time hasn't it that a big snazzy journal is going to say like oh science well is it exciting and new and did you find something good if not yeah and then everything that gets published is all of the positive results and ignoring all the potentially more negative results that could have come out the same kind of thing and it just it means that you know people go down dead ends all the time because there's a study that actually someone else has done and it came up with a negative result and so people then don't bother publishing it and then another team will go oh that's interesting let's explore that and so you can have so much time wasted and very similar things what we're talking about when it comes to criminal trials you know like is there is there some subtle biases going on we're not saying anyone's doing anything deliberately but where if you have a result and you want to get that you expect that as your answer you might slightly tweak the analyses that you do you might think well we got nothing when we run this test let's try running this other test so the idea with a registered report is that you say upfront everything you're going to do oh so this stops the the p hacking as well you have to say which analyses you're going to run in advance as well okay yeah and in this case so I'll tell you bit about the actual study as well that's been published because it's kind of cool.

10:22Another benefit is that you can do quite ambitious projects because you know it's going to be published. There's no risk to it. So this specific paper was a study of users on Blue Sky, social media platform, and it looked at 2 ,000 users before and after the last US election in 2024. And the researchers were trying to figure out if the algorithms that the platform uses, you know, they are tailored to drive engagement. If by using those algorithms, you know, to decide what goes top of your feed, if that amplifies this kind of toxic, outrage driven kind of political divisive content. Is this one of those where they are working with the social media platform and getting them to tweak the algorithms?

11:08Okay. So Blue Sky is one of the, in fact, I think now the only social media platform that will let you do this, that give this access to researchers. And so they ran three different algorithms. They had this one, which was the kind of classic social media algorithm, amplifies engagement. They had one which was just most recent first, reverse chronological. And they have their own tailored algorithm that tried to kind of dampen down the extreme voices to create more diversity and make it hopefully a little bit less toxic. And this took it took about a year. The researcher who led this project said it was like running a startup for a year and they had to hire a software engineer and everything.

11:45So this was just like such a huge project to do. And one of the reasons that they were able to do it was because they knew because it had already been, you know, their research methodology had been approved. they knew that it was going to be published either way. Yeah. And for researchers, being published is a big part of getting more funding and things like that. And so do you want to know what the results were? Yeah, I kind of hope it's a negative result just to prove the point. A few things. So they found that, yes, these engagement kind of based algorithms do bump up your prevalence of moral outrage and political content.

12:21So they put the algorithm in. What were they measuring? They were measuring what users saw and how politically sensitive or divisive that was. So did they see more kind of toxic posts as a result of the algorithm, depending on which algorithm they had? What did they see? So what did people see? How prevalent these different types of posts were? And indeed, yeah, they found that if you're trying to maximize engagement, you do end up with seeing a lot of divisive and toxic content. And then the other thing they found was they were looking at how much that affected people's kind of perceptions of social norms and what the world thought.

12:57You know, if you're seeing all these outrageous posts, you know, what does that make you think about the people who are around you? And it did increase perception that there was like partisan division. But actually people underestimated how prevalent toxic content was. possibly because the researchers said when people saw these very like awful toxic posts they also saw people condemning that and criticizing it and commenting yeah as well as looking at like yeah more engagement is more of certain types of posts potentially negative posts they also actually asked the people yeah how did that influence of it yeah how they viewed their social world around them and so the other thing they did was ask them did surveys as to how much they were enjoying the platform and here is what I think is the really good news of this study is that people reported um equal or even greater enjoyment of the platform when they had this algorithm that was deliberately designed to reduce toxic content so they were able to make a social social media platform that was still engaging and enjoyable but wasn't didn't have that kind of like oh there's a car crash but I can't help looking at it element that leaves you feeling a bit dirty once you've been like scrolling for a while so it does it does seem like you could make a platform that would actually be a just a much nicer healthier place to hang out and that is possible that world is out there but I feel so cynical that I'm like I don't think that doesn't seem right that's not how social media works but maybe and the other thing though that was interesting which may not have been included if it wasn't a registered report, was a kind of negative finding in that it didn't, seeing all of this toxic content didn't influence people's own behavior.

14:41So that was one of the questions was, if you are, you know, at an end of this fire hose of like people being angry and political and, you know, moral outrage, does that also make you behave in that way and they found that it didn't which is kind of good news how how did they how did they work that one out how did they um well they looked at the posts that people were making themselves and all of this they did like content analysis they had nlms and trying to figure out you know like what was what was the main point of each post that people wrote and and you know particular keywords and things that would be associated with more divisive language and so yeah that was something which the researchers said was kind of a negative finding like you would think that people with themselves maybe become a little bit more riled up and toxic themselves if that's like post to match exactly but that did not happen did the people volunteer for this they paid them blue sky oh they paid them but they were real users so that was another thing actually that came out of it was in the peer review process they had been planning to use like you know when you just get volunteers like students like hey come in and do this study and the peer reviewers suggested that maybe they should use real users who already kind of have a vested interest yes it is their real life their real world online the results are likely to be more credible as a result of that and that was something that peer reviewers suggested and they were able to work in so yeah basically you're collaborating with your peer reviewers before you've even done the experiment exactly and we we mentioned that this has been sort of a topic of discussion for for ages why has it taken so long you know it's 2026 and this is the first one of these that they've done so it was 2023 that nature first started allowing them okay and then i guess you know it's not that crazy that a study like this would take you know i think it's the 2024 election that they were looking at and then they did all the analysis and they had to do all the prep for that so um just you know science science takes a while um but yeah you're right in that nature was certainly not the first journal by a very long stretch to have these registered reports what's changing in nature what's in this editorial is that actually it's being expanded now so this particular example with the blue sky study was one of those classic ones where you're testing a hypothesis but nature is now saying that they'll accept a wider range of studies like ones where you're comparing two different scientific methods or you're just collecting a huge data set on behalf of the scientific community kind of different types of studies and in all the fields as well initially it was restricted to social science and behavioral science and I think neuroscience now it's going to be for all the different fields that nature studies so if you out there have an idea um basically it can be not anything but there's a lot of different things uh different types of study that can be registered reports now and it's it is a slog right like there's two different different separate rounds of peer review and it can be quite intensive but um more peer review that's what everyone's really asking for out here how long have people been saying i wish there was just more can i peer review a bit more please that'd be great um but you know it makes it really does seem to make studies more rigorous and yeah so i think the results really do speak for themselves so it's a it's a really good practice and let's hope we get more of them and there is an editorial on this very topic which you can read on nature.com or we're just going to put a link to it in the show notes for your convenience along with a link for where you can sign up to the nature briefing where you can get stories like these and other ones directly to your email inbox and I think that's it for today but don't forget if you want to drop us a review, some stars.

18:10Tell Nick and Ben how nice it was to have Lizzie and Sharmini step in for a briefing chat. Yeah, big us up, big us up in the comments, give us some stars and see you next time. Bye-bye. While every other channel is fighting for your customers' attention, podcasts are where they've already given it. No one accidentally listens to a podcast for 45 minutes. They choose to be here. They trust the voice in their ears. And when that voice talks about your brand, it doesn't sound like advertising. It sounds like a recommendation from a friend. Acast gives you that trust at scale, digital precision, post-read authenticity, and performance data that proves it worked.

18:48Don't fight for attention. Buy it with Acast. Learn more by visiting acast.com slash advertise.