What's In A Face: How technology uses our faces

0:00This message comes from AT &T. Staying connected matters. That's why AT &T has connectivity you can depend on. or they'll proactively make it right. That's the AT &T Guarantee. Terms and conditions apply. Visit att.com slash guarantee for details. This is the TED Radio Hour. Each week, groundbreaking TED Talks. Our job now is to dream big. Delivered at TED conferences. To bring about the future we want to see. Around the world. To understand who we are. From those talks, we bring you speakers and ideas that will surprise you. You just don't know what you're going to find. Challenge you. We truly have to ask ourselves, like, why is it noteworthy?

0:44And even change you. I literally feel like I'm a different person. Yes. Do you feel that way? Ideas worth spreading. From TED and NPR. I'm Manoush Zomorodi. And today on the show, what's in a face? do you have any idea how long a face will stay in your mind like once it's in there is it there forever i actually don't know but i mean as an example i have come across faces that i remember seeing from when i was younger than 10 so oh wow yeah so i have come across like school teachers or classmates or people i remember in my neighborhood when i lived there as a child and it's a lot harder to explain where I've seen them.

1:33Oh, so you're saying it's more just a sense of familiarity? Yeah, and it's almost like this intuition or this kind of ping and I know, like, I am so confident I've seen that face before. That happens instantly. This is Yenisa and she is what's called a super recognizer. So super recognizer, firstly, I find that term very cringy. Oh no. Basically what it is, is super recognizers are people who are maybe on the top 1 % to 2 % who are very good at remembering faces. And I'm told that it's about 80 % or so of the faces that we see, we remember. So it's very high in comparison to the average person.

2:23Yeni knew she was always good at recognising faces. But about five years ago, she realized that her ability was really unusual. I just turned on the TV and I happened to come across this show about people with different abilities. And I saw this gentleman was based in the UK and he was a police officer. And they made him do a test where he was standing in a really big train station with lots of people going through. You've got to try and find my four actresses who have hidden themselves away in the crowd or be wandering around. And he was shown photos of, I think, a handful of different faces, and he had to pick out the people that he'd seen in the photo.

3:09But the trick was they would, you know, put a wig on them or they'd be wearing a different hat or glasses. Black leather jacket, blue jeans. Brilliant. Absolutely brilliant. And he was able to, I think, pick them all out. When I saw that, I just got goosebumps and I just had this really strong confidence that somehow I'd be able to do those tests. Is it the lady in the black jumper, cream top and blue jeans? Brilliant. Yeni took some tests online and she did really well. So she got in touch with a researcher in Australia where she lives who confirmed that she was indeed a super recognizer. I ended up visiting their lab in Sydney and they put some sort of sensor detector so they saw where my eye movements, how it worked when I was exposed to a face.

3:58And it's not that I pinpoint on one feature. I would not focus on the eyes or nose or mouth or the shape of the face. It's just the whole, the face as a whole leaves kind of an imprint in my head. so have you ever found like your ability useful then like other than like you know fun party trick yeah I mean when I was in uni I worked at a clothing store and we I mean I ended up catching a shoplifter because we had a team meeting and there was a particular shoplifter who would repeatedly steal like the highest priced item in the store and they had this CCTV footage of her and it was just this really grainy black and white photo and they showed it to us during our team meeting in the morning and they stuck it on the wall and I looked at it and I was like all right I don't know if I'll be able to catch that person I didn't really think much about it.

4:57An hour or so into my shift that exact person walked in and I just knew straight away it was that person even though the photo was really grainy I just knew. What did you do? We had security guards in our store, so I just had to, we were wearing walkie-talkie type of things, and I just told them, yeah, she's here. She's just walked into the store, so maybe you guys should go and have a chat with her. They ended up catching her, and then they had to call the cops in, and so that was my one crime-fighting experience. From what I understand, a lot of super recognizers work in or work with law enforcement is that, or in some kind of security capacity.

5:37Is that not something that you sort of thought, well, you know, I could actually make money off of this? I mean, at one point I think I did consider it, but I think like it's still very new and the research in this area is still developing. I know like countries like the UK, like their police enforcement have started recruiting officers who have that ability, But I always thought it was a little bit creepy that I, I don't know if creepy is the right word, but I always thought that, you know, it would be perceived as being a bit like I was a stalker or something. Yanni sees and processes faces in an extraordinary way.

6:22But technology is quickly passing her superhuman abilities. Most of us already use facial recognition to unlock our phones and tag people in photos. Governments, law enforcement, and companies can use cameras and algorithms to collect and identify us. But where will we draw the line? Today, what's in a face? ideas about the promise and peril in turning the human face into an everyday digital tool for anyone to use.

6:57I was actually literally just today talking to a facial recognition vendor. What about? So they're in the middle of filing a patent where artificial intelligence or machine learning system will look at your face and determine how you feel. This is Bloomberg tech columnist Parmy Olson. Which will allow them to analyze the faces of stock market traders and bond traders to get a sense of where the market is moving based on the emotions shown on the faces of these traders. In a way, that sounds maybe a little bit innocuous, if not a very odd way and potentially a disastrous way to determine where the market is going.

7:41I don't know that that would work. But I think the question is, well, what happens when all these different vendors and stakeholders have access to our faces and can maybe get to a point where they want to start drawing inferences about us based on our faces? Now, even if making market decisions based on the minute facial expressions of day traders sounds far-fetched, Parmy says the basic technology behind it is not. So these systems are essentially trained on millions and millions of actual photos of people. And the more data it has, the more accurate it can get. And I think the concern is that this technology is so widespread and so actually not that difficult to build.

8:36Some of the technology is open source. There are billions of images of faces on the Internet. It's relatively cheap to do it. Yeah, and it's so cheap that you have written about how this software is now used pretty widely in retail, even gas stations, convenience stores. Yeah, I think the main reason that retailers want to use facial recognition in their shops is to actually look for unwanted individuals. So there was a chain of stores in the UK that hired a security system, a facial recognition security system to be installed. Let's go to Aylesbury, southern England, and to a Budgen store. Parmi Olson continues from the TED stage.

9:25Now, Budgins in this particular town has been having trouble in the last few years with people coming in and stealing meat from their refrigeration aisle. So a year ago, they installed some new technology from a company called FaceWatch. And through their usual CCTV cameras, FaceWatch's computer and software would scan every single face that came into the Budgins and match it up against a watch list. Now, this watch list is processed by FaceWatch and Budgeons can also add to it if they suspect someone of stealing. And I called up the Budgeons and asked how they thought it was working. And the staff member there told me that his phone gets pinged up to ten times a day with an alert to say that someone has walked in the store who matches the watch list.

10:16So if that happens, he might call the police if it's an aggressive person, or he might just say, hey, you're on CCTV. And actually, it works pretty well, he said. He thinks it's helped. But there's a few concerns about FaceWatch. So first of all, to get on the watch list, you don't have to be arrested, and you don't have to be charged by the police. There's no real legal due process. And the other thing is that to be uploaded onto the servers of FaceWatch to be on a watch list, you can be on it for up to two years and you won't be taken off. So this is a security company that relies on watch lists.

10:57And anyone with clearance, I guess that could be a store employee, they could add someone to the list and then what, that information is shared? Yes, that's right. Each store would have their own watch list and they would share the watch lists with each other. So you'd have an even bigger watch list And yes, the people who are on these systems, this is a private system. This is not something where there's a court order or a warrant or anything like that. This is totally done privately by a business. It's their own private watch list that they've put together. So, Parmi, one of the underlining problems with this kind of mass surveillance is that sometimes the algorithms are wrong.

11:39Right. When I talked to one of the people who worked at one of these stores, they said that about 25 % of the time the system was wrong. So they would get the alert, get told that person had walked in, then they'd walk around and they'd see actually it wasn't that person. And so they had to really be careful to trust that the system was correct in real world when, you know, the lighting isn't that good and the image might be a little bit grainy. Not surprisingly, the system was getting it wrong one out of four times. Wow. And I can imagine someone thinking like, OK, well, it's a grocery store.

12:16But if you're talking about a situation involving law enforcement, that could get quickly escalate, I would think. Yeah. So a police officer has a body cam with facial recognition or a camera on their van with facial recognition and they detect someone. And if that person has increased melanin in their skin or they're black, essentially, then it is more likely to make a mistake in identifying that person. And the reason is that the databases that these facial recognition models are trained on typically have way more white people than black people. And so the system just isn't trained enough on black people, so it doesn't identify them properly.

13:01It makes more mistakes. And that has happened. And it's probably going to continue to happen, too. When we come back, what are we willing to stomach in a face-tracking-filled future? On the show today, what's in a face? I'm Manoush Zomorodi, and you're listening to the TED Radio Hour from NPR. Stay with us.

13:33This message comes from the Michael J. Fox Foundation. Did you know that every six minutes someone in the U.S. is diagnosed with Parkinson's? It's the fastest growing brain disease in the world. Chances are you or someone you know is impacted by Parkinson's. The Michael J. Fox Foundation is on a mission to find a cure. Now is a critical time for Parkinson's research, and your support will help drive progress toward a cure. Get involved today. Visit michaeljfox.org slash get involved. This message comes from NPR sponsor Informatica. Everybody's ready for AI, except your data. Informatica offers the only end-to-end AI-powered cloud data management platform that connects, manages, and unifies your data across virtually any multi-cloud or hybrid environment.

14:21Informatica accelerates your AI readiness and scale at a pace that allows you to pay for only what you use without compromise. Get your data ready for AI at informatica.com slash AI. Informatica, where data and AI come to life. This message comes from NPR sponsor, Walmart. Did you know that Walmart is one of Fortune's 100 best companies to work for in 2025? It's no surprise. Walmart is investing$1 billion in training and development to help associates build lasting careers with or without a degree. In fact, 75 % of Walmart management started in hourly roles. With benefits like health care, paid parental leave, and tuition support, Walmart isn't just a place to work.

15:09It's a place to grow. Learn more at walmart.com slash spark. Before we get back to the show, I want to ask you to please consider becoming a member of TED Radio Hour Plus. You'll get extra advice, stories, and expertise from TED speakers every other week and no ads ever. And you'll be supporting public radio. Listener support is crucial to keeping us going. Go to plus.npr.org slash TED or give it a try right in the Apple Podcasts app. It's the TED Radio Hour from NPR. I'm Manoush Zomorodi. And on today's show, what's in a face and how companies will use them to make money. We were just talking to Bloomberg columnist Parmy Olson.

16:03Each one of our faces has a face print. It's like a fingerprint, except it's a string of numbers that corresponds with the image of our face. And there are lots of databases of people's face prints out there on the Internet. And even if they aren't identifying us by name, our faces can be tracked, categorized, and profited off of. For example, a few years ago, Walgreens installed cameras in some of their stores that identified shoppers by age, gender, and then displayed targeted ads. Yo, new Sprite? Like, do you look male and around 20 years old? Buy a Sprite. Buy a Sprite. New Lipton green iced tea.

16:4450 and female, maybe a green tea. Your Subaru has a driver monitoring system. Carmaker Subaru's new vehicles use facial recognition tech that create user profiles and even alert a driver if they seem distracted or tired. And then there are casinos. So there is a casino in London which has facial recognition cameras dotted around all the different rooms. And it uses that so that when high rollers walk into a certain room, then the staff get an alert on their phone which gets sent to an encrypted chat app they use called wicker and and then they notify each other like oh so and so this hostess should go up because that's the high rollers favorite we know that they like this particular type of food and this particular type of drink and so they can actually provide a better service and they call it their white glove service and i remember asking the head of security well are the patrons actually a little bit put off by that?

17:47And he said, not at all. Not a single one thinks that that's even the slightest bit creepy. They just see it as part of the service. It's what they expect. And I think it's a nice little allegory for just how this kind of surveillance is going to eventually come to serve the rest of us as consumers, that the convenience will ultimately be something that we just take for granted and we won't worry too much about the price that we're paying with our privacy. And I think that is just the way it'll go. I mean, I guess I can see the appeal if you're opting into a luxury service. I mean, that just feels very different than being tracked while you're walking down the street or going into a store.

18:29I just don't think that that is something a lot of us would sign up for. Yeah, I would say that facial recognition definitely has become a controversial subject. And I think that's made it difficult for brands, for advertisers who might be able to benefit from using it to target their brands at people. They've had to take a step back. Yeah, there are a number of retailers now, I think this is what you're referring to, who are facing lawsuits for surveillance, for gathering data on their customers without consent. But really, like for now, it just feels like whack-a-mole because we have not figured out as a society what we think is OK and what isn't OK when it comes to face tracking.

19:13Yes, that's right. Companies who develop these kinds of systems need to be very careful. There needs to be more ethical oversight of how these systems are developed. And right now, that's only going to come from regulation, which is a couple of years away, yes, but also campaign groups. And there are some really good civil liberties groups in the United States and Europe who are really keeping an eye on this and just helping keep companies on their toes. If there wasn't the amount of kind of upset that had been created around facial recognition, I think there'd be a lot more advertisers using it right now.

19:43But because people have really rung alarm bells about it, then I think that's made companies really just take stock and just sit back and just say, okay, let's just be a little bit more cautious about how we use this. And I think that's a really good thing. Given that there still aren't major regulations out there around facial recognition, I mean, are we just at the point of no return here? I mean, can't go back. There definitely needs to be more laws and regulation, but we have sort of gone past trying to force companies to design algorithms in a way that are safe and ethical because the algorithm's already out there.

20:23But there is a law coming from the European Union called the AI Act, and it actually bans all forms of facial recognition for surveillance by police unless it's for trying to combat terrorism. So that's a pretty blunt rule. And I mean, that's going to be the first kind of comprehensive legislation around the use of artificial intelligence algorithms. I think the issue with it is that it is so broad. It's not just about facial recognition. It's about all forms of AI. So whether that's recommendation systems on social media or facial recognition, you know, it covers a lot. And so enforcing it, I think, is going to be difficult.

21:09You know, one of the reasons all this tracking is possible is because we have accepted the idea that cameras are in our pockets all the time. They're on our doors. They are all over public spaces. And we're OK with it. We are OK, largely, with being surveilled. Yeah. There's something like 20 million homes in the U.S. have a video doorbell. The thing about ring doorbells that I think is really interesting is that actually the studies that have been done about just how effective these cameras are in reducing neighborhood crime show that the evidence is really flimsy. There's actually not much evidence that they do reduce crime, but the big impact is on human sentiment.

21:56So the owners of these cameras feel a greater sense of security and a greater sense of control. But then on the other hand, we also collectively come to accept that our behavior is being watched. So, yeah, take that how you will. I think we are just an increasingly surveilled society. And I think people are just like slow-boiled frogs. We're increasingly accepting of it because it's just what's happening for better or worse. That's Parmi Olson. She's a tech columnist at Bloomberg, and you can see her full talk at ted.npr.org. And earlier, we heard from super-recognizer Yeni Sa, who works as a translator in Australia.

22:38On the show today, what's in a face? Often, to understand how technology will change our lives, we just need to watch a movie. like this one released in 2021.

22:58So The Champion was a film shot and made in Poland. So everyone's speaking basically Polish or German. This is Mike Seymour. He's a researcher at the University of Sydney and works in the film industry in special effects. And it's a great film. It's a true story about one of the first members of Auschwitz who was a boxer. Terribly moving story, but of course, only in Polish or German. And usually when there's a foreign film that wants to break into the English-speaking market, there are three options. Dubbing it, so we get somebody else to voice over a different piece of dialogue, but of course, the lips aren't right, so it looks kind of odd.

23:37Or there's subtitles. Or we have the new version of what we call facial reenactment. Facial reenactment. It's a new technique that Mike and his team used on The Champion. We got involved as part of a team to convert the entire film to English. So now if you were to watch the film in English, every actor speaks as if they'd been shot in English. The Dutch King of Warsaw, totaling almost a hundred fights in bantam and featherweight. So we've replaced effectively the actors' faces with their own faces, saying the lines in English. He looks more like a small rooster plucked off its feathers than a champion.

24:19Okay, so Mike, when I watch this English version, it's seamless. It's like their mouths, their faces, everything looks like it was originally shot this way in English. Is this common in the industry? Is it normal? Well, it's the first time anyone's done it in the world, but hopefully it's going to become normal. As Mike went on to explain, months after filming, the actors re-recorded all their lines in English as cameras taped their voices and facial expressions. And then through a process called neural rendering technology, their faces were replaced. So it looks like the film was just shot twice in two different languages.

24:58Come on, Jenny. Come on. The film industry is always pioneering new tech to trick our eye, to make someone or something look real. But over the last few years, Mike has been developing ways to use these techniques in our real lives. Yeah, could we take this tech and just sort of use it outside the film industry that fascinated me, where I say, well, hey, I don't want to get shaven and put on a suit for my important meeting today, So I'll just flip a switch and get digital makeup and I'll look a whole lot better and a whole lot smarter. And I would be able to say speak in Korean when I absolutely can't speak in Korean.

25:38And that, we hope, would facilitate much more genuine communications across cultural divides. Wow. You're saying that maybe one day if I have relatives all over the world who speak all different languages, But maybe one day we could do FaceTime and it would sound as though I was speaking fluent Swedish and they were speaking back to me. Well, they can't speak English, but that we would hear each other's native tongue and wouldn't know the difference. It would look as though I could speak fluent Swedish, but it wouldn't look like I was not myself. Yes, there is a lot of modern technology that's very sophisticated that would, we think, benefit from being able to have an extra layer of communication that you get from face-to-face interaction.

26:26We're kind of this nexus point where that's possible. Mike Seymour picks up from the TED stage. We're interested in being able to see if we can put a face on technology because how would you react when a computer reacts to you with a smile? Would a six-year-old learn maths better if there was a six-year-old teacher on the screen? What about if it was a slightly older version of herself? Would a grandparent having a cup of tea be more likely to check in with a computer system if they didn't have to log in and type? They could just talk to a virtual agent that actually was somebody from their past.

27:05This is what we're excited to explore with digital humans. Our ability to produce digital humans up until recently has been quite limited, but we're now seeing interactive digital humans starting to appear. The doors are opening, we are at an inflection point. We have this perfect storm of faster GPU graphics cards, new artificial intelligence, deep learning algorithms and great advances in game engines. It's an incredible combination of things coming together. This tremendous nexus of points is just providing us with an extraordinary opportunity of things that we can do. The important thing about this technology is that we can now use this to get these faces to work with us in real time.

27:47In other words, this is a really key point, the faces that we're talking about can talk, interact, and see us. Okay, putting faces on our technology. Tell me more about how you see this working and the reasons why we would want it. Yeah, I mean, there are a lot. Already in New Zealand, there is automatic sign languaging. So if somebody is speaking a digital human signs for the deaf community, you might have an assistant sitting in on a Zoom call that you can ask to help book future things, take notes, do stuff. In aged care, you could have an assistant that logs in with somebody each day and make sure that they're okay and are lucid and they've taken their pills, not to replace a healthcare worker, but just simply to make sure that they're okay and facilitate them staying in the home longer.

28:39And so in a world where we're saying, hey, even to use the phone, there are no buttons now. You have to swipe up, swipe left, do all this stuff. People are like, I have trouble with that. And so we could bring a face from their past that would be the one that they interact with that technology. You don't think that would be odd to someone that if you said, well, this is your sister. She's not actually your sister. She just kind of looks like your sister and she's going to help you use your phone. That, I don't know. That might freak me out. Yeah. You know, you've just touched on a really interesting point.