AI discrimination

0:00This BBC podcast is supported by ads outside the UK. I am Nina Khrushcheva, the great-granddaughter of Nikita Khrushchev, the leader of the Soviet Union in 1962. And I'm Max Kennedy, the nephew of U.S. President John F. Kennedy. We explore what was a terrifying moment in history. The story of the Cuban Missile Crisis. How close the world came to nuclear war. And what they had to do to pull it back from the brink. The bomb, Kennedy and Khrushchev. All episodes available now, wherever you get your BBC podcasts.

0:39Welcome to Tech Life on the BBC World Service, the programme about tech and how it affects your life. I'm Chris Vallance and this week we'll be looking at how artificial intelligence can discriminate against us. I'll be finding out how that happens. That's in a moment. And on the other side of the coin, could it be AI to the rescue? A computer model that predicts the behaviour of people who go missing might help emergency services find those lost in the wilderness. And TechLife's gaming expert gets hands-on with the new Nintendo Switch 2.

1:33As a Tech Life listener, you'll know that artificial intelligence is being integrated into almost every area of life, in business, in public services, and in the tech we use every day, from phones to cars. We've reported on many of the benefits it can bring, but we've also alerted you to some of the downsides. One downside is AI discrimination. We're used to human discrimination, racism, sexism, religious bigotry, but could machines have inherited our prejudices? It's an area that Dr. Jen Schrader knows well. She's a sociologist at the Paris Institute of Political Studies. Now she's leading a team from an independent international expert panel that will study how AI could affect people's chances to get work, start businesses, and generally be included in the economy.

2:25So what is AI discrimination? We all know a lot about what we've heard in the news about artificial intelligence, which AI is short for. But when it comes down to how AI works, it tends to make very biased decisions about who gets a job, maybe a loan or housing, because the data that it's trained on isn't really, you know, allowing it to base decisions on merit, but really on patterns that either reflect or perhaps reinforced inequalities that already exist, right? And And then those get transferred online. And so it's really it's learning from examples that are found on the Internet. And that Internet data contains biases, partly because it's on the Internet, but also because it reflects the biases in society.

3:26Is that basically the picture? I love how you summarized that, right? Because sometimes we might think, you know, AI is just out there and it's magic, right? But it's really math trained on data. And data inherently has those biases. So my research that I've been doing over 15 years has looked at who is actually creating the online content that now is being used by AI. And what I found even 15 years ago is how people who have higher levels of education, in particular, are much more likely to have their lives, their experiences, their views, their opinions posted online. And now we have all these what we call AI training data sets that are being culled or amassed or collected from this existing data, which is already skewed toward the elite.

4:32So can you give us some real examples? Where do we see these biases occurring? Yeah, so I think one of the most common examples on the job application process is that because there are algorithms screening through resumes or even if someone just enters in their information through a job application platform, even someone's name, right? So if it's a more white sounding name, for example, often those applicants will not be as favored as people who have different sounding names or who have an education level or a school and have attended a school that is more elite. The algorithm doesn't even allow someone who may not have that type of elite background or schooling to come forward and participate in an interview because they've already been screened out.

5:32And I think it's that invisibility that happens that is so common. And we also see that as well, very common in the finance sector, when people try to get loans, maybe they live in a low income zip code area, so they're not even considered, or if they're trying to get just basic credit, people who, you know, may be deemed more risky because of, you know, their financial behavior may be different. That affects people on so many different economic levels, which is why I'm really interested in kind of looking more broadly at some of these impacts. Is there also bias against people in the global south, if you like?

6:15Because a lot of the data that's generated is generated from technically advanced nations, which tend to be in the global north. We definitely see bias toward people from the global south. One persistent problem with the Internet in general and the data that it's drawing from is it tends to draw from more dominant languages. And so when, for example, chatbots for customer service may not recognize accents differently, may not recognize more working class or global south dialects. So it may be something as simple as people's concerns not being addressed through a chatbot to more extreme examples, once again, of bias and hiring people not recognizing, for example, if someone went to school in the global south, that institution may be ranked much lower.

7:12I think the other area that is really key to thinking about the role of the global south in AI is that there's so much work that needs to be done to keep these systems going. So we're familiar or more familiar with the massive amounts of electricity that AI systems need. But what I think people may not be as aware of is the massive amount of real human work that goes into building these AI systems. Right. So we tend to think that, again, it's all robots. But the reality is there are people doing piecework, particularly in the global south, to make sure that all of the data is constructed in a way that makes and powers these AI systems.

8:02And they are getting paid at a dramatically low rate. Yeah, the large outsourcing industries, aren't there, essentially for doing things like tagging images. You know, if you want an AI to understand what a tree is, you have to show hundreds of examples of trees. and that requires a human being, someone who knows what a tree is, to sort of tag the data manually. I mean, I guess there are kind of concerns around that as you've raised about the amount that these people earn and I suppose you also raise, I guess, a question about whether the fruits of the development of AI are distributed equally.

8:39There's a sort of bias in that way. Absolutely. The companies and the corporations that are benefiting the most from AI, especially the development of generative AI platforms, are based in the global north. Again, they're using work from the Global South, but a lot of those companies where these workers are doing the type of tagging that you mentioned and the piece work, really, piece by piece, data point by data point, they're often companies that just really come and go, where workers aren't getting a living wage, they aren't getting health insurance, etc. Meanwhile, the headquarters are largely in the global north and the workers who really power AI systems aren't really benefiting from the massive explosion of attention that financial systems have placed on investing heavily in AI.

9:40But it's more investing in the global north than the global south. Can we get rid of the biases in AI? Getting rid of bias, completely getting rid of bias in AI means that we need to completely get rid of bias in society in general. And I don't say that lightly at all. In fact, getting rid of bias in AI is not a pure technical solution. Now, while transparency would be helpful, while understanding the code that's being used would be useful, until we have a more representative data set to begin with, which means we've addressed inequalities around literacy and access and education, it will be impossible for AI to be completely without bias.

10:29My thanks to Dr. Jen Schrader of the Non-Profit, Non-Governmental International Panel on the Information Environment.

10:59And what they had to do to pull it back from the brink. The bomb, Kennedy and Khrushchev. All episodes available now, wherever you get your BBC podcasts.

11:23You're listening to Tech Life on the BBC World Service with me, Chris Fallons. Last week, we covered another problem with AI. It hallucinates, it makes stuff up. We asked if you'd found any examples. Alice McVeigh is an author who lives in the UK, and she heard the show. She emailed us this voice message explaining her experience of using ChatGPT. I put in my own name and learned to my delight that Alice McVeigh, that's V-E-I-G-H, only one of us, had written a brilliantly successful series of children's novels in the 1940s. A, I don't write children's novels, never have, never will. B, I wasn't born until the 60s, so no, I'm not impressed.

12:09Not impressed, I do not understand it. Thanks for listening, bye. Alison McVeigh there on her experience of AI hallucinations. We've also been asking you for examples of tech you simply can't do without. Rachel from Exeter in the UK didn't leave her surname, but she did WhatsApp us to say she thought last week's episode was great and she can't live without Instagram at the moment. Thanks for getting in touch, Rachel. We're loving all your examples you're sending in of tech you rely on every day. If you haven't done so already, you can send us a voice note or a text message on WhatsApp. Our number is plus 44 330 1230 320.

12:49or send us an email to techlife at bbc.co.uk. Still to come, all you need to know about the latest gaming console about to hit the market.

13:14well we've left the warmth of the studio and i'm now outside the global headquarters of tech life that's the BBC in Glasgow in Scotland it's a it's a typically chilly late spring day looming grey clouds overhead a sort of colour of wet cement and in the distance I can just see the hills and beyond the hills there there are mountains and if you follow the local news every so often you hear about people who get lost big effort for a mountain rescue from the police to try and find them but maybe artificial intelligence can help can help predict where people who get lost might end up and that's a project that my next guest Jan Hendrik Ewers is leading the research on at the University of Glasgow using artificial intelligence to help in the search for missing people.

14:15Tell me about how your system works. So our system really works by approximating how a missing person might behave right so we're working with aggregate data over hundreds of thousands of cases and we can make some good statistical assumptions based off of this data. We then have a simple agent that traverses over the landscape and if we run that a couple of million times we end up with a sort of a heat map that tells you where the probabilities are perhaps higher of finding the missing person. So how do you figure out where the missing person might be? What enables you to predict where they might be?

14:54So what we're looking at is how long this person or this person within a category has moved for, how far they have maybe dispersed from their target destination, where they may have ended up. And through this, we can then kind of prime our model to behave like this person. Okay. So for example, me and you are roughly the same age range. If me or you were hiking out in the hills, we would probably behave in a very similar way if we get lost. Okay. So we might try and self-rescue and that self-rescuing might involve, you know, seeing a building in the distance moving towards that or seeing a road and traversing that whereas a younger person maybe up to like five years old might not have that intuition and internalized and then kind of another example if someone was fell running for example they're moving much quicker and maybe moving along better paths and if they were a ski mountaineering as many people do in the highlands again they're moving very differently across the landscape so we can encapsulate that within our model and fine-tune that to the person that's gone missing.

16:00So the AI effectively knows how different types of lost person might behave and it knows that because you've got lots of data from previous hunts for missing people that it's trained on. Exactly so you know the more data we have the better the model is. Right now you know all the The data sets kind of agree that there's roughly 50 unique behavioral profiles that we're working with. But if we can collect more data, we can fine-tune that even more to the person at hand. Have you successfully tested this out? I wish. It's all in the simulation world right now. But what we're doing is validating the model.

16:40As you can imagine, if we were to deploy this right now, there's a lot weighing on this. If it's wrong, then you're potentially endangering someone even more than if it was right. Data collection is a big move for us. Next, we're going to apply for some grants at the University of Glasgow to develop the system even further and work with the mountain rescue teams to do some data collection. Likewise with the Police Scotland Air Support Unit. We saw the helicopter fly by a second ago. They're very interested in this kind of technology. The helicopter is extremely expensive to keep up in the air.

17:10and if they can shave seconds off, not only does it save lives potentially, it can also save a lot of taxpayer money. Because I suppose the time is critical, isn't it? Because in Scotland, the mountains aren't as big as some other parts of the world. But one thing we do have is a lot of pretty wet, windy weather, the kind of thing that if you're out in it for a long time, you can suffer from exposure. Yeah, so every minute is really critical in a search and rescue scenario. If someone's hypothermic, they don't have any ability to really generate any body temperature, and they generally might have seconds or minutes left.

17:48So getting to them is really paramount. You mentioned helicopters. Of course, one thing that AI can do is direct air vehicles, including unmanned air vehicles. Yeah, exactly. So this prediction of where people are, that's kind of the first part of my PhD research. The second part is using AI models, so using deep learning from this prediction. So we have a heat map over an area telling us where people might be. We can then use AI to make informed decisions on where to fly a drone, their efficiency over an area to search the area. And of course, drones can deliver things. And we've seen in other countries that drones have often delivered medical equipment, defibrillators, that kind of thing.

18:32So that presumably is an option for rescuers to get some help to them before the people arrive. Yeah, 100%. I mean, if we can get these people some space blankets or silver tinfoil blankets, that might be a chance of prolonging their time exposed to the elements. Well, look, I'm going to make a suggestion that we change location because I want to ask you some other questions about different settings for your tech.

19:10Well, we've just moved a little way. We're outside the Science Centre, just across from the BBC, and there's a little play area for children. You can probably hear the sound of them. There are lots of different kind of musical sounding, I don't know what you call them, sort of toys, that they can play with out there. Could this be adapted? Because one of the things people worry about a lot are children getting lost, but they're not often lost in the wilderness. They're usually missing at parks and places like that. Could your system be adapted for that? You would require to do a lot of data gathering, which is very difficult with children, obviously.

19:49But if you can understand how children behave under different scenarios, then that's something that we can 100 % model. In a large park, for example, the child might be moving towards other obstacles or other objects, such as an ice cream truck and that, which is intuitive, but it's hard to model that without real-world data. Around buildings and that, it gets a little more difficult because one thing that our model does is work a lot on line of sight. Within a larger city, you have obstructions in terms of buildings. The stats also do show that kids in the wilderness tend to hide quite a lot, and that's something that's quite difficult to model.

20:26As you can maybe tell from my apprehension, it's quite a hard topic, but I think with more data, we can definitely work towards it. And what about people who are in some way cognitively impaired? Maybe they're in distress, maybe they're suffering from a condition like dementia and they get lost. Again, how well can the model cope with that? So, dementia scenarios is a very big one for the Police Scotland Air Support Unit that I've been working with quite a lot. If a person with dementia leaves the home that they're in, they have a hard time navigating back. and if it's wintry with an old person, that's really, really dangerous.

21:06So we have a lot of data on dementia cases. We can take the psychological profile and adapt that to elderly people with dementia and the model is set up to work just as well. How easy will it be to deploy the system in other countries? That's a great question because I think it would be very easy. If we wanted to apply this to southern Germany, for example, we would just have to collect similar data points from their scenarios or potentially Australia or Japan. It really doesn't matter where in the world you are, this model works more on behaviour rather than on terrain. That's Jan Hendrik-Ewers of the University of Glasgow on how AI tech could help find the lost.

21:55and that we found our way back to the studio time to bring you news of a hotly anticipated sequel it's news to delight gamers and maybe send a shudder of fear through parents who might end up having to pay for it nintendo are about to release the next version of the nintendo switch Switch console. TechLife's Tom Gherkin is one of the fortunate few journalists given early access to try it out. So Tom, what is the Nintendo Switch 2 like? It's really, really interesting. Now the original Switch, its whole hook was it's a hybrid kind of console. That means it's portable like a Game Boy was, but it also hooks up to your television like a GameCube or a PlayStation.

22:41It can do both and the nintendo switch 2 is the same but of course more powerful the original switch came out in 2017 it sold 150 million units yeah it's the third best-selling console in history it did incredibly well the switch 2 has got a lot to prove to show that it can be on the same level as that it's super powerful it's got so much under the hood when it's in your hands it feels great It's light, comfortable. The screen is so bright in your eyes. It feels good, and it looks good. Of course, one of the things that Nintendo, sort of sets Nintendo apart from other consoles has been its controllers.

23:20With the Wii, you had those sort of innovative controllers that track the motion of your arms, so you can play games. Anything new on the Switch with that regard? The Switch 2 does have something sneaky underneath the hood. Actually, it's got a lovely thing with the controllers. On the original Switch, they slid upwards. Now they're just magnets. They pull off the side of the screen. which is very nice, actually. You can just yank it as hard as you can and it'll come off. But the clever thing it does is they put a little optical unit inside them, which means that both controllers, one in each hand, can work like mice.

23:48So you can be playing a video game and then slam it down on the table and get more precision, or you can play games that were made for PC originally on a home console, which is actually really clever, because if you think about that market, there's so many PC games that need to be played with a mouse. They need that kind of precision. One of the things that's attracted people to Nintendo has been the price. Cheaper than its two big rivals. How does this work out for price? The Switch 2 is, well, I think actually cost around the same as a PS5 or even an Xbox. So that is really expensive. They've gone for a much more premium package.

24:27I spoke to Chris Dring about this. He runs a newsletter, The Games Business, where he looks specifically into hardware sales. and, well, he had some strong opinions about it. What will be key is whether or not Nintendo can reach the family audience that's so important to them, that they're all about. Because to begin with, everyone who buys this will be Nintendo fans, people desperate for a new machine. Will families see this as a reasonable price? Perhaps in a different market, maybe. I feel with the economic circumstances that we will find ourselves in, it might be a bit too far. Well, as Chris said, finances for a lot of families, very stretched.

25:01But the console itself isn't the only expense, is it? A big expense is the games. In different markets, the prices are a little bit different, but in dollars, it's$5 to$10 more for the most premium game Nintendo have advertised. That's a physical copy of Mario Kart World, and presumably a game that people would rather want with their system. We don't know what their other games will be priced at. They've announced one price for one game. They've announced a different price for Donkey Kong. And those two prices are for the physical editions. It's cheaper yet to buy the digital editions. I spoke to Ellen Rose, the co-editor of Outside Extra, and she told me about the price of those games.

25:37We are in a cost of living crisis and games, as much as I love them and sometimes they feel like my entire world, they are a luxury. They are a luxury. And I think, you know, they've got to be careful, like not to price people, too many people out. Ellen Rose there. So how does the Nintendo Switch compare with its rivals? The Switch 2 is much more powerful than the original Switch in terms of what's behind the scenes, in terms of what it can display on its screen itself, 1080p compared to the original 720p. When attached to a television, it can now display in full 4K with a fan helping to power that.

26:16But what you're seeing on screen on a Switch 2 is certainly not anywhere near the visuals of what you can see on a PlayStation 5, let alone a PlayStation 5 Pro. The thing to remember is what the name suggests and what I keep saying. You can take that Switch with you. You can take the Switch 2 with you. And it means that though games will certainly look better on those rival consoles, with it having more power, it might well have a lot of the same games as the rivals do. But rather than just being able to play it on your television, you can also play it when you're on the bus, when you're out and about.

26:48And that's a very, very, very big difference. and Nintendo clearly banking a lot on that portability. TechLife's Tom Gherkin there. Thanks very much, Tom.

27:02Well, it's game over for this edition of TechLife. Remember, we'd still love to hear about the tech you simply can't do without, whether it's a supercomputer or an egg timer. Our email address is techlife at bbc.co.uk or you can WhatsApp us on plus 44 330 1230 320. Please include your name and where you live. Today's Tech Life was produced by Tom Quinn, edited by Monica Soriano and presented by me, Chris Vallance.

27:47I am Nina Khrushcheva, the great-granddaughter of Nikita Khrushchev, the leader of the Soviet Union in 1962. And I'm Max Kennedy, the nephew of U.S. President John F. Kennedy. We explore what was a terrifying moment in history. The story of the Cuban Missile Crisis. How close the world came to nuclear war. And what they had to do to pull it back from the brink. The bomb, Kennedy and Khrushchev. All episodes available now, wherever you get your BBC podcasts. what you guys 감 Michelle