The Hidden Biases Built Into Every AI System | Dr. Joy Buolamwini

30:34And we know that there's more biases in these data sets against genders or nonconforming. So all of these different ways that the more data that gets collected on us can be used against us in ways that I think people don't really realize. Yeah, absolutely. And when I think about schools as well. Oftentimes we see AI being adopted within the school, yes, for educational purposes, but also at times for security purposes, right? There have been school shootings. It's understandable. People want to say, whatever we can do to prevent the next one, let's go for a tech solution as opposed to maybe a policy solution that could go further.

31:17And so I'm actually thinking about a teenager in Baltimore who was accosted and handcuffed by police at his school because he pulled out a bag of chips, Doritos. But the system they were using detected those Doritos as a gun. And now when you have a situation where supposedly there's a weapon involved, then it justifies the use of force. So I think it's also important when we're thinking about these spaces that are meant to be for learning, or you're even seeing the introduction of AI surveillance in spaces like hospitals. I was very surprised to learn in some instances, people have used facial recognition in ambulances.

32:03And I'm thinking about these are some of the places where you're most vulnerable, right? Like what opportunity is there for consent and so forth. And for someone that's listening that thinks, how could the AI make that mistake? So how could AI confuse or falsely predict a bag of Doritos for a gun? What's happening on the technical side that AI is making those inaccurate predictions? You know, there are glitches in the system. And because these are pattern recognition systems, sometimes it gets the pattern wrong. Very similar to was my face the pattern it expected versus the white mask. And then you add things like if you're doing surveillance cameras, you can have grainy footage.

32:46The lighting conditions can all change. There's so many variables that go into it. As a computer scientist, when you're thinking about computer vision, you think about illumination. What's the lighting looking like? When we test it for our own benchmark, sometimes we do the best possible conditions, right? You and I in a studio looking directly at the camera. This is optimal. We would call it a constrained environment. AI is in the real world. Bring in the rain, bring in the shadows, bring in the grainy footage. So when you have AI systems that have been trained in more pristine environments, now actually hitting the real world with edge cases that were never considered in the training data set and so forth, you can come up with these errors.

33:31But I think it's so important to also realize it's not just about the ways in which AI makes mistakes, because accurate AI systems can also be abused, right? Accurate AI systems can be used to track people that you might want to do harm to as well. So you have problems when there are misidentifications. And even if we had the perfect biometric surveillance, which we do not, right, we now then have a biometric surveillance state that would be the dream of many authoritarian governments. And even when you think about trading off these conveniences for our own privacy, and we hear about a future of wearables.

34:13And some of this does sound great, right? If we can avoid going to the hospital altogether or needing to call that ambulance because we have a device that lets us know your blood oxygen is dropping or your heart rate is beating abnormally and we don't have to head to the hospital in the first place, that is a win. That is an exciting future. Except for when those types of wearables malfunction because of the color of your skin or because of your gender. I'm sure you read that study too with the blood oxygen wearable devices that weren't working properly for people with darker skin, in particular black skin.

34:50And so it was showing that black skin patients had normal blood oxygen levels when they didn't. And then people weren't getting passed to the ICU that needed to go to the ICU. People weren't given oxygen when they needed oxygen. So there are all of these kind of insidious ways where bias can show up. And this is why I always say, if there's AI in your workplace, you are an AI company. So if you are a doctor now that has AI in their operating room, you have to understand all of the different ways that AI may not work for people because now you're at that touch point. Of course, it has to also go upstream.

35:27Who's building these devices? As you talked about from the design standpoint, what data went in to train them? And clearly not enough data that's representative of the world that we actually live in, but then all the way downstream. And I think the correction for when that discovery was made was even worse. The doctors just added a few basis points and said, okay, anytime somebody has darker skin, just add a higher oxygen number to whatever it is that the reading is. Again, completely misreading the situation. And a few things there, right? Because you know, you and I are both Black, our skin type differs, right?

36:03And then if we get blessed by the sun, you know, that might differ as well. And so with my research, it was also really important that we were really nuanced, right? When we were talking about the actual skin color, skin reflectance, and all of that. You bring up this story, and I don't think I've really shared this publicly yet, but I want to share it with you and your audience as well. When I came to MIT, in addition to building that mirror project, I also took a health course where we had to build a pulse oximeter. According to my pulse oximeter, I was dead, right? You know, and it kind of reminded me of the white mask incident.

36:48So to actually test out the homework project, we realized it wasn't going to work on my skin, right? And then we had some Indian team members, so we ended up finding the person with the lightest skin so we could get that project done. Little did I know, right, it would have all of these truly life or death implications when you're thinking about triage for a pandemic and COVID happening. In that same class, I met somebody named Alicia Chong Rodriguez, and she was actually building a health tech company focused on women's heart health, right? So just going back to the wearables again. And as I was working with her, I learned heart disease is the number one killer of everybody, basically women, of course.

37:34And then in the US, Less than a quarter of research participants when it comes to clinical trials around women's heart health, you know, around heart disease. Less than a quarter of research participants when it comes to clinical trials around heart disease are women. You know, it wasn't even until 1993 women had to be included. So given all of that, going back to the past dwells within our data, it was also the case that you have these consumer wearables that are trained on a mainly male data set. So her intervention with Bloomer Tech is the smart bra, right, to actually capture intimate details about women's heart health and moving beyond the halter monitors where you go to the hospital stressed, then they're like, calm down.

38:26We're going to see how your heart reacts versus, OK, let's actually get you in the real world where you're calm. And so she had identified this major data gap. And she's now, you know, built this incredible organization that I'm proud to support. But I don't know that we would have even gotten to that if not for that class, where according to the pulse oximeter I'm building, from not having my face detected, to being read as dead. I'm talking to a ghost right now, a ghost from MIT. A ghost from MIT. It's wild. So no, those I'm glad there has been more attention on this, but not enough, which is to say that I think sometimes some of the most dangerous applications of AI in our imagination are the good uses because they're like we've been talking about surveillance and all of this.

39:18with health, we want you to get better. Right. You know, and that story sounds great. And sometimes we don't question the assumptions. So with the same scrutiny, we would want to look at surveillance to AI applied to surveillance. We want that scrutiny applied to health so we can actually realize the promise of these systems, not just for a select few. Right. I'm totally I think most people would love to be in a future where before they don't have to go to the hospital because 10 years before something's gone wrong, they had a ring that flagged something while they were sleeping. That is a great state for everyone.

39:54And that ring was 50 bucks. But we can't get there if we don't acknowledge all of the ways those types of devices will not work for certain people. And I think it's so imperative that we confront any societal bias, any stereotype that exists. If it can be found in the data, if it can be found in a movie script, if it can be found in how people are cast, it can get automated into the future. I was reading a study recently from, I think it was Stanford, Berkeley, and Oxford, and it was showing how the more experienced a woman is in her career, and therefore the older she would be, AI could be working against her.

40:33Because if you think about all of the images that we would see on the internet, so if you do a Google search of women in a certain profession, a lot of stock photos would come up, or a lot of different images from past movies that were really successful. And who gets cast in those roles or who gets hired for those stock images of the female lawyer or the female doctor? She's probably much younger. So AI systems are getting trained on all of this type of imagery. And so when they asked AI systems, and these types of systems now get used in recruiting, and it worked even worse for the more professional or prestigious a job, the worse it scored women that were older and more experienced, when it asked the AI system, okay, generate a resume for a female physician or a female lawyer, it always had her age much younger and as a recent grad.

41:21And so that means if you're an older woman with tons of experience and you apply for a role that you are well qualified for, because you are older, the AI could flag you as an outlier. And then you don't even, you are completely dismissed from that role without even knowing it when the resume says you are a perfect fit for that job. There was another study that came out that was showing when women were using certain AI tools to help them negotiate, they were being advised to negotiate lower in terms of how much to be compensated than compared to their male counterparts as well. So following on from what you're saying, right, not being given your due despite experience and value.

42:07And would that be because if an AI system is trained on historical data of salaries and pay, women have historically, because of the pay gap, women have been paid lower. So an AI system would see that and then anchor down in a salary for a woman? Is that could be one of the reasons why AI would tell if it's a female negotiating for a salary, actually aim low? It could be, right? And that's where you actually have to do the analysis and see what's going on. I remember when we were doing the gender shades paper, which was my MIT research that showed the skin type bias mapped to racial bias and the gender bias.

42:48When we just looked at gender and when we just looked at skin type, right, we assumed that, okay, it's going to overall work better for male faces and it's going to overall work better for lighter skin faces. Then we took it a step further doing a subgroup analysis and intersectional analysis and we found that for different systems different groups had various types of performance and so in one case for example the best performance was on darker skin males. So this is why I'm always very curious to dig into a particular system to find out what's going on because we might find an even richer story, right?

43:30It might be, oh, it's because of this particular profession, or this was the database that was more available when they crawled the websites. I know in Unmasking AI, I talk a lot about power shadows. And I think what you just described is a great example of a potential power shadow. As we are talking about how does bias get into AI systems, and we identified skewed data sets, my question is, why are these data sets so skewed anyways? So I started to look at the mechanism behind how that data was collected. And at that time, it was, let's use publicly available photos. Let's use elected officials.

44:10So if you're going to use elected officials as your source for your data set, then it wasn't so surprising to see that these data sets were 70 % or more male. And then if you look at the global representation of who's empowered, actually is around there, right? Similarly, when we started to ask, okay, there are more darker skinned people in the world than lighter skinned people. Why are these data sets overwhelmingly 80%, right? Lighter skinned individuals. So we got even further to say, how do you detect the face in the first place online. And they were using face detection systems, kind of like the face detection system where I literally had to put on a white mask.

44:57So it meant that even if you had images of dark skinned individuals, the automated face detector wasn't going to include them in the data set in the first place. And then you add the added bias of, you know, there is colorism, all kinds of isms in terms of who gets media attention. And when you put all of that together, then the power shadow where you had much lighter skin representation isn't so surprising, right? So going back to your example, when we look at who's encouraged to negotiate, maybe if you actually did break down salaries by gender and the AI is reflecting that data, that's a power shadow embedded, right, that then makes the problem worse.

45:42And we're going to link to your gender shades paper because this was, I remember reading it when it first came out, and this paper made shockwaves in the world. This really was a huge wake-up moment for the private sector too, the response, so we're going to definitely link to that in the comments or in the episode description. And what about all of those kind of AI memes and AI, I guess there would be AI memes, those trends that happen where you can use AI to show you and depict you in 20 years, or you could use AI to depict you in what you would have looked like in the 70s, or what you may look like if you had a family, all of these different trends that go viral, and they seem really fun, and they're usually really funny.

46:29Are there any impacts of participating in AI systems that are scanning our face to make those predictions? Absolutely. I always say follow the data and we know that data travels. In Unmasking AI, I was talking about one of these trends that was going on, you know, to, I think this one was to see how much older, what your face will look like a few years down the line. And then there was another trend that was show me in a new profession, right? And so the males were getting professions like astronaut, doctor, women's photos, and we continue to see this, they were having their clothes taken off or being put into skimpy wear and so forth.

47:15And part of that, again, goes back to how these data sets are composed. If you look at a lot of the imagery of women online, if we look at the entire web, it's going to be hyper sexualized images. And then you also get, I think what I found, all of this is disturbing, but even more disturbing. This was happening with underaged individuals, right? And so now you have somebody where they might upload an image of their 14-year-old self. And that image then gets created in another form. And then that can also get circulated as well. And so what should people on that note be aware of when it comes to sharing photos of your family and sharing photos of your kids?

48:05And I know that so many people knit out in different ways. Some people will blur the child's face. Some people don't really know and they'll just share the photos because you just want to update your friends and family about what your kids are up to. And then some people have very strict rules around that. What should parents be aware of when they're sharing photos of their kids online, just so they have some more information about it? Yes, that data travels. And so it can end up in a database somewhere else that's actually used for surveillance. And I think you even had this with real school photos being taken, where the photographer at the school ended up uploading those faces to another type of data set because face data sets are extremely valuable.

48:50Biometric data, all of these examples we were talking about that enable surveillance, they need high quality face data. So an easy way to get high quality face data is to do it under the guise of a trend. That's fun. And so now I don't have to go scraping somebody's ring doorbell, I can invite you in. And you might volunteer because you don't know that's what's actually happening. Because oftentimes the terms of use say that they can do basically what they wish or what they will with your face data. The only reason Facebook had to pay$1.4 billion in fines and then some more is because they deployed their facial recognition technologies in states that had biometric rights protections.

49:43And in most areas, you don't necessarily have those protections. So, you know, when I think about my family members and they all ask me, like, should I post the kids, all of this? They already know Auntie Joy is going to say, put a cover on it, that kind of thing. And also, So who do you really want it to be shared with in the first place? Because maybe there's a smaller way to share it versus broadcasting on the Internet where it can easily be scraped. For example, Clearview AI has 20 billion face photos that they have scraped online. Government agencies use Clearview AI. I was just looking at the inventory of AI use cases.

50:31And right now, Clearview AI is listed as a vendor to help with facial recognition for national security reasons. And I was looking at the part within this disclosure where are you able to challenge it, right? And the entry said not applicable. So knowing how much is out there in the first place, if I were a parent, I would very, even not a parent, right? I would limit, you know, the amount of exposure in terms of posting images of your kids and also thinking about consent, right? When they're older, this might not be something that they actually wanted out there for the entire web. So I think thinking about smaller ways, you know, of sharing the photos with the people you love ultimately.

51:22And then there's also risks of identity theft and voice cloning and all of that. So the more data that floods the web as the child gets older and you could use an AI, the better AI gets at making those predictions to predict what they may look like. And then it creates all sorts of cybersecurity and identity theft risk as well. but you had written an op-ed in Wired and I think it was about two years ago and you wrote that you can envision a future where there's two groups of people who emerge there's the one group who essentially just accept that our biometric data is captured how we walk how we talk how we look there's cameras everywhere and they've just accepted that and then there's the second group you call the faceless and the faceless resist and essentially try to opt out of this society can you paint a picture of what that future is that you're envisioning and what would be the benefits and risks of being on either side?

52:18Yes, no. So in that Wired op-ed, they asked me to imagine a future, right, where I saw some of this ubiquitous AI surveillance tech going and it could look at facial recognition. So I thought about these AI-fueled surveillance states, right, and then I also thought about free state or free face societies which is where I would like for us to live where you can go out and not be worried that someone's going to capture your biometric print but for those who don't live in the free face society and their cameras everywhere I started thinking of what would it be like to go faceless maybe this means you get certain kinds of prosthetics that are attached to your glasses.

53:04Maybe it means the glasses have certain kinds of coating. Maybe we start to learn from other cultures where there's already a tradition of face covering. Even when I go to the airport right now, I usually have something over my face and dark glasses and things of that nature. And so I start to think the ways in which people might want to make themselves invisible to a surveillance state because the government, has not protected them. And we don't have to actually go too far out into the future. We can actually go back to the past. When we think about protesters in Hong Kong, where they were like, be like water.

53:47And it got to the point where people were covering their faces and also smashing surveillance cameras that masks were banned, right? And so I think to see that there have been government actions on whether or not your face is visible tells you how important it is to actually have power over it. And I do believe I also got into what that might mean for children in the future. To be biometrics naive might be a luxury, right? That your child has never been scanned in certain ways or put into particular systems as well. And then I thought further, right, if you don't live in a free face society, maybe one of the most intimate things that will be in the future is for somebody to actually see your true face.

54:38And there's so much there. You could imagine a world where your wearables skew how the world sees you. And And yeah, there's this economic divide in families based on who gets to opt out, who gets to buy the wearables that allow you to remain anonymous or to give you full autonomy over your own face. And I mean, I suppose the ideal scenario is that we don't have to go there. And the truth is we don't, right? We're not necessarily there yet. we're starting to walk down a path where there's surveillance everywhere on the corporate level, on the public sector level. But that doesn't have to be the end kind of dystopia that we end up in.

55:20First and foremost, what are the things that the average person can do today in their own life to try to be a part of a movement that gives us more agency and autonomy over our biometrics, over where we're surveilled, and gives us more rights in this environment? I think it's always important to remember if you have a face, you have a place in the conversation about AI and the conversation about surveillance. And also, if you have a voice, you have a choice. So I think it's important to remember the power of our voice. Sharing stories, letting people even know that opt out is an option. So often, even when I share, you know, that it's expanding to 430 airports, it might be the first time that somebody has even learned that they have the option to opt out.

56:09So I would say where it is not mandatory, just think twice before automatically submitting your face to something. Ask if there is an alternative just as your default way of being, because right now the norm that's being established is if you're asked to submit your biometrics, your face data, you just go along. I think another thing that I've learned in terms of storytelling, right, and I'm a poet, you know, all of these various things, was when we did the documentary Coded Bias. I honestly didn't want to share my experiences about coding in a white mask. I found it kind of embarrassing. But I decided it was worth speaking up, not because I necessarily knew the Algorithmic Justice League would form or there would be this impact that there has been, but because I thought about the next person, right, who would be in a similar situation.

57:08and I wanted to say I at least did my part. And in speaking up, I realized that others realized that, oh, these experiences I was having that I thought were just happening to me or were in isolation or maybe I did something wrong or there's nothing I could do about it. That wasn't true because others saw that story. And so when we literally just went on a world tour with Coded Bias, I remember being in Nairobi and a young software developer coming up to me and she was almost in tears. And she said, when I saw you in coded bias, I knew I could be in this space. Right. I remember when we were showing it in the UK, Rhodes House, Oxford University, a student stood up and said that they study what they study now.

57:57Right. Because of unmasking AI, because of coded bias, because of storytelling. And that continues to remind me of the power of our voice, not just because we have all the answers or we have some big technical degree or something like this. I was doing this as a student. People see, you know, the doctor title now, but I like to remind them that this was when there was not that kind of credential behind my name, but saying this is worth saying something about right now. And so I think it's so important that people don't feel overwhelmed. Oh, I don't have a technical background. This isn't a conversation for me.

58:35If you have a face, you have a place in the conversation because AI is shaping society for all of us. And you're the expert in your lived experience. And I think that is something to keep in mind and exercise your voice when there are new AI systems being adopted at your school, in your workplace. So for example, the New York public schools, they just recently released an AI policy that is available for comments. Go read through that, right? And let your voice be heard in terms of the questions that you have. I think these are all great opportunities to use your voice that isn't predicated on your prior knowledge is based on the fact that you're a valuable human being.

59:19And these technologies are being deployed in ways that impact you, that impact your neighbors, that impact people you care about. So I think it's so important to remember that power of the voice. Also the power of no. Oftentimes how this happens is there wasn't resistance. We saw when there was resistance to Amazon Ring, they cut, you know, a whole collaboration because there was a larger risk. And I think sometimes people forget that they have the power to say no. And that power of refusal actually shifts markets because ultimately you do vote with your dollars and they do want your cash. So I think that's also important to keep in mind as well.

1:00:07I love that. Yeah, you vote with your dollars. You can say no. You can not participate in some of those AI trends we talked about online. Telling your neighbor about the impact of uploading photos of kids online, where that could go. And then just asking for the school policy on AI, you know, what predictive, do we use predictive algorithms or do we use algorithms at the school with kids? And how are they used? What data is captured? All of those little acts are actually acts of resistance. But then when you, you know, you could be somebody listening to this right now that thinks, OK, I could do that.

1:00:38But they say now over what 70 percent of companies use AI in hiring. AI is now used in medicine. It's used in our vehicles. So AI, in some ways, there's a lot of it that already surrounds our world. What can be done about the AI that's here today evaluating people? Is biased AI just something that we have to accept? And there's a line that you wrote in your book, and I quote it all the time. And you say, data isn't destiny. So what do you mean by that? Is there more that can be done when it comes to the bias in these systems? I think it pushes to, you know, collective action where, yes, you can do what you can in your local vicinity, your sphere of influence.

1:01:27But we also do have to push our lawmakers and our policymakers and our regulators. When we're talking about Facebook deleting a billion face prints, it's because there was a biometrics privacy act, you know, that was passed. And so I don't think we can step away from the importance, right, of actually having laws. We also see that litigation helps to move companies and move decisions. We recently gave Megan L. Garcia our Global AI Justice Award. She was the mother of Sewell Setzer III, who unfortunately lost his life to chatbot suicide. side. You know, he was encouraged to no longer be here after interactions with the character he made on character.ai.

1:02:21And actually, she has a legal background. She decided to use that to push forward a case, you know, in her son's honor. And I do think more is going to happen on the litigation side. And we need more on the legislative side as well, right? If we're thinking about the Defiance Acts or all of these examples of AI harm, right, where children are being, you know, stripped of their dignity in various ways online. It is important that we have laws in place. And that does mean having the political will. And I think something that encouraged me so much, you're talking about the reach of gender shades.

1:03:04I know as a grad student, when I was working on And I'm thinking, I don't know how far this will go. But it actually got IBM, you know, Microsoft later on Amazon to no longer sell facial recognition to law enforcement. You had laws be put on the books. You know, I remember when San Francisco banned facial recognition and the police said I landed there that day. They were calling it Ban Francisco and so forth. But I bring up these examples to say when people resist, when there is push back, then there is response. What happens is when we're complacent, it becomes business as usual. And it's up to us to say this is the kind of society we want.

1:03:50We are demanding no to these actions, even if you have some of the most powerful foes. I'm so privileged to have recently joined the board of the NAACP Legal Defense Fund. Right. And I think about the historic cases that they've argued in the past that made my educational path even to be at MIT to do this research possible. When I think about Brown versus Board of Education or even their ongoing fights with birthright citizenship right now and 85 years of saying these communities matter. We believe in justice and we are going to strategize. We're going to organize. We're going to play the long game, knowing that Thurgood Marshall was fighting for decades to make some of these changes happen.

1:04:42understanding that the Dahl test itself was a story to be told. I realized the white mask story was following in that legacy even without my knowing it. And so when I look at the history of sustained resistance, I see that it actually does change the course of history. And we are right in that history flow where we get to make these decisions. And I can see myself telling my children my grandchildren to come right what it was to be a member of the opt-out club what we were doing at the borders and so forth why we have a free-faced society now or some free-faced zones and things like that i want to be able to tell the next generation that we thought we used our voices while we could and and i love that a paper led companies and some of the most powerful some of the biggest companies in the world to change the products that they put on their shelves right So all of these ways that wherever you are in your journey, maybe you're a student, maybe you're a voter, maybe you can change what products you choose to buy.

1:05:48All of that is part of the act of resistance and part of using your voice or using your wallet to participate in building towards a future that we actually want to live in. And that's the thing about the future, right? The future isn't some far out state. It's the decisions that happen today. That is how the future gets made. and what about when it comes to tech companies themselves or there could be somebody that's listening they're building an hr resume hiring tool using artificial intelligence what should they be aware of so let's say that they've procured ethical data sets and everything has been you know lawfully procured together but what should they know about bias what needs to happen in these coding rooms so we stop getting these types of ai products on the shelf that only work for every ninth person.

1:06:36Yes. Well, one thing I think about is algorithmic hygiene. So you wouldn't just floss once and think you're done with your dental care for the rest of the year. I floss January 1st. I'm good. And so I think sometimes when it comes to thinking about issues of bias or AI harms, people just want to floss hard once and think they're set. And so this leads me to this idea of continual checking, right? Making sure you're exercising that algorithmic hygiene. I think the other part that's so important is affirmative consent and to let people actually decide what they're using with informed information.

1:07:22So I think sometimes people are scared to share the limitation of their system because you want to hear the AI promise, right? But I think it's really important for developers to say, this is who it was trained on, right? Data sheets for data sets, responsible AI cards. There are many ways of doing this where the whole idea is, this is how it was trained. This is what it was made for. And these are the known limitations, right? And also in doing that exercise, you might realize you might not want to put it out if these are some of the known limitations as well. This actually reminds me of when I became a spokesmodel for Olay.

1:08:05We did this big decode the bias campaign talking about AI bias as it pertained to beauty. Right. And so as I was getting ready for the campaign, because it was in vogue and there was a commercial, all of this, I did all the right things. I went to bed on time. I drank water. I was exercising. Of course, I use the products, right? But this was knowing, you know, that there was an evaluation that was coming of me. And so in that way, I think it's really helpful, right, to think of yourself being evaluated, showing your work so people can make an informed decision about using your product. But so you also have an informed understanding of where it works and where it doesn't.

1:08:54I think the other thing that I've seen is the importance of speaking up internally, even though it can be hard. One way people aren't as aware of for how the gender shades paper was used were people inside corporations saying, let's learn from that. So we don't necessarily make that mistake. Or maybe we can build more inclusive data sets. I remember hearing from some of the team members from Project Douglas, which ended up leading Google to create more inclusive camera software, saying that they actually used the gender shades paper internally as a way of gaining resources. You know, people, money, dollars to go behind this to create a different version or something that actually reflected the communities that they cared about.

1:09:47And so I do think it can be definitely difficult. And there is a cost. I think of my dear colleague, Dr. Timnit Gebru. She used to be the head. I say used to be the head of AI ethics at Google. and when she spoke up there was a cost to her job even though she was literally warning of things that are now taken for granted as ai harms like the environmental risk and so i also and i talk about this in the book unmasking ai kind of grappling with when you speak up versus not and also understanding what's at stake i remember finishing my mit phd and saying and just so you know, we had a legal team, right?

1:10:33To also understand that you are weighing different risk. And sometimes you are in more of a position to speak up. And sometimes you're in a position to influence somebody else who has more power and who can do a bit more of a cover to speak up. So that's why we invite people to share their stories, report.ajl.org. Maybe you're seeing something, but you're not in a position where you can say something out loud right now, but you know, that this is important for other people to know. So I think it's also important to increase that surface area of how you can make a difference, but asking better questions like who might be missing right from this data set?

1:11:17How have we tested and evaluated this? Yep. On all the products, whether you're in HR or you're in marketing, who was probably in this data set? Have we done, Have we double-checked various different profiles to ensure that our product works? And again, you don't necessarily have to have technical expertise just to present those questions to the builders in the room. You can just raise them. And so for people who want to be a part of your mission or just keep up with it to follow what it is that you're doing, to follow the Algorithmic Justice League, how can they get involved? Where should they follow you?

1:11:52I've already, I shout out your book. We did a live the other day or a couple of weeks ago. And people ask me, what are the books that you read? I always recommend Unmasking AI. It's such a great book, especially if you're you don't have to be technical to read it, right? It's much more the lived experiences of living alongside these systems. So that's a great place to start. But where can people follow you, get involved? Where should they find you? I would definitely say become a member of the Algorithmic Justice League newsletter.ajl.org. You can be part of the broader movement to make sure that technology works well for all of us, not just the privileged few.

1:12:28I think also follow us on our social medias and things of that. But I think more importantly to me when I think through ways of supporting is truly educating yourselves and the community around you. So that might be watching a documentary like Coded Bios is now on YouTube, reading Unmasking AI, or maybe you're reading Empire of AI by, you know, Karen Howe. There are just so many great people, and we've created a library on AJL.org where you can find many more voices to have a better sense of what's going on as well. We're forever asking people to be part of the opt-out club. So if you've gone through TSA recently, you can go to tsa.ajl.org and literally share your lived experience so we can continue to create research that keeps organizations accountable.

1:13:25And that's one thing I've been proud of the Algorithmic Justice League to be able to do, because sometimes it's like, oh, I'm not a researcher. I don't have this background. I don't have that. you have your lived experience and that is so valuable for sure. And then I'd say keep supporting artists. I do have a poem that I had wanted to share. There are actually two poems, but I'm thinking maybe the X coded poem would be more apt. I wrote this poem precisely who will die as I was thinking about the use of AI by the military. So here we are. Some say AI is an existential risk. We see AI is an exterminating reality, accelerating annihilation, augmenting destruction.

1:14:16I heard of a new gospel delivering death with the promise of precision, the codename for an old aim, to target your enemy, the other, reduced to rubble, face erased, name displaced as drones carry out in a formation that spells last shadow. AI wars first fought at the doors of our neighbors. Next, the bombs drop on your private chambers. Cease to believe that fire on fire will deliver peace. Precisely who will die, zeroed out by ones and guns. And then take you from the low to the high. I'll leave with a bit of a palate cleanse after that one. Okay. All right. This one. To the X-Coded. To the X-Coded.

1:15:18Resisting and revealing the lie that we must accept the surrender of our faces, the harvesting of our data, the plunder of our traces. We celebrate your courage. No silence. No consent. You show the pathway to algorithmic justice requires a league, a sisterhood, a neighborhood, hallway gatherings, sharpies, and posters, coalitions, petitions, testimonies, letters, podcasts, research, and potlucks. People asking questions like Sinead. Dancing and music. Everyone playing a role to orchestrate change. To the X-coded and freedom fighters around the world, persisting and prevailing against algorithms of oppression, automating inequality through weapons of math destruction.

1:16:12We stand with you in gratitude. You demonstrate the people have a voice and a choice. When defiant melodies harmonize to elevate human life, dignity and rights, the victory is ours. Thank you. I can't think of a better note to end this podcast on. Dr. Joy Boulamwini, it has been a pleasure. Thank you so much. You shared so much with us today. I learned a lot beyond the other end of this conversation, and I've known you for years. So thank you so much for making the time. And I know we're going to have you back on again soon.