Hilke Schellmann: Is Your New Boss a Robot?

22:40And some companies actually do have humans watching all of these, but some companies also use AI to rank people and do that. So we see that more and more. And we see this often like entry-level jobs. We see this in like retail companies, fast food. Like it's called high turnover, no, high volume, high, I don't remember. So it's in jobs. Jobs that generally have people where people are coming in quickly and leaving quickly. Exactly. Like they're not going to be there. It's not a career job. So people are going. Sometimes it's a career job. Oh, but it's just like high turnover. But it's a high turnover or you have like lots of candidates that you have to go through.

23:17Got it, got it, got it. So, for example, like Goldman Sachs said a few years ago for their summer internship, they had like over 100 ,000 applications. So they have to like go through these applications and like narrow down the pool. So you use like resume screening AI. You use like video interviews. You can use games. We see like personality. These games are supposed to find your personality while you're clicking on balloons, pumping up balloons. They find your personality. all kinds of ways to assess you without maybe putting in a whole lot of work because humans are expensive to do this work and also sorry to say this but a lot of humans they do suck at hiring because we have bias we have human bias but this is the conundrum though so so this is this is the thing that's like weird now just for this part of it is my reflex my reflex when i hear something like that is to go, oh no, this is not good.

24:12How can you have AIs screening people's interviews? But then on the other hand, I go, if you have 100 ,000 people applying to a job, let's be honest, I don't think there's any human who's going to get through those 100 ,000 applications. I don't think there's any humans. And I wouldn't be shocked if there were like a bunch of humans who were skipping through this before because they were just like, it's like auditions in a way. At some point, the person's tired. You want to get them when they're fresh you want to get them when they're in the mood yeah not when i wonder is there a world where like does the ai make it better then you know i wish i could tell you that um so we don't know we don't know i've asked many many companies um to let me come in as a researcher and like sort of look at like here's your traditional way of hiring here's uh your ai hiring and have this like run at both times and then sort of double check like you know the people that they i said that would be high performers, did they actually turn out to be high performers?

25:09And I have not seen a company do this or want to share this with me or with anyone. I think it's because, I don't know, there's like a lot of turnover in HR, like these processes don't work that well. And I think what we already know, so what we know from a survey of C-suite leaders, like sort of leadership and companies of over 2000s in Germany, the UK and the US, when they asked them if your company uses AI tools. Do they reject qualified candidates? And almost 90 % of the leadership said yes. So they know that their tools reject qualified candidates. They still use it because I guess the efficiency from using AI versus humans, it's just much, much more greater.

25:57But it's not that we know that one process is better than the other. I mean, we do know that humans are very biased in hiring and even the best anti-bias training is not going to get out of it. And, you know, we all know the shortcuts, right? If you see somebody on your resume that they went to Harvard, you're like, oh, they must be smart. No. They're not. Well, we know from social signs. This episode, Eugene Koza learns about the world. She's like, wait, what are you telling me? But, you know, so this is where I feel like we stumble on the first conundrum. generally, generally, machines like predictability.

26:37Yes. Right? Algorithms like predictability. That's what an algorithm is fundamentally sort of trying to do. It's like patterns and, you know, yeah. And a pattern is a predictability, right? The conundrum or the paradox of being human is that the biggest breakthroughs that have come from humanity have often come from the pattern breakers. The person who didn't think correctly, the person who didn't fit the algorithm, the person. The outliers. Yeah, the outliers. So I wonder if companies in moving all of their resources towards efficiency and patent recognition might go the opposite direction of innovation.

27:14Because it's almost like the misfits and the mistakes are sometimes the ones who give you the biggest flips. Do you know what I'm saying? Yeah, totally. I feel like the solution has caused a problem. Well, we were speaking about how many people had applied to Goldman Sachs. And I think if it wasn't for technology, would you still get that many applications? That's interesting. Would 100 ,000 people from all over the world show up at the address to put in their resumes? So I think technology also allowed easy access. Absolutely. Because I also think there's people who know they don't qualify, but would do it anyway.

27:45So why would you put a human through all of this? But also I think it's a box ticking exercise for some companies as well. I think some companies don't want to hire anybody, but they'll just put out a thing that says we want to hire somebody. then they'll end up doing the internal process anyway because if you're going to trust people with people's monies and files and information you'd want someone that you know so I think companies know exactly what's going on but they're just sending out hope and I think once you advertise a job it's a great way to advertise your company as well yeah I mean sort of like people online you know they often joke because obviously some people obviously are very aware that companies use AI and now a lot of you know I think it felt very like passive and and sad for a lot of applicants until sort of LLMs and ChatGPT and other AI came around, where now it's like much easier for me as an applicant to generate a resume.

28:35There's actually now AI. So now it's AI warfare. Yes, it is AI versus AI. I'm going to use the AI to apply for the job. They're going to use the AI to grade me. I'm going to use the AI to pass the grades. Exactly. I'm going to try to use AI to like outsmart the AI. There's actually AI programs that now apply for you. So you don't even have to do anything. So there's all kinds of stuff. But like the question is like, well, what are we then doing here? Yeah, like what are we doing? What are we doing? That is a great question. That becomes the question. What are we doing? Because if the AI is hiring the people who are using the AI to get the job, that the AI has hired the people, then that's what I mean.

29:14It's like we have to ask the fundamental question, wait, what was the point of this process in the first place? Because multiple studies have shown Humans are terrible at predicting the future Especially when it comes to hiring Right A lot of the time when you're hired You're hired because the person sitting across from you Saw something in you that they considered correct for the company But a lot of the time it's just wrong Yeah You know what I mean? It's just wrong And then people don't do well And they're like, well, that didn't work But the prediction is wrong You know what I'm saying? Yeah And so now I almost feel like We forgot what the whole point Of an interview was Like I'm not a historian But if I was to bet I would think an interview Was just to be like Let me see what your vibe is It was a vibe check It was a vibe check Yeah But it turns out like Vibe checks Not so great actually Like because you At predicting who's a good employee Yeah But also like A vibe check is like Finding people who like Like are often like Have the same background as you They speak like you They vibe with you Exactly So you find the same people again Yes Which you know we kind of know that like diversity is good for companies.

30:21Also, like, I mean, I think that's why we have, you know, fewer women, people of color in leadership positions because we have underestimated them as humans in hiring for decades and promotion decisions. So we have like sort of a lack of diversity already because of human bias and sort of the vibe. You know, when you come to a job interview, you want nothing more but like somebody, you know, like the HR manager or the hiring manager to like you. And then you start talking about like, Like, well, what school did you two go to? Like, what did you? If you walk in wearing the same shirts, oh boy, it's on.

30:51You like this, you know, sports team, yada, yada, yada. And that chitchat feels like very good for humans to make a human connection. But it's actually really bad because that brings the bias in. And now as a hiring manager, I'm like, oh man, you went to the same school as me. It's so cool. I see you in a completely different light than other people. And I'm supposed to look at like, what are the capabilities and like your skills that you need for the job? Not if we went to the same school, but we as humans do that. And that's where a lot of the bias comes in. The unfortunate thing is you might think, well, AI is like a pattern machine that just finds patterns, right?

31:26And it will just look at your capabilities, your skills, and find the most skilled person. But what we've seen in some of the AI tools, when I talk to lawyers and others who get access to these tools, when like an AI provider, they built the tool, an AI vendor, and a company may use their tool. Sometimes they bring in lawyers and do their due diligence. Like, how does this tool work? And what they found out is when the lawyers looked at it, that the tool used, some of these tools used kind of problematic keywords. So, for example, one tool. Oh, there's the Amazon story that you wrote about. Yeah, the Amazon story is one of them.

32:02The Amazon one was pretty insidious. So this was like if you had the word woman or women on your resume, you got downgraded. Because, you know, the tool had learned over time. You give it resumes of people who currently work here or who maybe made it to the last round of hiring, sort of labeling them as these are the successful people. Well, if you work in a tech company and you probably have a gender disparity already built in from maybe previous bias, you kind of replicate that, right? If the people who are in the role, if you use their resumes, the machine does what it does best. It looks for patterns.

32:38And it finds out, wow, women are less successful here. So we should downgrade them in the hiring process. There were some applications in the story where Amazon was hiring people. And their system basically went on its own, doing its job as it had been told. And it went, oh, I've noticed women's soccer team, women's baseball, women's anything, does not match with the people who are currently at the top of Amazon. They don't have that word on their resumes, basically. Exactly. So this person is less likely to be like that person. So we're going to downgrade that. But this had nothing to do with your actual qualifications.

33:18Wait, did AI do that or did someone who put the input to the AI do that? No, AI did that. No, there's no input. This was a middle issue. There's no input at all. Yeah, you have to think about like, you know, sort of present day AI. What we do is like we give the AI just the data we have and have it like we call it unsupervised learning. Have it like figure out what do these people have in common and who should we hire. So it looks at patterns in the resume lake that you give it. And I guess it scans all of the words. And then it does what it does best. It does a pattern analysis and finds out.

33:54One other example was if you had the word Thomas on your resume, you also got more points. If you had the word what? Thomas. Thomas? Thomas. Like the name Thomas. Or in another case, it was words like Syria and Canada. What, those got you up or down? That got you up, actually. If you had Syria and Canada. If you had the combination, you were hired. Yes. Wait, wait, but now, but now. If your name is Thomas on top of that. From Syria via Canada. Yeah, I was like, no, but now, so here's my question, though. Does that mean that people could, are there tricks that people could use now? So if I was writing a resume today, could I just write somewhere randomly?

34:31Syria, Canada. Passions, reading about Syria. Canada, I like it. Maple syrup. Thomas you know what it is Thomas Thomas Thomas Thomas Thomas Thomas and then well so I think the problem is that like most tools are like individually calibrated to each company so I could only get hired at Amazon by doing this well Amazon had that women's problem but they say they changed that they also say that their machine learning algorithm was never used solely to make hiring decisions but no one would say that it was like I mean which company would be I don't think I've seen a single story where a company has come out and said, yeah, man, we were just using a computer to choose who was coming here.

35:13All of them go like, no, no, this was not the only thing. This was merely a pilot program that determined. You know, the more you guys talk, the more I realize, are we under, you're a journalist, you know this, are we underplaying the role that biases have played in our lives? People choosing whatever it is that represents a certain group of people or a company even based on what they think the taste of the population or demographic is. Do you understand what I'm saying? Yeah, yeah, yeah. So you're thinking generally or in the hiring process? Not in the hiring process. Because if you're going to work for a company and the person sits there goes, I think you'd be great here because of what, what, what, what.

35:53Now we're going, because I think biases always, and I could be wrong, always comes in when we speak of race, gender, or religion. Once you've ticked those three boxes, we're like, yeah. But how many places have we gone to where there's that mix because of someone's biases who decided maybe people who are six foot with muscles should be in construction because they look like this, they sound like this, they talk like this. Actually, they'll be great for this job. So how many of us are beneficiaries of biases? I think a lot of us are beneficiaries, and a lot of us also have been sort of the victims of bias, and probably unbeknownst because, you know, you go in for a job interview or you send in your resume, and most likely is you get rejected, right?

36:38Because there's only so many jobs that are being given out. So the question is like, were you rejected? And I think most of these humans think, oh, well, I was rejected because I wasn't the most qualified candidate. Or it might've been that you've been rejected because your name is Thomas. Or in one actually instance, there was the word African-American that was used to weigh resumes. And another instance, there was if you had the word baseball on your resume, you got more points. If you had the word softball On your resume You got fewer points So That's probably Gender discrimination I would give you I would give you Zero points for both In my company I would be fair You say baseball You say softball I would detract points Trevor do you see How it circled back How those And this was not A baseball position The question is like You know What does it have to do With baseball But you know In a way I know this This is gonna sound A little crazy But like I can sort of Understand these ones And when I'd read the examples in your work, I would go, this sort of makes sense.

37:37I can see why they've made a mistake here and they can rectify it. But there are some examples that you've given that blow my mind. For instance, there's one story that you go into of a guy, I think by the name of Mike, and he's working for Bloomberg or he's trying to get a job at Bloomberg or something. And please help me understand this because from what I understood, I'll say it and then you let me know if I'm right or if I... He had to play a game like Candy Crush type stuff of popping balloons and then he got fired because of how he popped the balloons? He didn't get fired but he did apply to a job.

38:16Okay. He was based in Barcelona and he was based in Barcelona and applied to a job in London and he got a link immediately after applying saying like, hey, go to this link and you know, I sort of feel like Like we as job applicants, we sort of force consumers off this tech, right? Because if you want the job and you get an email with the link saying like, hey, you have 48 hours, click on this link, play this game. What are you going to do? Are you going to do it? Yeah, you're going to play the game. And he was like, while he was doing it, he was like, this is weird. Why is this asking me this question?

38:47It sounds like the beginning of a horror movie. Do you want to play a game? Why do I have to do this? Like it sounds great. And I think a lot of applicants technically like it better than answering 100 questions about like, are you the life of the party? Like I rather pump it balloons. But when you realize, wait, is this the only criterion I'm going to be judged on? How well I like pump balloons or like in one of the games I had to hit the space bar as fast as possible. And while I was doing that, I was like, you get like 15 seconds or so to do that. And I was like, what does it have to do with the job?

39:18Like in what jobs do you have to hit the space bar as fast as possible? Maybe it's like a company where like there's like big gaps between people's names. Maybe there's like, maybe you're working at a company where it's like Suspenseful Pause Incorporated. incorporated maybe it's like i mean i want to know what this job is now where somebody out there is just like yeah maybe it's a company maybe it's a company that had to cut costs because all the enters the enters on the keyboards were broken and now they have to hire people who can use space to get to the next line because you can't just press return you can't just press come on come on and then that boss was like you know we need we need people can press the space bar get me the fastest space bar presses in the world.

39:56We found them. We found them. But you know, I mean, what's interesting, like that actually, that suite of games was used by like multinational companies around the world. We're talking like legitimate, not some random company. You're saying this is used by like big name companies. How fast can you press a space bar? This is one of the many games that you have to play. And you know, they say they're not actually like looking at your capabilities of hitting the space bar. It's like finding out how much like, you know, how risk averse you are, like what your personality is underneath this. Like, are you somebody who likes challenges or not?

40:31I guess who takes any order that you're given. I'm sure even the time between you deciding are you going to press the space button or not actually maybe counts. Oh, that's interesting. Did you really think about this instruction? I don't know if that counts, but I did talk to an industrial organizational psychologist who said, yeah, we looked at all of those things. and actually the people that take longer until they start playing, they're actually less successful. But he said, we are not using that criteria. I called it, my man. That's my... Called it. You called it. You did call it. Yeah. But so we don't know exactly, but all of these, like every space bar hit and everything that I do obviously gets recorded somehow and can be used.

41:17But the question is like, on a good day, our personality is such a low predictive measure to measure how good we are going to be in a job because it also turns out like I can overcome things in my personality, right? Like, I don't know if any one of you have, like, I try to, you know, I used to be like really shy. I didn't like to talk to strangers. I know it's part of my job. I like calling people on the phone and chatting with them, but like going to like a party, like a reception with actual people, I don't know, and like going up to them. It's like, oh, I used to hate it. And then I was like, it's part of my job.

41:52And I made it a game to challenge myself. So I was like, I'm going to make games for myself. You just walked into parties with a keyboard and you're like, how fast can you hit this space bar? You win. Nice to meet you. Nice to meet you. I'm Hilka. We can be friends. This is my research.

42:15That would be, I should have done that. That would have been much more interesting. But what were the games? What did you do? No, no, the game was that I have to approach strangers and like say hi. Oh, you did this for yourself? Yes, yes. And what was your reward? My reward was just like, well, getting to know people and like learning about them. I like this. So this was how you overcame it for yourself. You went, I'm afraid of speaking to people, so I'm going to make it a game where I just walk up to a stranger. That's what I tell my journalism students. What happened when it didn't go well? Well, I'm still here.

42:46So I was afraid I was going to get decapitated, right? People are nice. They're like, what the? But, you know, I'm still here. And, you know, sometimes people were just like, eh, and like just left me standing there. You realize it's not as bad as you thought. Yes, but you see, this is AI again having, let's say if this was a program, you would score higher because you're a woman. It's easier for women to do that than a man to do that. Oh, that's interesting. If I walk into a random room, then there's a bunch of women there. I'm like, hey, guys. I should take a game where I'm trying to be social like stranger danger psycho oh man but for for a woman it's much easier so the bias is kicking again if I go to a Midwest town as a black man from Africa and I walk in there and there's truckers and I go howdy folks no one's gonna say hi to me that was a good howdy though yeah you like that you nailed that I'm in if my eyes were closed when you walked in?

43:43Close your eyes now. Howdy, folks. Hi. That was not bad. That was not bad. I'm in. Darn. Once I look up, things might change. So you see how biases is informing how, what the outcome ends up being? But it wasn't a bias challenge. It was just like a personality like overcome challenge, right? Because we all have like certain things that we like to do and we don't like to do. You were not biased. They were. On the other, on the receiving side of it, they were like, here's a woman. She's smart She's nice She's saying hi Less threatening That's true Exactly So the bias is kicked in So the same applies When an HR manager Is sitting across someone Who they look at and go I wouldn't want to be stuck With you in an elevator On the 14th floor At six at night Yeah but then that Raises the question then Is there ever going to be A world without bias And is that what We should be looking for I mean look We can all wish But we know That that's never Going to happen Like we humans are biases machines.

44:43Yeah, but now that the machines are doing the job, could it be possible? And I know, I'm not saying it will, but I'm saying could it be possible that the AI, because here's what I think about in what you're saying. We're living in a world where we know that biases exist. We know, right? So whether it's in courts, whether it's in law enforcement, whether it's in jobs, whether it's in schools, doesn't matter. We know that bias. Social settings. Social settings. Bias exists, right? Now, AI has gotten involved. And we see the AI mirroring many of our biases. Yeah. But the difference is with AI, we can actually see it.

45:28We couldn't see it before and we couldn't prove it. We had to conduct weird studies. Before, you couldn't say this company didn't hire anyone because they didn't say baseball or because they had women or because they said black. But now you can actually look at the data and go, oh, damn. And I sometimes wonder if it'll be easier. And again, this could be the optimistic side of me, but I sometimes wonder if it could be easier for us to address bias in society because we actually have concrete data now that shows it and we get to blame it. We don't have to blame each other. We'd be like, oh my God, look how racist AI was to you.

46:04I'm sorry, my friend. Yes, HR AI is a Trojan horse. You're right. Do you see a world where that's possible? Yeah, yeah, yeah. I mean, I wish companies would actually look at these tools more closely. I think the general notion, though, is they buy it from a vendor. The vendor sort of services the algorithm over time and makes sure they still run and there's less bias. They check if there's gender and very basic racial bias in there. But they never look at, does it let people with disabilities through or something like that? Right. Like and it also we don't see a whole lot of companies actually checking how are the decision being made.

46:45And I think that's sort of where the problem lies. Like if we actually somebody would look at the thousands of keywords resume parsers used to predict if you're going to be good at the job, they would find those keywords that are learned from lawyers and other places. And, you know, those are keywords we shouldn't be using. We should be looking at like your skills and your capabilities and not if you are on the baseball team or not. And I came to this as a human. I remember for the first time talking to a lawyer about this, and I was like, well, maybe I found something that humans couldn't. In this case, it was playing lacrosse in high school that was a predictor of success.

47:19And I was like, maybe it found out for this whatever insurance job or sales job. It was really good to play lacrosse in high school. It found this hidden gem that we humans couldn't. And the lawyer started laughing. And he was like, God, you think like a human? I was like, really? What? He's like, it's a pattern machine. It does a statistical analysis. For whatever reason, like playing lacrosse in high school, a bunch of people who were in the job have that criterion. Yeah, because if you're playing lacrosse, you come from a certain family. Yeah, it doesn't mean that lacrosse has anything to do with your success.

47:51And in fact, he's like, well, if it's like playing team sports, what's with all the other team sports? Like, why weren't they included? Why do you get more points for baseball and fewer points for softball? But it's essentially, I think, as a non-American, it's the same game, just a bigger field. You see, Hilke's on my team. Minus points for both. Like how you saw pickleball and... Oh, beach ball. We call it beach ball. Don't bring pickleball into this, please. Please, let's not bring... Trevor doesn't want to talk about pickleball. Don't press anything. We've got more What Now? after this. Finding the music you love shouldn't be hard.

48:26That's why Pandora makes it easy to explore all your favorites and discover new artisan genres you'll love. Enjoy a personalized listening experience simply by selecting any song or album, and we'll make a station crafted just for you. Best of all, you can listen for free. Download Pandora on the Apple App Store or Google Play and start hearing the soundtrack to your life. The crispy chicken crunchwrap slider that you love is back at Taco Bell. And it's back with attitude, because now you can enjoy the all-white meat chicken nuggets inside a slider-sized crunch wrap with Hidden Valley Diablo Ranch.

48:57What do you love most? The crispy chicken nuggets, the crunch wrap slider, or maybe the Hidden Valley Diablo Ranch? Yes, the answer is yes. The crispy chicken crunch wrap slider, a brand new classic, is back. Only at Taco Bell. Ever start a workout and feel like you've got nothing in the tank? When you need energy, you need carbs. But not all carbs work the same. Meet Glow Gummies, delicious glucose gummies made with fast-acting dextrose for quick energy when you need it. They're vegan, gluten-free, and made without artificial colors or flavors. Plus, they don't melt in the heat or stick to your teeth.

49:31Toss them in your pocket and go. Head to GlowGummies.com. That's G-L-O-W-Gummies.com and use code PODCAST20 for 20 % off.

49:46you know what i realized speaking to you guys about this because i i wanted to know as little as possible about the topic so i can get enlightened in real time is companies how's that going very well because i'm i'm because i've worked in retail before in south africa and and i've realized that hr has always been the enforcer and the goon of the corporation because when you come in they're the first people to ask you what do you like but basically they're trying to see, do you want to fit in here and be here? And where do you come from first? Then when you get let go, you do what they call an exit interview.

50:16And that will help them not hire a person like me ever again. So I use public transport. I went to a township school. So they knew that all of those factors and my age as well and how long I stuck around in that job. So they know the propensity of me sticking around longer or doing something wrong or right, according to them, is based on how long I stayed and where I come from and what changes I've made in my life since I started working there. So they could predict if someone earns this much for this long at this age from this background, the money will start becoming too little for them to be here.

50:48So AI now is doing that at a rapid rate. Instead of saying we don't want women, it will cut out words like soccer and blah and blah and blah and blah. And then the people that say those words, maybe they get hired because likelihood is they are men. How many kids do you have? How far from the job you live? And what are you willing to do for this job? I was going to say, like, you know, like when you think about it, like how these kinds of statistics and prediction works, it precedes AI by a long time, right? Like we know statistically that if you have a longer commute to your job site, you are much more likely to quit statistically.

51:26But is that fair? And, you know, know we've seen companies um trying to use this like zip codes and stuff to then say like okay well we only hire the people that are right you know live in the zip code right around our store location because they are less likely to quit but like does it really have that that that's a criteria that has nothing to do with the job it doesn't say anything about your capabilities and if you're going to be good at the job it only says something about your situation yeah and you know and also like well first of all like there are people who do the two-hour commute each way and they do a fabulous job.

51:57So you're cutting out all those people and it's not their fault. And then on the other hand, you also have to look like we live in very segregated communities in the United States. There's historical redlining. So if you like start taking out zip codes, you might actually like take out huge swath of like African-American population or Asian-American population. I think that's true around the world, to be honest. It doesn't matter where you're from. A real problem. And we sort of see this kind of statistical bias get replicated again and again. But now we have this like layer of objectivity and we don't interrogate the tools again.

52:33Plausible deniability to enforce more of the biases. How did you know that? I think it's plausible deniability of the companies that use it and buy from the vendor because then they can't be taken. You know, it'd be very hard to have a court case where you say like, well, you knew that YouTube was biasing women and there's like 2 million people that apply to this, 2 million women that apply to this company and you use the bias algorithm on them. So suddenly you have potentially 2 million claims. That's why we see what I think is sort of a cloak of silence around this because companies obviously don't want to come out.

53:08I've had so many people who work in HR tell me after the book came out, oh yeah, we use that tool that you talk about and we, you know, stopped using it. And I'm like, oh, really? I was like, well, that's good. I'm glad you did. They're like, yeah, we sort of realized we had the same questions. We found the same things that you found. And we just didn't think it was fair. And I was like, okay, can you talk about this? They're like, oh, absolutely not. But we need to learn. Like, we'll never get better. We can't put pressure on the vendors to build better tools if we don't know how the tools work and if there's any problems in the tool.

53:41I just looked at a fraction of these tools. Like, I tested some of them myself. I worked with like scientists to test them. I looked at like, you know, I spoke with like whistleblowers and like lawyers who like work in the space. But I have just a sliver of the whole sort of world out there. Like we need to do a whole lot more, but I don't think it's in the company's interest. They want something that, you know, like sort of saves them money in HR. It's always a cost center. HR never generates money or talent acquisition, however you want to call it. And so in a way, they want to save more money, have less labor involved.

54:14and they don't want to like hire people and now start like picking apart the algorithms, then it might not work. And what are they going to do then? They just spend so much money in it. So when you look at what they're doing, it seems like, and maybe I'm going to a dystopian conclusion, but I've read through some of the companies that you've investigated and some of the tools that they've used. It feels like it's becoming more and more pervasive. So first companies just looked at what you submitted to them, your resume. Then companies started scrubbing what the world knew about you. And then now, because of the way data is shared, I'm even seeing stories where they're saying some companies may be able to go, you know, as far as your social media.

54:58I mean, one of the craziest examples I saw, which I don't know how true it is, is like your Uber rating is a possibility in a future, which sounds like something China was doing or trialing, by the way. Yeah, with the social... Yes, remember that. We're like basically in China. Social score, yeah. If you have a high social score, you get to travel and you get like certain benefits of society. But if you're like jaywalk, visit grandma. That's me. No, really. And so, but now when I think of that, I'm like, are we heading towards a world where a company can hire you or fire you looking at your Spotify playlist going, oh, this?

55:34Oh, no. Oh, no. Oh, yeah, yeah. I mean, look, some psychologists say that like the way we behave is very predictive. And they can find certain ways. Like there was a big finding a few years ago, and I think it was like that a lot of computer scientists are really into manga comics. So the question is like, well, if you look at then resumes, should you hire the people that like mangas? Because you know they're going to be good computer scientists. But what is it with the people who are great computer scientists who just are not into manga? Like that's not fair to those people, right? So that's sort of the problem with these shortcuts.

56:12But I sort of do feel like there is a dystopian vision that I sort of felt like at one point I was like, wow, maybe at one point we're just not even going to do a job interview anymore. A company will just tell you if you're hired or fired or if they don't want you based on all of the social exhaust, the data exhaust. We sort of leave around and companies can predict who we are. It turns out we did test the sort of personality testing that is being used on social media. It doesn't work. But it's still being used. It doesn't actually stop people from using shitty technology. That's sort of the bad part here, right?

56:51But it doesn't actually work to predict what the people are doing. It does make me think of a dystopian world, though. Like just this idea that you will be hired before you've applied for a job. I just think of like us in the year 3000 or something and a van just pulls up, the door opens and they're just like, welcome to the job, Eugene. We know you better than you know yourself, soldier. And you're like, what are you talking about? But you might not even be wrong. In my conspiracy mind, I'm thinking that AI tools are just a big giant facade for data harvesting. Companies know if what they're offering to the public is still viable, learning institutions know who are the most likely candidates for them to start giving or keep giving the courses that they're giving because we forget that high learning institutions are just businesses as well.

57:43Oh yeah, totally. And some of them use this kind of technology. To find out who's still interested. One way video interviews. And yeah, I mean, I think what fundamentally comes down to, it's kind of funny, what I have learned by bringing AI into the talent acquisition hiring space, I learned how bad our old processes are, like job interviews, actually really bad. Because it sort of filters out the people who are good about talking about doing the job. As opposed to doing the job. So we have this competence versus confidence problem. People who come off as confident, we often think, well, that person speaks so confidently about this.

58:20They must be really good. It turns out that they're more often than not men. And that doesn't mean actually they're competent. What? Us men? No. Never. What? So, you know. Little old Ash Hilka? No. As always, not all men, but a lot. Us little old men acting like we know more than we do. Ash? Come on, Hilka. Mansplaining? What?

58:56Wait, I think this is highlighting yet again the same point again of saying that biases have gotten at this far. I've often heard people who go, if I'm in a criminal trial and I'm thinking of what kind of lawyer to get, I want someone who's talkative, who's out there, who's loud, but the person who handles my finances must be quiet, you know, reserved and frugal and they'll know how to handle my finances. You know what I'm saying? Yeah, yeah, yeah. I've never heard about this talkative lawyer, but I'm sort of like Yeah, we have an idea. You are someone who goes, razzle-dazzle. We've seen the lawyers that represent rappers.

59:27Yeah, charisma. Yeah, you want, and it's interesting to exactly what you're saying. If I hear you correctly, you're saying, in a way, it seems like we are expanding and scaling on a foundation that was already broken. Yes, absolutely. The way we hired was already broken. Like job interviews are broken, like sort of looking at, and, you know, resumes have very little predictability because, you know, like you put certain like things, you need to have this skill and this skill in the job. And you put that on everyone who applies for the job. 99 % of the people will have that on their resume. So and you can't find like things like teamwork.

1:00:06Are you a good collaborator? Yeah, you don't know. How are you going to know that from a resume? How are you going to know that from a job interview? You can ask like questions like, well, tell me how you overcome, you know, really a challenging situation at work. But you can train for that. Like the best way, you know, one of the best way to predict if you're going to be successful, this will come to no surprise to anyone, is to put you in the job. And then you can find out if you're going to be good at the job. That is, yeah. Look at that. Totally doesn't work for most companies to hire 100 people and then let 99 go at the end of the month.

1:00:37But sort of my hope sometimes is like, wait a second, like we have virtual reality. Like we have other ways. Like could we put people in the jobs and actually have them do the jobs, the most important parts of the jobs? And then figure out like how they actually are at the job. And I think that would also give candidates a way to sort of understand better what is this job actually? Have you suggested this to companies? Because I like this idea. You. I really do. You. I really do. Think about it. I do think it is a little bit more complicated than just what I'm saying. Because a lot of jobs have different capabilities and different things that you have to test for.

1:01:14Right. And some of it is hard to test. But we need to be better. Or some total cynics in this world have sort of suggested, you know what? if you want to hire use a random number generator because that is at least fair you have the same fair chance as you you and you to get to get picked obviously everyone is like it's also a way to go bankrupt as a company I mean that's like a whew I'm all for like but that's also like chaos there's random and there's chaos you know what I mean if you're going to say to people a random number just bring the person in yeah I don't if they have the basic capabilities oh okay so you're going okay Okay, so you're going basic capabilities and then you've got the qualifications and it's random.

1:01:56I'm in for that. I'm down. Yeah, try that. I'm down. Wait, so you know what I want to move on to is like the, we're talking a lot about hiring. Yes. Your work really delves into keeping the job, which I think a lot of people aren't aware of and might even be more terrified to find out about. Oh, yeah. What we see at the surveillance at work. Yeah, like for instance, and I know there was an explosion of this during COVID once people were working remote and then companies were like, we need software to know whether people are actually in their underpants or not. And we need to figure out like what people are doing at home.

1:02:30But now companies are starting to deploy AIs that not only see how like active you are, but they try to predict whether or not the company should fire you, not based on what you're doing now, but what the company thinks you might want to maybe do or not. Yeah, I mean, I think it's often like, you know, it's called like a digital neighbor or something, Sort of like the idea is like you are a vice president of sales of North America, so they might be a vice president of sales in Europe. And one of them is like might be more successful or not. That's actually kind of vague and hard. But for the sake of this example, we'll assume, OK, maybe the European person is better at their job.

1:03:13And so then an AI will like sort of take in all of the digital traces that you leave. How many emails you send? How many Zoom meetings you attend? Are you a bully in Zoom meetings? Do you speak up? Like you can kind of assess a lot of different things and then tell the person in the US like, hey, the person that has your job in Europe and like sells more or whatever, like is more successful, they do this. Why aren't you doing that? It's sort of like a clone of like looking at all of their, everything that gets recorded. And, you know, it's sort of like, I don't know, we have different ways to be successful.

1:03:47Like maybe you write 500 emails, the next person is successful by doing like 100 in-person meetings a week. That's probably not possible. But, you know, maybe they do 50 a week. Who knows? But we sort of, and you know, what does it mean to be successful? Like we had this like whole thing, probably don't remember this, and I might be dating myself, but there used to be like algorithms in New York City to assess teachers like 20 years ago or so. Like every parent was like, I wouldn't know about my teachers. Well, Well, it turns out like these algorithms were terrible. And a lot of teachers were like put in rubber rooms because their students didn't gain enough knowledge in a year.

1:04:24But it could be that they were already at the top. Wait, the teachers were put in what? They're called rubber rooms when like when like teachers were not in the classroom anymore, but they were still on the payroll of the Department of Education. They called them rubber rooms at the time. Rubber rooms. Yeah. Because in my head I was like picturing a room. They had to go somewhere to work. It sounds like a room in a rubber? No, it sounds like a cell. I think it wasn't a cell. Okay, no, because you just went through that. You were like, they put the teachers in rubber rooms, and then I was like, wait, they did what to them?

1:04:52Oh, they just called it a rubber room. I don't actually know the history of that. Good question. Yeah, I want to know. You want to know about the rubber rooms? Yeah, no, no. If someone's taking me to a rubber room, I want to know what a rubber room is. I would actually take you to the rubber room. Oh, wow. Oh, my God. I don't know if I want to go there. Wow. You got to get paid for free? You don't have to do nothing Wouldn't you want to be in a rubber room Play squash Good place to fall in a rubber room I still can't believe how Digital peeping Tom And the digital tattletales It's just everywhere now It starts like super benign With your green light on your email Are you active or not That's sort of a way And then we see when people realize Oh everything gets recorded we see sort of what we call productivity theater you know that people like slow it down did you say productivity theater?

1:05:47yeah it's like like sort of gaming the algorithms so you sort of so like acting like we're busy exactly so like in the morning like you check in on Slack and be like hey everyone good morning like 7.45 crazy and then you turn around and take your dog for a walk and then you don't show up at your desk at 10 but smoke and squeans you were like you were productive at 7 45 uh well you know an algorithm will now be able to uh understand that you haven't said any emails in an hour you've just done though you have in a single sentence unraveled one of the greatest mysteries i have struggled with working in an office i remember the first time and only time i worked in an office I was always shocked by how some people Were just constantly sending emails and messages And I always felt like they were unnecessary And they were always at random times Sometimes on a weekend I was like what But now when you put it that way I go they weren't working They were trying to maintain the appearance of working Productivity theater So you just like yeah you send a message at 6am and people are like, man, you up at 6 a.m.?

1:07:02Yeah, wow. Emails at 3 a.m.? What the? Well, you just don't stop working. Yeah. And, you know, I do think that - Meanwhile, you just left the club. Send. Schedule send. Schedule send. Schedule send. Oh, schedule send. Look at this. Yeah. But, you know, think about it. Like the office was like sort of always a place to look for productivity, right? Because you had a manager look at everyone who was working. And if you left early, that was not so good. Even though, you know, we know that some people just like set at their computers or the internet and didn't do any work, but they were physically at their seats.

1:07:38We didn't have the technology to actually like sort of see every one of their clicks and what they're doing. And now we do. And sort of, we can sort of look at everything you do. But like, the question is like, is this kind of analysis really meaningful to understand how many emails you sent? does that actually have anything to do if you are productive or successful? What does successful in this job mean? Those computer systems you're speaking about, I remember reading about how Warehouse is also using it. This is something that I hope people understand will be pervasive across all jobs. Because if you work in an office where you're using a computer, they can track your clicks, they can track your typing, see what you're doing and how you're doing it.

1:08:21but in warehouses I've seen that now they're deploying AI camera systems that see how many employees take bathroom breaks or don't take bathroom I swear how long you spend in the bathroom how quickly you actually move one package over to the next how and they look at that how many items do you put in a box per minute per hour I imagine your bladder your bladder is the reason that because you've got a smaller bladder than another person you're getting fired Technically that would be illegal But yes Yeah but they wouldn't say It's because of that Because they would just go like You take excessive bathroom breaks Yeah Or you're falling Under your productivity Exactly Because the other people around you They're hitting these numbers Why aren't you hitting those numbers As a conspiracy theorist I always say Who is benefiting from this Who is Because I look at COVID And you explain to me How tough COVID was in the city Yeah But if you look around the world how many running shoes have suddenly become in fashion, how many running clubs, how many running apps are being used, how many outdoor activities, hiking, you name it, that people are now having invested themselves in and investing a ton of money in because they missed being outside so much because it was taken away from them.

1:09:36Could it be that people that fund startups are now having the time of their life because they realize there's these educated people who are trying to get into the job market with these kinds of expertise and these kinds of interests But maybe they're not going to get in there. So how about we give them a hand and make money out of them? Sure. I mean, I think the way we see this kind of technology benefit is usually the companies because that's where the money is, right? Is an individual going to buy a success company, success AI? We don't really see it. It's not really a market, right? The same way for job applicants.

1:10:13There is some AI where you can sort of test your resume and the job description. But we see like vastly outnumbered AI for like vendors, the people that make the employment decisions, those folks, because that's where the money is. Like I sometimes dream of like, you know, we were talking about bias and I was like, you know, wouldn't it be cool if you have like a bias detector in job interviews that pings the hiring manager? Like, stop talking about your schooling. Like, you know, this is like where bias creeps in or at least analyze afterwards. So you get like real time feedback like, hey, you shouldn't really ask those questions.

1:10:47Like stick with the structured interviews in a job interview, for example. And we don't see that because I don't think there's really a market there to do that yet. You know, I sometimes feel like, you know, wouldn't it be cool? Like I have a young kid. So like if you're like a parent and you have a little AI who's like, hey, you really shouldn't get so upset with your kid. You should really say, I like how you did this and this. But like, I think a lot of parents wouldn't want to do that because as soon as you have the data, somebody else, like Child Protective Services or whoever can come in and look at that and be like, the way you talk to your kid, no good.

1:11:25Like no one wants that, right? You're not fit to be a parent. We'd love to hire you as a manager at our company. How's your bladder? You have the personality to enforce the algorithm. So actually, let's talk about that then. as somebody who's investigated and gone down all of these rabbit holes, as somebody who's seen how AI is affecting who gets hired and how you get hired, who gets to stay in the job and how they get fired. Yeah. As somebody who's done all of this work, I'd love to know what you think some concrete solutions could actually be, like where we see progress, where we see solutions.

1:12:00Is there something, let's break it down. Is there something lawmakers can do? Is there something that companies can do? And then is there something that just workers can do? Yeah. So I do think there's room for improvement in all levels. So I do think that there could be better laws here. For example, what we see, you know, the funny thing is like, I am originally from Germany, but I remember talking to the former head of talent acquisition at Vodafone, which is a huge telecommunications company in Europe and other parts of the world, not so big in the US. And he was laughing. He's like, you know what?

1:12:39We use AI in hiring now. And when you want to upload your resume, there's Germany and the rest of the world. Because Germany has this one funny thing that once you're working in a company and you have, I think, more than five employees, they can have a workers' council. It's not a union. Sounds like it, but it's different. And the workers' council, there's actually a law and they get to co-decide technology in the workplace. So some of the surveillance technology we don't see happening in Germany because the, you know, this workers council has to be notified. And I think a lot of companies shy away from using some of this like very intrusive AI tools.

1:13:16But in the United States, for example, like anything that happens on a work computer belongs to the company. So like, don't do it like private Slack messages, like private surfing, like all of that can be recorded by the company and it belongs to them. So you want to be very careful of that. So I think there needs to be many more privacy protections. And I think companies should tell, should be mandated to tell their employees what kind of software they use in them. So for example, like some of it is like very basic, but like if you suddenly print a lot, that might be an indication that you're at flight risk.

1:13:46So maybe the company lawyer should be looking into what you're moving away from your computer. like those kinds of like sort of digital tales. You know, I think companies should tell us and maybe there should be a way for like employees to co-decision making. Because some of the time, you know, if you're working a nuclear power plant, maybe you do want AI to scan for like exposure to radiation. I would want that. So, you know, there might be cases where this is like actually really helpful. And maybe everyone agrees that like, you know what, printing is a problem. You shouldn't be printing so much and you shouldn't like move files and there could be an indication that you're leaking, yadda, yadda, yadda.

1:14:24Like maybe we can make a decision together, but we don't see that. So it's like all top down and people are, you know, these kinds of tools and decision makers are being used on them and they don't even know it. And I think that's really unfair and there's no way to push against that. I think also like companies need to be much more skeptical when they buy these AI tools, not believe the hype that this is going to solve all their problems. They're going to hire the best people, like actually show me, show me the evidence, like show me how it works. I'd be happy to look at it. And, you know, I'd be open to it.

1:14:59Like maybe an AI is better. Wouldn't that be great? But we need to know. We don't actually know that kind of stuff. So we need to interrogate these algorithms, understand the processes underneath them and really critically assess them. I think that's where maybe humans are coming in in this world. So we need to be much more skeptical there. And then as like the applicant for jobs, that's the hardest part because there isn't necessarily something you can do except like call your congressperson and sort of be aware what is out there and like try some of the tools. Like, you know, there's definitely better ways to, like, have a machine-readable algorithm and there's things you can do.

1:15:37But, you know, when, like, 5 ,000 people apply for one job and they close the job description after, you know, they close the job portal after 24 hours, there's nothing we can help you with there. Like it's, you know, it's sort of like a bigger societal change to be much more skeptical about these tools and put pressure on lawmakers, decision makers to do a better job here and to just be more transparent. Like one of the stories like of Martin like came through because he lived in the European Union and knew about the laws and he asked for the data. Like there is a general privacy protection law and you can ask for your data that companies have on you.

1:16:16And that's how he found out that the company used AI, which was against the law, la, la, la, la. So he got a settlement. He actually started a case. So that was like a goldmine for me. I call him patient zero. Because he's sort of the first person who, like, encountered these kind of AI tools in the hiring phase and then actually got the data on himself, right? That's, like, gold to me. So we could sort of unravel and talk about the case because we had the data. And we don't have anything like that, at least on a federal level in the United States. So there's like way more work to be done to make this better.

1:16:49And I do think in general, like I do like I think we talk a lot about like sentencing guidelines with AI to send people to prison. Should you get a mortgage? And I think those are all very consequential decisions. And we absolutely need to take a closer look at those and look at them critically. But I also think hiring is really important, too. Like it matters if I can pay the bills, like it matters if I can put food on the table. Also, happiness is tied to our jobs for many people. We spend enormous amounts of hours at our jobs. So it better be something we kind of like, at least. So it matters if I get the job or not.

1:17:27So we really should be scrutinizing these kinds of systems if it makes decisions on humans. If it makes decisions about my spam and it doesn't work, I'll find another spam filter. Fine, great use for AI. But for hiring in these critical human decision-making where human lives are at stake, you've got to be much more skeptical, scrutinize these tools. And then we probably have a chance of building a better world. Well, I will say there's one part of the equation I'm very grateful for, and it's that we have an intrepid investigative journalist who's doing the work. Thank you. Sometimes you wonder, does my work have an impact?

1:18:02But I do think sometimes when I show people my videos from eight years ago about the emotion recognition of facial expressions, and they're like, wow, that could be so easily biased. And I was like, wow, I guess our work sort of like has made a difference because eight years ago we were all looking at like, whoa, who knew? This is so cool. And now everyone is like, oh, wait a second. Like if there are only like, you know, more men than women in the data, la, la, la. And I was like, wow, there is like sort of a much more education around AI and bias and all of those things. And I think it has made an impact slowly but surely.

1:18:37Slowly but surely. but I'll tell you now I know for my next job I've got something to think about when we get out there in the streets and from my side please work with me on hiring let's do like summer thank you very much you know from me you know from Syria from Canada and Thomas we just want to say thank you very much let's flip a coin on hiring and see how it works thank you very much thank you

1:19:03What Now with Trevor Noah is produced by Day Zero Productions in partnership with Sirius The show is executive produced by Trevor Noah, Sanaz Yamin, and Jess Hackle. Rebecca Chain is our producer. Our development researcher is Marcia Robiu. Music, mixing, and mastering by Hannes Brown. Random Other Stuff by Ryan Harduth. Thank you so much for listening. Join me next week for another episode of What Now?

1:19:35Pandora makes it easy for you to find your favorite music. Discover new artists and genres by selecting any song or album, and we'll make you a personalized station for free. Download on the Apple App Store or Google Play and enjoy the soundtrack to your life. You fell in love at first, Crunch, and now the crispy chicken crunchwrap slider is back at Taco Bell. But with a twist, it's crispy all-white meat chicken nuggets inside a slider-sized crunchwrap. But now you can enjoy them with the kick of Hidden Valley Diablo Ranch. Or try the tortilla chip-coated nuggets on their own. They're waiting for you.

1:20:07The crispy chicken crunch wrap slider, a brand new classic, is back. Only at Taco Bell.