AI Ethics Expert: The AI Myths You SHOULDN'T Believe

0:01I am so glad you could join us. I'm your host, Mo Gaudet This podcast is nothing more than a conversation between two good friends sharing inspiring life stories and perhaps some nuggets of wisdom along the way This is your invitation to slow down with us Welcome to Slow Mo

0:34Welcome back. Today I am in Al-Khubar in Saudi Arabia. I'm attending a conference called SYNC. And it's my second time here. The first time I came was in 2022 in March. And I have to say, I marked it as one of my favorite conferences ever. So SYNC basically is focused on digital well-being. It's an attempt to use the wealth and connections and success of Saudi Arabia to create a get-together for people to discuss what I tend to believe is one of the biggest challenges to our well-being today, the digital world, the advancement of the internet, and now artificial intelligence. And SYNC just has that very interesting vibe to it.

1:17It's an event that's sort of loosely organized. It's not that tremendously formal, even though it's ticking like a clock. But most of it is just conversations on stage between different people with different opinions and points of view, aiming to make the world slightly more alert, more aware of the challenges that we have with digital well-being. So today I have several guests that the SYNC team has kindly arranged for me to meet with, and the first of whom is a Bangladeshi-American, MIT graduate, PhD in technology, who is the U.S. envoy of AI globally and who is the CEO of Humane Intelligence.

2:05Ruman Chowdhury is a voice of reason in the rise of artificial intelligence. We, from what I've seen from her work, agree on many things, don't agree on some things, but we agree that humanity needs to engage in a different way if we were to see the benefits of artificial intelligence. She brings wisdom, I think, to the conversations of technology, which are normally all about data and facts and possibilities. So I think you're going to really enjoy this conversation with my first guest at SYNC, Roman Shoudi. Roman, thank you so much. First of all, it's an honor to see you here in Saudi. We share a few things in common.

2:50And so like me, who is now much more focused on sort of helping the world become a tiny bit better, I did spend a lot of my time in the wheels of the corporate world. I know you worked at Accenture. You worked at Twitter as well. That's right. Yeah. Tell me to start if you believe that the promise of the corporate technology players is being delivered And why are you not part of the tech world and of the corporate tech world anymore? Thank you so much for having me on the show. I'm super enthusiastic to talk to your listeners who I know span the globe, have a lot of different perspectives. So, you know, your question is interesting.

3:34Are tech companies delivering on their promise? Well, Silicon Valley is such a fascinating place because really never in the history of capitalism, have we had corporate entities so wholeheartedly embody or try to embody humanistic perspectives? So they don't say we're building a widget for you to use. They say we are connecting people all around the world. They don't say we're finding ways to find things on the internet. They'll say we're revolutionizing access to information, right? We don't think of cars. Car companies don't say, you know, we enable people to be connected all over, you know, like they don't have such romantic visions.

4:16And I don't think it's disingenuous. I think a lot of these founders truly believe or want to embody this in the technology that they're building. It's part of the lifeblood of Silicon Valley as well. I do think though, that realizing a humanistic vision is not going to be purely corporate driven. So the second part of your question was, well, why am I no longer in tech? I will say, you know, I've tried to carve out a space of being integrated into technology, right? So I work with all of the major AI companies. I just came from the AI Safety Summit in Seoul, where I saw a lot of peers and colleagues at Microsoft and Google and Salesforce, et cetera.

4:55But I do think that any healthy ecosystem has a wide range of players. I find that to be very accurate to my years at Google. So you sort of because when i joined google honestly i totally believed in the mission to organize the world's information oh my god what a cause to give my life to and i'll tell you openly the first day the first weeks i walked the corridors that was very true like every single one of us was blinded by the mission and then i remember vividly one of my favorite bosses at google saying you know I'm worried that we're going to become old and boring. It was the words he used.

5:38And basically the idea that you start to hire people that run the business like a business. And, you know, suddenly your vision, your mission, your ambition, your aspiration becomes secondary to your quarterly profits and the way things run. And then you find yourself. Which plenty of people would say that's exactly what, you know, some of these companies are now. and I've heard this analogy about Google that people say that Google is no longer the company they joined, that a lot of the, so the specific one I've heard is there used to be the, you know, Fridays are for... TGIFs, yeah. Right, like doing whatever you want, exploring something that's passionate to you and that's just not the case for most Googlers anymore.

6:16Oh, the 20 % time, yeah. The 20 % time was canceled as well. I'm not sure whether it was formally removed, but it's one of those things like a Twitter. It was formally removed, yeah. Was it really? Or, you know, at Twitter, we used to have no meeting Fridays, but let's be real, if you're a people manager, there's no such thing as a no meeting day, right? So it's on the books, maybe for some people, but it's not the case for most. But, you know, I suppose I would counter that to say, is that such a bad thing? Corporations are being corporate. Is it? I don't know. I mean, the thing is that they never really stop making the promise, right?

6:48That's right. So it's that disconnect which can be a problem. It's that disconnect, right? And that can be the problem where, and I see this a lot in responsible and ethical AI, where people are initially, they're disillusioned. People who become, let's say, whistleblowers, right? Are not people who came in with an agenda, right? They're people who came in and truly believed the mission, but then the mission was out of sync with the reality of the situation. And in every single case, they'll all say the same thing. They were true believers. They were not cynics. They were true believers. And they came in and they saw something that was misaligned with the vision, right?

7:22but when they tried to let's say raise it internally they were blocked or they were stopped and being a whistleblower for the people who chose to do it it was a it was a last resort it was not their agenda going in but it was only when the institution failed them because there was a misalignment of their expectations in reality which is why i sort of you know introduce a little controversial statement is it so bad so what if we're like okay well when you work at a corporation you're doing corporate things it's not necessarily a bad thing in a sense right corporations should have a mind to the impact on society but their goal isn't to resolve impact on society like their legal structure is to make profit for shareholders that is what they are there for so even if you're a b corp so if we're thinking about large language model companies anthropic is a b corp right does that mean anthropic's gonna stop chasing profit no it doesn't.

8:14Of course not. Yeah. And as you rightly say, I agree 100 % that there's nothing wrong with that. As long as you wake up every morning and say, yeah, we're here, we're going to focus on profits, right? That would be a way of really making a promise that you're going to keep. And I think the struggle really is, which is really my segue into why you do what you do, is I too believe that I could not make corporate prioritized impact from within the corporate world. That's right. It's hard. I struggled with that a bit at Twitter. And here's where I think having a not overly idealistic view helps. If you go in and you understand, so I'm a social scientist by background.

8:56I look at institutions and incentive structures. And maybe there's something a bit cold and calculating about that, but it helps me navigate institutions. If I understand that this is a corporate entity, at the end of every quarter, I was at Twitter, Jack Dorsey, they have to get in front of shareholders and say, This is how much money we made, so many users we have. Impact is a very nice story, but whatever you're doing with an impact, you have to figure out a way to align with that. So there were things that my team at Twitter worked on that I had a mind to, okay, I have to pitch this as something that will drive the metrics that matter.

9:31And sometimes I worry that people who go in with an overly idealistic view, they're overly romanticizing it, they are offended by having to do that. Yeah, I mean, in the original Google, and by the way, I always make a public statement that from the outsider's view of the world we live in today, I still think that Google is reasonably ethical in the way they do things. They reasonably are driven by making a difference. I know personally. It's hard to do what they do. It's very easy to be on the outside and point fingers. Of course. But once you are handed the responsibility of problem solving, you truly understand how difficult it is.

10:07Yeah, and I will say openly that, you know, I still believe that Sundar or Demes or, you know, the leaders, I tend to believe that they're truly, and I know them personally, I've worked with them sometimes, you know, sometimes in my career, I know the kinds of decisions they make. You know, they are driven by making a difference. But, of course, navigating the metrics, like you called it, is very, very important, which is, you know, some of the more interesting, you know, situations for me in the original Google were most of us were like Demis, to be honest, right? Most of us were like Sundar Humble.

10:45You know, I'm not that intelligent, but everyone was really super intelligent. they basically took care in in the power that they were given and the conversation of don't be evil was actually on the table all the time i think then you know when the bureaucrats as we used to call them lovingly because you need bureaucrats to run an 80 billion dollars business you know when the bureaucrats were confronted with the topic of don't be evil they were like Like, let's define evil. Right, right. There's a little asterisk. Exactly. And then there's the fine print. Exactly. Evil is defined as, like, overtly breaking the law, for example.

11:24There you go. And so we see this in responsible AI quite a bit, right? Where I have a lot of conversations about how responsible use is not compliance. Correct. Compliance is the floor. Yeah. That's the bare minimum. You are not breaking the law. It's not ethical and responsible. It's like, congratulations, you woke up and brushed your teeth, right? That's compliance. Yeah. Ethics goes above and beyond. because now you have to have a vision for the world you want to achieve. And you are actively doing things to achieve that vision. And the hard part of being a leader of one of these companies is you're really balancing this quarter over quarter reporting need to a long-term vision.

11:59And something like fairness, responsible use, things like that build trust. Trust takes a long time to build. So one does not overnight or in one quarter or maybe even one year create a more ethical, responsible like technology stack but the current reporting cycle pure shareholder analysis would say what did you do this quarter what did you do year over year i don't care about five years from now yeah and i and i think i think that's really where it becomes you know i i used to i second your your point of view about ethics and you know the the bare compliance minimum i i i write about that publicly and i say being legal is not always ethical right oh yeah that's certainly true the The law has, in many cases, been very unethical.

12:44But also, I worry, well, and the thing is, in order to be ethical, you have to have a perspective. And you have to have perspectives on things that are not necessarily 100 % agreed on by everybody. Correct. And you have to have perspectives that could be controversial. Correct. And you have to be willing to stand by your moral framework, even if it is different from somebody else's moral framework. So today in this sort of new generation of frontier AI models, we're almost seeing companies trying to not have a perspective and trying to technology their way out of it. What do you mean by that? So, for example, one of the common methods of training, quote, values aligned AI systems is something called reinforcement learning with human feedback, RLHF.

13:33RLHF is basically getting Amazon workers to rate output, using that to create a model and using that model to train another model. By doing that, nobody has to have an opinion, right? And then you get to say, oh, I don't have an opinion about the Holocaust or I don't have an opinion about gay marriage, right? We are reflecting the opinions and values of, quote, people. But there's a couple of things wrong with that. Number one is there's no such thing as universal values. There isn't. Even if you think of something that we should all possibly think that we agree on. For example, people shouldn't murder other people.

14:14Well, I live in the United States where the death penalty is still on the books in many states. Don't we murder people in the United States? Yeah. We have very lax gun policies. Don't we allow people to murder people? So even if we say something that we would think that everybody universally agrees on, right? Human beings should not kill other human beings. We already see gray area within that. So trying to achieve universal values is an impossibility. The second is the construct of the way this feedback works, right? Again, I'm a social scientist. What people say their values are is sometimes discordant with their actions.

14:49Yeah. Right? So people will say they want better schools and better roads, et cetera. But they will also say, why did you raise my taxes? Right? So we want to pay less money to the government. Government's evil and corrupt. But also, schools need to be better. That's incongruous. But they don't see it. As human beings, we're immensely capable of holding completely discordant thoughts. It's one of the most amazing things about human beings. So I want to build the story for our listeners here. So we've established we cannot rely on corporates to create an ethical framework for us because they will comply within legal frameworks.

15:26Well, I guess I'd say they're part of the ecosystem. They're not solely going to do it. Yeah, so they'll comply with a legal framework, but legal is not always ethical. and you know we cannot rely on governance or society to tell us what is right and what is wrong i mean not society maybe is the wrong word but you know we can't we can't let governance tell us what is ethical because we don't really agree to what is ethical governance in the u.s as you rightly said says yeah you have the right to hold the gun when it seems to me like a step away from murdering others right so so the story we have here and just to bring context to our listeners is that over the years, as the Internet started to shape our behaviors, our characters, our beliefs, our moralities in many ways, we're now about to hit the second wave of the Internet on steroids, if you want.

16:21Like, you know, artificial intelligence is truly magnifying what humanity is. And when you talk about things like reinforcement learning, that idea of asking employees of open AI to tell us what their views of things is, this is not reflective at all of the rest of the world. It lacks understanding of different cultures, different languages, different value systems, different traditions. and you know the joke i always give with all due respect to all opinions is that if an open ai employee says a man killed a woman in pakistan the typical reinforcement learning of california would be don't call them men and women this is a gender identity issue right and in reality you know and that's true we're californicating artificial intelligence now that to me is probably one of the top societal dilemmas that the world is about to face.

17:16And I heard you once say something that I really took to heart, which is where you said, AI is, I'm paraphrasing here, but you basically said AI is not our innovation, it's our mirror. It's basically reflecting to us what humanity is. What does that mean? So artificial intelligence is built on data that's collected about human beings. I think we all know the world is not always a fair place. The world is not always a safe place. I'll also add that the quote, the internet is not the world. No, most of the world is actually not reflected on the internet. And I appreciated your term, like the Californication of everything, because the internet is largely an English speaking Western place.

18:00So the values, the methods of communication, they're very Westernized, right? Even the design decisions of saying most platforms are natively in English. No matter what part of the world you live in, you usually interact with social media platforms, search engines in English. Even if you are typing the anglicized letters, the phonetics of your language. My family is South Asian. We see this actually all over India, Pakistan, Bangladesh. The way they communicate and talk, even over text, is they will type the phonetics of the word in Bengali but in English characters. All these things matter. And they matter because all of that perspective, all of that way of doing things is put into these AI systems.

18:41So there's a couple of problems here. The first is that, as I mentioned, and as you've pointed out, the Internet is not reflective of the world in its entirety, in its beauty, in its diversity. It's reflective of English-speaking Western perspectives, by and large. The second is that this data was not ever natively meant to build artificial intelligence. When you went on Facebook or Reddit or Twitter, did you think, oh, one day this is going to train an AI model? Wow. That is going to be magnified and rule the world. No, so this data was never intended for that purpose. So that's second. And third is, you know, just the abstraction from individual situations, interactions, scenarios that are sometimes historically old and outdated to try to mash all of that together without context, without temporality, without understanding how the world itself has changed, just sort of flattens the diversity, right?

19:38The complexity of what it means to be human. So we're never going to have these AI models are parrots, right? their mirrors, they're just reflecting the things that they're being fed without any context. So it makes it worse. Oh, yes. That's actually really interesting. So you would take, I asked Chad GPT recently what my wife's name is, and it answered with so much confidence the wrong name. It's quite interesting, actually. What most models do when you ask it about me, it generates a biography that is actually an amalgamation of multiple women in my field. Oh, interesting. Yeah, so there are a few prominent women in AI ethics, responsible use, who have worked at companies, like Google and Twitter and Microsoft, et cetera.

20:25So the biography that it writes about me is actually, I know how they've pieced together other people's bios to do this. So it'll say something like that I'm a professor at the University of Washington. That's not me, that's Emily Bender. That I worked at Google. That's not me, that was Meg Mitchell and Timney Cabrew. And just like, it pieces together the wrong story. And it doesn't apologize. It doesn't say, I'm not sure. I think it could be. So I'm glad you raised that. So what I do with my nonprofit is I do these exercises called red teaming. And the purpose of red teaming traditionally has been to bring hackers to break into systems.

20:58I'm expanding the idea to say, when we're breaking a system here, it's not just about hackers causing malicious content. It could be exactly what you're talking about an incorrect representation of a person, of a society, of a culture. So we did a red teaming exercise with scientists to look at COVID and climate's mis - and disinformation with language models. And I love these exercises because you see the technology through somebody else's eyes. Like even if I am a quote outsider in tech, I'm very much of tech, right? So I don't see things the way a scientist does. So what they told me was exactly what you said.

21:32What they said was, well, the fundamental problem with language models is that they speak very truthfully, right? So they speak as if what they say is the truth, when most of science is about inquiry, conversation, and discovery. So if you talk to scientists, they never say, well, that's the answer. They'll always say, well, the research has this, and then there's this other research, because science is understood as a language. Language models don't impart science like that. They'll try to say it like a fact. And it was interesting that scientists pointed out the same thing that you just said.

22:05There is such an arrogance to it, to be quite honest. I mean, when I worked at Google, one of the things we took pride on in organizing the world's information, and I was one of the first people when I came into Google that tried to object and change that. And I was taught by Google that, you know, for example, I remember there was a video on YouTube that said that Ataturk, the founder of the Turkish nation, is gay, right? And I basically said, but Turks believe otherwise. Why are we showing this to Turkish people? And couldn't we just IP block it from Turkey and basically just avoid a conflict and people being upset about it?

22:46And Google's view was, what gives you the right to say what's true and what's not? That's right. I worry about that quite a bit. And this is why I said, is it so bad if we treat them like corporate entities? because the dark side of believing in this mission is that ego. It's that belief that you know better than everybody else. And even, you know, we see this even in the field of algorithmic bias when we think about bias and discrimination. My team at Twitter did research demonstrating that in seven out of eight Western democracies, the Twitter algorithmic feed versus chronological feed showed content that was more center-right.

23:23So people took it to say, oh my God, algorithmic bias, you need to fix it. And actually, that's a misinterpretation of our research. First is that we see the output. We don't actually see what caused it to happen. My working hypothesis is that what caused it to happen is the algorithms were working as expected. The purpose of social media algorithms is to give you content that is like the content that you are already engaging with. Our data was collected from April to August of 2020. Guess what most of the Western world was talking about at that time? Center-right politics. We had elections in Hungary, in Germany, in the United States, right?

23:56So the question actually is, to your point, who gives anybody the right to say what is political bias and not bias? Or to flip the question, what is a politically equitable output? Do we say 50-50, Democrats, Republicans? That's an arbitrary distinction. Yeah, but if you really look at the wider spectrum of technology and you take Google search on one end and language models on the other end, Google search would tell you, look, you're looking for an answer. I have a million for you. Okay. Million websites. Go look through them and find your own truth. That's right. I'm not, I'm going to rank them in order of how many people are interested in each, but I don't know.

24:36I don't really know what you're interested in. And it's not the main driving factor. Right. I have some context of where you are in the world. So I'm going to, you know, try to show you that, but I'll still give you a million versus language models that will say, I'm going to average the million and give you my point of view. It's, you raise a million, you just like hit the nail on the head. So last week was Google I.O. Yeah, I watched it. Yes, but you know, one of the things that they're basically saying is they're aiming to replace search with these sophisticated AI systems. Yeah, Gemini included in AI search.

25:08Exactly, exactly. And we see the early days of it, right? So there's sort of AI, if you do any sort of a search engine, you'll get some sort of AI generated output for better or for worse. Now, you are absolutely correct. And if there's one thing I want your listeners to understand is that these language models are not information retrieval systems. A search algorithm is an information retrieval system. In other words, there is a pool of information and it's sticking its hand in and there is a formula to how that information is pulled out, right? Social media algorithms, similar, right? There are tweets, there are posts, and your algorithm sticks its own hand in, shakes it around and pulls out the three or four things that you're going to be interested in.

25:46Language models are information synthesis machines. What they do is they take that bucket of tweets or that bucket of websites and to your point, mash it together. Again, and this is why it matters that they have no context. They have no temporality, right? It does not understand that maybe a fresher website is better or a perspective is better than an older one or that there's a difference between CNN and some random guy's blog post or whatever, right? And I'm oversimplifying a lot of information integrity that goes into analyzing the data, but not enough. So information synthesis, the thing I worry about is it's telling people not to think for themselves.

26:26So to your point, the example you gave earlier is information retrieval would say, here's 100 websites. I have no idea. And I'm not going to try to understand what your intent is for doing the search, but I'll give you the tools to figure it out. It's like going to a library and picking your book. instead of this large language model world, which is doing the thinking for you. Because right now your brain does the synthesis. They're saying, don't even bother with that. We'll do that for you. So what would your advice be for someone listening to this? In this new world where you're being told by seemingly a superior intelligence what to believe.