There Is No ‘Existential Threat’ From AI Says Timnit Gebru

0:00AI can make businesses faster. It can also make mistakes, expose data, and execute the wrong instructions faster. ThreatLocker helps organizations set boundaries AI cannot ignore by controlling what applications can run, what they can reach, and how they interact with sensitive resources. Innovation needs more than good intentions. It needs limits. Control AI with zero trust at threatlocker.com slash wired. That's T-H-R-E-A-T-L-O-C-K-E-R dot com slash wired.

0:34From Wired, this is The Big Interview. I'm senior correspondent Lauren Good sitting in this week for Katie Drummond. So in the past few weeks, you've probably noticed that the topic of AI safety is everywhere. Recent testing is showing advanced AI models acting autonomously, quote unquote going rogue, and hacking computer servers without human permission. And all of this has led some AI insiders to issue these doomsday scenario predictions. To help make sense of it all, I'm talking today with Dr. Timnit Gebru. Timnit is the founder and executive director of the D.A.R.E. Institute. D.A.R.E. stands for Distributed Artificial Intelligence Research.

1:19Timnit is also a co-founder of Black in AI, a nonprofit working to advocate for the Black community in the field of AI. And early next year, she's coming out with a book called Deep Unlearning, The Rise of AI and the Radicalization of a Tech Idealist.

1:40Timneet, thank you so much for being here on The Big Interview. Thank you for having me. To set the table, you are someone who came to public prominence in the tech world by being fired by Google for raising the alarms on large language models. This was based on a paper that you had written. We're going to talk more about that later in the episode. And so I think you are uniquely placed to contextualize and ground us in this moment of concern about AI. You know, for me, even as a Wired reporter who's been covering this really closely for a long time, you know, the discourse around AI safety has been pretty remarkable the past few weeks.

2:20I'm going to assume that a lot of our listeners know this already. A few weeks ago, a young AI researcher, Edanthropic, basically stepped down and in his resignation post on X indicated that a lot of people within the company feel that we are facing this major existential risk with AI. And then a few people responded and there was this whole pile on and discourse about the real existential threats of AI. And when you and I spoke in the immediate aftermath of that, you said that you felt that particular narrative was a distraction from the real problems that we're facing with AI. I would even go further than that and say that it's dangerous.

3:01It's not just a distraction, but it's harmful discourse. And so the reason being, before I answer your question, you know, I try to do this. I want to give a little brief history of how this narrative has been going on for a long time, because I wrote a Wired op-ed in 2022. We wrote a statement in 2023 when this happened. Elon Musk and Peter Thiel have been saying this since 2013. Prominent people have been saying that AI is an existential risk to humanity. And the way it goes is usually, and this is exactly the same rhetoric every three years. So I can regurgitate whatever I wrote 10 years ago to respond to these things.

3:41For example, the Future of Life Institute. And what is that? Yeah, the Future of Life Institute was founded by Max Tegmark and Jan Talon, a billionaire who led Anthropics Series A funding. And he was a Skype co-founder, correct? Skype co-founder. Jan Talon also funds Meter, which is the report of an auditor, so-called third party whose report went viral about these rogue agents hacking hugging face. So to the public, it might seem that there are so many different entities coming together saying the same thing. But just like the tobacco industry and fossil fuel industry playbook, there's a playbook here where the same billionaires who founded and funded Anthropic stand to benefit it so much more from its IPO more than anybody else.

4:32Also founded and funded the institutions warning about existential risk, quotation marks of AI. Also founded and funded the third parties that are being cited right now as if they are like an independent entity. So that I want to ground the public in that fact. And I can say more about that. Okay. Tell us why you thought the machine got there. It's a distraction, right? It's even more than a distraction. It's harmful. For example, the reason I was telling you all these facts is that these people, the funders, the founders, the investors who stand to benefit and profit the most from these companies IPO-ing are saying this, that they have been the ones seeding this narrative going back multiple decades, actually.

5:17People should ask why. If I stand to get lots of money from a particular company, why do I seem to be the person saying this particular company might build something that kills us all, right? It sounds counterintuitive. But the first thing is that if you convince the public that you're building something so powerful, it's beyond even your control. It's beyond something that we've seen before. is beyond something that we don't have existing regulation for. This is why I have a lot of respect for Lina Khan. She said there's no exception to the current laws we have for AI companies. And they talk as if there is an exception.

5:59You're saying whatever you're building is so powerful that it's beyond anything we've seen before. It's self-aggrandizing. It's saying we're the future makers here, but the future is going to be really scary. They don't always say the same people. you have to pay attention to what they say in the news cycles. When they say that AI can cause an existential risk, they also say that AI can stop climate change, bring us world peace, and eradicate poverty. AI can do all of these things because it's powerful. It can bring us utopia. But also, if the wrong people do it, and we don't have guardrails or whatever, it can also just kill us all.

6:35So the same people believe these things. There's only a few people who might be in one of these camps. When you're telling people that you have the super powerful machines, what you're speaking to, first of all, is to your investors. You're saying that, you know, you want to get your hands on these super powerful machines. You're talking to governments saying, you don't want your adversaries to get their hands on these super powerful machines. You want to get your hands on these super powerful machines. And you're also telling any regulating bodies that whatever regulation they're thinking about should be about these fictional, super powerful things.

7:11And imagine if I'm talking about potentially eradicating all of humanity, things like pollution, data centers, they sound kind of small potatoes, right? You're talking about copyright protection for artists, actual lawsuits that are going on right now. Government should not waste its time thinking about this stuff. If you waste your time regulating us on these small topics, you risk China getting its hand on this super powerful machines. And you don't want that because at least us, trust us, we can build the machine God that can help us. And we're telling you we want to be regulated, but you don't want China to be the one.

7:50That's the first one. The second one, second issue is you can always abdicate responsibility. We're going to take a quick break. We'll be right back.

8:05AI can make businesses faster. It can also make mistakes, expose data, and execute the wrong instructions faster. ThreatLocker helps organizations set boundaries AI cannot ignore by controlling what applications can run, what they can reach, and how they interact with sensitive resources. Innovation needs more than good intentions. it needs limits. Control AI with zero trust at threatlocker.com slash wired. That's t-h-r-e-a-t-l-o-c-k-e-r dot com slash wired.

8:41In our prior conversations, we've talked about the usage of certain language or phrases to describe this modern era of AI and how you take a shoot with some of them. For example, in our previous conversation, you said something about autonomous weapons. After that was published, you told me you later regretted saying that because you would have phrased it differently. We've talked about PDOOM. We've talked about how when you say that agents have gone rogue, such as OpenAI, you're essentially removing culpability from the engineers who built those tools. I can't help but wonder if maybe there are groups that are, we've even talked about the word alignment.

9:13So I was just going to say, I wonder if there are groups that are actually more aligned than they realize around safety and transparency, but who are deploying different phrases to describe what is essentially the same thing. Some of the reasons that I regretted saying these words is not that the term autonomous weapons has, I have any issue with that. It's a legal term that has, even landmines are considered autonomous weapons legally. It's just that the people in the existential risk camp have co-opted it to mean something different. When I said the term, I knew I'm like, huh, I wonder if this is going to be taken in that way.

9:53And it was because the people talking about existential risks of AI took it to mean what they're saying, right? Which is like some kind of superhuman, super intelligent machine doing stuff on its own or whatever. Just to bring it back to the latest conversation about this, I was at the UN General Assembly this week, I was a part of some side panels, so I wasn't actually in the UN, GA, nor the Security Council. But there were some prominent tech folks there, including Sam Altman from OpenAI and Dario Amadai from Anthropic. Dario reportedly said, if managed poorly, I even believe AI could be a risk to humanity as a whole.

10:30His competitor, Sam Altman, said we could lose control of the future to AI. So explain how it is that these folks who once again have these, you know, companies that are valued at nearly a trillion dollars or more, supposedly planning to go public very soon. Explain again how these companies that have commercial interests in selling AI are now here on the world stage at the U.N. saying, I don't know, there could be there could be some risks here. right? And how, you know, is this part of regulatory capture? Are they going to be looking for ways to self-govern before there's a regulatory crackdown on this?

11:14Absolutely. It's part of regulatory capture. And in the same paper where I introduced a test reel bundle, I talk exactly about this regulatory capture and especially what has happened at the UN. Briefly after, you know, Sam Bankman-Fried, as you know, was an effective altruist. And And briefly after he went to prison, the effective altruists were briefly scrutinized by journalists. And journalists were talking about, there was a political article, for example, about the regulatory capture. In fact, let's talk about right now. What can be done? What kind of governance do you think is needed for AI?

11:45So, I mean, we've been talking about regulatory capture has been happening for a long time. 2023, same thing. Sam Alman said the same thing. We need the world cooperation, et cetera, et cetera. The EU AI Act regulated. and then he threatened to pull out of the EU. So you just have to see what they've actually done. When there's actual regulation that holds them liable, they threaten to pull out or they lobby super hard to water down that regulation. On the other hand, they're going around telling these multilateral bodies that there has to be world cooperation, et cetera, et cetera. And then again, what is this doing?

12:21It is A, selling themselves as organizations that are creating super intelligence, right? And so that's already marketing. And B, they're making everybody scared of anybody else who might be creating such things. So they don't want open-rate models from China. They don't want this competition. I am not following Chinese regulation that closely, but they're not talking about existential risk. They're talking about deep fakes and they're talking about like real things that need to be regulated. So the first one is marketing yourself as creating some super powerful, unprecedented things for which we don't have existing regulation, which is not true.

13:00We have existing regulation. But the second one is what they're doing is when someone tries to enforce the existing regulation, they either threaten to pull out or they lobby hard so that they don't have this existing regulation. And so the reason I keep on going back to 10 years ago, five years ago, three years ago, is that the same in my book, I say, the same movie on repeat with different heroes. So what is the solution for governance right now, if you had to propose it? Very, very simple things. First of all, former FTC Commissioner Lina Khan outlined five things. One is deceptive marketing practices.

13:38There's a law for deceptive marketing practices, and you can go after companies for that. Two is transparency, documentation. Before you put something out there, you should be able to tell us where all the data came from and actually document. This simple thing they don't do and they will never do. I'm going to tell you that they will fight tooth and nail to do the simple thing of documenting data. Labor exploitation of data workers. That's another one that they don't want to talk about. We are here in the world, in the clouds, talking about super intelligence where you have hundreds of millions of people around the world painstakingly labeling data, even pretending to be chatbots.

14:19There are data workers called AI impersonators, right? We now know that OpenAI hires people to look at your conversations with their chatbots. So people should be careful. Don't believe that, you know, you have privacy. Right. 404 Media reported that the new MetaMuse chatbot is actually a person on the other end who's responding. Things like this. Very simple. data transparency, labor exploitation, you should not be able to steal data from people. Even the first three things I talked about, data documentation, labor exploitation, if they had to abide by laws like that, and they were not allowed to steal data and not documented, the market calculation, right now the market calculation is not working.

15:04But with these additional measures, It just would not work whatsoever. Right. So you would automatically slow them down and have to make them accountable for something. Back in 2021, you co-authored a paper titled On the Dangers of Stochastic Parrots. Google had approved it initially. Then it was being reviewed. There were parts of it that were in dispute. You said, look, if you want me to remove my name from this, I'm not going to be a part of this. you ended up decamping from Google. That paper was later presented at the 2021 ACM Conference on Fairness, Accountability, and Transparency, and people still reference it.

15:42Can you briefly explain for someone who's never heard of stochastic parrots before what it has to do with the AI we're using today, what it means? Stochastic parrots is a metaphor to help people understand what large language models do. And large language models are trained on vast amounts of textual data on the internet trained to output the most likely sequences of text given their training data. They power most of the chatbots that we see today, whether it's cloud, whether it is ChatGPT. But when I wrote this paper, ChatGPT hadn't come out yet, but we saw the race to build larger and larger language models.

16:17And so that's the danger of building larger and larger language models that we were describing in this paper. That they would essentially parrot people? To parrot is to repeat back without understanding, right? So there was this whole, this whole existential risk narrative was happening back then too, if you can believe it. And so there was all this conversation about how OpenAI had claimed that GPT-2, the precursor to GPT-3, that powers chat GPT was too dangerous and too powerful to release. There was this conversation about whether GPTs can be ethical or they're creative and all this stuff. And so we really wanted to ground the conversation in the real issues.

16:58One of them, one of these issues is the environmental catastrophe, which a lot of people are now seeing, but we discussed it back then. And that was one of the main sections that Google people were unhappy with, the environmental costs. The other one is not documenting your data because you say you have too much data to document. The other one is deceiving people into believing that there is a mind behind the textual outputs, that they're interacting with. And so there, that's where we really wanted to explain that these systems are parroting the patterns of their training data. And it's very dangerous when you're outputting text like that because when you have plausible sounding text or very fluent text, there's so many different kinds of issues that can occur besides you believing that there's a mind behind a machine.

17:51In that paper, we gave an example of this Palestinian man writing good morning, which was translated to attack them. And because of that grammatical correctness and there were no cues that the translation could be wrong and people believed the translation. So the other issue of believing there is a mind behind the machine is what we call automation bias. You over-trust automated systems. And if you believe that this thing is an all-knowing machine, then you're going to over-trust the errors that you get, right? So we're seeing this with medical scribes where I was just reading an article, another article, where medical scribes that were summarized said this woman was microdosing on mushrooms since a woman has never heard, never discussed mushrooms, never done mushrooms.

18:42She saw it on the notes. So none of the doctors, nobody, nobody checked. Because again, if you believe, this is why I believe the super intelligence existential risk discussion is not just distracting, but dangerous. Because with automation bias, with over trusting these machines already, if you believe that they are nearly super intelligent instead of error prone, large things want us parroting things, then you're going to be less likely to check. You're going to be less likely to put checks and balances and regulation. And you're seeing things like misdiagnosis based on medical errors and things like that, which are serious.

19:20We're going to take a short break and then come right back.

19:27AI tools can write code, access files, connect to the Internet and execute commands at a speed no human can match. That productivity is powerful, but it also means one bad instruction or compromised tool can quickly become a security incident. ThreatLocker helps organizations use AI without losing control. Put zero-trust guardrails around tools like ClaudeCode and control exactly what they can execute, access, and connect to. Set clear boundaries around the applications, files, systems, and online destinations your AI tools are permitted to use. Because approving an AI tool should not mean giving it unrestricted access to your environment.

20:03Control what runs, control what it can do, control what it connects to. Learn how ThreatLocker can help you safely embrace AI at ThreatLocker.com slash Wired. That's T-H-R-E-A-T-L-O-C-K-E-R dot com slash Wired.

20:23It's funny, I must be too much of an elder millennial because you say that there's too much automation trust. And I'm like, I'm so distrustful. I've got to call the bank and it's a robot. I'm like, nope, nope. I don't want to talk to it. Yeah, exactly. So one of Anthropix co-founders, Jack Clark, recently posted something on X. I know you know what I'm going to say. Yeah, I do. He basically he put stochastic parrot in quotes and said he it was a mimically fit cognitive virus that spread from 2021 when your paper was out to 2025. It temporarily blinded many gifted people to the nature of AI progress, burned up crucial years of research, he says.

21:02He later says the use of this frame causes people to materially underestimate what AI systems can and can't do. When you saw Jack's tweet post on X, what was your initial response to that? I was not surprised, by the way, let me just tell you. Because the effective altruists have been, I was telling people this is a talking point they're telling lawmakers now. because every time I said something, they were like, oh, you should not take seriously someone who still takes the stochastic parrots thing seriously in 2026. So this has been a talking point of theirs for a while. The idea is that the research is outdated, right?

21:38Yeah. And it's not. And that's what I want to say. It's so ludicrous that we're even saying this because these are definitions of what large language models are. What large language models are has not changed, will never change. Large language models are large language models. Now, Now, you might have large language models in a separate system that's trained in a, you know, which they have now reinforcement learning agents. But our paper was about large language models, and that's never changed. And these chatbots still have large language models as a basis. And we are seeing it's so ridiculous that Jack Clark is talking about overestimating systems, because what I'm seeing is the examples that I just told you.

22:20It's over trusting these systems, not having checks and balances and ending up misdiagnosing someone's breast cancer to the wrong side. So there's a direct line between LLMs being stochastic parrots and misdiagnosing, giving a medical misdiagnosis using an AI tool. Because why? How does that actually, how does one lead to the other? because large language models are stochastic parrots. The stochastic parrots, they don't understand what's inside the text. So you cannot expect them to be factual. Even if you see something like the AI overview, I had another example where I was calling an oncologist friend of mine to ask about a specific medication and whether it was appropriate for a specific use because I read the academic paper saying that it was not and I wanted confirmation.

23:14and my oncologist friend was like, oh, look at the Google overview. It says that it's appropriate. But I read back, but it turned out not being. Why? I encounter that all the time with Google AI overviews. Because they are stochastic parrots trained to give you the most likely sequences of text based on their training data. So this is not grounded in world models. Emily M. Bender has been trying to say this in so many languages for a long time. She has another paper. Also one of the co-authors on the paper, correct? Co-first author with me to say that the research is outdated is ludicrous because right now, as we are hyping up these superintelligence and all that, there are news stories that are going unnoticed, which is about the complete opposite scenario that is actually happening in the real world.

24:01My understanding is that some of the biggest critiques people have had about focusing so much on research from that era may not be incorporating the thinking or the reasoning. There's no reasoning. Or even the recursive intelligence that is happening. Is there none? There is no thinking. How is there none? This is something, by the way, that at the Berkeley AI conference earlier this summer, I heard someone from Google talking about recursive intelligence. Of course they are. The New York Times just said a big story about how the scientists and researchers right now, that's what they're looking towards, recursive intelligence, the AI learning from other AI.

24:37Is that a reality? It sounds like you're saying that's not a reality. No, because, you know, there's one problem with AI researchers, which is aspirational naming and aspirational stuff. So machine learning, that's aspirational naming. But machine learning is real. But the naming is aspirational. The machine not necessarily. So this happens all the time, right? So just because there's curriculum learning in AI, it's a field. But let me give you a paper from my former manager, Sammy Benjo, who is a superstar in machine learning, not as famous as his older brother, Yoshua. And I asked him, aren't you tired of your whole life being like debunking the whole reasoning thing every single paper you write?