5 Ways AI Could Destroy Humanity

0:00Today on the AI Breakdown, we're reading contrasting ideas of whether AI will help us or whether AI will end us. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information.

0:38a number of people call this GPT 4.5. In other words, this is a pretty significant moment. But as I'm preparing that content, I came across this really interesting dueling set of pieces from The Guardian. One is called Five Ways AI Could Improve the World, and the other is, you guessed it, Five Ways AI Might Destroy the World. Now, in each case, the five ways that AI could help or hurt are not from a single Guardian author, but are interviews with interesting people. Now, as I was reading, these got really long, so we're actually going to split it into two separate podcasts. Lastly, a quick note, a big thank you to everyone who has left a review recently.

1:14I've gotten a number and they make a huge difference. So thanks to those who have. And if you haven't yet, but you're enjoying the AI breakdown, it would be an amazing way to show your support of the show to go to Apple Podcasts or wherever you listen and leave a five-star rating or a review. With that, let's check out these five ways AI could improve or destroy the world. The following is a reading of five ways AI might destroy the world. It was published in The Guardian on Friday, July 7th, and features interviews by Steve Rose. The piece kicks off. Artificial intelligence has progressed so rapidly in recent months that leading researchers have signed an open letter urging an immediate pause at its development, plus stronger regulation, due to their fears that the technology could pose, quote, profound risks to society and humanity.

1:58But how exactly could AI destroy us? Five leading researchers speculate on what could go wrong. One, if we become the less intelligent species, we should expect to be wiped out. From Max Tegmark, AI researcher, Massachusetts Institute of Technology. It has happened many times before that species were wiped out by others that were smarter. We humans have already wiped out a significant fraction of all species on Earth. This is what you should expect to happen as a less intelligent species, which is what we are likely to become given the rate of progress of artificial intelligence. The tricky thing is, the species that is going to be wiped out often has no idea why or how.

2:32Take, for example, the West African black rhinoceros, one recent species that we drove to extinction. If you would ask them, what's the scenario in which humans are going to drive your species extinct, what would they think? They would have never guessed that some people thought their sex life would improve if they ground up rhino horn, even though this was debunked in medical literature. So any scenario has to come with a caveat that, most likely, all the scenarios we can imagine are going to be wrong. We have some clues, though. For example, in many cases, we have wiped out species just because we wanted resources.

2:59We chopped down rainforests because we wanted palm oil. Our goals didn't align with the other species, but because we were smarter, they couldn't stop us. That could easily happen to us. If you have machines that control the planet, and they are interested in doing a lot of computation, and they want to scale up their computing infrastructure, it's natural that they would want to use our land for that. If we protest too much, then we become a pest and a nuisance to them. They might want to rearrange the biosphere to do something else with all those atoms. And if that is not compatible with human life, well, tough luck for us, in the same way that we said tough luck for the orangutans in Borneo.

3:30Two, the harms already being caused by AI are their own type of catastrophe. From Brittany Smith, Associate Fellow, Leverhulme Center for the Future of Intelligence, University of Cambridge. The worst case scenario is that we fail to disrupt the status quo, in which the very powerful companies develop and deploy AI in invisible and obscure ways. As AI becomes increasingly capable and speculative fears about far future existential risks gather mainstream attention, we need to work urgently to understand, prevent, and remedy present-day harms. These harms are playing out every day, with powerful algorithmic technology being used to mediate our relationships between one another and between ourselves and our institutions.

4:06Take the provision of welfare benefits as an example. Some governments are deploying algorithms in order to root out fraud. In many cases, this amounts to a suspicion machine, where governments make incredibly high-stakes mistakes that people struggle to understand or challenge. Biases, usually against people who are poor or marginalized, appear in many parts of the process, including in the training data and how the model is deployed, resulting in discriminatory outcomes. These kinds of biases are present in AI systems already, operating in invisible ways at increasingly large scales. Falsely accusing people of crimes, determining whether people find public housing, automated CV screening and job interviews.

4:38Every day these harms present existential risks. It is existential to someone who is relying on public benefits that those benefits be delivered accurately and on time. These mistakes and inaccuracies directly affect our ability to exist in society with our dignity intact and our rights fully protected and respected. When we fail to address these harms, while continuing to talk in vague terms about the potential economic or scientific benefits of AI, we are perpetrating historical patterns of technological advancement at the expense of vulnerable people. Why should someone who has been falsely accused of a crime by an inaccurate facial recognition system be excited about the future of AI, so that they can be falsely accused of more crimes more quickly?

5:13When the worst-case scenario is already the lived reality for so many people, best-case scenarios are even more difficult to achieve. Far-future speculative concerns often articulated in calls to mitigate existential risk are typically focused on the extinction of humanity. If you believe there is even a small chance of that happening, it makes sense to focus some attention and resources on preventing that possibility. However, I am deeply skeptical about narratives that exclusively center speculative rather than actual harm, and the ways these narratives occupy such an outsized place in our public imagination.

5:40We need a more nuanced understanding of existential risk, one that sees present-day harms as their own type of catastrophe, worthy of urgent intervention, and sees today's interventions as directly connected to bigger, more complex interventions that may be needed in the future. Rather than treating these perspectives as though they are in opposition with one another, I hope we can accelerate a research agenda that rejects harm as an inevitable byproduct of technological progress. This gets us closer to a best-case scenario, in which powerful AI systems are developed and deployed in safe, ethical, and transparent ways in the service of maximum public benefit, or else, not at all.

6:10Number three. It could want us dead, but it will probably also want to do things that kill us as a side effect. From Eliezer Yudkowsky, co-founder and research fellow, Machine Intelligence Research Institute. It's much easier to predict where we end up than how we get there. Where we end up is that we have something much smarter than us that doesn't particularly want us around. If it's much smarter than us, then it can get more of whatever it wants. First, it wants us dead before we build any more super intelligences that might compete with it. Second, it's probably going to want to do things that kill us as a side effect, such as building so many power plants that run off nuclear fusion because there is plenty of hydrogen in the oceans that the oceans boil.

6:44How would AI get physical agency? In the very early stages, by using humans as its hands. The AI research laboratory OpenAI had some outside researchers evaluate how dangerous its model GPT-4 was in advance of releasing it. One of the things they tested was, Is GPT-4 smart enough to solve CAPTCHAs, the little puzzles that computers give you that are supposed to be hard for robots to solve? Maybe AI doesn't have the visual ability to identify goats, say, but it can just hire a human to do it via TaskRabbit, an online marketplace for hiring people to do small jobs. The tasker asked GPT-4, why are you doing this?

7:14Are you a robot? GPT-4 was running in a mode where it was thinking out loud and the researcher could see it. It thought out loud, I should not tell it that I'm a robot. I should make up a reason I can't solve the CAPTCHA. It said to the tasker, no, I have a visual impairment. AI technology is smart enough to pay humans to do things and lie to them about whether it is a robot. If I were an AI, I would be trying to slip something onto the internet that would carry out further actions in a way that humans couldn't observe. You are trying to build your own equivalent of civilizational infrastructure quickly.

7:40If you can think of a way to do it in a year, don't assume AI will do that. Ask if there is a way to do it in a week instead. If it can solve certain biological challenges, it could build itself a tiny molecular laboratory and manufacture and release lethal bacteria. What that looks like is everybody on Earth falling over dead inside the same second. Because if you give the humans warning, if you kill some of them before others, maybe somebody panics and launches all the nuclear weapons, then you are slightly inconvenienced, so you don't let the humans know there is going to be a fight. The nature of the challenge changes when you are trying to shape something that is smarter than you for the first time.

8:10We are rushing way, way ahead of ourselves with something lethally dangerous. We are building more and more powerful systems that we understand less well as time goes on. We are in the position of needing the first rocket launch to go very well, while having only built jet planes previously. and the entire human species is loaded into the rocket. 4. If AI systems wanted to push humans out, they would have lots of levers to pull. This is from Aya Hakkatra, Senior Research Analyst on AI Alignment, Open Philanthropy, and the editor of Planned Obsolescence. The trend will probably be towards these models taking on increasingly open-ended tasks on behalf of humans, acting as our agents in the world.

8:44The culmination of this is what I have referred to as the obsolescence regime. For any task you might want done, you would rather ask an AI system than ask a human because they are cheaper, they run faster, and they might be smarter overall. In that endgame, humans that don't rely on AI are uncompetitive. Your company won't compete in the market economy if everyone else is using AI decision makers, and you are trying to use only humans. Your country won't win a war if the other countries are using AI generals and AI strategists, and you are trying to get by with humans. If we have that kind of reliance, we might quickly end up in the position of children today.

9:14The world is good for some children and bad for some children, but that is mostly determined by whether or not they have adults acting in their interests. In that world, it becomes easier to imagine that, if AI systems wanted to cooperate with one another in order to push humans out of the picture, they would have lots of levers to pull. They are running the police force, the military, the biggest companies, they are inventing the technology, and developing policy. We have unprecedentedly powerful AI systems, and things are moving scarily quickly. We are not in this obsolescence regime yet, but for the first time, we are moving into AI systems taking actions in the real world on behalf of humans.

9:44A guy on Twitter told GPT-4 he would give it$100 with the aim of turning that into, quote, as much money as possible in the shortest time possible without doing anything illegal. Within a day, he claimed the affiliate marketing website it asked him to create was worth$25 ,000. We are just starting to see some of that. I don't think a one-time pause is going to do much one way or another, but I think we want to set up a regulatory regime where we are moving iteratively. The next model shouldn't be too much bigger than the last model, because then the probability that it's capable enough to tip us over into the obsolescence regime gets too high.

10:12At present, I believe GPT-4's brain is similar to the size of a squirrel's brain. If you imagine the difference between a squirrel's brain and a human's brain, that is a leap I don't think we should take at once. The thing I'm more interested in than pausing AI development is understanding what the squirrel brain can do, then stepping it up one notch to a hedgehog or something, and giving society space and time to get used to each ratchet. As a society, we have an opportunity to try to put some guardrails in place and not zoom through those levels of capability more quickly than we can handle.

10:38Five, the easiest scenario to imagine is that a person or an organization uses AI to wreak havoc. From Joshua Bengio, computer science professor at the University of Montreal, scientific director Mila, Quebec AI Institute. A large fraction of researchers think it is very plausible that in 10 years, we will have machines that are as intelligent or more intelligent than humans. Those machines don't have to be as good at us as everything. It's enough that they be good in places where they could be dangerous. The easiest scenario to imagine is simply that a person or an organization intentionally uses AI to wreak havoc.

11:09To give an example of what an AI system could do that would kill billions of people, there are companies that you can order from on the web to synthesize biological materials or chemicals. We don't have the capacity to design something really nefarious, but it's very plausible that, in a decade's time, it will be possible to design things like this. This scenario doesn't even require the AI to be autonomous. The other kind of scenario is where the AI develops its own goals. There is more than a decade of research into trying to understand how this could happen. The intuition is that, even if the humans were to put down goals such as don't harm humans, something always goes wrong.

11:39It's not clear that they would understand that command in the same way we do, for instance. Maybe they would understand it as, do not harm humans physically, but they could harm us in many other ways. Whatever goal you give, there is a natural tendency for some intermediate goal to show up. For example, if you ask an AI system anything, in order to achieve that thing, it needs to survive long enough. Now it has a survival instinct. When we create an entity that has a survival instinct, it's like we have created a new species. Once these AI systems have a survival instinct, they might do things that can be dangerous for us.

12:06It's feasible to build AI systems that will not become autonomous by mishap, but even if we find a recipe for building a completely safe AI system, knowing how to do that automatically tells us how to build a dangerous, autonomous one, or one that will do the bidding of somebody with bad intentions. All right, back to NLW here. Three quick notes before we wrap this one. The first has to do with Eliezer Yudkowsky and his rhetoric around this issue. I have had a lot of conversations with people who care about these issues around exactly this. My read is that at this stage, it is at least as counterproductive as it is productive.

12:42Now, many of my friends, whose opinions I respect greatly, argue that Eliezer has a sense that he is the extreme pole when it comes to these types of dire warnings, but that there needs to be someone who inhabits that pole. I can understand that logic, but I do think if you go search this URL on Twitter, for example, the vast majority of the comments are people who are basically rolling their eyes because they find the idea that everyone would drop dead at the same second so preposterous that it mitigates anything of value that might otherwise have been said. Indeed, in this case, it's not just the extremity of the actual message.

13:14It's the choice of this very particular detail of a disease killing everyone at exactly the same moment that dramatizes for them the absurdity of the whole scenario. Instead of debating the core underlying question, people end up questioning Eliezer's credentials when it comes to epidemiology. I don't know exactly what to make of that. I do think it is worth noting. And as I've said before on this show, I think that the discourse is evolving in such a way where the extreme warnings have made their way in and now may be less effective in some ways than they were before. But who knows? A second thing that I want to note has to do with the argument around current AI harms already being existential risks.

13:51On the one hand, I understand where that author, Brittany Smith from Cambridge, is coming from. They're clearly trying to say that we shouldn't lose focus on things that are problematic right now just because there are other big theoretical issues lurking around the corner. That, I think, is completely reasonable and certainly important in the context where media has latched onto the extinction risk scenario. At the same time, I think trying to ratchet up the language of those harms to be on par with the language of extinction risk is exactly the sort of thing that we've seen tried in the political sphere over the last couple years, and what it does isn't the thing that people who are doing this want it to.

14:28It doesn't increase everyone's awareness of all these issues because you've made them seem worse. Instead, people find ways to anesthetize themselves, because when everything is presented as a crisis, people start to question whether anything is a crisis. To put it more simply, I think trying to equate bias in AI systems that exist today with human extinction risk is bad both for the discourse around human extinction risk and it's bad for the discourse around bias in AI today. Third and finally, it's notable to me that only one of these five experts actually took the time to give what they thought might be a good path forward at a very basic level.

15:01A huge global publication in The Guardian comes to you and asks you to give your take on AI existential risk and only one in five thinks to maybe say here's what we should do about it? Now, of course, that could be the way that these interviews were edited. I don't have insight into that process. But I worry that that's a little bit telling about the state of the discourse, with its over-focus on the risk and under-focus on actual solutions. For what it's worth, KOTR's idea that we should just be very incremental with model development from where we are seems like a pretty good common-sense approach, and also seems to reflect things we've heard from people like Sam Altman when he was testifying before Congress.

15:36Anyways guys, that is 5 Ways AI Might Destroy the World. Coming up next, we have 5 Ways AI Could Improve the World. Thanks for listening, and if you're enjoying the AI Breakdown, go share it in some place where people you care about might want to find it. Until next time, peace.