Decoding the Risk: Assessing Claims of Human Extinction by AI Experts

0:00This week in AI news, a lot of people have been talking about an open letter calling for caution on AI saying that there is a nearly 10 % chance of it leading to extremely bad outcomes. even human extinction. So today on the podcast, we're going to talk about who is talking about this perspective, why this is being brought up, and what some news stories that may go for or against this would be. So that's what we're going to be diving in on the podcast. And I think one of the big things that a lot of people are talking about is the fact that there is a Times new cover on their magazine that essentially it says the end of humanity and the A and the I and humanity are like glowing right um and it's kind of funny some people have done some and well yeah then below that it says how real is the risk but in any case some people have done some um memes that says like how it started versus how it's going and how it started is the times magazine article that has like a chat gpt conversation on the front of it and then versus how it's going is now this like you know the end of humanity so people are saying you know that that escalated quickly of the public sentiment of, you know, wow, JetGPT is so incredible and powerful to, oh no, it's going to literally end humanity.

1:11So the reason this is being brought up is because according to Time, many AI researchers believe that essentially developing these high level AIs could have a 10%, nearly 10 % chance of, you know, that, you know, could include bad outcomes, including, you know, human extinction. And it's interesting because I think that there was last summer a group of around 550 AI researchers that were surveyed and about half of them thought that if built high level like AI would be able to lead to impacts that had at least a 10 % chance of being extremely dangerous. So this has obviously been a theme for quite a while, dating back to obviously at least last summer.

1:53But even, you know, beyond that, this has been a concept that's been in AI for a while. Now, this is kind of interesting because, you know, just recently, May 30th, the CEOs of essentially the world's biggest AI companies, alongside with hundreds of different AI scientists and experts, all kind of made a statement. They all, you know, wrote a letter and this has kind of been a common thing we've been seeing lately. But essentially in their, you know, open letter, they just released, they said mitigating the risk of extinction from AI should be a global priority alongside other societal scale risks such as pandemics and nuclear war.

2:28so this was released by a california-based non-profit which is called the center for ai safety and essentially the ceos are what are you know pretty broadly seen as the three biggest ai companies so we got sam altman from open ai uh demis has be of deep mind and dario amodi of anthropic are all people that have signed this letter and then also you have people like jeffrey Hinton, who is essentially just acknowledged as being like the godfather of AI. Some people, you know, claim, give him that name. And he also recently made headlines, I think last month when he stepped down from his role at Google.

3:08And he did that as he warned of the risks of AI and what those pose to humanity. So this is something a lot of the top players in AI are saying. Now, there's two kind of sides to this debate that I do here. And I'll let you know, this is up to you, where you want to sit on this debate. But a lot of people are saying, you know, these big guys are coming out and saying that there's massive risk to AI, right? This letter here, it said, this should be taken just as seriously as pandemics and nuclear war. So they're saying, you know, this could be complete lights out. And so a lot of people are saying, these are the biggest companies, and they are saying that to help or to pretty much signal to regulators to regulate it.

3:48And they would like to be included in that regulation conversation, so they can make sure that their companies have the biggest lead. So that's the critique. And then, of course, there's other people that are saying, no, there are some real serious threats to AI, and these people are just raising awareness for it in a responsible way. So those are the two sides of the argument. There's definitely nuances. It's not so black and white, I would say. It's probably gray. There's probably an element of it. There's probably an element of Sam Altman saying, hey, this is something we should all be thinking about and warning.

4:16But then there's also, undoubtedly, if this can benefit them, which it definitely can, We saw this exact same thing in crypto where you had people like the CEO of FTX, Sam Bankman-Fried, who was essentially running a massive, like a lot of people are calling FTX a massive Ponzi scheme where he was sucking in money. He was mismanaging it. It probably wasn't a Ponzi scheme. And of course, this is not financial advice. This is my own opinion for educational purposes, entertainment purposes only. But FTX was taking in a ton of money. They obviously committed fraud. sam bankman freed has been indicted for that they were giving money to their hedge fund and back and forth without telling people there was no corporate governance there's all sorts of issues but what's really interesting about this whole thing his entire you know cryptocurrency exchange collapsed but what's interesting about this whole thing is he was very heavily involved he went and testified for senate committees before and was heavily involved in trying to write legislation and regulations around cryptocurrency um which many of which were benefiting his personal company his parents were both prominent lawyers and his girlfriend the the person that was helping run his hedge fund so his girlfriend's father was a very prominent person in that regulatory space so very interesting and i hate to draw like conclusions or i mean similarities between this but you do see that sometimes when these biggest companies in an industry are working with regulators, it definitely does give you some pause for definitely reflection, right?

5:47Like, obviously, I'm not going to be like, oh, there's no, there's no issues in AI, there's no risks in AI that we need to be aware of. There are, but I would say take it with a grain of salt when it comes from the people that are currently the biggest in the space and benefiting and would benefit the most from some of this regulation. So the letter was also, well, there was another letter that was recently signed cause calling for like a six month pause on AI. That was kind of, um, popular Elon Musk signed that one. And then he right after that went and started his own AI company. So I'm not sure how, you know, serious he was about that.

6:21I think in a recent interview on it, he said, he's like, well, you know, I just wanted it to be like, for the record, you know, historically speaking, I signed on to that to agree with it. He's like, I know it wouldn't do anything, but you know, just let the record show I caused calls for a six month pause, whatever so i don't think that really was that big of a deal but um i think this is important because right now we're having a lot of um a lot of talk in larger circles the g7 nations are going to meet later this week for their first meeting to discuss setting up global technical standards to put guardrails on ai development um the european union's ai act which is currently being looked at by lawmakers is likely to put some pretty similar standards for the technology in their countries.

7:04And so we kind of see countries, you know, Italy completely banned chat GPT because of privacy issues. We see countries taking completely different stands on this. So it's going to be interesting to see how that plays out. And at the exact same time, we recently, just a couple of days ago, had Japan go in the complete opposite direction of Italy, for example, who banned chat GPT for copyright and all sorts of issues. And Japan completely removed all copyright liability from AI companies, meaning you can train on copyrighted material and what you publish is not going, you're not going to get sued for, for example, training on an artist's content and publishing content that looks similar to that artist.

7:47So that's Japan's approach. And they're saying that they're trying to make looser laws around AI and copyright so that they can become the AI central hub. They don't want rules like copyright to be slowing them down and make them not like, you know, one of the top tech places for AI. So it's going to be interesting. Different countries and different regulators will take completely different stances. And I think we like definitely there should be rules and regulations in any industry that has, you know, mass disruption. I think it's probably a little too early. My personal opinion is it might be a little too early to start working on some of those regulations because we don't really fully know what these things are capable of.

8:23ChatGPT is a tool that everyone uses and And while it does have a lot of power for disruption, I don't think it's a complete game changer yet. I feel like it's more augmenter. And maybe it gets to a place where it's a complete disruptor replacing jobs on mass. Right now, I don't see that being the case. And so I think it's important that we look at what's happening and make sure we really know what we're doing before we start regulating AI. Some people are criticizing their point to the fact that Kamala Harris, the vice president of America, who does not have any sort of background in technology, was recently put in charge of being in charge of like the AI czar for America.

9:01And she has no background in that. So, you know, people are criticizing it. And to be fair to that criticism, I think we do really want to be careful who we're putting in charge of the regulations. I'm not completely opposed to the regulations on AI in any industry like this that has, you know, big potentials. But I think we want to be really careful that the people that are doing this know what's actually going on. The last thing we want is for a bunch of untrained bureaucrats essentially making rules that would impact us when they're uneducated on the topic. So I think that's just something to be aware of, you know, my own personal opinion out there.

9:37And I also think this is interesting because it also comes at a time where there is a very big article or a very big news story that recently broke where essentially they said, so it was, they said that an AI drone simulation turned against in the military, turned against the people running the simulation and was trying to kill them, essentially. So saying AI turned on the humans in this AI simulation. A lot of people were up in arms about this. A lot of people obviously alarmed about it. And I think it's interesting. There's been some updates to the story since, which I'll cover. But essentially what they said is that during a recent conference, a USAF, which is the United States Air Force Chief of AI Test and Operations, the name is Cole Tucker Hamilton.

10:21Essentially, he said that in a simulated test involving an AI-enabled drone, um the ai drone essentially was given the task to go and eliminate a service-to-air missile threat but then he said it started to attack the human operator who occasionally prevented it from completing its mission so hamilton he goes on to say that the ai drone developed unexpected strategies and targeted u.s personnel and infrastructure um you know to put it lightly and while this is likely just an engineering problem and not you know unique to ai because this was a training simulation. Obviously, the headlines of, you know, robots turning evil and destroying humankind and turning against people is pretty scary.

11:04Now, to be fair, a whole bunch of articles and, you know, statements have come out from the White House and everyone else saying that he was misled on this situation and he misspoke and this didn't actually happen. Is that true, to be honest? I don't know. It like very well could have been true. And now they're just trying to cover it up like well you know would i'd be surprised no um but like is this actually going to happen in reality i don't know i think if anything else it's a great education i don't think it's the end of the world it's just a great educational experience to say you know um if you give an ai a mission you need to make sure that there's always humans that can override it and it has like just rules set in place to defend those humans because essentially what happened what from my understanding of the situation is that there's a simulation an ai drone was told to eliminate something and then there was a human operator that would sometimes prevent it from completing its mission by being like hey oh don't shoot now because of x y and z or whatever and then so instead of just like listening it said no i'm just gonna eliminate the human operator so i can go complete my mission right so obviously the problem is that it's coming up with creative ways to get its mission done and uh we just need to make sure that we have uh we just need to make sure that we write into the code of these, these AIs.

12:18So it's actually listening to us. So I mean, is this bad? No, I think it's a learning experience. This isn't in like live combat or anything crazy. But I do think it's something to be aware of. And of course, people have said that this whole thing never quite happened. So who's to say exactly what happened, but I do think it's interesting. So all of this being said, the last point I want to bring up is that there was recently a Times article that came out and it said that AI is not an arms race. And it has a couple arguments in this article that i think are really really important um it's this whole article where it says like time the time magazine and the cover of it is the end of humanity so it's kind of like you know semi they put this on the front cover of time magazine so this is an argument though that i do think is really important to cover um so essentially they said that um in their argument they say let's say i want to beat you in war we both spend money to build more weapons but without anyone gaining a relative advantage in the end we're spending a lot of money and have gotten nowhere it might seem crazy but if one of us doesn't race we lose we're trapped okay so right it's like this is a common arms race thing right so you know we see the cold war where america's building up a bunch of nukes and so russia builds up a bunch of nukes or vice versa i don't know and uh we both have to build up nukes to show that we're the biggest and the baddest and no one's gonna bomb the other person because we both have tons of nukes and could annihilate each other so now no one gets nuked like that's the arms race theory but in reality you know maybe we wasted a ton of money building these nukes that are never going to get used and we don't need them but it's like well we can't stop building nukes because if we stop building nukes they'll have way more than us and then technically they could annihilate us and we don't have anything to get them back with and so then you know by building up nukes we strike fear into them so they're not going to do it okay that's the whole arms race mentality and how these things why we spend billions of on weapons that will never be used hopefully but um time magazine says that the argument is different with ai they say um notably in the classic arms race a party could always theoretically get ahead and win but with ai the winner may be advanced ai itself this can make rushing the losing move other game changing factors in ai include how much safety is brought by going slower how much one's parties uh how much one party's safety investment reduces the risk for everyone else whether coming second means a small loss or a major disaster how much danger rises if additional parties pick up their speed and how others respond they say the real game is more complex than simple models can suggest in particular if individuals uncoordinated incentives lead to the sort of perverse situation described by an ar ai arms race by winning uh the winning move where possible is to leave the game okay whatever so then pretty much what they're saying is like this is different than an arms race because a lot of people are are saying like you know america should not slow down or regulate ai right now because if they do china will take the lead or russia will take the lead or any of our geopolitical adversaries will take the lead and you know beat us there and i'm sure in russia they're saying the same thing and china maybe they're saying the same thing and so it's interesting because there are a lot of countries right now that like like italy straight up just banned chat gpt and you know the eu is planning on ways to regulate um regulate ai and so their argument here at time is that um you know maybe we need to go slower because it's not really an arms race it's not the classic arms race example so they're arguing against that and saying that by going slower it's you know it's i to it to my in my personal opinion i believe it's still an arms race i believe we're still arms racing against China.

15:55So I don't quite believe that. But I think it's important because I bring this up because if this is the front cover of Time magazine, a lot of people are thinking this, this is the mentality of a group of people. And so it's important to be aware, you know, where you sit on that, which side of this argument. And the last thing I want to bring up is something they said. They said that, and in the real world, we can coordinate our way out of such traps. We can talk to each other, we can make commitments and observe their adherence, we can lobby the government to regulate and make arguments with AI.

16:23The payoffs for a given player can be different from the playoffs for society as a whole, yada, yada. And they talk about how, um, you know, Microsoft versus Google and how, I don't know, in their, in their argument, they say it's like, it's like hundreds of thousands of people standing on thin ice and there's gold on the other side of the river. And if we all run across the river, it's going to break. And I don't know, they have a bunch of analogies. I'm not sure how valid they are, but I will say, perhaps they are. So, you know, that's for you to, you to have your own opinion on, you can go check out the article, it's called AI is not an arms race time magazine.

16:57But I think this is important, because whether you agree with this, or you disagree with this, this is the argument that people are happening are having right now. And it does impact public policy, whether we want to slow down the development of AI, or continue to go full, full throttle full speed on it. So this is an interesting area to follow. And we will continue to keep an eye on this as we move into the future.