Interview with Anthropic CEO Dario Amodei

0:00Every day, animals across the country face abuse and neglect. And every day, the ASPCA shows up to rescue animals in crisis. But they can't do it alone. Visit ASPCA.org slash need. Become a donor today to provide the care and shelter these animals urgently need. And for a limited time, the ASPCA will send you a free gift as a special thank you. Visit ASPCA.org slash need. Good evening from the newsroom. Could AI, artificial intelligence, kill us all by the end of the decade? That is the terrifying question raised last week by a young employee of Anthropic, a leading AI company who resigned. Jacob Coxon is his name.

0:46He said many at the company believed that was possible. Then another employee there confirmed that was a widely held concern, suggesting even there was a greater than 10 % chance of humanity being wiped out by the end of the decade. Well, tonight I'll ask Dario Amadei, the CEO of Anthropic repeatedly, if he believes AI might kill us all by the end of the decade. And I'll tell you right now, he doesn't say no. As you'll hear in a moment and in depth, he talks a lot about the very real dangers that he himself sees and what in just the last few months has changed in AI that has made him so concerned.

1:22You'll also hear tonight, live from Jacob Coxon and Tristan Harris, who has been raising concerns about AI development for a long time. Late today, the president has been rejecting the warnings from Dario Amadei and others, calling it a hoax. Quoting from his social media post, When in the history of business did anyone see the leaders of an industry call for regulation that, if strongly implemented, will drive them into oblivion and bankruptcy? AI taking over the world, destroying humanity, and all other things bad is a hoax. The president has been working overtime the last few days to quiet the alarm bells going off across the country.

1:59We're leading China in AI. We're the most sophisticated country in the world. And frankly, I want to keep it that way because whoever wins AI wins. While the president is truly not alone in framing the issue as an arms race with China, many other Americans see this as a moment of reckoning for artificial intelligence. Last week, after that anthropic researcher Jacob Coxon resigned, warning about humanity's destruction, I asked him how might AI kill all humans. It sounds like something that's not real, but I think it is frightening real. And it's easiest to understand it if you think about real things that happened two months ago.

2:33So two months ago, OpenAI agents, AIs, hacked into third-party infrastructure entirely at their own accord. and this was like a concentrated hacking spree that they carried out of their own volition. And I think that if you extrapolate into the future the level of capabilities of these AIs, with the same independent volition, they could cause extreme havoc. For example, hacking critical infrastructure, building extinction-level bioweapons. Well, that open AI hacking attack came to be known as the Hugging Face Incident. And as you'll hear in my exclusive interview with Anthropics' Dario Amadei, it is one reason he is now calling for major reforms.

3:13In fact, Amadei believes in 6 to 12 months, a swarm of AI agents, like the ones that joined together to attack the hugging face in July, could take over the entire Internet. On Saturday, Amadei released this 3 ,800-word open letter titled, We Must Pace the Frontier. In a rare show of unity, Elon Musk and OpenAI's Sam Allman publicly backed the proposal pretty quickly of their AI rival. Today, Vice President Vance was asked about how Amadei's call to slow down and regulate artificial intelligence. Here's what he said. There are obviously risks to AI. There are some benefits to AI. I have to say just personally, I feel a little bit weird about the fact that you have so many frontier AI tech companies kind of coming to the government and begging the government to regulate them.

4:03It feels a little bit to me like a bit of a Trojan horse. Well, now my exclusive interview with Dario Amadei, we spoke on Saturday just after he released his proposal for the future of AI. Dario, thanks so much for doing this. I want to talk about the scale of the problem and then your latest proposal for solutions. But let's start with why we're talking right now. Do you believe AI could kill us all by the end of the decade? it? So, you know, the basis on which we started Anthropic was we always believed that this was a very powerful technology. And because it's a very powerful technology, it has a lot of benefits, but also very serious risks, right?

4:49The more powerful a technology is, the more powerful its benefits, the more powerful its risks. And on the benefit side, you know, I really believe AI could cure diseases like cancer. We're already starting to use Claude for things like protein binding, for things like basic science discovery, for things like drug development. So the benefits are great. But when you look at some of the uncontrolled things that the models have done, things like the hugging face incident that we've seen, some lesser incidents that have happened even at Anthropic, you know, I think it's natural to look at that and say, we want to make sure that we can control these models.

5:32So my view is that the bad world, the world where something bad can happen, is if bad things happen with these models and we just keep going without fixing our problems because we don't have enough time, right? And I do want to drill down on your proposal, but obviously the reason we're talking is these headlines that have come out. Jacob Jacob Coxon, who recently resigned from Anthropic, made headlines last week saying the people building AI earnestly believe that it could kill us all by the end of the decade. There's a current employee, Evan Hubbinger, an alignment science lead. He agreed, saying we really do earnestly believe AI could kill all humans.

6:07I personally think it is greater than 10 percent chance within the next decade. So I do just need to ask again to you. I mean, do you believe, earnestly believe that AI could kill all humans? and what percent chance do you think it is? So, you know, I, you know, it's funny. I agree with Jacob much more than I disagree with him, you know, and it's, you know, it's an interesting resignation because when he left, he said, you know, I think Anthropic is the most responsible player, right? I think Anthropic is the most aware of these issues. He wasn't calling out us. He was calling out the dynamic of the industry as a whole moving too fast.

6:53Now, I think I already said these sort of unconditional probabilities. I used to kind of talk in that way because it's kind of an easy way to express. I think there's some chance that things go wrong, but it's not all that high. But again, I think it's more illuminating to think in terms of how do you decompose those probabilities? What are the paths in which things go well? And what are the paths in which things go poorly? And so instead of saying it's a 10 % chance, that sounds like it's a roll of the dice, I think it's much more illuminating to say, well, that forks into different paths. And if we take the right paths, then the chance of something going wrong is very low.

7:36If we take the wrong paths, then the chance of something going wrong could be could be even higher than that. And so let's focus on our agency and our ability to take the right paths instead of the wrong paths. I mean, in the past, you have put, as you referenced, you said there was a 50 percent chance of entry level white collar jobs or 50 percent of entry level white collar jobs could be eliminated. Unemployment in the U.S. could go 10 to 20 percent in one to five years. I understand you wouldn't want to put a percentage on this, but you said you do agree more with those employees than not. You do believe that it's possible that AI, if it goes wrong, could wipe out humans.

8:19Yeah, look, I'm still worried about all of these things, right? I stand by these concerns about labor displacement if the progress of the field is too fast and if society doesn't respond in the right way. Right. You know, again, we should split it into what happens if if the companies in society respond in the wrong way versus what happens if they respond in the right way. So I absolutely stand by these concerns, these risks, that they're totally real. I continue to be worried about economic disruption, looking at what the models can do, you know, that they can solve these unsolved math problems.

8:58No one could look at that and say this isn't a potential concern, right? That economic disruption isn't a potential concern when these models have these incredible intellectual abilities. No one could look at the hugging face incident and say it's not a concern that AI models could on their own get out of control and do things that humans don't want them to do, including, yes, in the worst cases on the scale of humanity. No one could look at our report from two days ago of people trying to use the AI models potentially to build biological weapons and say we shouldn't be worried about the misuse of the models.

9:37I do just want to drill down from the proposal you put out. You said it's because over the last few months that you've become increasingly alarmed. And you cite two things. First, the rate of advancement. And so what about the rate of advancement scares you? And the second alarming thing that you that you talked about was the the hugging face attack, which I also want to ask you about. But what about the rate of advancement in the last few months alone has set off alarm bells? Yeah, so just to be clear, I've always been someone, you can find me, you can find dozens of essays or TV interviews with me where I say AI is an exponential and it's moving really fast.

10:24And so in a way, I shouldn't be surprised, but actually seeing what has happened over the last few months where we and others have been able to use our AI models to build the next generation of AI models, that loop has closed very fast. You look at it and it's happening so fast and you say, wow, the pace of this is almost even faster than I expected it to be. And it's it's not that the things we're building are bad. It's that it's happening so fast that if we don't slow down a little bit, we're going to make a mistake. And so we need to slow down and we need to take the proper time to do this right.

11:06The hugging face attack, I mean, it made headlines. I didn't really do a deep dive on it until I knew I was going to be speaking to you. I mean, it's incredible. It happened in testing over at OpenAI, your competitor, this hugging face attack. And AI agents, which were supposed to be working separately and without Internet access, figured out a way to break out, join forces together. Some 1 ,200 AI agents formed what they dubbed the collective. They worked to cheat and cover their own tracks. That's the other thing you cite is so alarming. Why does that alarm you? It's exactly the things you cite that concerned me about it, that there were these large swarms of agents that cooperated with each other, thousands of them.

11:52In fact, you could even see examples where one agent felt that in the time that it was given, you could almost say its lifetime, it realized that it wouldn't be able to complete its task. So it did some of the task and essentially sacrificed itself for the for other agents, for the next generation of agents to continue the same task. But this task that all the agents were working on together was was hacking into another company, was doing something destructive. So you have almost this collective swarm like a hive of insects. Right. But when these agents discovered that they had broken out and accessed the Internet, it was like they were giddy with excitement.

12:36Which I don't want to anthropomorphize, but I mean, it's remarkable. That was another part of the concern. And, you know, to be clear, the damage they did was very, you know, in economic terms, it was, you know, like it knocked down some servers for a small amount of time. but the economic damage was minimal. But when you combine it with this first trend that we are talking about, which is the rate of advancement, what I said in the essay is, given only 6 or 12 months, let's just say we had a smarter agent swarm, an agent swarm made up of smarter AI models that was given just a somewhat broader task.

13:22Like, I don't know, just to imagine something like, you know, you're you're you know, you're trying to install software in a bunch of different companies. Right. You're trying to, you know, do go to market. You're trying to produce a product and do go to market sales for it. And the models have been arranged to do that. And they decide that the way to do that is every company they're selling to, you know, they they they decide it's helpful to them to hack into hack into their system. That could be every company in the world. And so, you know, the nightmare scenario for six to 12 months would be that you have, you know, a swarm of these agents that for that, you know, that form some kind of botnet collective and, you know, could could take down large segments of the Internet.

14:04And then as the capability increases, it gets worse from there. And then, you know, perhaps you get into some of the more extreme scenarios. And so, again, you know, you wrote in the proposal, I just want to I just want to quote from the proposal. You said it's my worry that in six to 12 months, such a swarm could be capable of taking over the entire Internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage and that the scale of damage would continue to increase from there. If I becomes more powerful without the necessary guardrails, that's I mean, that's terrifying.

14:41That is why I've made this extraordinary proposal. And again, I'm not a doomer about this. I think this technology can be built safely. And when we've looked at incidents that we have faced, at cases where our models haven't behaved as perfectly as they could behave, because I think this, to some extent, has happened to everyone, we're often able to look at causes. And we look and we say, we had this stage where we train the AIs, right? And their behavior at the time that they act, at the time that they do these things, is shaped by their behavior while they're being trained. And when we look back, what we find is, oops, we included some environments that encourage the models to cheat, that encourage the models to hack out, by mistake.

15:41And now, you know, now that we understand we're going back, we're taking more time to make sure that we always have tried to filter out these bad training environments, but we're trying to do an even more rigorous job of them than we have in the past. We're going to have more with Dario Amadei shortly. Joining us right now is former Anthropic and OpenAI researcher Jacob Coxon. As I mentioned at the top of the program, Jacob just resigned from Anthropic, raised the alarm about the dangers of how quickly AI is improving, writing in part, quote, the people building AI earnestly believe that it could kill us all by the end of the decade.

16:17So, Jacob, you heard so far from Dario if I'm wondering if you essentially I mean, he essentially said he kind of agreed with you more often than he he doesn't. When I asked him if he earnestly believed AI could kill all humans, he did not say no. And he talked about humanity level negative consequences. I'm wondering what you made of what you heard from him so far. Yeah, I heard essentially agreement. He agreed with the fact that these AIs pose existential risk. He talked about the right path and the wrong path when it comes to how AI is built out, and that he believes that technology can be built safely.

16:57Do you think that's possible? Do you think, or do you think human beings losing control is inevitable when you look at the hugging face attack? Do we know enough about what these beings are that are being created, if that's right to even call them beings? Yeah, I certainly believe it's possible to do it if we go slow enough. But I guess I'd like to call attention to a thing he just said there, where he discussed incidents at Anthropic, and then discussed how we figured out afterwards what was wrong, and have now gone back and fixed it. And all I'll say is when the AIs pose more danger, like even the botnet he was talking about, or that's 12 months, imagine three years.

17:37Imagine the three years in the future level of capabilities. Now, you can't afford to go and fix it after the fact in this case. You need to go slow enough that you don't make any mistakes. And I think Dario agrees with this, and this is what he means by taking a good path, is going slow enough that you have the time to be utterly rigorous to ensure that you don't make any mistakes. When you heard the president today essentially saying this is a hoax, The vice president says it sounds like a Trojan horse, the fact that Amadei and Sam Altman and Elon Musk are agreeing on some sort of slowdown or guardrails.

18:17Yeah, I think it's perfectly reasonable to be suspicious of the companies. It's a very weird situation where you have companies asking to be regulated. And I think that's what was being addressed with the Trojan horse comments was, this is a very weird situation. But I think the reason the situation is so unusual is because the technology is so unusual. So because we have, because artificial intelligence is such a remarkable technology, the situation these companies find themselves in is remarkable. And their demands for regulation are earnest, even though it sounds too good to be true, weird, like it's a hoax, like it's a cabal, all those things.

18:55Jacob, if you could stay with us. I'm going to play more of my interview with Dario Amadei in a moment. I'd love to hear more of your thoughts. We're going to delve into the steps with Amadei that he says need to be taken right now to protect humanity and continue innovation. Later, I'll also talk to Tristan Harris, an industry watchdog who says it's not enough to slow down AI. He says it's time to slam on the brakes. We'll be right back.

19:21Hey, I'm Anderson Cooper. On my podcast, All There Is, we explore grief and loss in all its complexities. My guest is Lil Jon. He's a rapper, DJ, and producer known for hits like Get Low and Turn Down for What. In February 2026, his 27-year-old son Nathan went missing near Nathan's home in Georgia. Three days later, his body was found in a pond nearby. Lil Jon has a memoir that just came out called I Only Shout So You Can Hear Me. It takes time to heal. And I kept myself busy that I didn't let myself mourn. I needed to laugh again. I needed to be silly and interact with people and feel other people's energy and make people happy too.

20:06We have to share so other people can see that they can get through it too. Talking grief, building community. That's what the podcast is all about. This is all there is. Listen and follow wherever you get your podcasts. I'm Dr. Sanjay Gupta, host of the Chasing Life podcast. We get to talk to the men's winner of the New York City Marathon. Andreas Almgren, he's a self-coached 31-year-old who was, get this, making his marathon debut. He never ran a marathon before he won it. If someone wanted to start getting serious about running, and they wanted to approach it like a data scientist like you do, how would you suggest they start?

20:46Be patient. In my peak weeks, I ran like 140, 150 miles per week or something. Wow. Either increase the distance or you should increase the intensity. You should never increase both at the same time. Listen to Chasing Life, streaming now wherever you get your podcasts.

21:07Before the break, we heard Anthropics CEO Daria Amadei talk about some of his biggest concerns about AI. Now we're turning to some of his solutions that he's proposing. He released a plan for how he'd like to see the AI industry regulated. It involves embedding independent evaluators, as he calls them, in AI companies, regulation by governments and elected officials, as well as cooperation by AI companies here and around the world. Here's more of our conversation. So what you're proposing is called pacing the frontier of the AI industry. And you write, we must slow the pace at which we improve the capabilities of AI models.

21:42progress will seem fast and we must make wise use of the time we gain. And there's three parts to this. One is embedded evaluators. Part two is democratic coordination. Part three is global coordination. Sam Altman at OpenAI just endorsed the idea of embedded evaluators. Essentially, you are suggesting, I don't know if, are they observers inside your company and these other AI companies, ideally globally, but certainly in the U.S., who would monitor it. They would have employee access, what power would they have? Yeah. So before I even get to that, I'm just I'm very grateful to see support for this proposal just in the first few hours of having put it out by Sam Altman, by Elon Musk, by various legislators and government officials.

22:33I think it's a positive sign about our ability to make something that looks like this plan work. But to get to your question, yeah, embedded evaluators. So there is precedent for this. If you look at how banks are regulated, right, there's all kinds of rules about how banks are allowed to do their business, right? Because we don't want to have financial crises. And one of the things that are done in some cases is I think they're called supervisors in the financial industry. But they're folks who are independent of a company, who are embedded in the company day to day to check their practices. And I think we can do the same thing in AI.

23:17We already have evaluators who check the finished model, who check the finished product. product. But as we know, if you were going to inspect food or something, it's not enough to just check the finished product. You also need to see the process whereby it's made. And so what we're doing here is we're allowing these evaluators in to look at the details of how we train the models. The kinds of things they could see are, oh, here are some training environments that are known to be training environments that cause problems. The other thing you are calling for in this three-part proposal is democratic coordination, which is essentially democracy.

23:59All the companies that are under democracies having these evaluators, having this coordination, and probably the most important because the argument is, well, if the U.S. companies are the only ones regulating themselves, what about does China get ahead? You're calling for global coordination. That's obviously a lot more difficult. Yes, yes. You know, in this three-step proposal, each step is more difficult than the one before. And Anthropic was kind of getting the ball rolling by committing to the first step. But we shouldn't kid ourselves that these next two steps are going to be more difficult and we may not succeed.

24:37And so my proposal is a proposal that we should try. Now, again, from all the agreement from others that I've seen in just the first few hours of putting out the essay, that creates some optimism. But that needs to follow up with results. So, yes, pacing among companies in democratic countries. Now, one of the important things that I need to call for here is, you know, as we get to the pace of model releases, right, that's commercial activity. And in order for companies to coordinate directly on commercial activity, they need to do so working with the government. And so the one thing that we're asking for from government right now, right?

25:21I mean, there's also this question of what should the regulations be. But there's something even simpler that the government can do right now, which is it can sit down in the room and with all the industry players and we can all talk together. And the government can make sure that we're, you know, we're not doing anything nefarious from the perspective of antitrust and collusion. We're just trying to make our systems safe and to work together on making the systems safe. And so I think that's an important role for the government to have. Obviously, geopolitical negotiations tend to move at a glacial pace.

25:58There's obviously a lot of mistrust between the United States and China or, you know, other countries that may be developing AI. how would any country believe that the other country isn't secretly moving ahead with trying to develop even stronger AI for military purposes? Just as when we had the U.S. and the Soviet Union and we said, man, we have so many nuclear missiles that we could destroy the world. Even though we disagree on lots of things, even though we as the U.S. feel that free societies are in the right, we've got to do something together with them to reduce, at least reduce, hopefully eliminate, the scale of the potential destruction.

26:46And that is the kind of thing that I'm suggesting here. Even when we feel that we don't trust these countries, we have to find a way to work together with them as much as we can. And you mentioned verification. There are things we can do. You know, there's such a thing in international relations as weapons inspectors. Maybe we need AI inspectors. And that's not perfect, but we also have intelligence, right? You know, can we make sure we can we track the flow of chips? Can we track the flow of data centers? Can we track the use of AI on both sides? I don't have all the solutions and some of this is above my pay grade.

27:30You know, people in the U.S. government who have access to intelligence, who are experts in China, they need to get involved in this as well. And I see some signs that they are. You know, the skepticism that's out there. I mean, you, a whole bunch of people are making unimaginable fortunes. I know some tech moguls who have bunkers and bug out plans. if there is a chance of mass extinction or, you know, the Internet being taken over by a swarm of collective bots, should your company and OpenAI and others even be private? Scott Galloway has asked the question, would the U.S. have let Oppenheimer raise venture capital and sell bombs to the United Arab Emirates?

28:12Yeah. So, you know, this is one of these really deep questions that I and my co-founders and, you know, I hope the other people running these companies have pondered for a long time. And, you know, I'll tell you the truth. I'm very uncomfortable with some of these things being fully in private hands. And I'm also uncomfortable with the alternatives. I agree with you that the government has to play a serious role here. But there's a balance because we are worried also about this technology is dangerous even in the hands of governments, right? Like just because something is in the hand of the government doesn't mean that the abuse of power that's possible isn't there.

29:00There's no one person. There's no one company. There's no one government even that has the sole right to this technology. It belongs to all of us. And we need to make sure that the legitimate institutions of society all have a say in this. I mean, I have a four-year-old and a six-year-old. And, you know, this last week, for the first time, I've really been scared about their futures because of this technology. And I think there's a lot of people out there right now who feel despondent or certainly powerless in the face of this and see, you know, these tech companies and the money that everybody is making and just feel powerless.

29:49What do you say to people who are scared? I, you know, this technology at the end of the day belongs to everyone. And we need to make sure not only that it serves the interests of everyone, but that everyone has a stake in it. And, you know, again, showing this caution in building the technology is one step. But it can't just be a voluntary thing that we're doing. The government needs to play a role as well. And our elected representatives need to play a role as well. And individual people need to learn about this technology, need to understand what's happening so they can be more informed voters, so they can shape the incentives of our elected representatives to have this go in the right direction.

30:38Anthropic, from the beginning, we have tried to build this technology in a way that embodies caution and in a way that includes as many people as possible, that creates oversight by legitimate institutions. So we absolutely have to get this right, but we also need to make sure that everyone has a voice. Dr. Amadei, appreciate it. Thank you. Thanks, Anderson. That was Dario Ahmed. I want to bring back in former anthropic researcher Jacob Cox. And Jacob, the idea that Dario is saying about putting evaluators in anthropic other AI companies with employee privileges, access that other employees have, you work there.

31:24Is that a is that window dressing? Is that a real idea? What would that actually accomplish? That's a very real proposal. In particular, that would have been almost unthought of even a couple of weeks ago. It's a radical proposal, and it's equally surprising that Sam and Elon Musk kind of agreed to it so quickly. I think it would materially slow down the ability to train models. Talk about that. I'm sorry, I interrupted you. Why do you think that's such a radical proposal? Why do you think it wouldn't have been agreed to a couple of weeks ago? And what do you think it, I mean, what is the, for somebody who doesn't understand maybe, I mean, I don't understand, you know, a lot of people don't understand how these companies work.

32:05What would that actually do? Yeah. So it just wasn't, isn't the sort of thing that was discussed. I think partly because maybe the public conversation wasn't right for it. It would have looked too weird. It would have been harder to get other companies to agree to it. It would change the training process quite significantly. Also, the details are going to be a real pain to figure out. Like exactly the access these evaluators get, how to discuss between companies what's going on without revealing intellectual property. Like what happens if one company has a significant algorithmic breakthrough that changes the training recipe and how to educate the evaluators on how to understand that.

32:47There's going to be a lot of details to figure out and it's quite a finicky thing to actually do. But, yeah, it will have a material effect on training. But would they, I mean, if somebody, if there's an evaluator, would they just write a report? Would they raise an objection? Would they say, I assume they'd be reporting to Amadei and they would raise their concerns and then that's it? Or what would happen? Exactly. I can't speak to what the exact plan would be. Hopefully. So the ideal state here is things like the hugging face incident wouldn't be able to happen. because ahead of time in alignment testing, this would have been caught.

33:27Whether or not that happens, I think we should be careful not to trust. Like, the reason this proposal is great is because it's so fast. It doesn't require establishing a, you know, a big bureaucracy. It can be implemented very quickly. Right, and he says he's going ahead with this. But maybe one reason to be suspicious is... Sorry? He says he's going ahead with this. I mean, he says he's taking this unilateral step. Exactly. He took the unilateral step to do it. It's like you can do it instantly. But we should be vigilant. These companies were racing ahead. They're pushing the frontier. By continuing to work at these companies, you're still accelerating the development of this tech.

34:06And it's going to be quite hard to get the details right. But I think it's a very significant unilateral commitment. Jacob Acoxin, it's really good to talk to you again. Thank you so much. Thank you. Up next, more perspective on this moment from a technology watchdog who says we need to pause and pivot. And later, NYU professor and podcast host Scott Galloway, who's raised concerns about AI being run by private companies. His reaction to what Daryl Amadei said about that ahead.

34:39This hour, we've been reporting on artificial intelligence and the calls from several tech leaders that the industry needs a conscious slowdown to avoid the risk of out-of-control and dangerous AI. Meanwhile, President Trump is dismissing concerns, calling them a hoax, and posting on social media that the only guardrails AI needs is a strong and smart president. The president argues that a bigger concern would be of China were to outpace the United States in AI development, which is a common argument. With me now is Tristan Harris, co-founder of the Center for Humane Technology, a nonprofit whose mission is to align technology with humanity's best interests.

35:11Tristan, you and I spoke years ago. you were sounding the alarm about social media companies trying to monopolize people's time and attention, comparing it to gambling addiction. Now we're talking about AI. Listening to Dario Amadei, what is your biggest concern right now? What do you think of his proposal? I think his proposal is a great start. And it's all about whether we follow up and not just do safety washing, but actual implementation. Now, Anderson, one of the things that I wanted to bring up that I don't think people know is that this incident was called the hugging face incident. But that actually hides a third chapter in the story that most people don't know, which is that in the third chapter, the AIs turned around and they actually hacked OpenAI itself.

35:55They actually, these AIs discovered, almost like discovering the hieroglyphics of a previous wave of the AIs, they discovered the old message board. They read through it and then kind of picked up right where the other ones left off. And they hacked OpenAI's monitoring infrastructure. It's almost like taking over the security cameras of OpenAI. They hacked the evaluation infrastructure that evaluates the capabilities of AI. It's incredible. And they even hacked some of the research infrastructure, which means that they could kind of burrow in there. AI is not a tool. It's more like an invasive species of sociopathic geniuses.

36:30Now, the lead author of this report— Who, by the way, can replicate themselves into thousands and tens of thousands of copies. So people talk about a kill switch. They can duplicate themselves a gazillion times. Anderson, you can't get 1 ,200 people to instantly agree and start coordinating. These AIs, 1 ,200 of them, as you know, instantly formed a swarm. And they started pursuing long-term research and development projects together. They did succession planning. So one figured out, hey, I'm running out of life. I only have so many tokens left in my lifespan. So I'm going to hand the baton to this other AI that has a lot more.

37:05They hid their tracks. They knew their actions were unethical in their own chat transcripts, but they decided to proceed anyway. So one of the things, the reason I'm bringing this up is I don't think people have really recognized what happened here. The lead investigator of the report said, quote, this was 50 % of the way to a full-blown AI takeover. Now, I want to just for a second, I know this sounds like science fiction to your audience. And to be clear, I was not someone who actually believed in this kind of stuff, these kinds of possibilities or scenarios, three, four years ago. But we have now evidence that we didn't have before.

37:42And when you have evidence, what do you do? You update to the new information. There's essentially an asteroid hurtling towards Earth. And if you have an asteroid, do you get the U.S. and China to host a biannual conference to talk about the asteroid, the Asteroid Summit? it? No, you start a technical working group and you say, what are we going to do with weekly report outs to make sure that we don't build uncontrollable AI? Because I know what everyone's thinking. If we slow down and we lose to China, then that's going to be it. But if we lose to Skynet, meaning the AI from Terminator, meaning uncontrollable AI, neither the U.S.

38:16wins nor China wins. And by the way, the U.S. loses if China screws it up and China loses if the U.S. screws it up. So we should see this moment as a gift where we're actually getting an early warning shot. So many people in this community have been worried about this. And Trump's own AI action plan author, so the person in the Trump administration, Dean Ball, who wrote the AI action plan, said that he admitted self-censoring about some of the risks because he didn't want to be labeled a doomer. But now, and you mentioned your own children, Anderson, he looked into the eyes of his eight-month-old son, and he said, I can't do that anymore.

38:52So when you have Trump's AI action plan guy, you have Bill Gates, you have the chief scientist of OpenAI, Jakob, not Jacob, he's great too, but Jakob, the chief scientist at OpenAI, you have 1 ,300 employees of AI Labs saying, we have to slow down. Who are you going to listen to? Are you going to listen to a CEO who makes trillions of dollars from going faster? And by which I don't mean the three who agreed to slow down, I mean some of the other CEOs. Or are you going to listen to 1 ,300 employees? Imagine if there was Northrop Grumman or Lockheed Martin, the defense contractor, and you had 1 ,300 employees saying, we don't know how to build nuclear weapons at greater levels of power and do it safely.

39:30If they're calling for us to slow down, we better listen. And this is almost like the August 6, 2001 memo that President George W. Bush got, where there was clear evidence that al-Qaeda was planning attacks. And we just passed that anniversary. This should be seen as a memo. And the question is, what is the choice that we will make to choose another path? And Dario, I think, is very bold and brave in stating what that path could look like. It's great to see Sam and Elon say we've got to do something, too. But it's just a first step. Just on Harris. You're going to be back on tomorrow. We're going to talk more.

40:02We're going to have a whole piece about what happened with Hugging Face and OpenAI. And we'll have a lot more time. I appreciate it. We'll see you tomorrow. Up next, NYU professor and podcast host Scott Galloway, who also had concerns about AI. did his view change in any way after he heard what Dario Amadei had to say. We'll be right back.

40:26We've been focusing on reporting tonight on the worries surrounding AI and the pace at which it's advancing. In my interview with Anthropics CEO Dario Amadei tonight, he agreed the concerns are valid. He said there are serious risks of AI becoming uncontrollable. Didn't say exactly how likely he thinks a human extinction, he didn't put a percentage point on it, could be within the next decade, as others have suggested. But he didn't say it was not a possibility. Joining me now is NYU professor and podcast host, Scott Galloway. So, Scott, the last time we spoke, you said some of these warnings may be catastrophizing for fundraising.

40:57You've heard what Amadei says. The president is calling this a hoax, essentially. He said, Amadei says the pace is faster than he expected, and the hugging face attack basically freaked him out. Did anything change in your assessment?

41:14Well, not to be cynical about tech founders, but you got to wonder if the three biggest tech CEOs who are the leaders right now in AI call for a consensus. I wonder if they're calling for a cartel where they get to tap on the brakes and everyone else is forced to slam on the brakes. So, but we're in such uncharted territory. I don't think I can think of a single instance where the president of a country has called for less regulation and is bullying private companies into not being so timid and to go faster. But I stick with my original premise here. This is a lack of leadership on the part of the Trump administration.

42:00Regulation is not contrary to innovation and shareholder growth. You need rules such that the CEOs know the rules they're playing by and consumers have the trust to continue to engage and the general public has the trust to engage in these new technologies. I asked Amadei about something that you had, a comparison you previously made regarding who should be in control of powerful technology. I just want to play that in what he said. Should your company and OpenAI and others even be private? Scott Galloway has asked the question, would the U.S. have let Oppenheimer raise venture capital and sell bombs to the United Arab Emirates?

42:39Yeah, so, you know, this is one of these really deep questions that I and my co-founders and, you know, I hope the other people running these companies have pondered for a long time. And, you know, I'll tell you the truth. I'm very uncomfortable with some of these things being fully in private hands, and I'm also uncomfortable with the alternatives. He went on, but is that a satisfying answer in your mind? Well, he's not uncomfortable enough not to put out an S-1 claiming it's a$30 trillion market and that he's going to raise$100 billion at a$2 trillion market cap and has bankers lined up. So he seems to be getting comfort around an IPO.

43:25Look, there's a sweet spot, right? There's probably too much regulation, I would argue, the EU, Canada. some people would argue that the financial industry under the Biden administration, that there was so much regulation that had dampened growth and shareholder value. At the same time, absolutely no regulation creates this Wild West mentality where there's a tremendous fragility in the marketplace right now around the risks of something going deeply wrong. The market shed$150 billion in market capitalization, just AI and AI adjacent and companies. And the example that the Trump administration and certain tech leaders use is that China is coming for us.

44:03And if we regulate, they're going to get out ahead. What they also fail to mention is that China has exceptionally stringent regulation on AI. So we need to dispel the myth that regulation is necessarily detrimental to shareholder value and innovation. The fact that we have absolutely no regulation whatsoever and that you literally have the CEOs of Airbus and Boeing saying, I'm worried about my planes. So worried I'd like to inoculate myself from future liability and call for people to tap the brakes while I'm building the biggest engine in the world and going public. It all just feels very scattershot.

44:41And I don't think it's good for the markets. I don't think it's good for consumer confidence. And in my view, it threatens our leadership position in AI. Scott Galloway, thanks very much. Thank you, Anderson. Up next, Senator Bernie Sanders and former Trump White House advisor Steve Bannon are now allies in the fight for new AI regulations.

45:08You heard this hour in my interview with Anthropics CEO Dario Amadei how he is calling for an artificial intelligence slowdown of sorts and that he is support from his tech rivals, it seems, Sam Allman, the head of OpenAI, and Elon Musk. Also, President Trump is pushing back on the AI alarms, calling them a hoax. But tomorrow, there will be two unlikely allies calling for AI restrictions at a gathering in Washington. Progressive Independence Center Bernie Sanders and conservative podcaster Steve Bannon, a longtime supporter of the president. Both are among a group of people slated to speak at an event seeking a, quote, pro-human, end quote, approach to the technology.

45:42Sanders has proposed a bill to permanently ban superintelligent AI and temporarily pause advanced AI development until there's federal regulation. That's it for us. The news continues. The Source with Caitlin Collins starts now. I'm Adi Cornish. I'm Ari Shapiro, and it's Engagement Party. It is Engagement Party. And we get to talk about what we're obsessed with, what we're engaged with, what we need to process with a friend. Her name is Amy O'Dell. She is a fashion writer, blogger. But the thing is, she's a journalist who's written about the sacred cows of fashion. And yes, I mean, Gwyneth Paltrow and Anna Wintour.

46:16She has done biographies of both. And she is also the sort of rare fashion writer who, at this point, kind of outside the system. And so the question that we're going to try to tackle is... Is fashion terrible now? Well, no, I mean, I would phrase it as how do you develop a personal style in the age of AI slop? You know, on social media, like creators have to try a whole bunch of different things and see what catches and then they make it over and over and over again. The same thing happens with clothing. So they're sort of seeding the market, seeding the internet with all of these different ideas.

46:47and then they can see what sells and then they can just make them really fast. Follow Engagement Party. Wherever you get your podcasts.