A New AI Safety Report (and Why the Media Loves the AI Extinction Narrative)

0:00Today on the AI Breakdown, a new report funded by the State Department of the State Department main episode, and frankly, I didn't even get into everything that I could have. So for that reason, we're going to skip the brief today. We will be back with our normal format, however, tomorrow with a profile of a couple new pieces of software that some folks are getting very, very excited about, including what some are calling the most advanced AI coding app that they've ever seen. However, for this episode, we are talking all about this AI safety report and even more, the media coverage surrounding these issues.

0:58Welcome back to the AI breakdown. A new report has been flying all over the media wires and the Twitter sphere, and probably the most referenced article about it is this piece in Time magazine called Exclusive. U.S. must move decisively to avert extinction-level threat from AI, government-commissioned report says. I will be clear right up front that I went from interested to see what this report had to say, to fairly disgusted with the media coverage around it in the 24 hours that I've been paying attention. I'll explain a little bit more about why, but first, let's talk about what the report actually says.

1:34To get a sense of the position, time begins. The U.S. government must move, quote, quickly and decisively to avert substantial national security risks stemming from artificial intelligence, which could, in the worst case, cause an extinction-level threat to the human species, says a report commissioned by the U.S. government published on Monday. The report reads, current frontier AI development poses urgent and growing risks to national security. The rise of advanced AI and AGI has the potential to destabilize global security in ways reminiscent of the introduction of nuclear weapons. The executive summary is called Defense in Depth, an action plan to increase the safety and security of advanced AI.

2:08The executive summary reads, The recent explosion of progress in advanced artificial intelligence has brought great opportunities, but it is also creating entirely new categories of weapons of mass destruction and WMD-enabling catastrophic risks. A key A driver of these risks is an acute competitive dynamic among the frontier AI labs that are building the world's most advanced AI systems. All of these labs have openly declared an intent or expectation to achieve human-level and superhuman artificial general intelligence, a transformative technology with profound implications for democratic governance and global security by the end of this decade or earlier.

2:38From there, they say that the frontier lab executives have acknowledged these dangers, but continue to push anyways, and so they conclude, there is a clear and urgent need for the US government to intervene. This action plan, they write, is a blueprint for that intervention. Its aim is to increase the safety and security of advanced AI by countering catastrophic national security risks from AI weaponization and loss of control. So what actions do they propose? Well, they say they follow a sequence that begins by establishing interim safeguards to stabilize advanced AI development, including export controls on the advanced AI supply chain, leverages the time gained to develop basic regulatory oversight and strengthen U.S.

3:11government capacity for later stages, transitions into a domestic legal regime of responsible AI development and adoption, safeguarded by a new U.S. regulatory agency, and finally extends that regime to the multilateral and international domains. Here's the way that Time sums it up. The finished document recommends a set of sweeping and unprecedented policy actions that, if enacted, would radically disrupt the AI industry. Congress should make it illegal, the report recommends, to train AI models using more than a certain level of computing power. The threshold the report recommends should be set by a new federal AI agency, although the report suggests as an example that the agency could set it just above the levels of computing power used to train current cutting-edge models like OpenAI's GPT-4 and Google's Gemini.

3:50The new AI agency should require AI companies on the frontier of the industry to obtain government permission to train and deploy new models above a certain lower threshold. Authorities should also urgently consider outlawing the publication of the weights or inner workings of powerful AI models, for example under open-source licenses, with violations possibly punishable by jail time. And the government should further tighten controls on the manufacture and export of AI chips, and channel federal funding toward alignment research that seeks to make advanced AI safer. Even Time calls the report's recommendations, quote, previously unthinkable.

4:19Time also talks about how the report hones in on hardware as a good area to regulate as a way to slow AI down. Quote, Regulating the proliferation of this hardware, the report argues, may be the most important requirement to safeguard long-term global safety and security from AI. It says the government should explore tying chip export licenses to the presence of on-chip technologies, allowing monitoring of whether chips are being used in large AI training runs. Now the response to this report has, as you might expect, fallen almost completely along the political dividing lines of the AI safety folks on the one hand and the accelerationists on the other.

4:50Prominent AI safety voice Max Tegmark writes, a report commissioned by the US government finds that future AI poses an extinction-level risk. I agree. Ed Newton-Rex, who is the person who is in charge of audio at Stability AI but left over their copyright thinking, tweets, This is an extremely important report. Commissioned by the U.S. government, it outlines two key existential risks from AI, weaponization by rogue actors and escaping the control of humans. And it strongly recommends governments take these threats seriously, whatever the potential upsides of AI, and act fast. On the flip side, there are folks like Riva who tweets, Report commissioned by the U.S.

5:22government says that the U.S. government are best poised to manage AI. Shocker. Eric Hartford from Abacus writes, What's next? An AI prohibition? AI speakeasies? Amjad Massad, the CEO of Replit, writes, To save you a click, the government wrote a$250K check to a handful of pre-committed AI doomers to write a doom and gloom report, so they wrote one. To which George Hotz responded, it's called manufacturing consent. And honestly, these were some of the tame responses. Daniel Jeffries writes, I'll keep it simple. Here's what I recommend. Take any report that seriously includes a discussion about AI killing us all and pitch it right in the trash.

5:53Never allow anyone with this delusional viewpoint within 5 ,000 miles of a policymaker. But Nick Marta, the technical lead on AI governance at Mozilla, gets at one of the issues that I think is core here. Nick writes, I will now be referring to every NSF-funded paper as a government-commissioned report. Brilliant communications, though. Just go around saying the U.S. government commissioned this first-ever action report, get a flattering news article about it, and then monetize that traffic. What he's pointing to is the fact that to get a copy of the action plan, you have to sign up with your name and email on the Gladstone website, which is the group that was behind this research.

6:24Now, let's talk about Gladstone for a minute. Here's how Time describes the firm. Jeremy and Edward Harris, the CEO and CTO of Gladstone respectively, have been briefing the U.S. government on the risks of AI since 2021. The duo, who are brothers, say that government officials who attended many of their earliest briefings agreed that the risks of AI were significant, but told them the responsibility for dealing with them fell to different teams or departments. In late 2021, the Harris' say Gladstone finally found an arm of the government with the responsibility to address AI risks, the State Department's Bureau of International Security and Non-Proliferation.

6:54Teams within the bureau have an interagency mandate to address risks from emerging technologies, including chemical and biological weapons, and radiological and nuclear risks. Following briefings from Jeremy and Gladstone's then-CEO Mark Beal in October 2022, the Bureau put out a tender for report that could inform a decision whether to add AI to the list of other risks it monitors. The State Department did not respond to a request for comment on the outcome of that decision. The Gladstone team won that contract. So basically, this is a team of four people, or at least it was four when they won this$250 ,000 contract in October 2022, as it appears that the third co-author of the report, who was Mark Beal, a former Defense Department official, has subsequently left Gladstone to start a super PAC aimed at advocating around AI policy.

7:34His PAC is called Americans for AI Safety and also officially launched on Monday. Writes Time, it aims to make AI safety and security a key issue in the 2024 elections, with the goal of passing AI safety legislation by the end of 2024. The PAC did not disclose its funding commitments, but said it has, quote, set a goal of raising millions of dollars to accomplish its mission. Time also notes that the Harris brothers were previously the entrepreneurs behind a company that went through Y Combinator, and writes, the pair brandished these credentials as evidence that they have the industry's interests at heart, even as their recommendations, if implemented, would upend it.

8:03Now, let's read the tweet thread that Edward Harris did around this. He writes, Here's what we've been working on for over a year. The first U.S. government commissioned assessment of catastrophic national security risks from AI, including systems on a path to AGI. TLDR, things are worse than we thought, and nobody's in control. Harris continues, We started this work with concerns, but no preconceptions. We knew there were solid technical reasons that AI could eventually pose catastrophic risks. but we went in looking for reasons to change our minds. We found the opposite. Our overriding goal was to get to the truth.

8:31To do that, we had to do more than just speak to policy and leadership at the AI labs. We also connected with individual technical researchers, many of whom are way more concerned than their labs let on in public. Many of these folks came forward on condition of anonymity to share stories. He then goes on to tell a number of anecdotes and pitches the framework that they came up with. Now to me, one issue here is of course, debate over the substantive issues herein. The policies advocated for here are absolutely extreme relative to where most people are, but I don't have any problem with people having these debates.

9:01It's totally fine for a group of researchers to dig into these issues and come to the conclusion that these are the right ideas. We live in a free society where everyone's positions, even extreme positions, gets to be debated. More broadly, I don't mind there being a great, even fierce debate between the extremes of accelerationism and safetyism at either ends of the discussion. What's frustrating to me, and I think to many who are observing how this is being presented, is in fact in the press coverage. I read you the Times headline, U.S. must move decisively to avert extinction-level threat from AI, Government Commission Report says.

9:32VentureBeat says State Department-backed report provides action plan to avoid catastrophic AI risks. In short, these reports make it seem like President Biden himself commissioned this report, that this was a central and key asset in U.S. government decision-making, rather than what it appears to have been, which is a group that had a pre-existing concern around a particular issue, spent a bunch of time building relationships and trying to find others who might resonate with that position, and then got a grant for a quarter million dollars to write a report based on that position, which for all we know, might have either made waves inside the specific department within the State Department that paid for the contract, or might have been chucked right in the trash can.

10:10What's irresponsible, in other words, is the presentation of this like it is the State Department and the U.S. government's position. It lends it an authority which it should not have, and which warps that debate that I'm so encouraging of there being. The issue, of course, is that this is not somehow out of the norm for how press and media treat these issues right now. Sometime around this time last year, the press went from never caring at all and never thinking about any sort of AI-related issues to being completely and utterly in love with this extinction narrative. As that's happened, we've seen American attitudes about AI decline precipitously.

10:45And of course, I don't want to blame that entirely on the media. There are plenty of reasons why someone might be concerned about AI in terms of job loss or whatever other issue there might be, to say nothing of the overall trend of declining trust in big tech. But it certainly makes it very hard to have a rational, wide-ranging conversation about these issues when people are being pummeled constantly with the idea that Terminator is going to come end their world. Now, I don't expect AI safety advocates to have any issues with this. It's likely to me that they'll see this as simply winning the battle of ideas.

11:14But I find myself personally radicalizing in the other direction because I think people are getting such a warped and one-sided conversation. And apparently I'm not the only one who feels that way. Just before I pressed record on this show, I noticed that a new organization called the Alliance for the Future had just announced itself with a mission effectively to counter these types of narratives when it comes to artificial intelligence. John Esconis writes, It's time to take back our future. I'm proud to be a small part of the Alliance for the Future, a new political advocacy organization aiming to give the future, our future, our children's future, America's future, a voice in American politics, and to push back against a growing conspiracy against it.

11:50Alliance for the Future is an alliance, and it's a big tent. We've got EACCs, American Dynamists, Libertarians, Civil and Otherwise, China Hawks, Cypherpunks, and more on board. What we have in common is an enemy, the people who would rather cancel America's future than give up the power and wealth they currently enjoy, or their neurotic desires to exert control. Today's battleground is AI. Most Americans have little idea about how AI actually works, how it differs from Hollywood fantasies, and about how the benefits they, and especially working-class Americans, could enjoy are being canceled by an organized cabal of Baptists and bootleggers.

12:19Now, I had started to wonder at what point we were actually going to see a counter-response to the doom and gloom narrative, and here we have it. I still tend to think that talk to the average person, and they're going to be way more in the middle than either of these extremes, and in that middle space, there's going to be lots of common-sense approaches that involve neither prohibition nor complete ignoral and ignorance, but until those more moderate or just still being formed opinions get coverage in the media, I think we're a little bit doomed to be subject to just endless back and forth between the narrative extremes.

12:52Hopefully this AI breakdown space can counteract that trend at least in small ways by at least presenting all sides of this, and yet, as much as this audience has grown, it remains still small, relatively speaking. But I certainly wouldn't want to be talking about anything else, and so I'm glad to have you all here on this journey. For now, that's going to do it for today's AI Breakdown. Until next time, peace.