AI Leaders Call for International Panel on AI Safety

0:00Today on the AI Breakdown, we're looking at two pieces that paint a very different vision of the future and the future of AI than Mark Andresen's Technooptimist manifesto. which we read earlier in the week. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.net for more information about our Discord, our newsletter, and our YouTube channel. All right, well, as I suspected, Mark Andreessen's techno-optimist manifesto was very controversial among listeners of the AI breakdown. Some of you were rah-rah, let's go, thank you for reclaiming the moral high ground for technology.

0:41Some of you liked parts but also disagreed with some of the more extreme positions. Some of you really didn't agree and thought it was just a rich guy talking his own bags. And some of you hated it so much that you begged me never to read anything from this buffoon again. Now, of course, as I said then, I present this type of content with the goal of giving you a really diverse array of perspectives on the future from a central vantage that wants to understand and absorb all the perspectives in order to come away with one's own best thoughts. But given that spirit, I thought it would be a perfect use of this weekend's long read to look at two essays that paint a very different picture.

1:22The first is from Stephen Levy in Wired and is a direct response to Marc Andreessen. And the second is an op-ed in the Financial Times from two other notables from the field that, again, paints a very different vision of the role of government in the AI future. We'll start with Stephen Levy's piece, What the Techno Billionaire Missed About Techno Optimism. And just by way of context, Stephen Levy is someone who has been around Marc Andreessen for now going on two and a half decades. Stephen begins, As a general rule, any essay that includes the one-sentence paragraph, I am here to bring the good news, is written by someone who wants to take your money, your vote, or your soul.

2:00As far as I know, Marc Andreessen isn't running for office, but the techno-optimism manifesto he posted this week is definitely bullish on inflating his already bloated wallet and narrowing the broad arc of human existence with a relentless pursuit of new and even risky technology. Andreessen's bolt from late-stage capitalism's Mount Olympus, i.e. Silicon Valley's Sand Hill Road, landed this week to a mixture of kudos and outrage. He posits that technology is the key driver of human wealth and happiness. I have no problem with that. In fact, I too am a techno-optimist, or at least I was before I read this essay, which attaches toxic baggage to the term.

2:34It's pretty darn obvious that things like air conditioning, the internet, rocket ships, and electric light are safely in the win column. As we enter the age of AI, I'm on the side that thinks the benefits are well worth pursuing, even if it requires vigilance to ensure that the consequences won't be disastrous. But Andreessen's screed isn't just about how great it is that we humans are a tool-building bunch. It's also an over-the-top declaration of humanity's destiny as a tech-empowered super species. Ayn Rand resurrected as a Substack author. If this essay had a soundtrack, it would be Wagner's Ride of the Valkyries.

3:04Andresen might have missed out on making an early investment in Uber, but he's all in on the Ubermensch. He even cites Friedrich Nietzsche as one of his patron saints of techno-optimism. Perhaps a better title for this essay would be the Techno-Billionaire Manifesto, as it attempts to justify not only an unquestioning pursuit of technology, but the late-stage capitalism that provides out-of-whack rewards for the system's winners. Like Andreessen. In his argument, the market-based techno-capital machine is the infallible generator of merit and production. Never mind the astonishing income inequality that has dragged the world down and fomented destructive political unrest.

3:36Money, proclaims Andreessen, is the only motivator capable of producing the giant technological leaps that advance humanity. This will be news to the inventors of the internet, who were civil servants and academic geeks with zero-profit motive. In fact, for many years, they were adamantly opposed to any commercialization whatsoever. Andresen does proclaim that he opposes monopolies in regulatory capture. Maybe he believed that when his browser company Netscape was buried by Microsoft. But that's a hollow declaration from someone who sat on the board of Facebook, now Meta, for 15 years. I'd love to peek at the minutes to see how often he has invaded against monopoly and lobbying in board meetings.

4:08Andresen argues that advanced technology creates abundance that lifts all humans. We believe there is no conflict between capitalist profits and a social welfare system that protects the vulnerable, he writes. But though he might not perceive it from his home in Atherton, California, the country's richest zip code, the country he lives in presents a counter-argument. While the U.S. has the most advanced technology in the world, the life expectancy of its citizens has dropped. Surely he knows of the homelessness problem in America's cities most glaring in nearby San Francisco. He might even have read that the vast majority of average Americans can't afford to buy a home, and that 40 % would struggle to cover an unexpected $400 expense.

4:42The techno-capital machine doesn't seem to be working for them. But don't worry, Andreessen cites an Andy Warhol quote celebrating how well our system works because poor people and rich people alike can enjoy a Coca-Cola. Let him drink sugar water. Speaking in the first person plural, without identifying any cosigners, Andreessen writes, we believe in an absolute rejection of resentment. Yet the essay oozes with resentment particularly of the unnamed critics who disagree with him. We are being lied to, he charges using language of the demagogue. Though he doesn't name any critics, Andreessen warns readers of chivalrous that have been used as part of a quote, vast demoralization campaign, many, he says, derived from communism.

5:16I wonder why Joe McCarthy didn't make his patron saints list. What are these despicable terms? They include sustainability, tech ethics, and risk management. Do you endorse social responsibility? Must be a commie. Andreessen professes to be open to criticism himself, repeating physicist Richard Feynman's line, I would rather have questions that can't be answered than answers that can't be questioned. But if you disagree with Andreessen about AI's untrammeled beneficence, and suggest we proceed with caution due to ethical risks or a fear that algorithms will snuff out humanity, he literally accuses you of a capital crime.

5:45He writes, Any deceleration of AI will cost lives. Deaths that were preventable by the AI that was prevented from existing is a form of murder. To me, the abiding mystery of Andreessen is, what happened? When I first met him in the mid-90s, he was a joyful, energetic lad from the heartland. The young Andreessen struck me as supremely self-confident and a very fast talker. But despite his intensity, there was a welcoming Joao DeVille to his conversation. Now he is a billionaire triumphalist extolling the genius of an anarcho-capitalist system that provides millions with iPhones and autos but denies healthcare and homes to millions.

6:15Solving those problems would seem our worthiest goal. But like other like-minded tech moguls, he instead dreams of having billions of humans living in outer space. Yes, I'm a techno-optimist, but in lowercase. I will co-sign one of the vital questions Andreessen raises in his essay. What world are we building for our children, and their children, and their children? Our solutions are very different. Alright, back to NLW here. I wanted to read that one because while it is a stinging rebuke and critique, it strikes strikes me as done in good faith, and comes from someone who might like to have a conversation with Marc Andreessen about this.

6:48It's not from someone who hates tech a priori, and I think it asks a lot of reasonable questions. I certainly agree with some of the specifics, like the utter weirdness of being accused of murder for asking if AI should be slowed at all. It's such an aggressive argument that it effectively draws a line in the sand that says there can be no discussion on these points at all. I personally am of the bias that we've had far too many of these line in the sand, us versus them, my team or their team, and anyone who disagrees as a murderer kind of conversations, and they just haven't worked. But I gotta tell ya, and this is my optimistic side coming out, having a forcing function to drag these conversations into the world is a lot better than if we never had them, even if the source of it drives you nuts.

7:29But like I said, I didn't want to necessarily just spend this time on critiques of that piece. Instead, I also wanted to offer at least one alternative vision. Very central to Andreessen's argument is a clear, utter lack of faith in government as even a partner in the development of the future. Yes, he frames it in language of communism and old terms that come from the mid-20th century, but the point that's clear is that government is firmly on that enemy's list. An enemy of progress, in other words. There are many who feel, however, that when it comes to the significance of the AI revolution and properly figuring out how it fits and how it services the greatest portion of humanity, that government is going to have a very significant role.

8:08One that, yes, they might not be exactly prepared for, but is coming for them regardless. This is a little less grandiose and a little more specific, but in the Financial Times this week, former Google CEO Eric Schmidt and Inflection CEO Mustafa Suleiman wrote an essay called We Need an AI Equivalent of the IPCC. An international panel on AI safety could offer an objective body to help shape protocols and norms. They write, AI is here. Now comes the hard part, learning how to manage and govern it. As large language models have exploded in popularity and capability over the past year, safety concerns have become dominant in political discussion.

8:44For the first time, artificial intelligence is top of the intray for policymakers the world over. Even for those of us working in the field, the rate of progress has been electrifying. And yet, it's been equally eye-opening to see the extraordinary public, business, and now political response gathering pace. There's a growing consensus this really is a turning point as consequential as the internet. Clarity about what should be done about this burgeoning technology is a different matter. Actionable suggestions are in short supply. What's more, national measures can only go so far given its inherently global nature.

9:13Calls to just regulate are as loud and as simplistic as calls to simply press on. Before we charge headfirst into over-regulating, we must first address lawmakers' basic lack of understanding about what AI is, how fast it is developing, and where the most significant risks lie. Before it can be properly managed, politicians and the public need to know what they are regulating and why. Right now, confusion and uncertainty reign. What's missing is an independent, expert-led body empowered to objectively inform governments about the current state of AI capabilities and make evidence-based predictions about what's coming.

9:44Policymakers are looking for impartial, technically reliable, and timely assessments about its speed of progress and impact. We believe the right approach here is to take inspiration from the Intergovernmental Panel on Climate Change, or IPCC. Its mandate is to provide policymakers with regular assessments of the scientific basis of climate change, its impact and future risks, and options for adaption and mitigation. A body that does the same for AI, one rigorously focused on a science-led collection of data, would provide not just a long-term monitoring and early warning function, but would shape the protocols and norms about reporting on AI in a consistent global fashion.

10:16What models are out there? What can they do? What are their technical specifications? Their risks? Where might they be in three years? What is being deployed where and by whom? What does the latest R &D say about the future? The UK's forthcoming AI Safety Summit will be a first-of-its-kind gathering of global leaders convening to discuss the technology's safety. To support this discussion and to build towards a practical outcome, we propose an international panel on AI safety, APAIS, an IPCC for AI. This necessary, measured, and above all achievable next step will provide much-needed structure to today's AI safety debate.

10:48The IPAIS would regularly and impartially evaluate the state of AI, its risks, potential impacts, and estimated timelines. It would keep tabs on both technical and policy solutions to alleviate risks and enhance outcomes. Significantly, the IPCC doesn't do its own fundamental research, but acts as a central hub that gathers the science on climate change, crystallizing what the world does and doesn't know in authoritative and independent form. An IPAIS would work in the same way, staffed and led by computer scientists and researchers rather than political appointees or diplomats. This is what makes it such a promising model.

11:20By staying out of primary research or policy proposals, it can avoid the conflicts of interest that inevitably come with a more active role. With a scope narrowly focused on establishing a deep technical understanding of current capabilities and their improvement trajectories, it would be cheap to run, impartial, and independent. Built on a broad international membership. Given that much of the most cutting-edge work in AI is undertaken by businesses, ensuring sufficient transparency from leading companies is essential. And IPAIS will help here even before legal mechanisms come into play, establishing a trusted body to report into, creating expectations and standards around sharing to provide space for maximal openness in a tight commercial market.

11:56Where full access isn't possible, it would still aggregate all publicly available information in the most comprehensive and reliable form. Trust, knowledge, expertise, impartiality, these are what effective, sensible AI regulation and safety will be built on. Currently, they are lacking. We believe that establishing an independent scientific consensus about what capabilities have been developed and what's coming is essential in developing safe AI. This is an idea whose time has come. Now, I should note before I dig into the analysis here that in addition to Mustafa and Eric, Dario from Anthropic, well-known geopolitics guy Ian Bremmer, and others including Reid Hoffman are also part of this proposal.

12:34So what to think of this? Well, on the one hand, it's clearly not a panacea. No independent international body could possibly hope to be a cure-all. AI policy is inherently a space that is going to require a huge number of voices, and unfortunately we are not going to get away from politics being a part of it. However, the recognition that right now, policymakers are largely learning about this by one of two sources, either one, on the one hand, the companies building the technology, who even assuming good faith still have an unbelievably high incentive to push towards policy that benefits them, or on the other end, an increasingly well-coordinated campaign of AI safety folks with deep pockets, who are convinced that we are teetering on the edge of global collapse, it kind of leaves a lot of space in the middle.

13:17The idea that there is a body whose specific mandate is to absorb a huge amount of information, in fact, all the information that is possibly available, and just provide a more scientific gut check, frankly, to the loudest voices in the room, I think has a lot of merit to at least discuss. Indeed, I think that many of these issues that will find their way into AI policy are a great example of how you can have a topic where 80 or 90 % of the people even are somewhere in the middle, not quite sure what to think, looking for more information, open to lots of possibilities, see some good, see some bad, but where on the flip side, 90 % plus of the discourse is by one of the two extreme sides screaming at one another.

13:54Now I understand in this case why those voices get raised. If one believes that humanity is actually sleepwalking, or even worse, speedrunning into its own doom, you're going to be pretty passionate about that. But I do think it behooves us to have the counterweight to that not be only the dismiss-it-out-of-hand, and if you try to slow down AI, you're a murderer kind of voice, although we do need that extreme too, but also perhaps some middle-spaces institutions and people who want to drag on their waiters and muck around in the messy nuance. If this body can be a part of that? Not all of that, but a part of that?

14:27Why not? I agree it seems like an extremely cheap option that's basically nothing but net value. Anyways, guys, there you have it, a very different perspective than the one we heard on Monday. Hopefully this makes you feel more balanced, a little more yin and yang in concert as you head into this fall weekend. I, of course, appreciate you guys listening or watching wherever you are. And until next time, peace.

14:55Thank you.