Warriors, Doomsayers, Reformers: Understanding the Factions in AI

0:00Today on the AI Breakdown, we're discussing the three factions in the battle for artificial intelligence. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube channel, our newsletter, and our Discord.

0:24Hello, friends. Happy weekend. Well, usually I do a long read Saturday and then something a little bit different on Sunday. But this week, there was another candidate for the long read that really is not only about the essay itself, but about the provocation and the discussion that comes after. And I was too excited to not set it up as a framework because I think that you're going to have a lot of fun debating this as well. So what we're going to do is read excerpts from the piece first, and then we'll get into some of the discussion. Now, the piece came from the New York Times. It was published on Thursday and is called The AI Wars Have Three Factions and They All Crave Power.

1:02It was written by Bruce Schneier and Nathan Sanders. Schneier is a security technologist and lecturer at Harvard Kennedy School, while Dr. Sanders is a data scientist affiliated with the Berkman Klein Center at Harvard University. The two begin, there is no shortage of researchers and industry titans willing to warn us about the potential destructive power of artificial intelligence. Reading the headlines, one would hope that the rapid gains in AI technology have also brought forth a unifying realization of the risks and the steps we need to take to mitigate them. The reality, unfortunately, is quite different.

1:33Beneath almost all of the testimony, the manifestos, the blog posts, and the public declarations issued about AI, are battles among deeply divided factions. Some are concerned about far future risks that sound like science fiction. Some are genuinely alarmed by the practical problems that chatbots and deep fake video generators are creating right now. Some are motivated by potential business revenue, and others by national security concerns. The result is a cacophony of coded language, contradictory views, and provocative policy demands that are undermining our ability to grapple with a technology destined to drive the future of politics, our economy, and even our daily lives.

2:06So basically what these guys are saying, their setup, is that underneath the conversation around AI safety, AI regulation, AI policy, AI ethics, there are these different cohorts who have their own set of motivations, their own set of goals. And that in order to interpret and understand what different people are saying as they are advocating for different policies or different solutions, these authors believe that you need to understand those three factions and where they are coming from. As they put it, to understand the fight and the impact it may have on our shared future, look past the immediate claims and actions of the players, to the greater implications of their points of view.

2:40When you do, you'll realize this isn't really a debate only about AI. It's also a contest about control and power and how resources should be distributed and who should be held accountable. Indeed, they write, beneath this roiling discord is a true fight over the future of society. Should we focus on avoiding the dystopia of mass unemployment, a world where China is the dominant superpower, or a society where the worst prejudices of humanity are embodied in opaque algorithms that control our lives. So their contribution to this conversation is to try to decode the language behind some of these different factions by giving them public light.

3:15First up, they write, are the Doomsayers. And any of you guys who have spent any time around AI will know that this is a loud group that has gotten much more of a megaphone over the last five months or so. They write, The loudest perspective is a frightening, dystopian vision in which AI poses an existential risk to humankind, capable of wiping out all life on Earth. AI in this vision emerges as a godlike, super-intelligent, ungovernable entity capable of controlling everything. It's likened to monsters like the Lovecraftian Shoggoths, artificial servants that rebelled against their creators, or paperclip maximizers that consume all of Earth's resources in a single-minded pursuit of their programmed goal.

3:50It sounds like science fiction, but these people are serious and they mean the words they use. Now, in terms of who is among this camp, the two point to the two 2018 Turing Award winners, Jeff Hinton and Yoshua Bengio. However, they point out that this faction is also boosted by a, quote, class of tech elite that has enormous power to shape the conversation. The two point specifically to the rise of effective altruism, which of course in many ways is at this point in popular imagination most connected to the disgraced former founder of FTX, Sam Bangman-Fried. But the authors find some problems with this cohort.

4:22They write, Many doomsayers say they are acting rationally, but their hype about hypothetical existential risks amounts to making a misguided bet with our future. One example he gives is Yon Talin, a long-termer and EA-aligned person who is also the co-founder of Skype. They write he has, made dismissive noises about climate change because he thinks it pales in comparison with far-future unknown unknowns like risks from AI. The technology historian David C. Brock calls these fears, quote, wishful worries. In other words, quote, problems that would be nice to have in contrast to the actual agonies of the present.

4:53However, the main issue, they say with the doomsayers, is the absolute implausibility and unrealism of simply hitting pause on technological development. They conclude, while we shouldn't dismiss the Hollywood nightmare scenarios out of hand, we must balance them with the potential benefits of AI and most important, not allow them to strategically distract from more immediate concerns. Let's not let apocalyptic prognostications overwhelm us and smother the momentum we need to develop critical guardrails. The second category they call the Reformers. They write, While the Doomsayer faction focuses on the far-off future, its most prominent opponents are focused on the here and now.

5:28We agree with this group that there's plenty already happening to cause concern. The way that they describe this group is, quote, a distressingly familiar vision of dystopia, a society in which humanity's worst instincts are encoded into and enforced by machines. The Doomsayers think AI enslavement looks like the Matrix. The Reformers point to modern-day contractors doing traumatic work at low pay for OpenAI in Kenya. They point out that many of the people who belong to these groups are from historically marginalized groups to begin with and so who have a sense of the stakes when it comes to encoding bias in this new generation of technologies.

6:00Another subset of the group focuses on the problems of social media and how it's created problems with hate speech, misinformation, disinformation, and challenges to democracy. The authors write, While doomsayers and reformers share the concern that AI must align with human interests, reformers tend to push back hard against the doomsayer's focus on the distant future. They want to wrestle the attention of regulators and advocates back towards present-day harms that are exacerbated by AI. And this, of course, leaves their third category, which they call the warriors. They say other groups of prognosticators cast the rise of AI through the language of competitiveness and national security.

6:34One version has a post-9-11 ring to it, a world where terrorists, criminals, and psychopaths have unfettered access to technologies of mass destruction. Another version is a Cold War narrative of the United States losing an AI arms race with China and its surveillance-rich society. Some arguing from this perspective are acting on genuine national security concerns and others have a simple motivation, money. These perspectives serve the interests of American tech tycoons as well as the government agencies and defense contractors they are intertwined with. They then get into the arguments around regulatory capture, which will be well familiar to this particular audience, saying that people like Sam Altman and Mark Zuckerberg have an incentive to try to get onerous regulations because they are already in the leading positions and have the ability to comply expensive as though it might be as compared to others who might come up and challenge them in the future.

7:17The authors also point out that many of these warriors have the ear of people in power in Washington. They write, The warrior's narrative seems to misrepresent that science and engineering are different from what they were during the mid-20th century. AI research is fundamentally international. No one country will win a monopoly. And while national security is important to consider, we must also be mindful of self-interest of those positioned to benefit financially. They argue that fears about existential risk of AI are really fears about the threat of uncontrolled capitalism. They argue that regulatory solutions do not need to reinvent the wheel.

7:47Instead, just doubling down on the rules we know limit corporate power, etc., etc., etc. I'm actually not going to go deep into their particular interpretation because I'm much more interested in the way that they've broken apart this industry than in these folks' interpretation of what we should do. Although, of course, I will link to this so you can go decide for yourself if they have good ideas. I think a bunch of things about this piece. One, I love that it exists. I think it's going to add a lot of nuance to the conversation. And I especially love that it's in a major mainstream publication like the New York Times that already has good advancement for getting beyond strict binaries, right?

8:18I think especially helping people understand that when it comes to AI safety and risk, there are these competing factions of people who have very different ideas about where we should place our emphasis. I think that one can make an argument that characterizing the AI safety folks as doomsayers is inherently undermining to their position. But then again, they, like everyone else, have to deal with branding and marketing to get their ideas across to society, so there you go. I think that they accurately represent that there is a whole different cohort who is worried about a very different set of issues, and that both of those two groups, although they are talking about AI risk, are at loggerheads because their intended remediations are so wildly different.

8:54Now, of course, any thoughtful person will tell you, and that acknowledging things like extinction risk doesn't mean that you don't want to work on AI bias for systems that exist right now. They may be less related than they seem, but they are not mutually exclusive as they are sometimes presented. Where I think this particular piece kind of falls off the rails is in the warrior section, the categorization of this third group. There are a bunch of reasons it falls off. I think that the authors are right to identify that there is a warrior class that relates to AI. We've been talking about it all week.

9:23The CIA is building its own chatbot for the 18 security agencies that make up the American national intelligence community. The NSA just announced that it has its own new AI security center. Every week we get some new announcement about some new thing that the Pentagon and military are doing in terms of AI weaponry, AI and geopolitics are completely intertwined, and they are right to identify that that is a big group. Where I'm not sure they're so correct is the easy lumping in of the AI tech billionaires with that cohort. Some certainly are. Palantir certainly is. On a previous long read, we actually read a piece by Palantir CEO Alex Karp that makes this argument explicit, it, but others are a lot less so.

10:02I do think there is actual value in trying to understand where those big tech leaders are coming from, if only for the reason that like it or not, they have incredible power to shape how this all turns out. I think that in many ways requires nuance that we're not giving it. Just writing off these players as looking for regulatory capture undermines them as individuals who are dealing with a lot of competing and conflicting forces, but it also isn't going to lead us to make good policies. Much more, there's lots of reasons to think that it's just not that true. The article also doesn't get deep into the different factions within the technology community.

10:37Some of the most ardent, future-forward, regulation-like tech entrepreneurs that I know have become extreme AI safety advocates. But likewise, there is another entire movement not even really touched on here in, of course, the accelerationists. Now, they may not be big enough for a couple of Harvard professors to warrant notice, but to the extent that Silicon Valley and the extended technology industry are shaping our discussions of AI, which I believe that they are, leaving out big categories of people probably isn't going to get us to a better conversation. Maybe the right thing to do would be for our AI Breakdown community to come together and create a slightly more comprehensive taxonomy of all the different cohorts and groups that exist in here.

11:14I certainly think we need one for the shruggers, by which I don't mean some Lovecraftian monster that you just missed, but the people walking around shoulders raised just trying to figure it all out and not being super sure about what's the right approach because there's so many different ways that things could go. Still, like I said, right at the top, I think that it's a good thing, a net easily good thing that this article and articles like it are starting to appear. It may not be as nuanced as I like, but it's a hell of a lot more nuanced than we were getting just a few months ago. And that my friends is progress.

11:42Anyways, that will do it for today's episode. I appreciate you guys listening or watching as always until next time. Peace.

11:54Thank you.