AI for War? Debating the Military Use of AI

0:00Today on the AI Breakdown, we're debating the military applications of artificial intelligence. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube, our Discord, and our newsletter. For this week's AI Breakdown long read, we are turning to a topic that I think is getting more and more coverage. And that is the relationship between AI and warfare. Now, of course, in the context of the Oppenheimer movie just coming out, There are many people who are asking, is artificial intelligence another Oppenheimer moment?

0:39And if so, what are the lessons that we take away from that? So what we're going to start to read today's conversation is a piece that ran in the New York Times earlier this week called Our Oppenheimer Moment, The Creation of AI Weapons. The piece was written by Alex Karp, the CEO of the pretty controversial Palantir Technologies. Karp writes, In 1942, J. Robert Oppenheimer, the son of a painter and a textile importer, was appointed to lead Project Y, the military effort established by the Manhattan Project to develop nuclear weapons. Oppenheimer and his colleagues worked in secret at a remote laboratory in New Mexico to discover methods for purifying uranium and ultimately to design and build working atomic bombs.

1:20He had a bias towards action and inquiry. When you see something that is technically sweet, you go ahead and do it, he told a government panel that would later assess his fitness to remain privy to U.S. secrets. And you argue about what to do about it only after you have had your technical success. That is the way it was with the atomic bomb. His security clearance was revoked shortly after his testimony, effectively ending his career in public service. Oppenheimer's feelings about his role in conjuring the most destructive weapon of the age would shift after the bombings of Hiroshima and Nagasaki.

1:49At a lecture at MIT in 1947, he observed that the physicists involved in the development of the bomb, quote, have known sin, and that this is a, quote, knowledge which they cannot lose. We have now arrived at a similar crossroads in the science of computing, a crossroads that connects engineering and ethics, where we will again have to choose whether to proceed with the development of a technology whose power and potential we do not yet fully apprehend. The choice we face is whether to rein in or even halt the development of the most advanced forms of artificial intelligence, which some argue may threaten or someday supersede humanity, or to allow more unfettered experimentation with a technology that has the potential to shape the international politics of this century in the way nuclear arms shaped the last one.

2:28The emergent properties of the latest large language models, their ability to stitch together what seems to pass for a primitive form of knowledge of the workings of our world, are not well understood. In the absence of understanding, the collective reaction to early encounters with this novel technology has been marked by an uneasy blend of wonder and fear. Some of the latest models have a trillion or more parameters, tunable variables within a computer algorithm, representing a scale of processing that is impossible for the human mind to begin to comprehend. We have learned that the more parameters a model has, the more expressive its representation of the world, and the richer its ability to mirror it.

3:00What has emerged from the trillion-dimensional space is opaque and mysterious. It is not at all clear, not even to the scientists and programmers who built them, how or why the generative language and image models work, and the most advanced versions of the models have now started to demonstrate what one group of researchers has called sparks of artificial general intelligence, or forms of reasoning that appear to approximate the way that humans think. In one experiment that tested the capabilities of GPT-4, the language model was asked how one could stack a book, nine eggs, a laptop, a bottle, and a nail, quote, onto each other in a stable manner.

3:31Attempts at prodding more primitive versions of the model into describing a workable solution to the challenge had failed. GPT-4 excelled. The computer explained that one could arrange the nine eggs in a three-by-three square on top of the book, leaving some space behind them, and then place the laptop on top of the eggs, with the bottle going on top of the laptop and the nail on top of the bottle cap, with the pointy end facing up and the flat end facing down. It was a stunning feat of common sense in the words of Sébastien Boubec, the French lead author of the study who taught computer science at Princeton University and now works at Microsoft Research.

4:01It is not just our own lack of understanding of the internal mechanisms of these technologies, but also their market improvement in mastering our world that has inspired fear. A growing group of leading technologists has issued calls for caution and debate before pursuing further technical advances. An open letter to the engineering community calling for a six-month pause in developing more advanced forms of AI has received more than 33 ,000 signatures. On Friday, at a White House meeting with President Biden, seven companies that are developing AI announced their commitment to a set of broad principles intended to manage the risks of artificial intelligence.

4:31In March, one commenter published an essay in Time magazine arguing that, if somebody builds a too-powerful AI under present conditions, he expects that every single member of the human species in all biological life on Earth dies shortly thereafter. Concerns such as these regarding the further development of artificial intelligence are not unjustified. The software that we are building can enable the deployment of lethal weapons. The potential integration of weapon systems with increasingly autonomous artificial intelligence software necessarily brings risks. But the suggestion to halt the development of these technologies is misguided.

5:02Some of the attempts to reign in the advance of large language models may be driven by a distrust of the public and its ability to appropriately weigh the risks and rewards of the technology. We should be skeptical when the elites of Silicon Valley, who for years recoiled at the suggestion that software was anything but our salvation as a species, now tell us that we must pause vital research that has the potential to revolutionize everything from military operations to medicine. A significant amount of attention has also been directed at the policing of language that chatbots use and to patrolling the limits of acceptable discourse with the machine.

5:31The desire to shape these models in our image and to require them to conform to a particular set of norms governing interpersonal interaction is understandable, but may be a distraction from the more fundamental risks that these new technologies present. The focus on the propriety of the speech produced by language models may reveal more about our own preoccupations and fragilities as a culture than it does about the technology itself. Our attention should instead be more urgently directed at building the technical architecture and regulatory framework that would construct moats and guardrails around AI programs' ability to autonomously integrate with other systems, such as electrical grids, defense and intelligence networks, and our air traffic control infrastructure.

6:06If these technologies are to exist alongside us over the long term, it will also be essential to rapidly construct systems that allow more seamless collaboration between human operators and their algorithmic counterparts to ensure that the machine remains subordinate to its creator. We must not, however, shy away from building sharp tools for fear they may be turned against us. A reluctance to grapple with the often grim reality of an ongoing geopolitical struggle for power poses its own danger. Our adversaries will not pause to indulge in theatrical debates about the merits of developing technologies with critical military and national security applications.

6:39They will proceed. This is an arms race of a different kind, and it has begun. Our hesitation, perceived or otherwise, to move forward with military applications of artificial intelligence will be punished. The ability to develop the tools required to deploy force against an opponent, combined with a credible threat to use such force, is often the foundation of any effective negotiation with an adversary. The underlying cause of our cultural hesitation to openly pursue technical superiority may be our collective sense that we have already won. But the certainty with which many believed that history had come to an end, and that Western liberal democracy had emerged in permanent victory after the struggles of the 20th century, is as dangerous as it is pervasive.

7:15We must not grow complacent. The ability of free and democratic societies to prevail requires something more than moral appeal. It requires hard power, and hard power in this century will be built on software. Thomas Schelling, an American game theorist who taught economics at Harvard and Yale, understood the relationship between technical advances in the development of weaponry and the ability of such weaponry to shape outcomes. To be coercive, violence has to be anticipated, he wrote in the 1960s, as the United States grappled with its military escalation in Vietnam. The power to hurt is bargaining power.

7:45To exploit it is diplomacy. Vicious diplomacy, but diplomacy. While other countries press forward, many Silicon Valley engineers remain opposed to working on software projects that may have offensive military applications, including machine learning systems that make possible the more systematic targeting and elimination of enemies on the battlefield. Many of these engineers will build algorithms that optimize the placement of ads on social media platforms, but they will not build software for the U.S. Marines. In 2019, Microsoft faced internal opposition to accepting a defense contract with the U.S.

8:14Army. We did not sign up to develop weapons, employees wrote in an open letter to corporate management. A year earlier, an employee protest at Google preceded the company's decision not to renew a contract for work with the U.S. Department of Defense on a critical system for planning and executing special forces operations around the world. Quote, Building this technology to assist the U.S. government in military surveillance and potentially lethal outcomes is not acceptable, Google employees wrote in an open letter to Sindhar Pichai, the company's chief executive officer. I fear that the views of a generation of engineers in Silicon Valley have meaningfully drifted from the center of gravity of American public opinion.

8:47The preoccupations and political instincts of coastal elites may be essential to maintaining their sense of self and cultural superiority, but do little to advance the interests of our republic. The wonder kinder of Silicon Valley, their fortunes, business empires, and more fundamentally, their entire sense of self, exists because of the nation that in many cases made their rise possible. They charge themselves with constructing vast technical empires, but decline to offer support to the state, whose protections and underlying social fabric have provided the necessary conditions for their ascent.

9:14They would do well to understand that debt, even if it remains unpaid. Our experiment in self-government is fragile. The United States is far from perfect. But it is easy to forget how much more opportunity exists in this country for those who are not hereditary elites than in any other nation on the planet. Our company, Palantir Technologies, has a stake in this debate. The software platforms that we have built are used by U.S. and allied defense and intelligence agencies for functions like target selection, mission planning, and satellite reconnaissance. The ability of software to facilitate the elimination of an enemy is a precondition for its value to the defense and intelligence agencies with which we work.

9:48At Palantir, we are fortunate that our interests as a company and those of the country in which we are based are fundamentally aligned. In the wake of the invasion of Ukraine, for example, we were often asked when we decided to pull out of Russia. The answer is never because we were never there. A more intimate collaboration between the state and the technology sector, and a closer alignment of vision between the two, will be required if the United States and its allies are to maintain an advantage that will constrain our adversaries over the long term. The preconditions for a durable peace often come only from a credible threat of war.

10:16In the summer of 1939, from a cottage on the North Fork of Long Island, Albert Einstein sent a letter, which he had worked on with Leo Zillard and others, to President Franklin Roosevelt, urging him to explore building a nuclear weapon, and quickly. The rapid technical advances in the development of a potential atomic weapon, they wrote, seemed to call for watchfulness and, if necessary, quick action on the part of the administration, as well as sustained partnership founded on, quote, permanent contact maintained between the administration and physicists. It was the raw power and strategic potential of the bomb that prompted their call to action then.

10:46It is the far less visible but equally significant capabilities of these newest artificial intelligence technologies that should prompt swift action now. Now, the New York Times did an interesting thing where they had someone on their staff basically respond to Karp's piece. That piece is called Not Everyone is Against AI Weapons Research. Here's why. He writes, a raft of opinions have been offered on the race for artificial intelligence, most often calls for restraint, for caution, for a halt, for a slowdown, or for a complete pause. But whom are we arguing against? In a guest essay, Alexander Karp, the chief executive of Palantir Technologies, which develops software with military applications, provides us with a clear version of the other side of the debate.

11:24He argues for embracing the battlefield-changing potential of artificially intelligent weapons, and he makes the case that, as with the development of the atomic bomb, it would be irresponsible for the United States not to lead this effort. If you look at the wars around the world without rose-tinted glasses, it's hard to imagine AI-powered weaponry not soon playing a decisive role on the battlefield. In recent years, drone combat has changed the nature of infantry and mechanized warfare, and it's also hard not to imagine that given the recent progress on AI, more fully autonomous and lethal packs of battle drones might be unleashed to cloud the skies in whatever strife flares up in the years to come.

11:56It's not that I want to hear this argument because I agree with it. If the question is, should we build creative and intelligent deadly machines reminiscent of the Terminator, the answer to me is clear. You don't, and you should build consensus with your adversaries on that point. The guest author Jeremy Eshkanis goes on, Karp is not a disinterested party here. His company stands to benefit from increased investment in military AI. At the same time, in his writing and public speaking over the years, he has maintained a consistent perspective and vigorous critique of the culture of Silicon Valley that considers itself above the grubby problems of national defense.

12:27His consistency makes him an excellent candidate to make this case. I hope you'll read Karp's essay to understand better the type of argument that will most likely prevail in the halls of military power unless there's a strong immediate global effort to defuse it. If current dreams of artificial intelligence are realized, this technology will soon become a disruptive force throughout society and an autonomous, fearsome weapon for those who would use it to dominate the battlefield. Without credible evidence that countries like China, Russia, Iran, and North Korea would sit out this arms race, I find it hard to imagine a version of Karp's view not ultimately becoming our reality.

12:59All right, so back to NLW here. The thing that's really interesting about this piece is that it's really not so much about artificial intelligence, except insofar as it is the technology with big disruptive potential. What it's really about is our sense of global conflict heading deeper into the 21st century. The key thrust of this piece is an argument that democracy, Western liberal democracy, has not won to nearly the degree that we think it has, and that there are other powers in the world who will use whatever tools are at their disposal, including advanced AI, to ensure that the 21st century is not another W in the column of Western liberal democracies.

13:35In many ways, then, artificial intelligence becomes the context to have a much bigger, wider debate. And it's one that in many ways America has never really had with itself following the end of the Cold War. Sure, we lurched into the free trade era, but I'm not sure that we ever really got together to decide how we wanted to define America's intended role in the world heading into the 21st century. Now we find ourselves in a period of tension, tension with ourselves. On the one hand, there is an undeniable desire that cuts across political perspectives to withdraw from messy foreign wars, to focus more at home, and yet at the same time, the world is not sure it's willing to let us just stay back.

14:13Or at least it is showing that there are major consequences if we do. So one part of this discussion has to be America and its role in the world. At the same time, I don't know that we should view AI as totally subservient to that conversation. In other words, there is a universe in which, despite great power struggle between China and the US, there are rules and norms that the parties decide to obey because of the massively negative consequences if we don't. Is there an equivalent, in other words, of mutually assured AI destruction? No matter what, it's a discussion that's heating up. For example, a new paper from the American think tank, Center for a New American Security, argues that China's military AI advances are increasing the likelihood of an outright conflict with the U.S.

14:56The executive summary starts, two tectonic trends in the international security environment appear to be on a collision course. The first trend is the intensifying geopolitical rivalry between the United States and the People's Republic of China. The second trend is the rapid development of artificial intelligence technologies, including for military applications. They write, Taken together, the emergence of military AI will likely deepen U.S.-China rivalry and increase strategic risks. Now, this particular paper, which I'll link in the show notes, argues, like CARP, that part of the answer is to, quote, build U.S.

15:27military AI capabilities to stay on the cutting edge. And if you take away nothing else, it should be that the conversation is here and happening now. Anyways, guys, that is where I will leave it for today. This is a topic which we will get into much more, I'm sure, as events demand it. For now, I want to say thanks again to those of you who have listened or watched this episode, and until next time, peace.