The Fight for AI's Soul Following OpenAI's Leadership Battle

0:00Today on the AI Breakdown, we're reading from a series of opinion pieces about the state of the industry following OpenAI's crazy leadership challenges over the last two weeks, not to mention the one-year anniversary of ChatGPT. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube, our Discord, and our newsletter. Hello, friends. We are back with a long-reads episode of the AI Breakdown. And today, instead of reading just one piece, we are going to read some excerpts from a number of pieces.

0:34This week was the one-year anniversary of ChatGPT, which launched on November 30th, 2022. And between that and Sam Altman's official return as CEO of OpenAI, there's been a lot of context for reflection on what comes next. So the first piece we're going to read from comes from the Los Angeles Times, who frankly tend to be fairly anti-AI with their coverage, although not universally so. And the opinion piece is called Opinion. OpenAI's drama marks a new and scary era in artificial intelligence. The piece is by Darren Acemoglu and Simon Johnson. The piece begins, Salm Altman's dismissal and rapid reinstatement as CEO of OpenAI, the creator of ChatGPT, confirms that the future of AI is firmly in the hands of people focused on speed and profits at the expense of all else.

1:19This elite will now impose their vision for technology on the rest of humanity. Most of us will not enjoy the consequences. Now, in the next paragraph, they make a claim which is not backed up, saying, We may never know what really happened on November 17th when the board fired Altman, But the most likely interpretation is that the members of the board were troubled by Altman's commercial emphasis and the headlong rush to develop new powerful models of generative artificial intelligence. And yet, outgoing board member Helen Toner said explicitly that slowing down OpenAI was not the reason for the dismissal.

1:48I share that because you have to put where the authors are coming from in context to get the most possible from their piece. Now, another telling part of their perspective is in the next paragraph, which reads, It's encouraging to think that there are still people in Silicon Valley who worry about guardrails because digital technology has already done plenty of damage to jobs, wages, and democracy. For example, this sector brought us Facebook and social media, which have been used to fan the flames of hatred all around the world in the name of engagement and selling more digital ads. At the heart of their argument is this paragraph.

2:15Disruption and uncontrolled growth have become religion for the tech industry, and Altman has been one of its most dedicated high priests. Yet unsustainable growth rates and large losses are not supported by the logic of the traditional capitalist market system. Venture capital created this way of operating, but OpenAI doesn't need traditional VCs because it has Microsoft, which has already committed$10 billion to the company. Top Microsoft executives stayed focused on their goals during the Altman crisis. Hire the talent, promise them unlimited money to spend, and press the pedal to the metal.

2:42Worse, the speed imperative is boosted by the predominant vision in Silicon Valley, which cares little about social responsibility or what happens to people. They then go on to rail against Marc Andreessen and his techno-optimist manifesto, and basically argue that Andreessen doesn't care about any real issues. And overall, the piece concludes this way. In Washington, D.C., any whiff of regulation or sensible guardrails drives top tech executives apoplectic. The tech bros have embraced the full-fledged libertarian fantasy in which they are the indispensable men. In The Shape of Things to Come, published in the early 1930s, H.G.

3:11Wells imagined a dystopian near future, in which aerial bombing campaigns came close to destroying civilization. But after more than 20 years of disaster, Wells imagined a new global elite controlling aviation technology would emerge to impose world peace. Wells was right about the danger posed by the unbridled and unprincipled development of technology. But his work of science fiction about a dictatorship of the elite holding the keys to the future of the world is just as disturbing today as it was in the heyday of European fascism. Now, worth noting that these guys are MIT professors and authored a book called Power and Progress, our 1 ,000-year struggle over technology and prosperity.

3:43You can probably tell from my tone that I don't necessarily share the starting point that these guys do. I think I have a much less cynical belief about the intentions and compassion of many in Silicon Valley. I also don't disagree with the dismissal of all technology innovation over the last 20 years, simply being reduced to something that, quote, fans the flame of hatred around the world. However, I do think that this point of view is important for a couple of different reasons. One, this is an increasingly common point of view. Technology has absolutely lost narrative warfare when it comes to the public.

4:15It has been assaulted from all sides of the political aisle, Facebook as the big boogeyman because Russian bots beat Hillary Clinton in the 2016 election, and of course Silicon Valley censorship of conservative voices on the right. Between that and very real questions that people experience around whether the trade-offs that society has made between cheaper stuff and worse jobs, the reality is that rightly or wrongly, people are more inclined to be distrustful of technology and its leaders now than they were, for example, 20 years ago. This is a point of view, in other words, that can't just be dismissed or written away.

4:45What's more, even for one who is not inclined to this base set of views, they bring up an incredibly important question which is going to come into view ever more and which the Sam Altman OpenAI episode does put a fine point on. How much do we accept that a technology with as much potential to shape the texture of the world as artificial intelligence has is controlled by a very, very small number of people? When you ask that question, it doesn't necessarily mean a priori, that you are implying that a better answer is, for example, government control, even if that would be the preferred methodology of many of the people who write essays like this one.

5:24Instead, it's simply about the importance of asking the question, which, by the way, isn't a question that I think people like Sam Altman are really wanting to shy away from. Altman has talked about the idea of needing there to be some kind of democratic governance for these types of technologies. It's just that there's not an easy answer to how to actually make that real. And in the meantime, there are incredible market pressures to compete. Those are real and undeniable forces. And even if one comes out on the side of let the markets do their thing, I think it's right to at least reconcile with them and try to make an intentional opinion rather than just let these forces happen to us.

5:58So, an essay that in many ways I disagree with, but one with very important questions to chew on. Next, we turn to something that's a little bit more directly applied to OpenAI, a piece from Kaval Desai published in The Information called, To continue innovating, OpenAI should return to its non-profit roots. The subheader, many important tech breakthroughs originated in research organizations, but the innovations are most often commercialized elsewhere. Now in this piece, Cavall argues that ultimately it's not really important why exactly there was dysfunction at OpenAI. Instead, he says, we should look at the history of innovation to understand the relationship between research labs on the one hand and commercial bodies on the other.

6:36He writes, research organizations have developed many revolutionary technologies, but most often, these innovations have been commercialized and popularized by other for-profit companies. Consider, researchers at Bell Labs invented the transistor, but Fairchild Semiconductor commercialized it. IBM's TJ Watson lab pioneered work on dynamic random access memory, but Intel made the first commercial DRAM chip. Most relevantly, the transformer architecture was invented at Google and popularized by OpenAI via ChatGPT. There are many reasons why the businesses that nurture groundbreaking research have repeatedly failed to capitalize on those innovations commercially.

7:09They include misalignment of corporate culture, employee motivations and incentives, investors differing time horizons and return expectations, and differences in risk tolerance. A researcher motivated to do groundbreaking innovation and ensure it is both novel and safe may view a rush to commercialization as rash. Conversely, those who aspire to beat competitors to market with an innovative product don't want to wait to ensure every corner case is accounted for. Waiting to achieve perfection may allow others to reap the rewards, under its original non-profit Research Parent. This experiment to have the best of both worlds under one roof might have been ingenious, but clearly failed to resolve the misalignment between research and commercialization.

7:43The company should take heart that its predicament is not unique, rather a recurrence of a fundamental organizational conflict in tech innovations. In its post announcing Altman's firing, the previous board of OpenAI made clear that it is a non-profit with the responsibility to advance its mission of ensuring that artificial general intelligence benefits all of humanity. History suggests for that mission to have widespread impact, the company ought to reverse course, go back to its roots as a research organization, and leave commercialization to others. Hopefully, the new board will pursue that path.

8:11So a couple follow-up thoughts here. The first is not bloody likely, especially after this scare. The second is that I think the piece sort of underestimates the extent to which the attempt here is of a team and the folks led by Sam Altman who believe that this transformation is screaming down the pipeline and believe that the only way to do it safely is more or less not to leave it to someone else. Now, there are plenty of others, especially folks in the AI safety community, who have made the point that just because someone else is doing something extremely dangerous doesn't mean you should do it too.

8:42And there's also more than a fair chance that there's sort of an ego aggrandizement going along here that on some level thinks that they're the only people who can thread this difficult needle between mission and profit. But I think in many ways, it seems to me that this weird structure has proven a little bit more resilient than all of us first thought as this happened. At the end of the day, Altman is back, but in a less powerful version. He is not on the board. He doesn't control the board. And maybe he'll get there over time. But even Adam D 'Angelo, who was very antagonistic during this whole thing, it seems, was able to remain.

9:14The board might not have Helen Toner and a couple of those other people on it, who are most against Sam Altman, but its supremacy remains. And even after all this, Microsoft isn't getting a voting board seat. So maybe we give it a little bit more time before we write it off as a total failure. David Brooks of the New York Times leaned back on his time spent in the open AI offices with those teams to write a piece called The Fight for the Soul of AI. He asked this question centrally, can one organization or one person maintain the brain of a scientist, the drive of a capitalist, and the cautious heart of a regulatory agency?

9:46Or, as Charlie Warzel wrote in The Atlantic, will the money always win out? Brooks sums up some of the wildness of this industry when he writes, the people in AI seem to be experiencing radically different brain states all at once. I found it incredibly hard to write about AI because it is literally unknowable whether this technology is leading us to heaven or hell, and so my attitude about it shifts with my mood. Now, one thing that Brooks does note is that he expected OpenAI to be very different than it was. I didn't meet any tech bros there, or even people who had the kind of we-are-changing-the-world bravado I probably would have if I were pioneering this technology.

10:18I confess I have a history of going into these tech workplaces with a degree of defensive, humanistic snobbery. These people may know code I tell myself, but they probably don't know the literary and philosophical things that really matter. I was humbled at OpenAI. He then goes on to list a group of people who can only be described as polymaths, although hopefully they don't describe themselves that way in their LinkedIn and Twitter profiles. Brooks continues, as impressive as they all were, I remember telling myself this isn't going to last. I thought there was too much money floating around.

10:44These people may be earnest researchers, but whether they know it or not, they are still in a race to put out products, generate revenue, and be first. He also notes that even back then, before all of this, there were clearly deep divisions over safety. Brooks concludes, nobody really knows who is right. But the researchers just keep plowing ahead. Their behavior reminds me of something Alan Turing wrote in 1950. We can only see a short distance ahead, but we can see plenty there that needs to be done. I had hoped that OpenAI could navigate the tensions, though even then there were worries. As Brad Lightcap, OpenAI's chief operating officer, told me, the big thing is, is really just maintaining the culture and the mission orientation as we grow.

11:20The thing that actually keeps me up, if you're asking honestly, is how do you maintain that focus at scale? Those words were prescient.

11:39All right, so here is my non-consensus and probably unpopular take about all of this. The episode that we just experienced is wild. And even if we hadn't been through that, the one-year anniversary of ChatGPT would have provoked a lot of reflection. However, to the extent that there is a silver lining in all of this, I think that it has put front and center even more than were before some incredibly important questions that the industry, both AI specifically and tech more broadly, but much more significantly society needs to be asking itself. And I don't just mean in terms of policy. Policy ultimately should follow, not create, should reinforce, not require, in other words, the social contracts that governs how we interact with each other in the world at large.

12:23The world in which there is advanced generative AI to say nothing of artificial general intelligence is a world that is changed. It is one that requires new conversations about things like the social contract. And that is true whether you are a full accelerationist or someone who thinks we should stop now. My read, nascent though it may be, is that those conversations are getting a little bit more pointed, a little bit more sophisticated, and a little bit more specific in the wake of all of this. And so perhaps what seemed from the outside to be just a totally insane episode will have been valuable after all.

12:57But of course we can't know that now, so we're just going to have to wait and see. Anyways, friends, that is going to do it for today's AI Breakdown. Appreciate you listening or watching as always. And until next time, peace.