Dear AI Companies: Stop the “Doom Trolling” | AI Reality Check

0:00The last three years have been sort of exhausting for me. You know, as a computer scientist and a technology commentator, I was excited by ChatGPT when it was first released. I mean, the effectiveness of generative AI at both understanding and producing structured language was cool and unexpected. It was sort of like when you first saw that pinch-to-zoom feature on an early iPhone. So it seems self-evident to me that, hey, with enough experimentation, we would for sure find some impressive applications for large language models. And I was curious to learn what they would be. But then almost immediately, the discourse surrounding AI became cloaked in a mantle of dread and hype.

0:44A few months after ChatGBT's launch, for example, Yuval Harari, Tristan Harris, and Azaraskan published an alarming New York Times op-ed about what the arrival of this tool foretold. Here's what they wrote. AI's new mastery of language means it can now hack and manipulate the operating system of civilization. By gaining mastery of language, AI is seizing the master key to civilization from bank vaults to holy sepulchres. later in the article those authors predicted by 2028 the U.S. presidential race might no longer be run by humans and then perhaps most notably in their conclusion the authors declared we have summoned an alien intelligence now at the time I remember that that seemed out of proportion with how large language models actually worked so a few weeks later I responded with a long New Yorker piece that I wrote that was titled, What Kind of Mind Does Chat GPT Have?

1:40And this led to some media calming, some fear calming media interviews. I got to spend some time up on the hill explaining auto-aggression to some senators. But ultimately, it was too little too late because soon after this sort of initial pop of dread and fear around AI, the AI companies themselves embraced the strategy of trying to unnerve and scare their own customers. A month after my New Yorker article, for example, OpenAI CEO Sam Altman signed an open letter that argued, and I'm quoting here, the risk of extinction from AI is on scale with nuclear war. Anthropics Dario Amadei got more specific, claiming on multiple occasions that there was a 25 % chance that our AR future would go, quote, really, really badly.

2:31He was referring there to the end of the human race. The near future wouldn't be any picnic either. Amadei argued on multiple other occasions that 50 % of entry-level white-collar jobs would be automated in the next one to five years. Sam Altman agreed with his general sentiment and helped fund it out of his own pockets. Experiments in universal basic income because, as he explained multiple times, he was convinced that some sort of guaranteed income from the government was the only way that we would avoid having to eat our pets once AI took all of our jobs and cratered the economy. Last summer, while appearing on Theo Vaughn's podcast, Altman's voice cracked as he tried to reckon with the all-powerful technological demons he was summoning.

3:13He compared their work to the Manhattan Project. Earlier this spring, Anthropic released a report saying that their new language model Mythos was so good at finding cybersecurity vulnerabilities that they couldn't release it public. They gave really anxious briefings to reporters and world leaders about this new danger. This led Tom Friedman, who I think was at one of those briefings, to dutifully write a column that called Mythos terrifying. He said in his column, holy cow, super intelligent AI is arriving faster than anticipated. This is not a publicity stunt, Friedman insisted. Then six weeks later, Anthropic threw some basic guard rails on the model and released it anyway.

3:55So maybe it was a little bit of a publicity stunt. Then finally, earlier this month, Anthropic put out a new report that argued that the success of their cloud code software development agent increased the probability that AI may one day soon start improving itself until it gets so powerful that humans can no longer control it. The report featured a scary animation of robots multiplying like cells in a Petri dish and many brightly colored charts with somber captions. They concluded the report by saying that, well, there's nothing they could do about it because, you know, China. All right. All of this type of fear mongering has had a massive impact.

4:33Polling numbers show most Americans now distrust these technologies and think they're much more likely to cause harm than good. Essentially, anyone now who tries to keep up with AI news, even if they're just sort of casually reading some headlines, will find themselves living a life riven with anxiety and dread. And I'm not exaggerating about this. I want to read you an excerpt from an email that I received just last week from a software developer. Here's what he told me. I am tired. Every day I hear that me as a developer will be replaced. Every day I hear that self-improvement is right around the corner, that everything will be bad, wrong.

5:09I don't even know what to believe anymore. These people ruined my mental health and don't even know who I am. Just casually tossing around predictions after predictions after predictions. I'm afraid my son will be living in some dystopian world. Reading messages like this is heartbreaking. It feels like COVID-19 all over again. But is this all necessary? Why are companies like Anthropic and OpenAI simultaneously trying to terrify us about AI while sucking up investment dollars and working to advance the technology as fast as possible? Why do their CEOs cosplay as profits instead of trying to convince us about the usefulness of their products?

5:50Does any of this make moral sense? Do we have to put up with this? Well, it's Thursday, which means it's time for an AI reality check episode of this show, which is the perfect opportunity to go looking for some measured answers to these questions. As it turns out, last week, I channeled my frustrations here into a New York Times op-ed that called out these companies and declared their communication strategy, which I've taken the calling doom trolling, to be morally indefensible. So what I want to do today is I want to get into the details of my op-ed and then conclude with some concrete suggestions for those of you who are fed up with being relentlessly told by these AI companies that you have no choice but to shut up and take your abuse, how to push back.

6:36All right, so let's get into it. As always, I'm Cal Newport, and this is Deep Questions, the show for people seeking depth in a distracted world.

6:51All right, so I want to start by reading some passages from my article itself. I have it here for those who are watching. The title that we gave it was Dear AI Companies, The Doom Trolling Needs to Start. Okay, stop rather. Let's start with the introduction. Here's what I wrote in the very beginning of the piece. technology revolutions in the digital age are typically accompanied by optimism and excitement recall steve jobs basking in thunderous applause as he introduced the iphone in 2007 the major ai companies seem to be following a darker and weirder strategy they like to solemnly describe the harms that their models will cause while acting helpless to do anything about it i think this is a big deal This idea that this is new.

7:38We're used to it, but it doesn't mean that it's normal. Technology leaders used to say, let me tell you why this is cool and why you should be excited about it. Today, they're doing something so much darker and weirder. Okay, returning to my article, after I give some examples, including in particular that last anthropic report about recursively self-improvement among their tools, I said the following. like a cat leaving a dead bird at your doorstep. Anthropic catalogs the grim future that its products might produce, shrugs its shoulders, and then returns to its furious efforts to make these warnings a reality.

8:15I then elaborate. Let's call this strategy doom trolling. It's one of the defining and most arresting properties of our current AI moment, and I've come to believe that it is morally indefensible. again this is a big claim but i'm saying once we recognize the weirdness of this communication strategy where again like dropping a dead bird at the doorstep you know anthropic be like hey we might lose control of ai because of our tools carry on and then they just walk away and get back to trying to raise money for their ipo it's weird let's call it let's give it a name let's call it doom trolling and i say this is morally indefensible all right i want to go through here carefully my argument for why I think it's morally indefensible.

8:58So here's what I write. I say, there are really only two options for the intentions of AI companies when they engage in doom trolling. The first is that they actually believe that the systems they're building have a non-trivial chance of producing hugely disruptive events, from destroying the economy in the best case to wiping out our species in the worst. If this were true, every reasonable ethical system would argue that there is only one acceptable response to immediately stop working on any product that might accelerate such a future and lobby with all of your resources to help force other AI companies to do the same.

9:30From a moral perspective, any other reaction would be monstrous. All right? So if they actually do think the technology they're working on has even a non-trivial chance of causing massive societal or even existential harms, all ethical systems say, yeah, you've got to stop and do everything you can in your power to stop other companies from doing the same. Forget your IPO. Forget your stock holdings. Forget your money. This is the future of the human race. Of course, that's what you should be doing. But for you to say, yeah, this might lead to the extinction of humans, this trajectory we're on, a product development, but China, and then getting back to trying to do a trillion-dollar valuation, that's morally monstrous.

10:13All right, here's the second option, though. I say the second option is that these AI companies aren't really concerned about these risks and that they're injecting those doses of unresolvable doom for other reasons. They might want to amplify the perceived power of their technology at a time when they're setting up their initial public offerings or they hope their performative reports and somber interviews will help them compete for more engineering talent coming from a Silicon Valley culture that's steeped in this type of doomerism. DaVinci Capitalist and AI advisor David Sachs recently suggested that Anthropic was using fear-mongering tactics as a method of regulatory capture, which can impede upstart competitors.

10:50Any of these reasons would mean that these companies are laundering the anxiety of millions to improve the financial fortunes of a vanishingly small number of major stockholders. This cynicism would be equally monstrous, right? So the second option is like they don't really believe that building harnesses on top of fine-tuned language models is going to lead to the automation of our economy or the end of our species, but they still talk about it like it would. I say that would be just as bad because that means you're basically alchemizing anxiety into money for a small number of people who have early stock positions in your company.

11:24That's what it would come down to. That cynicism would be equally monstrous. What do I believe is true, by the way? I don't know what they think, but most East Coast computer scientists who aren't sort of infected by the quasi-religious eschatology of Silicon Valley would say, no, language models with harnesses built on top of them are not a trajectory that's going to exponential up to automation of all jobs and certainly not to super intelligence and the death of economy. This technology has its uses. They're more narrow than we think. That's why we're still stuck mainly doing highly structured language type things like coding, which required years of effort to get these custom neurosymbolic harnesses to make it useful.

12:08It's why it's not affecting the economy. It's why so many companies now who invested a bunch in AI outside of software development are now moving backwards. And even those who've invested in AI as software developers are now having to reduce the token budgets because they're not getting enough return on their investment. It's not magic technology. It's not the chip from Terminator 2 that allowed the cyborgs to come alive. That's what I feel about it. Now, whether the companies themselves actually maybe wrongly believe these harms are true or if they're being cynical, I don't know. But either option is morally indefensible.

12:42Okay, I want to scroll down. I get into a lot of details in the article, read the whole article for the rest of details about how we might respond societally from a government perspective, from a court's perspective. But I want to jump down to my conclusion here. I say, as a computer scientist and a digital ethicist, I'm both optimistic about the possibilities of AI and confounded by the terrifying and grim way that current technology leaders insist on talking about it. This could have been a period of hopeful innovation, but instead our emotions are being manipulated by Silicon Valley's self-serving and morally untenable addiction to doom trolling.

13:17This communication strategy has to stop. The harms it's causing to the public's mental health has arguably outweighed the benefit that AI has so far delivered. All right, so I try to be pretty clear here. You have to stop. treat your products like products. Explain to us their benefits. Make the pitch for why the cost is worth it. And of course, like any other consumer product company in the history of consumer product companies, they say, no, of course we're not developing anything that's going to cause wide-scale harm. We can, of course, generate very useful tools based off of large language models without having to be traversing a trajectory towards the extinction of the human race.

14:05we're fully liable for our products, and we're trying to build, this is going to help you write code. This is going to help you do your calendar. Just treat it like a normal product. Now, of course, that could be terrifying because if you look at the actual number of these companies, they don't justify anywhere near the giant valuations they want for the IPOs. Why do they want those giant valuations? Because they're in a measuring contest, so to speak, with Elon Musk. They're like, I want a big number two. show I'm a big man. But we might have to just keep terrifying people so that we seem so important that we can ride the hype waves.

14:41I really am fed up, and I really do think it has to stop. All right, so I put this article out there last week. You know, it hit a nerve. It was trending on X for a while, which I got to say, by the way, made me quite nervous. The trending headline was something like, Cal Newport accuses AI companies of something, something, something. And I got to say, in general, to see your name and the word accuses trending on Twitter. Historically speaking, it is not good. In this case, it was okay because it was me doing the accusations. That made me a little bit uncomfortable, but I'm glad that the article was getting some pickup.

15:13Steven Pinker retweeted it and demanded that AI companies, quote, stop the self-serving and fatalistic doom trolling, end quote. Ed Zitron called it, quote, the best thing I've read about AI in years, end quote. David Sachs, who I talked on the article, got involved. It became a whole thing. But, you know, it got out there. But here's the question that a lot of people have asked me since. All right, what should we as individuals do next? The article made a lot of calls for what the company should do. It made calls about what the court should do. It made calls about what the government should or shouldn't do.

15:43What should we as individuals do here? What's the call to action to the person who is fed up with being force-fed anxiety day after day by these AI companies? Well, here's the clearest way I can summarize my suggestion. stop playing along with the doom trolling game don't pay attention to any statements from ai companies that are stated in the future tense care about the products they offer now and whether those products are currently worth the time investment and money required to use them When you encounter scary news articles about AI, remember that most journalists don't have access to some sort of special information or powers or prognostication.

16:32They're just replaying or relaying the dark vibes that the AI company leaders are putting off. So you don't have to put special stock in the fact that a major publication is saying something scary about AI. It is just a direct reflection of the doom trolling that the AI companies themselves are doing. I want you to remember two things about the current leaders of these major frontier AI labs. First, they are weirder than you think. I think we all seriously underestimated the degree to which the quasi-religious ex-risk singularity culture in Silicon Valley has warped the way that many Silicon Valley leaders think and talk about human value technology in the future.

17:11This is why I think doom trolling got a early foothold, especially in the media, is that we sort of rationally assumed, oh, these are the people who know this technology best. So if they're concerned, we should be concerned too. Now a few years in, we're realizing that the person that we trusted maybe had a couple screws loose. It reminds me of the classic Simpsons episode where the car company gets convinced that Homer is a genius and they let him design his own car. And it's a terrible disaster and a flop. They realize like, oh, maybe our idea that this is the person who knows best about what someone wants in a car was wrong.

17:47It reminds me of this sometimes. Like these guys, these hardcore serious engineers know what they're talking about. And then we see them at their sort of metaphorical robes on as they're doing their demon summoning dance around the GPU on top of an altar. Like, uh-oh, maybe we shouldn't have been so quick to assume everything they're saying was true. The second thing I want you to keep in mind about the leaders of AI companies right now, this way they've been talking, this sort of doom-drenched rhetoric has been very helpful financially for them. It makes them seem all-powerful and therefore worthy of much more investment capital than the reality of their business model, which is that they are essentially a money-losing natural language version of Google, an OpenAI's case, and a software developer utility company in Anthropics case.

18:37That reality is not worth a trillion-dollar valuation. But the reality of we're going to invent the last tool ever that's going to run the whole economy, there's enough hype around that. You're like, I want to write a check. It puts them in that sort of meme stock territory that SpaceX benefited from. So there's a financial incentive for the way they talk. Keep that in mind. All right. I also want you to keep the following in mind as well. If you begin rejecting doom trolling and show like really any resistance to these gloomy messages, people in your life who are drowning in AI anxiety will begin to use you as a safety blanket.

19:13they will semi-antagonistically challenge you to explain whenever any new sort of stupid feature or benchmark chart is released why this doesn't mean that the Terminator robots are in fact coming and here's what I want you to remember once you're in this situation it's not your job to disprove the remarkable claims of the doomers, it's the doomers job to convince us that the remarkable claims are likely you do not have to play the safety blanket for people around you it's this weird asymmetric inversion we've had that instead of the companies having to say why they think we're going to have, you know, AI ruling the world, you have to somehow argue why a 10 % increase of Fable 5 on the Epic 6 benchmark doesn't mean that.

20:00It's a crazy inversion. It's not your job. It's the doomer's job. Remarkable claims require remarkable evidence. Finally, I want you to use the term doom trolling as much as you can. Words have power. My greatest hope for this piece is that by normalizing this new term, by getting it as a part of our lexicon, it will effectively undermine or ridicule the communication strategy of this sort of like, guys, I'm sorry to report, but our latest product, it's probably going to It's probably going to lead to data centers having to devour your pets for organic energy sources. That's probably going to happen.

20:41Anyways, our stock will be available next week. See you soon, right? If it kind of ridicules that, it makes it harder for them to do. I want it to be culturally almost impossible for Anthropic to put out one of these stupid white papers with animations and charts where they act all somber. Like, oh, we're so safety aware. and then doing no changes at all in their product offerings or research programs. So let's use that term. Hey, stop doom trolling and get back to explaining to me, what are you selling? How much are you making? Does it justify your valuation? Let's get back to the real product basics.

21:17All right, that's probably enough of me ranting about doom trolling. I'll be back on Monday with an advice episode of this podcast, which should be a nice break from all this doom and gloom. then next Thursday I may or may not have another AI reality check. They're going to be a little bit spottier this summer just because I'm traveling a lot. So some weeks I'll have them, some weeks I won't. But they're not going away, so never fear. And of course, you can always subscribe to my newsletter at calnewport.com where I send a dispatch from a distracted world every week. And that'll ensure you're getting at least your daily dose of me trying to understand our current technological world and how we can push back and thrive within it.

21:52And remember, until next time, care about AI, but not everything. you read about it.