

6 Oct 2026 · 1h 6m · 24 chapters

The episode argues that today’s “race to superintelligence” is driven less by shared ideals than by rival, hidden motivations among three leaders—Sam Altman (OpenAI), Dario Amodei (Anthropic), and Demis Hassabis (DeepMind/Alphabet)—and by a compute-scaling strategy that accelerated after 2017.
Kevin Roos is a journalist and author of The AGI Chronicles, reporting on the inside story of AGI development and company rivalries.
Superintelligence is AI vastly smarter than humans at everything; progress is rapid because capabilities scale with compute spending. Insiders at frontier labs are portrayed as increasingly terrified of an “intelligence explosion.” Rivalries and safety/commerce tradeoffs push all sides toward “speed, speed, speed.”
2017 GPT-1 work at OpenAI; Dario Amodei’s “Big Blob of Compute” memo predicting scaling laws; AlphaGo’s undefeated run by DeepMind (Demis Hassabis). OpenAI’s GPT-2 demo to Microsoft CEO Satya Nadella that secured $1B; ChatGPT’s accidental November 2022 launch after a Claude rumor. Amodei’s atomic-bomb analogy (Leo Szilard) and Anthropic’s shift from safety-only research to commercial Claude, including funding pressures after Sam Bankman-Fried.
Written by AI. May contain mistakes.
Understanding what superintelligence means and its implications.
“one month of Security Edge Preferred on them.”
Discuss the advancements in AI over the past decade and recent developments.
“superintelligence you could think of as an AI system that is smarter than everyone at everything.”
Explore the personalities and motivations of key figures in AI.
“and then the sort of, you know, the non-public thing that I've seen is that the people at the companies who are building this stuff, I guess this is not a secret anymore, are terrified of it.”
Delve into Sam Altman's approach to leadership and AI safety concerns.
“peculiar dudes, respectively, Sam Altman, Dario Amadei, and Demis Hissabis.”
Learn about the high-stakes demo that secured funding for OpenAI.
“and you can just do things that change the world without permission.”
Learn about the pivotal meeting between OpenAI and Microsoft that led to a billion-dollar investment.
“And they've got this very high pressure meeting with Satya Nadella and his leadership team at Microsoft.”
Explore the challenges Microsoft faced with AI chatbots and how it influenced their investment in AI.
“that was exactly what they wanted to see.”
Discover the surprising launch dynamics of ChatGPT and the misconceptions within OpenAI.
“And whether you're a listener who absolutely despises AI or loves AI, like the release of ChatGPT absolutely changed business, technological, and economic history.”
Understand how a false rumor about a rival's chatbot led OpenAI to expedite ChatGPT's release.
“many of them didn't think this thing would be very popular.”
Delve into the dramatic events surrounding Sam Altman's firing and reinstatement at OpenAI.
“that forced them to take their GPT technology and turn it into a chatbot when they had been thinking about an entirely new or different product or business model?”
Learn about Dario Amadei's motivations and views shaped by historical lessons from nuclear physics.
“But essentially, Sam engineers his own comeback with the help of OpenAI's employee base.”
Discover why Dario Amadei resonates with Leo Szilard's story and its implications for AI ethics.
“And if you've read that book, and I highly encourage people to read that book.”
Explore how Dario Amadei parallels Leo Szilard's moral imperative in AI development.
“that the only moral thing to do now that the secret is out is to build it before the Nazis do.”
Discuss Dario Amadei's fears around AI misuse and the competitive climate at OpenAI.
“Lincoln Rejuvenate does not operate unless the vehicle is in park when the vehicle is off or if any door is open.”
Learn about the reasons behind a key figure's departure from OpenAI.
“So he had been thinking about this for years.”
Explore how Anthropic transitioned from safety-focused to commercially successful.
“Yeah, it's the central question of Anthropic.”
Discuss the challenges Anthropic faced in funding and strategic directions.
“that he sort of concocted this rationale of the, he calls it the race to the top.”
Understand the contrasting philosophies of AI leaders Sam Altman and Dario Amadei.
“who I think owns 13 % of the company's equity at one point.”
Learn about Demis Hassabis and DeepMind's unique approach to AI.
“They both have to release chatbots that make tens of billions of dollars a year.”
Explore the differences between game-based AI learning and language model training.
“It has way more material on him than I was able to fit in my book.”
Explore the contrasting philosophical perspectives on what constitutes superintelligence.
“Like philosophically, I was just thinking as you were saying that.”
Understand the rivalries and differing legacies of key figures in AI, including Sam Altman and Demis Hassabis.
“So Sam Altman and Daria Amadei hate each other.”
Delve into the political maneuvers and strategies of AI leaders and their implications.
“started this whole infrastructure cycle and has continued, has been very successful at using other people's ideas to build the power, his power and the power of his company.”
Learn about Elon Musk's pervasive influence in the AI narrative and his dealings with key players.
“I mean, you can go back and look at these papers he was writing in 2016 and 2017.”

Get the gist, explore an idea or ask about something said.
On Plain English with Derek Thompson, what are the key points of this episode?
Automatic transcript. May contain errors.
0:06Today, understanding the people who are building superintelligence.
0:10Derek Thompson: I want you to think back to 2017. In some ways, it probably just feels like yesterday. Nine years ago, your neighborhood probably looked exactly the same. There were cars, coffee shops, phone in your pocket. But in the history of artificial intelligence, 2017 is the distant past. ChatGPT doesn't exist. Anthropic doesn't exist. The mega project of AI construction that's currently driving American economic growth, it was on nobody's horizon. But the following things all happened in 2017. Number one, OpenAI, the nonprofit co-founded by Sam Altman and Elon Musk, started work on GPT-1. Five years later, this technology would serve as the guts of ChatGPT.
0:54Also in 2017, a physicist at OpenAI named Dario Amadei wrote a secret 26-page internal memo entitled Big Blob of Compute in which he made the argument that AI research should resemble a massive infrastructure project where bigger models with more data using more chips would be the path to superintelligence. Nine years later, this thesis has proved eerily accurate as the entire AI build-out has been motivated by the idea that AI's capabilities scale with compute spending. And number three, also in 2017, AlphaGo, an AI model, went undefeated against the world's top-ranked Go player. The model came from the company Google DeepMind, run by Demis Hassabis.
1:41Three years later, that company would use similar technology to solve the famous protein folding problem, an achievement that many scientists considered impossible. For this, Hassabis would win the Nobel Prize. Nine years later, it is not a stretch to argue that the entire frontier of AI has been shaped by these three men. Sam Altman, the CEO of OpenAI, Dario Amadei, now the CEO of Anthropic, and Demis Hassabis, now chief scientist at Alphabet. But who are these men? Who are Sam, Dario, and Demis? I'll sometimes read or hear coverage of this space that makes it sound like the men building advanced AI are all the same.
2:24They all want the same things. They are powered by the same greedy motivation. But these men are not the same. And today's episode is about the hidden motivations and rivalries among these three men that have arguably brought us to the precipice of superintelligence. What books and ideas inspired them when they were young? What do they fear? What do they really want? My guest is Kevin Roos, the author of a new book, The AGI Chronicles, The Inside Story of the Race to Create an Artificial Superintelligence. This is a story of bitter rivalries and secret documents, and a technology moving faster than the people building it can understand, let alone control.
3:08Derek Thompson: I'm Derek Thompson. This is Plain English.
3:40This episode is brought to you by JPMorgan Chase. In today's unpredictable world, geopolitics isn't just a backdrop. It's shaping markets, business strategy, and risk. The JPMorgan Chase Center for Geopolitics brings together former senior policymakers and seasoned practitioners to translate fast-moving global events into clear, decision-ready insights. From conflict and great power competition to energy and supply chain disruption, the center helps clients anticipate what comes next. Read the latest insights at jpmorganchase.com slash center for geopolitics.
4:41one month of Security Edge Preferred on them. Plus, ask about a$500 prepaid card. Limited time offer, restriction supply, requires 300 megabits per second internet, Security Edge, Security Edge Preferred, and Wireless Connect. Call today. Kevin Roos, welcome back to the show.
4:57Kevin Roose: Thank you for having me back. Always a pleasure.
4:59Derek Thompson: Kevin, what is superintelligence and why do you think we're on the precipice of building it?
5:06Kevin Roose: superintelligence you could think of as an AI system that is smarter than everyone at everything. There are sort of these gradations of AI capabilities. People talk about AGI, artificial general intelligence, as a computer program that is human level or better at most tasks or most intellectual tasks. Superintelligence would be the level beyond that where we have something that is vastly smarter than us at everything that we could conceivably do.
5:34Derek Thompson: Your new book is called The AGI Chronicles, The Inside Story of the Race to Create an Artificial Super Intelligence. It seems from your reporting that you think this is a race that is on the precipice of achieving something quite dramatic in the near future. What have you seen, let's just say in the last 12 months, that have made you think we're really making progress toward this goal as you've defined it?
5:58Kevin Roose: Well, let's back up more than 12 months to 10 years ago when the book starts, which is we had these AI systems. They could do things like identifying objects in images or translating between languages or helping you with autocorrect on your email. But those were narrow systems. They could only do one thing. They couldn't be repurposed to do 20 different tasks. Now we have these general models, Claude, ChatGPT, Gemini, that can do a wide range of things near or above human level. We have systems that are capable of solving Millennium Prize problems in math, of discovering new protein structures, of helping people, you know, billions of people around the world every day in various ways.
6:47Kevin Roose: So I think that's what I've seen is just the continued march of this technology. and then the sort of, you know, the non-public thing that I've seen is that the people at the companies who are building this stuff, I guess this is not a secret anymore, are terrified of it. They have been for years and they're not getting any calmer as we approach what they think could be some kind of intelligence explosion, which we could talk about. That's sort of another concept that they think is not far off and I'm inclined to believe them.
7:23Derek Thompson: One thing I think is so strange about this entire project is how small this world is. And when we talk about the Frontier Labs, we were talking about three companies leading the race to build superintelligence. They are OpenAI, Anthropic, and Google DeepMind. Those companies are run by or have until very recently been run by three particular and peculiar dudes, respectively, Sam Altman, Dario Amadei, and Demis Hissabis. I want to spend the bulk of our time trying to understand these men, who they are, what motivates them, their strengths, their weaknesses. And let's just jump right into it. Sam Altman, Paul Graham, the former head of Y Combinator, once famously wrote this about Sam Altman.
8:04Derek Thompson: Quote, you could parachute him into an island full of cannibals and come back in five years and he'd be the king. End quote. What makes Sam Altman so good at becoming powerful?
8:16Kevin Roose: He has this superpower, which everyone he's ever worked with, all of his mentors, they have recognized this in him. And he had it from quite a young age of figuring out exactly what someone wants and giving it to them, or at least appearing to give it to them. So back in his days running Y Combinator, people would sort of talk about this power of his, and they didn't know where it came from, but people like Paul Graham recognized it very early. He was very good at reading the room, at figuring out, you know, this person expects me to do this or wants me to do this, so I will do that. And it led to him becoming one of the most successful young entrepreneurs and investors in the Valley.
9:02Kevin Roose: The flip side of it is that he would often get into trouble because he would tell one person one thing that they wanted to hear and then tell a different person something completely different that they wanted to hear. And he sort of got this reputation as a sort of slippery people pleaser, which is, I think, a criticism that many people who have worked with him over the years have identified and have made in pretty consistent ways. He just seems to be in the business of both being quite ambitious, but also of being quite eager to please.
9:34Derek Thompson: I mean, one way to say this is that he is a brilliant dealmaker in the mold of a J.P. Morgan. Another way to say it is that he's a bullshitter. And I wonder, after all of your reporting, whether you feel like underneath all of that deal-making, there is a core philosophy. Like, just to put my thumb on one distinction that might help you answer that question, there's a 2015 blog post that Sam Altman wrote where he called machine intelligence, quote, something we should be afraid of. And when he founds OpenAI with Elon Musk, he founds it as a nonprofit because he thinks that this is a project that should not be a typical commercial business.
10:17Derek Thompson: But now he's running a business that is by some accounts valued at 800, 900 billion, a trillion dollars. And later colleagues who work with him are very skeptical of the degree to which he's serious about AI safety at all. So one way to ask this question is to say, you know, does he believe in AI safety? But what I'm really asking here is, does he believe in anything outside of just making the deal that increases his power?
10:43Kevin Roose: I believe he has a core. I don't know what it is. I'm not in his inner circle of people that he confides in. I've spent time with him. I've talked to him. I've had encounters with him as a journalist. but I've always sort of understood that he was playing the part in those interactions and wasn't necessarily showing me his true self. And for what it's worth, I think most executives do this. They have kind of their persona that they put on. I think the thing that kept coming up in reporting on Sam and the people around him is that he is a person who believes first and foremost in agency, He has this famous phrase, you can just do things.
11:26Kevin Roose: And this is something that he believed at Y Combinator. He would tell these young startup founders that he was investing in, like, you don't have to wait to like change the world. You don't have to like, nobody has to give you permission. You can just go do things. And that is a core part of his belief about himself and about those around him. Like, another way of saying it is like, there's no grownups, like you are the grownups. and you can just do things that change the world without permission. He's also a fervent believer in scale, the power of scale. You know, people would tell me these stories about, you know, he would walk into a meeting and he would just tell everyone, you know, tell the people in the meeting to like add a zero or two zeros to whatever budget they had for their project going forward or, you know, increase the scale of a training run just because like he thought that bigger was better.
12:17Kevin Roose: So I think that love of scale is probably part of his experience running Y Combinator and being an investor before OpenAI. But certainly OpenAI's success has confirmed that just having this kind of delusional self-belief and belief in the power of scaling is actually what you need to win in AI. because this is something that didn't really occur to me until midway through the book. Like belief is an active ingredient in AI success. You have to believe that the scaling laws will continue, that putting billions or trillions of dollars into building data centers and acquiring compute will pay off.
12:58Kevin Roose: Like you have to, like the belief is the enabling force that allows you to win. And if you don't have it, no amount of technical expertise can save you.
13:09Derek Thompson: Your book has several examples of this delusional self-belief that Sam Altman demonstrates. And one example is as OpenAI is scaling their GPT technology, and they've just come out with GPT-2, but they realize that they're going to lose the money spigot from Elon Musk. And so they need to get a billion dollars from someone. And a billion dollars at that point in the AI super cycle is a lot of money. And so they decide to put on a little demo for Microsoft CEO Satya Nadella with GPT-2 to wrest a billion dollars from Microsoft. Can you tell this story and what this story represents about Sam and the company?
13:50Kevin Roose: Yeah, it's an incredible story. And it hadn't been reported before. And I was sort of delighted to find it out because it does really illustrate who these people are. So they go up to Redmond for the day from San Francisco, Sam and a few other open AI executives. And they've got this very high pressure meeting with Satya Nadella and his leadership team at Microsoft. And they've got this new model, GPT-2, that is still, it's not good by today's standards. Like you would never use it for anything that you needed done, but it can sort of complete a sentence. It can sort of generate coherent text and it can do all of this like more flexibly than any of the chatbots that had come before it.
14:33Kevin Roose: So they go up, they run this demo, they bring a Microsoft laptop borrowed from one of the OpenAI executives' girlfriends because they're like, Microsoft will, you know, will think more favorably of us if we're using Windows. So they borrow a Windows laptop. They go up, they run the demo, and they're showing some sort of canned prompts that they've prepared that they kind of know GPT-2 will be able to do. And then Satya Nadella asks, you know, can we see this thing? And another Microsoft executive sort of wants to try their own prompt that wasn't part of the canned demos. And so they're sort of looking around each other.
15:09Kevin Roose: They're like, can we do this? Can we do this? Yeah, I guess we have to do this. So they put in a prompt that's like, you know, Satya Nadella was hired as a CEO of Microsoft. And the first thing he did was blank. And then they hit enter and they're sort of like sweating as they wait for this thing to come back. And then it spits out the completion, which is that on his first day, Satya Nadella fired all the employees and pivoted the company to Azure, their cloud platform. And this draws a big laugh. And it's sort of an amazing moment in time. Like an AI model has never been able to do this before.
15:40Kevin Roose: And it was very high stakes. Like had this sentence completion not worked, they might not have gotten the billion dollars from Microsoft. But as it turned out, that was exactly what they wanted to see. And they invested a billion dollars and the rest is history.
15:55Derek Thompson: It's an amazing story. And one other piece that I think is really important is that Bill Gates at the time was quite biased against AI chatbots because Microsoft had had a terrible experience with a chatbot named Tay or something. Do you remember Tay?
16:07Kevin Roose: It was like a glorious - You're the chatbot nerd here.
16:10Derek Thompson: You've had all of the deep, intimate experiences with chatbots. I have a thing for deranged chatbots, Derek.
16:16Kevin Roose: And Tay was one of the more deranged ones we've had. It was this like experimental chatbot that Microsoft built pre-LLM. And they put it on like Twitter and people just started like messing with it and like turned it into like essentially a white supremacist within like 12 hours. It was like spewing like Nazi propaganda. And so the lesson that Bill Gates and other Microsoft executives learn from that is like, don't make chatbots. That's like, that's a bad idea. Yeah, and I think that's a really important ingredient in what makes a story so dramatic
16:48Derek Thompson: because Microsoft is not beginning from like a neutral place of maybe chatbots are good, maybe chatbots are bad, we have no experience. Like, they're like, we've done chatbots, they're terrible. Who knows, maybe if we, you know, invest a billion dollars into this company, our chatbot might, you know, engage with a New York Times reporter and, you know, proposition him for marriage. You know, thank God something like that would never happen to them. Yeah, right. That would never happen. I'm a huge sucker for like understanding the internal experience of like hinge points in business history. And whether you're a listener who absolutely despises AI or loves AI, like the release of ChatGPT absolutely changed business, technological, and economic history.
17:29It was so interesting to read your reporting on just what an effing nothing burger the folks at OpenAI thought ChatGPT would be. At one point, they're taking bets on how many people are going to sign up for this chat bot. Most people think 10 ,000. The highest bet at the company is Sam Altman saying no more than 1 million. I mean, we're talking about a technology now that has a billion, a billion users around the world or more. Can you flesh out the details of why OpenAI launched ChatGPT? Why, ironically, it started with a rumor about a rival company launching their own chatbot.
18:11Derek Thompson: And just what happened behind the scenes that gave birth to ChatGPT itself in November of 2022?
18:18Kevin Roose: Yeah, it's an incredible story because it truly was an accidental hit. The people at OpenAI, as you said, many of them didn't think this thing would be very popular. And I think to explain why, it's like we had to sort of step back in the time machine to like the sort of summer and fall of 2022. At the time, GPT-4 was just sort of finishing being trained. This was the most capable model. It would not be released until the spring of 2023, but OpenAI employees already had seen versions of it and we're playing around with it. And they were all racing to get this thing ready for a launch in early 2023.
18:59Kevin Roose: And then in November, 2022, so a couple months before the planned launch of GPT-4, this OpenAI co-founder, John Shulman, hears this rumor at a party where he hears that Anthropic, which is a splinter company that's sort of formed out of a group of dissatisfied OpenAI employees is going to release Claude, their chatbot. Now, the rumor is not true. Like they were, they had no plans to release Claude as a commercial chatbot. They were sort of testing it in a private Slack channel with like friends and family. But this gets sort of passed back to Sam Altman and the folks at OpenAI. And they're like, shit, like we have, we have this plan to release GPT-4 in a few months, but we think that this chatbot Claude might be really good and might kind of upstage us.
19:51Kevin Roose: And we don't want that to happen. So John Schulman proposes this sort of rushed launch of a thing that was originally called Chat with GPT 3.5, which was basically to take their old model that had already been out for like six months for developers and put kind of a free, easy to use chatbot interface on it and just throw open the doors as what they called a low-key research preview. Basically, it was not their best model. internally. And I think that's why a lot of people at OpenAI had low expectations because they're like, this thing sucks. It's like six months old. We already have this better model that has finished training that we've been using ourselves.
20:31Kevin Roose: That's a lot better. Why would we like pump the brakes, throw a new skin on this old model and like put that out as if it's supposed to be impressive. But they did it anyway. They decided to make it this low key research preview. They had low expectations, and then it just caught fire. They had badly misjudged how impressive the things that they were using internally would look to the outside world.
20:57Derek Thompson: Really, really quick question. Do you think OpenAI would have inevitably released a chatbot with GBT 4.0, GBT 5.0 eventually? Or was there something about that dynamic of believing that Claude would become public that forced them to take their GPT technology and turn it into a chatbot when they had been thinking about an entirely new or different product or business model?
21:21Kevin Roose: No, they were fully planning to release GPT-4 as a chatbot. And actually they were planning with Microsoft, which had this exclusive partnership with OpenAI, had plans to build it into their products and make chatbots as well. So it was going to happen in the spring of 2023. but the launch of ChatGPT, you know, brought that up in the timeline to the fall of 2022.
21:48Derek Thompson: Good, it's good to know that. Okay, last question about Sam Altman. Tell us why he was fired and how his return of the company changed the culture of the company. And because we don't have time for you to explain all of the reasons why he was fired, why don't you just shade your answer toward the most salacious of those reasons?
22:06Kevin Roose: You want the dirt? Well, he was fired because because the nonprofit board that governed OpenAI had been configured in a way that gave the more sort of safety conscious members of the board a temporary majority. And these four members of the board were also getting worried about how honest Sam was. This is, again, a pattern that has been observed by people close to Sam for many years. In some ways, it was exactly the reason that Dario Amadei and the Anthropic co-founders had quit in late 2020. They just sort of saw Sam lying and observed that he would say things to different people that were contradictory.
22:54Kevin Roose: And so Ilya Satskiver, who was an OpenAI co-founder, former chief scientist, also on the board, starts sort of assembling this rap sheet on Sam Altman and Greg Brockman, the president of OpenAI, who was also on the board at the time. And he and Mira Maradi, who was then OpenAI's chief technology officer, start kind of secretly compiling this dossier about all of the times that Sam and Greg would, you know, would say something to one person and a different thing to another person or would undermine executives or pit them against each other. They're just kind of bad management practices. But displaying that this was a consistent pattern, they send it to a member of the nonprofit board, and they basically plot this coup that they're expecting will be greeted with a hero's welcome.
23:47Kevin Roose: They're like, oh, if we're seeing this stuff about Sam, then everyone else must be seeing it too. And we will be greeted as liberators, essentially. And they carry out this thing on a random Friday in November, and they just are sort of amazed and bowled over by how much that does not come to pass. The employees rally behind Sam and Greg. They demand their reinstatement. They threaten to quit and go to Microsoft. And there's a lot of good, gory detail in the book about how that all went down. But essentially, Sam engineers his own comeback with the help of OpenAI's employee base. And five days after he's fired, he's reinstated.
24:29Kevin Roose: People have heard this story a zillion times, but what's new and what I was interested in are the sort of dynamics of this employee-led resistance and how and why they decided to band together to sort of restore the position of this guy that I think a lot of them had misgivings about, even at the sort of rank and file level. So it really is like one of the most fascinating five day periods in business history, I think. And so I devoted two chapters to those five days because I was just like, it's not going to get any juicier than this.
25:04Derek Thompson: Yeah, a lot of chicken nuggets ordered, a lot of Boba T's consumed. And you even have stuff about how like the the image of the chicken nugget and Boba T became like the hammer and sickle emblem of like open AI's counter revolution. It's like the war medals of people who survived this period. It's a really, really colorful and strange story. That closes this sort of mini chapter on Sam Altman, who I think it's important to say emerges from this dealmaking tradition at Y Combinator, is a dealmaker. And that is both his strength and his weakness. It's his strength in terms of being able to secure money, secure compute, which they are lapping the field in.
25:41Derek Thompson: That is the amount of chips that they have to train and run inference on their AI. But that's also his weakness in a way. It was almost his downfall because he's so much of a dealmaker without a clear moral or philosophical center that people feel sometimes that it doesn't exist and therefore must trust him for it. And in a way, that's very, very different from our second character, Dario Amadei. Dario now the CEO of Anthropic, but up until, I mean, in the first few years of OpenAI, he was their head of research. Before we get into his own personal history, I want to talk about his motivations.
26:17Derek Thompson: You report that one of Dario Amadei's favorite books is Richard Rhodes' Pulitzer Prize-winning history, The Making of the Atomic Bomb. And if you've read that book, and I highly encourage people to read that book. Great book. Or really just read the prologue. The hero of both the prologue and the entire book is Leo Seilard. Leo Seilard. I don't know how to exactly pronounce the Hungarian name. I think it's Szilard, but it's fine. Szilard? Okay. It's S-Z-I-L-A-R-D. Let's call him Leo. Who is Leo? And what does it tell us about Amadei that he sees himself in Szilard's mode?
26:51Kevin Roose: So Leo Szilard was a Hungarian physicist. And he has this discovery, this breakthrough. He's the person who came up with the nuclear chain reaction, the thing that paved the way for the splitting of the atom and eventually to the atomic bomb. And he has this realization, there's this very dramatic scene where he sort of has this realization when he's waiting to cross a street in Richard Rhoades' book. And then once he has this realization and proves it out, he tries to keep it a secret. He goes around to all of, he sends telegrams to all the scientific labs, all the nuclear, you know, sort of ish labs in Europe, begging them not to look into this.
27:32Kevin Roose: or not to publish their findings because he is terrified that the Nazis will discover this secret and use it to build bombs and use it to win the war. And he tries to classify it as a secret with the British Admiralty. He tries to patent it to sort of keep it out of the public view. And he fails at that. The secret gets out. And after the secret gets out, he has this moment where he realizes that the only moral thing to do now that the secret is out is to build it before the Nazis do. So there's this sort of amazing sequence in Richard Rose's book where he basically petitions the president with the help of his friend Albert Einstein to start the American nuclear effort that becomes the Manhattan Project.
28:22Kevin Roose: He then signs up for the Manhattan Project. And basically his theory is if we don't build it before the Nazis do, the world is going to end. or we're all going to be living under Nazi rule. And Dario Amadei not only read this book, it's his favorite book. He bought copies for early Anthropic employees. Like on your first day at Anthropic for years, you would get handed your onboarding packet and a copy of the making of the atomic bomb. Which is like 1300 pages, by the way. So this is not light reading. Not light reading. And he gave out a Szilard Award at OpenAI to like people who would make sacrifices on behalf of safety.
29:01Kevin Roose: So like really idolizes this guy and has patterned his career after Szilard's. And, you know, the obvious parallel here is that Dario Amadei discovered a secret. He was the essentially discoverer of what became known as the scaling laws, this method of improving AI performance by just throwing more compute and data and parameters at a model. And he tries to keep it a secret. He like, you know, really, really tries hard not to let anyone else know about this discovery. And then once it gets out, he decides that he has to build AGI or super intelligence first before, you know, not the Nazis, but, you know, China or Sam Altman or someone else builds it before him.
29:46Kevin Roose: So this is the sort of the meta narrative of his career is that he sees himself as essentially a modern day Leo Szilard working to build a powerful technology in a safe way so that others don't build it in an unsafe way first.
30:01Derek Thompson: Yeah, I don't think you have to necessarily accept the entire argument for Anthropics existence to at least appreciate the sort of internal logic of the sentiment. I'm building something that could be the most dangerous thing in the world, but I have to build it as fast as possible because it's important for us to have the thing first. And like that internal logic does not make sense for most products, but it made sense for Einstein and Szilard with the atomic bomb. It made sense to Oppenheimer to a certain extent with the atomic bomb and everyone at Los Alamos. And it makes sense to people working at Anthropic and to a certain extent also OpenAI.
30:34Derek Thompson: That's just a part of the weird self-justification that motivates a lot of activity in the Bay Area that I think in many ways seems completely alien and sometimes downright immoral to people outside the Bay Area. But I think that Szilard is an important skeleton key in terms of understanding at least how these people think of their own project. You can think they're wrong, but it's still an interesting skeleton key to understand their motivations. Amadeh loves to write essays. He's written a bunch of essays. Anthropic circulates all the essays for all the journalists. One essay that I had not read in a while is one you write about in your book, his 2017 26-page internal document entitled Big Blob of Compute.
31:17Derek Thompson: You were referencing this in your previous answer. And he argues essentially that AI researchers overthink things. They try to sort of finely tune all their models. But at the end of the day, progress comes from this simple recipe of make the models bigger, get more data, get more compute, and scale, scale, scale. Can you explain just how important this paper has been in terms of pointing to the future of and almost incepting the future of the AI super cycle. Because reminding myself of the paper in your book, I was like, it's a little bit Babe Ruth pointing to the left field stands and then hitting the whole run.
32:01He kind of just like says, we're going to spend trillions of dollars on this thing. And then that's basically exactly what's happened. So tell me, big blob of compute, what was Amadei's thesis in 2017, nine years ago? And how has it come to pass?
32:17Kevin Roose: Well, I hope you have not read Big Blob of Compute yet, because I think this is the first time it's been published anywhere. This is not a document that Amadei has published or even really talked much about publicly. So this was something, an internal research memo that I obtained that I'm going to publish the whole thing on my Substack this week, I think, because I just think it's an important historical document for exactly the reason that you just said was like, he points to left field, he says, this is how it's going to go. and then it essentially goes exactly like he predicted. So this was a paper that predates the LLM.
32:51Kevin Roose: GPT-1 had not been trained yet, but it is essentially the scaling hypothesis in miniature. He says, essentially, you're all doing this wrong. By you all, it means like other AI researchers. He says, you're building these small, narrow, single purpose models to do discrete tasks. You're building language classifiers. You're building image recognition systems. You're building speech recognition systems. And you're training them at a much smaller scale than would be optimal. You're not putting enough compute or enough data into them. And you are engineering these kind of narrow solutions to your problems.
33:32Kevin Roose: When the real way to create better AI models is to just make them bigger. Like, this is a principle from nature. Animals with bigger brains, all things else being equal, are smarter than animals with small brains. So if you want to improve the generality and the performance of your systems, just create this massive unstructured computational capacity, this big blob of compute. And he talks about these seven sort of variables, these knobs that you can turn, like how much data you put in and what goal you give the system. and then you essentially let the intelligence fall out of the model rather than trying to like code it in by hand.
34:12Kevin Roose: This was a, I think this is the most important document maybe ever in AI research because it, it, it came at a time where open AI was really struggling. They were trying to do these sort of narrow things. They had a robotics team that was trying to build a robot hand that could solve a Rubik's cube. They were trying to teach AIs to play video games, But none of it was generalizing very well. Like they would have one success in one little area, and then they would like, it would just wouldn't matter. And big blob of compute really steered them toward language models because language models were a thing that you could put a big blob of compute behind that would get better the bigger you made the blob.
34:56Kevin Roose: And that was the key insight that led them to sort of bet, to go all in on scaling up language models. And it's why we have the trillion dollar build out that we have today, because they realized and they proved that the bigger your blob of compute was, the better your model was.
35:16Derek Thompson: And there's another paper published in the exact same year that it essentially hooks up with, right? Attention is all you need because you get both big blob of compute saying we need to secure as much possible computing power as possible. combined with Attention is All You Need, a paper that comes out of Google that essentially says the transformer could be an incredibly useful, generalizable model to do this thing that all these smaller models can't do. And essentially when those streams collide, you get this explosion of this modern super cycle of AI building. Is that fair to say that like these two 2017 papers sort of collided in hyperspace and created like the modern AI moment?
35:55Kevin Roose: Absolutely. I mean, it wasn't enough to just have the big blob of compute. You also needed an architecture that could accept a big blob of compute. You know, there were all kinds of language models before 2017, but they didn't work that well, in part because they just like, you couldn't scale them in the same way that you could scale this new kind of language model that Google had developed, the Transformer, which was basically a way to radically increase the amount of compute and training that you could do on a large set of text. So you needed both. You needed the piece from OpenAI was the belief in scaling and the big blob of compute and the piece from Google, which they didn't even recognize at the time because they didn't have the belief in scaling.
36:44Kevin Roose: They just thought, oh, this is a cool thing to a cool way to make our language models better at like translating and, you know, writing emails. It's not going to like they didn't think this would be a foundational technology, which is why they released this paper publicly. So they they sort of dropped this thing in open AI's lap, the transformer. Dario sort of sees this and runs with it, along with some other researchers at open AI at the time and outcomes the AI revolution. This episode is brought to you by ServiceNow. Look, I have my dream job. I get to explain complicated ideas to folks who have better things to do than read white papers.
37:21But even dream jobs have not-so-dreamy parts, the stuff that gets in the way of the actual work. That's where ServiceNow's AI specialists come in. They don't just tell you what you should do about your busy work. They actually do it, start to finish, cases closed, requests handled, no extra work for you. That way you and your team can spend more time on what matters, which for me is finding that one elusive stat that just makes everything click. To learn how to put AI to work for people, visit servicenow.com. This episode is brought to you by Jira by Atlassian. Your work is scattered. A thread here, a DM there, five different docs related to the same project.
38:04And now it's not just your team hunting for context. It's your AI agents too. Jira taps Atlassian's teamwork graph to bring that context together across Jira, Confluence, Slack, GitHub, and more. It makes it easy to share with the agents you already use, like Claude, Cursor, and GitHub Copilot. Assign them work directly or connect your agents through MCP. The result, less token waste, more accuracy, and more time moving work forward. Learn more at jira.com. That's J-I-R-A dot com. This episode is brought to you by Lincoln. What if your vehicle could help give you a reset? Lincoln Rejuvenate is designed to do just that.
38:45This multisensory feature combines expansive visuals, digital sense, and a seat massage for the driver, and is available in the 2027 Navigator SUV. Visit lincoln.com to learn more. Lincoln Rejuvenate does not operate unless the vehicle is in park when the vehicle is off or if any door is open. Do not use the feature in a closed garage or in other enclosed areas. See owner's manual for important operating instructions.
39:12Derek Thompson: In 2019, OpenAI releases GPT-2. And at this point, Dario Amadei is still the VP of research at OpenAI. The release of GPT-2 does a few things. One, it proves Amadei right. They use something like a thousand times more compute than with GPT-1, and they can see that the model is generalizing. It's getting smarter across a bunch of different domains, including those it's not specifically trained on. I guess that is generalizability. But the release of the model also really scares Dario Amadei. Why?
39:47Kevin Roose: He is worried that a text completion engine that is good at spitting out plausible generated text could be used for bad ends. It could be seized by propagandists or foreign disinformation campaigns. It could be used to pump out misinformation and fake news on an industrial scale. He knows that GPT-2 is not going to take over the world. It's just a text completion engine. But he is worried about how bad actors might use it to do unscrupulous things. And there's a big debate inside OpenAI about what to do with this. I mean, it helps to know, I think, that Dario Amadei is a sort of neurotic by temperament, But he is also part of this community of AI safety warriors in the Bay Area.
40:45Kevin Roose: You know, at one point in 2016, he's like living in a group house with Paul Cristiano and Chris Ola and Jared Kaplan and sort of all these now like leading lights of AI and AI safety. and they are all just sitting around at night, like reading posts from LessWrong, you know, and other like AI safety focused publications and talking about them and debating what the sort of existential threats from AI might be. And this was considered insane behavior in 2016 because nothing actually worked yet, but they were sort of looking, peering into the future and imagining when systems did work and could be improved through scaling, what that would mean for safety.
41:30Derek Thompson: Dario Amadei obviously no longer works for OpenAI. He's the CEO of Anthropic. You talk about all of these different rivalries that have been created in his final months at the company. It's basically like it's a three-front war. It's Dario versus Ilya Sutskiver. It's Dario versus Greg Brockman, the president. It's Dario versus Sam Altman. What can you tell us, again, shading towards salaciousness here in your brief answer to help us understand why Dario Amadei felt like he had to leave OpenAI and start a new company.
42:00Kevin Roose: So he had been thinking about this for years. I think this was not a rash decision, and he had actually sort of thought very seriously about quitting before, mostly because he thought that Sam Altman was dishonest. He had this sort of running list of times where he thought he had caught Sam lying to him or saying something that wasn't exactly true, or just doing things that seemed like bad judgment. Like this example that I just love that I was so happy to be able to reveal in the book because it was very salacious and very funny was that apparently Sam Altman met Kanye West at a party and kind of offhandedly mentioned that they were doing this private beta test for GPT-3 and offered to let him in.
42:45Kevin Roose: And, you know, Sam likes talking with celebrities and having celebrity friends. And you can sort of see why he would want to like show off this new model that he had built to this very famous person. And then he brings this back to OpenAI and they're like, what the fuck, man? Like, Kanye is not well. He had just posted a video of himself like peeing on a Grammy award. He was not yet like, you know, talking about Hitler, but like things were not going well for him mentally. And here Sam is offering to let him into this product that they think could be quite dangerous in the wrong hands. And so they sort of put a stop to this.
43:23Kevin Roose: There are like, you know, dozens of incidents like this. And the way it was explained to me is that none of them on their own would have been sort of worth quitting over. But together, the sort of snowball effect meant that one day in 2019, he essentially just sort of decided he couldn't do it anymore. He was not going to be successful at steering open AI towards safety from the inside. He wanted to leave and bring his co-founders with him and start something new.
43:50Derek Thompson: I think one question that I've had about Anthropic is how did they go from being a company that splintered off from open AI for the purpose of protecting this technology from the forces of commercialization only to become in just a few years even more commercially successful than open AI? I mean, how can you not even defend, just describe why Dario and the other leaders of Anthropic went from essentially thinking Anthropic should never release Claude as a consumer product to releasing Claude as a consumer product that makes tens of billions of dollars for Anthropic a year?
44:42Kevin Roose: Yeah, it's the central question of Anthropic. How did this small, I mean, when I first visited Anthropic in 2023, it was a couple years after its founding, but they had not yet released like a good chatbot that people were actually using. And it was like 150 people and they were all cooped up in this office and they had sort of no business strategy. It was just kind of like a research lab, but ChatGPT had come out and they had sort of been convinced, in part by Dario, that they needed to compete. Like if they wanted to be relevant on safety, they had to have a model that was as good as any on the market.
45:27Kevin Roose: And I think part of that was seen internally as kind of like Dario just kind of justifying something that he wanted to do anyway, because he hated Sam Altman so much and Hope and AI so much and couldn't bear the thought of losing to them, that he sort of concocted this rationale of the, he calls it the race to the top. Basically, if you are in the lead or close to the lead, you get to sort of lead by example and everyone else has to follow you when it comes to safety and making your models more aligned. So this becomes his sort of, his bet. But you're right, at the start, they had no intention of making money.
46:02Kevin Roose: I talked to lots of early Anthropic employees who were like, yeah, like it was not even a subject that came up. We thought we were going to build safe AI systems and do safety research on other people's models. They had this list of crazy ideas for how they could make money, including having Claude start a hedge fund and give them trading tips. Didn't end up working out. But they eventually decide, well, two things happen. One is AGI turns out to be more expensive than they had bargained for. So in this founding document that Dario writes in 2021, it's called Vision for a New Co., he predicts that Anthropic would need about$10 billion to build a full-fledged AGI.
46:45Kevin Roose: And as we now know,$10 billion is a vast undercount. You need like hundreds of billions of dollars, possibly trillions of dollars, to build something that will fulfill the sort of original vision of AGI. So they need more money. and they have raised a bunch of money, but they are going to need more. And so they have to have something to sell to raise that money. So that's part of it. There's a part of this story
47:11Derek Thompson: that you absolutely have to include, which is that their top benefactor is a little old man named Sam Bankman-Fried. Yes. He is the initial investor who buys, who I think owns 13 % of the company's equity at one point. Yes. Is Sam Bankman-Fried. And so once Sam Bankman-Fried is in jail, Well, then not only do you need a lot of money, you also no longer have access to the person who owns 13 % of your equity. So you need to go further and you can't rely on EA billionaires. And so you put that together and you say, okay, big blob of compute. We need hundreds of billions or tens of billions of dollars.
47:49Derek Thompson: We can't rely on philanthropy. We have to go through commercial ends. That means we have to build a commercial product. Like I can see step by step. It just doesn't necessarily make it like the most moral step by step logic.
48:00Kevin Roose: No, and I think they've done a lot of moral, I think there have been some stances that have been both the right thing for Anthropik's mission and the right thing for their business. And some decisions that have been more the right thing for their business than their mission. I think these deals that they've done with Gulf states, their willingness to raise money from dictatorships, I think has been, you know, their own employees have flagged that as sort of an instance of hypocrisy. And I think they will continue to face these pressures because they are now, you know, a huge company. And they are the fastest growing company by some measures in the history of capitalism.
48:47Kevin Roose: and yet it is still not big enough. They still have not reached where they are going. And so they will continue to have to raise money and make commercial sacrifices.
48:56Derek Thompson: And this is what I think is maybe the most interesting part of this framework because, and I'm quoting from your, or summarizing your book here rather than coming up with this idea on my own. But I think it is both very important to point out that Sam Altman and Dario Amadei are extremely different people motivated by not only different philosophies of the world, but even literally different books, like Sam Altman is a Ray Kurzweil and, you know, Nick Bostrom guy, super intelligence. And then you've got Dario Amadei reading about the making of the atomic bomb by Richard Rhodes. But despite their extraordinary differences and mutual self and mutual loathing, they both end up in the same bucket of Accelerate.
49:40Derek Thompson: They both have to release chatbots that make tens of billions of dollars a year. They both have to raise money from benefactors that aren't necessarily liberal capitalist. And so no matter where you start out in this race, you end up in the bucket of speed, speed, speed. I mean, that does seem like an important takeaway from this story.
50:02Kevin Roose: It's a funny thing, isn't it? It's like whatever your motives are when you start, you end up racing right alongside everyone else who's racing with very different motives, but you're kind of checking your sides to make sure that no one's pulling ahead of you. I think that's a really important point. And I think it's part of what is freaking out the insiders at the labs, the people like Jacob Coxon, the people who are writing essays and open letters about how we need to slow down. They see their bosses engaging in this kind of ego-driven brinksmanship where it's like, it is clear that part of the reason they are racing so hard is because they can't stand the thought of these particular rivals winning.
50:54Kevin Roose: And I think it makes them very uncomfortable. And it makes me very uncomfortable too.
50:58Derek Thompson: I want to introduce our last character here, Demis Asabis. He might be the least famous of these three men, but he is the only man on this list who's won a Nobel Prize for their work in AI. And there's absolutely a version, I think, of the future where his contributions might in 20 or 30 years be seen as the most important because of how significant the science contributions of DeepMind are. Let's just start in the biggest picture. Who is Demis Asabis and what does he want from superintelligence?
51:27Kevin Roose: Demis is a fascinating character. There's a great biography of him by Sebastian Malaby that just came out. It has way more material on him than I was able to fit in my book. But basically, the story of Demis Asabas is he is a child chess prodigy. He is a competitive chess player, very highly ranked in the world for his age group, sort of a brilliant and competitive mind. People have described him to me as the most competitive person in the world. There's this sort of famous anecdote about him, which is that this outfit known as the Mind Sports Olympiad starts this competition called Pentamind, where it's basically like an all-around Olympics of games, essentially.
52:14Kevin Roose: And they are trying to find sort of the best game player in the world. And Demis Asabas learns about this contest, decides that's going to be him. He masters all five of the games included in the pentamide and wins the contest five out of the first six years. So he is the most competitive person in the world. And he's also very interested. He's a cognitive neuroscientist by training. He has a PhD in that. And so he's very interested in this question of what is intelligence and how can you build it into a machine? And this is the thing that obsesses him. He becomes sort of convinced that AGI is possible long before this is common wisdom.
52:49Kevin Roose: People sort of look at him as like a fringe exotic, like who's this weird British man talking about the coming machine god. And they set up this little company, DeepMind, and they start making progress toward, their path that they choose is teaching AIs to play games. So they start with Atari games, then they move on to Go and Shogi and all these other games, StarCraft. But their thesis is basically that you can train an AI to play a game, get really good at the game, and then sort of expand the domain to include things that are less and less like games and more and more like tasks in the real world, and that that's how you're going to build AGI.
53:31Derek Thompson: And this is a very different approach from OpenAI and Anthropic. I mean, they are not going the language, the large language model approach. In fact, you've got lots of quotes in your book of Demis and people working at DeepMind explicitly rejecting the language approach, saying language is like a simulation of intelligence, but it's not intelligence itself. Can you help us understand how game-based approaches to growing AI differ from open AI and Anthropics' emphasis on transformers and large language models?
54:03Kevin Roose: Yeah, they're essentially competing philosophies and competing approaches to creating powerful AI systems. So the approach that DeepMind uses and specializes in, the approach that helps them build this system, AlphaGo, that beats the best human Go players. And that's a very famous moment in the history of AI. That was known as deep reinforcement learning. This is basically trial and error. You give an AI system a task, make a Go move that is likely to increase your probability of a win. And you give it a reward if it does that, and you give it a penalty if it doesn't. And then you just run it in simulation millions and millions of times, It gets a little better every time, and eventually it surpasses the best human Go players and becomes superhuman.
54:51Kevin Roose: Their view is that a perfect, elegant AI system should be able to learn without any human data whatsoever. You should just be able to stick it into a novel environment like a Go board or a chessboard or something, and it should be able to kind of bootstrap its way to superhuman performance because it is learning from its experience. Open AI and transformers and LLMs are based on essentially the opposite view. They are built out of human data. Take the entire internet, jam it through a transformer, come up with a system that is extremely good at predicting the next words in a sentence, and use that to increase the overall intelligence of your model.
55:34Kevin Roose: There's this very famous debate inside DeepMind in these years known as the language is enough versus reward is enough schools of thought, which is basically the sort of microcosmic version of deep mind versus open AI and anthropic. Do AI systems develop better intelligence as a result of experiencing things and learning through trial and error? Or do they develop better intelligence because they learn more of language and language is what we use to encode intelligence in humans?
56:07Derek Thompson: I love that. Like philosophically, I was just thinking as you were saying that. I remember that part of the book, but I didn't have this particular thought as I was reading it. The difference between those approaches philosophically is maybe a little bit like this. Tell me if this metaphor is wrong. Language is enough. It's essentially like saying, if someone could memorize everything that had ever been written in science and great literature and human psychology, they would be super intelligent. They would be able to, if they could hold it all in their head, make connections and map new recombinations of ideas that would make them essentially super intelligent.
56:43Derek Thompson: That's language is enough. And then there's reward is enough is kind of like, what if someone could live a million lives and experience in those million lives, how some decisions lead to happiness and some decisions lead to unhappiness and some decisions lead to making a company that makes$10 billion. And some decisions lead to making a company that sucks. you would in those million lives learn all of the positive and negative reinforcements that would lead you to becoming super intelligent and sort of kind of interesting to think of like like it's like like sort of a borgesian way of sort of summarizing these ideas like would you rather if you wanted to be super intelligent memorize every word produced by all of the geniuses in human history or just live a million times it's like a great way of putting and that
57:27Kevin Roose: That is almost entirely true. And as it turned out, in the end, you needed both. Today's AI models are a combination of LLMs and reinforcement learning. So they all sort of converged on this hybrid solution. But there were years where OpenAI and parts of Google were very interested in scaling up language models. And the people at DeepMind just said, like, that's crazy. You cannot build superintelligence out of Reddit posts. Like, it's just not going to happen. And they were wrong. And they have since, you know, realized the error of their ways and are now building LLMs, too.
58:11Derek Thompson: So Sam Altman and Daria Amadei hate each other. Altman and Hassabis met at the Vatican in 2015, 2016. As you reported, they also don't like each other very much. I have a question about why they don't like each other. But I'm going to put that on ice. Read the book if you want to know why they hate each other. Last question about Hassabis specifically. AlphaFold, which is the protein folding technology that won the Nobel, which emerged from this RL theory of AI, is a triumph. But Google doesn't have an AI consumer business that rivals open AI or anthropic. And Hassabis initially dismissed large language models, as you've just said, in a way that I think has cost Google significantly on the commercial front.
58:57So it's a mixed legacy. And I wonder if you, if you stop the clock today, how do you see his legacy?
59:06Kevin Roose: I see it as, as, um, both an incredible scientific legacy. I mean, as you mentioned, he's the only one out of any of these guys who has a Nobel prize, who has produced something that has actually revolutionized a scientific field today.
59:25I think his theory of change was correct in the broadest sense of like he believed in scaling,
59:33Kevin Roose: he believed in reinforcement learning, he was early on a number of these things, including just the belief in AGI itself. But I think he, the die may have been cast when DeepMind was sold to Google. I don't think they necessarily had a choice. They needed to build bigger blobs of compute to run their experiments. They needed deep pockets. They needed to partner with a hyperscaler or someone with the willingness to spend millions or billions of dollars on what they were doing. But I think that ever since he sold DeepMind to Google, he has been, in some cases, fighting against the commercial incentives and the pressure, the short-term pressures of being part of a large search-based internet giant.
1:00:24Kevin Roose: And, you know, he explicitly tried to get away from Google. There's some reporting in the book about this thing, Project Mario, which was sort of his attempt to kind of carve DeepMind out of the larger Google apparatus. But that didn't work. And then he ended up running a division of Google for, you know, years with, you know, thousands of employees, which is not what he wanted to spend his time doing. Most more recently, he's been sort of kicked upstairs to this, like, you know, uh, this sort of executive chairman position. Um, and Google, you know, deep mind is firmly part of Google now in a way that it, it hasn't always been.
1:00:59Kevin Roose: So I think there's a tragic element too, of like, here's a guy who really wanted to use AGI to unlock the mysteries of science and to discover, you know, new, new things in physics and math and biology, and who, you know, sort of by no fault of his own, was stuck running a, you know, a large division of a commercial search giant that made most of its money through ads on search results. So look, I think it's too early to count Google out. They have lots of money. They have lots of smart people. And I think as long as Demis is there, They will have, you know, his sort of inspiration, his sort of drive.
1:01:42Kevin Roose: And he's very inspirational to the people there. But I also think there is this reality that they are just a more traditional company than either of these other labs. And they have lots of competing incentives and feuding internal teams and bureaucracy to navigate. And it's just going to be a lot harder for them to move quickly.
1:02:01Derek Thompson: So summing up, we've got Sam Altman, who I think of as kind of the JP Morgan of AI, like the mogul, the dealmaker, someone whose company bet on transformers, ship Chachubiti, started this whole infrastructure cycle and has continued, has been very successful at using other people's ideas to build the power, his power and the power of his company. Big blob of compute was Dario's idea, but who leads the labs in compute secured by a mile? it's open AI. And so that's, there you've got, there you've got Sam. Dario, I see, as you described as something more like a moral crusader in the Oppenheimer mode.
1:02:44Derek Thompson: Um, someone who's both an inventor again, like Oppenheimer and, but also someone who's, who's safety crusading both inspires some people and also rubs some people the wrong way. You've got a lot of that in the book about how at open AI, some people were just so tired of Dario using the moral argument to try to win every single corporate fight. And there's Asabas, whose historical analog is harder to come up with because in some ways he's kind of like the Nikola Tesla, like a great inventor who was a little bit in the shadow of his contemporary, but it doesn't work because he's like a Nikola Tesla who worked at GM or something because he's, as you said, one blocker for his success is that he's been inside of a bureaucracy rather than leading his own thing outside of that conglomerate.
1:03:30Derek Thompson: Final question to you, now that we've sort of talked about their different motivations, different inspirations, who do you think got the most right?
1:03:41I think scientifically,
1:03:44Kevin Roose: Demis and Dario got the most right, both by betting early on various forms of scaling. Sam Altman's not a scientist. I think he got the politics more right than they did. I think he understood what it would take to make all of the deals, to secure all of the compute. There's this incredible scene in the book where he's basically scheming to get around Elon Musk, one of his nemeses, to get into the good graces of the newly elected Donald Trump. And I think he is a savvier political operator than either of his rivals and maybe than anyone on Earth. I mean, he might be one of the savviest dealmakers and operators we've ever seen.
1:04:35Kevin Roose: And I think that has been his downfall at times too. So yeah, I think they've both, they've all gotten pieces of it right, but no one has gotten everything right. I think Dario's gotten the science more right than anyone. I mean, you can go back and look at these papers he was writing in 2016 and 2017. It's like he's an oracle who sees the future, but he is too honest. His fatal flaw, as someone put it to me who's close to him, is that he cannot tell a lie to save his life. I mean, you could see him at that gaggle on the White House lawn the other day, just sort of fidgeting and being visibly agitated and nervous while all of the other AI CEOs are there just kind of smoothly kissing up to Trump and kissing the ring.
1:05:22Kevin Roose: And so I think that is a quality I admire in him, but it is also something that has caused him and Anthropic a lot of trouble.
1:05:32Derek Thompson: Finally, I think it should be said, we've talked about three people in this book. There's a lot of characters in the AGI Chronicles and one character that's a little bit like a Forrest Gump who just appears in every single story. um elon musk not only co-founded open ai with sam altman he also tried to hire dario amade to spacex and tried to acquire deep mind and uh demis's work uh uh i guess after he initially went to google in addition to funding some of demis's work so it it is remarkable how like just cropping up through the book scene after scene is elon musk being like hey by the way i'm I'm holding a string here.
1:06:09Derek Thompson: Yeah. It's an amazing story. I'm really, really glad you wrote it. It's a wonderful, wonderful book. Thank you so much. Good luck with it. Kevin Roos, thank you so much. Thanks, Jared.
1:06:31Are you looking for support in your weight management journey? Zepbound Terzepatide may be able to help. ZepBound is a prescription medicine used with a reduced calorie diet and increased physical activity to help adults with obesity, or some adults with overweight who also have weight-related medical problems to lose excess body weight and keep the weight off. ZepBound is approved as a 2.5, 5, 7.5, 10, 12.5, or 15 milligram injection. ZepBound contains terzepatide and should not be used with other terzepatide-containing products or any GLP-1 receptor agonist medicines. It is not known if ZepBound is safe and effective for use in children.
1:07:11Don't share needles or pens or reuse needles. Don't take if allergic to it. Or if you or someone in your family had medullary thyroid cancer, or if you've had multiple endocrine neoplasia syndrome type 2, tell your doctor if you get a lump or swelling in your neck. Stop ZepBound and call your doctor if you have severe stomach pain or a serious allergic reaction. Severe side effects may include inflamed pancreas or gallbladder problems. Tell your doctor if you experience vision changes before scheduled procedures with anesthesia, if you're nursing, pregnant, plan to be, or taking birth control pills.
1:07:43Taking ZepBound with a sulfonylurea or insulin may cause low blood sugar. Side effects include nausea, diarrhea, and vomiting, which can cause dehydration and worsen kidney problems. Talk to your doctor. Call 1-800-545-5979 or visit zepbounds.lily.com. Ulcerative colitis and Crohn's disease symptoms don't pause for the moments that matter. That's why understanding your treatment options matters. Discover what's possible with Trimphia. Gucelcomab. Trimphia is a prescription medicine used to treat adults with moderately to severely active Crohn's disease and adults with moderately to severely active ulcerative colitis.
1:08:22Serious allergic reactions, increased risk of infections, or lower ability to fight them and liver problems may occur. Before treatment, get checked for infections and tuberculosis. Tell your doctor if you have an infection, flu-like symptoms, or need a vaccine. Talk to your doctor and explore what's possible. Or call 1-800-526-7736 to learn more about Trimphia and visit TrimphiaRadio.com.
Sam Altman, Dario Amodei, and Demis Hassabis are three of the most important figures in AI superintelligence, but each of them have very different ideas about what this technology should become and what it could mean for the future.
Kevin Roose, author of 'The AGI Chronicles,' joins Derek to explain the people behind the AI race: the books and ideas that shaped them, what they want for the future, and how their rivalries helped drive the industry to its current moment.
If you have questions, observations, or ideas for future episodes, email us at [email protected].
Host: Derek ThompsonGuest: Kevin RooseProducer: Devon BaroldiAdditional Production Support: Ben Glicksman
Comcast Business. Get AI-powered threat detection for small business.
Learn More at https://www.lincoln.com/
Learn more about your ad choices. Visit podcastchoices.com/adchoices