


Kevin Roos analyzes Anthropic’s IPO S-1 (“The First Real A.I. Test”) and what it implies about AI demand, compute markets, and regulation. He also discusses AI progress, “AGI-pilled” thinking, and how labs handle safety (frontier red teaming) amid weak government oversight.
Kevin Roos, author of The AGI Chronicles and co-host of Machine Gods; he breaks down Anthropic’s filings and broader AI industry dynamics.
Anthropic’s revenue grew ~10x yearly from near-zero to billions; net revenue retention is ~900% (existing customers spending ~9x more). Anthropic faces massive compute/infrastructure contract obligations (~$800–$855B over a decade), so long-term profitability depends on continued rapid growth. AI compute is scarce, driving long-term “lockup” deals with hyperscalers and compute providers.
Anthropic buying compute from Google and SpaceX; AI solving Navier-Stokes and other major math/science problems; Anthropic frontier red team discovering a model could make a chemical weapon.
Written by AI. May contain mistakes.
Discussion on food trends and a free phone promotion.
“State Fair food is getting smaller and pricier.”
Analyzing Anthropic's recent IPO and its implications.
“Normally, when I bring an author on, we talk about their book.”
Discussing Anthropic's revenue growth and financial challenges.
“So I see revenue in the first half of the year,$16.3 billion.”
Exploring Anthropic's extraordinary customer spending habits.
“Usually you're happy with 10 % revenue growth year over year.”
Assessing Anthropic's future valuation and the demand for AI.
“Their customers on average are spending nine times as much on their models as they did last year.”
Analyzing the current state of AI technology and market dynamics.
“a show about technology, power, and how the future will be determined.”
Tracing the evolution of AI technology and its current implications.
“All of this is happening as neocloud stocks have been bouncing around, some going off a cliff.”
Discover the recent advancements in AI and their implications.
“Like the Navier-Stokes equation, which is one of the seven hardest open problems in mathematics and which an open AI model solved just a few weeks ago.”
Learn about the evolution of AI from basic functions to complex models.
“So I think what's happening in math, what's happening in coding, The question is whether that is contained to those domains or whether that is kind of going to spread to every domain as the systems get more capable.”
Explore the concept of being AGI-pilled and its implications.
“And maybe that could just continue forever or until you run out of compute.”
Identify the three main groups in AI development and their motivations.
“You group them into scientists, seekers, and survivalists.”
Discuss how AI developers perceive public sentiment towards AI.
“But I do think that is a sincere belief that someone will build this.”
Examine the parallels between AI development and the Manhattan Project.
“A lot of the people in this book, and Dario Amadei, again, comes up, you know, liken what they are doing to the Manhattan Project, right?”
Discusses the failures of government in regulating AI and the potential consequences.
“They were responsible in some very direct way to the public sector and to the public.”
Explores the responsibilities of AI companies regarding safety and regulation.
“We barely have enough capacity in the government to run tests and evaluations on these models to see whether they're dangerous or not.”
Examines the psychological impacts of AI and the skepticism surrounding existential risks.
“is that there's so much focus on, understandably, extinction risk, you know, the P-doom, but less on near-term harms, the people who have developed AI psychosis, the teenagers who have killed themselves.”
Delves into the definition of AGI and its practical uses in various fields.
“Everyone's got a different definition of AGI these days.”
Shares personal experiences with AI tools and their impact on daily life.
“What is a thing that AI could do in your life that would make you go, oh, this is AGI or this is as smart or smarter than me?”

Get the gist, explore an idea or ask about something said.
On What Next | Daily News and Analysis, what are the key points of this episode?
Automatic transcript. May contain errors.
0:00State Fair food is getting smaller and pricier. We'll have room for fried cookies. Okay, you never overspend on food. What's your deal? What's his deal? He went to Metro and got$450 in value on the new Samsung Galaxy A37 5G for free and no activation fees. Free phone? Oh, sign me up. Get that more for your money feeling only on Metro by T-Mobile. Bring your number and activate$40 period plan. $450 based on device discount waived activation fee. Not available if with Metro or T-Mobile in the past 180 days.
0:33This episode is sponsored by Smart Travel, a new podcast from NerdWallet. You know that one friend who always finds the best travel deals, picks the right cards, and somehow ends up in first class for the price of coach? Smart Travel is like that friend, but in podcast form. They cover all things travel finance, like the quick math for knowing when to pay with points or just swipe, or which travel cards actually pay you back, and which just make your wallet heavier for no reason. Travel smarter and spend less with help from NerdWallet. Follow Smart Travel wherever you get your podcasts. Hello.
1:12You are used to hearing a lot from us, but now we want to hear from you. We are running a listener survey from now through the midterms and a little bit after. It's for What Next and for TBD, And we're going to use that information to talk about how the show should look and sound in the future. You can find the survey at slate.com slash whatnextsurvey. Again, that's slate.com slash whatnextsurvey. We really appreciate it. It'll help us a lot. And thank you.
1:43Normally, when I bring an author on, we talk about their book. But this week, we had a wonderful accident of timing. Anthropic had just filed its IPO prospectus. It's also known as the S-1. That's the name of the form at the Securities and Exchange Commission. And who better to break it down and analyze it with us than Kevin Roos, author of the AGI Chronicles. So I see revenue in the first half of the year,$16.3 billion. That's a lot of money. A 14-fold increase from the year before. A loss of$80.7 billion. Mostly accounting charges. The number that really stood out to me is somewhere between$800 and$855 billion in compute and infrastructure contracts over the next decade.
2:29They got to pay that no matter what. The big question when I look at this, can Anthropic make enough money to cover those costs over the next 10 years?
2:39Kevin Roose: I think the answer depends a lot on if they keep growing as they have been growing. this is one of the most extraordinary s1s i've ever seen you and i have both looked at a lot of s1s in our time i'm not sure i've ever seen a company i'm sure i haven't seen a company go from having essentially zero revenue uh just three years ago to now having um the revenue that they they have today uh they've been 10xing their revenue every year since they started making any money. That's crazy. Which does not happen in capitalism. Like that is not a number. Usually you're happy with 10 % revenue growth year over year.
3:24Kevin Roose: 10x is unheard of. And the number that stuck out to me from this S1 is actually something that was a little buried in the details, but it was a number called net revenue retention. This is basically of your existing customers, in this case, the businesses that use Anthropix models, how much are they spending this year versus a year ago? Same customers. Usually if you're a software business, you're happy if that number is 120%, 130%. Maybe the customers are spending a little bit more than they did last year. Anthropix net revenue retention, according to this filing, is 900%. 100%. Whoa. Their customers on average are spending nine times as much on their models as they did last year.
4:14Kevin Roose: So that is, I've never seen that before in any public company filing or any filing for any company about to go public. And I think that just sticks to, or it speaks to the incredible stickiness of this product among the large businesses that have come to rely on Anthropics models. Do you think this IPO is different from the other mega IPOs? Like, is it different from the SpaceX IPO? Is it different from whatever will be the coming OpenAI IPO? Like, is there a different story that the Anthropic S1 is telling you? Yeah, I think it really forces a choice about what you think this company is and what it's going to do.
4:55Kevin Roose: Because if you're valuing it based on its current revenues or current costs, you may think, well, this doesn't seem like a company that is worth one to two trillion dollars. That's, you know, there are only a handful of companies in the world that are worth that much. They all bring in significantly more revenue than Anthropic does. At the same time, if you believe that the story that Anthropik is telling about AI is true, then this technology is going to get more and more powerful. The use of this technology in business will increase. The demand for AI will increase. And all of that will happen very, very quickly if what they're saying about this being a technology that's on an exponential curve is true.
5:44Kevin Roose: So you are making a bet not just on this business, but on the trajectory of AI. And that's something that I feel like the market hasn't really decided yet what it thinks about AI. Today on the show, Kevin tells the story of AI's trajectory, technological breakthroughs, vicious rivalries, and the companies that brought us to this moment. I'm Lizzie O 'Leary, and this is What Next TBD, a show about technology, power, and how the future will be determined. Stick around.
6:20State Fair food is getting smaller and pricier. We'll have room for fried cookies. Okay, you never overspend on food. What's your deal? What's his deal? He went to Metro and got$450 in value on the new Samsung Galaxy A37 5G for free. And no activation fees. Free phone? Oh, sign me up. Get that more for your money feeling. Only on Metro by T-Mobile. Bring your number and activate$40 period plan. $450 based on device discount waived activation fee. Not available if we've met your T-Mobile in the past 180 days. Your call has been forwarded to an automatic voice message system. A missed call doesn't feel like anything.
6:56No red flag, no notifications screaming at you. It just disappears into your call log. And nine times out of ten, that was someone ready to hand you money. That's why today's episode is brought to you by Quo, spelled Q-U-O. The business phone system built so you never miss a call. Here's what I like about it. All your calls, texts, and voicemails live in one place, so anyone on your team can pick up a conversation, see the full history, and respond fast. No more, wait, who talked to this customer last? And if you're worried about after-hours leads slipping through, Quo's optional built-in AI agent can answer questions and even book appointments while your team's offline.
7:35It's the number one rated business phone system on G2, and over 90 ,000 businesses already trust it to stay reachable. Money is on the line. always say hello with Quo try Quo for free plus get 20 % off your first 6 months when you go to Quo.com slash tech that's Q-U-O dot com slash tech Grainger knows if you're a procurement manager at an industrial manufacturing facility you're balancing inventory and budget while sourcing critical supplies across the operation too much inventory? cost climb too little? production slows Grainger's$2 billion in on-hand inventory helps you get what you need when you need it, so you can help keep budgets balanced and lines moving.
8:17Call 1-800-GRAINGER, click grainger.com, or just stop by. Grainger, for the ones who get it done.
8:28Let's talk about the market right now. All of this is happening as neocloud stocks have been bouncing around, some going off a cliff. I guess the best way to describe them is cloud providers. They are built specifically for GPU compute, AI workloads. There are also worries about corporate debt, how much debt and what yields these corporate bonds are getting. Investors seem unsure. Elon posting 200 plus neoclouds are going to see guppy level death and consolidation. Elon posts a lot of things. Does he have a point?
9:02Kevin Roose: I think there will certainly be some neoclouds that don't do well. And just so people understand, there are kind of two types of businesses that are supplying compute to the AI companies. There are the so-called hyperscalers. That would be the Googles, the Microsofts, the Amazons of the world. And then there are the neoclouds, which is sort of anyone that has popped up in recent years to serve the demand for AI chips and AI compute. so that would be the core weaves and there are a bunch of other companies like that I think there are probably going to be winners and losers in both of those categories but right now the big picture is that the AI companies are severely compute constrained they do not have nearly enough compute to serve the demand for their models and so they are buying it from everywhere they can to be able to just keep their services up and running and to fulfill the demand.
10:04Kevin Roose: So that's why we see, for example, Anthropic buying compute from both Google and SpaceX, AI, SI, whatever they're called today. Elon Co. Yes. I mean, they're spending billions and billions of dollars with each of those companies and Amazon to just sort of get as much compute as they possibly can. And that's why you see these sort of long-term arrangements that are for hundreds of billions of dollars because they want to lock up compute for years into the future. And they're taking on debt to do that. They're striking these long-term deals. But that is the reason is because they are confident that they will be able to use all of that compute and then some both to train new models and to serve the models that they've already trained.
10:48I think one of the things that's hard is if you're not deeply involved in this world, if you're a regular person who catches the news every now and then, sees clips come across their social media feed, you're like, wait, I don't understand. Is AI successful? Is it a bet on the future? Is it a giant scam? What's happening here? Because the headlines are really hard to make sense of, especially in the age of giant IPOs.
11:12Kevin Roose: Yeah, I totally understand the confusion. I think there's also been a lot of people, I don't know, kind of cheering for a bubble pop in this weird way. Like they kind of hope that all of this turns out to be sort of a fad because the alternative is that like everything's gonna change and that's really uncomfortable for people. So I think, you know, I've become fairly convinced, not that we're not in a bubble, but that this is a durable technology that people are finding uses for this stuff, that it is a story of actually tremendous success. Like think about that figure I just quoted you. The largest companies in the world who are using Anthropic are spending nine times more on it on average this year than they did last year.
11:56Kevin Roose: To me, that suggests that actually these companies are getting a ton of benefit out of this, that it is not just them sort of doing work to look busy and getting caught up in token maxing or whatever. That suggests to me that like these companies are actually at the beginning, not the end of what they will spend on AI. Another figure that stood out to me in the Anthropic IPO filing, the top fortune companies by market cap are spending roughly about 0.1 % of their total expenses on AI with Anthropic. So if that number were to even grow to 1%, that would represent another 10x increase in the spending on these companies.
12:41Kevin Roose: And if that goes to 10 % or 30 % of their costs, I mean, you can see where these projections come from. I really think the sort of fact that I get to talk to you as the Anthropik numbers came out is fascinating because I feel like Anthropik and Dario Amadei's mind is a huge part of your book, right? You follow, I mean, you follow all of the companies, but he's in there a lot. And there's this part where he's like in Italy, talking to a horse, like spending some time, I don't know, like writing long treatises. And then I look at the S1 and there's a part where the company writes, the advent of machine intelligence is one of the seminal moments in human history.
13:27And that reads like what I saw of him in your book. Number one, I do agree. And number two, like, is it now? Is this the advent of machine intelligence?
13:41Kevin Roose: I think, yes. I think it is the advent of machine intelligence. And I think it is a seminal moment in human history. Now, I have been accused by some of having AI psychosis. So maybe discount what I say. But it is just empirically true that AI systems today are solving some of the hardest problems that humanity has ever struggled with. Like what? Like the Navier-Stokes equation, which is one of the seven hardest open problems in mathematics and which an open AI model solved just a few weeks ago. They also released several more breakthroughs on massive scientific and mathematical problems just this week.
14:27Kevin Roose: So I think what's happening in math, what's happening in coding, The question is whether that is contained to those domains or whether that is kind of going to spread to every domain as the systems get more capable. You know, part of the reason I wrote the book is because I wanted to just trace the evolution of this technology over the last decade. Because if you go back to 2016, when a lot of these people were starting to get interested in AI, none of this stuff worked at all. Like we had Siri and Alexa and image recognition and some speech recognition and translation features in some apps. But like there was no large language model.
15:14Kevin Roose: There was nothing that could write its own code. There was nothing that could take in a prompt in English and spit out a PowerPoint deck or a business memo or a working software program. And so I was just curious, like, how did we get there? And so the book is really a story of Dario and his contemporaries kind of waking up and realizing, like, this is happening. Like, we have these systems now that seem to obey these sort of scaling principles or scaling laws where the bigger you make them, the smarter they get. And maybe that could just continue forever or until you run out of compute. You told Nilay Patel, friend of the show, friend of yours, that you were AGI-pilled.
15:59What does that mean?
16:00Kevin Roose: It's a term that people in San Francisco use to basically – so I think for me there are a few things that being AGI-pilled consists of. One is just a complete inability to think about or make any decisions that are not in some way related to AI or your feelings about AI or the future. like it is um it is a kind of pathological obsession and i grant that i have it i think in the extreme you get people who are making decisions like you know not contributing to their 401k anymore because they think well either we're all going to be dead in you know by the time i'm ready to retire or money is going to be meaningless and the robots will be meeting all of our material needs and so why save for retirement i am not that agi you still save for retirement.
16:48Kevin Roose: I do. I do. But I think, look, I think the basic thing that people who are AGI pill share is just that they assume that the current pace and trajectory of AI progress will continue. That the gains that we've seen where, you know, two or three years ago, the leading AI systems in the world could not tell you the number of R's in the word strawberry with any reliability to now solving millennium prize problems, the hardest open problems in math. Like that happened in two or three years. Being AGI-pilled means you think there's at least some chance that that pace of progress will continue, that there is no natural plateau at human level intelligence, that these systems are getting better and smarter as they get bigger, and that that will continue.
17:38Kevin Roose: I don't think it means that you have to think this is going to happen tomorrow, or buy any of the sort of doomery or optimistic predictions about how quickly society will change. I just think it means you believe that the straight lines on the graphs will keep going. I think one of the things that really comes through in this book is the amount of time you spent with the different people working on these products and the factions and the rivalries and the people who really hate each other and the people who spend a tremendous amount of time going to parties together. You group them into scientists, seekers, and survivalists.
18:16Who is who, and how does it shape these companies they're running?
18:22Kevin Roose: The scientists are the people who are primarily interested in the technical challenge of building AGI, of building the human-level artificial general intelligence, and who also are interested in using AGI for science. So someone like Demis Hassabis, the former CEO of DeepMind, he is very interested in how can we take this machine intelligence and use it to understand the natural world, understand biology. He won the Nobel Prize for using the system called AlphaFold to predict the structures of hundreds of millions of proteins. The scientists are really motivated by the puzzle of scientific discovery and using AI to help them crack that.
19:08Kevin Roose: The seekers are the people who believe that we are sort of destined to create a superhuman machine intelligence. It is almost the next natural phase of evolution that we would create our digital successors. And they just kind of want to see what's out there. And I would put Sam Altman and Elon Musk into this category. these are people who are motivated by curiosity they feel like they're you know they a lot of them read a lot of science fiction as kids and came to the conclusion that you know it was humanity's destiny to introduce this superior form of intelligence into the universe and then the survivalists are the people like Dario Amadei at Anthropic who are very worried about AI they think it could blow up the world they think it could literally cause human extinction or disempowerment, but they feel like it is their duty to build it safely before someone else builds it unsafely.
20:06Kevin Roose: So they are like trying to do what the people in Armageddon did and like get onto the asteroid and like blow it up with a nuclear bomb. Do you think they get the level of antipathy that a lot of people, a lot of Americans have toward them and have toward AI right now? I think they get it. They see the same polls that everyone does. I think they understand that people are very angry about the possibility that AI could replace jobs. I just think they feel like in some sense, this is going to happen, whether people like it or not. They believe that this is inevitable. They sincerely do. It's, you know, they'll tell you like the recipe is not that hard.
20:53Kevin Roose: You got data, you got compute, you got algorithms, you put it into a big slurry and mix it all together and outcomes superhuman AI. And look, I don't know that I buy that it is inevitable. I think that is a very convenient argument for people who also want to be racing each other for all these different and often personal reasons. But I do think that is a sincere belief that someone will build this. And whether it is them or their rival or China, it is their moral obligation they feel to build it first. I mean, you have this very funny moment in the book referencing the billboards that are on the 101 or there's one that's like ERP without the PTSD.
21:38I mean, I remember landing in San Francisco and sending your colleague Mike Isaac a text being like, what are all these billboards? I don't understand them. These are crazy. But I think if you are in that world and you're moving among the rationalists, effective altruists, doomers, like that all feels like it's all the precipice of some phenomenal understanding. And I wonder if you feel like part of your job is to translate that to normies. And if you ever feel like you're getting sucked into the world so much that you are, I don't know, wearing their glasses, right? Seeing it from their perspective.
22:22Kevin Roose: I absolutely do feel like it's my job to translate. In fact, I've described myself as a kind of translator. or taking the conversation that's happening inside the AI bubble. I mean, they trust you. And exposing it to the outside world and helping people understand and demystifying some of what is going on here. Like, I think that is absolutely my job. And I think it is all of our jobs as journalists to, I mean, what could be more journalistic than taking a very important conversation that is happening in coded language in secret and making it accessible to outsiders? So that I absolutely feel like is part of my job.
22:58Kevin Roose: As far as whether I've gone native or whether I'm seeing it through their glasses, like, I don't know. You tell me, Lizzie. I don't know. You're the one who said you were AGI-filled. I'm just curious. I feel like if you were having the same conversation about, say, social media in 2010, you know, I remember there was this period where, like, everyone in San Francisco was running around using this new thing called Twitter. and they were using it to like share, you know, what they were eating for breakfast and where they were going for coffee. And I remember people saying like, oh, look at these weirdos.
23:35Kevin Roose: Like this will never catch on. Normal people don't want to share photos of their breakfast with their friends. And you fast forward 10 years and it's like the world is run by Twitter addicts. And people now, you know, to their own detriment sometimes use this to broadcast not only what they're eating for breakfast, but also like, you know, the president's on it and all the president's advisors are on it and everyone in business and AI is on it. And so I do actually think there is some predictive value in knowing what the weirdos in San Francisco are doing and how they're spending their time and what they're getting excited about.
24:12Kevin Roose: Because the pattern we've seen over and over again is that what people are doing in San Francisco is what people are doing everywhere in a few years. We'll be right back after a quick break.
24:26State Fair food is getting smaller and pricier. We'll have room for fried cookies. Okay, you never overspend on food. What's your deal? What's his deal? He went to Metro and got$450 in value on the new Samsung Galaxy A37 5G for free. And no activation fees. Free phone? Oh, sign me up. Get that more for your money feeling. Only on Metro by T-Mobile. Free your number and activate$40 period plan. $450 based on device discount waived activation fee. Not available if we're Metro or T-Mobile in the past 180 days. There's one thing that will forever change the way you sleep, and it's the softest 100 % organic cotton bedding from Bowlin Branch.
25:04You'll fall asleep faster, wrapped in the most luxurious comfort. You'll never sleep hot again with airy blankets, cloud-like duvets, and breathable sheets. You'll experience pure comfort on night one and feel your bedding get softer with every wash. Discover the difference with 15 % off your first order at BolinBranch.com with code BUTTERY. Exclusions apply. See site for details.
25:29A lot of the people in this book, and Dario Amadei, again, comes up, you know, liken what they are doing to the Manhattan Project, right? To the creation of the atomic bomb. And I'm so interested in how they think that through, because I think it's hard to unpack a lot of the legislative proposals that we've started to see come out and the fact that people in Washington are talking about AI regulation. And like when you're dealing with something that is both knowable and utterly unknowable, right? If let's say there's recursive self-improvement and things keep going, how to actually rein that in?
26:08Like how do you think they reckon with that possibility and responsibility?
26:14Kevin Roose: I think it varies by company. I think there are companies like Anthropic that have been very active and were very early in their life as a company in trying to shape AI regulation. They were instrumental in getting the chip export controls put on NVIDIA and other American companies to stop them from sending their most advanced AI chips to China. Those are still in effect. It's a meaningful difference. It's a meaningful reason why China has not caught up to America. in AI. They just don't have the compute for it. But I think they sincerely believe that this will be a matter of national importance, that it will be as strategically valuable to have the most advanced AI as it was to have the most advanced atomic bombs, to have atomic bombs at all in World War II.
27:13Kevin Roose: It reshaped the global order. It arguably won the war. And so I think there are people, including Dario and his fellow executives there, who see what they're doing as very much akin to the Manhattan Project. They are building this technology. They feel like it is their responsibility to build it before the bad guys do because they think it's going to remake the world. But of course, one major thing, right, is that like, I think you could arguably say no one in the government truly understands what they are doing. Yeah, I mean, that is absolutely true. And then we get to this like self-regulation question.
27:52Kevin Roose: Right. The biggest difference between the Manhattan Project and Anthropic is that the Manhattan Project employees were government workers. They were responsible in some very direct way to the public sector and to the public. The public sector has just completely dropped the ball on AI. All of the important development has taken place at private companies. And more than that, the government has also failed to bring in enough people who understand the technology to be able to effectively regulate it. Well, and the ones who were there are, frankly, accelerationists. Yeah. And the sort of right wing, you know, what my co-host Casey calls the Andreessen Horowitz wing of the Republican Party has ascended and is basically hell bent on removing any roadblocks to continued acceleration.
28:42Does that frighten you?
28:43Kevin Roose: Of course. It's terrifying. I think it is untenable, ultimately. I think there will be something, some calamity, I hope it's a small one, that makes that position seem ludicrous. But yeah, right now we are just careening toward the cliff. And I think in the absence of federal regulation, we'll probably get some stuff at the state level. I think the AI companies themselves will have to do some amount of self-regulation. Because if you leave it up to the Andreessen Horowitz wing of the Republican Party, you're not going to have anything. And it's going to be too late by the time you realize that you actually do need some guardrails in place.
29:23like there's this moment in your book and i wanted to get it right it's in february of 2025 and like the what the frontier red team at anthropic realized this that one of their
29:35Kevin Roose: models can make a chemical weapon yeah i mean that was and they're just like oh my god everybody into the hotel room let's try and work that out yeah and and frankly it's not even clear who they would have called at the government to have that message heard and acted upon because we just don't have the capacity in the government to respond to that kind of thing. We barely have enough capacity in the government to run tests and evaluations on these models to see whether they're dangerous or not. So in the absence of any public sector capacity, the labs have had to kind of build their own regulatory infrastructure, including these so-called frontier red teams who do test the models for dangerous capabilities before they're released.
30:16Kevin Roose: But it should not be up to them. We don't let airplane manufacturers test their own airplanes. We have the FAA and the NTSB. We don't have pharmaceutical companies just asking us to take them at their word that their drugs are safe. We bring in the FDA to run the trials and conclude whether they're safe or not. I think one of the things that is confusing, right? Because I don't think anyone would would listen to you or listen to any of these CEOs and say, yeah, that's not scary, is that there's so much focus on, understandably, extinction risk, you know, the P-doom, but less on near-term harms, the people who have developed AI psychosis, the teenagers who have killed themselves.
31:00And I wonder if you think that's one of the reasons why people are more skeptical of the messages they hear from the frontier labs about these bigger questions, because it seems like they yada yada'd past some of the other ones.
31:15Kevin Roose: I just don't think that's true. Like I, I'm, I talk to these people constantly. I understand what they're worried about. They are worried about all of it. They were worried about the near-term psychosis stuff. They are worried about the long-term existential stuff. It's funny, actually, if you go back like a year or two, maybe not even that far, maybe like six or eight months, people were saying, you know, skeptics, people who just don't buy the existential risk stuff, they were saying, oh, well, these doomers, they keep talking about these like science fiction risks, like automated cyber attacks taking down the electrical grid.
31:53Kevin Roose: But they're not focusing on the problems in the here and now. And then you fast forward to today where people are saying that literally the same people are saying, well, we shouldn't talk about these far-flung science fiction risks like bioweapons. We should talk about the near-term risks like AI taking down the electrical grid through an automated cyber attack. So it's like, yes, it is, I realize, like, these things are not happening literally at this minute. But that does not mean that it is a long-term risk. It means that it might be a month or two away from happening. And so I think there are basically people who just believe we should only worry about things when they are staring us in the face and already causing catastrophes.
32:33Kevin Roose: And I am not one of those people. I like to worry about things that might happen a month or two from now. And I don't think we're in the realm of science fiction to worry about this stuff happening. If AGI exists, what would you use it for? I think it does exist. You do? Yeah. Tell me more. I use it for everything. Everyone's got a different definition of AGI these days. But the basic idea is you have a system that is capable of doing a broad range of tasks at or near human level. And we have that today. I use AI to, you know, I'm starting a company. We use it to, you know, draft legal documents, to look at contracts, to deal with just basic office tasks.
33:21Kevin Roose: I have little agents that I can sort of hand off tasks to and they go out and do them. and sometimes they make an error, but then again, humans make errors all the time. And so my bar for AGI is not this thing has to be like a godlike super intelligence. I think it just has to be like - It's not the president's bar. No, it's not - You're not calling it super intelligence. Well, it just has to be like pretty good at a lot of things. And I think we have systems today that I would classify as pretty good at a lot of things. But then where do you not use it? Like as I'm curious, as a power user, are you like, oh no, this has to be me?
33:51Kevin Roose: I don't use it to write. So that's an area where I've just decided Like I will use it for all the parts of writing that are not writing. But you have a long part in the front of the book about that. Yeah. I use it for research all the time, constantly transcription, fact checking, editorial review. But I actually think it's important for me as like a discipline and a practice, something that I make my living doing to like sit down and write the words myself. there may come a day when that's not necessary where I don't have any advantage where I'd be making things worse by getting involved rather than outsourcing but for now I still feel like I have something to add to that creative process so I don't use it for writing I don't use it for like my my kid for like teaching my kid I mean he's only four but he's little yeah I still feel like that you know we use it for other things I've used it to like make little like songs that are amusing to him or coloring book pages uh for you know whatever he wants to draw that day but I don't use it for like like reading him bedtime stories or something just because I feel like that would be depriving him and me of something that we both value other than that I'm using it for most things are you I'm a mix I think I probably use it less than a lot of other people.
35:11I use some for interior design. Here are the dimensions. What would a custom wardrobe look like in this room? And for some research, pull me six cases where section 230 did X, Y, Z. But yeah, and for transcription. But I don't use it to write like you. I think I use it a lot less than a lot of other people.
35:33Kevin Roose: Can I ask you the same question I asked Neelai, which was, what would you have to see? What is a thing that AI could do in your life that would make you go, oh, this is AGI or this is as smart or smarter than me? Reliably reduce my mental load. Like I know that that's what Marissa Mayer's new company, right, like she wants that to happen, right? It's like targeted toward women like me, like mothers with jobs, right, who have like 18 different things in their heads all the time. But I've yet to see something that can stay true enough to who I am and come up with ideas that feel interesting to me.
36:17Kevin Roose: Okay, that's a pretty good definition. I'll check back in six months and see how it's doing. I like Lizzie Bench, they're calling it. Kevin Roos, thank you so much for coming on. Thanks for having me. This was great. Kevin Roos is the author of The AGI Chronicles and the co-host of the podcast Machine Gods. And that's the show for today. Just a reminder, please fill out the listener survey. It helps us so much. You can find that at slate.com slash whatnextsurvey. Again, slate.com slash whatnextsurvey. What Next TBD is produced by Evan Campbell, Patrick Fort, Madeline Thames-Ducharm, and Micah Phillips.
36:56Paige Osborne is the senior supervising producer of What Next and What Next TBD. Mia Lobel is the executive producer of podcasts here at Slate, and Ben Richmond is our senior director of podcast operations. If you want to yell at me about AI or anything else, you can track me down on Blue Sky, where I am at Lizzie O 'Reilly. Thank you so much for listening. Talk to you soon.
37:44We'll see you next time. Get AI tools for work. Upgrade to Google Workspace with Gemini at goo.gle slash workspace with Gemini. That's goo.gle slash workspace with Gemini. Some days, getting the work done isn't the problem. Staying locked in is. AlphaBrain is Onnit's daily nootropic formula for cognitive support, designed to support memory, focus, attention, and mental clarity. Its formula with science-driven ingredients include theanine and huperzine A to support focus and memory, and it's caffeine-free. So when you're deep in the work, AlphaBrain helps support enhanced productivity and tune out distractions.
38:21Get focused on what matters. Shop AlphaBrain now at onnit.com.
What is AGI—and is it crazy to say it’s already here? Anthropic’s targeted IPO valuation indicates that many don’t think it’s far off.
Guest: Kevin Roose, author of “The AGI Chronicles”, former NYT host of “Hard Fork,” current host of NPR’s “Machine Gods.”
Want more What Next TBD? Subscribe to Slate Plus to access ad-free listening to the whole What Next family and across all your favorite Slate podcasts. Subscribe today on Apple Podcasts by clicking “Try Free” at the top of our show page. Sign up now at slate.com/whatnextplus to get access wherever you listen.
Podcast production by Evan Campbell, Madeline Thames-Ducharme and Patrick Fort.
Paige Osburn is the senior supervising producer of What Next and What Next TBD.
Hosted on Acast. See acast.com/privacy for more information.