The Scientists Studying the End of the World

0:06Hey listeners, it's Tyler. The political scene is off this week for the 4th of July. But given everything that's been happening in the world right now, we thought it might be a good time to revisit a conversation I had last year with New Yorker staff writer Rivka Galchen. Rivka and I discussed a piece she had just written about a University of Chicago course dedicated to studying the world's various existential threats, from AI to nuclear war to climate change, and which one might actually bring about the end of the world. Here it is. Just like AI was able to come up with an antibiotic we'd never come up with, it would be able to do the same thing for a pathogen.

0:45I'll take the transmissibility of this and the virulence of this and put them together and it's like a home CRISPR, kind of. New Yorker staff writer Rivka Galchin has a story in this week's magazine that looks at the world around us and asks, are we doomed? Tonight, a stark warning that artificial intelligence could lead to the extinction of humanity. Russia says that it has begun tactical nuclear weapons drills close to Ukraine. Ocean temperatures are warming at an unprecedented rate with 40 % of global oceans currently experiencing a heat wave. Rivka often covers offbeat science stories for the magazine.

1:26Most recently, she wrote about the field of existential risk, a discipline that examines the threats to life on Earth from climate change to nuclear warfare. I wanted to speak with her about how we can think through something as unfathomable as the end of the world. You're listening to The Political Scene. I'm Tyler Foggett, and I'm a senior editor at The New Yorker.

1:54Hey, Rivka. Thanks so much for coming on the show. Hi, Tyler. Thank you for having me. So to report this piece, you attended a University of Chicago class called Are We Doomed, which looked at the most serious threats to our world and at how we think about them. What are these threats, at least according to the class? Ah, well, it's not an exhaustive list, of course. But I came to understand that the field of existential risk does cluster around nuclear threats, biothreats, climate change threats, and emerging technologies, which currently is dominated around a discussion of AI. Basically, threats from things we don't understand yet and we don't know what might happen when they become pervasive.

2:36You mentioned this term earlier, existential risk. Can you explain, like, what that means? I guess I'm wondering whether we're talking about total doom as in like human extinction or something that could just be like a fundamental threat to the way that we live. One person who came up in the course a lot is Toby Ord, who's a central thinker in this. And in his book, The Precipice, he makes a point that he wants to think not only about extinction, but he wants to think about something that would prevent humanity from being able to bloom, for lack of a better term. And so he's sort of including events that basically make it impossible for humans to flourish.

3:18Are we not already there? Yeah, that's a good question. You know, I think about, you know, we talk about climate change and how fewer young people seem interested in having children for that reason. Although, interestingly, in the piece, and we'll talk about this later, but it seems like some of the young people in the class that you sat in on are actually not as pessimistic when it comes to, you know, reproduction and whatnot. But yeah, I guess how far down the path of annihilation are we? Well, you know, one thing I loved about this course was that they didn't really have the answers. So it's more like a kind of cacophony of the whole spectrum of people saying, you know, this AI thing, people can't even come up with a scenario I find plausible.

3:58An expert will say something like that. And another expert will say, basically, I don't even know why we're talking about this. It's over. It's unpreventable. So the fact that people who have devoted enormous, enormous amounts with their time and intellect, that they're disagreeing, I'm not saying that that's a source of hope, but I thought that that was an interesting and important distinction. And I remember around the conversation of nuclear, Governor Jerry Brown came to visit the class and he was sort of talking about conversation with Admiral Perry, who he admires very much. And he said, well, I mean, do you have any hope?

4:34He's like, yeah, I do have hope because the future hasn't happened yet. That was like the only kind of positive gloss he had to offer on nuclear. So it sounds like Brown was like an expert speaker who came to the class, but who actually teaches the class on a day-to-day basis and what is the format of it? Can you like set the scene a little bit for us? Absolutely. So just like existential risk calls on thinkers from many different fields, it was sort of handy and apt that the course was taught by two professors in very different fields. Daniel Holtz is an astrophysicist, publishes a lot of work on black holes and gravitational waves, but also has a role on the Bulletin of Atomic Scientists, which is a group at the University of Chicago that's been there since shortly after the Second World War that has basically been thinking about this question.

5:24We actually have some audio of Professor Holtz talking about the class, which listeners can hear now. Unsurprisingly, there are aspects of the class that are somewhat of a downer. Many of the experts are completely freaked out and worried about the future prospects for humanity. There are so many profound interconnected risks, and there's a consensus that we're at a uniquely dangerous moment in the history of civilization. So you might think that the class is pretty dismal, but I find it to be profoundly hopeful. The students are inspiring. They're engaged. They're highly capable. They really care.

5:59And they want to make a difference. Yeah, there's no doubt that the situation is dire. But a running theme of the class is that we have the capacity to address all of these threats. We just have to take the risks seriously and do the work to avert disaster. So he was one of the professors. and then the other professor James Evans is also sort of someone whose list of interest and expertise is quite overwhelming. So he's both a sociologist and a computational scientist and who kind of comes at these questions from many different angles but not just his expertise, his personality I think is an interesting part of how he comes at the questions which is kind of like, you know, he's the sort of guy in the kind of 1930s movies who falls down a chimney and then pops back up and is smiling.

6:47He has a sort of playful, totally non-fear-oriented way of interacting with the world. And here's Professor Evans talking about the class. I was interested in designing and teaching this class with Daniel because it focuses on the things that matter most for our species, surviving and thriving at a time of increasingly interconnected risks and emerging threats of our own making. Moreover, knowledge about these possible futures are beset with uncertainty. Many of our tools for scientific knowing reach their limits here. We don't have many human experiments. We have one. If it fails, we cannot run it again.

7:25Across the fossil record, the average lifespan of a mammalian species is 3.5 million years. From this perspective, humanity is a toddler, and it feels almost inconceivable what our world will look like in 100 years, let alone 3 million. As countries and communities, we are making short-term tribal decisions. But how can we think and responsibly act as a species into the long horizon? I was excited not only to teach about risks in this class, but unleash our students' imagination about how to think and know together at a planetary scale so we can make wise decisions for the long term. These professors are intense.

8:05I mean, obviously they're brilliant, but I'm also suddenly remembering why I was an English major. Of all the existential risks that the students mulled over, what subjects were they most engaged by or concerned about? Yeah, you know, they did like a survey several times in the course of the class. So in the first class, they sort of asked the students to rank their, you know, rank your existential risk. What are you most concerned about? Climate won by a lot. And then as the course evolved, and this might be the influence of the different guests they have or the influence of the world as it is, nuclear just sort of sailed to the top, which, of course, is something that I think that generation and frankly, my generation thought of as sort of in the rear view mirror mysteriously.

8:54I don't know why we thought that. But, you know, it's something that the older generation, when I talk to people 20 years older than me, they talk about having so many dreams about nuclear Armageddon as like a 10-year-old. And I don't think we grew up like that. I'm wondering why it is that we aren't talking about nuclear warfare more often. I mean, maybe it's just not my social circle that's talking about it, but it seems like there was a point. I mean, I think it was like shortly after Russia invaded Ukraine and we were talking about Western support for Ukraine. That was something that was very much in the news, but not so much anymore.

9:29There was a lot I learned on that that I had both never thought about and certainly didn't know. And Jerry Brown was actually quite great at talking about this. Like he said, you know, it's been 75 years since it's been dropped. That gives you a weird sense of confidence. And he's like, that is just pure luck. And he sort of brought up several instances of which there are many where one or two people were the kind of bridge between something horrible happening or not happening. And he really emphasized and emphasized how many other thinkers on this feel this way, that it was a launch by mistake is the most likely thing to make this happen.

10:09So he said he goes to Congress, he tries to talk to everybody about it, and everyone looks at him like he's just a total fool. Like he really, like, he said it's not just you guys who aren't thinking about it. Like he's like a preacher for it, and there's no one sitting in the pews. It's like a really empty church. And yet his argument was very persuasive. What were you left most concerned about after observing the class? I mean, are you now very concerned about a nuke going off? I'm just curious about what you thought the most alarming thing that was brought up during the sessions was. It was interesting because all of the speakers and even some of the readings on AI were so charming and witty and smart that in some funny way it got in the way of being able to take it as seriously.

10:56even though it was sort of compelling in the sense that it was formless. And, of course, in terms of maybe fear rather than risk, formless things are, of course, very fearful because we kind of can't draw a box around them. But I guess related to that, I was very compelled by the synthetic bioweapon becoming a very easy thing to make with AI. So that was the one that was, I guess, because I'd never really thought of it before. Can you explain that a little bit? Is that like manufacturing a like a virus or is that like, you know, the milk is poisoned, like synthetic bioweapon? It's not a phrase you hear that often.

11:33Yeah, and I'm sure I'm being. It brings a lot of really terrifying things to mind. And so I just want to make sure I'm terrified of the right thing. Yeah, no, it is basically building a sort of extremely lethal pathogen or getting a few copies of an extremely lethal pathogen that is also highly communicative that will make COVID seem like sort of nothing. and, you know, throwing it out into the community. Can you say more about how AI would be involved in this process? Like why the idea of a synthetic bioweapon, especially one that's, you know, manufactured on a large scale, is just so much more possible with AI?

12:06Well, I think there's many different mechanisms that it could be, but like one that's maybe very easy to picture is, you know, we're already using AI in a kind of exciting, positive way where it's like coming up with an antibiotic we'd never come up with before because it can sort of fold and change and go through all these kind of just a much, much larger set of ideas to come up with something. So just like AI was able to come up with an antibiotic we'd never come up with, it would be able to do the same thing for a pathogen. Like I'll take the transmissibility of this and the virulence of this and put them together.

12:42And it's like a home CRISPR kind of. Totally. No, it makes me realize that when we talk about the threat that AI poses, we're so much more focused on short term. I guess it's not a small thing for AI to like, you know, completely destroy journalism. But I feel like we're so focused on like, what will it mean for Amazon reviews? And will we not be able to trust them anymore? And meanwhile, you know, there's the potential for this. So I'm glad that the students at UChicago are focusing on the right things. One of the concepts that comes up a few times in your piece is this idea of nonlinear outcomes.

13:16And I was wondering if you could speak about that a little bit. Yeah, I love the word nonlinear because it gets like a pretty big idea into, I can't count the letters very quickly, but like 10 or whatever, a really small number of letters. You know, in our life, we're very accustomed to things that change a bit and then change a bit more and then change a bit more. and we're very accustomed to things that go up and they go down and they go up and they go down, like stock market or something like this. But we're less accustomed to, even though we talk about it in relation to the environment, like a total systems collapse where like a small change makes not much of a difference, makes not much of a difference, makes not much of a difference, and then an entire set of species sort of from the apex predator on down is sort of destroyed and gone.

13:59It's very difficult to model what are those variables, what are those moments, Like, what will be the final break and then, you know, the current that will change directions and disappear? Like, I just think it's a very useful word. It's sort of capturing a sudden and dramatic change. I guess I'm wondering how you think that we should think about that phrase in relation to existential risk. Like if, you know, the existence of nonlinear outcomes is the kind of thing that makes it extremely difficult to calculate how close we are to doomsday. Or if it's more likely that if we do reach that point of doom, that it'll be because of something like a nonlinear outcome where it's like you start with something that seems very mundane and then all of a sudden it, you know, destroys a species.

14:47You know, I don't feel confident about sort of talking about like how that should frame how people think about the field of existential risk or even how people personally think about troubling trends or troubling changes. But I did think it was just like a kind of useful lens to put over the thinking so that gradualism doesn't seem like the norm. And then I do feel like there's sort of people who spend a lot of time, including Professor Evans, actually trying to identify what is it that will make a system, whether that system is an institution, whether it's an ecology, whether it's a weather system, whatever it might be, what makes it sort of suddenly collapse or suddenly explode.

15:31And there are people who spend a lot of time thinking about that and thinking through and trying to identify those critical points. So Rivka, I'd like to talk to you more about how young people are thinking about our possibly cataclysmic future. But first, we're going to take a quick break. You'll hear more of the political scene from The New Yorker in just a moment.

16:06So one of the more surprising things in your piece is that the students in the classroom seemed somewhat upbeat. You know, these are students who are all young and who may live to see some of the worst outcomes that they're talking about in this class. And yet they just seem kind of undaunted. And I'm wondering if you think that's because, you know, if that's just like an atmosphere that was kind of cultivated in the classroom or if there's something else going on. I mean, there must be some strange selection bias, like maybe the people who signed up for that course, although my initial thought might be that these would be people sort of trembling in their shoes, is not actually the group of people who would want to sign up for the class.

16:43That's a good point. It's like if you're really concerned about nukes, you're not going to go take a class where you have to think about them all the time. You'll do what I do and work here and, you know, edit pieces about other things. Yeah, maybe. I actually felt like it was almost like the students were much more buoyant than the speakers and maybe than almost all the adults in the room. I mean, they were adults as well. That's fascinating because it seems like so much of the political discourse right now is focused on the idea that young people, particularly young people who are thinking about voting, are disillusioned right now and pessimistic.

17:20And so I guess I'm curious about how to reconcile that general belief or maybe it's a misconception with what you were seeing in the class. Yeah, I mean, I wish I confidently felt the class was generalizable because this was, I don't know, a very impressive, thoughtful, creative, kind of amazing group of people. So, you know, that would be great if like that really is very representative. And I'm sure that the sort of generalized media depiction of kind of TikTok watching nihilists is also incorrect. I have my own, of course, like personal psychological speculations thinking about what's changed in my head from when I was young to when I was older.

18:05And when I was young, in a wonderful way, things were sort of real. Everything seems sort of reversible. And there's an enormous strength in that, even if it's not necessarily accurate, because of course, now that I'm older, I also have this inaccurate sense of almost total lack of autonomy, which is not true either. And so, I don't know, I'm always hesitant to say anything, as we all are, about like what it is to be young or what it is to be middle-aged, what it is to be old. But I do think in a frightening way and in a totally wonderful way, there's something plastic about reality when you're young.

18:43You feel like you can change it, which is, of course, very true, but not totally true. And also like that you can kind of get a do-over. I feel like you still have that. But I remember as a kid when someone came into our house and stole all the money a couple times. And I remember being confused that we weren't giving back the money. I don't know. I just remember thinking, but that's not fair. Money was taken from our house. Why aren't the police reimbursing us? Obviously, most people are smarter than that. But just that sense of that there's a certain way things should be and that they will be that way if we just sort of push up against it.

19:24And there's some truth in it. One thing that stood out to me when reading your piece is that the class discussions didn't seem that overtly political. Like there's one student who argues for a progressive response to climate change, but that's basically it. There's no mention of, you know, like the election, even though there's a lot of talk about how crazy it is that a president can just press the nuclear button and that's it. And so I'm wondering why you think that's the case. You know, that was a question that was on my mind again and again, because like almost everyone my age and many ages, like the sort of very present tense, very American political discussion feels, you know, at the top of my head in the morning and then in the afternoon and then in the evening.

20:08And I sort of asked the students several times, like, did they think there should be more presence of politics in the class or was it something that was on their mind? And I mean, I got a pretty across the board, like, you know, it's just not a very productive conversation. I don't even talk about this with my friends. It's just, you know, people have their different taste. Like it did feel sort of like a no-go zone, which I'm sure is not universally true among students or among young people. But it just seemed like everyone had this idea that there was something to be gained from a different aperture.

20:48You know, I asked the professors about it, too. I think they were just like, there is a different aperture. And one thing that I thought was interesting to me that I noticed inside of myself over the course of the term was I thought, like, it seems so weird that we talk about the election in terms of like, basically like two dudes. because a lot of the thinking was about seeing sort of a pattern and seeing like a pattern of the structure of how things work. And I just thought, oh, this is really a sort of steamship this way or steamship that way. It's kind of almost irrelevant who the person is and our long line of kind of agreements and disagreements and approval and disapproval.

21:32Like it seems so much smaller than overall, like, you know, can kind of go plinko board down one path, or you can kind of plinko board down the other path. It made me think about a Bill McKibben piece that ran while I was attending the course. He was just like, this is a climate election. This is a climate pivot point. It can go this way or it can go that way. I mean, did he even mention Trump or Biden's name? Like, it was a different aperture, and I thought it was kind of clarifying. I guess I want to push on that a little bit. I guess I'm wondering if it's truly a different aperture or if it's basically a progressive aperture without saying the word.

22:10Just because when you think about concerns over environmental calamity, you know, you kind of associate these ideas with one party more than the other. And maybe that doesn't necessarily need to be true, you know, for the rest of history. But I feel like right now it's like if you are signing up for a class where there are concerns about climate change, you can just assume that the people who are talking about these concerns will probably be liberal, right? Well, yeah, and you're at the university. I mean, you've got all these sort of demographic things like that. But, you know, I mean, I had some strange and, to be honest, unsettling conversations with students saying like, well, you know, four years of one, four years of the other.

22:48What was the difference? I mean, my head almost exploded because I had such a long list of differences. But I mean, I think at this moment in history, there is one party that is just like so much further away from reality and transparency and honesty. You know, they're both political parties. They're like full of millions of problems. But one of them is so deranged. And as Noam Chomsky said, like the Republican Party is the greatest threat to the planet Earth that has ever existed. And I don't know, it's really hard to pretend that there's a sort of interesting other way of framing what is likely to happen the more red power is held at the federal level.

23:34You know, it just seems like, I don't know, how do you have that conversation and feel like you're being honest? Well, even though the students didn't seem that interested in classic party politics, it does seem like in order for them to address some of their biggest concerns in terms of doomsday threats, they would still have to work within the political system that we have. And I'm wondering what kinds of ideas came up in the class about how our system should respond to these problems. Oh, yeah. Every week, always at least one of the readings was a super pragmatic reading. reading. So whether it was AI, you know, there is a Center for AI Safety.

24:10They've sort of outlined interventions that they think would make a big change. And the same thing for the week of nuclear. There's a lot of kind of wonderfully boring ideas about how to change certain systems to reduce the risk. That was always like one of the elements of the readings. We'll have more after the break.

24:49So obviously it's a very complicated, somewhat reductive question to ask how close are we to the end. But how close do the doomsday experts think that we are to total annihilation? Yeah, so that's always fun to think about. And it coincided with the beginning of the course that the Bulletin of Atomic Scientists, which makes an annual announcement about sort of how far we are from midnight. It's like a Christmas card. Exactly. So they make this annual announcement after having many, many, many, many, many conversations with and between experts. And when the doomsday clock was begun, it wasn't actually begun as a clock that was going to move backward and forward.

25:31It was just sort of the cover of an issue of the Bulletin of Atomic Scientists whose goal was to sort of bring awareness to the horrors of nuclear weapons. But, you know, it was a clock and it sees the imagination. It was set at seven minutes to midnight. It just seemed whatever. It just seemed like that seems really close. I thought it would be like an hour from midnight or if it's seven minutes to midnight, that would make it seem as though 2025 is like not going to come. Right. Exactly. Exactly. Exactly. And one of the students sort of pointed out like one of the problems with the clock is we just only think of clocks going forward.

26:04And the clock actually did go backwards. It's sort of. I see. It went backward towards the end of the Cold War, I think. I can't remember if it was 13 or 17, some beautiful prime number of minutes away from midnight. So that was like the kind of furthest back it ever was. And this year they announced it's 90 seconds to midnight. And last year it was also 90 seconds to midnight. And the year before that, it was 100 seconds to midnight. And, you know, they're very talk a lot about how this is, you know, it's a talking point. It's not quantifiable, really. There is no sort of great way to feed a lot of numbers into a computer and come out.