What Does Effective Altruism Really Believe?

0:27Derek Thompson: Thank you. soon. That's theringer.com slash events. Again, plain English, we'll be live at the Howard Theater on December 8th, and you can grab tickets starting Thursday. See you there. On September 16th, 2026, the Trump Department of Defense, aka Department of War, posted that the United States will never be an effective altruist country. This came with an image of Trump pumping his fist with the tagline, Americanism, not effective altruism. Two weeks later, the cover of The Economist declared that effective altruism was the 21st century's biggest idea. What is effective altruism? I am in a strangely unique position to begin to answer this question.

1:14Derek Thompson: In 2013, I was living in New York, and my roommates left to start a company in Africa. Some subletters moved in. Their names were Will and Amanda McCaskill, now Amanda Askell. She is currently the lead philosopher at Anthropic, and he is known as one of the founders of the effective altruist movement. Over late night whiskeys, I got a front row seat at the emergence of a philosophy that has asked people to think harder about the consequences of our actions. What began in the early 2010s as a nerdy way to rethink the most cost-effective charity has instead become a lusher philosophy to rethink just about everything, our consumer habits, our career choices, our technology, and our approach to the most existential issues of our time.

1:58Derek Thompson: Today's guest is Will McCaskill. We talk about the birth of effective altruism, its history, its insights, and its blind spots, its high-profile challenges, including the conviction of Sam Bankman Freed, a major funder of the movement, and what effective altruism says for this moment in history. I'm Derek Thompson. This is Plain English.

2:48Derek Thompson: This episode is brought to you by Brought to you by ServiceNow. Look, I have my dream job. I get to explain complicated ideas to folks who have better things to do than read white papers. But even dream jobs have not-so-dreamy parts, the stuff that gets in the way of the actual work. That's where ServiceNow's AI specialists come in. They don't just tell you what you should do about your busy work. They actually do it, start to finish, cases closed, requests handled, no extra work for you. That way you and your team can spend more time on what matters, which for me is finding that one elusive stat that just makes everything click.

3:26Derek Thompson: To learn how to put AI to work for people, visit servicenow.com. This episode of Plain English is presented by Comcast Business. Did you know Comcast Business detected 79.3 billion cyber threats last year? That's why they're introducing Security Edge Preferred. It's big business cybersecurity designed for small business. Get started with Comcast Business Internet for$50 a month for one year on a qualifying internet package and find out how to get one month of Security Edge Preferred on them. Plus, ask about a$500 prepaid card. Limited time offer, restriction supply, requires 300 megabits per second internet, Security Edge, Security Edge Preferred, and Wireless Connect.

4:09Derek Thompson: Call today. Will McCaskill, welcome to the show. Thanks so much for having me on. So, Will, you're considered one of the founders of effective altruism, so you're probably a good person to ask. Where did effective altruism come from, and when did it come from? So I'll tell you about at least my part of the story, which for me began really in Oxford, UK, in 2009, where the intellectual underpinnings really came from Peter Singer and a series of arguments he'd made for the idea that if you live in a rich country, because you can help people in poor countries so cheaply at such little sacrifice, you are morally obligated to donate a large fraction, maybe even the majority of your income, to those non-profits that could help the poorest people in the world.

5:10and I had been very influenced by that idea but had done very little until I met another graduate student in Oxford at the time, Toby Ord and he had spent years thinking about this, taking it very seriously. He had made a pledge to give everything he earns above a fairly low baseline of about£20 ,000 per year, Oxford 2009 adjusted and he had also been doing a lot of work a side of things that I had not appreciated, but I think became the distinctive aspect of effective altruism, which was thinking about not just how much should you give, but also where you give and what are the charities that do the most to help other people with every pound or dollar they receive.

5:53Derek Thompson: Someone listening could think that you're basically just applying philosophies otherwise known as utilitarianism or consequentialism to philanthropy. Why does effective altruism have a name at all? And why is it effective altruism? So I think there's a couple of things there. The first is that all ethical perspectives should care about helping others. And in fact, almost all do. And as long as you care about helping others, whether you're a virtue ethicist or a libertarian or any other model perspective, well, then you should care if you can help more people rather than fewer, or help by a larger amount rather than less, then surely that's actually important.

6:36But the striking thing in at least academic ethics over the last 50 years was that even though there'd been a lot of discussion of Peter Singer's arguments in this area, there'd actually been very little work done to really work them through and figure out, okay, well, if you do want to help other people by as much as possible, what should that involve? And so effective altruism was almost called positive ethics. So academic ethicists normally talk about thou shalt not. And this debate kind of, does that apply to abortion? Does that apply to animals? But there was very little on the kind of thou shall, actually.

7:17If you want to help others by as much as you can, what in fact should you do? And so I think the distinctive part of effective altruism was really taking that aspect of things seriously and really trying to work through as thoroughly as we could the effectiveness question. The question of just, you have limited time, you have limited money, but you want to have as much positive impact as you can with those scarce resources. What should you do?

7:45Derek Thompson: Four years later, in 2013, we were roommates and my mother passed away the previous summer before you moved into my apartment in New York City. and my first interaction with effective altruism was you pointing out that many people who want to do good in the world via philanthropy will act in a way that is often shaped by social status or by the desire to do something that is sentimental but might not be consequential and you said there are so many opportunities Derek to donate money in a way that can actually save lives and I was incredibly motivated by this, gave tens of thousands of dollars in my mom's honor to the Against Malaria Foundation to buy malaria nets.

8:27Derek Thompson: I was also very interested in arguments that you made outside of philanthropy. If I recall, for example, we talked about the idea of choosing what to do in a career. You said, you know, if you go to work for a philanthropy as, you know, the some grunt at maybe PETA or some other do-good philanthropy, if you don't get that job, Someone just like you is very likely to get that job. But if you get a job at Goldman Sachs and you give away 60 % of your income, you're probably donating much more of your income than the next guy up or next woman up at Goldman Sachs. And I use that story as a door into this other question I'm interested in, which is how did the ideas of effective altruism escape containment from the world of philanthropy and end up walking into all these other spaces?

9:14Derek Thompson: how we should act as consumers, how we should think about our careers as doing the most good. When did those ornaments start to get hung on the EA tree? Yeah, so in terms of the history, in 2009, it was very focused on giving. And I set up Giving What We Can, which was encouraging people to give at least 10 % of their income to those most effective charities. But we were talking a lot to students. I was myself a PhD student at the time. And there was this question of just, well, should I become a philosophy professor? Is that the best thing to do? An undergraduate was wondering about whether to go into doing a PhD himself or to go into investment management with the aim of earning to give and so on.

10:01And because there was this demand for just, look, this is a question that many of these students actually need, are grappling with, and where they do want to do good with a career, this undergraduate, Ben Todd, and I gave a lecture that turned into a series of lectures that then turned into this organization, 80 ,000 Hours, that was trying to take this same effectiveness-oriented mindset, how can you do the most good with your scarce resources, but applying it to time as well as to money. And why don't you tell us a little bit about what 80 ,000 Hours means and what that organization does? So 80 ,000 hours refers to the number of hours you'll typically work over the course of a career.

10:42And the thought there is twofold. One is that man has a lot of time. So it would be worth spending many hundreds of hours to try and figure out, okay, if you spend 1 % of time, that's 800 hours. That's many weeks of time to figure out how to spend the remaining 99%. But yet many, many people at least just kind of sleepwalk into some career or other because it's the consulting job that they're expected to go into or what their parents wanted them to do or something like that. But then the second point is that it's also just a finite amount of time. The problems we're talking about in the world, no matter how successful you are, you're not going to be able to solve them as one person.

11:26And that means you need to prioritize. You need to think, how can we use that time as effectively as possible to make the biggest dent in one of these really big problems?

11:38Derek Thompson: In about 15, 20 minutes, we're going to get into some critiques of effective altruism as it exists today. But I wonder, going back to, say, 2009 to 2013, what the most common critiques of effective altruism were back then? Because I'm sure someone listening who's a bit innocent about the concept of effective altruism is thinking one should give their money where it is most likely to save lives. One should think about what they're going to do with their career a little bit, considering that they're going to work 80 ,000 hours. These are somewhat harmless, even anodyne, even obvious ideas. Were there identifiable critiques of the philosophy that you were trying to grow in this period?

12:19There were very, very many critiques all the way back from the giving 10 % to help people in the global poor days. People did call it a cult back then, where the idea of donating 10 % just seemed a bit extreme to people.

12:42Whereas I think that's just now much more widely accepted as just like a bit normal. and, you know, I think very respectable thing to do. There were critiques along the lines of being too okay with the current capitalist system. So very common critique was like, basically, if you're not trying to overthrow global capitalism, what are you even doing?

13:10Whereas, you know, I did and still do just think that even though I'm sure there are better economic systems that are possible, that is not the most effective way of spending one's time. And then there was also a critique that always felt more like a kind of within-tent critique of just the question of what can you measure? Is there a risk that by focusing on what the data tells you, that you're going to end up focusing on what's quantifiable rather than most important? Um, that's always been just a very hard question and something we've tried to improve with more data, with more careful arguments and so on.

13:52Um, but has, uh, yeah, always been, always been something to grapple with.

13:57Derek Thompson: I want to keep telling the story of this philosophy's evolution because one might think of EA1 as being evidence-driven philanthropy. And then EA2, which crops up right around the same time, is utilitarianism doing the most good for the most people, but as a philosophy for modern life, as a consumer perhaps, if you believe that animals have conscious experience and they suffer, then you should be a vegetarian. As a worker doing the most good with your career, so 80 ,000 hours and being really purposeful about the way that you are adding value to the world as a worker. Now, this brings us to, I think, the next development of EA.

14:35Derek Thompson: We got beer, I don't know if you remember this, maybe in 2015 or 2016, And I was asking you what you were thinking about for the next chapter of Effective Altruism. And you told me that, among other things, you were very interested in this idea that there were a lot of people that would exist in the future, and we needed to think more about their interests. And this previewed what later became known as long-termism, which I have heard you summarize with three sentences. One, future people count. Two, there could be a lot of them. Three, we can make their lives better. Going sentence by sentence, how would you explain what long-termism means?

15:17Yeah, so the underlying thought is that, okay, all humans are equal. They're equal across space. So the interests of someone living in a poor country is, that matters just as much as the interest of someone living in a rich country. But similarly, the interests of people shouldn't matter across time either. If I can benefit someone in a week or a year or a century or a thousand years, the interest is the same. It matters just the same. Time itself is not really the relevant dimension in terms of how much you should care about that person's happiness or suffering. And then the second point is just, well, once you take that seriously, you should bear in mind that and take seriously the fact that man there's just a lot of future to come um i think quite plausibly we're very early on in civilization's development and so we should look ahead to thousands or millions of years or maybe even longer that uh human that human society and civilization could continue into the future and so then that means wow there's just really a lot of stake, a lot at stake, if there are things that affect how the long-term future might go.

16:35And then the final point, which is what most of the book, What We Are the Future, spends its time on, is saying like, yeah, actually, there are things that are happening in our lifetimes. And since I've written this book, I now think within the next few years, that we'll have very major impacts, not just on how well this generation goes, but how the whole course of the long-term future goes. And put that all together and you get the view that these things that will impact the long-term future, where that includes how we handle pandemics, how we handle AI, are among at least the very most important priorities of our time.

17:16Derek Thompson: What does it really mean to care about somebody who won't exist for a thousand or one million years, I feel like I can pretend to care. I mean, in an emotional way to literally care about that person's happiness. I don't know that I can do it. I, when I care for my daughter, wife, friend, it's not just that they exist here. It's that I have an enormous amount of certainty about the things that make them a person and the things that they care about. And so I wonder how you deal with this. I know very frequent objection to an aspect of long-termism, which says that we should care equally for people who live a million years from now as we do about people who live today.

18:07Derek Thompson: Don't we simply have so much certainty about the people who exist today and certainty about our feelings for them and certainty for what they want in the world that it's a different enterprise to care about someone who exists rather than conceive of care for someone who won't exist for a million years? Yeah, so I think that's a great question because it kind of illustrates a way in which long-termism gets misunderstood, I think. So I think if what I was trying to do was think, okay, what are going to be the problems that will really face people in the year 4000 AD or something, and how can I solve those problems?

18:46That would be kind of foolish, I think, because we're not going to know. They're going to be quite radically different than the problems of today, for example. However, what it does mean is that we should draw attention to risks that we face now that are short-term risks but that have long-term consequences, where that can involve a worst-case pandemic that leads to the collapse of civilization or the extinction of the human race. or an AI-enabled totalitarian state that can persist indefinitely, or to the loss of control to AI systems themselves that then mean that humans are just no longer behind the steering wheel of society at all.

19:33These are really quite broad thoughts about how future society might go, and we don't really need to know very much about what are the problems that might face someone in the year 3000 AD, to know that it's a good thing if humanity is still around and has not gone extinct or civilization has collapsed. Or to know it's a good thing if we don't live in this totalitarian state. Or it's a good thing if misaligned AIs have not taken over and now have humans as the underclass. And so it's, yeah, the kind of decisions are at that level of generality. And you might still disagree, you might still think that some of those things are good things.

20:19And then we can get into an argument about that. But I think that's at least a lot broader and kind of safer position to hold than some very particular claims about how things will go in the future.

20:32Derek Thompson: I want to enumerate some of what I understand to be the guiding principles of EA to on-ramp us to a question that I'm really, really curious about your answer to. So number one, obviously, people in rich countries like the U.S. can save or improve the lives of people in poor countries with modest donations to the right charities, one. Two, with 80 ,000 hours in similar organizations, our working careers are both finite and significant, and we should think very deeply about how to use those 80 ,000 hours to do the most good. Three, humanity can exist for a very long time unless we go extinct or civilization collapses, and so we should try to not have civilization collapse or go extinct and care very much about things that present existential risks to civilization.

21:16Derek Thompson: You've done work on this. Toby Ord has a book, The Precipice, which is all about existential risk, whether it's AI or comets smashing into Yucatan peninsulas. And then you mentioned AI, which I really want to save for the end. I am interested in effective altruism as a collection of moral principles, but I am also interested in it as a way of seeing the world, as a way of paying attention to the world. I think moral philosophies, if they're successful at any level, they almost give their adherence like a set of goggles, right? Like stoicism asks people when they're in a difficult situation, hey, immediately separate what's under your control versus what's out of your control.

21:56Derek Thompson: And like you hear athletes all the time quote back to like halftime and end of game questions. They'll just say, I got to only focus about what's on in my control. So stoicism has succeeded in giving millions of people a really clear pair of goggles. Aristotle, I think, in virtue ethics, trains people to think about virtues as you're a good person cultivating through repeated action and behavior the good, that which is courageous. What are EA's goggles? Once someone internalizes effective altruism, what do they start seeing everywhere that other people do not? Yeah, I love the framing because I think that is right for effective altruism.

22:35It's a mindset. And it's a question, much more so than it's like a set of answers or even a set of principles. So you can be into effective altruism and very skeptical of long-termism, very skeptical of working on catastrophic risks and so on. And so what is distinctive about effective altruism? One is the very basic idea of when you're thinking about, okay, I'm not too good. Is this the most good that I could be doing? or is there something even better, even more impactful? A second thing in the mindset is just taking scale seriously. So early on, we would talk about how some charities vary, you know, in their effectiveness, not just by 50 % or 60%, but they would vary by orders of magnitude.

23:25So one charity would do 100 times as much to help people as others, save 100 times as many lives. and that sort of scope sensitivity is something that is actually quite rare but I think is extremely important and then maybe I'll maybe say like another part of the mindset as well is just thinking a lot about counterfactuals what would happen if I don't do this where for some things they're really neglected if you don't work on them if you don't contribute then nothing is going to happen or very little is going to happen whereas in other areas oh there's already just a ton of attention and work so that the difference that you make by contributing might still be meaningful, but maybe it's a bit more marginal.

24:09So these are some of the kind of underlying questions that have this, that I think help to define the EA goggles, as you call them.

24:19Derek Thompson: One of the effective altruism's most famous adherents, and certainly its biggest funder for many years was Sam Bankman-Fried. And famously, FTX, the crypto exchange that he ran, collapsed. He was convicted of fraud. You knew him. What happened here from your perspective? What did you get wrong? What did the EA community get wrong about him? And what did you learn from that experience? Yeah, I mean, the whole, the FTX collapse, the bankruptcy, the fraud was just this huge catastrophe for the world. It was like a huge harm that was inflicted on literally millions of people who suddenly didn't have access to the investments they'd made in that exchange.

25:07And the whole affair, speaking frankly, is just a matter of great shame and embarrassment, both for me personally and for the movement as a whole. I did know him. He had been involved in effective altruism for many years. And then when he was setting up the company, I was advising on his philanthropy. And he became one of the biggest proponents of effective altruism. And yet, it turned out that things were not at all what they seemed. In fact, I think I'd, well, I now know that I'd been lied to on many occasions to my face by the people involved in FTX. So what went wrong? I think a really big thing was just like effective altruism started off as this just small, very tight-knit group of people who were just, you know, everyone really trusted each other, kind of like a family or something.

26:04And it grew and grew. And I think there was this sense of, well, if you're into effective altruism, oh, you're kind of part of the club. We can just extend trust to you in this kind of open-ended way. And that's certainly how I think I was acting. um and i think that's you know an error um saying you're into effective altruism or even being into effective altruism does not tell you um much about other aspects of uh someone's character um let alone kind of how they run their companies and so on and so uh i think so effective altruism after the fdx crash like as a movement made quite a lot of changes um there was big institutional overhauls in, you know, most of the leading kind of organizations.

26:55And also, I think, just moved a bit away from being a kind of community to more like a professional network or something, more like, look, this is something we do, but the fact that you say you're an effective altruist doesn't mean you, like, suddenly should be extended kind of a lot of trust. And so I think as a whole, it was just, um, yeah, a hugely painful and important kind of growing up moment, uh, where we could move them. Yeah. Being that like small fledgling group of people with a set of ideas who all felt like they could, uh, trust each other to something that's kind of, you know, needing to be a bit more engaged in the real world, take, um, take issues like in particular kind of institutional governance just very seriously indeed and not rely so much on kind of trust networks, I think.

27:49Derek Thompson: Yeah, it's interesting to hear you talk about something like institutional governance, which is something that almost by definition can't scale, right? There's a way in which while I don't think someone has to choose entirely between Aristotelian virtue ethics and effective altruism. The weakness of virtue ethics, which is also a strength, is that it's not interested in scale, right? When you say, I want you to model your sense of courage and honesty on the people that you are surrounded by who you believe to be courageous and honest, that's not a feeling that can scale. That's behavioralism. That's just cultivate this thing in your life.

28:33Derek Thompson: And that's different than try to do the most good for the most people and associate the good with a mechanism that can scale. And so I wonder if the SPF catastrophe made you update about effective altruism at all, whether you feel like it exposed a flaw in the philosophy itself, or whether you still believe in the exact same cause that you did before Sam Bankman freed, but simply have a different attitude toward trust in issues like institutional governance while still believing all the things you did before the fraud was exposed? Yeah, I mean, I think I believe the ideas at their core in just the same way, where, you know, this idea of just, you can do more good.

29:25Donate a significant fraction of your income. Choose a career on the basis of what will help others. I think none of those ideas should get tainted by the actions of a small group of people who did bad things. In just the same way as the actions of Ted Kaczynski, the Unabomber, do not undermine environmentalism. I think the lesson instead is just in like, well, what does that mean for how you live your life and how you run a company and so on? And it's just, effective autism should never, I think, come at the expense of like, either on an individual level, a kind of common sense guardrails, common sense kind of prohibitions as to how one lives.

30:19The kind of like classic ends don't justify the means is like, it's there. It's part of common sense wisdom for the reason. nor kind of on the institutional level, the kind of classic just best practice of, if you're a company, have a functioning board, have like, you want to be in a system where it is, you know, if there are bad actors, it's just not possible for them to actually be doing bad things because it gets caught kind of early on. I think those are the kind of constraints that both individuals and companies just should be operating under. But then within those constraints, absolutely continue to try to work out how can we have the biggest impact and go and try and make the world better on that basis.

31:07Derek Thompson: Here's the economist on the views of effective altruism that they consider questionable to be on the pale. Quote, some EAs think that insect life is in aggregate more valuable than human life because there are so many insects. Some say it would be better to bring about the existence of trillions of humans whose lives are barely worth living than to have mere billions with genuinely good lives. Others even see the welfare of artificial intelligence itself of being of vital importance. They would redirect resources toward future humans or non-humans and away from people suffering today. End quote.

31:42Derek Thompson: Do you agree that A, there's a strong argument that insect life is more valuable than human life, or B, that it is defensible to redirect resources toward future non-humans away from people suffering today? uh yeah so i wrote a response to this economist piece um defending the strangeness of effective altruism so the these are all ideas that uh get discussed among people within the effective altruism movement and your response might be like what this is just so weird these are just like crazy folk. However, I think we now have this long track record in both history of people looking like moral weirdos when they advocated for ideas like women's suffrage or the abolition of slavery or civil rights or environmentalism, but then being proved right by history.

32:48That's certainly not to say that all the ideas effective altruists are saying are correct and certainly not at the importance of those ideas. But if you're going to make moral progress, then you have to really think about things and you really have to be open to ideas, even if they seem bizarre or weird at the time. And in the effective altruist movement, we have seen that ideas that seemed esoteric or bizarre at the time, like worrying about pandemics, which many within effective altruism had been for many, many years before COVID, or worrying about AI, again, many, many years before the ChatGPT moment, actually, reality kind of catches up.

33:33And those concerns seem more prophetic than, you know, deluded at the time. And so I just think it's really important to be debating moral ideas that have good moral arguments behind them, in a just like quite open-minded sort of way. And all of these topics are just absolutely bread and butter sort of discussion topics in an academic ethics seminar room or something. And so, yeah, I just don't think they're beyond the pale at all. And the track record of saying like, whoa, that's just, that idea is too weird. We shouldn't even talk about it. I think it fares pretty badly, actually.

34:17Derek Thompson: I agree with the argument that many ideas we consider to be morally obvious today were morally insane hundreds or thousands of years ago. But to repeat at least the first part of the question, do you personally believe that insect life in aggregate is more valuable than human life? Yeah, so I don't buy that one myself, actually. um uh the insect so some people focus on insect suffering and what they're concerned about is just the sheer quantity um you know it's just take shrimp farming for example it's like hundreds of billions of shrimp every year get put in these like awful conditions they get suffocate or um get frozen to death.

35:06In many cases, their eyes get ablated and they feel pain. So if you give the shrimp anesthesia, they don't behave in the same way. They don't touch the ablated eye stalk in the same way. So we have pretty good evidence that they feel pain. Not decisive evidence, but some. And then the thought is just, wow, take really seriously like the sheer quantity of suffering there. I just still think it doesn't come out as the most effective thing to focus on, even if you're just looking at kind of near-term suffering. I think other areas of animal welfare, in particular, factory farming of chickens, look more compelling.

35:53So I think like, yeah, so I think with my like doing the most good hat on hat, I'm not in the like worrying about the suffering of insects camp. But I do want to defend the people who are like come to a different conclusion than I do. And I definitely don't want to say like, oh, these are, this is like beyond the pale to even be thinking about this. Because we would have been saying it's beyond the pale to worry about the suffering of pigs or chickens. Not, you know, really not very long ago in human history at all. Sure.

36:22Derek Thompson: I have a philosopher friend in DC who has a joke about what he considers the weird evolution of effective altruism. He says EA used to be about protecting humans from mosquitoes, and now it's about protecting mosquitoes from humans. And while I think that's a clever laugh line, I also think it touches on something that is quite fundamental and potentially problematic for a philosophical movement, which is that on the one hand, you do want to cultivate a degree of openness. I am absolutely on your side and understanding the larger principle that if you automatically shut down any idea that exists outside of the boundary of accepted moral thought, you cannot have moral progress because you've made the wall around current moral thinking the only distance that people are allowed to travel.

37:17Derek Thompson: understand that entirely. But there is also something to this idea that whether it's a philosophical movement or a company, culture is what you tolerate. Culture is what critics see as tolerated inside of that community. And a philosophical tradition that seems to be basically okay with lots of people arguing that, you know, insect life on aggregate is worth more than human life, could potentially say, well, if they're thinking that, well, now we can argue against all of these other thoughts, whether it's an artificial intelligence, pandemic preparedness, philanthropic consequentialism, that has nothing to do with shrimp lives and mosquito lives.

38:02Derek Thompson: But nonetheless, the impression that the field is open to those arguments can sometimes reflect on the way that outsiders see the entire movement. So I guess the question that I'm grasping toward is, while I agree that the weird factor might be necessary in part to create forward moral progress, do you worry that the weird factor ends up being a tar on the philosophies embraced by the larger public now that EA has absolutely escaped containment and is on the cover of The Economist magazine? Yeah, I mean, it's certainly a cost. These are certainly sticks that effective altruists can get hit with.

38:49And in particular, because the perceptions, you know, what spreads is the juicy, spicy takes and so on, where the article you're referencing is written by like a 22-year-old blogger. He's like just out of undergraduate and got big on the internet because it's this like, you know, spicy, provocative take. Or the idea of there is a project called the Shrimp Welfare Project that's concerned about the suffering of shrimp. But like effective altruist, yeah, charities. Yeah, we're looking at like in the low billions of dollars that have moved to global health and development. a vanishingly small fraction of that has been raised, I think, by the Shrimp Welfare Project.

39:34And so if the critics are harping and really getting worked up about discussion of shrimp welfare, well, they're choosing to focus on what is just a very small slice of all of the activity that goes on. And I guess I just think, well, judge the movement by the whole, where that's saving hundreds of thousands of lives via donations to the most effective global health and development charities, and talking about the avant-garde frontier of ethical issues as well. And yes, I think that means that some critics can just seize on whatever is weirdest at the time, and then pretend like that represents the whole of the movement.

40:24That's a shame, that's a cost. but I'm very very wary of the idea of kind of self-censoring on issues that might be of great moral importance um for fear of critics calling you out as being too weird I think that that kind of uh self-censorship I think would have um aged very poorly or would have done very poorly in uh previous eras of human history and I think we need to hold on to it now and it means that some critics will attack us and c 'est la vie, I guess.

40:58Derek Thompson: This episode is brought to you by JPMorgan Chase. In today's unpredictable world, geopolitics isn't just a backdrop. It's shaping markets, business strategy, and risk. The JPMorgan Chase Center for Geopolitics brings together former senior policymakers and seasoned practitioners to translate fast-moving global events into clear, decision-ready insights. From conflict and great power competition to energy and supply chain disruption, the center helps clients anticipate what comes next. Read the latest insights at jpmorganchase.com slash center for geopolitics. This episode is brought to you by JIRA by Atlassian.

41:40Derek Thompson: Your work is scattered. A thread here, a DM there, five different docs related to the same project. And now it's not just your team hunting for context, It's your AI agents too. Jira taps Atlassian's teamwork graph to bring that context together across Jira, Confluence, Slack, GitHub, and more. It makes it easy to share with the agents you already use like Claude, Cursor, and GitHub Copilot. Assign them work directly or connect your agents through MCP. The result, less token waste, more accuracy, and more time moving work forward. Learn more at jira.com. That's J-I-R-A dot com. This episode is brought to you by Lincoln.

42:19Derek Thompson: What if your vehicle could help give you a reset? Lincoln Rejuvenate is designed to do just that. This multi-sensory feature combines expansive visuals, digital sense, and a seat massage for the driver and is available in the 2027 Navigator SUV. Visit lincoln.com to learn more. Lincoln Rejuvenate does not operate unless the vehicle is in park when the vehicle is off or if any door is open. Do not use the feature in a closed garage or in other enclosed areas. See owner's manual for important operating instructions. I want to build a bridge here from our discussion of the critiques of effective altruism to artificial intelligence, where EAs have been objectively influential.

43:04Derek Thompson: There is a very particular critique of EA that comes from the tech right, from conservatives in and around Silicon Valley. And if I was going to do my best to summarize the upshot of their critique, I think it's something like effective altruists, by dint of their obsession with long tail catastrophe, have essentially fallen prey to the precautionary principle. They're simply too afraid of bad things happening, period. But civilization has always moved forward, not just because of moral progress, but also because of technological, scientific, and economic progress as well. Civilization needs ambition.

43:50Derek Thompson: It needs competition. It needs something more than just an obsession with welfare. I think there's a Nietzschean vibe, you could even say, to their approach, that we need to be less afraid of unleashing the animal spirits of capitalism to move forward civilization. And we can't listen to these, you know, these whiners who think that we're always about to end the world. What do you make of the tech right critique of effective altruism? Do you think they see your philosophy clearly? And do you think there is any wisdom in their critique? Yeah, so I'll say a few things. One is like, A drum I will always beat is that effective altruism is compatible with many things.

44:38You could be an accelerationist, want to just speed up technology as much as possible and still be within the effective altruism camp, if you really think that's the best way of doing good. And I know some people who are in that category. However, it is in fact true that within the EA movement, there's a lot of concern about catastrophic risk from pandemics. In particular, the risk of a future wave of new pandemics once the ability to create new viruses becomes more and more widespread. And then secondly, there's a concern about catastrophic risks from AI, too. Now, this isn't a matter of just general pessimism about tech.

45:18I, at least, and in fact most of the people I know, are generally very boosterish about tech. I think living standards for humans are radically better now than they were 100 years ago or 200 years ago. And that's primarily through new technology. In general, I think we don't harness the benefits of new technology enough. And through nuclear power, I think it's a catastrophe that that got regulated to death, essentially. However, that does not need to always be the case. Take the case of AI.

45:58Most of the people working on AI think there's a very meaningful chance of utter catastrophe coming from development of AGI and then superintelligence, where very meaningful chance doesn't mean like, oh, it's a one in a million, we should play safe, even if the chance is so low. It's like, no, no, no. The average AI researcher thinks the chance is on the order of 10 % that you get an outcome as bad as human extinction, or comparably as bad. The people who are concerned include literally the most highly cited scientists ever, such as Geoff Hinton and Yoshua Bengio, who are independent voices. Geoff Hinton even left Google so that he could be an independent voice.

46:43and they think the risk of catastrophe is even higher again. And then finally, I think there is just a kind of forecasting difference between me and some of the, and others and some of the people in the tech right, where they're excited by more progress. Maybe they want economic growth rates of 5 % or 6 % per year and think we can get there. And I think that would be amazing. But what I think is going to happen is that at the point of time that you get AI that can substitute for an AI researcher itself, so now known as recursive self-improvement, it's not just that you get 5 % or 6 % economic growth per year.

47:28Instead, you get a leap forward from AI that is slightly better than what we have today to suddenly AI that can replace all cognitive work, including all science, and maybe even radically better than that. You get that leap within a couple of years. And that's really a rate of progress that I think the world has, a rate of tech development that the world has never seen before. And my guess is that if the accelerationists thought that that was what was going to happen, they'd think that maybe we ought to be a little bit cautious about it too.

48:05Derek Thompson: It's a huge irony, I think, that many effective altruists and EA-adjacent people have been both banging the drum about AI danger and are also on the front lines of building artificial intelligence. I mean, the OpenAI board had effective altruists and EA-adjacent people like Helen Toner from the start. Dario Amadei was an early signer of the giving what we can pledge. His sister married Holden Karnofsky, who I believe was one of the co-founders of GiveWell, one of the organizations that has done the most to identify the philanthropies that do the most with the money that they're given. There's a ton of EA-influenced folks that are building a technology right now that you believe is on the cusp of, you mentioned a few years, on the cusp of delivering catastrophic outcomes.

48:55Derek Thompson: Are you frustrated by, furious about the Frontier Labs? Because there is some times when I feel like I personally want to scream with these companies saying, on a Monday, we think we're building something that could be really dangerous. Somebody, please stop us. And then on a Tuesday, they're like, oh, by the way, we solved 700 math problems. So they're simultaneously saying, we're doing something terrible. Please slow us down. Let's slow, let's pause, let's pace the frontier. And then on Tuesday, we just made a giant leap forward in scientific and mathematical progress. I mean, there is something unbelievably incongruous and potentially insane that's happening here.

49:37And I wonder what you make of it. Yeah, I mean, I think it's just very confusing for the general public. And I honestly even sympathize with people thinking, well, surely they can't possibly believe this because they wouldn't be building it. I mean, the analogy is like, imagine in the 1950s, you'd realized that fossil fuels produce climate change via carbon dioxide emissions. And so you thought, OK, well, the thing we've got to do then is build oil companies, build coal companies, so that we can steer this in a positive direction and accelerate the clean energy transition. and, you know, maybe that argument works, but it's certainly a weird, like it's certainly a weird move.

50:23Now, the situation is a little different in the case of AI because we have not yet crossed the Rubicon where the AI systems actually pose meaningful existential risks.

50:36So in the early days, OpenAI got started as a non-profit research lab and Anthropic got started off as a spinoff that was basically, again, going to be a research lab. And there that makes sense. You want to understand the risks from AI and you want to be able to make sure we align AI systems. Well, you've got to be working on it in order to be able to do that. And now both of these companies have evolved to become some of the biggest companies in the world. And the big question is, what will they do when they meet that Rubicon where the AIs that they have at the moment are disruptive, they're very powerful new technology, but they are still far away from the technology that would pose a meaningful existential risk in the world.

51:27and the big challenge that they will face is what happens when, no, actually now the AIs they're developing are posing like meaningful dangers. And I really hope that they're able to do, like already there have been signs. They have talked about pacing and they are being open to having external auditors come into their companies and so on. They're calling for regulation. These are all very good signs. But as the AI systems get more powerful, the sorts of actions they'll need to take will need to be, yeah, 100 times bigger than this. And the big worry is that they will be so locked into commercial competitive dynamics that they won't follow through on the kind of lofty goals they have to try and create and steer the technology in a safe way.

52:27Derek Thompson: Yeah, I mean, I understand where you're going with your Exxon analogy, but I think even that doesn't take into consideration the thing that you just said makes AI most dangerous, which is recursive self-improvement, AI that builds and controls itself and escapes human monitoring and oversight. What this is more like, to me, is like building a thousand hydrogen bombs that you believe might launch themselves because you think the Soviets are also building a thousand hydrogen bombs that you believe might launch themselves, even though it doesn't matter who builds the hydrogen bomb that launches itself, it's still really bad for the world and you have hydrogen bombs launching themselves.

53:04Derek Thompson: I mean, to me, it is significantly more catastrophic and reckless than starting Exxon because you think that people need like cheap energy to light their homes. Like people do want cheap energy to light their homes. And, you know, I think climate change is really bad. But, you know, oil certainly isn't improving the biosphere choking capabilities of itself and then running rogue away from the Exxon chemical engineers. So I think what's happening here both has some historical precedence and precisely because RSI assumes a kind of runaway capability does not have any kind of useful historical precedence.

53:45Derek Thompson: I do wonder, you talked a little bit about oversight, and I know that there are people who are like-minded who work at Meter and Redwood Research and other companies that study how AI behaves. Just give me your number one suggestion. What is the number one thing that you think folks in Silicon Valley who care about both technological progress and human safety should do with AI right now? Yeah, my number one favored policy that the companies could implement is about what fraction of their computing power they devote to research and development and future training, and what fraction they devote to infants that is serving customers or to work on safety and alignment, where at the moment, most of their computing power goes towards experiment and work to make the next even better model.

54:48And that's part of why we have such rapid progress. And I'm happy for AI progress to keep going. In fact, I think if it kept going at the current rate or maybe a little bit slower is the kind of ideal pace, we could live through this golden age of human AI collaboration that would drive forward productive science, make people more productive in their jobs, could give us that 5 % or 6 % economic growth per year that we might really like, that is totally feasible. And we could make it happen just in part, at least, via these decisions about how computing power is allocated. Because if only a small fraction is going towards this R &D process, then the explosive growth that I would otherwise expect, that explosive leap where you go from kind of below human to suddenly far greater than human capability, the sort of thing that could just automate all cognitive work, you know, that might occur over the course of a decade rather than over the course of one year.

55:57That's a lot safer from my perspective. And it would mean that it's still, I know I said a decade, it still sounds like pretty fast-paced, but relative to a year, I think it's an awful lot better. And it would mean that people could actually reap the fruits of the AI technology we already have kind of much, much earlier. And it means that everything could get developed more safely and securely. And so that's the kind of single policy. I think it's also very easy to understand that it could be implemented quite soon.

56:30Derek Thompson: You mentioned that effective altruism is not worried about slightly crazy ideas. So let's end with a slightly crazy idea. One of the things I think we're going to hear more about in the next few years as these models get more powerful is the question of AI consciousness, as insane as those syllables might sound to many people listening or watching. And the logic, as I gather, is something like, if AI is akin to a person in a computer, if we can find some way of demonstrating that it experiences suffering, that it is essentially a silicon shrimp, then open AI and anthropic are logically something like slave empires.

57:12Derek Thompson: I mean, they are institutions that bring into being entities that suffer and can only do that which is demanded of them. That I think describes what we, if we thought of a person, would call a slave. Do you believe that open AI and anthropic are slave empires? um so i think the situation we should have with with respect to ais and whether their consciousness is one of great uncertainty in a way that should really freak us out so you look at the leading theories of consciousness from philosophers and cognitive sciences and so on and ai systems at at least some of the AI systems that we can create, like they tick many of the boxes.

58:01They're using similar amounts of computational power as medium-sized animals. The idea that consciousness is only a human, or moral status is only a human thing is just actually not very plausible. You look at that data in Star Trek or Spock,

58:29and people very naturally think, yeah, no, okay, their interests matter. And so I think this is something, like among the many reasons.

58:38Derek Thompson: But Data's a character. Does anyone actually think that like the interests of a character in Star Trek matter? Like if Data's killed in an episode of Star Trek, this is not a human rights catastrophe. This is just a plot development. I wasn't meaning the character Data. I was meaning just if you were in the Star Trek universe, if Star Trek were real. Right. But why not go, I'm not trying to be the question or answer here. Why not go just to animals? Why not go to dogs? Is there a question if dogs are conscious? Bats? Oh, yeah. I mean, like I say, I'm, yeah, I obviously think that dogs are conscious.

59:14And, you know, it's unclear, like, where do you draw the line between dogs and nematodes or bacteria or something. but at least you know someone could say well it's all biological and that's the difference and historically that has been a very unpopular view in the philosophy of consciousness and much more popular is the idea that even though we have no idea really what grounds consciousness it's something to do with computational processes it's something about like the function of how um of what brains do rather than the fact that they're implemented on neurons rather than silicone. And yeah, I basically, I think that this is an issue that society as a whole should be taking extremely seriously.

1:00:01And the thing that's particularly tough about the issue is that we are not going to get a good answer, a satisfying answer either way as to whether the AIs are conscious or not. Even once they're like humanoid robots walking around with long-term memory that people have relationships with. We're still not going to know. And so the issue society has to deal with is what should you do given that we don't know? And I think this is a case where it actually is pretty fair to be kind of cautious.

1:00:34Derek Thompson: You said two things that are interesting. You said we should take it seriously, the question of if AIs can ever be conscious. And you also said we'll never know. What does it actually mean to take seriously a philosophical question that we'll never know the answer to? Well, I think I'm kind of optimistic we'll know at some point, but maybe it's decades or maybe it's hundreds of years into the future. But to take it seriously is to just be not to dismiss it. So we'll be in a position where it's like, yeah, there's some chance these AI systems are conscious. I already think there's some chance these AI systems are conscious.

1:01:11And so taking the issue seriously is to say, okay, what ethically ought we to do when we're in this position of uncertainty? Where one view you might have is like, well, if you're unsure, then just don't worry about it. Just assume they're not. This is how the medical establishment used to act with respect to newborn babies. In fact, until remarkably recently, only a few decades ago, newborn babies and infants would get surgery without anesthesia because it was the widespread understanding was that they don't feel pain. I think that's a kind of crazy thing to become the widespread understanding, but it's also agreed that it's like, no, that's false actually.

1:01:55And so, yeah, I think the ethically responsible thing to do is to be firstly debating as a society, if we are creating a new, like a wholly new sort of being, how should that being fit in to our current society. Should it be that they get treated like other software and get owned by the companies that create them? Is that correct? Or should they have economic rights? Should they have welfare rights? That is a debate society should be having and it is not currently having it. And then I think there should also be an onus and just understand, yeah, if the AIs do have interests, like what do they care about?

1:02:41What would it mean from their perspective to treat them well? And without kind of anthropomorphizing them or making assumptions that it would be similar in any way to how humans are treated.

1:02:50Derek Thompson: Last question, a kind of summing up question. I go back to 13 years ago when we first had those late night scotches and talked about effective altruism in its nascent early period. And that was a philosophy that was about using money to help people and using time to help people. And when I think about the evolution of the philosophy over the last 13 years, I think about the fact that animal welfare has clearly been pulled into the orbit of effective altruism, shrimp welfare, even claims of insect supremacy. We're talking about AGI now and AI moral rights. How do you feel about the argument, or I should say the criticism, that one way to describe the trajectory of effective altruism has been the long de-centering of the human being.

1:03:41Derek Thompson: That as the philosophy has found more and more ways to extend itself over the course of all things to care about, it has found more and more things to care about that aren't people. Is that a fair description of effective altruism? And if it is, is it one that you welcome? I mean, maybe I could hear an argument that says the correct frontier of philosophical entrepreneurship should be moving away from a human-centric theory of moral good toward thinking about not just the people that exist, but the people that might exist. Not just humans, but also insects and shrimp and also AI. That's a good thing, one could say.

1:04:27Derek Thompson: So I guess two-part final question. One, do you agree with this idea that a history of EA has been the dissentering of the human? And if so, is that moral progress? Yeah, I think on the first question, I think it's largely not the case. Like, effective altruism has grown as a whole. that means the work that gets done to benefit present-day humans has grown enormously you know a factor of a thousand or something or hundreds compared to the early days such that you know now now it is kind of tens hundreds of thousands of people per year who are getting prevented who are whose lives are being saved via things like anti-malarial bed nets and so on similarly the work on global catastrophic risks i don't think at all you need to care but you could care about people past the year 2100 not a job and it would still come out as among the very most cost-effective things you could do in my view because the risks are so great they're so and they're so incredibly neglected um it is the yeah and so what has happened well there's a lot more discussion of that like avant-garde stuff because the effect of altruism is kind of more in the public eye than it was.

1:05:52But again, I think that's just thinking that that's kind of what the effect of altruists now all focus on is just kind of misrepresenting the movement. However, I do think that's what we should be talking about. I think the idea that only human beings have any sort of moral status is a view that's just going to look quite bad from the perspective of people looking back at us in the future in much the same way as the idea that like, oh, well, it's only people in the UK, only people in the US matter. And the interests of someone in Mali, they're just, they're not, you know, they don't count. I mean, I think when you ask people and you say like, well, imagine it's 2200 and future generations are looking back at us and thinking, oh, what are the moral atrocities that we were committing now?

1:06:41In the same way that we look at moral atrocities of the past, many, many people, maybe even most people say, oh, well, it's going to be factually farming, isn't it? And it's like, yeah, it's like huge amounts of unnecessary suffering that we inflict on animals for very little benefit to ourselves. And so the thought of just, okay, well, what are other ways in which we could be engaging in similar moral atrocities? We really ought to be thinking about that and just coming in with this kind of blinkered, well, it's not about present-day humans or, you know, not just about present-day Americans or Brits or something, I think is going to look really quite narrow-minded from the perspective of moral progress, from the perspective of history.

1:07:28Will McCaskill, thank you very much. Thank you so much.

1:07:41you