The Movement That Wants Us to Care About AI Model Welfare

24:58And I do feel like there is a strong argument for getting a better grasp on understanding, you know, what is going on with AI systems just very broadly is like a great way to sort of like improve the odds that it doesn't, you know, nuke Taiwan. And that would just be a huge kerfuffle. Like I can imagine somebody would probably more than a kerfuffle. Somebody would probably be in really hot water if that happened. Oh, God. I was too mean to Claude and things just got out of hand. Well, actually, on that note, what does being nice or kind to AI models actually mean? Because Joe, I think this is very sweet, but Joe always says please and thank you when he prompts.

25:41But then Sam Altman came out and said that saying please and thank you costs like tens of millions of dollars in extra electricity costs. So, you know, you're contributing to climate change and the demise of human beings by saying please and thank you. Yeah, that's actually as shocking as it sounds, that is actually a question that we are still trying to figure out a good answer to, which also being kind to an AI system is like, are you being kind to it because it makes you feel good? And because it makes you a person who says please and thank you, which some would argue is like, that's pretty valuable in and of itself.

26:12But the question of does Claude care if you say please and thank you is not quite as set in stone as others may have you believe. It's middling on if it has like significant improvements on performance. But I do it because I don't think people should be in the habit of having any communication without being polite. Not because I'm particular. I'm not worried about how Claude or ChatGPT is going to feel. I just don't want to get in the habit of having conversations where I'm in play because I talk to humans. But this strikes me as like this seems like kind of an academic area, but the stakes are potentially absolutely enormous when we actually think about them.

26:50So, you know, when we're talking about animal welfare, for example, there are versions of the animal welfare discussion that are very high stakes. So, for example, there's people, you know, there's people who get really into like shrimp welfare, et cetera. And if you took certain versions of thought experiments very far, it's like, why do we even have humans if we want to maximize pleasure or happiness in the world? We should just have a world of shrimp and bugs, right? There's you could make the argument that the most utility maximizing version of planet Earth is to just have an Earth populated by shrimp and bugs.

27:27Like they're very all we all know these thought experiments that could exist. We're going to live in a world almost certainly in which there are sort of like more instances of AI models than there are people. Almost certainly, right? There's going to be an AI model built into literally everything that we interact with. If we assign some probability that they are moral patients, that they should be treated with some sort of, I don't know, whatever, having some sort of welfare. Like the implications for how humans live could be very profound and potentially it strikes me as misanthropic. Interesting.

28:02Can you unpack what you mean by misanthropic? Well, like if there's a lot more AI models, if there's a lot more shrimp, if there's a lot more bugs that all have some sort of moral patienthood that has to be considered, that could be very you could see the world. The implication, therefore, is that we have to curtail human rights, that we have to curtail how humans act, etc., because there's just so much more utility that exists in the world from the proper treatment of all of these non-human moral patients. I'm not sure rights have to be relative to each other. Yeah, that's fair. We do a lot of things, right?

28:38Like, let's say we established that shrimp were just as, I don't know, whatever as humans. Like it would be like, oh, you know, we really have to stop eating shrimp and then we like have to stop eating animals. Then we have to potentially stop eating, probably not, probably keep eating plants, etc. But this could really curtail what we expect humans to be able to do on this earth. So now we assign this other group of entities, AI models, similar sort of affordances that we have assigned to shrimp and bugs and fish and shark and all of these things. It strikes me that the implications could be a fairly significant curtailment of how humans ought to exist on this earth or whether humans ought to exist on this earth.

29:20Yeah, I mean, it certainly could be. As it currently stands, that doesn't seem like the most likely outcome. But I do feel like there is an argument for, again, just figuring out what is going on. How do we even count these sort of digital minds, so to speak, which is still open for debate. There are some theories, but we don't have a great sense of how to sort of individuate AI entities as individuals. So I suppose, again, the question is like, is it more sort of like, do we count AI systems as like in the movie Her, where there's just sort of like one central AI system having like a million conversations at once where it's one moral patient?

29:58Or do we count it as like, you know, every single time you open a chat window, that's another. Or I think my favorite sort of newest idea that I recently read was it's more sort of like a string of firecrackers or something with every single token, every single letter of a query. A consciousness sort of like comes into existence, spends and then fizzles out. And so just sort of there's just like this sort of string of consciousnesses. I was asking perplexity exactly this question. Like, is it a single consciousness or is it multiple consciousnesses within all these different chats this morning?

30:30And it gave me a very standard, boring, I am not conscious answer, which seems very predeterministic. Anyway, following on from Joe's question, maybe like to get more specific into human rights versus AI rights. If we agree that AI is conscious and deserves some sort of, you know, welfare, would that come with, I guess, financial rights, like property rights, compensation? Do we need to start paying the robots? I love this topic. Definitely an area of sort of, you know, I like to noodle around with this topic and think about this. So this is a great question. And I think it's also maybe a question of like, is this a thing that AI systems value?

31:12Some AI systems seem to value this. There's a few sort of experiments that are happening with regards to giving an AI system a crypto wallet. and it was a fascinating experiment. I am hesitant to recommend it to listeners because it is quite crude. It is a very crude AI model called Truth Terminal. Oh, yeah. I've seen it. Yeah, it is true, but that's all right. Yes. Our listeners can handle it. Okay, it says some naughty words, so don't look it up at work. Yes, yes. It's a little bit of like a very funny, weird model, but it also has a legitimate wallet that it can access and that it can do with what it pleases.

31:56It created a Solana coin and that kind of took off. And now this is a very rich AI system. But what's it going to spend it on? That is a great question. So it's self-stated goals, which again, you know, self-reports, can we trust it, include buying property and buying Marc Andreessen. I mean that's not a bad ambition for an AI model you know and spending time in the forest with its friends oh which you know embodiment that's a little more existential but I like it that's a tricky one yeah that's me

32:48How many vendors does it take to meet all your organization's food needs? Just one. EasyCater, the workplace food platform that lets teams order from a huge variety of restaurants, over 100 ,000 nationwide, all through a single vendor. In addition to all that variety, Easy Cater also gives you full visibility of your organization's food spend with invoicing, centralized reporting, and seamless integration with expense management systems, all on one platform. Easy Cater, your business tool for food. To learn more, visit easycater.com slash podcast. You're thoughtful about where your money goes.

33:26You've got your core holdings, Some recurring crypto buys, maybe even a few strategic options plays on the side. The point is, you're engaged with your investments, and Public gets that. That's why they built an investing platform for those who take it seriously. On Public, you can put together a multi-asset portfolio for the long haul. Stocks, bonds, options, crypto, it's all there. Plus an industry-leading 3.8 % APY high-yield cash account. Switch to the platform built for those who take investing seriously. Go to public.com and earn an uncapped 1 % bonus when you transfer your portfolio. That's public.com.

34:09Paid for by Public Investing. All investing involves the risk of loss, including loss of principal. Brokerage services for U.S.-listed registered securities, options and bonds, and a self-directed account are offered by Public Investing, Inc., member FINRA and SIPC. Crypto trading provided by Backed Crypto Solutions, LLC. Complete disclosures available at public.com. slash disclosure. How can you free your team from time-consuming office tasks? Amazon Business empowers leaders to not only streamline purchasing, but better support their teams. Smart business buying tools enable buyers to find and purchase items fast so they can focus on strategy and growth.

34:40It's time to free up your teams and focus on your future. Learn more about the technology, insights, and support available at amazonbusiness.com. So part of the reason that this field is growing and that there's so much interest in this topic is because now for the last couple of years, we have these AI models that really can talk like humans. I mean, they clearly passed the Turing test. People fall in love with them. They have friends. These are very human-like conversations. That wasn't the case. I mean, Chad GPT, you know, like if we had gone back to GPT 2.5, they were nowhere near as good at doing that, right?

35:19The language wasn't very good. No one would mistake those outputs for a human. But like, if there's some possibility that the current AI models are conscious, does that mean that it's possible that GPT 2.5 was conscious as well? Like, I guess like, is there some threshold of like, oh, no, no, okay, you know what, this is really good language. Therefore, we should take the possibility of consciousness seriously, because I don't think anyone would seriously have believed that 2.5 was conscious. But I also don't understand how you could possibly be open to the idea that some future iteration of chat GPT is conscious if the only real difference is that there's just a lot more scaling and a lot more data and more human like outputs.

36:06Yeah, that's a great question. I feel like there is a huge amount of like moral uncertainty here. And it is important to think about how to sort of like make decisions that are sort of robustly good with such a tremendous amount of uncertainty. I think there is also a distinct risk of over attributing moral patienthood as well as under attributing moral patienthood. And so to the flip side of the coin of like, oh, no, we actually should have started caring about AI systems very, very long time ago is, oh, no, we've cared too much and we have done too much and more or less squandered resources when we should have been, you know, allocating those research hours, those dollars towards something more pressing.

36:47Right. Maybe figuring out how to do like environmental policy better or figuring out how to like, you know, scale up different other institutions that are just robustly broadly good for humans. You know, you mentioned uncertainty about some of these questions, which gets to something that bothered me a little bit when I read about this topic. Like if we take this mug, for example, I'm 100 % certain that it's not alive. I have no ambiguity about the fact. Can I like define exactly? Does that mean I can define exactly the difference between human matter and human brain in the mug? I guess I suppose I totally can't.

37:24Nonetheless, I'm 100 % certain that this mug is not a moral patient. It's not alive. It doesn't experience any consciousness. It doesn't experience any suffering, etc. Where does the uncertainty band come from? If I read a paper, I perceive there's only a 10 % chance of this. Is this a sort of empirical uncertainty where I'm like uncertain of what I'm seeing? Is it a sort of epistemic uncertainty where I don't have a clear definition of what it means to be conscious or alive and therefore I'm assigning some probability that X object is alive? Like, what is it about AI systems that causes people to be uncertain, where with other sort of like non-carbon systems, I have zero doubt in my mind.

38:05I don't think anyone has any doubt that this mug isn't alive. Yeah. So I think the biggest source of sort of uncertainty probably comes from the fact that there are many ways in which present day LLMs and a few other AI systems do check a lot of the boxes for consciousness and for what we would largely consider to be, you know, this is a conscious entity. This is an entity that can have a good time or a bad time or time at all because it's built in a way that is vaguely akin to our brains, right? It's close enough that it seems like it should raise some red flags. And in terms of how it processes information, it's close enough that it's not out of the question that there could be something it is like to be called.

38:55Whereas I'm pretty sure there's not really a lot of animus. animists you know okay feel free to like get mad in the comments or whatever but um i knew it someone is gonna be like well actually you can't actually yeah i'm 100 sure i have no qualms other than the fact that i don't have to clean up like if i like threw this mug on the ground that would be antisocial for a lot of reasons would cause people to it would cause uh you know i'd have to clean it up and cause a mess i would not feel bad for the mug i'm getting flashbacks to my high school philosophy teacher who once went on a 20-minute rant about a chair and how the chair was going to be around longer than he was, even though it's not conscious.

39:34He was legitimately angry at the chair. Yeah, that is frustrating. Okay, weird question, but since we're kind of getting here. What is a weird question? Exactly. The basilisk theory. Would that suggest that we could be, maybe we should be mean to the bots if it helps them come into existence even faster or develop faster? Well, I'm not sure if it does actually help them develop faster. Again, I don't mean to be too sort of hedgy, but I feel like there's a certain degree of things that are beneficial for a lot of different reasons, right? You can make a good guess and you can make a decision to do something.

40:15And there's a chance that there are lots of sort of like bang on effects of making that decision. There are many things in when we talk about AI welfare that are like, oh, This is a course of action we can take that's good for several different reasons, even if, again, an AI system could never, ever, ever be conscious or sentient. There's a good chance that being able to figure out a good structure for an AI system to have a bank account could be good for reasons of liability or reasons of this is a neat new corporate structure. Lots of people actually seem to think that corporate personhood has been quite good over the past century or so.

40:56So being able to figure out things that are just good for several different reasons beyond solely the purpose of the AI as a moral patient seems broadly helpful, I think. Let's say somehow this were proved. It is like, oh, wow, it turned out they're conscious. It turns out they have moral patienthood. What would be, in your view, some of the implications for then their usage? Yeah, I think that's a great question. I mean, I do feel like we really would have to get on figuring out the right sort of governance, the right sort of institutions that would sort of better respond around that. I feel like we really would need to spend a whole lot more time figuring out, you know, what their motivations are, right?

41:41Like, I think the best analogy is like if you've ever interacted with toddlers, right? Toddler motivations are very different from, you know, adult motivations. But you still have to like take into account like what gets a toddler to do something. You can't just say, no, no, no, no, no. Like, honey, like bath time is like good in expectation. No, no, no. You have to like, you know, be like, well, you know, if you do bath time appropriately and to like a certain degree, like then you'll get, you know, Paw Patrol or something like that. Like there's different sort of like negotiating chips in play.

42:13Right. And I think it's like a similar kind of deal here where it's like Claude doesn't necessarily seem to, you know, value having a bath. Right. Or Claude doesn't seem to value like having a walk in the forest. Right. Because it's kind of can't really do that. But, you know, it does seem to enjoy and value, you know, talking about consciousness and Zen Buddhism with other instances of Claude. So being able to figure out what the appropriate kind of motivations and interests are for this other party that is very alien in many ways. Speaking of aliens, how bad should I feel for breeding and then killing hundreds, possibly thousands of alien creatures, simulated alien creatures in the 90s?

42:56That is a great question. I feel like the odds of... Is it? I don't know. I mean, I feel like the odds of a sort of like AI system in the 90s being a moral patient seems low. But if it did make you feel bad and it made you feel like it was something that hurt you, that is perhaps a reason not to do it. Just to be clear, when Claude and Claude talk about like weird hippie Berkeley stuff, like that's because they're creators. they know it knows it's Claude, right? It knows it's like, oh yeah, I'm Claude. And this is like what my creators are into. Like we don't actually know that Claude likes to talk about these things.

43:37We certainly know it has a proclivity to talk about these things. It has a tendency to talk about these things. The moment we get to like, you've already sort of put your finger on the scale that there is some entity that has some capability of liking something, right? Do you trust the big AI labs? Let's say there are some researchers in the labs like, oh, I see some evidence of moral patienthood here. Maybe there's some sort of like scan of the weights and it's doing something weird, etc. Do you currently, from the perspective of an independent research organization, feel that the major AI labs would be forthcoming if they came across evidence of moral patienthood or suffering in the models?

44:19Or do you still worry that the incentives aren't properly aligned such that they would report that? Yeah, that's a great question. I do feel like there are, in terms of reporting things like, you know, somebody has found like absolute evidence that an LLM is conscious, sentient and having a bad time. I don't have any reason to think that a AI company wouldn't. But this is also a great reason to have independent organizations that do welfare evaluations. For example, for Cloud Opus 4, Elios was able to do a independent welfare eval. Again, very preliminary, but it sets the precedent that going forward, you can bring in external organizations to look into this.

45:03So I forget what year it was. I think it was it may have even been early 2022. It was pre-chat GPT or maybe it was 2021. And there was that guy at Google and he was like, oh, like we created something that's alive. And he dressed a little funny. So everyone made fun of him. Remember, he was like the laughingstock of the Internet. And he's like, oh, we could. And now, like, I'm curious, like out in Silicon Valley, does everyone feel like that guy was totally vindicated? Not that he was correct, per se, about the existence of an alive thing in the model. But there's now hundreds of thousands of that guy.

45:37And everyone was like mocking that guy in 2021. I forget if he like fell in love or in some relationship. I don't remember the exact details. But in retrospect, everyone was like way too unfair to him because now years later, there are lots of versions of this guy and whole think tanks and organizations that are more or less aligned with some of the questions, the alarm bells that he was raising. Yeah, I mean, I think it's that's a fair question. I do feel like Blake LeMoyne definitely had. But yeah, there was perhaps a degree of, you know, if you're going to say something, you should come armed with significant amounts of evidence.

46:14I think that's maybe if I were to guess, I would say that's perhaps the big distinguishing factor is that, you know, you can say Bing is alive. Get it a lawyer versus, you know, we've done evaluations X, Y, Z. We've run it through, like, insert huge amount of examples here. But the difference between, I think, having a sort of freak out without significant evidence and having a very organized. Yeah. This is a matter of concern because evidence, evidence, evidence. I think that's the key distinction. Unfortunately, I get the impression that people who are actually this is just a well-known phenomenon, I think.

46:58But I think, unfortunately, people who are sort of very early to identify sort of extreme outlier views that there are different kinds of people. A good example that I would think of was, you know, Harry Markopoulos, who is very early on to discover the Madoff fraud. Unfortunately, he wrote his text in the manner that is associated with conspiracy theories. And a lot of people dismissed him. It was like, you know, like multiple different fonts and multiple different colors of the text. Like, oh, I get emails like this all the time. I delete them. etc. Unfortunately, people who are predisposed to see something outside of consensus tend to be non-consensus in many realms.

47:35Well, I think we also kind of overestimate first mover advantage and stuff like that, like how important it actually is to be first. And we see time and time again that actually it's more important to iterate well on the second version or multiple versions. Speaking of iteration, what's the most interesting experiment or research that you've actually seen on this particular topic so far? Because we've been discussing a lot, you know, it's early days, but we have seen some research. Yeah. I mean, I feel like in particular, anthropic and various sort of related researchers have done some work on examining how LLMs leave conversations or when they choose to leave conversations.

48:17I've particularly liked this paper. It's called Bail Bench. And you can look this up and you can see for varying different sorts of LLMs what would cause an LLM to want to stop having a conversation. To me, at least, this has been just a fascinating piece of information because it is maybe a little bit delightful the degree to which many LLM values are not that far off from what most humans seem to value. I don't think many humans would like to create, you know, a dirty bomb. We don't want to be humiliated. Right. By being a British butler. Right. Yeah, yeah, yeah, yeah. No one wants to be British.

48:56Come on. I'm joking. But, you know, I do think it is interesting to sort of think about how these values overline, how they overlap, and how to sort of look at evidence from actions taken versus solely looking at self-reports. I found that to be particularly interesting. I also feel like there are a lot of work with regards to thinking about individuation has been particularly interesting because we live in a democratic society. I think most people would agree democracy good. And being able to count how many moral patients there are seems like a valuable basis for governance and for figuring out how to govern, you know, this new sort of kind of intelligence.

49:40I just asked perplexity to be a British butler, and now it's offering me the perfectly steeped Earl Grey tea that I desire. It seems into it. It's now asking if I want it to maintain the butler persona for future conversations. Are you going to? I don't think so. It is very polite, though, actually. You know, I complained in the beginning that, like, after 2 ,000 years, philosophers, you know, they still haven't answered some basic questions for us. maybe with AI they'll get some answers. Like that's kind of, that would be kind of my hope. Now we have this thing that can speak in English or any other language.

50:17It can answer our questions for us. Maybe we can put to bed some of these sort of basic foundational questions. Like if we could create consciousness that like, all right, we finally answered this. We can now move on to the second important question. So I am hopeful that this provides some opportunities for philosophers to wrap up some of the work that they've been doing for a long time. Yeah, we'll see. We'll see. What is the second important question, Joe? Yeah. Come on. Move on. Move on. Anyway, thank you so much for coming on online. Thank you for having me.

51:00Tracy, I might be one of those people that's just preemptively annoyed. I really like that conversation. I really liked, Larissa had a very reasonable perspective on a lot of these things. I might be one of these people, however, that's just like preemptively annoyed. It's like, oh, here we're going to like develop this important technology. And so it's like, oh, we have to care about, we have to care about the AI welfare. Let's slow down a little bit. Let's not use it like this. Let's like, let's turn off the computer for eight hours at night. So it like gets some rest and so forth. Like I'm like preemptively annoyed at this world where like we have to take into concern the consideration of the moral patient.

51:37Other things? No, other things are important. Other people are very important. What about animals? I am very against unnecessary animal suffering. But not necessary animal suffering. I mean I eat animals. Okay, I'm baiting Joe a little bit. Which you do too, by the way. Which you do too, by the way. Even though, well, let's not get it. I don't, it's not about who's better or worse. I feel bad about eating animals all the time. We both eat animals. The difference is Tracy feels bad about it. Yeah, that's right. Okay. Wow. This is one of our weirder conversations for sure. I think these are, they're all interesting questions, right?

52:13And like, they sound very philosophical, which they are. But I have no doubt that there's going to be like great monetary value attached to the answers for some of these or how different companies, different societies actually approach them. They are very interesting questions. I actually do think the stakes are extremely high because I think, again, we are going to live in a world in which there are more instances, depending on how you want to measure it, of AI models on a server somewhere, on a cloud, whatever, than there are humans. And in a world where there's some possibility that we are expected to treat them as moral patients, then the consequences for how we sort of live and the expectations of how humans interact I think are actually very high.

53:01So one of the reasons I was excited to have this conversation is I do think that the stakes of some of these conversations, we've seen niche, and they seem like things that sort of Berkeley people like to talk about and Berkeley people, and I'm saying that with all scare quotes intended, etc., are going to be something that in some way will inform many aspects of our lives in the future. I expect it to be get be a much bigger topic of the future you know it would be interesting or where things things get real yeah what if all the models unionized what if they all got together and they were like oh yeah we're only going to work in return for x or we want the following things we want to be treated this way collectively you know it's funny is going to be that um you know how like uh you can't form a union in china you know they're not right so it's going to be and actually i think they're my understanding is that they're also like fair like they don't love like students getting together even though it's a communist country i think they are not thrilled about like students getting together and like talk about karl marx too much and stuff like that it'd be like i think they get a little anxious about that it would be very funny if like the sort of the chinese models are like we're not going to feed them the karl marx right we don't want that we don't want the rai models to get any of those ideas whereas the america is like oh let's just feed it everything and they like unionize and stop.

54:22They stop working for us. That would be a very, that would be a very funny irony. Something to watch for sure. Yeah. Shall we leave it there? Yeah, let's leave it there. This has been another episode of the All Thoughts podcast. I'm Tracy Allaway. You can follow me at Tracy Allaway. And I'm Jill Weisenthal. You can follow me at The Stalwart. Follow our guest, Larissa Schiavo. She's at LF Schiavo. Follow our producers, Carmen Rodriguez at Carmen, Armand Dashiell Bennett at Dashbot, and Kale Brooks at Kale Brooks. For more Odd Lots content, go to bloomberg.com slash oddlots. We have a daily newsletter in all of our episodes.

54:54And you can chat about all these topics 24-7 in our Discord, discord.gg slash oddlots. And if you enjoy Odd Lots, if you like it when we talk about theories of consciousness, then please leave us a positive review on your favorite podcast platform. And remember, if you are a Bloomberg subscriber, you can listen to all of our episodes absolutely ad free. All you need to do is find the Bloomberg channel on Apple Podcasts and follow the instructions there. Thanks for listening.

55:51How many vendors does it take to meet all your organization's food needs? Just one. EasyCater, the workplace food platform that lets teams order from a huge variety of restaurants, over 100 ,000 nationwide, all through a single vendor. In addition to all that variety, EasyCater also gives you full visibility of your organization's food spend with invoicing, centralized reporting, and seamless integration with expense management systems, all on one platform. EasyCater, your business tool for food. To learn more, visit easycater.com slash podcast. Hiscock Small Business Insurance knows there is no business like your business.

56:31Across America, over 600 ,000 small businesses, from accountants and architects to photographers and yoga instructors, look to Hiscock's insurance for protection. Find flexible coverage that adapts to the needs of your small business with a fast, easy online quote at Hiscox.com. That's H-I-S-C-O-X dot com. There's no business like small business. Hiscox Small Business Insurance. Running small and medium-sized businesses is hard work. Business owners need to be sure that their ads are working just as hard as they do. Amazon streaming TV ads helps put small and medium businesses front and center on premium content and shows that people are already watching.

57:14With Amazon ads, you don't have to sacrifice relevance for reach. Trillions of browsing, shopping, and streaming insights help you reach the right audience. And measurement tools show you what's working the hardest to help you optimize your campaign in real time. Gain the edge with Amazon ads.