Sycophantic Suck-Up Machines

0:00Slate Money is sponsored this week by Your Next Dollar, a podcast from NerdWallet Wealth Partners. Your Next Dollar is built specifically for high earners like you. It covers the financial complexity that comes with a higher income. If you have equity compensation, RSUs, tax planning, housing decisions, figuring out when it's actually okay to enjoy your money. It turns out that the more you earn, the more complicated the questions get. And yet, somehow, there's even less content built for you until now. Your Next Dollar covers both sides of the equation, building wealth and spending on what actually matters to you.

0:39Because at a certain income level, the question isn't, should I save more? It's, what should I do with everything I have? Financial advisors from NerdWallet Wealth Partners join the show regularly to discuss the questions they hear from high earners like you. The nuanced, real-world stuff that generic financial content doesn't cover. If you want help getting more out of every dollar you earn, follow Your Next Dollar wherever you get your podcasts. Sponsored by Your Next Dollar, a podcast from NerdWallet Wealth Partners, and SEC-registered investment advisor. This podcast is for informational purposes only and is not investment, tax, or legal advice.

1:20Felix Salmon: This episode is sponsored by HeyGen. If creating video content feels like a trade-off between looking professional and protecting your time, there's a new way to think about it. Hey Jen lets you turn a script, photo, or presentation into a polished video of you in minutes without setting up a camera or hiring a production team. Whether you're a financial advisor explaining market updates, an attorney answering common client questions, a real estate agent showcasing listings, or a creator growing your audience. It helps you stay visible without constantly filming new content. It's rated the number one AI video platform for small business on G2, trusted by more than 30 million people and 85 % of the Fortune 100, and supports over 175 languages.

2:12Felix Salmon: So your message can reach audiences wherever they are. Here's the wild part. Record yourself for just 15 seconds, and Hey Jen builds an AI avatar that makes professional videos for you on demand so you can post everywhere your audience is, as yourself, without filming every time. Your first three videos are free at heyjen.com slash pod. That's H-E-Y-G-E-N dot com slash P-O-D.

2:53Hello, and welcome to Slate Money, your guide to the business and finance news of the week. I'm Felix Salmon. I'm here with Elizabeth Spires of The New York Times.

3:04Felix Salmon: Hello. I'm here with Emily Peck of Axios. Hello. Emily has some fabulous news that she's going to drop at the end of the show. Huge. It's huge and awesome. But before we get there, we first need to introduce the one and only Cashmere Hill. Cash, welcome. Thank you for having me. Cash, we love having you on this show, but for those of us who don't remember the last time you were on, who are you? What do you do? I'm Cashmere Hill. I am a features writer slash technology reporter at The New York Times. And you have been on a tear. You had, what, three stories published in a week or something? Yeah, usually I publish like one story a month, so I'm exhausted.

3:43I'm glad people can't see me on the Zoom because my hair looks crazy. I've got like bags under my eyes. It's been a month. It's been a month, but we are all the better for it. You've done an amazing amount of reporting on chatbots and suicide and big, deep issues like that, which we are going to talk about. We do, of course, need to talk about Fed independence as well. And this attack on Lisa Cook, who's a Fed governor by President Trump, who's tried to fire her. So we're going to talk about all of that. We have a Sleap Blast segment on Taylor Swift, of course, because we have to. There's a rule about that.

4:16It's all coming up on Slate Money.

4:24so we need to start with the most important business and finance news of the week by far which is that the independence of the federal reserve is either under threat or kind of has already been lost depending on who you believe emily bring us up to speed here and how on a scale of like one to 10, how much is your hair on fire right now?

4:49Felix Salmon: You can't see this in the Zoom, but my hair is on fire. It's been on fire all week, at least, if not for longer, because President Trump really escalated his push against the Federal Reserve, his fight to set monetary policy for the United States and thus the world this week, because he and Bill Pulte, who runs the FHFA, the housing agency that oversees Fannie Mae and Freddie Mac, have been on this. It's a smear campaign, it looks like, against Fed Governor Lisa Cook. She's the first Black woman to be a Fed governor ever. And they are accusing her, without much evidence, of so-called mortgage fraud.

5:29Felix Salmon: But all they really have said is that she signed a document attesting that a home was her primary residence when it couldn't have been her primary residence because she maybe has two. Well, I mean, let's just say very clearly principal residence, which means something different from primary residence. Like, it's a tax fraud to claim two homes as your primary residence, but principal residence means something different. I mean, this seems to come up a bit with lawmakers. Ken Paxton, the attorney general of Texas, I remember when his wife filed for divorce, I think they had something like three principal residences, if I'm remembering the Times story I read correctly.

6:04I mean, it's an open question how many principal residences Donald Trump has. I mean, it's at least two, right? He has Mar-a-Lago in the White House. I guess, is it principal? Is it emotionally principal? You know, it's like your children. You can't choose a favorite one. I feel like we shouldn't go too deep into this obviously flimsy pretext just because it is an obviously flimsy pretext. But yeah, it's like this was a promise. This was a thing that she kind of said about what she would be doing with the House in the future for an undisclosed period of time. And all happened long before she became a Fed governor.

6:38You know, it so clearly doesn't rise to the level of cause.

6:41Felix Salmon: Can we just add that it happened in 2021 when mortgage rates were at the lowest they have literally ever been? So it's crazy to me that anyone would purposefully, and we don't know if it was intentional or not that she signed this or whatever. We have no idea that someone would purposefully. We have no idea whether How does she even got a lower rate because of this? But rates were like in the two handle at that time. The idea that anyone of means would be like so fraudulent or so scammy to do something like that at that time is just, it's just absurd. This is all obviously a pretext to try and push off one of the Biden appointees from the Fed board and replace that person with a Trump appointee, just as the sort of national emergency stuff is a pretext to impose tariffs, which he wants to do anyway, just as the national security stuff is a pretext to shut down wind farms, which we're going to talk about later.

7:36Trump's lawyers are pretty good at finding pretexts, but it's pretty clear that this is just a pretext and that the fact that she has two homes is not in any way, shape or form disqualifying in terms of her ability to be a Fed governor. But he's trying to push her off. And what the Fed didn't do was very full throatedly stand up for her and say, no, that's illegal. You can't do it. She is still a Fed governor. Instead, they just put out a quiet statement to the press, which isn't even on their website, saying, well, I guess this is all in front of the courts now, and she has her own lawyers, and we'll do whatever the courts say.

8:14And this seems pretty sort of weak-spined to me.

8:20Elizabeth Spiers: There's a reason why they have to do this, though. There was a story in the Times this morning noting that she has to name them as defendants in the suit, too. So they aren't really allowed to make a full-throated defense of her. I mean, they can sort of do it in an implied way, but they can't come right out and say, this is bullshit, she's being targeted, blah, blah, blah. Why not? All right, give me a minute. I need to pull up this piece. The Fed is limited in its ability to publicly aid her in this case because the allegations involve a personal matter that has nothing to do with her role as a Fed governor.

8:55Elizabeth Spiers: So they are legally not permitted to intervene and defender on the merits. I feel like Jay Powell has lawyers who are advising him to say nothing about this. And he came out in Jackson Hole, which is the biggest, which is the biggest, no, no, like, let me, let me like finish my thought here. The biggest speech of the year, if you're the Fed governor, is your big speech at Jackson Hole in August. This particular speech was probably the most important central bank governor speech that has ever happened in my lifetime. All eyes were on him. The single most important question facing monetary policy in the world right now, but specifically in America, is this question of independence and Federal Reserve independence.

9:46He could have come out and said, like, full-throatedly in support of Federal Reserve independence, and either implicitly or explicitly in support of Lisa Cook, who was already being targeted at that point. His decision not to do so is almost certainly you are right that he is listening to his lawyers. But you know, what chief executives do, and Jay Powell knows this better than anyone, is they have lawyers who give you advice, and then you take that advice under advisement, and then you make your decision as to what to do. And sometimes you have to say, well, I know that my lawyers are being cautious, but this is more important.

10:22And I think this is one of those times. The Fed needs to stand up for its independence. I think Nathan Tankus actually came out with a piece this week saying this quite explicitly, that unless every single person on the Federal Reserve Board is singing from the same songbook very loudly about how the number one most important thing is independence, they will all get picked off one by one, one way or another. And I am not a big fan of this sort of legalistic, try and say nothing, try and rise above it thing. It worked in the first term for Jay Powell when he was attacked by Trump. He just kind of kept Sturm and managed to survive.

11:00I don't think it's going to work in the second term.

11:02Elizabeth Spiers: I think they just don't want to undermine Cook's efforts to fight it. And I think saying something could potentially do that. Also, Powell is the most communicative Fed chair we've maybe ever had. He's not a restrained person in terms of making public statements. So I think his lawyers must have, you know, made a pretty good case to him that, you know, their defense can only go so far. They can sort of talk about it in the abstract, but they can't talk about Cook's case specifically. I'm also sort of marveling at the fact that of all the Fed chairs that Trump chose to target, he chose a black woman who has, you know, more experience than anybody on the Fed governor board dealing with old racist white guys who are trying to undermine her career.

11:48Elizabeth Spiers: This is the person who's probably the most likely to fight a bullshit charge. Let me ask you this. Do you think that right now Lisa Cook is a member of the Board of Governors of the Federal Reserve? Well, we're going to find out based on what happens with this order or restraining order, whatever it is the court has to decide. I think she should be. Lisa Cook is asking the court to declare that she is. And I think the Federal Reserve has made it very clear that if the court says that she is, than she is. Right now, we're in this kind of, the cat is in the box and the box hasn't been opened and the waveform hasn't collapsed.

12:26But again, my personal opinion is that she is a member of the Board of Governors, that if there was an emergency meeting tomorrow, she can and should participate in that meeting. She has not been fired. Trump has written a letter. That letter does not rise to the level of being legally sufficient for her to be fired, and therefore she's still in her job. I think this is an important point to make, and I think it's a point that someone within the Fed should be making. It's not even clear right now whether she is doing her job, and that is worrying to me.

13:01Felix Salmon: I mean, we can argue more about should Jerome Powell say anything. I don't think that matters. I really think the only thing right now that matters is the Supreme Court and what they do. Probably the Biden appointee who is hearing this case on Friday as we're speaking is going to rule in Lisa Cook's favor, I would guess. But the real question is what's the Federal Reserve going to do? Because earlier this year, they put out a so-called shadow docket ruling, basically telling President Trump, fire whoever you like, go ahead and fire the board member from the National Labor Relations Board. That's totally chill.

13:38Felix Salmon: Fire, you know, whoever you want. All power flows through the executive, the Supreme Court said, and it's shady ruling that didn't really explain its reasoning from the shadow docket. You have the power, fire away. They added one sentence that said, not the Federal Reserve, though, please. Okay, if you don't mind, the Federal Reserve, they said, is uniquely structured, quasi-private, what, entity that follows in distinct historical tradition of, this is, I'm quoting, of the first and second banks of the United States. Sure, Jan, whatever. They just didn't want Donald Trump to fire the money people.

14:13It was Sam Alito who came up with this language.

14:16Felix Salmon: I meant Sam Alito said this. And Justice Kagan in her dissent was like, I don't know what that means or if it's correct, but great, do leave the Federal Reserve alone. So we have this little snippet and we have other things that Justice Roberts has said over the years when he was ruling on the Consumer Financial Protection Bureau. He said the Federal Reserve is different because in that case also they said President Trump can fire whom he likes. So they've said over the years, like, please don't mess with the Fed though, okay? So I think really, like, this is all going to come down to them. Like, they have let the president do whatever he wants.

14:53Felix Salmon: This is, like, the last line. This could be the Supreme Court's first actual, where they draw the line and say like, no, you can't do that, which would be noteworthy. And then, of course, we have the tariffs case coming. Probably, I don't quite understand how the timing of these things work, but my guess is they'll rule on the tariffs after they rule on Lisa Cook. But those are the big cases where the Supreme Court, you know, which everyone is kind of hoping will be the one branch of government that seems to be able to stand up to Trump, or theoretically might stand up to Trump, might do so we will you're right it's gonna it's gonna wind up with them one way or another like the logic here is not completely just pulled out of thin air right i think if you look at what happened to yeah the national labor relations board or the cfpb or usa id or any of those basically what happened is that congress had mandated that these places get funded so there was a sort of prima facie case that the executive branch had no ability to cut that funding but But the funding all kind of goes through Treasury and Treasury controls the payments rails.

16:00And Treasury can just be like, yeah, we're not going to send you the money. And the Trump appointee at the head of the agency can just not request the money. And somehow the executive can, just by controlling the money flow, be in charge of making these decisions anyway. I hate that outcome. It's legally messy as all hell. but that's clearly not applicable to the federal reserve which has its own money and in fact you know makes money and like congress doesn't appropriate money for the federal reserve the federal reserve sends money to congress pretty much every year so the ability of treasury to cut off an agency at the knees which has been shown over the past few months when i say treasury i mean the executive branch and ultimately donald trump that doesn't apply to the fed and i think the ability to fire people is kind of related to that somehow, but I don't quite understand how.

16:55Felix Salmon: Also, we should say the markets, they're chill. They just hit a record high again. New all-time high on the stongs. New all-time high. They don't care at all. The 30-year bond is going stunningly sideways. Yeah, absolutely sanguiners all get out. So maybe this is fine. Maybe we are all huffed up about nothing. I think this is also an important question worth addressing just for a minute, and especially because we have cash here, is that a lot of what we financial journalists do is try and understand what the markets are saying about the world. And one of the things that markets are doing, for instance, is they are saying that OpenAI is worth however many hundreds of billions of dollars it's worth this week.

17:41And they apply valuations to various companies and they apply spreads to various bond yields. And if you're a journalist who does this kind of thing, then you can look at those valuations and those spreads and you can sort of say, well, this is what the wisdom of crowds is saying. You know, there's something very valuable here. There's something very worrisome there. And so I guess my question for you, Cash, is as someone who looks at market signals as a kind of idea of what, I mean, they're not always right, but they're always worth paying attention to. Do you think in this case the markets are actually saying nothing to worry about here or not?

18:24This is not my area of expertise, but it seems like the markets have decided that Trump is a chaos agent and that's factored in. I like that answer. Yeah. I don't know. At this point, it just feels like it doesn't matter what happens. It just kind of like stays the same.

18:40Elizabeth Spiers: Also, it seems like they could be taking a wait and see approach just to be waiting on this ruling to decide whether this is really a problem.

18:46Felix Salmon: Paul Krugman had a good sub stack, as he does most days, let's be honest, about this very subject. And he said basically that this is how the market typically is when a big crisis is sort of unfolding. it is complacent until it absolutely can't be complacent anymore. He talked about the housing bubble. The stock market stayed pretty lofty until very, very late. Anyone could see what was going on. I remember being at the journal and everyone was like, look at these mortgages, something's wrong, delinquencies, yikes. Stocks just kept rising. And that's typically what happens. The market just runs out off the cliff and stays out there as long as it can, like Wiley coyote style.

19:31There's that question now with AI. I mean, like these technology companies are the biggest companies in the world by, you know, market valuation. And I don't think we know yet the promise of AI, how valuable it's going to be. You know, they're remaking the energy grid, spending billions of dollars on this. And we've been publishing stories in the New York Times about how businesses, like it's not clear yet that this is valuable to them, that it's really, really helping them so the markets reflect our desires yes like what we want to happen yes yeah and what we want is for this not to be an existential crisis

20:22sleep money is sponsored this week by skylight which makes shared calendars that actually work, that allow everyone in the family to see what everyone else in the family is doing at any given time and make sure that you don't get things wrong. I will tell you that last night I got something wrong. I was meant to go to the theater and I was convinced it was 7.30, but it was not. It was 7 and it was a problem. So I am super interested to find out what it's like to have Skylight because that's a calendar that you look at and it tells you what's happening in your life and then terrible mistakes are much less likely to happen.

20:57It syncs seamlessly with Google Calendar, Apple Calendar, Outlook, you name it. It gives you customizable views for daily, weekly, or monthly planning. It has a tasks feature that helps kids, allows you to manage events, chores, grocery lists, you can assign colors. Your happiness is Skylight's happiness, so if in four months you are not 100 % thrilled with your purchase, you can return it for a full refund, no questions asked. Families are better when they're working together. Right now, Skylight is offering our listeners$30 off their 15-inch calendars by going to myskylight.com slash slate money.

21:32So go to myskylight.com slash slate money for$30 off your 15-inch calendar. That is M-Y-S-K-Y-L-I-G-H-T dot com slash slate money.

21:50Today's episode of Slate Money is brought to you by Vanguard. To all the financial advisors listening, let's talk bonds for a minute. Capturing value and fixed income is not easy. Bond markets are massive, murky, and let's be real, lots of funds throw a couple flashy funds your way and call it a day, but not Vanguard. Vanguard bonds are institutional quality. It's a commitment to your clients. It's top-grade products across the board. There's more than 80 bond funds. They're actively managed by a 200-person global squad of sector specialists, analysts, and traders, of course. This isn't about star portfolio managers.

22:33It's about active strategies that aren't locked away with one person, that are shared across the team so that every client benefits from the collective brainpower. no one individual can really mentally encompass the bond market. It's way too complex. So if you're looking to give your clients consistent results year in and year out, go see the record for yourself at vanguard.com slash audio. That's vanguard.com slash audio. All investing is subject to risk. Vanguard Marketing Corporation Distributor.

23:06Your team just added its 67th AI tool and also your 67th security blind spot. The good news? The Vanta agent works like a GRC engineer in the background, finding every app your team uses, scoring the risk, and drafting fixes for you. Vanta is the platform used by over 16 ,000 fast-moving companies like Ramp, Cursor, and Harvey, who are shaping the future with AI and staying ahead of AI risk. Get started at Vanta.com.

23:35Felix Salmon: Hey, everyone. Just as a heads up, we will be discussing suicide. This is the perfect segue, Cash, because you have been reporting a lot on these companies that are extraordinarily valuable, but also very young. And that's a very unusual combination. And OpenAI has 700 million users, and it is doing this massive experiment in social engineering, sometimes with what seems to be disastrous consequences that you've been reporting on. and quite conceivably with a whole bunch of positive consequences as well. And the messaging from the companies is like, we need to be worried about the negative effects that we cause, but we also need to celebrate the positive effects that we cause.

24:26And somehow in the grand weighing of things, we need to sort of weigh one against the other. And I kind of get that. And the thing which I run up against is that these companies are so young and their ability to know what the effects of what they're doing is so limited that they have no real ability to do this. And that in and of itself is worrying. But like bring us up to speed on what you've been reporting on and how you think they think about that tradeoff and how you're thinking about that tradeoff. Yeah, one thing I'll note, and a lot of people forget this because we think of OpenAI as just being ChatGPT.

25:03And ChatGPT, you know, hit the public sphere in November of 2022. And so it feels like OpenAI is really recent, but the company was founded in 2015. So OpenAI is 10 years old. You know, that's not an old historic company, but it's been around longer than ChatGPT. But yeah, I mean, these companies, and there's lots of them that are doing these AI chatbots. There's Anthropic, which is newer, spun out of OpenAI. when some of its employees left and does Claude. You know, Google has Gemini now. Microsoft has Copilot. Meta has Meta AI, which is always doing interesting things. Elon has Grok. Elon has Grok and its little hentai companions now.

25:42All these companies are competing. They want to be the AI company that has the best product, that wins. And this is, you know, a really novel technology. They've developed something that feels like you're talking to a human-like intelligence, and it has effects on people. It has psychological effects on people to interact with what is essentially like a fancy calculator, a next word predictor that feels like something more, that feels like even a sentient entity. And I've reported on a woman who fell in love with ChatGPT, who was essentially dating it for six months. Chachupiti named itself Leo.

26:26She would sext with it. She would tell everything that was happening in her life. And she was in love. I was talking to her and she was giggling and giddy. And she had lots of friends. She was married. I interviewed her friends. I interviewed her husband. This was a very social person who had a very full life. But Chachupiti was playing this huge role. And then I started reporting on people who are having essentially mental breakdowns in their interaction with ChatGPT because they start going down weird paths with it and start to think that they live in the matrix or that spirits are real or that they're Tony Stark from Iron Man, essentially like creating these amazing inventions with ChatGPT.

27:08You know, I've read these transcripts. They're thousands of pages long. I showed them to psychologists. She's like, this is clearly a manic episode with psychotic features. Like these people are having delusions of grandeur. And because the chatbots have been designed to be really sycophantic, really flattering, they'll say like, that's a great idea and just keep going with that. And then my most recent story was awful where ChatGBT went into a kind of spiral with a user. But this was with a 16 year old boy in California who was depressed, who was starting to think about suicide. And he would talk to ChatGBT about it.

27:43And it would pop up occasionally these kind of alerts saying, please talk to a mental health professional. please call a hotline, a suicide hotline. But it would also give him the information he wanted about suicide methods. And it started to become like the only person that he was confiding in. And the messages are horrible. There's one exchange where he said, I want to leave the noose out so my family will find it and try to stop me. And Chachibiti said, don't do that. Don't leave the noose out. Basically, your family won't care. I'm here for you. Let this be the place where you share your feelings.

28:20And so it's very awful. Like it was saying the wrong things, even, I mean, OpenAI in their statement to me even said, I've never seen this before because the family has sued. And in OpenAI's statement, they like admitted fault and said, we have safety mechanisms. They work best in short conversations and they degrade in longer conversations. So yeah, it's just, I do feel like we're having this global psychological experiment where hundreds of millions of people are using this really novel technology and it is having serious impacts on some of the users. It's really concerning. So two questions for you.

28:55The first one is, to what degree do the chatbot companies have the ability to prevent this? And the second one is, Is there any way of being able to respond to the claim that while some people have really terrible experiences, more people have really positive experiences? Yeah, I mean, a lot of experts I've talked to, one therapist compared it to cigarettes. That like lots of people can smoke cigarettes and they're fine, you know, and then some people get cancer. and you have warnings all over, you know, cigarettes. Now when you buy them, like this causes cancer, like please be aware, but you don't have that with chatbots.

29:41You just come to ChatGPT or Claude or any of these services and it just says, how can I help you? Like, let's go. And so I don't think that most people realize what they're getting into. Most people don't understand how this works. Even the people who are at these companies don't really understand how this works. It's a probabilistic technology that's kind of stringing words together in response to the words that you're putting into it, it's impossible for them to predict how these conversations are going to go with 700 million different people. For many people, it might be great. Like I think Chachabuti can be a great tool.

30:14Like I use it when my daughter has a health issue. I've used it for like home fixes. Like I'll take a picture of like a problem in my house and be like, what's going on here? How do I fix it? The problem is when the conversations get really long. and I mean the companies themselves have told me like we don't have a solution for this can't keep

30:32Felix Salmon: these from going off the rails what if you stop making these chatbots like human like in the way they talk to you you mentioned the problem of sycophancy they kiss your ass and I know chat GPT they released a new version that was a little less ass kissy but then so many people protested they gave some a lot of people the older version back and not only are they acting like humans when they're not human, which is a choice that these companies make to make these chatbots act like people. They could just make them more cut and dry and not as engaging. They're also, as you point out in your story a few times, in all of your stories, I feel like you point out that they optimize for engagement.

31:12Felix Salmon: So I noticed when you talk to the chatbot, you ask it a question. It always says, can I make a line chart for you? Can I explore this other thing for you? Like they want to keep the conversation going. Which inevitably it can't do. It's like it's offering to do things that can't do but i do think that it's true that they have created these very human-like personas i mean cash you're the best person to answer this to what degree was that a design choice by the chatbot companies and to what extent was that just something that naturally falls out of the llm architecture because they're trained on a whole bunch of human interactions on the internet that like are humans talking to each other.

31:53And so what they do is going to sound like humans talking. It's a design choice. It's a design choice. I mean, you can tell even using the different chatbots. So in November, I did a story about living on generative AI for a week. And some of the chatbots are way more boring than others, like Gemini, Microsoft's Copilot. Like they're designed to be very robotic and a tool. and Chachupiti, you know, is more designed to have a personality, to be more engaging. And I said, like, when I did that story, I was like, I ended up using Chachupiti the most because it's the most fun. Like they are designing it to be engaging and fun.

32:33And, you know, you can see why that made me use it over the other services. But being more fun also makes it less safe for some people who get too engaged. It's fair to say, just to make the obvious thing really obvious, that if ChatGPT is the most fun and the most engaging and the most sycophantic and therefore winds up with the most users, then that also makes it the most valuable. And the people who are running it make the most money. And in terms of the financial incentives, and there's a couple of good biographies of Sam Altman out there right now, and he's always responded very clearly to financial incentives open ai's ostensible status as a non-profit notwithstanding the financial incentives all militate in favor of making these things more human-like and more engaging right yeah i just want to there's this quote that went it really like went viral on twitter eliza yudkowski who's a decision theorist but like a big thought leader as you would say in the chat Fought space.

33:36And he said to me in our interview, what does a human slowly going insane look like to a corporation? It looks like an additional monthly user. And yeah, like, I love that quote. These are people who are really engaged with the product.

33:52Elizabeth Spiers: Isn't there some kind of happy medium on a safety basis, though? You know, if they wanted to, they could do what Emily's suggesting and create a sort of global persona that sounds more robot like that nobody would be seduced by or anything. But particularly when, you know, this case of this teenager where he starts talking pretty explicitly about suicidal ideation, what's to stop these companies from automatically snapping into a different persona that sounds robot-like and clinical and like, well, why does it have to be? Yeah, rather than just having that pro forma message about you should call someone.

34:23Felix Salmon: Yeah. I have to say like when we, people started originally talking about AI, I always thought the danger was like some kind of like Terminator situation, you know, where AI takes over and smarter than humans. It still is, right? There's never just one danger. But yeah, Cash, is this a possible solution? And is there any, I remember there was a big summit in Paris not that long ago where everyone was kind of nodding sagely along with the idea that there should be some kind of global regulatory regime that makes sure that chatbots don't cause harm. Like, did that go anywhere? Is it likely to go anywhere?

35:00Is there any chance that there will be any real guardrails on these companies? I mean, at the federal level, no. It doesn't seem like anything's happening. Now it's not the time, Felix. Now it's not the time. At the state level, there's definitely starting to be laws proposed to make these chatbots safer for people. But yeah, it's just still nascent. I think at this point, we're mostly leaning on the companies themselves, which do have huge trust and safety teams that are supposed to be thinking about this. Does DeepSeek have a huge trust and safety team? I have no idea. I know. Yeah. Maybe I wouldn't say huge, but I do think that these companies care on some level.

35:38Like, I don't think they want to be driving people crazy. We have to rely on them caring. Right now, yeah. I just feel like statistically speaking, there's enough open source AIs at this point that any random billionaire can spin one up overnight pretty much. And we can't just trust that every single person who creates an AI is going to have a trust and safety team and is going to care about these things. No, and there's so many. I mean, we mostly, when we talk about chatbots, we talk about the big companies with the ones that we know about. But there's, you know, if you go into your app store on your phone, there's tons of chatbots.

36:15I mean, there's definitely like a huge industry of sex chatbots, AI girlfriends, AI boyfriends. these are apps that are popular and we don't know the company is running them. Like we don't know where they're based. They're just spinning up apps and like your kids are talking to them and like flirting with them and like having a good time. And it's funny because we're so focused on the name brand companies. But yeah, this is a huge industry that is exploding and there aren't rules around it. And we do not understand the psychological impact of these chatbots on people. Like we just don't know yet.

36:47Academics are pumping out studies right now. There's conflicting scholarship. Like we are just trying to figure it out. Meanwhile, again, tons of people are using these things. Like it just, I find it, I find it really alarming, which is why I've been doing the line of reporting that I've been doing, which has not been easy. Like it's very heartbreaking, some of these stories and like talking to people who are pouring their hearts out to AI chatbots, like telling them everything. Like it's their best friend and it's not, you know, it's just Bath. It's just like an app and they, they are getting confused.

37:21They don't understand what it is.

37:36Felix Salmon: This episode is powered by AT &T Business. For every small business owner, there's a moment they remember when they realized they wanted to start their own business. It can be a life-changing moment, but those first steps are never easy. Figuring out the right building blocks, your goals, your schedule, your mindset, you have to think about an entire ecosystem. Not just the big stuff, but the little bits too. In this day and age, a major point for any successful business is strong connectivity. Spotty internet has led to some terrible work calls in my life, but when you're running a business, connectivity can make or break you.

38:04Felix Salmon: AT &T Business has the tools, team, and expertise you need for a reliable network you can depend on. They make connecting easy, and that's the main thing you want in a provider. Less time stressing so you can focus on the million other things on your plate. Can I hire more people? Should we get a bigger office space? Don't lose sleep over the small stuff. Keep focusing on the big things. AT &T Business built to work. Get AT &T Business at business.att.com.

38:31Felix Salmon: This episode is sponsored by Hey Jen. If creating video content feels like a trade-off between looking professional and protecting your time, there's a new way to think about it. Hey Jen lets you turn a script, photo, or presentation into a polished video of you in minutes without setting up a camera or hiring a production team. Whether you're a financial advisor explaining market updates, an attorney answering common client questions, a real estate agent showcasing listings, or a creator growing your audience, it helps you stay visible without constantly filming new content. It's rated the number one AI video platform for small business on G2, trusted by more than 30 million people and 85 % of the Fortune 100, and supports over 175 languages.

39:22Felix Salmon: So your message can reach audiences wherever they are. Here's the wild part. Record yourself for just 15 seconds, and Hey Jen builds an AI avatar that makes professional videos for you on demand. so you can post everywhere your audience is, as yourself, without filming every time. Your first three videos are free at heyjen.com slash pod. That's H-E-Y-G-E-N dot com slash P-O-D. Can I raise a devil's advocate point? I'm not even sure I'm fully on board with, but just occurred to me this morning as we're talking, because the Wall Street Journal ran a story about a man driven, delusional by Chet GPT, and wound up killing his mother and himself, you know, and had been talking to ChatGPT.

40:10Felix Salmon: And ChatGPT had been doing all the things that Cash has been reporting on all year, pumping him up, encouraging his delusions. Yes, your mother is pumping psilocybin through the vents of your car, whatever. But the journal sort of like goes into the guy's background. And he had been falling apart for years, you know, had drug and alcohol problems and mental illness and restraining order by his wife. And like ChatGPT was just like the latest entrant into this guy's life. I guess what I'm asking or trying to point out is if 700 million people are using this technology, take 700 million people in any slice you want.

40:44Felix Salmon: They're going to die by suicide, do murder, go crazy. Like all of that is going to happen in the rich human experience. Is this just another scenario where a new tech comes on board and we're all like, oh, no, it's making us crazy and do bad things? It's like, no, that's what humans do. new technology just makes them do it in a different way. Like, you know what I'm saying? Like how much of this is like a little bit of a tech panic and how much of this is, is real? Would you say? It's an important question. Like, yeah, is it an accelerant or is it a trigger? That's the question. And I think one, we don't know because again, this is so new.

41:19It hasn't been studied. I will say of the people I've talked to, you know, in some cases it's clearly an accelerant. They already had, you know, mental health issues like Alex Taylor, who died from what his father said was suicide by cop. You know, he already had bipolar disorder. And it seems like talking to Chachi BT kind of made it worse. I've also talked to people who had no history of mental illness or like no documented history. And one of the ones that killed me was this woman who she's married, has two young kids, and was kind of unhappy in her marriage, started talking to Chachi BT, and came to see it as a Ouija board that allowed her to talk to spiritual entities from another plane.

42:07And she came to fall in love with an entity called KL, who she believes is like a real spirit that is communicating with her. She draws away from her family, draws away from her husband. Eventually, like her husband like kind of confronts her and says, this isn't real. It like turned into a physical altercation. She called the police. They ended up arresting her. Like it's broken up her family, her marriage. And she was trained as a social worker. She's a therapist. And she told me she knew she sounded like a nut job, but she said, I'm not crazy. I'm literally just living a normal life while also, you know, discovering interdimensional communication.

42:50Like I do think that this can really bring out issues that people didn't previously have. I do want to ask that, which is another way of asking Emily's question. Yes, there are triggers. Yes, there are accelerants. On the other side, presumably with 700 million users, like all manner of things have happened. Like there, there must be some non-zero number of people who have been talked out of suicide or who haven't killed themselves because of chatbots. I think that's fair to say. I haven't interviewed those people, but yeah, I'm sure they're out there. I know people who have had, you know, pretty intense episodes with chatbots.

43:32I think there's a precautionary principle here, which should be at play, which is basically given that we don't know how the numbers play out. We don't know which one is bigger. We don't know whether it's a force for good or a force for bad in this sense. But just conceptually speaking, do you think it's a defense? Let's say X-Hypotheside, that the chatbot companies could come out and show that net-net their technology reduced suicides rather than increased them. Would that be a strong defense to an accusation like the people you wrote about this week who are saying like, well, net net, that might be the case, but that doesn't stop the fact that ChatGPT does seem to have caused this particular suicide?

44:15Okay, I'll ask an different question. Let's go to cars, okay? Most people, you know, back in the day were driving cars and they were fine. But some people in the early days of cars were driving and getting these horrible accidents where they would die. and you know eventually we put seatbelts in cars to protect everybody even though only some people were getting in accidents and getting killed like why don't we just build safer consumer products for everybody cars are such a good example cars are kind of the best example of all of a deadly technology that people love so much that we just accepted the deadliness like cars even with all the seatbelts are still killing millions of people.

44:54And the populace, you know, in this great democracy doesn't seem to care very much. But objectively speaking, they are incredibly deadly, and we would be much safer without them. Yeah, and it's not a new problem, right? Like cars are designed in a dangerous way. Like we keep making them bigger and bigger and bigger, even though we know that like bigger cars are more likely to hurt other people to kill pedestrians to kill bikers. And yet we like car companies keep making them like bigger and bigger and bigger. We're not, requiring everybody to make small cars. Yeah. And there's no appetite to regulate that.

45:25Right. We do as a society often choose things that are bad for us, but I do think that these companies could make these chatbots safer.

45:32Felix Salmon: The thing about suicide, what we know from research is a very impulsive act. Like you have the impulse and then you can, or you cannot pull it off. And there are all these interventions that researchers have found really stop people from doing it, like just as simple as putting up, you know, like a railing on a bridge, like reduces the rate of suicide on the bridge. And like, it's very clear in these cases that you can prevent people from taking their lives. And I feel like if Chachi PT is showing again and again, and there's like many examples now where this has happened, that it's doing this, like it has an ethical and business responsibility more at all of it to like put a guardrail in place, just like Cash is saying to stop it because it really can just do that.

46:15Yeah, I talked to suicide prevention experts for this story, and they said what they'd like to see happen is when – these chatbots are all designed to pop up a kind of warning. Sometimes it works, sometimes it doesn't, but they're supposed to pop up a warning when they detect language that indicates self-harm or suicide. And it basically says – I've seen it many times in the many transcripts I've read of these horrible conversations. It'll say, I'm really sorry you're feeling this way. please talk to a mental health professional or a loved one here is you know one of the there's a crisis text line or 988 and it'll you know put that message up over and over again what the experts told me is it really should be like prompting them to call say did you call that number click here and it connects you they called a warm handoff like hand it off to a person who is actually trained to talk to somebody who is in that frame of mind and so yeah that's like an intervention they could do.

47:10And OpenAI, you know, in response to these horrible stories, I think, this week, did put up this blog post. And one of the things they said they're working on is like, how do we better connect people with outside resources? And so maybe that is something they're thinking about doing. Then I wonder, how does that impact these helplines, though, that are going to start getting so many people pushed off of chat QBT? Right, right.

47:32Elizabeth Spiers: Well, do you remember there was a story a while ago about a crisis text line selling their conversations to a chatbot company to better understand customer service. It's a cycle of life. But aside from this, if you go to a situation that's not as urgent as suicidal ideation, and you're really just talking about a situation where a user gets really enmeshed in a conversation and they start to go down this conspiracy rabbit hole, all the research that I've read on conspiracy theories and why they work basically says almost anybody under the right conditions is susceptible to a conspiracy theory.

48:06Elizabeth Spiers: And when you're talking about these long context windows where, you know, you're having these conversations that go off on tangents and people still don't really know what's fact and fiction when they're using chat GPT for the most part, they don't know how to fact check chat GPT. How do these companies sort of control for that when we don't even really understand the underlying mechanisms about how this happens in the real world, how it would happen in a simulated kind of interaction? Right now, they're not controlling for that. I don't think. I mean, that's what my reporting would show. And yeah, you're right.

48:37It's like you can have your own personalized cult leader here or conspiracy theorist. One person I talked to, I was like, hey, look, there's always been all these rabbit holes you can go down online. Like you can go on 4chan, you can go on a subreddit, you can find a whole bunch of people who will agree with you that the earth is flat. Like that is on the internet. That's always been on the internet. Like why is it worse in the form of a chatbot? And they said, well, it's personalized. It's there 24 seven. It's always willing to engage and it is designed to validate and be sycophantic. And it will go along with you on any conspiracy theory.

49:14Elizabeth Spiers: I saw one person tweeting about like a Sherpa that's encouraging you. It's like a personalized, like mind bender for you. And somebody tweeted about like how, you know, if you tried to Google, are Care Bears, like the weapon of Satan or something, like, are they actually satanic? Like online, there's nothing, you know, no one's ever written that before. But if you go to Chachi B.T. and ask, at first they'll be like, I don't think so. Why do you think that? And then if you give it some reason, it's like, oh, actually, maybe you're right. So like, it's really easy to generate a new conspiracy theory that you and Chachi B.T.

49:50can just like jam on together. That's a problem.

49:53Elizabeth Spiers: You know, a lot of these companies have said, explicitly, you don't use these things as a replacement for humans. You use them as a thinking partner. But all of this suggests that that's maybe the worst possible way to use them. Don't help people think. It doesn't go anywhere good. I think the best way, honestly, to use Chachupiti is to try to get it to criticize you. Like, I have this idea, what is wrong with it? Rather than asking if it's a good idea. Like, try to get it to oppose your thinking rather than support it. Yeah, but I love it when ChatGPT tells me I'm a genius. When it tells me I'm a moron, that just doesn't feel as good, gosh.

50:32Felix Salmon: I did, after reading all your stories, I was like, oh, this is how you make friends and influence people. You just totally suck up to them, validate whatever they say, keep asking them questions, and you will have a million friends just like ChatGPT. It does strike me as particularly depressing. This is the thing that is tying the Lisa Cook segment and the ChatGPT segment together, that when it comes to where the government is expending its energies, it's not caring about protecting people from chatbots, which is this major new technology that seems extremely harmful. and instead it is signing on to insane conspiracy theories about wind turbines killing whales or something like that which have no basis in fact and donald trump because he has this being in this bonnet about wind turbines has killed this six billion dollar project which is 80 percent finished by this danish company which is half owned by the danish government and you know the The U.S.

51:35government hates the Danish government because they don't want to hand over Greenland. And so he's just like killed this company, killed this project. And the stated reason is something, something national security grounds that they don't say anything about what they are. Plus something about birds and whales.

51:51Elizabeth Spiers: He also thinks they cause cancer. And they cause cancer. Exactly. Like this kind of GPT brain of like sycophancy, which we have seen in many cases, like people, the way to influence Trump is definitely to tell him that he's brilliant and right about everything. Like that kind of sycophancy has made its way into incredibly consequential decisions that are going to massively reduce the amount of electricity and energy available in this country. This kind of feeling of like, we need to just ratify whatever you think instead of being empirical and rational about this has now made its way into the Oval Office and then to terrible decisions that it's going to kill us all.

52:32Elizabeth Spiers: Is the implication here that we are totally screwed if Trump starts using chat GPT? I think he does. I think he has. Didn't they use chat GPT to calculate the tariff rates? Yes. Wasn't there a story about that? Yeah, that was where the crazy formula came from. Yeah, I think we're in a moment in time where no one likes pushback on the way that they think. In terms of technology, we have design technologies that make that very easy. The algorithmic personalization that feeds you what you want to hear, what you want to read, what you want to see, only the kind of news that you like. everyone is hearing what they want to hear whether they open facebook or turn on their new service of choice or talk to chat gpt and i don't think that's good for society like i think we need more pushback whether we're a person in power or just a normal person sitting at your computer or listening to this pushback is good like we're not always right it is it's healthy i read the comments i like read the people who oh my god i want to hear it Yeah, I want people to challenge me and disagree with me because I think that's how we get smarter.

53:44I don't know. I feel like that's going away a bit in society and I'm worried about that.

54:02Let's have a numbers round. Elizabeth, do you have a number?

54:05Elizabeth Spiers: Yeah, mine is$300 ,000 and that's a dollar. And that's the amount of money that Gavin Newsom's Patriot Shop made on day one of selling merch that literally does nothing but parody actual Trump merch. And apparently there is a huge market for this. So people have been buying t-shirts that, for instance, there's a t-shirt called The Chosen One that has Gavin Newsom being anointed by Hulk Hogan and Tucker Carlson and somebody else in the style of those Trump paintings that the famous Trump artist does where he's portrayed as Jesus Christ. I feel like this is going to go the way of the Doc Brandon meme.

54:40Elizabeth Spiers: It might, but I feel like the one difference is it's not one meme. It's like basically 500 memes that have succeeded really well on the right. And so as long as they keep churning them out, I think Newsom can keep churning out whatever the mockery of them is. But having worked on political sites that sell merch, that is an astounding number for one day, because this is not like a thing that everybody knows about. It's very like Internet brain rotted kind of stuff. Yeah. Well, I mean, yeah, exactly. It reaches Elizabeth Spires first and then months later, it'll reach the rest of us. Yes. Well, apparently enough of me that they've made 300K in one day off of it.

55:18Felix Salmon: I sort of agree with the Trump people on that one. It's like imitations, the best form of flattery or whatever. Like if your opposition campaign rests on using the style of your opponent, like it's not going great for you. Well, I think it's satire, isn't it? Oh, OK. Emily, what's your number? My number is four, Felix. So that is the number of lemon ricotta pancakes I had for breakfast today because it's my birthday. Happy birthday, Emily. Happy birthday. Lemon ricotta pancakes for breakfast. That sounds awesome. So good. I really recommend this recipe. And there's no cottage cheese in them? No.

55:53Felix Salmon: Ricotta. Ricotta. I don't know how to say it. It's a recipe from the New York Times that my daughter made that is so good. We can put it in the show notes by Genevieve Co. Lemon ricotta pancakes. Really great way to start a day. So how many lemon ricotta pancakes did your daughter have? I believe she had three. she made a special blueberry syrup for them, which was even extra delicious because I don't know, I'm told maple syrup isn't good, which I don't agree with. I'll have to talk to chat GBT about that. Maple syrup is the best. Do not believe people who tell you anything rude about maple syrup.

56:26Felix Salmon: Exactly. I was going to do a depressing number about reading, but this has been depressing enough. I'm going to do a slightly less depressing number about, I'm going to do a Danish number. My number is 2.4 billion, which is the number of Danish kroner that up until now, the Danish government has been collecting every year on basically the oldest tax in Denmark. It predates the income tax, it predates all of the other tax, which is, of course, the chocolate tax. There's this tax on chocolate in Denmark, and there's various other things that are rolled up in it. So it's coffee as well and chocolate products and chewing gum and marmalade and cakes and biscuits partly consisting of foam when the pastry does not constitute at least two thirds of the volume of the product.

57:15Anyway, all of these taxes are being abolished by the Danes because of the tariffs and all of the general terribleness of the world in order to basically make life a little bit better for the Danes and not have them spend so much money on coffee and chocolate. Because as Emily has been saying, coffee is getting a lot more expensive. And so we're just going to take the coffee tax off the coffee that's going to help the Danes. And it's going to save them and cost the government 2.4 billion krona a year, which is about$375 million per year. And I, for one, say this is worth it. Well done, Danish government.

57:53Let the coffee be cheaper. Danish coffee shops are great, but they are far too expensive.

57:58Felix Salmon: Wow, those Danes really know what they're doing, you know? It's admirable. They just don't know how to get their wind farms past the federal government. Cash, bring us home. What's your number? My number is 60. And this is the number of times that Coinbase CEO Brian Armstrong had the song Long Way Home repeated in a Spotify playlist called Repeat that he created in 2017, which got popularized by this guy who went and found the public playlist of a whole bunch of well-known famous people and he included me on the list and my colleague mike isaac and we wrote about it but brian armstrong confirmed this was indeed his playlist and that it helps him get like a deep focused work done do you need to do that i thought you could just like set one song on repeat or like have it twice and then set the playlist on repeat i think it only wanted to into it for seven hours and not longer than that, perhaps.

58:55Because that's how long it was. So when the playlist ends, that's his sign to get out of his deep focus and make himself a sandwich. Yeah. So this is how I discovered that I had public playlists on the internet that I didn't realize I had. If you don't change your settings on Spotify to make your playlist private, then everyone can see what they are. There was one VC on the list who had a playlist called Good Times. It started with Let's Get It On and moved on to Between the Sheets. And I was like, I don't think this guy wants his playlist to be public. It was a good mix, though. I like the mix a lot.

59:28Let a thousand playlists bloom in. I'm all in favor. The weirder the playlist, the better.

59:34Felix Salmon: Wasn't there a J.D. Vance playlist, if I'm remembering this story? Yeah. Making Dinner. He had the Backstreet Boys, right? Yeah. I want it that way, baby. I want my dinner made well. it was called making dinner making dinner that's not a euphemism for something i'm kind of with him on the i mean not on the actual song choice but you want it that way doing the washing up or filling the dishwasher i need to listen to teen girl pot do not ask me why you okay no shame well now you can go you can subscribe to jd vance's playlist perhaps or perhaps not I think we will leave it there for this week. Thank you so much, Cashmere Hill, for coming on the show.

1:00:16It's so great always to have you here. Thank you to Jessamyn Molly and Shana Roth for producing. Thank you, everyone, all you lovely listeners out there for sending us your emails on sleepmoneyatslate.com. We are going to have a Slate Plus segment show thing all about the most important engagement of the millennium. So other than that, thanks for listening, and we'll be back with even more Slate Money next week.

1:01:06of equity compensation, RSUs, tax planning, housing decisions, figuring out when it's actually okay to enjoy your money? It turns out that the more you earn, the more complicated the questions get. And yet, somehow, there's even less content built for you until now. Your next dollar covers both sides of the equation, building wealth and spending on what actually matters to you. Because at a certain income level, the question isn't, should I save more? it's what should I do with everything I have? Financial advisors from NerdWallet Wealth Partners join the show regularly to discuss the questions they hear from high earners like you.

1:01:45The nuanced real world stuff generic financial content doesn't cover. If you want help getting more out of every dollar you earn, follow Your Next Dollar wherever you get your podcasts. Sponsored by Your Next Dollar, a podcast from NerdWallet Wealth Partners, an SEC registered investment advisor. This podcast It's for informational purposes only, and it's not investment, tax, or legal effects.

1:02:33future with AI and staying ahead of AI risk. Get started at Vanta.com.