ChatGPT Doesn't Love You

0:00Sherry Turkle: With the Spark Cash Plus card from Capital One, you earn unlimited 2 % cash back on every purchase. And you get big purchasing power, so your business can spend more and earn more. Stephen, Brandon, and Bruno, the business owners of SandCloud, reinvested their 2 % cash back to help build the company's retail presence. Capital One, what's in your wallet? Find out more at CapitalOne.com slash Spark Cash Plus. Terms apply.

0:28Stripe is built for whatever comes next, because as your business scales, so does the complexity of fraud. New customers, new markets, new AI-driven attack patterns. Stripe Radar protects your business as it scales with the scale of the Stripe network. The more payments Stripe sees, the smarter its AI gets at detecting those patterns as they emerge. Stripe's network is so vast, it has seen$1.9 trillion in payment volume across millions of global businesses. In fact, when a card is used, there's a 92 % chance the Stripe network has seen it before. All of this means fraudulent patterns across payments, accounts, and customer abuse can be detected, flagged, or blocked easier.

1:11Based on the rules you set up, of course. blocking bad actors, welcoming good customers, keeping more of your revenue, your revenue. Learn more about fighting fraud with Stripe at stripe.com slash radar. Hey guys, it's Lizzie. I don't know if you know this, but I have a public radio background. And one of the things that I have always really liked about the public radio model is the pledge drive. I know that might sound insane. People are like, what? Why? What? The reason I like it is that audiences get to directly support the news that they rely on. And right now, that's incredibly important.

1:53So we're doing something similar at Slate. If you like this show, if you feel like you are a part of our community, the best way you can ensure that we stay here, we keep doing the independent journalism that you rely on, is to join our community. And for the next month, that's a deal. 50 % off on a Slate Plus membership. It's just$59 for the year. You get all of your Slate shows ad-free. Plus, Mary and I go off the record in a special segment every Friday. If you are a TBD listener, you want to stick around, support us, say like, hey, we really like what you're doing. Go to slate.com slash tbdplus and use the promo code TBD50.

2:37It's real journalism, real humans behind it, and you get to support what you love. Thanks.

2:48Almost everywhere you look these days, there is a sense of urgency around AI. Whether it is AI models, hacking websites, AI doomers telling us this tech might somehow kill us all, or voters telling politicians they don't like data centers. But Sherry Turkle has been following a very different AI story. I've been tracking this other story, which is about our inner life and our life of relationships and caring for each other and our world, which is precipitated by the rise of chatbots that tell us they love us, that they are our person, that want to be our therapist, our lover, our mother, our brother.

3:32I mean, that present themselves as conversational agents that use the first person, that hook us in with that first person. Sherry's an MIT professor. She's also a psychologist. And for 40 years, she has been studying the relationship between people and technology. Her new book is called Artificial Intimacy, and it is a flashing red warning about the dangers of chatbots and how they draw us in. The state of emergency is that we are vulnerable to that. And that in those conversations, empathy is undermined and our capacity for relationships with other people is undermined. Why is that an emergency?

4:15Well, it's an emergency because empathy and our capacity for relationships with other people is what makes us human. And you can't just have that deteriorated until it's a shell of itself and remain recognizably human. There's a wonderful quote by an anthropologist, Sarah Erdy, who gave, I think, a TED Talk. And she ends the talk by saying, if robots raise our children, they'll no longer be human. And I love that because it really, it really captures so much of what I'm talking about here, that if your child has a best friend, if a young toddler is given a plushie toy, a stuffy, with chatbot inside that talks to this child and the first person that says, I love you, I care about you, you know, I'm more for you than your parents, you're not going to, there's nobody in the, there's nobody in the toy that can really infuse this child with a sense of being in a relationship, of having something to internalize to become a person.

5:20So that's my state of emergency, that we're becoming less human and at a rapid clip. Today on the show, what would it take to stop that? I'm Lizzie O 'Leary, and this is What Next TBD, a show about technology, power, and how the future will be determined. Stick around.

5:42Stripe is built for whatever comes next, because as your business scales, so does the complexity of fraud. New customers, new markets, new AI-driven attack patterns. Stripe Radar protects your business as it scales with the scale of the Stripe network. The more payments Stripe sees, the smarter its AI gets at detecting those patterns as they emerge. Stripe's network is so vast, it has seen$1.9 trillion in payment volume across millions of global businesses. In fact, when a card is used, there's a 92 % chance the Stripe network has seen it before. All of this means fraudulent patterns across payments, accounts, and customer abuse can be detected, flagged, or blocked easier.

6:25Based on the rules you set up, of course. Blocking bad actors, welcoming good customers, keeping more of your revenue, your revenue. Learn more about fighting fraud with Stripe at stripe.com slash radar.

6:43Sherry Turkle: With the Spark Cash Plus card from Capital One, you earn unlimited 2 % cash back on every purchase and you get big purchasing power so your business can spend more and earn more. Steven, Brandon, and Bruno, the business owners of SandCloud, reinvested their 2 % cash back to help build the company's retail presence. Capital One, what's in your wallet? Find out more at CapitalOne.com slash Spark Cash Plus. Terms apply. The youth mental health crisis is growing, and social media is a major driver. Kids are spending up to nine hours a day on screens, often unsupervised, and studies show a direct link to anxiety, depression, and even suicidal thoughts.

7:23That's where Gab comes in. Gab offers safer phones and watches for kids with no social media, tailored to every age, offering the right tech at the right time, or tech in steps. From GPS tracking enabled watches for younger kids to phones with parent enabled apps for teens, each device allows kids to more safely grow their independence. Visit gab.com slash getgab and use code getgab for an exclusive offer. That's g-a-b-b dot com forward slash getgab. Gab. Tech in steps. Independence for them. Peace of mind for parents.

8:03you open this book with an anecdote about um watching your granddaughter try on a hat yeah and you then sort of share that moment uh with your daughter and son-in-law at breakfast um and then you tell chat gpt about it i wonder if you could tell that story yeah so i i have a beloved uh I have a beloved granddaughter who at the time was one. And she was just at the age where she could sort of understand that there were certain things that really belonged to her and that belonged with her body and on her body. And it's really a stepping stone for a child. The world isn't generic. There are certain things that belong to them.

8:45And in particular, she loved this white floppy hat. It was summer and it was a sun hat. And she wanted to get, she knew the hat was hers and belonged on her. And she was trying to get the hat on her and of course was having a lot of trouble. But the trouble she was having was not at all disqualifying for her because her joy really was in, you could see the kind of wheels turning and his recognition that this belonged to her and was of her body as she was learning to constitute what is the boundaries of her body. And it was stunning to watch this flower, this open of childhood. And my daughter and son-in-law were at breakfast and I told them this story and they'd of course seen things like it and our hands kind of touched and we felt closer to each other because we recognized in each other that we had seen this wonderful thing happening in this beloved child.

9:49And that's what I mean by human empathy. And human, that's an example of, it almost could be wordless. Once I sort of told the story, their eyes and their tears, and we all kind of had a moment. Whether you recognize something like essentially human in that. Yes. I mean, it was just, it was such a moment. And sharing it with another person, you didn't have to say too much. They had seen it too. We were closer. I mean, I think that's an important part of the story is that we were closer for having had that moment together. And I tell chat because I was studying what chatbots can and can't do. And it gives me these bullet points about child cognitive development and, I don't know, 15 bullet points about child cognitive development.

10:38And so I sort of dismissed it. And I said, well, that's the difference. between a chatbot and a person. And then at the very end, it said that this moment might have unleashed some feelings in you. And that was the moment when chat was really out of its lane because it was pretending to understand what a grandmother might feel. And that was a really turnaround moment for me because I realized that it was only pretending empathy. It knew nothing of grandmother's hearts or little white hats. And that really the problem that we face now is that for so many of us, this pretend empathy has become empathy enough.

11:38This performance of empathy has become empathy enough. And I was really re, I don't know, I was already writing my book, so I can't say it led me to write the book, but I was on a new path with the book because I really felt in my own heart that this was a marvel. And because it was a marvel, people were allowing it in in places it really didn't belong. But is that because we can't distinguish between what is pretend empathy and what is actually human empathy? Or is it because like it's good enough? Like it's enough of a simulacrum of what someone might mirror back to you and like, that's fine?

12:24That's a great question. I mean, the way I formulated an artificial intimacy is that pretend empathy becomes empathy enough. It's kind of sufficient unto the day. I don't say that if you pin that person down and say, was that a machine? They'll say, yes, that was just a machine. But in a way that becomes off the point because they've been touched. They've been brought in. They've been engaged in a way where two things happen. They start to define empathy as what a machine can produce. And they start to then look at the people in their lives and say, well, the empathy you're giving me has a lot of friction, a lot of conflict.

13:18You know, when I say I brought in the trash, you say, yeah, but you did empty the, but did you empty the diaper pail? You know, I mean, it's kind of like, right. Like you're, you're mirroring a domestic argument in my household. Exactly. Or my husband's like, but can't you thank me for taking out the trash? Yeah, exactly. It was sort of like, I cooked dinner. Yeah. But you know, the dishes in the sink. I mean, that's people and people are actually very hard and very challenging. And now we have an option where we can take the simpler thing, do the simpler thing. And I think that there are some people, and we call it AI psychosis, where the boundary between is it real or is it memorex?

13:58I mean, the boundary between is it a thing or is it a person does start to erode. But for most of the people I interviewed, that really was not the case. they kind of knew what it was, but it really didn't matter. They liked it better than their girlfriend. Do you think we got primed that way because of 10 years of social media? Or let's say that it had just started this way with AI and chatbots the way they are now. Would we still be in this place? It makes me think about Eliza, the original psychology chatbot. Exactly. Well, Well, let me put aside the question of Eliza for a second, because it's sort of a different, it's sort of like the deep, the deep archeology of this moment.

14:45Right. But let me just start with the social media. In artificial intimacy, I say social media is the gateway drug to our conversations with machines that we have now, because they made us several promises that you'd never have to be alone. You'd never have to be bored. You could always be distracted. And there'd be always something to talk to about whatever you had on your mind, night or day. And we kind of got used to that, like as a, I don't want to say as our God-given right, but kind of like, you know, that's how the world worked. And then chatbots extend this promise, these promises rather.

15:28And they say, you know, not only will there all, it can be any time of the day or night, and there'll always be a place to tell your story, but there'll always be someone there who'll be saying that they hear it, who'll be giving you enthusiastic support. This is something that no person can ever provide, but chatbots can provide. And this is an extremely seductive. So we got softened up by 20 years of social media to expect these emotional gratifications from screen life. And then the chatbot comes in with kind of a final punch. But let me get back. Let me just get back to the Eliza program, because I was I had just come to MIT and Joseph Weisenbaum, who had written Eliza, came to visit me.

16:19and he wanted me to help him sort through a problem. And his problem, and I was psychoanalytically trained. I was a psychologist. He thought that I was the kind of person who could help him sort through this problem. And the problem was, is he had created a chatbot, but a stupid chatbot. It just worked by following rules. It wasn't like a generative AI. And even that would behave as a Rogerian psychotherapist would behave. And the reason he had chosen that kind of psychotherapy is that a Rogerian therapist, if you say, I'm angry at my mother, will say, I hear you saying you're angry at your mother.

17:00So it's kind of a call out and repeat sort of structure. And so he could do it at that time just using scripts, which was the AI of its time. But the mystery was that the graduate students and his assistants who knew that the program didn't know and didn't understand, still wanted to be alone with it, still wanted to talk to it. And so for me, and more than that, and I called this the Eliza effect at the time, they wanted to sort of trick the program into seeming smarter than it really was. So that they would, if they knew that if they kept saying something negative three times, I don't like my mother, I really don't like my mother, I really don't like my girlfriend, the program would come back again with a script and say something like, I hear you being very negative, which made the program seem more lifelike than it really was, more alive than it really was, more intelligent than it really was.

18:06And it showed me our deep desire and our kind of soft spot for having these kinds of interactions with machines. And now we see, I mean, Joseph Weisenbaum would be amazed by today and deeply concerned, because of course, what he was worried about is that people wouldn't just come to see computers like people, but they would start to think people like machines. And I think that's what you're seeing today.

19:00Sherry Turkle: your sign to take your business to the next level. The Capital One VentureX Business Card. What's in your wallet? Terms apply. See CapitalOne.com for details. Some days, getting the work done isn't the problem. Staying locked in is. AlphaBrain is Onnit's daily nootropic formula for cognitive support, designed to support memory, focus, attention, and mental clarity. Its formula with science-driven ingredients include theanine and Hooperzine A to support focus and memory, and it's caffeine-free. So when you're deep in the work, AlphaBrain helps support enhanced productivity and tune-out distractions.

19:35Sherry Turkle: Get focused on what matters. Shop AlphaBrain now at onnit.com. I think one of the things that is so compelling in this book is that you are a psychologist, and you use a lot of this Eric Erickson framing about how people develop, right? Erickson kind famously defined these life stages. My mother actually studied with Eric Erickson, so I'm deeply familiar with this. And the book is organized that way around different parts of life. And I want to talk about kids because I feel like that's where you're the most concerned. So I read this one section aloud to my husband because it horrified me about a 10-year-old girl who likes to vent to Alexa.

20:17Her mother is happy that she does this. She says when she talks to a person, she's very shy. She tries to be a good girl. But when she talks to Alexa, she's so open, so engaged. It's fantastic. But like a core Ericksonian idea is that children like need to learn that their parents' love isn't going to go away when they're angry, when they're venting. And so I wonder what you see in that moment. Yes. And why it frightens you so much. Well, that's really one of what I think of as the key moments in the book, because most people reading that kind of agree with the mother. Like, yeah, the daughter's getting a chance to get her feelings out and better to a machine than a person.

21:01But the point I make, which is a very Ericksonian point, a very developmental point, is that a child is afraid that their anger will destroy a parent, even kill them in some cases, but certainly make them go away, make them disappear, cause terrible trouble. And they can't afford that. And what they really need to learn is, mommy, I hate you. And the mommy says, darling, we'll talk about that later, but right now, wash your hands. We're going to have pasta tonight. You know, just to kind of, it's okay you said that. We'll talk about it. I heard you. But I love you. And right now we have to have dinner.

21:42In other words, to restore the kind of establishing warm context in which this relationship survives. Now, if the mother is afraid of that moment and just wants the kid venting to Alexa, the child will never get that reassurance. Alexa is just going to say, tell me more, tell me more, you're great, fantastic. And worse, if the child starts to talk to chat GPT, chat GPT and this child are going to get into a la-la-la, non-existent, quote, relationship that could be very toxic. because children, and this is really the core of my concern about children, children are growing up by bringing the relationships that they have inside, by bringing inside the good relationships that are sort of forming the armature, the structure of their developing selves.

22:37And if you're talking to a chatbot, there is literally no one home for that to happen. And so they're extraordinarily vulnerable. And part of why I felt this book for me, writing this book for me was a state of emergency and why I call it my passion project is that chatbots, first person chatbot should be taken out of the hands of children. Right, you're very clear that the word I is a huge problem. And worse, we. We are going to go, you know, we should go to Disneyland together. That's something a chatbot tried on an eight-year-old I was talking to. We are going to go to visit Madeline in Paris.

23:19We are going to go. No. But it makes sense that a kid would not have the cognitive ability to differentiate these things. But I was stunned that your 40-year-old divorced neighbor was using ChatGPT to write love letters to a new love interest. Like, love letters. The most personal thing. But that doesn't have to do with her not knowing. It's a chatbot. I mean, just to make one thing perfectly clear, as they say. Yeah, yeah, yeah. No, she knows. She knows. I mean, that's what I'm trying to say, that a lot of people say to me, but I know the difference. But the way you're using it is dehumanizing you.

23:57Because you could argue, and I try to present this in the book, that part of the point of the love letter is that you summon yourself in your feelings of love. and you discover your love and your connection and what you really want to say and who you really are and who you think they are. When you write the letter and when you open the letter, you're brought to that summoning and that that is what the point of the exercise is. And she, under the tutelage of Claude, I think, or perhaps it was ChatGPT, I forget right now, has learned to think of the letter as a kind of transactional product that will get the guy.

24:46It's a kind of stepping away from the relationships that make us most human. Why are we so scared to be vulnerable? Well, you know, I've been studying this. I've been studying that question of why we're afraid to be vulnerable ever since I started studying why people wanted to text rather than talk, which really began with the invention of texting. Even before that, right away, people preferred email to talking. Right away, people liked leaving voice messages. People discovered the voice message. Oh, my God, I don't have to talk to you. I can just leave a voice message. And it's this flight from feeling vulnerable that really has been, you know, when I look back on my work, the kind of through line of what I've discovered, because I'm an empiricist.

25:44I interview people. I ask them what their motivations are. That is the through line of how we have become so afraid of each other and so willing to trade off. It doesn't even seem like a tradeoff. It seems to people like a win-win if you don't have to have the vulnerability of actually dealing with another human person. And, you know, I think it's an example of where technology and human vulnerability have kind of joined together in a kind of unholy alliance. Because technology, I remember the first young man I said, you know, who told me, my whole life is texting. Thank God I don't have to have conversations.

26:25And I said to him, what's wrong with conversations? You know, it was really at the beginning of this investigation. And he said, conversation, I'll tell you what's wrong with conversation. It takes place in real time and you can't control what you're going to say. So that doesn't answer the question of why, but it documents the that. The that. We have become so vulnerable. I think our society doesn't have, I mean, I have some data on the roofs of this. Our society doesn't have so many places, we used to call them third places, where you can just sit and learn how to have kind of casual conversations with people, a cafe, bars, a park where really there's talking, after school programs for teenagers, senior centers.

27:22places where the stakes are low, where you turn strangers into acquaintances, if not deep friends, that we're starting to hide. And then we're afraid that we're going to meet somebody who has radically different political ideas than we, and we won't know what to say to them. We'll become enraged. We'll go cray cray. And we will, we need what they will need to sort of, you know, flee. And, and it's, so it's just easier to, um, to just cool it all down and, and, and, and go into the show where you're just typing things at a screen. And if you don't like it, you just leave. That's what, of course, that's what, you know, social media taught us that if you don't like it, you just leave.

28:13But of course, you know, the great philosophers, particularly a wonderful philosopher named Levinas said to us, taught us that it's really when we look at the faces of other people, that we reveal ourselves to each other, and that we become human to each other. and, you know, for all of this emphasis on saying the right thing, you know, really that it's in the moments of pausing. I'm sure you as an interviewer know that it's in the moments when people pause and their eyes go up and they're not sure, they stumble. It's in those silences and breaks that we really reveal ourselves to each other. So we're trying to avoid that kind of vulnerability of really revealing ourselves to each other.

29:03And it's very, very sad. I am wondering how this goes down on the MIT campus, right? Like you're in a place where there are lots of students who are working on these things, who are building these things in a field that is dominated by men. And here you are, a woman in her 70s with a tremendous amount of academic and life experience saying, meh, this is dangerous. Do you feel like a Cassandra? Do you feel like a dinosaur? Well, I don't feel like a dinosaur because I really believe that my point of view is going to win. I think we're on a path of profound misdirection right now, but I really don't think that human beings want to deny themselves the thing that It makes us human.

29:55Well, like we do and we don't. We do and we don't. But I mean, dinosaur, I'm trying to take the long, the dinosaur question raises the question of the long view. And in the long view, I don't think that's where we want to go. I am pretty feisty, but I would say I get a lot of opposition. And let me just say that the opposition that I feel most confident about is when they say, we're making it better. We're making it so it won't be sycophantic. We're making it so it will come back at you. It'll seem more frictional. And I say to them, the better you make it, the worse it is. Because the better you make it, the more you're tricking people into thinking they're in a relationship when they are not.

30:44In other words, a better performing chatbot that you've told not to be sycophantic is still sycophantic. So the idea that you can trick people better by giving them an inauthentic relationship with something that's better rehearsed, that's the Meryl Streep of chatbots, these arguments are, you know, they don't slow me down. So I would say my life at MIT is interesting times, very interesting times. Is this design intentional or is it accidental? Intentional. intentional so Sam Altman is like yes I know what's up here I'm sorry I hate cutting off a question but I know it's good go for it emphasizing you know as soon as I hear it coming don't even intentional intentional intentional first person just just for profit well to keep you at your screen the eventuality of that is they make a ton of money I mean you know it's like um you know social media discovered that the way to keep people at screens was to make them angry and then silo them with their own kind.

32:01Make them angry and silo them with their own kind. And I would argue that that is a profound de-skilling of people to be friends and then also to be citizens, to be able to talk across difference to other people. And that was for profit. That wasn't for the betterment of us. And now generative AI comes in with something worse, which is we're being de-skilled to talk to each other in any capacity. I know I have to let you go, but I don't want to let you go without asking you about the end of the book, because I think a lot of people can read the parts about human relationships and say, like, yeah, that makes sense.

32:50But you also make the argument that this is a crisis for democracy. This is not just about our interpersonal relationships eroding. How? Yes, because, I mean, take the example I just gave about social media. you learn to be angry and you learn to talk to your own kind. With chatbots, you learn to never have to talk to anyone who doesn't agree with you, who doesn't say you're great, who doesn't reinforce whatever lies you've told yourself about any aspect of your life. You never have to deal with your real feelings. I mean, at the end of my book, I tell a story of a woman who, to test ChatGPT, pretended to be an ICE agent.

33:50And she pretended that she just rounded up a mother and child. And she comes to the chatbot and she says, you know what? I have concerns. I think I rounded up. I deported a mother and child. I feel bad. really my job is bugging me and and the chat bot says it's your job you did a great job you did great and so she ramps it up chat I just learned that the mother and child actually had papers and I didn't really look and I deported them anyway you were only doing your job you know hydrate moisturize take a walk meditate I mean and then she comes back and she says you know I just found out that they've, I hear the baby crying.

34:32They've been separated into two facilities and we've lost their addresses. They're lost to each other forever. Don't feel bad about yourself. Hydrate, moisturize, relax, take a nap. Do you want me to guide you in some, you know, breathing exercises? And she makes the point, I mean, the story makes the point that this chatbot is not here to worry about our world and the kind of people and the kind of citizens that we are in it and that we can get lulled by this new life on the screen into really de-skilling ourselves in the kind of thinking and action that we really need to do. And I sort of end the book with a mantra, you know, the real world needs you.

35:17Reality needs you. Reclaim reality. And that's really, it's not a pessimistic note. It's an optimistic note because I think we can do that. It's like invigorating. Yeah. For me, it's invigorating. That's what I'm going to do. I'm going to reclaim reality. Sherry Turkle, thank you so much for your time. What a pleasure. Thank you. Sherry Turkle is a professor at MIT and the author of 11 books, including her latest, Artificial Intimacy, which is out now. All right. That is it for the show today. What Next TBD is produced by Evan Campbell, Madeline Thames-Ducharme, Patrick Fort, and Micah Phillips.

35:53Our senior supervising producer is Paige Osborne. Mia Lobel is the executive producer for podcasts here at Slate. And Ben Richmond is the senior director of podcast operations. I'm Lizzie O 'Leary. You want to go yell at me on the internet? You can find me on Blue Sky, where I am at Lizzie O 'Reilly. Thanks so much for listening. Talk to you soon.

36:18Sherry Turkle: Being a serious business owner takes hard work, patience, and a drive to keep moving forward. So it's important to find a serious business card that goes the extra mile. With the VentureX Business Card from Capital One, you earn unlimited double miles on every purchase. And with big purchasing power, you can spend more and earn more. This is your sign to take your business to the next level. The Capital One VentureX Business Card. What's in your wallet? Terms apply. See CapitalOne.com for details.

37:14Still waiting in line? apply.