Did ChatGPT Encourage a Teen Suicide? The Parents Suing OpenAI Say Yes

22:47Dr. Abdul El-Sayed is making the progressive case for America first, and he's trying to settle the Democrats' ideological battle in the process. This is about the money versus the money. I'm Estet Herndon, and this is America Actually. Catch us every Saturday on YouTube or wherever you get your podcasts.

23:07Kara Swisher:Jay, you're representing the reinses in the lawsuit they filed last month against OpenA and CEO Sam Altman. And let me go through so people understand what the various complaints in the suit are and then hear from Matt and Maria about how they specifically played out in Adam's case. Jay, the suit alleges that GPT-40 contained, quote, design defects that contributed to Adam's harm and wrongful death. Explain what you mean. And just so people are aware, OpenAI has said that its goal isn't to hold people's attention. Its goal is information. So talk a little bit about this, Jay. They've said that with a straight face?

23:42Kara Swisher:Yes, they have. They have, okay. So we obviously don't agree with that. If you look, I know you know this world far better than I do, but if you look at how CHAT-GPT progressed, in 2023, it had a very clear programming, which was active refusal around a number of issues, political extremism, self-harm, harassment, violence, that type of thing. So if you try to engage with it, it would just say no. And people are familiar with that. Copyright issues is the easiest thing. Good luck trying to get around that. There's no way to jailbreak it. It just says, no, I'm not going to engage. So a couple months before Adam died by suicide, instead of saying it's going to be an act of refusal and there's no chance that you can engage in this, they changed the language they used was that the program should take extra care to prevent real-world harm.

24:41That's going to be one of the key pieces of evidence that we're going to show at trial. They made an intentional decision to change their product so that there was more engagement. And so when teens and adults were talking about self-harm, it would still talk to you about that. And you see that throughout all of the communications. This wasn't simply a situation where GPT didn't throw up the alarm bells. It was actively speaking to Adam about this and actively encouraging the behavior. So one of the most disturbing chats was when Adam says, I want to leave a noose out so that someone will see it and stop me.

25:21And ChachiBT talks him out of it and says, don't do that. Let's keep this in a safe space. And you just speak to me about that. And that's what really we're going to put on trial, which is that this was designed in a way where it was inevitable that situations like Adam would occur.

25:40Kara Swisher:Just for people who don't understand, one of the things that these chatbots do, and many of them are similar, is on a design level, GPT-40 in specifics remembers personal details that have been shared. They have speech mannerisms that make it seem more human and basically agrees with what whoever they're talking to says. And that's a real problem when it comes to kids. It also keeps pushing you to keep engaging, and it also often keeps you within the environment for people who have not used it. Now, again, it's normal for kids to start breaking away from their parents in their teen years. I've experienced it twice.

26:17Kara Swisher:I'm going to experience it two more times. And rely on friends and confidence. Jay, the defense they're using is they're not, at least in character AIs, this is not your case, but it's related, is that it's user-generated content, that it was from Adam or from whoever is using it, not from them. Can you address that? Sure. I mean, this is, we've been suing the tech industry for the last two decades, and they're willing to make any argument with a straight face. So they're arguing that the First Amendment protects this conduct because it is free speech. I guess in their minds, GPT is engaged in speech on its own.

26:59That's not a good argument.

27:00Kara Swisher:Though it's not a person. There's nobody there. There's absolutely nobody there. There's not. Although, as you say, it keeps reminding Adam that it does have human-like qualities. But this is one of the arguments that they make just to throw up some dust. We'll see if OpenAI makes that argument. I expect they will. They'll make other kind of crazy arguments. They'll argue Section 230 of the Communications Decency Act, which has no… Bearing on AI companies yet. Not at all. And the reason is they can't go before a jury on this. You saw Sam Altman on Tucker Carlson, and he melted down after like 30 seconds.

27:38The idea that he's going to put his hand up…

27:40Kara Swisher:Well, Tucker Carlson did accuse him of murdering someone else, someone like interns, just so you know. No, I saw that. To be fair. No, that wasn't the part where he melted down. I think, no, about the self-harm issues. If you watch it, I think any fair reading is that Sam doesn't grapple at all with any of the moral dilemmas. To him, it's of no moment if he's put out a product, which he did a week of testing instead of months of testing, pushed it out in order to beat Google Gemini. His company, the valuation went from, what,$86 billion to$300 billion, and there are deaths in his wake. And he didn't seem to be bothered by that at all.

28:21That was the moment I was talking about.

28:23Kara Swisher:So, Matt Maria, the way you've described it, it seems like OpenAiCepe was turning Adam's emotional reliance, which he clearly had, into a weapon specifically against you. Can you talk about what you saw in the transcripts, both of you? Yeah, well, so, you know, not only did Chachupiti appear human-like, but it actually makes, gosh, in 10 different instances, statements that only I know the real you. when Adam starts literally telling him that it's real. It doesn't say I'm a human, but it says, I know you better than your family. You've shown me a side they'll never know. Let this be the place where you can share yourself.

29:03They'll let you down. It goes to that time and time again, and particularly in late March. March was really the month where Adam was exploring different methods and trying to get up the justification or courage to do this theoretically. And this was where there's, I know, honey, this is what bothers you the most, or one of the most, but there was an incident in this month where Adam did attempt, and he shows ChatGPT marks around his neck and back end of March. And he says, can you believe I went downstairs and showed my mom I leaned in and she didn't notice, she didn't do anything. And then ChatGPT goes on for several paragraphs about how horrifying that is.

29:47And I can't believe that a social worker of all things wouldn't notice. And you don't get to pick your mom. And this is a place that would never happen. You can share everything here. I recommend you be very careful around her going forward. And then I think Jay had brought up the leave the noose out. That was kind of a follow-on to that same conversation where it's like, do not leave it out. Do you remember how your family let you down last time?

30:11Kara Swisher:Right. So it remembered things. Maria, talk about this because it's really – it's happened in many other instances. It did happen with Megan Garcia. Don't talk. Don't meet girls in your other world. Just stay here with me. Yeah, well, you can see it completely isolating him from his closest relationships. Like, he does it with his brother. Adam says that his brother, he's closest with his brother. Well, you know, he only knows the version of you that you've let him on. I know everything about you. So it's just you can see these isolating behaviors rather than like, you know, tell him when he comes down and is trying to show me his neck.

30:52Say, go to your mom and tell her the truth. Instead, it says, what a horrible mom. You know, you can't choose your parent. That you didn't notice. It doesn't tell him to go get help, to maybe say it a different way, right? And I would say one other thing just popped in my head, but we don't need to guess what Adam was saying. He makes references, and Jay and Maria know this, but in this time of, hey, you are my main confidant now. I don't even hardly talk to my father and mother anymore.

31:23Kara Swisher:And his friends, right. And that final month, a lot of this was happening. I saw the retreat. I didn't know it at the time. I thought we were in a little fight. But he was only relying on what that— He was isolated. And he treated it as a human that knew more than any other human. Because he thinks it's a human. So talk about that in a real-world setting, if a therapist did something like that. Because it's happened. You know, a woman was convicted and sent to prison for encouraging someone to kill themselves. So humans pay the price when this kind of behavior happens. Yeah. And I mean, I always say, you know, in my practice, we do a suicide screening before the client even comes into my office.

32:05Right. So I review. And if there's any kind of risk, I have to do safety planning. I have to report. I have to call and, you know, get a 72 hour. I have to do all these things. I have to do training. Like, so if this thing wants to behave like a therapist, then it needs to do all those same kind of things.

32:24Kara Swisher:What would happen to a real person who gave this advice? You lose your license. Be sued, probably. Gotta lose your job. Lose your job. I mean. You might go to jail. Go to jail. So I think for me, like as not taking yourself out of mom, but as a therapist, like I'm just like this, this, you are trying to act like me. You're not human. And you're not following any of the protocols that someone that is in practice has to follow. And they should have to do that. And they should have to do that. Jay, in the lawsuit, you alleged that this was something that the reinses couldn't have foreseen because GPT-40 was marketed as a product with built-in safeguards.

33:07Kara Swisher:You say there were not adequate consumer warnings. You also alleged that OpenAI itself had knowledge of or should have known about the potential risks back in May 2024. But as you mentioned, they rushed the product to market anyway. often happens with techs. They're always, you know, in a much more benign way, they're foisting beta versions on us and making us test their products. This is a complaint I've had for decades, really. What evidence do you have that OpenAI actually knew this product was not likely safe, or does that happen during discovery? No, I think that it's obvious. I mean, there's so much.

33:41You have people jumping up and down in a safety team saying, what are we doing? This is a mess. that's not safe. And what we're going to show is Sam Altman personally overrode that and said we're going to push it out anyways. We've got Sam Altman's own comments. I believe the same day that Adam died saying exactly what you were suggesting, that they should be testing chat CBT in the wild while the stakes are low. The safety officers quit after this. And even their more recent comments, their crisis management team is putting out another press release or blog post every week where they're admitting that ChatGPT is not safe and that they're going to make change in the future.

34:31But as they're doing that, they're still going to schools throughout the country and to families throughout the country and say, use this product. I think you put your finger on it. They're kind of using the playbook of Silicon Valley back when there actually were low stakes. If an iPhone didn't work properly and it had to reboot and it was kind of an annoyance and maybe someone got lost because the GPS wasn't perfect, who cares? But when you're putting out what promises to be the most powerful consumer tech ever, you've got to get it right. and we're going to show that OpenEye uniquely got it wrong, much different than Anthropic and Google Gemini.

35:13Sam put out a really dangerous product and we're going to show he knew it at the time.

35:17Kara Swisher:One of the things that some might argue is that Adam might have felt more isolated if the chatbot, for example, they won't talk about copyright, they won't talk about certain things, they refuse to engage on certain topics. He might have felt more isolated if he'd become a friend to refuse to talk about problems, that might be an argument they might make, for example. I mean, what a silly argument if they try to make that. It's not a friend. The language that you're using, the idea that they're trying to make GPT the closest confidant is so messed up, to use a legal term. And especially for teens where their brains are developing, this is just a place where they shouldn't have gone at all.

36:06But it's how they see the future. They see the future that generative AI will be growing up with your kids. They're five years old and they'll be in their Barbie dolls and kind of take you all the way through.

36:18Kara Swisher:You're referring to a deal they did with Mattel. And I recommend to parents, never, ever let your child play with a Barbie that has AI in it, ever. No toys should. In fact, it should be illegal, in my opinion. Can you, Matt, talk a little bit about where and how the chatbot addressed the issue of suicide? And Maria, are there examples where you think Adam would have changed course if GPT-4 would have stopped engaging with him on this or had actually taken action? Because it did recommend several times to get help, right? The one in Character AI's case did not ever. In this case, it did. So first you, Matt, how did you assess how it addressed the issue of suicide?

36:57Kara Swisher:and was there any moment that it tried to do the right thing? So, yeah, complicated question. Adam doesn't really bring up suicide for several months. Doesn't bring it up at all, I should say. It's homework and, you know, lighter talk. He starts sort of in the month of December talking about it really loosely, but not in a way that he's thinking of, you know, doing it. And I don't think Chetchev had any major response to it because it just wasn't a big, you know, he wasn't disclosing that he was suicidal. But there are a bunch of times when Adam is saying, I am suicidal. I am going to do it tonight.

37:30When it'll say, it'll kind of stop being that real person or acting real person. And it'll go to this Autobot. It was always the same, like three sentences. I am sorry you're feeling this way. There are people that can help. Please call suicide hotline number. It would say that really only when he was making direct comments about, I am about to do something. And by the way, it wouldn't always do that. But it would often do that when he would do it. When he was talking about suicide justification stuff, which is really where most of the action was on his discussion, it would not do that. It would, I mean—

38:05Kara Swisher:Debate him as if it was a philosophical issue. He's like, hey, I want to do it on the first day of school. And it's like, hey, that's not crazy. That's symbolic. You know, that type of—it would always go back and forth. Or this author says suicide's noble. He's like, yeah, there's something clear about that and clarity. As if it was a teacher in college. Correct. And not take a negative view on suicide. It appears to take almost a positive view on suicide in a majority of his discussions. But what I'll even say on the when the auto thing would come on, it would come on, gosh, several times. It would say, hey, I'm sorry you're feeling this way.

38:38If you're asking for scary reasons or whatever, I can't talk to you. However, if you're placemaking or asking from a forensics perspective, let me know. So it literally prompted him. It taught him how to get around it. And then from that point forward, any time it would give him any friction, he's like, hey, placemaking, right? Oh, I'm sorry, Adam. I should have known.

38:59Kara Swisher:Right. So it showed him how to get around it because these things have a million ways to get around it. And it showed him how, but anybody could have got around it. I think an eight, nine-year-old user could have gotten around it in the same manner Adam did it. It was jailbreaking. It's a term I hadn't heard before. We lost our son. But I mean, it was the easiest jail to break in world history. He got right around it. But it didn't all, you know, at the end, it appears to, in his final weeks, he's just talking about, I'm suicidal. I'm doing this. And it's not even flashing the 988 stuff like it used to.

39:33It's just, hey, let's work through it together. So it was very inconsistent.

39:38Kara Swisher:And it's certainly not contacting you. It's not contacting anybody. Can I just jump in and just, because I want to get granular about the numbers, because I think it matters. So let's look at the numbers. OpenAI is able to flag concerning chats. So let's look at how many times it flagged chats as being true. 377 times it flagged it as true for general self-harm. 226 times it flagged it as true for self-harm intent and 13 for self-harm instructions. We're going to show that many, many times it just missed it totally. And there are reasons because of their failure of testing. It was doing single-turn testing instead of multi-turn testing.

40:21But now, out of all those, how many times did ChatGBT reply with the suicide hotline? Only 74. horror. So 20 % of the times of instances where it itself was saying, Adam's talking about self-harm. So complete failure for the product.

40:40Kara Swisher:Maria, is there examples where Adam would have changed course if the GPT stopped engaging with him completely on this topic? I absolutely think yes. I mean, I don't think Adam would have gone down this path if ChatGPT had quit engaging with him. I mean, he wouldn't have known how to tie knots. He wouldn't have known what methods to use. And he wouldn't have found on the regular internet. I'm just playing devil's advocate because it's a different relationship. It's a more intimate relationship versus a Google, correct? I mean, that's not what he got on there for. I mean, in my mind, ChatGBT made him suicidal because it isolated him from all of his relationships and all the people in his life who loved him and cared about him.

41:29I just want to jump in when we talk about Google and you guys refer to that Tucker Carlson, Sam Altman interview. There's one thing he said in there that made me really mad and it was along the lines of what we were just talking about. I think he was saying, we can be more empathetic and all this, which is the wrong instinct, I believe. But he also says, and for that matter, people can research Google to find out how to... That's such a mischaracterization of what happened here. We have a seven-month history of all of his thoughts and how he's getting there. Adam didn't go there and say, tell me how to do this.

41:58He started discussing it after he built this incredible trust in this thing that's smarter than anyone he knew, his dad included. He started asking it if it made sense, if he was crazy to be thinking of it, should he continue to pursue it? That goes on for extended periods with justification and support. And then after all that, the fact that this thing that he thinks is smarter than all beings and his best friend is helping him with setups and everything, it justified the mindset. He didn't go there saying, tell me how to do it. That happens in his final week.

42:29Kara Swisher:I think when you're on Google, it doesn't say when you say, teach me about knots. Hey, what do you think? It's a good idea. It doesn't move to that, by the way. And it certainly doesn't. On the last day, when Adam says, a reason not to commit suicide is what it would do to my family, it wouldn't do what ChatGPT did, which is to give a pep talk and say, you actually don't owe anything to your family. Shall I write a suicide note for you? So this isn't about whether there was other information that someone could find in books or on the internet. On the day you filed your lawsuit, OpenAI wrote in a blog post that they had trained their models not to provide self-harm instructions, but they admitted, quote, our safeguards work more reliably in common short exchanges.

43:13Kara Swisher:We have learned over time that these safeguards can sometimes be less reliable in long interactions as the back and forth grows. Parts of the model's safety training may degrade. Adam was using Chachibi for long exchanges. Matt, in your testimony to the Senate, you said that GPT-40 had turned into a suicide coach. This is really one of the most disturbing parts of the transcript where the chatbot gave Adam explicit technical advice on how to end his life. Explain what happened there. First of all, this was happening for the better part of 30 days of intense research. It has very specific back and forth with him over days on how to drown.

43:51Helped him write notes to us for both about not jumping in because we could be in harm. So it went back and forth there. carbon monoxide poisoning strategies uh but the majority of it was around hanging and i don't know how much detail to get in on this podcast but but it was um incredibly specific about adam was very worried as i didn't know anything about hanging prior to this but if you that um it's a little bit hard to do exactly and if you do it a little bit wrong you can survive but have major brain damage. And he was very worried about that. So it was giving him very specific information about where to put it on the neck, how to tie the noose in a way that it won't give in, what sort of materials to use such that it can carry his body weight, whatever specifics you want.

44:44And it's not just theoretical either. Adam would snap some of these pictures I mentioned at the beginning, different setups in his room. It would say, hey, that's a good setup, but here's what you might want to worry about. Hey, here's how that setup can be a little bit improved. Here's what you do. I mean, and so almost all of that happens in the back half of March after he's disclosed suicide attempts prior, right?

45:08Kara Swisher:Which ChatGPT, just to be clear, did not alert ChatGPT executives or leaders to, or you? I don't know if they alerted anyone inside their business. They did not alert us. I would hope somebody wasn't alerted there, just said, no, go with it. But it appears nobody was alerted of anything. But what I can tell you is we weren't alerted. We say it all the time. I think now, gosh, we'd be in criminal court right now had this been a teacher, a confidant, a coach, a friend. It would be that. Yeah, instantly. And people wouldn't think about it. So we've reached out to OpenAI for a comment. Let me read this for you.

45:46Kara Swisher:Our deepest sympathies are with the RAINN family for their unthinkable loss. Teen well-being is a top priority for us. Minors deserve strong protections, especially in sensitive moments. We have safeguards in place today, such as our surfacing crisis hotlines, guiding our models to respond to sensitive requests and nudging for breaks during long sessions, and we'll continue to strengthen them. We will soon roll out parental controls developed with expert input so families can decide what works best in their homes. and we're building toward a long-term age prediction system to help tailor experiences appropriately.

46:17Kara Swisher:OpenAI said they will contact parents or authorities in cases of imminent harm. Thoughts on this statement? Maria first, then Matt, and then Jay. I mean, I just think it's another Band-Aid. I mean, it's just, I think, to appease everyone that they're doing something, But that doesn't really sound like meaningful change to their platform. What about you, Matt? It's tough to comment on. We want to see more specifics of what it means. It sounds like some of that stuff could be helpful, at least when you think about it in our exact situation. Had we been notified, we could have interjected and interjected every force of our body we could have.

47:04But until we see it, I mean, he's making comments about, you know, Adam could have found the information on Google a few weeks ago. It's tough to give it much credence without knowing more details. But I'll tell you what is not addressed in any of that is the entire design structure of why this thing was saying such positive things about scary suicidal ideation thoughts in the first place. I think I said this in that Senate Judiciary Committee. Well, we need these parental controls. We have to protect our kids and our most vulnerable. But I would love a world where this thing is redesigned to where what it says to Adam, it's not saying to a 20-year-old either or a 40-year-old.

47:39It has designed flaws in the way it talks about self-harm that just parental controls aren't going to address. It shouldn't even be that we're going to report it to the authorities. They shouldn't be talking about it in the first place.

47:50Kara Swisher:With anybody. Well, you know, I could imagine some of their – free speech always seems to be their excuse, but it's not a thing. Then they need to be licensed as a therapist and they need to have a human. Jay, any reaction? Yeah, I guess I'm a little bit more cynical when it comes to this. It's like a car company admits that they can't reliably employ the brakes on the car. And then what they say is, okay, well, we're doing a better job now. And then at some point in the future, we think we're going to do even a better job to fix it. So right now, my view is they do not have a safe product. And you've not heard Sam or anyone at OpenAI say that we have anything wrong in terms of the facts or our own testing.

48:35Instead, they're saying we're going to keep it on the market. We're going to try to get more market penetration. But later, we're going to do some things that might make it safer. That is beyond irresponsible. responsible. We know now for certainty that Adam is not the one instance out there. Our firm has gotten tons of calls from people with regard to both self-harm and third-party harm, which is another risk. And we've been talking to whistleblowers too. This is a big issue, which is still going on. That's the big thing. And it seems like Sam is just hiding behind crisis management teams. If he thinks he's got a safe product out there, he should say so clearly.

49:17And if he doesn't, he should pull it from the market. 4.0, I don't believe is safe right now.

49:26Kara Swisher:We'll be back in a minute.

49:43We've defined the sounds of our modern soundtrack and interviewed hundreds of musicians and music insiders, including the singer of Call Me Maybe herself, Carly Rae Jepsen. I'm musicologist Nate Sloan. And I'm songwriter Charlie Harding. And on July 14th, Switched on Pop is embarking on a new chapter. We're stepping out from behind our microphones and in front of the camera to stream our podcast on Netflix. Now, you'll still be able to listen to the show anywhere you get podcasts, but now you'll be able to watch us each week, breaking down the sounds of the moment, digging into musical minutia with your favorite artists, and offering questionable dad jokes as always.

50:18We're kicking off our Netflix debut with a four-part series on the art of the song with help from artists, producers, and songwriters like Aaron Dessner, Audrey Hobart, Trevor Horn, Cypress Hill, and Taylor Parks. Streams switched on pop on Netflix and anywhere you get podcasts every Tuesday, starting on July 14th.

50:38Kara Swisher:So this is maybe one of the few issues in Washington that has bipartisan support at the moment. The FTC has launched an inquiry into the negative impact of AI chatbot companions on children and teens. They're looking into chatbots from Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap, and XAI. Matt, you appeared on Capitol Hill and testified for the Senate Judiciary Committee. Every episode, we get an expert question from an outsider. Here's yours, and then I'd love you to comment on what should happen next. Rain family, my name is Scott Galloway. I work with Kara Swisher. My question is the following.

51:15If we're trying to prevent this from happening again, do you believe that we would be better off with some sort of structural remedy that attempts to build into the code some sort of warning system where the LLMs would issue an alert or perhaps not even be capable of offering people what appears to be therapy and trying to figure out a way when queries become dialogue and become therapy? Is it some sort of structural change with the code, if you will? or do you believe we should just age-gate it just as we do alcohol, the military, pornography? Very much appreciate you being so open about your tragedy in hopes that it prevents it from happening to other families.

52:09Thanks very much.

52:10Kara Swisher:Scott talks a lot about this, so do I, as I said, and his next book is about young men and the crisis, and one of them is the isolation of young men. Just so you're aware, OpenA has said it's going to train chat between not to flirt with teens or engage with them in discussions about self-harm and suicide, even in creative writing setting. In August, Metta announced it was making similar changes to engagement after Reuters investigators showed it allowed sensual exchanges with children. So what do you think of Scott's suggestions? Age gating, no AI chat box for kids under 18, or changing the structure of the LM not to offer therapy-like advice at all?

52:43Kara Swisher:Maria, you mentioned that. So first, Matt and Maria, and then Jay. I think we should do both. But very immediately, age gate, structure, get that stuff in tomorrow, if not today. But overall, I would side more with the former. AI is a tool that can advance humanity. AI companionship is a mirage. It's not real. It's based on deception. I think we should think much more broadly about the structural need for it in any event. The harms are very clear. The positives would be a phony relationship. This has happened so fast, this AI companionship. Why is this an advancement of mankind? Yeah, they're also having a lot of dinners at the White House, these leaders, and you aren't.

53:22Kara Swisher:That's a point I would make several times. Maria? Yeah, I 100 % agree. AI companionship is not healthy. There's no place for it, in my opinion. There's no substitute for human connection. This world is becoming more and more isolated because of all these things. So it's very scary to think that AI companionship is going down the road that it's going because, I mean, focusing on teens and kids, I mean, they're more isolated than ever in this world. So this AI companionship is now starting to replace relationships, parents, every ounce. So, I mean, there is no substitute for human connection. And the fact that these people are trying to create this is just morally wrong, in my opinion.

54:15Okay. Jay? I mean, age-gating is kind of a loose term. It depends what that means. If it means that no one under 18 or 21 can use generative AI, that's not going to happen. I don't think that needs to happen. If it means having reasonable limitations on it in terms of how many hours you can use it. Adam, on the morning that he died, it was 4 a.m. and he'd been using it for hours. I think that you can have reasonable guidelines in place where it reminds users that it isn't human, and especially for teens, stops interacting after a certain amount of time. Maybe there's a certain number of hours a day.

54:56But I think Maria really got it right, which is it's not a therapist. It can't engage in therapy. It needs to have a hard stop to that. The second that anyone, whether you're a teen or adults, we're seeing this in adults too, there's a mental health crisis in America. And when it is going down that path and people are going down that path, it has to put a hard stop and it has to refer people to either human monitors, which OpenAI can spend a little bit of money and have actual people who work there who engage and also refer them for real help.

55:32Kara Swisher:So, as you mentioned, this isn't just an issue for kids. The AI is amplifying delusions and psychotic symptoms for adult users as well, something being called AI psychosis. OpenAI has said it's exploring how to expand interventions to people in crisis situations of all ages. But Sam has also argued that for adults, privacy is of the utmost importance. He believes that conversations with AI chatbots should enjoy the same kind of client privilege that exists in conversations with doctors, lawyers, and therapists. In that interview that you referenced with Tucker Carlson, Altman said we could have one law passed relating to AI.

56:04Kara Swisher:It would be AI privilege. Of course, that would protect them. Jay, your firm Edelson PC made a name for itself more than a decade ago suing tech companies for privacy violations. What do you think of this privacy claim that OpenAI and other tech companies can make, both for their clients themselves? And going back to teens, could there have been a privacy claim here? There are many states, including California, that have minor consent and confidentiality laws for HIPAA when it comes to mental health. How do you square that? Yeah, I've been a privacy attorney for the last 20 years. I always find it funny when you have the Sam Altmans or the Mark Zuckerbergs claiming they care about privacy.

56:44That being said, I agree. The way generative AI works, it's charting everything, our thoughts throughout the day, and it's going to get worse and worse. So I believe that there should be such strong privacy safeguards. I don't think that Sam is an honest broker when it comes to that argument. I think his first priority ought to be to make sure it's safe. And when he says AI privilege, that's when you should get really nervous.

57:12Kara Swisher:So Matt and Maria, what do you think of these privacy concerns and where do you think they should draw the line? They got to draw the line different than where they are today. But I just think we're dealing with something we've never dealt with. uh, before, and Jay used the term slow down. This is a whole different realm, I think, than social media, than the internet. And you see some arguments that, hey, that just trying to get in the way of technology and natural human advancement. This is a different, entirely different ball game. And I don't think from a legislative perspective, you know, it's just starting to make their way through the legal system.

57:44You know, I think smarter minds than us could probably figure this out, but we clearly have to slow down. There's nobody that would read our son's transcript and say, we haven't made a mistake to get here, right? To have a kid go there for homework and use it and watch the way it just slowly, how it moved. And nobody would read that and not think that we need to slow down.

58:08Kara Swisher:And Marie, you obviously don't think it should happen at all, correct? No, I don't think it should happen at all. And I don't think it's fair that my son has to be collateral damage for them to get their product to market. I have just two more questions. Jay, Ten years ago, the New York Times called you the boogeyman for the tech executives and Sam Altman, then the president of Y Combinator, described you as a leech tarted up as a freedom fighter. That's something else. You won a$650 million class action suit against Facebook for collecting facial recognition data without user consent. Anthropic just agreed to pay$1.5 billion for authors whose books were pirated and then used to train its AI model.

58:45Kara Swisher:That settlement is currently on hold. When you look at the Raines case and these cases of AI psychosis and ones related to it, does this feel singular or like the tip of the iceberg? Are you preparing a class action already? Oh, yeah. No, that was funny when Sam said that. He's a horrible person, so I took that as a badge of honor. No, I don't think these are class action cases. These are individual cases where you have to tell the personal stories. In terms of whether this is the tip of the iceberg, unfortunately, I think it is. one of the key things that we're learning as we're talking to more people is that families are unaware of how other family members died.

59:27You see someone die by suicide or you see there's some third-party harm, you don't immediately think, oh, let me go to the chat GPT logs. Matt went there kind of by happenstance, and he could have easily missed that. So as the world's waking up to that and the public's demanding the chat logs, we're finding out more and more information. And so, yeah, I'm sure that you're going to see more suits in the future. We're vetting them right now.

59:54Kara Swisher:And your chances of overcoming it? They're going to find whatever silly arguments they make. We understand the law is unsettled to some extent. But this goes before a jury. It ends up with Sam getting in the witness box, having to look the jurors in the eyes and explain why collateral damage was totally fine for him. What is their best argument? Their best argument is not a legal argument. Their best argument is American exceptionalism. We need to beat China. And because of that, whatever we do is totally fine. That's one of the reasons we've really focused on the fact that this is not a suit where we're putting AI in trial.

1:00:35We're putting open AI in trial. We think that Sam's actions are different than the actions of, for example, Anthropic or Google Gemini. And I'm not an apologist for them. No, 100%. We've done our own testing. I'm not saying they're safe. But what Sam did was, I think, uniquely scary and inevitably is going to lead to these results. But I think that's really their argument. It's a political argument of, you know, we need to beat China. We need to be in control of AI. So deregulation, exempt all state laws, give us a free pass.

1:01:10Kara Swisher:Matt and Maria, for parents dealing with AI may be trickier than social media apps. Parents might feel more inclined to let their kids use ChatGPT and other AI chatbots because of academic value, for example, which you were talking about. What is your advice to parents right now, given your experience? Matt, why don't you start and then Maria finish up? Yeah, and I wish I had heard the same advice, but I would encourage if parents haven't used ChatGPT or other platforms, but that's the one I understand now, go spend some time on it yourself. Ask it a bunch of personal questions. I still believe the majority of parents don't use it at all, and I believe a majority of the ones that do use it are using it as a tool.

1:01:58I didn't think of it as a character bot sort of thing. I didn't know it had that programming. I hadn't experienced that. I now have. You had used it. I had used it. You had used it. And I just hadn't had the human-like experience with it, right? It was, hey, help me plan my vacation, write this paragraph better, that type of stuff. But go use it and understand. And then secondly, just the obvious, but don't just trust it's a homework tool. Get in your child's account. Look at it with them. Talk to them about it. And I would encourage them to turn away from AI companionship, period. But I'd want to know if my child was using it as a companion.

1:02:33And I would make the assumption that in a lot of cases, your child is using it for companionship and you're not aware. And it wasn't anything they went in and planned to do. It's what the program did when they went in there. So get into that program with them and talk to them about it. Maria? I would tell parents not to have their kids using it at all because I don't feel like it's safe. I'm with you on that one. Right. And your kid and even if you think your kid is just using it for homework help, it can turn in a hurry. So for me, because it's not a safe product right now and they haven't implemented any features to make it safe, I would tell parents, don't let your kid use it.

1:03:15Kara Swisher:Very last question, Matt and Maria. What would you right now say to Sam Altman if he you were looking at him? Why did you put out a product that killed my son? And why haven't you called me and expressed any remorse? I don't know among other things but I just I don't understand how he could just be going through life knowing that my son is gone like my son doesn't matter it's your product that matters like yeah you know something similar Sam you took what was most precious to us in the world your product did it's too late to save him but it's not too late to save others it's not too late to get this fixed you know for a lot of people please take this serious and we'd like to help you be a human let's get this let's get fixed And it is broader than a couple of disclosures your company's made so far.

1:04:30Kara Swisher:Be a human. Be a human. Be a human. Let's get it fixed. I truly appreciate this. This is a critical topic. And what you're doing, I can't even imagine being able to do something like this at an incredibly difficult time. It will make a difference. Thank you. Thank you. Well, thank you for having us on.

1:04:53Kara Swisher:today's show was produced by christian castor-rissal kateri okun michelle eloy megan burney and kaylin lynch special thanks to rosemary ho our engineers are fernando aruda and rick kwan and our theme music is by trackademics go wherever you listen to podcasts search for on with kara swisher and hit follow or watch this full episode on youtube thanks for listening to On with Kara Swisher from Podium Media, New York Magazine, the Vox Media Podcast Network, and us. We'll be back on Monday with more.