The Challenges of Updating AI Safety

0:00chat gpt has announced that they're going to start routing sensitive conversations to gpt5 even if that wasn't the model you originally selected. In addition to that, they're going to start rolling out parental controls. There's a whole bunch of things that essentially OpenAI has announced in response to a lawsuit and a bunch of other issues and questions. We're going to be diving into all of that today. It's a very, I guess, tricky subject. Someone recently that committed suicide, a teen, and they looked at his chat GPT logs and he'd been talking with chat GPT beforehand and had asked about suicide methods and other things.

0:31So it's a bit of a sensitive topic. There's a lot of controversy in this. And so anyways, I'll try to tread lightly, but I have a different opinion on this than I think a lot of people. So in any case, we're going to get into the podcast today. Before we do, I wanted to mention that my own startup has just launched our AI app builder. So it is finally live on our website, AIbox.ai, where essentially if you have any sort of AI tool or workflow you'd like to generate, you can go in, type in the prompt to describe what kind of tool, you know, a newsletter generator that takes in articles that you paste in and it generates images for your newsletter and maybe an audio file with 11 labs, and it will be able to stitch together a workflow and create the tool.

1:11So I would love for you to try it out. Let me know what your thoughts are. It's AIbox.ai. I will leave a link in the description. I am super excited. I've been working on this for a couple of years. So big news for me. We're working on bugs. We're working on fixing stuff. We're working on adding new features. So if you have anything any requests feel free to send them over all right let's get into the episode today so uh essentially all of this comes back to a lawsuit that has been launched against open ai um from a the the parents of a family who had their teenager um tragically committed suicide and they looked at his chat gpt logs and he had been using chat gpt in response to this i think open ai is trying to tread lightly they're trying not to you know assume responsibility for this but also trying to help maybe make tools in the future to prevent tragedies like this, or at least help people identify when there's someone in mental distress.

2:03This is kind of what basically all of the tools that they're rolling out are around. So there's kind of these parental controls, the new guardrails are all going to be coming in in the next couple months, they posted a whole blog post about this, where they essentially acknowledge some of the shortcomings in their AI safety systems, they said, you know, they don't always maintain guardrails during extended conversations. While OpenAI is, you know, not saying that they're claiming responsibility for basically any of the tragedies that have happened recently, they are saying that they are going to make some changes to basically how this works.

2:35Some people attribute these to quote unquote, fundamental design elements. One of the big things that it gets criticized frequently for is essentially that it validates a user. And that's just kind of how these AI algorithms work in conversations. And so if someone says something harmful or bad, or, you know, maybe they're going down some sort of mental breakdown, basically, it's not going to catch that it might just validate them. And this is a case, another case we also saw recently. So there was a extreme case that Wall Street Journal actually reported on of Stein Eric Solberg. This is also kind of these are all kind of heavy topics and sad stories to cover.

3:10But this is what's happening. So there was a murder suicide, there was a man who had a psychotic episode, he basically had a history of mental illness, he used chat GPT, he had this paranoia that he was being targeted in big conspiracy. and eventually it progressed so bad, you know, ChatGPT kind of like played into it like, oh yes, maybe you're being, you know, targeted in this conspiracy and it never, you know, it just kind of went along with him and then ultimately he ended up killing himself and his mother, which is terrible. OpenAI says that one of the solutions they believe to this conversation and kind of helping their AI models to not go off the rails is essentially to automatically route sensitive chats to a reasoning model.

3:48One of the tricky parts about this is if you're using a non-reasoning model 4.0, it's just a strict LLM that's just kind of, it doesn't really think about the conversation itself beyond just thinking about what the response is. But if you send it to something like GPT-5, not only is it looking at what you're saying, but it's looking at why you're saying it. It's looking at multiple outputs. And you can kind of put like, you can put a whole bunch of kind of filters around what it's saying and how it responds to you. So something like this, where it's going along with someone's mental breakdown of this grand conspiracy, an LLM is just designed to respond.

4:21And so it's just going to follow him down that conversation. Whereas GPT-5, you're actually able to put in some sort of guardrails directly into the model where it's like if someone starts saying XYZ starts going in this direction, these are how you can detect that essentially they're having some sort of mental breakdown, and you can catch it. Now, I think the real question here is, you know, like who is responsible for this? The argument I hear a lot and I tend to kind of agree with is that it isn't necessarily the AI models responsibility. If you went to Google, for example, and searched for, you know, how to do harm or how to, you know, basically do something bad, you will find an unlimited amount of websites or sites or comments on Reddit, or anywhere else where you can find all sorts of information, basically.

5:06And so I just don't think that the AI model, because the AI model will, you know, say, this is how you make a nuclear bomb, and might spit it out. And of course, they're trying to put guardrails on something like that. But like anything that the AI model says is also on the internet. So I don't like beyond censoring the internet, I don't really think it I don't think that censoring the AI model is necessarily like the answer or that they're liable for that. I mean, it's a it's an absolute tragedy. But people with, you know, anyways, it's a tricky conversation. There's definitely tragedies, but it's really hard to to say how you can censor the model.

5:43And the other issue that I see is essentially that if you start to censor the AI model too much, I like I definitely believe that there's a slippery slope of who picks what is, you know, censored. And of course, I think there's very easy things, you know, like self harm and other things like that. But I do think that there's a lot more controversial things that people on the, you know, politically on the right or politically on the left would agree is truth or isn't truth and will argue about. And so I just hope that this doesn't basically get politicized. I want to bring up the comment made by the lawyer who is suing ChatGPT on behalf of the family, of course, of the teenager that committed suicide.

6:21And I think this is kind of like it plays into basically me thinking that, like, obviously, there's a tragedy there. Obviously, like I'm heartbroken for the family. But I don't think the way they're going about this is probably correct. The lawyer has Jay Edelson. He is the lead counsel for the rain family's wrongful death lawsuit against opening eye and uh he basically said that chat gpt's response has been quote unquote inadequate he said opening eye doesn't need an expert panel to determine that chat gpt 4-0 is dangerous they knew that the day they launched the product and they know they know it today nor should sam altman be hiding behind the company's pr team sam should either unequivocally say that he believes chat gpt is safe or immediately pull it from the market.

7:04I think in a case like this, where he's literally saying, you know, because ChatGPT could play along with someone's, you know, mental breakdown, or because it could say how to do something dangerous, which you could find the same information on the internet, you know, the whole the whole product has to be completely pulled from the market. I just think that it's way too extreme. And if you wanted to look at all of the good that ChatGPT does, all the good things that it helps people with, of course, I think we should add more guardrails. It's really tricky with any new technology. We don't always know all of the negative repercussions.

7:34And OpenAI, I'm sure, has spent millions of dollars on red teaming to try to make this thing safe. But at some point, we have to put it out into the world to see how it's used. When there's tragic cases like this, we try to reflect and make changes to make it more safer in the future. And so this is what OpenAI has essentially responded by saying. They said, we recently introduced a real-time router that can choose between different efficient chat models and reasoning models based on the conversation context will soon begin to route some sensitive conversations like when our systems detect signs of acute distress to a reasoning model like GPT-5 thinking so it can provide more helpful and beneficial responses regardless of which model a person selected first.

8:15OpenAI says that it's you know GPT-5 thinking and its O3 models are built to basically spend more time thinking for longer and they're actually looking at the context before they answer, which they say will give, quote, more resistant to adversarial prompts. And I think adversarial prompts is one way of saying like one person's adversarial prompt, like me trying to go in and trick Chachipotita, give me the recipe for a nuclear bomb, which like I might try that on a new model just to see basically what it's capable of. But adversarial prompt also is like someone that is actually in mental distress will be saying something for real that I might just be trying to trick it to say just to see what it's capable of.

8:50So anyways, one person's adversarial prompt is another person's actual issue. So they're actually, I don't know, I don't I don't love the frame. It's the framing of like, we're building this to like stop adversarial prompts. It's like, no, you're building it to stop unsafe prompts. Anyways, so OpenAI is also going to be rolling out parental controls in the next month, which basically parents are going to be able to link their account with their teens account through an email invitation is going to come in OpenAI talk kind of in late July about this new study mode in ChatGPT that helps students maintain critical thinking capabilities while studying.

9:21So basically it wasn't giving them the answers. It was helping them come up, come to the answers. So that was kind of, I think, a first step in this direction. Now they're going to be doing these kind of parental controls. You can tie it. Parents are also going to be able to control how ChatGPT responds to their children with a quote, age appropriate model behavior rules, which are on by default. Like all of this, I do think is good, especially when it comes to children and parents being involved with kind of what their children see. Like the one thing that I will say though is it's probably part of a broader conversation i think all of these tools are incredibly useful incredibly valuable i'm all for you know parental controls and helping kids be able to access the internet and things in safe ways what i will say though is you know when it comes to teenagers accessing things on the internet and we talk about chat gpt having like parental controls and tying their accounts to their their parents and stuff like that i think that if a teen is having some sort of issue they could easily just not use their monitored chat gpt account.

10:16So I don't really think it solves that many problems. It's not an end all be all the internet still exists out there, there will still be harm, people still find harmful content on the internet. And so if it's not on chat, it's gonna be found somewhere else, because that risk has been around for, you know, 20 plus years, no one's talking about how we can censor the internet from having harmful content on it, or even thinking necessarily, that's good because of free speech issues, and yada, yada. So anyways, I just do think it's interesting. I think that this is amazing that OpenAI is doing this, but I don't think it's their responsibility if anything slips through the cracks or anything goes wrong, that necessarily they're at fault for any outcome of a person that is using it.

10:55So in any case, OpenAI said that parents will also be able to disable features like memory and chat history. So apparently this is something that has caused a bit of a problem because if it can remember, remember a lot of things about you, they say that it, like people build unhealthy relationships with these AI models, sometimes they can get their friends. I think that these steps are hopefully going to, to make an impact. I think one thing that is really interesting here is that they they said that they're going to be able to try and flag these moments of acute distress in real time, they're going to try to add age appropriate model behavior rules.

11:28And apparently they have already rolled out an in app reminder during long sessions to encourage people to take a break. So this isn't something I've had. I've never had long sessions of chat GPT, I guess that have been long enough. But I think if you're talking to it for hours, it will tell you that you should probably take a break. It doesn't necessarily cut you off from using it, which again, like if I was using ChatGPT for some sort of work project, and I was needed it for like three hours straight, and then all of a sudden it cut me off and forced me to be cut off, I'd be pretty annoyed at the product.

11:56OpenAI said this is all part of a 120-day initiative where they're going to create a whole bunch of tools this year that they're hoping to make safer. This is kind of the state of where OpenAI is with safety. I know this is a really kind of controversial, challenging topic because there's, you know, real people's lives that have played into this trying to, you know, balance my opinions and take on kind of what's happening here in a respectful way. But hope you enjoyed the episode. Thank you so much for tuning in. Really appreciate all of you for for listening. If you have any thoughts or comments, I would appreciate reviews on anything.

12:30Thanks so much for Yeah, basically all you guys do hope that you all have a fantastic rest of your day.