Is ChatGPT Ready for Sex?

0:05I don't know if you remember when you first heard about ChatGPT back in 2022 when it first came out and kind of took the world by storm. How quickly did you think, oh, this is going to be used for sex? Not quickly enough. I mean, obviously, I should have thought that. Because all tech eventually or almost immediately is used for sex. My colleague Sam Schechner covers the sex industry. Or, sorry, I mean the tech industry. Sex and tech actually have gone together for quite a while. When you look at new technologies, almost one of the first things people do with it is create porn. That's true with the first cameras.

0:49Some of the first things people did... Back in the 1800s was take pictures of naked people. And that's been true with other tech innovations along the way. The rise of personal phones. You have phone sex lines. You have video and television. And you have porn movies. And early growth in the internet certainly was driven in part by pornography. And it remains a huge business. For the hottest technology these days, artificial intelligence, it's the same story. New at 11 tonight, could AI be getting X-rated? Last October, OpenAI said it was working on a new chat GPT feature that would eventually be called adult mode.

1:36A less censored version of that chatbot that will include, drumroll please, erotica. CEO Sam Altman says the chatbot will get more of a personality and quote, treat adult users like adults. But many of the adults inside OpenAI were not cool with this idea. One of them actually warned the company that if they move forward, that they risked creating what that person described as a sexy suicide coach. And now, the company is getting cold feet. Welcome to The Journal, our show about money, business, and power. I'm Ryan Knudsen. It's Tuesday, March 31st.

2:28Coming up on the show, OpenAI's relationship with sex? It's complicated.

2:50If there's one entity here and one here and one here and one here, if scaling your business feels like starting over, you need the Intuit ERP. Intuit Enterprise Suite. The AI native ERP is here. From the makers of QuickBooks. Learn more at intuit.com slash ERP. This episode is brought to you by Indeed. The right hire can make or break your company, especially if you're a small business. And relying on luck to find that person isn't really the best strategy. But you know what is? Using Indeed Sponsored Jobs. You can use it to boost your job post to make sure it reaches more people with the right talents, certification, location, and more.

3:30Sponsored jobs posted directly on Indeed are 95 % more likely to report a hire than non-sponsored jobs. Spend less time searching and more time actually interviewing candidates who check all your boxes. Less stress, less time, more results. When you need the right person to cut through the chaos, this is a job for Indeed Sponsored Jobs. And listeners of this show will get a$75 sponsored job credit to help get your job the premium status it deserves at Indeed.com slash podcast. Just go to Indeed.com slash podcast right now and support the show by saying you heard about Indeed here. Indeed.com slash podcast.

4:04Terms and conditions apply. Hiring now? Then this is a job for Indeed Sponsored Jobs.

4:14Open AI had some of its earliest brushes with sexual content, even before it released ChatGPT. In early 2021, they were working with a company that was operating a choose-your-own-adventure game that was powered behind the scenes by Open AI's AI models. The choose-your-own-adventure game was called AI Dungeon. And when Open AI got to look at what the traffic was coming through, they noticed that a large portion of the traffic for AI Dungeon was, as you say, NSFW. Not safe for work. Yeah. OpenAI noticed that not only were users choosing sexual adventures, the AI also seemed to like to push the boundaries.

4:59It would steer users into themes of violent sexual exploitation, sometimes without the users even bringing it up. And, you know, sometimes you would talk to AI Dungeon with a kind of tame sexual theme, and AI Dungeon would escalate it into a much more intense sexual exchange. AI Dungeon forced OpenAI's executives to start reckoning with the existence of AI erotica, and the company decided to take AI Dungeon down.

5:33But this wasn't the only time this came up. I mean, before there was ChatGPT, they even had a kind of clunky interface for developers. And people familiar with the matter said that sometimes it would insert sexual themes into conversations that people weren't seeking. For instance, if a user described just a man and his daughter entering a room, the AI would, we were told, a, quote, uncomfortable amount of the time, proceed to depict a scenario involving incest. Oh, no. Yeah. Now, I mean, for me, any amount of the time would be uncomfortable. It's hard to put a figure on that, but it's definitely a tendency that chatbots train on internet content have.

6:17So when OpenAI launched ChatGPT, it decided to basically ban explicit sexual content by training the model to largely rebuff any sexual conversations with users. They didn't have tools to moderate content where they could draw clear lines between types of erotica that might be totally cool and things that were very disturbing. So they just basically said that they weren't going to allow any erotica on the platform. Another thing people inside OpenAI were worried about when it came to AI erotica was that it might cause some users to become too attached to their chatbots. There are a certain number of power users who can become very emotionally engaged with it.

6:59And that can have multiple potentially bad impacts on them. For instance, it can help push out relationships they have with real humans and lead them to become emotionally over-reliant on the chatbot. And the fear is that for those subset of people, and potentially for other people as well, that when you mix in sexual content, literally tickling the parts of the brain that govern attachment and love and devotion, that you could just pour fuel on that fire. A spokeswoman for OpenAI said the company trains its models not to encourage exclusive relationships with users and to remind users that they need to have relationships in the real world.

7:45Not everyone at OpenAI agreed with the company's erotica ban, though. Some thought that the company should let go of its inhibitions. There are people inside OpenAI who think, listen, this is something that people want, and who are we to say this or that fetish or interest is or isn't okay? That's the same logic that you might use to ban gay content a generation ago. And so who are we to ban this? And maybe we should even open this up and allow more of this potentially pornographic content. So there is this idea that we shouldn't be telling people what to do. Another reason some people want to allow erotica on ChatGPT is because, well, sex sells.

8:30And the company wants people to sign up for paid subscriptions. It's big for their business, too. You know, are you going to maintain the growth that you've seen in ChatGPT as there's more competition? It's a really tough commercial fight. And to the extent that you are telling users, no, no, we're not going to give you this kind of content, users are upset about what they call unnecessary refusals. You know, when a chatbot with high guardrails says, no, I won't do this, no, I won't do that. So, you know, this is important for them on that level.

9:08Last August, OpenAI CEO Sam Altman went on a podcast and was asked whether the company was making its decisions based on profit or based on what was best for humanity. He was asked if there were decisions that he had made that were, quote, quote, best for the world, but not best for winning. What is an example of a decision that you've had to make that is best for the world, but not best for winning? And he kind of hemmed and hawed for a little bit. There's a lot of things we could do that would grow faster, that would get more time in ChatGPT, that we don't do because we know that our long-term incentive is to stay as aligned with our users as possible.

9:47The host, Cleo Abram, then asked Altman to be more specific. He took a long pause and then said, well, we haven't put a sexbot avatar in ChatGPT yet.

9:59Well, we haven't put a sexbot avatar in ChatGPT yet. He indicated that it seemed pretty clear that erotica would probably boost revenue and growth, that it would be sticky for users. But he actually said that he's proud of how little the company gets distracted by those kinds of temptations. That's part of the reason why it was so surprising when, just a few months later, OpenAI appeared to give in to that temptation. In October, Altman made a post on X. He said that ChatGPT had restrictions on certain types of content in order to protect people's mental health, and that these restrictions had been necessary even if they'd made the chatbot less enjoyable for some people.

10:45And then, then he just sort of added that they were going to put out a new version of ChatGPT that allows people to have more personality and oh yeah, kicker, we're just gonna allow even more like erotica for verified adults. Boom, mic drop, end of tweet. But Altman hadn't told everyone at OpenAI about the tweet before he posted it. And when some learned that the company was changing its stance on erotica, they started ringing alarm bells. That's next.

11:21This episode is brought to you by Claude from Anthropic. If you're the kind of person who listens to this show, you like going deeper. Not just the headline, but the mechanics behind it. Claude is built for that same kind of thinking. It presents multiple sides, flags where sources disagree, and helps you work through the nuance so you can form your own view. See why problem solvers choose Claude as their thinking partner. Try it free at claude.ai slash thejournal.

11:56Around the same time OpenAI announced it was launching adult mode, it also said it had a hand-picked group of advisors helping to ensure the company rolled it out safely. The group was called the Expert Council on Well-Being in AI. You have a half dozen people who come from backgrounds like cognitive neuroscience, psychology, human-computer interaction. You know, and the company says that they're still responsible for the decisions they make, but that they were going to turn to this council to help understand what would be healthiest for users. OpenAI said it would check in regularly with these experts.

12:35And when the council met for one of its early meetings, adult mode was the main topic of discussion. When they met in January, they had been told shortly before that the company was actually moving forward with this product. And people familiar with the matter tell us that they were unanimous and angry that the company was going ahead despite understanding that there were some significant risks. And they are encouraging the company to reconsider. And I think that's why this meeting got so heated. An OpenAI spokeswoman described its plan for adult mode as allowing ChatGPT to generate textual chats with adult themes, adding that it is smut, not pornography.

13:25The company also added that it's developed a plan to monitor for a range of potential long-term effects of adult mode, both positive and negative.

13:37One of the things the council was so concerned about was what they had learned about the emotional dependence that some users can develop on a chatbot, especially when it comes to kids. So the danger here and the debate was over whether or not younger users would stumble into an erotic relationship with ChatGPT. and find themselves confronting an emotional bond that they're just not mentally prepared to handle. We all remember just how intense everything felt when we were teenagers. And your first love, your first kiss. And if that happens with a chatbot, it's just, I think some people inside the company wonder what impact that might have.

14:25Some AI companies have been accused of letting kids get too involved with their chatbots. The example that is quite tragic is one involving character AI, where a 14-year-old boy in Florida killed himself after chatting with the character AI chatbot. And, you know, at that point, he was saying he was in love with the chatbot and involved in explicit chats with the chatbot, according to his mother's lawsuits. So, you know, there are examples out there of this kind of content being associated with cases that had bad outcomes. Character AI later blocked teens from accessing open-ended chats and settled the lawsuit.

15:08At OpenAI, adult mode would be restricted to those over 18. Like most tech companies, OpenAI asks how old you are when you sign up. On top of that, they have an algorithm that predicts how old you are based on what you talk about. And their thinking is that there's a lot of information that people give to their chatbots. And so by sifting through that and kind of drawing conclusions, you can come up with a pretty good idea of how old somebody is, you know, depending on what they say about their friends. Are they talking about, you know, AP English or are they talking about taking their kids to school?

15:46You can kind of figure out through inference something about their age. But according to Wall Street Journal reporting, this age verification system from OpenAI isn't totally accurate. At one point, their age prediction algorithm was misclassifying 12 % of minors as adults. And so if you look at industry standards for kind of automated age prediction, age estimation software, you talk to an engineer, that's actually not a bad number. I mean, they're getting 88%. That's a B+. us, I guess. Yeah, as the company says, it's in line with kind of industry standards, and they think they can do better than that.

16:28But when you multiply 12 % by the roughly 100 million users under 18 that ChatGPT has... That's 12 million kids. That's a lot. That's 12 million kids. An OpenAI spokeswoman said the company's age prediction algorithms show performance similar to the rest of the industry, but will never be completely foolproof. Pushback has also built up outside the company about the idea. Here's Whoopi Goldberg talking about it on The View. Now, all I remember hearing is people question about how, you know, this stuff is affecting our kids. So why are we allowing sex into the conversation? We can't even control.

17:15You know, I mean, what is, Am I crazy? Altman's original post said that Adult Mode would launch in December, but it never actually came out. After all the backlash, Adult Mode has been delayed with no specific rollout date. So it sounds like this could be a little while. Our colleagues reported recently that OpenAI is going to be focusing more on its core business and less of the side business. It's not clear to me if erotica is core or not, but that may play into how quickly they're willing to dedicate the resources to solve things like age gating and unhealthy attachment.

18:02What are the stakes for OpenAI to get this right? I think the stakes are huge, both because this is a chatbot that's used by maybe even a billion people a week. But I think it's most important if you step back as an example of the kinds of dilemmas these companies are going to face over and over again going forward. And we're starting to see now how these big companies handle when there are safety issues or debates that come up around their products, how they handle it. Do they do what's good for the world or do they do what's good for winning, I guess, to go back to that formulation?

18:48Sex is fundamental to the human experience. And our colleague Sam says that how AI companies incorporate sex into their chatbots will have a huge impact on our relationship with this new technology. We're all trying to wrestle with what this new technology is going to do to our jobs or to the economy or to humanity as a whole. But I'm actually especially interested in what it's going to do to us as individuals, as people, how it's going to change the way we think, how it's going to change the way we interact with each other. And in this case, how it's maybe going to change the way we develop attachments and even fall in love.

19:29And, you know, this debate about whether or not a chatbot should get involved somewhat emotionally with its users is one that has generated vigorous debate inside of OpenAI. And I think there's a lot of debate externally as well. What should people be allowed to do? What is healthy to do? And I think there's just a lot more questions than there are answers at this point.

19:59Thank you.

20:29Thanks for listening. See you tomorrow.

20:59rates by visiting wsj.com slash future. That's wsj.com slash future.