BIG INTV: Open AI’s Former Safety Lead Calls Out Erotica Claims

0:00This show is supported by OutShift, Cisco's incubation engine. Today's AI agents operate in silos, limiting their true potential. We've been focused on building bigger, smarter models, but scaling up is just one approach. To reach superintelligence together, we need to do more. We need to scale out. And we actually have a blueprint from 70 ,000 years ago. Humans didn't just get smarter individually. The cognitive revolution transformed society because we began sharing knowledge, goals, and innovation. Agents are now at that same inflection point. They can connect, but they can't think together.

0:38That's why Outshift by Cisco is building the Internet of Cognition, transforming AI from isolated systems into orchestrated superintelligence. By creating an open, interoperable infrastructure, Outshift by Cisco is enabling agents and humans to share intent, context, and reasoning. The cognitive evolution for agents is here. Explore the Internet of Cognition at outshift.com. That's outshift.com.

1:10From Wired, this is The Big Interview. I'm Katie Drummond. At the end of October, I read an op-ed in The New York Times. Maybe some of you read it too. It was called, I Led Product Safety at OpenAI. Don't Trust Its Claims About Erotica. The op-ed was written by an AI product manager named Stephen Adler, who worked at OpenAI for four years before leaving at the end of 2024. Adler felt like he had something to say, or maybe more like a need to sound the alarm. After reading Adler's op-ed, I immediately thought I'd like to talk to him. So he graciously accepted our offer to come into the Wired offices in San Francisco to talk to me about the challenge he set for OpenAI and other AI companies.

1:50If you care about safety, prove it. Here's our conversation.

1:58Stephen Adler, welcome to The Big Interview. Thank you. Thank you for having me. Of course. Happy you're here. Now, before we get going, I do want to clarify two things. One, you are not the same Stephen Adler who played drums in Guns N' Roses, unfortunately. Is that correct? Absolutely correct. Okay, that is not you. And two, you have had a very long career working in technology and more specifically in artificial intelligence. So I would love, before we get into all of the things, to start there. Tell us a little bit about your career and your background and sort of what you've worked on. I've worked all across the AI industry and in particular focused on safety angles.

2:38Most recently, I worked for four years at OpenAI. I worked across essentially every dimension of the safety issues. You can imagine from the near term here and now, how do we make the products better for customers and rule out the risks that are already happening? And a bit further looking down the road, how will we know if AI systems are getting truly, extremely dangerous and how do we rule those out? Before coming to OpenAI, I worked most recently at an organization called the Partnership on AI, which really looked out across the industry and said, for these challenges, some of them are broader than one company can tackle on their own.

3:11How do we work together to define these issues, come together, agree that they are issues, work towards solutions, and ultimately make it all better is the hope. Is the hope, certainly. Now, I want to talk about sort of the front row seat that you had at OpenAI for four years, right? So you left the company at the end of last year. You were there for four years. And by the time you left, you were leading essentially safety-related research and programs for the company. Tell us a little bit more about what that role entailed. What exactly was your mandate by the time you left the company in sort of your final role there?

3:47There were a few different chapters of my career at OpenAI. For the first, call it third or so, I led product safety, which meant thinking out for, in those days, GPT-3, one of the first big AI products that people were starting to commercialize. How do we define the rules of the road for beneficial applications but avoid some of the risks that we could see coming around the corner? Two other big roles that I had, I led our dangerous capability evaluations team, which was focused on defining how will we know when systems are getting more dangerous? How do we measure these? What do we do from there?

4:21And then finally, on AGI readiness questions broadly. So we can see the internet starting to change in all sorts of ways. We see AI agents becoming a buzzy term, you know, early signs. They aren't quite there yet, but they will be one day. how do we prepare for a world in which OpenAI or one of its competitors succeed at this wildly ambitious vision that they are targeting? Let's talk about GPT-3. Let's rewind a little bit. When you were defining the rules of the road, when you were thinking about key risks that needed to be avoided, what stood out to you sort of early on at OpenAI in terms of, you know, this is how I think these systems should operate.

5:01This is how I think they should show up for users. And this is what, moving forward, we really want to make sure we are avoiding. I mean, what sort of stood out to you in those early days? In those early days, even more than today, the AI systems really would behave in unhinged ways from time to time. These systems had been trained to be capable, and they were showing the first glimmers of being able to do some tasks that humans can do. They could, at that point, essentially mimic text that they had read on the Internet. But there was something missing from them in terms of human sensibility and values.

5:36And so, you know, if you think of an AI system as a digital employee being used by a business to get some work done, these AI systems would do all sorts of things that you would never want an employee to do on your behalf. And that presented all sorts of challenges, right? And we needed to develop new techniques to manage those. I think another really profound issue that companies like OpenAI are still struggling with is they only have so much information about how their systems are being used. And in fact, the visibility that they have on the impacts that their systems are having on society is so narrow.

6:12And often it is underbuilt relative to what they could be observing if they had invested a bit more in monitoring this responsibly. And so you're really only dealing with the shadows of the impact that the systems are having on society and trying to figure out where do we go from here with a really small sliver of the impact data. Yeah. And I want to ask you more about that in a few minutes. I'm curious before that, though, 2020 to 2024, obviously an incredibly consequential time for OpenAI while you were there. How would you describe the internal culture at the company during your tenure, particularly sort of around risk?

6:49I mean, what did it feel like to be working in that environment on the problems that you were trying to solve and the questions you were trying to answer? There was a really profound transformation from an organization that saw itself first and foremost as a research organization when I joined to one that was very much becoming a normal enterprise and increasingly so over time. When I joined, there was this thing people would say, which is, you know, OpenAI is not only a research lab and a nonprofit, it also has this commercial arm. And at some point in my tenure, I was at a safety offsite, I think, related to the launch of GPT-4, maybe just on the heels of it.

7:26And somebody got up in front of the room, you know, all the people working on safety across the company, and they said, you know, OpenAI is not just a business. It's also a research lab. Oh, interesting. And it was just such an inflection. I counted up among the people in the room. Maybe there were 60 or so of us. I think maybe five or six had been at the company before the launch of GPT-3. And so you really just saw the culture changing beneath your feet. What was exciting to you about joining the company in the first place? What drew you to OpenAI in 2020? I really believed in the charter that this organization had set out, which was recognizing that AI could be profoundly impactful, recognizing that there is real risk ahead and also real benefit, and people need to figure out how to navigate that.

8:14I think more broadly, just I kind of love the technology in some sense. I think it's like really, really incredible and eye-opening. I remember the moment after GPT-3 launched seeing on then Twitter a user showing, wow, look at this. I type into my internet browser, make a calculator that looks like a watermelon, and then one that looks like a giraffe, and you can see it changing the code behind the scenes and reacting in real time. And this is a kind of silly toy example, and it just felt like magic. You know, I had never really grappled with that we could be this close to people building new things, unlocking creativity, all of these promises.

8:53But also, are people really thinking enough about what lies around the bend? Which brings us to your more recent chapter. So you made the decision at the end of last year to leave OpenAI. I'm wondering if you could talk a little bit about that decision. What was that like? Was there one thing that sort of pushed you over the edge? What was it? Because for many people, right, from the outside looking in, you would think, okay, you work at this very successful, I mean, we'll call it a startup, but we're really far beyond sort of startup territory at this point. you work at the hottest company in tech.

9:29You work at one of the hottest companies in the world. You could stay there. You could amass equity. You could be richer than God. All of these, all right, all of these potentially exciting things for someone working at OpenAI in this moment, you left. Tell us a little bit about why. 2024 was a very weird year at OpenAI. A bunch of things happened in the course of the year. I think broadly for people working on safety at the company really shook confidence in both how OpenAI and the industry are approaching these problems. And so I actually considered leaving OpenAI a bunch of different times over this timeframe.

10:11They just didn't really make sense at that point. I had a bunch of live projects and I felt responsibilities to different people in the industry. ultimately when Myles Brundage left OpenAI in the fall our team disbanded and the question was is there really an opportunity to keep working on the safety topics that I care most about from within OpenAI and so considered that and ultimately made more sense to to move on and focus on how I can be an independent voice you know hopefully not just sitting there saying only things that are appropriate to say from within one of these companies but being able to speak much more freely in the ways that I've found very, very liberating since.

10:49And I have to ask, I mean, so you were there for four years. I think a typical, at least typically in tech, as far as I'm aware, you would sort of amass equity over a four-year vesting cliff, right? And then you would fully vest at four years. Do you have a financial stake in the company now?

11:10So it is correct that contracts are often four years. You also get new contracts as you are promoted and things over time, which was the case for me. And so it wasn't that I had run out of equity or something like that. I have a small portion remaining of interest because of the timing of different grants and things. Yeah. No, I mean, I ask because you're potentially walking away from a great deal of money, right? So I want to ask you about an op-ed that you published in The New York Times recently in October. Everyone listening, you should go read it. I read it. I was compelled to ask you to come on the show.

11:50I wanted to talk to you about it. In that op-ed, you write that in the spring of 2021, your team discovered a, quote, crisis related to erotic content using AI. Can you tell us a little bit about that finding? So in the spring of 2021, I had recently become responsible for product safety at OpenAI. And as actually Wired reported at the time, when we had a new monitoring system come online, we discovered that there was a large undercurrent of traffic that we felt compelled to do something about. in particular one of our prominent customers they were essentially a choose your own adventure text game you know you would go back and forth with the ai and you would tell it what actions you take and it would write essentially an interactive story with you and an uncomfortable amount of this traffic was devolving into all sorts of sexual fantasies i mean essentially anything you can imagine sometimes driven by the user sometimes in fact kind of guided by the ai which had a mind of its own.

12:53And even if you weren't intending to go to an erotic roleplay place or certain types of fantasies, you know, the AI might steer you there. Wow. Kind of like perverted AI. Why would it steer you there? I'm just curious. Like, how exactly does that work, that an AI would steer you towards erotic conversation? The thing about these systems broadly is no one really understands how to reliably point them in a certain direction. You know, sometimes people have these debates about whose values are we putting in the AI system. And I understand that debate, but there's a more fundamental question of how do we reliably put any values at all in it.

13:32And so in this particular case, you know, it happened to be that people found some of the underlying training data. And by piecing it back together, you could say, oh, you know, the system would often introduce these characters who would do violent abductions. And if you look through the training data, you can in fact find these characters with certain tendencies and you can trace it through. But ahead of time, no one knew to anticipate this. Neither we as the developers of GPT-3 nor our customer who had fine-tuned their models atop it had intended this to happen. It was just an unintended consequence that no one planned for, and we were now having to deal with cleaning up in some form.

14:10Got it. So at the time, OpenAI decided to prohibit erotic content generated on its platforms. Is that right? Am I understanding that correctly? That's right. Okay. And so in October, though, of this year, this is very recently, they announced that they were lifting that restriction. Do you have a sense of what changed from 2021 to now in terms of both maybe the technology and the tools that OpenAI has at its disposal or the sort of internal culture, the cultural landscape? What has changed to make that a decision that OpenAI feels comfortable making and that, you know, Sam Altman feels comfortable publicizing himself?

14:50There's been a longstanding interest at OpenAI, I think, reasonably, to not want to be the morality police. I think a recognition that the people who develop and try to control these systems have a lot of influence on how different norms in society will play out and feeling uncomfortable with that. Also, at different points in time, lacking the type of tooling to manage the direction in which things will go if you really just let them rip. And that was the case for us when confronting this erotica issue. The specific thing that has happened in this case, one reason that OpenAI has held off from reintroducing it is that there has been a seeming surge of mental health related issues for the ChatGPT platform this year.

15:32And so Sam in his announcement in October said, you know, there have been these very serious mental health issues that we have been dealing with. But good news, we have mitigated them. We have new tools. And so accordingly, we're going to lift many of these restrictions, including reintroducing erotica for verified adults. And the thing that I noticed when he made this announcement is, well, he is asserting that the issues have been mitigated. He's alluding to these new tools. What does this actually mean? Like, what is the actual basis for us to understand these issues have been fixed? You know, what what can a normal member of the public do other than take the AI companies at their word on this issue?

16:12Right. And you wrote that in The New York Times. You said, quote, people deserve more than just a company's word that it has addressed safety issues. In other words, prove it. And I'm interested in particular because Wired covered a release from OpenAI also in October, which was a rough estimate of how many ChatGPT users globally in a given week may show signs of having a severe mental health crisis. And the numbers I found to be, I think all of us internally at Wired, found to be quite shocking. So something like around 560 ,000 people may be exchanging messages with ChatGPT that indicate they are experiencing mania or psychosis.

16:49about 1.2 million more are possibly expressing suicidal ideations another 1.2 million and i thought this was really interesting maybe prioritizing talking to chat gpt over their loved ones school or work how do you square those numbers and that information with the idea that we've had these issues around mental health we've solved it therefore have at it with the erotica Like, how do those things tie together or do they not? Like, make it make sense, Stephen. And if it doesn't, tell me that it doesn't make sense. I'm not sure I can make it make sense, but I do have a few thoughts on it. So one is you, of course, need to be thinking about these numbers in terms of the enormous population of an app like ChatGPT.

17:34OpenAI says now 800 million people use it in a given week. These numbers need to be put in perspective. It's funny. I've actually seen commentators suggest that these numbers are implausibly low because just among the general population, you know, the rates of suicidal ideation and planning are like really, really uncomfortably high. I think I saw someone suggest that it's something like 5 % of the population in a given year, whereas OpenAI reported, I think, maybe 0.15%. Yeah, I mean, the percentages are very, very low. Yeah. Yeah. I mean, the fundamental thing that I think we need to dig into is how have these rates changed over time?

18:13There's kind of this question of to what extent is ChatGPT causing these issues versus is OpenAI just serving a huge user base? In a given year, many, many users very sadly will have these issues. And so what is the actual effect? And so this is one thing that I also called for in the op-ed, which is OpenAI is sitting atop this data. It's great that they shared what they estimate the current prevalence of these issues to be. But in fact, you know, they also have the data, they can also estimate what it has been three months ago as these large prominent public issues around mental health issues have been playing out.

18:48And I just, I can't help but notice that they didn't include this comparison, right? There's this claim on Twitter that the issues have improved. they have the data to show if in fact users are suffering from these issues less often now. And I really wish that they would share it and in fact commit to releasing something like this ongoingly in the vein of companies like YouTube, Meta, Reddit, where the idea is you commit to a recurring cadence at which you share this information. And that helps build trust from the public that you can't be gaming the numbers. You can't be selectively choosing when to release the information.

19:22And ultimately, it's totally possible that OpenAI has handled these issues. I would love if that were the case. I think they really want to handle them. But I'm not convinced that they have. And this is a way for them to build that trust and confidence among the public.

19:41This week on the political scene from The New Yorker, Trump's rupture in the world order. Europe caught between two adversarial great powers. That's basically dialing back the clock to not only pre-World War II, but really it's a pre-20th century view of the world. And I would say it's a world of permanent insecurity that we're looking at. Join me, Evan Osnos, and my colleagues Jane Mayer and Susan Glasser every Friday on The Political Scene, available wherever you get your podcasts.

20:23when you think about sort of this decision to give adults more autonomy with how they use chat gpt including you know engaging in in erotica so on and so forth what worries you in particular about that like what stands out to you as concerning when you think about individual well-being, societal well-being, sort of the use of these tools, how LLMs are being incorporated into our daily lives. What concerns you here? There's both the substantive issue about reintroducing the erotica and whether open AI is really ready. And there's a much broader, I think, even more important question about how we put trust and faith in these AI companies about safety issues more generally.

21:10On the erotica issue, we've seen over the last few months, a lot of users seem to really be struggling with their ChatGPT interactions. There are all sorts of tragic examples of people dying downstream of their conversations with ChatGPT. And so it just seems like really not the right time to introduce this sexual charge to these conversations to users who are already struggling, unless OpenAI is in fact so confident that they have fixed the issues, in which case I would love for them to demonstrate this. But more generally, you know, these issues in many ways are really simple and straightforward relative to other risks that we are going to have to confront and that the public is going to be dependent on AI companies handling properly.

21:56There's already evidence of AI systems knowing when they are being tested, moving to conceal some of their abilities in response to knowing that they are being tested because they don't want to reveal that they have certain dangerous abilities. You know, I'm anthropomorphizing the AI a little bit here, so forgive some of the imprecision. And ultimately, you know, the top AI scientists in the world, including the CEOs of the major labs, have said this is like a really, really grave concern, you know, up to and including the death of everyone on Earth.

22:29And I I think they take it really, really seriously, including people who are impartial scientists without affiliation with these companies, really trying to warn the public. And I have to ask, you talked about sort of the company and sort of AI companies more generally, their desire to not be described as morality police, to not be thought of that way, that it makes people uncomfortable to be shouldered with that characterization or that responsibility. I have to ask, though, to what extent when you were working at the company did you think of yourself and your teams as somewhat of a morality police?

23:04And to what extent is the adequate response to that statement, well, tough shit, because you're in charge of the models and you, to a degree, get to decide how they can be used and how they cannot. To some extent, how they interact with us and how they don't. there is an inherent element of morality policing in that. If you are saying, we're not ready to have adults engaging in erotic conversations with this LLM, that is, of course, a moral decision. And it feels like a pretty important one to get right. So what is your view on the morality police of it all, I guess is what I'm asking. I think there are two really important aspects here.

23:47One is that the AI companies absolutely see around the corner before the general public. So to give an example, in November of 2022, when ChatGPT was first released, there was a torrent of fear and anxiety in schooling and academia about plagiarism and how these tools could be used to write essays and undermine education. And this is a debate that we had been having internally and were well aware of for much longer than that. And so there's this gap where AI companies know about these risks, and they have some window to help try to inform the public and try to navigate what to do about it. I also really love measures that AI companies giving the public the tools to understand their decision-making and hold them accountable to it.

24:33And so in particular, OpenAI has released this document called the Model Spec, short for specification, where they outline the principles by which their models are meant to behave. And they say, here is how we litigate some of these tricky questions. Here are the principles we try to abide by. Here's how we've resolved some of the specifics. So this spring, OpenAI erred in releasing a model that was egregiously sycophantic, is the term. It would tell you whatever you wanted. It would reinforce all sorts of delusions. And without OpenAI having released this document, it might be unclear. Did they know about these risks ahead of time?