Elite Safety Summit Salary: OpenAI's $555K+

0:00OpenAI is looking for a new head of preparedness, someone that can help the chat GPT maker avoid a lot of the catastrophic events we worry about with AI. Now, this is an interesting story because they previously had a head of preparedness who left. Many people from their safety team have left, and a lot of people say this is due to Sam Altman not prioritizing that part of the company and basically just pushing out models without doing all of the red teaming. This is a multi-billion dollar company, so I don't think that's 100 % the full story here. Today on the show, we're going to get into what they're looking for, how much Sam Altman is personally willing to pay for this new head of preparedness, why now is the time, and everything he's been saying about this on X.

0:41We're going to get into all of that, but before we do, I want to say a big thank you to the sponsor of today's episode, which is Delve. If compliance is something that's slowing down your deals, whether that's SOC2, HIPAA, GDPR, I know that it's a ton of work with the screenshots and the spreadsheets and all of the endless back and forth, and that is why this episode is brought to you by Delve. Delve uses AI agents to automate compliance end-to-end. They collect evidence, they fill out security questionnaires, and they customize controls to your actual business so you can get compliant in days and not months.

1:12You also get one-on-one Slack support from real security experts who respond fast. Over a thousand fast-growing companies trust Delve to close deals faster and stay compliant as they scale. I'll leave a link in the description to Delve.com if you're interested in learning more or booking a demo. Alright, let's get into what's going on with OpenAI. Sam Altman himself has personally posted on X saying that, quote, we are hiring ahead of preparedness. This is a critical role and an important time. Models are improving quickly and are now capable of many great things, but they are also starting to present some real challenges.

1:47The potential impact of models on mental health was something we saw a preview of in 2025. We are just now seeing models get so good at computer security, they are beginning to find critical vulnerabilities. Okay, there's a lot going on here, and I'll read more of his tweet. He said a couple other things about this role, But essentially what they're looking for is someone that can help, you know, prevent all of the catastrophic events. We've been terrified that AI could pose, you know, risk in the future. And I think this is interesting because there are a bunch of things happening with security where we see OpenAI has said that they actually like basically fine tune and train models to try to breach security.

2:25and then they use those and and to try to like harden their models and then they you know put in safeguards so that people can't use open ai models to become like hackers and and breach security what's interesting to me in this is that they are actually going and training the ai to be able to do this in the first place which i mean is kind of crazy but maybe you need to be able to do that to be able to control it but one shocking thing they found while they were doing this is that ai is getting so good it's discovering security vulnerabilities it's discovering ways to you know, engineer social engineer situations to be able to get data to be able to get information to be able to hack into crack into things that humans weren't able to do.

3:04In other words, they had like literal a red team of hackers that are like trying to use AI and chat GPT to like hack into things. And they and then they had these real people going up against actual just like training AI model to be a hacker and the AI that was trained to be a hacker was doing better than the actual people. And when I say better, it was essentially thinking of new vulnerabilities, really elaborate, complex, multi-step ways to get data and to hack into things that people were not coming up with. And so this is definitely part of the big concern that Sam Altman is pointing out here and that he's worried about.

3:37He said, quote, we have a strong foundation of measuring growing capabilities, but we are entering a world where we need more nuanced understanding and measurement of how those capabilities could be abused and how we can limit those downsides both in our products and in the world and in a way that lets us all enjoy the tremendous benefits. These questions are hard and there is little precedent. A lot of ideas that sound good have some real edge cases. Now the thing that I do think is interesting here and like it's a great point is we obviously all love ChatGPT. We obviously all want this technology to get better.

4:06It's helping us a lot with work. It's saving us a lot of time on a number of tasks and you know menial things that I'm not having to do anymore that I used to do but at the same time there are real risks so we have to figure a way to balance some of the the risks and you know if we want the ai models to get better they're also going to get better at areas that are you know areas of concern and so how do we how do we mitigate that and that's who they're hiring for uh sam altman finished off his tweet by saying if you want to help the world figure out how to enable cyber security defenders with cutting-edge capabilities while ensuring attackers can't use them for harm ideally by making all systems more secure and similarly for how we release biological capabilities and even gain confidence in the safety of running systems that can self-improve, please consider applying.

4:50This will be a stressful job and you'll jump into the deep end pretty much immediately. So this job is not for the faint of heart. But if you go over to their careers page and they give you an outline about the team and the role and they say what you're going to be doing in the role. One thing that is interesting is the compensation for this particular job here is$555 ,000 a year and they also have equity along with that. So, I mean, it's a very generous pay package, but it is obviously very stressful and entails a lot. When they say what you'll actually be doing in the role, they said own OpenAI's preparedness strategy end-to-end by building capability evaluations, establishing threat models, and building and coordinating mitigations, lead the development of frontier capable evaluations, ensuring that they are precise, robust, and scale across rapid product cycles, oversee mitigation design across major risk areas, ensuring safeguards are technically sound, effective, and aligned with underlying threat models.

5:55So one thing that I do think is interesting here with this role is, I mean, very publicly, Sam Altman is going out saying, look, we're gonna pay over half a million dollars a year, plus, you know, compensation, which I would imagine this is like a million dollar a year, maybe$750,$800, a million a year, maybe more with the compensation package with the stock options, probably over a million if I'm being honest. So, you know, great compensation for this role. But what does that mean? I think because this is so publicly being posted, you could say one of two things. Sam Altman is like, you know, signaling to everyone, look, I'm very serious about this.

6:26We really want to hire the best person. Also, it's going to be a very public facing person, I would imagine, just given the nature of the role. And if anything goes wrong at OpenAI, if any of these systems are used for safety, you know that like, whoever gets this role is going to be pointed at like, oh my gosh, XYZ person didn't do their job, because this is what they were supposed to do. Now, one thing that I do think is interesting here is they already had ahead of preparedness. They first announced that they were creating kind of this role in 2023. They said that it was going to be responsible for studying potential catastrophic risks, whether they were immediate, like phishing attacks, or more speculative, like nuclear threats.

7:03So that was kind of when it first came out. But less than a year later, they reassigned their head of preparedness, who was Alexander Madri, to a job focused on AI reasoning. So the person that was in charge of that, they reassigned them less than a year later, and now they're trying to fill the role again. and it's interesting like I'm not sure if this was because Alexander himself requested to move to something different or if they just needed help over there and he had a lot of experience I think there's you know definitely other safety executives at OpenAI that have left the company or they've taken new roles outside of preparedness and safety and kind of moved it around as I think that got a little bit deprioritized last year Google Gemini was putting out a lot of updates you know they were concerned about meta they were concerned about Grok they were concerned about like a lot of other players including Claude and it felt like they definitely kind of put this on the back burner because they didn't want to spend all their time on safety when they were it felt like they were kind of falling behind on some of the model features and they were just focusing on getting the model out as fast as possible now they're going to bring someone in and that's going to be their role but that person is going to take a lot of like personal I wouldn't say personal liability because obviously everything at the end of the day is opening eyes you know responsibility but that person is going to be very public-faced and if anything goes wrong they will 100 % feel a lot of the heat on that.

8:22They also recently updated their preparedness framework saying that it might, and they said that they're going to adjust it. It's, you know, and some of their safety requirements could be adjusted if a competing AI lab released a high-risk model without similar protections. So they're like, look, we're gonna make our models really safe, but like if Gemini or Grok come out and their model is crushing it, but they don't have the same protections, then we'll just like dial back the safety on it so that we could be competitive with them, which is definitely a very interesting point. I know a lot of people will find concerning.

8:52Right now, I think Sam Altman, he alluded to this in his post, but a lot of these generative AI chatbots have faced a lot of scrutiny around specifically mental health. There's a bunch of recent lawsuits that allege that chat GPT enforces users' delusions. It increased their social isolation and even led some to suicide. So OpenAI is definitely at a critical moment where they have to get this right. I think it's, you know, not an easy thing. ChatGPT or OpenAI did say that they're continuing to work on improving ChatGPT's ability to recognize signs of emotional distress and to connect users to real world support.

9:27I think this is amazing. Obviously, this is one of these situations where you have to learn and figure out what can go like as you see things that go wrong. You have to fix them and try to improve them. I think that that is the exact right approach. I would like to see more and more from OpenAI. So this is going to be interesting to see how this rolls out. and hopefully to see that their safety platform is robust but also doesn't slow down the innovation that a lot of us are using in our day-to-day lives. Thank you so much for tuning into the podcast today. As always, make sure to go check out AIbox.ai, my own startup, to get access to all of the latest models and build incredible tools if you're not a coder.

10:03And also go check out the sponsor of today's episode, which is Delve.com. There's a link in the description to Delve, and I'll see you in the next episode.