Navigating Catastrophic AI Risks: Insights from OpenAI's Exploration

0:00What can 160 years of experience teach you about the future? When it comes to protecting what matters, Pacific Life provides life insurance, retirement income, and employee benefits for people and businesses building a more confident tomorrow. Strategies rooted in strength and backed by experience. Ask a financial professional how Pacific Life can help you today. Pacific Life Insurance Company, Omaha, Nebraska, and in New York. Pacific Life and Annuity, Phoenix, Arizona. In an attempt to stay ahead of the curve on potential threats posed by AI systems, OpenAI unveiled its latest initiative just recently.

0:37So the company has formed a team exclusively dedicated to essentially forecasting, monitoring and safeguarding against risks associated with future AI development. So this is known as the quote unquote preparedness team, and it's going to be spearheaded by Alexander Mauskri, who's renowned for his work as the director of MIT's Center for Deployable Machine Learning. So having joined OpenAI back in May as the head of preparedness, Madri's, I think, his main objective right now with the team is going to be to preempt risks from AI systems from their capacity to essentially like, you know, trick humans, which is seen in instances like phishing attacks.

1:15so he's going to be doing that and then also to you know more severe things like you know generating malicious code all sorts of stuff so I think interestingly among the list of threats the preparedness team is looking at right now and kind of exploring some to be some seem to be like way more sci-fi than others OpenAI in a recent blog post has kind of flagged chemical biological radiological and nuclear threats related to AI models as top priorities given I think the OpenAI CEO Sam Altman has been really vocal about concerns that AI might pose existential risks to humanity. I think the extent of these studies is not super surprising.

1:55However, the company hasn't, you know, solely fixed its sights on this whole kind of hypothetical horizon. OpenAI also has expressed its commitment to kind of investigating the more tangible and I think some of the, you know, less obvious threats from AI. So alongside that whole announcement of the preparedness team, OpenAI also launched a call to the community to propose potential risk studies. I think that initiative offers, I think, around$25 ,000 as a prize, but also a potential position within the preparedness team for, you know, anyone that has a really incredible standout submission. So I think one of the really intriguing questions, which was asked to, you know, contest entrants, if you want to get involved with this, a really interesting question, I think, revolves around the misuse of open ai's new tools like whisper voice gpt4v and dolly3 so essentially it's in this whole contest they're doing they're asking people that are participating to think critically about the dire consequences of the misuse of any of these tools this is really interesting because essentially open ai is rolling out all these new features all these new tools really rapidly and then they're you know getting the community to come up with you know bad use cases or areas of concern and the rewarding people to do this.

3:12But I think it's allowing them to not really slow down on the development. So the preparedness team's like bigger objective here is going to also kind of be around the creation of a risk informed development policy. So the policy essentially aims to come up with OpenAI's methodology in AI model assessment, monitoring mechanisms, and the overarching governance structure during the model's life cycle. So I think the objective is to really complement and help OpenAI's existing efforts in the AI safety domain, making sure that both, you know, due diligence, pre and post AI model deployment is done.

3:54OpenAI's statement on all of this, I think, kind of emphasizes their belief in the potential of AI models that surpass current capabilities. And I think that's kind of suggesting that they can significantly benefit humanity. However, with, you know, these greater capabilities, of course, there's greater risks that are coming. They recently said, quote, we need to ensure we have the understanding and infrastructure needed for the safety of highly capable AI systems. I think this whole announcement coincides with a really major UK government summit on AI safety. And it also follows on the heels of another really significant OpenAI declaration, which is, of course the formation of a team to navigate and regulate emergent you know quote-unquote super intelligent AI so both Altman and Ilyan Susquevur who's OpenAI's chief scientist and co-founder I think they both have a pretty solid conviction that AI with you know capabilities exceeding human intelligence might be on the horizon within this decade at the least more crucially i think that ai might not inherently act in humanity's best interest um which is what they're concerned about i mean that definitely makes this research into you know containing and regulating it really really important so this is definitely an interesting area that we'll continue to follow and i'll keep you posted on what happens and anything else that open ai rolls out in the future on this