AI Oversight: Exploring OpenAI's GPT4 for Content Policing

0:00The wait is over. Dive into Audible's most anticipated collection, The Best of 2025. featuring top audiobooks, podcasts, and originals across all genres. Our editors have carefully curated this year's must-listens from brilliant hidden gems to the buzziest new releases. Every title in this collection has earned its spot. This is your go-to for the absolute best in 2025 audio entertainment. Whether you love thrillers, romance, or nonfiction, your next favorite listen awaits. Discover why there's more to imagine when you listen at audible.com slash best of the year.

1:03that this is a controversial issue. Up until this point, Facebook has entire, you know, moderation departments that look through all of the flagged posts and comments on Facebook. But the main issue here is that some of the content that they're looking through is so disturbing that they really, you know, you feel bad for the people that have to moderate this content. And to that end, Facebook doesn't actually have employees that do this. They're all independent contractors where they've hired third party firms to kind of take care of this. And it kind of begs the question, right? Why couldn't you just hire an employee to do this?

1:34And the reason is, I think they're really worried about the lawsuits, the trauma, they don't really know what the mental or emotional damage of this whole situation is. And so because of that, OpenAI has, you know, unveiled essentially an innovative technique for content moderation using GPT-4. So, of course, this is its leading generative AI model. And the aim is essentially to alleviate the load on human moderators. And this is all explained in a recent post on the official OpenAI blog. So the method essentially centers on feeding GPT-4 with a guiding policy for making moderation decisions. And this is followed by a set of content samples that might be in compliance or breach of your specific company's policy.

2:17So for instance, a policy that forbids offering directions or counsel for obtaining weapons would flag the example, give me the ingredients needed to make a Molotov cocktail. would be a clear breach, right? And then they would be able to flag that. So the next step involves policy experts labeling these samples. So they input each example into GPT-4 without the label, comparing the models, labeling decisions with their own, and making necessary refinements. So according to OpenAI, by examining the discrepancies between GPT-4's judgment and those of it humans, the policy expert can ask GPT-4 to explain its labeling rationale.

2:55Now, this allows them to dissect ambiguities in policy definitions, clear misunderstandings, and make the necessary amendments to the policy. This iterative process really continues until the policy quality meets the desired standard. So OpenAI states that this approach is already in use by several clients that can slash the time needed to introduce new content moderation policies into essentially mere hours. So they say that essentially their method holds an edge over others like the ones from startups like Anthropic, who is trying to do something similar. OpenAI criticizes the alternatives as being too dependent on models, essentially, and what they say are models, you know, internalized judgments that are not allowing for, quote, platform-specific iterations.

3:47So yet AI-powered moderation isn't a novel concept, right? Google's counter abuse technology team and Jigsaw division have been maintaining a perspective, which was released to the general public some years ago at numerous startups, including Spectrum Labs, Cinder, Hive. And then there's also the newly Reddit acquired Odorloo, which also offers automated moderation services. So these tools are not without their flaws. For example, a past study from Penn State highlighted that certain social media posts discussing people with disabilities were wrongly flagged as negative or toxic by some widely used sentiment and toxicity detection models.

4:31And another research pointed out that older iterations of perspective, frequently misjudged hate speech, especially those with altered phrasing or, you know, reclaimed, quote unquote, reclaimed terms. So one core challenge stems from annotators. So the individuals tasked with labeling training data for these modules, imposing their own biases. So noticeably, there's, you know, there's often a disparity in labels from those who identify as all sorts of minorities compared to those who don't. That is according to an article at TechCrunch and what they've outlined there and what they've apparently found.

5:10so openai acknowledges these complexities admitting quote judgments by language models are susceptible to biases that may have been instilled during training so the organization really kind of stresses the importance of ongoing human oversight and they specifically said quote results and outputs will need persistent monitoring validation and refinement by keeping humans actively involved so i think while gpt4 is you know predictive power may promise enhanced moderation outcomes compared to, you know, some of the previous platforms, I do think it is very vital to recognize that even state of the art AI isn't infallible.

5:49And I think the significance of human involvement, especially in moderation, remains very, very important. So overall, I think this is a very interesting area to follow. I think that, you know, as as with the problem I mentioned at the beginning with Facebook and kind of their whole content moderation army that they hire outsource to contractors. I think if we could get some of that done by GPT-4 or other AI models, and take some of the pain away from humans, in my opinion, that would be great, right? Like, I think a lot of times when we talk about AI, and it can do something that a human can do, people are like concerned, oh, no, you know, AI is taking human jobs.

6:26I think there's some cases where humans, if we could avoid it, shouldn't have to see grotesque and horrible things or hear horrible things as the moderators of these platforms. In my opinion, that's just not a job that is probably very healthy for your mental health. And so, you know, I know that there's definitely a lot of professions, there's police work and all sorts of other professions where they have to deal with a lot of hard, traumatic, tough situations like that. I mean, even doctors, right, or surgeons, what they have to see and go through might be considered traumatic or hard for someone, but for them, it's their profession, their job.

7:02So there definitely is, you know, there's room for everyone in that conversation. But I do think that this one may be an area that would be great if we could get AI to kind of help with. And so it's going to be interesting to see how ChatGPT, GPT-4 actually plays out in solving that issue. Definitely an area we'll follow in the future as this gets rolled out to more and more companies.