Toxic-Free Zones: Call of Duty's AI Initiative Against Voice Chat Misconduct 🌶️

0:00So the headline here is that Activision is taking some fairly concrete steps to address the longstanding issue of toxic conversations within the video chat feature of its popular game Call of duty. So this is something that since I was a kid has always been, you know, an issue a lot of people took issue with was essentially everyone that's jumping on the chat there. There's a lot of immature 11 year old, 12 year old boys, 30 year old guys, girls, all sorts of people saying what would be quote unquote considered toxic content. So in collaboration with Modulate, the game publisher is aiming to deploy what is called Toxmod.

0:36So this is an AI-based voice chat moderation system with the release of Call of Duty Modern Warfare 3 on November 10th. And the technology is going to, quote-unquote, identify hate speech, discriminatory language, and harassment in real time and deliver immediate penalties. Now, oof, some of those things are good, right? You know, things though like harassment. Well, I know theoretically we do definitely want to eliminate harassment. I'm sure there's going to be plenty of guys that are all playing with their buddies, joking around, quote-unquote harassing each other, and they're going to get penalized.

1:14So I'm sure there'll be some complaints there, and we'll dive into that later. In any case, I think this move adds a new layer to Activision's existing moderation tools, which include text-based filtering across 14 languages and an in-game reporting system where you can report people. again a lot of people are going to probably not like that michael vance who is the cto at activision underlined the significance of this collaboration stating quote there's no place for disruptive behavior or harassment in games ever tackling disruptive voice chat particularly has been an extraordinary challenge across gaming with this collaboration we're now bringing modulate state-of-the-art machine learning technology that can scale in real time for a global level of enforcement.

1:58While this partnership could bring a new era of, you know, quote unquote, trust and safety in gaming, I think it's also worth considering some of the potential downsides. So I would say that there definitely is a fine line between moderation and stifling free speech, which is raising concerns with a lot of people about AI essentially becoming a form of, you know, of thought control. Like you can imagine like T-Mobile or Verizon or some phone company having this kind of technology in there and deciding that something you said on a conversation was considered a hate speech or harassment and you know deciding that they're gonna deploy this to make it so if you say something bad it's gonna block it um i'm sure a lot of people would not be thrilled to have their conversations edited in real time um or you can imagine perhaps a more realistic which is probably already happening today situation where social media platforms are deeming your words as toxic and therefore um deleting your posts banning your account don't or just you know shadow banning not showing your content now i know that a lot of people are going to be like well that's good right toxic content is bad it's toxic nuclear waste is toxic your speech is toxic both of those are bad the problem becomes in judging what is toxic and not toxic and a lot of people have different uh beliefs around around that one person's concepts that they think are ideologically good or even one country's um would be considered toxic to another we know this.

3:21There's a lot of people with different ideas, opinions, religions, ideologies, all of it. And so it just is a tricky situation having that. And that's not something new, right? Social media companies have been wrestling with this for a long time. That's their own thing to figure out. But where this is coming into the forefront and is a new wave is now in real-time audio editing. So it's not just like, you know, oh, that person said a mean tweet or this person said something bad on Facebook, ban their reach. It's like they're speaking in real time. We don't agree with that toxic word, which perhaps could be fine.

3:55The problem, I think, is when it becomes ideologies, right? Like you question the government, whether you're conservative or liberal questioning the opposition government should definitely be a healthy thing done. And if any of this technology gets deployed by governments or other people to censor people's speech live, right? Like imagine if you were on Twitch or you're live streaming on Twitter, and Elon decided to censor a liberal's perspective on something because he didn't agree with it, or someone on Twitch decided to censor a conservative's perspective because they didn't agree with it. Definitely would not be appropriate steps, and so I think this is what the big concern is.

4:33So, an initial beta test for the Toxamod system is going to launch in North America on August 30th within existing games, Call of Duty Modern Warfare 2 and Call of Duty Warzone. This is going to set the stage for a broader rollout, excluding Asia. Interestingly enough, Asia did not make the cut. I'm not sure if that's for language reasons or for, you know, countries like China not allowing them to do this. Because maybe China's got their own system going. I don't know. In any case, this is in sync with the upcoming release of Call of Duty Modern Warfare 3, which initial support will focus on the English language with plans to include additional languages in the future.

5:11So, okay, that's interesting. It's only going to be in English, but still they're excluding Asia. Interesting. In any case, Mike Papas, who is the CEO at Modulate, expresses his excitement about the partnership, noting, We're enormously excited to team with Activision to push forward the cutting edge of trust and safety. This is a big step forward in supporting a player community the size and scale of Call of Duty, and further reinforces Activision's ongoing commitment to lead in this effort. so activision's anti-toxicity moderation systems have already flagged okay get this i i was pretty blown away when i first read this but their their moderation systems have already flagged over a million accounts for violating the call of duty code of conduct since they introduced it on call of duty modern warfare 2 so the company's um ricochet tech has also been a key player in kind of fighting against uh cheating in the game and other stuff but in any case specifically for their anti-toxicity moderation systems, I've already flagged a million accounts.

6:09This is so interesting to me. Like, this is a million users that are paying them for their product. They've flagged them with immediate penalties, banning their accounts, shutting off their speech availabilities, all sorts of things, right? It's just so interesting. You pay for something, and I guess the reason I bring this up is because it would appear that this is not just like a handful of bad actors it sounds like the toxicity parameters are set fairly high if they've got over a million people for this i'd be curious what the penalties are be curious uh those people's thoughts on this that's a lot of people like let's be honest a million people anyways although players can still report disruptive behavior tox mod aims to proactively catch violations that may otherwise go unreported.

7:01However, it's crucial to consider the broader implications of using AI for this purpose. My thought is, right, like, is this going to be flawless in identifying context, or is it going to stifle authentic conversations? And as AI-powered moderation becomes more, you know, pervasive, is this going to lead to a dystopian reality where our words are continuously judged and penalized, right? It's one thing in a video game, maybe there's a bunch of 12-year-old boys that are cussing, and you want your kid to go be able to play Call of Duty, without hearing his, the other people's toxic comments. Like maybe that's the one idea, but my concern is, does this move into the real world?

7:39Do you want your words, whether that's on social media, a live stream or your phone call or on Facebook messenger call or on FaceTime, do you want those words to be continuously judged and penalized for what you say? That's very, that is like a terribly dystopian outcome that I think we would really want to avoid. So I think for those concerned about privacy, the moderation system can be turned off in the settings, apparently. But this doesn't eliminate the overall question, which is, you know, as technology becomes increasingly intertwined with daily life, how much control should we surrender to algorithms for the sake of a, quote, better digital community?

8:14That's the question we are asking today.