

The episode argues that social media platforms are increasingly promoting “death on demand” content—graphic deaths and “snuff” clips—through algorithmic engagement, especially after guardrails were loosened. It also links this to AI-generated/AI-amplified misinformation around real-world events (e.g., riots in France), making it harder to trust what’s real.
violent content is rewarded because platforms optimize for clicks/reactions; automated moderation (AI) can misclassify content or wrongly suspend accounts; and normalization of graphic death harms younger users’ perceptions.
Charlie Kirk’s shooting clip (11M views in two hours), tiger mauling in Yorkshire, priest killed by falling cross in Ecuador, lightning-struck footballer, Christchurch livestream aftermath, and Nikki Katsura’s leaked accident photos.
Jeremy Vine (top commenter; online, joins discussion about AI/riots). Main hosts: Mariana and Matt (BBC “Top Comment”).
Written by AI. May contain mistakes.
Explore the dual narratives of protests in France and the impact of AI on public perception.
“But a lot of people have been saying we should do something on France.”
Discuss the alarming increase of disturbing content shared online and its implications.
“Because there is a certain quality of France that's so unbelievable sometimes when it's refracted to social media.”
Analyze the disturbing trend of snuff videos and the moral implications of sharing violent content.
“And what does it tell us about the shift within the social media companies and how they are recommending amplifying or not this kind of content?”
Examine historical instances of violence shared on social media and their consequences.
“It feels like there's been this gear switch.”
Investigate how online communities revel in sharing violent content and its impacts.
“the instantaneous mass distribution of them.”
Discuss the effects of violent content on children and the societal implications.
“And there used to be even subreddits like, you know, slash rwatchpeopledie that have been shut down, right?”
Discussing the disturbing trend of violent content being reshared online.
“And it's this hyper-reality point we make on Top Comment all the time.”
Exploring the free speech debate surrounding graphic content and its political significance.
“And by the time that you realise someone has died or this might be showing the moment of death or what have you, it's so viral they can't do anything about it.”
The importance of differentiating between politically significant and sensationalist death content.
“And I think the answer depends on whether it's something which we should be bearing witness to for a political reason or not.”
How social media companies have changed their approach to content moderation and responsibility.
“Because the live stream idea was only mooted and now it's been thrown out.”
The cultural implications of accepting violence as commonplace due to social media.
“We're just going to let people make their minds up about this content.”
Overview of social media policies regarding graphic violence and user experience.
“You can see someone killed, real people being killed by lightning strikes, by like falling to the floor of a giant stadium and so forth.”
Investigating the tension between social media moderation and political activism.
“It appears that this shutdown happened for around 90 minutes, but the account moderators were locked out for several hours after that.”
Exploring the complex relationship between Meta, Trump, and political moderation.
“Well, that is a question is like, is how MAGA is Facebook?”
The discussion explores how social media moderation impacts political movements and the role of AI in this process.
“Like they operate within their guidelines and so they're abiding by those.”
An examination of how AI-driven moderation struggles with nuanced content and the implications for free speech.
“increasingly reliant on ai and automated like it's cheaper it's quicker it's easier for them but it's not real people and the AI isn't always up to it.”
Discussion on the trade-offs between human moderation and AI efficiency, highlighting the need for human oversight.
“that have actually had the most effective moderation, obviously they are much smaller than these major companies, but let's talk about Reddit and let's talk about Pinterest.”

Get the gist, explore an idea or ask about something said.
On Top Comment, what are the key points of this episode?
Automatic transcript. May contain errors.
0:00This BBC podcast is supported by ads outside the UK.
0:30goals, or appetite for investing, Schwab has everything you need all in one place. Visit schwab.com to learn more. Running a business shouldn't feel like surviving a software group project. One app for accounting, another for inventory, another for sales. And somehow, none of them talk to each other. That's where Odoo comes in, an all-in-one business management software that brings every part of your business together, from sales and accounting to inventory and marketing, all in one powerful platform. No messy integrations, no bouncing between tabs, and best of all, no spreadsheets. Stop managing software and start managing your business with one unified system.
1:09Try for free today at odoo.com. That's O-D-O-O dot com. Matt, we've got a new top comment listener. People probably won't have heard of him, but he's got a question for us. Hi, Mariana. It's Jeremy. I'm at a railway station, so apologies if a train suddenly thunders by. Here's what's rage baited me though this week. So I'm looking obviously at the French story with a lot of concern of the smoke and the tear gas and the batons and whatnot and then the next wave appears of the story which is people jogging through the riots or people eating in a cafe while the riots go on outside and there are flames and they're smoking cigarettes in the cafe or two tourists arriving with suitcases and walking through the riot.
1:52And then the next thing I see is two old people who seem to have been smoked out of their house and they're on the street and they're both wearing their underwear and they're very old and they're in their thermals. And I think, my God, this chaos in France. And then somebody says, oh, no, that's AI. So suddenly then I track back and I think, well, if the old people are AI, maybe the people in the cafe are and the tourists are and the joggers are AI. Hang on a minute. Does France even exist? Now, for anyone who doesn't know, that was obviously Jeremy Vine joining the top commenters, loving the pod, mainly because he's very, he's actually very online, Jeremy is.
2:29But that is why he's struggling to navigate the content around the riots that have been happening in France, because there is a lot of it. In many ways, he's the perfect top comment listener, someone who is very active online, but can't quite parse reality from fiction. Welcome, everyone. message. But a lot of people have been saying we should do something on France. And of course, you know, it's something that we're all seeing a lot of on our social media. I think the most interesting thing is there's two versions of the protests. If you're in one left-wing silo, it's a left-wing protest led by students.
3:02If you're on another silo, it's evidence of immigration and Islam destroying Europe, which is a narrative we've now seen pushed by lots of right-wing leaders in the US and Europe. So I think that's very interesting. But amid all of that is a lot of AI content. The Pink Panther. Yeah, and the thing is, the Pink Panther was, I saw it and was convinced by it. I was like, oh, it's really, really good. It's convincing. It matches up with a lot of the other imagery you're seeing. It's not surprising that someone would dress up as a Pink Panther. And, you know, a lot of people have been romanticizing these riots.
3:33And like we should say, people's businesses have been looted and destroyed. And there were also protests that started off peacefully and then have obviously descended into this. Lots of people are making a point of just how French the whole thing is. And so stuff like Pink Panther, they're like, oh, this is so French. Like how French of them, like how chic of them. It's a red-way formula for virality. If you want to like just edit a baguette into someone's hand, then it meets people's idea of a French stereotype. And like, boom, you're off to the races. Here's your million followers and views.
4:02Yeah. Or like the picture of the couple who are having like a drink and there's like a fire in the background. That's real. Yeah, what is real? So that is real. But it's so easy, as per the question, it's so easy to be like, hang on, is this real? Isn't this real? And this is ultimately the real danger of AI, which is that we stop being able to trust anything anymore. I think Jeremy Vine's on to something when he's asking, is France even real? Because there is a certain quality of France that's so unbelievable sometimes when it's refracted to social media. Like, do you remember during the Louvre heist, there was the image...
4:34Which was a year ago, by the way. Really? One year to this day. during the Louvre heist there was that incredibly dapper like handsome detective looking guy that everyone was saying was a detective on the case he wasn't he was just like a random like well-dressed guy walking by but everyone was like this is AI and it wasn't he there really are men looking that good in Paris just daily while shit's going on anyway you're gonna start a new conspiracy theory instead of birds real it's like is France real it's France real no one's ever been there um Today, on Top Comment, Mariana, we're going to be talking about the return of snuff videos or trauma porn and how the Internet is now awash with images and clips of people dying in tragic and horrible ways.
5:17Yeah. And what does it tell us about the shift within the social media companies and how they are recommending amplifying or not this kind of content? And actually that connects to the other story we're going to talk about, which is all about a perceived overreach in moderation, a protest group in the US that had its page removed for a period of time. It led to lots of allegations of political censorship and accusations that Meta has got too involved. But actually, it seems like it's more about AI than it is about the people making the decisions.
5:55Matt, my feed over the past week, couple of weeks, has been overwhelmed with really traumatic content, what some people might call trauma porn, essentially clips that are depicting really horrible stuff and that are getting a lot of attention for it. There was that clip of the young man who was being attacked by a tiger in an enclosure in Yorkshire, a wildlife park, and then who died as a result of that attack. There was this live stream of a priest in Ecuador who gets hit by a cross that's falling and is killed. The footballer struck by lightning. There was also this guy performing at an American football game who like fell off a kind of trapeze type that I'm not quite sure.
6:36He was like he was rappelling down and he just plummeted straight. Yeah. And all of these clips are deeply disturbing. But I don't think my feed has been as a wash with this sort of stuff for a while. It certainly seems like what you're calling trauma porn, what I would call snuff videos, is back in a big way. And we're seeing more and more of it. And it comes at a strange time when the U.S. administration almost released its own live stream snuff video, arguably. At one point, Donald Trump approved the live streaming of the execution of Nadia Hassan, who killed 13 unarmed soldiers over a decade ago.
7:09But we're living in a very interesting kind of moral time where the kind of inherent value and dignity of human life and human death is being respected by neither the social media companies nor the sitting U.S. president, arguably. It feels like there's been this gear switch. I would say particularly since the assassination of Charlie Kirk, that video of him being shot was everywhere. Like I remember getting the message, Charlie Kirk's been shot at this event and just opening up my X feed. And the first thing I saw was this clip of him being shot. And it felt like the first time in a while that something so graphic and explicit had been recommended by the algorithms that I was able to just sort of watch it.
7:51Not even that I'd sort it out. It was being served to me. And it's something I get messages about a lot. People saying to me, oh, I had this thing pushed to my feed and I really didn't want to see it. And it was really graphic and it didn't have like a label on it. I mean, particularly X. And I would say X is crucial to this arc that we're going to talk top commenters through. Absolutely. And by the way, that clip, the exact moment of death, was viewed 11 million times. What, Charlie Kirk? Of Charlie Kirk. In the two hours in between the assassination and his death. He died two hours after the bullet hit.
8:22In that two hours, it was viewed 11 million times alone. That's just like insane numbers. Yeah. And there's something about, right, like we as human beings have always, throughout history been fascinated by, gripped by, horrified by very violent content. I mean, people used to go along to executions, right? People would watch gladiators fighting lions, all this kind of stuff. But what feels, I think, distinct about how this unfolds in the social media age is that the companies themselves are essentially sort of selling tickets to the Coliseum now. It's like, hey, come on down. And you're not necessarily going to have a choice about what you're seeing, you're just going to be pushed to this because everyone's engaging with it.
9:02Absolutely. Yeah. The appetite has always been there. It's arguably innate, right? It's arguably part of our evolutionary biology to be fascinated by this stuff. But as you said, what's changed is there's now a system in place for the recording of these horrible events and the mass distribution, the instantaneous mass distribution of them. That and we now live in a world where the US administration, you know, and prominent people like Donald Trump are also promoting them because it's not just the live execution. He also reposted the video of the man being mauled by a tiger in the zoo. And then shared an AI generated version of himself in the same scenario, while the family had very explicitly been saying all of this content is incredibly distressing for us at a time where our son brother has died.
9:46Now, the reason it's surprising that this content is making a comeback on social media in a big way is because there was this moment in 2019, just before the pandemic really, where there was a real reckoning, I would say, for the social media companies about violent content, especially during live streams. And it was a result of this mass shooting, this attack that happened in New Zealand in 2019, where a gunman at a mosque in New Zealand killed 51 people, injured dozens more, and they filmed the attack and live streamed it directly to Facebook. The GoPro attached to the rifle, I remember. Yeah, and it was incredibly disturbing footage.
10:23And this person was a very online shooter. They had engaged with all kinds of Islamophobic and far-right ideology and had shared this kind of stuff. And then they chose to carry out this killing, this attack on the mosque in a very sort of visceral, visual way. And there was widespread outcry that this had been live streamed, that people could see people they knew being killed, that anyone could see these people being killed on this live stream. And there was this real panic within the social media companies. And so they started to introduce not just Facebook, which is now Meta and owns Instagram and all the rest, WhatsApp, etc., but all of the social media platforms from that point onwards started to have kind of stricter rules around really explicit violent content, whether it be content that people are sharing that's been recorded elsewhere or whether it be live stream content, although that is obviously inherently harder to moderate.
11:19And it's a good thing they did do that because, of course, that Christchurch shooting that was live streamed did lead to copycat attacks. The shooter in Buffalo cited that as an inspiration. But I suppose the difference between that and these kind of snuff videos that we're seeing is that that's performed violence, whereas these are things that happen accidentally and are then shared. And I think there's two things going on here, right? There's the morbid curiosity that all humans have, which is completely natural. You shouldn't feel ashamed if you have that because it's innate. Everyone has it.
11:49Their question is, should it be pumped out to you en masse? But there's also a group on the internet who revel in these specific kinds of videos, who share them in a much more grotesque way, the kind of edgelord gore sharing communities that exist online. And these are the people who are making sure there's constantly like, you know, versions of these videos online, sharing them, reposting them frequently. And there used to be even subreddits like, you know, slash rwatchpeopledie that have been shut down, right? It reminds me of, I think, one of the earliest cases of this was in 2016 of Nikki Katsuras, who was an 18-year-old who died in a horrific car accident.
12:26and her body was so disfigured that the police didn't let the family come to identify the body because they didn't want to traumatize them. But the photos of her body were leaked. And for years, this horrible community on the internet was sending these photos to the family themselves, to the father. They were posting them, reposting them online. The family did everything they could. they tried injunctions, they tried lawsuits. There's nothing they can do. They eventually had to relocate because they were so traumatized. And I remember that that family was interviewed by Werner Herzog in one of his documentaries.
13:03They come to the conclusion that they see the internet as the antichrist. And I do think there is this very, for lack of a better word, evil part of the internet that revels in this kind of content. One of the saddest things about this stuff is that it's now being seen increasingly by children, right? And I remember going to my friend's house and her like 10-year-old little brother showed me a snuff video that his friends had sent him. And that's not like an anomaly. The latest Ofcom data shows that three quarters of 11 to 17-year-olds have seen harmful content. And we know from research that this causes aggression in children, it causes distress.
13:37But even in adults, right, there was research done with the Boston Marathon bombing, that the repeated exposure to the coverage of that event caused more acute stress than the people who had witnessed it directly. Yeah, and there's this group of people online who almost see violent content as a form of trolling, like the original definition of trolling, like they are using it to sort of like aggravate, get a reaction. And I noticed that particularly on X now that there are a huge number of accounts, I was messaging a few of them, who just relish in resharing the most horrible content. And it's this hyper-reality point we make on Top Comment all the time.
14:15But as though it's not real, like as though it is something funny and strange when you're like, no, this is a real person. Like all of that content we outlined at the very top of this episode. These are all real people with families who, like imagine having to watch a clip of someone that you love being killed. You can't think of anything worse, basically. Like it's so distressing. and I think that we need to talk about X because after Christchurch there were these policies that seemed pretty sort of strict really when it came to violent content especially when it came to the algorithmic amplification of that violent content and that content being rewarded and then when Musk took over Twitter and we've spoken about it loads and he got rid of lots of people who were working there and decided that a lot of the moderation the policies had been unfair, biased there are still, were still policies to stop that sort of stuff But it just felt like because you'd lost a huge number of people, like the guardrails came off, essentially.
15:11And because you don't have people proactively moderating that content in the same way they were before, it thrives. And by the time that you realise someone has died or this might be showing the moment of death or what have you, it's so viral they can't do anything about it. And so I think that the changes that Musk made in particular sort of set a new precedent for what was acceptable in terms of violent content online. And then we've seen across all of the social media companies, really, like Meta, for example, has also made changes around how proactively they get involved in moderation and so on and so forth.
15:40All of that stuff means that this kind of content is allowed to exist and then be promoted and engaged with in a way that it couldn't be before. And I think that's a really important thing for people to understand. It's not like the companies have gone, great, violent content performs really well. So let's like boost all of this horrible stuff for everyone to see. It's that they've created an algorithmic formula which rewards engagement and reaction. And if they don't put the brakes on early enough, that content isn't removed before it starts to get attention. Yeah. The Trump administration and its allies in social media will often cite this as a free speech argument.
16:15Right. But some might point out that the Trump administration and its allies are also looking to use the FCC to regulate and restrict content that mentions sexuality or gender dysphoria. So is the message that content discussing sexuality is more dangerous and more harmful than content in which you can watch someone die graphically? And I think that there is a free speech debate here, right? All these snuff eaters are becoming popular at the same time that we are exposed to a more significant form of death. You know, the death that's happening in Ukraine, in Gaza, for example. I mean, just the other day, we all saw an image of a child who died from a Russian strike in a push chair in Ukraine.
16:53And those kinds of videos and those kinds of images, one could argue there is a real free speech imperative in us being able to see that stuff. Especially when there's, for example, in Gaza, you know, restricted access for the press media from outside of Gaza to see what's going on. So actually social media has been this crucial portal to understanding what's happening on the ground when we can't otherwise, you know, look for ourselves as journalists. Exactly. So it's like, should you be allowed to and should you watch a video of a man being mauled by a tiger, right? And I think the answer depends on whether it's something which we should be bearing witness to for a political reason or not.
17:31If it's a man who's been killed by a crucifix falling or it's an accidental death, that death is almost meaningless except for it being morbidly curious to us, right? And if you believe that humans have like inherent value and dignity, then you shouldn't really watch that. You shouldn't share it and social media companies shouldn't push it. Some deaths, though, some snuff films or videos of people dying do have political potency. They have a reason, right? That's why it was considered important to share images of the Holocaust victims. I mean, George Floyd, that is a video of someone dying, which had a huge political importance to people.
18:02Many people thought that we should watch that, right? And restricting that kind of content would potentially restrict that kind of content too. But it's really important that we distinguish between videos and images of death that have that kind of political weight to them and the ones that are literally just some kind of morbid entertainment, which the social media companies do not do. The Trump administration is clearly aware of this potential for these images of death to have political potency because that's reportedly why the White House didn't live stream the execution of Nadeel Hassan because they were worried he would become a martyr.
18:32Because the live stream idea was only mooted and now it's been thrown out. And, you know, there was quite a lot of reporting that there was discomfort from people like J.D. Vance as well, who felt like this was not a good idea and not something worth doing. Because we never got as far as knowing how it would have worked. My immediate question was, oh, is this going to be live streamed on the social media platform? Would that be allowed? Well, exactly. So at the moment, technically no, but you could see a world where Elon Musk says, yeah, okay, I'll, you know, he's been very supportive of the Trump administration, says, okay, yeah, I'll share it.
19:00Or what happens is it gets shared on a third party platform, like a website, say. USgov.execution. Yeah, yeah, exactly. And then people clip it up. And then obviously it would be shared everywhere. Like once this stuff exists on social media, like once something is filmed, once something is shared, you cannot put a lid on it, this kind of content. And that was one of the issues they really grappled with when it came to that Christchurch shooting, because it wasn't just the live stream. It was that people were then resharing the content of the live stream everywhere. And it was incredibly hard to shut the whole thing down.
19:29When it comes to the social media companies, I think if we look at where they are now versus where they were when it came to Christchurch, I think the companies themselves shifted away from what was their original kind of like motto of like, let's do good for the world. We're great. Woo, cool. To like, oh, actually kind of everyone hates us. And we're in a bit of a lose-lose situation here. So like, who cares? We're not going to be the moral arbiters. We don't want to be the moral arbiters. We're just going to let people make their minds up about this content. And we're going to design a platform that rewards content that people engage with.
20:00And that's it. And we step away. And there are all kinds of issues with that. And a lot of the litigation that's happening in the US and possibly going to happen in other places focuses on the fundamental flaw of like, Your design is actually making moral judgments. It can't be amoral. And then the normalization point, and I think this is why it matters on a society level. I don't think it's necessarily that we all become more violent. It's just that we accept violence as part of our culture in a way that it's almost like we have for the past, particularly a couple of centuries, been trying to tame the human condition so that we actually don't all kill each other all the time or do bad things to one another.
20:39And it feels like social media is like unpicking that progress to some extent because of the way that it amplifies and rewards this sort of violent content. Yeah, there has to be a line. There has to be some dignity afforded to people who die. There has to be the widespread acceptance that death is a bad thing. This numbs us to that, right? Death is just there. It's everywhere for you to see. You can see someone killed, real people being killed by lightning strikes, by like falling to the floor of a giant stadium and so forth. and it reminds me of this kind of quote from the social reckoning which we talked about in our two episodes ago which i'm not sure if mark zuckerberg actually said this but he says we are a cork board his character in the film says this we don't control what people put on we just put the cork board up first of all a lot more people see the stuff on social media than they would on a cork board second of all if i put a cork board up i probably would be annoyed if someone put a bunch of snuff images on it i'd be like this get them off my cork board but this is that this is the mirror argument we are a mirror reflecting society back to you like we can't help that human beings like violence or engage with it and it's like that would be all well and good if people were seeking this stuff out for themselves but when you are actively pushing this to people and you are actively rewarding it and you get to this point where people are dehumanizing the victims of violence and it becomes like a joke i mean i think that blows their argument out of the water because their algorithms ultimately are not static they recommend stuff but also it's when we should all start to be worried because it I think particularly for younger people can have a huge impact on their perception of what's okay and not okay and can affect the way that we all cohesively exist which is kind of essential.
22:16Now in terms of the policies the companies do have when it comes to these sorts of posts X's violent content policy says you may share graphic media if it is properly labeled not prominently displayed and is not excessively gory or depicting sexual violence but explicitly threatening inciting glorifying or expressing desire for violence is not allowed and meta's policy on this is we understand that people have different sensitivities with regards to graphic and violent imagery to protect users from such content we remove the most graphic content and add warning labels to other types of content so that people are aware that it may be sensitive before they click through we restrict the ability for younger users to see content that may not be suitable or age appropriate for them by doing so we aim to provide an appropriate user experience while continuing to provide space for our users to express themselves.
23:06On the topic of moderation, social media companies recently came under fire for over-moderating. So the No Kings Coalition group, which is a protest group that is protesting Donald Trump, his administration, his deportation raids, has said that Meta shut down its Instagram account during a peak moment during their planned protest. It appears that this shutdown happened for around 90 minutes, but the account moderators were locked out for several hours after that. The No Kings Coalition is very angry about this. A lot of Democrats are very angry about this. Meta, for their part, have apologized and said that this was due to their automated moderating system.
23:46This message is for Meta. Why was the No Kings Coalition account on Instagram down for almost two hours today, which just happens to have been a national mobilization day for voting early? Meta, we see you. We see what y 'all are doing. We are not going to stand idly by and be silent. I'm not a liberal. I think these protests are ineffective and waste everybody's time. However, it really seems to me like this is a blatant act of censorship. I don't know what else to call it. Now, we don't know why Meta did what they did. But for something like this to happen with an account that has a massive following and a call to action, they need to explain why it happened.
24:26And the reason that the protest group are particularly concerned about this is because the midterms are coming up in the US. And so it's a key time for activists, protesters, anyone that wants to voice their political opinions to do so. And I think all of this is in the context, right, of Donald Trump and his administration in particular being real freedom of speech purists, essentially. Like they believe that they say they believe that freedom of expression is is essential and paramount and everybody should be able to say what they want. And yet then there are examples like this, which obviously this isn't about the administration itself.
24:56This is about the social media companies. But a lot of the cries about, hang on, is this a political decision? Why have you shut down our account? All of that kinds of stuff derives from what seems to be a fairly amicable relationship that exists between the major social media companies and Donald Trump, who has, you know, frequently expressed his support for and been seen with Elon Musk, Mark Zuckerberg. I mean, in the defense of these social media companies, they, from their point of view, whether they agree with his politics or not, they do sort of need him on side if they want to succeed in their various goals of making lots of money.
25:30Well, that is a question is like, is how MAGA is Facebook? Are they just playing the game of appeasing the party who's in power? And I think that's where a lot of this comes back to, right, is a lot of people see this on the left and Democrats as a quite simple story of Mark Zuckerberg is now MAGA line and he's doing Trump's bidding. The truth is more nuanced than that, but let's get to the actual story. The initial reason given by Meta with their apology was that this was due to the posting activity primarily of this account, which triggered their automated system as potential bot activity. But then the No Kings Coalition said, but we weren't really posting very much at that time.
26:10The explanation then kind of changed. It then said it was actually due to this account, its reading activity, the amount of other accounts and posts that this account was reading and that it was actually using a Claude-based agent to scan other posts. Perhaps it was using a Claude-based agent to maybe scan for mentions of no kings and kind of assemble them into a report for the runners of this account. We don't know exactly what it is. And their automated system in turn picked up the Claude agent as bot activity and shut the account down. That's what they're saying, which is different, right? Now, the argument that this could have been some kind of pro-MAGA activity on the part of Facebook, I don't think holds any water.
26:50I can see what people might think that. There's some precedent for like the policy team at Facebook. And let's be clear, their board and their C-suite now looks very Republican. You've got Dana White, the CEO of UFC in the long-term Trump alley. You've got Marc Andreessen, the venture capitalist who started Andreessen Horowitz and supported Trump. Nick Clegg was replaced by Joel Kaplan, who's a Republican, as their kind of public policy person. There have been cases where the policy department has fed into the moderation department. For example, in 2020, Joel Kaplan requested bans and penalties being lifted on certain right wing accounts like Breitbart or like Charlie Kirk.
27:31And we also know that during the Jan the 6th riots, for example, like lots of those decisions did have to be escalated higher, which you would expect in those situations. Like, for example, the decision about whether to keep Donald Trump's account on Facebook or whether to suspend it. Ultimately, they chose to suspend it. So there are examples of where decisions are made that do feel sort of inherently political. But Meta would argue, Zuckerberg would argue on the basis of their rules, right? Like they operate within their guidelines and so they're abiding by those. But this situation doesn't seem like one of those.
28:06I mean, to suggest that like Mark Zuckerberg or whoever has decided to like personally get involved in a specific protest group feels like a misunderstanding of how moderation now works on these platforms. Yeah, absolutely. I mean, look, on one hand, we wouldn't know, right? Because the whole thing about Meta is that they are a closed book, right? Mark Zuckerberg has a majority of the shares. We can't look at the decisions that the company's making, but would it really be that much of like a win for the Trump administration for Meta to shut down this account for 90 minutes? you know why so short of a time also i don't think that zuckerberg himself is like maga aligned i think he would probably describe himself as kind of you know a a liberal or libertarian and and a lot of tech companies like this they just try and appease the party that is in power at that time and that could be an explanation as to why he's changed the look of his board and his c-suite they're already also funding um some democrat non-profits in the run up to the midterm so they may also be hedging their bets.
29:07Yeah, and there is an argument to be made that obviously by removing a group's account that has a significant reach, that can have an impact on democracy, essentially, not a massive impact, but a small impact in that it does inhibit the organisation of those protests, the posting of content. And if that were to happen to a right-wing group that organises protests, there would be similar outcry. I think though also part of the confusion around this and suspicion around this is because meta itself changed not that they changed their story necessarily but rather that they didn't qualify their initial explanation and then when they qualified it it was essentially like look our automated systems ai kind of detected it as bot activity and that is the real story here right yeah this is about how moderation is becoming increasingly reliant on ai and automated like it's cheaper it's quicker it's easier for them but it's not real people and the AI isn't always up to it.
30:01And that's where we get examples of where things are removed, not with a political explanation, but because the AI, it can't make the same decisions that a human being would right now when it comes to particularly content that perhaps sits in a gray area of like, oh, is this allowed, not allowed? Or like here, detecting that there's a very high volume of posts and not being able to think, oh, actually, No Kings is this protest movement and they obviously share loads of things. It doesn't mean it's a bot. Like it can't do that right now. And look, I'm seeing the exact same thing all over social media.
30:28about TikTok initially shutting down the account that was advertising the Musk documentary film, which is obviously heavily critical of Musk. People on social media, you've got to stop just immediately assuming that this is all a politically motivated conspiratorial element of subterfuge going on. This is a story about these social media companies increasingly relying on automated moderating systems, which is important, right? That's also important in its own way. Now, the reason that this is getting so much traction the Elon Musk story on TikTok is because there has been a decision by other social media companies not to run ads for that film because of their view that it amounts to political advertising, which they don't do post Cambridge Analytica.
31:13And increasingly, our political discourse is going to be defined by one or another political party getting angry at social media companies for censoring or not censoring a certain bit of content, which just shows you how much power these companies have. But one thing that Senator Chris Murphy said, who is a Democrat, which I think resonates, is does it matter whether it was intentional or not? Does it matter whether this account was using bots to increase its traffic? What matters here is arguably that a social media company has the power to unilaterally shut down the main communication channel of a political movement at a crucial moment of national action.
31:52And, you know, whether they like it or not, They have become these kind of gatekeepers of so much of, you know, how politics works and what we see and how we view violence and how we see the world and all of those things. And yet they spend their entire time being like, we don't want to be the gatekeepers, but also like maybe sometimes we do, but also we don't. And it's that inconsistency that I think causes so much confusion. What I would just say on the automated moderation point, which is there are benefits, right, to using AI. it does mean that certain types of content, let's say like child sexual exploitation posts, like they could be picked up very quickly and removed very quickly and flagged very quickly.
32:27There are quite a lot of benefits to that. But the issue is still that we see mislabeling. We spoke on another episode of Top Comment about the Cornell 7 case, these gang rape allegations that had been made and people having their posts labeled because they contain the word rape, for example. And again, that's like where AI has been trained to spot certain words and to flag certain types of content, but it's not a human being. And if we look at the sites that have actually had the most effective moderation, obviously they are much smaller than these major companies, but let's talk about Reddit and let's talk about Pinterest.
32:59Their moderation system is proactive and very human. And actually like Reddit is basically run by people like themselves being like, we're going to take responsibility for like this place being a like good place, not a bad place. Serious question. Who actually can work Reddit? Because I keep getting banned on everything. Every time I post, it's like, moderator this, moderator that. Reddit can be so fucking useless sometimes. Why the fuck am I posting in the subreddit and I never can get a response because the moderators are always so fast to take the shit down for some rules. It doesn't even make any fucking sense.
33:33I posted on the r slash cooking subreddit because I didn't know what oil to use. A recipe I was making called for oil. What oil? And I asked it and not even two minutes later, they took it down saying, well, actually, that violates rule number 34, section C. if you didn't read the rules. I'm sorry my cooking question bothered you so much in the cooking subreddit. After all that, I still don't know what fucking oil to use. Yeah, of course, there's geeks. There's people that take things too seriously on Reddit. But at the same time, it's like, that's what gives it its charm, man. We're in this really rocky patch where until AI improves even more and the companies are making savings and cutting people and replacing them with AI, we're going to keep seeing examples like this that ultimately do impact and influence the world because of the decisions they're making about AI rather than it being with political intent.
34:23Exactly. Because when you take human moderation out of the equation, for all the cleverness of these algorithms, it essentially comes down to one decision that the company makes. Do we want to make these algorithms narrow or do we want to make them wide? If we make them wide, then they get all the bad stuff, but they also destroy a little bit of free speech. If we make them narrow, the free speech is protected, but then a lot of the harmful stuff gets through, right? You need humans to not have that, right? But why don't they want the humans? Because it's expensive. And this goes back to the thing I first said to you on that panel in Edinburgh like two years ago when we decided to do this podcast.
34:55If they're making like billions and trillions of dollars off of our collective suffering and attention, then they can spend a little bit, just a little bit, preventing the destruction of our entire society as well. Just like one pound. Just like one dollar. Just like a dime. okay if like Jeremy Vine you have a question about what is popping up on your social media feeds AI generated or not maybe you have had a post that's been labelled taken down it's bothered you get in touch our contact details are plus 44 330 123 9480 if you want to send us a WhatsApp or you can email topcomment at bbc.co.uk we actually threw Laurie under such we were like please contact Laurie I'm a producer
35:44Greetings malevolent munchkins, fiendish friends and devilish do-gooders. Welcome back to the home of the oxymoron. Evil genius! I'm Russell Cain and I'm delighted to be steering the ship that unceremoniously wrenches historic figures from their perfect pedestals so that we can decide whether they're evil, genius or a heavy concoction of the two. It's like the podcast version of telling your kids the ice cream van plays music when it's out of ice cream. Yes, it's evil. Yes, it's genius. Get on board now and listen to Evil Genius on BBC Sounds.
36:24Sometimes we all need an extra helping hand, especially with a tricky business problem you've been putting off. Claude is the AI for just that. It's the collaborator that understands your entire workflow and thinks with you, not for you. And Claude isn't just for quick questions. Hand it a whole task, like turning a folder of scattered notes into a report, and it does the work and hands back something finished that you can open and edit. Claude can now draft a dock or deck right in front of you, and Claude Design now lives right inside Claude. Describe the website or prototype you're picturing, attach your design system, and it comes back in your own design language ready to edit right on the canvas.
37:04It works with your files and the tools you already use, like Gmail and Slack. And whether you're debugging code at midnight, building a financial model, or strategizing your next business move, Claude extends your thinking to tackle the problems that matter. For problems worth solving, get started with Claude at claude.ai.solve. That's claude.ai.solve. And check out Claude Pro, which includes access to all of the features mentioned in today's episode. Claude.ai.solve
Why are we seeing more death on our feeds? From footage of an Ecuadorean priest being killed by a crucifix, to the man who was attacked by a tiger on camera, snuff content appears to be making a comeback. On this episode, Marianna and Matt examine how various platforms' responses to so-called trauma porn have evolved since an initial tightening of restrictions following the livestreamed Christchurch shootings in 2019.
Also this week, Meta has been forced to explain the deactivation of an activist Instagram account by pointing the finger at its auto-moderation systems. The company was responding to anger from No Kings protest organisers, who cited their official profile going dark as evidence of censorship and political interference. We dissect Meta's defence, and consider the broader implications of AI-led moderation becoming more widespread.
Plus - is France actually fake?
Top Comment is hosted by Marianna Spring and Matt Shea. The series producer is Laurie Kalus. The assistant producer is Sophie Millward. The technical producer was Ben Andrews. The editor is Justine Lang. The senior news editor is Sam Bonham.
If you want to get in touch about something that's popped up on your feed, our email address is [email protected] or you can send us a WhatsApp on +44 330 123 9480.