Whistleblowers, dolphin memes, and a million dollar toilet

0:00This BBC podcast is supported by ads outside the UK.

0:30to bbc.com and the BBC app. Find out more at bbc.com slash unlimited. So before we talk about what we're going to talk about, I want to dive headfirst into this toilet. And by that I mean there's basically this million-dollar toilet that's appeared on social media. This man called Nick Greenewald is selling projector-mapped ad space on his toilet. And it's like a really clean toilet, isn't it? It's like the sparkliest toilet. It's funny that you've picked up on that specific detail of the story. But yes, it is a very clean toilet. Because that's why it looks quite good, because it's got like the projected brands on the toilet.

1:07And it's like, whoa, this is a shiny toilet. My takeaway wasn't like how clean is a toilet. My takeaway is, wow, a guy's selling advertising space on his toilet and making loads of money doing it. Selling ad space on my toilet until I'm a millionaire. Part one. Here's how it works. I have three tiers of pricing. He's trying to make a million dollars. And I think this represents the entire internet where you kind of do something that's so ridiculous and create such shock value is so gross and then use that to sell stuff. Like when you think about Clavicular, HS Tiki-Toki, Andrew Tate. Like the Manosphere influencers.

1:43Yeah, all these Manosphere influencers. They're all kind of shocking influencers who are used by crypto companies, gambling companies, streaming companies, affiliate marketing schemes to promote their products. they are essentially just like toilets which advertisements are being projected onto. I'm sure they would love that analogy. And social media is essentially turning into one giant toilet with advertisements projected onto it. It's a toilet across the board really. Yeah. Very true. So what are we going to talk about today? We're actually not going to talk about toilets but we are going to talk about I guess in some ways what makes the toilets so viral which is the algorithms like the way that the recommendation systems promote certain stuff and I've been speaking to whistleblowers and insiders from the big companies about what's really going on behind closed doors.

2:27And also a new website called Your AI Slop Bores Me, which is going viral, has got us all to think about the so-called meat layer of AI, the bits of AI that are really human. So we're going to talk about Maltbook and we're going to talk about human fleet operators. And if you don't know what those things mean, you will by the end of the episode. and if you don't know about Gerald the Dolphin you will also know all about him by the end of the episode I still don't know about him I know far too much about him if you've seen him in his yellow hard hat then this is the podcast for you

3:08so you've been investigating how algorithms really work and you've been speaking to insiders and whistleblowers from all the major social media companies and you've made an amazing film inside the Rage Machine, which everyone should watch. And what I found so good about this film is it starts with the race riots in Sri Lanka, takes you forward to the Southport riots here, and you speak to people actually inside these social media companies who tell you how their algorithms may have helped contribute to those events. Yeah, and I think that one of the things that really struck me about doing this investigation was, in some ways, certainly you and I know, and I'm sure a lot of people listening know, that social media algorithms appear to be causing a lot of polarisation, a lot of division.

3:51These are the recommendation systems that essentially thrive on outrage, emotions, anger, etc. But what this investigation and documentary allowed me to do was understand what the companies knew and know about that and then how some of them made decisions that made it worse, even though they knew that their systems were contributing to harm in some way. I spoke to more than a dozen whistleblowers, insiders from across all of the major companies. I focused quite a lot on Meta and on TikTok because a lot of this is about what I've sort of coined as the algorithmic arms race. Yes, the algorithmic arms race, which it seems really ramped up when TikTok came onto the scene.

4:31Yeah. Yeah. So if you think back to when you were talking about like Sri Lanka and then other kind of violence that happened in other bits of the world, People might remember the role that Facebook played in violence that happened in Ethiopia and in Myanmar and places like that. At that time, and that's kind of pre-COVID, there was a quite widespread recognition that the companies had played a role and they took a bit of responsibility. Like Mark Zuckerberg sat in Congress and said, we're really sorry. You know, we didn't quite get this right. We've rolled out these features into these new markets.

5:00We're going to do better. Then the pandemic happens. There's quite a lot of pressure on the companies to deal with disinformation, conspiracy theories, hate, what we often call borderline content, like the stuff that's kind of harmful but legal. And then TikTok, kind of at the same time as COVID, bursts onto the scene, kind of takes everyone by storm, particularly because everyone's at home anyways, scrolling, you can watch short form videos so, so, so quickly. And what almost all of the insiders and whistleblowers I spoke to described was how that moment was just pivotal in terms of the companies deciding to take risks in terms of user safety so that they could compete with one another and make the product that was the most popular, that kept people hooked, that got them to engage because engagement equals money.

5:42It means you can sell ads. One of the things that was really interesting was being able to see a lot of the documentation. So for example, from Meta, one of the insiders I spoke to, a guy called Matt Motil, he was a senior researcher there and he effectively experimented on millions of users. So they would test out new features, but you didn't know that these features were being tested on you. and it meant that he had access to loads of different research that told you about new features and the impact they were having and everything else and one of the most striking bits of research that I was able to look at was that when they launched Reels to compete with TikTok so Instagram Reels was launched not long after TikTok became really popular they had all this research that showed that there'd been an uptick in harmful content particularly harmful comments on Reels like bullying and harassment violence and incitement and according to Matt this was basically because the attitude was move fast and break things, but it just meant that there was...

6:35Which is Meta's company slogan. Yeah. And therefore they just did not build the safeguards for this new product in the way that, in his view, he thinks they should have. Anytime you're introducing a new type of product, say reels, there's an elevated risk because the infrastructure that existed before, it's either completely absent or it's very immature. So it's hard to prepare sufficiently in advance of that launch. I was frustrated because there's a common trade-off between like protecting people from harmful content and engagement. Yeah, so TikTok really changed the game. And would you say that in the rush to make engaging content and to compete with those social medias to be as engaging as possible, to compete for young people's attention, that that safety and the kind of regulation of the content went by the wayside?

7:27It kind of feels like you are perpetually gaslit by the social media companies. Like when you put allegations to them, they send stuff back to you that seems to contradict the evidence that you're showing to them. But it's often quite hard because the evidence that you're presenting to them is the impact on users. So, you know, how they've been pushed harmful content or the way that it's affected them. Whereas to be able to sit with someone who works there and look at their computer and look at their dashboard and see, so this is someone who worked in the area that's called trust and safety. So dealing with cases that were being escalated around user safety, to be able to sit with them and go through the dashboard.

8:00And I did this over like many, many months and spent loads of time with them. And I really got to know them as well, which is kind of all part of that whistleblowing process. People do this at great risk to themselves. They may never work again in their industry. Yeah. And also, you know, the social media companies, you know, they have a reputation for being quite ferocious if they want to, legally or otherwise, with people who do leak stuff. Is there kind of like an omerta in Silicon Valley about speaking to the media about what really goes on in these algorithms? I think that interestingly, it used to feel like almost like it was a bit of a unicorn.

8:33It would be so hard to get someone to blow the whistle on this stuff or to talk to you. And actually often they would they would almost have to decide they wanted to have like a career as a whistleblower. Like you think of Frances Haugen, who quite famously came forward and spoke about Meta. Like she's built an entire career about online safety and campaigning on that front. It felt quite hard to get people who don't want to be public facing campaigners to leak and to speak in this way. But I feel like the tide is turning a bit. And I think some of that is to do with this engagement arms race and the fact that people within the companies were like, whoa, whoa, whoa, whoa, hang on a second.

9:04We know there's a problem here and we are making active decisions that are making it worse. OK, so what did this TikTok whistleblower actually show you on his computer? So on his computer, there was a dashboard of all these cases and you'd see how they were prioritised according to which ones you as a member of staff have to deal with first. So P1 means like top priority, P2 means second top priority and so on and so forth. And what they showed me over several months was how there were cases that related to politicians, say. So there was one about a politician who was in Iraq and was saying that it had been compared to a chicken.

9:38You know, like obviously not great. None of us want to be compared to a chicken, but not like immediately dangerous or harmful. And that had been ranked as P1. So top priority. But that's also, that is free speech. We should be able to call politicians chickens if we want to. So this is what this whistleblower said. they were like when i saw this i thought why like why is this top priority and then there were some other cases that i was able to look at so for example there was one involving a 16 year old girl in iraq so the same the same country and she'd had images of herself shared alongside sexualized images purporting to be of her so it was like someone was saying these were her nudes type thing and that was priority two not priority one so that was lower priority than this politician obviously that's not to say that this is the case across the board or even that there's a directive from above but ultimately what the evidence demonstrated and demonstrates is that there are some cases involving politicians that are prioritized over harm to kids the whistleblower felt uncomfortable about this they say that they raised it and their view was that this was about trying to stop tiktok being banned or regulated in certain regions because remember tiktok has particularly because bite dance owns tiktok which is a chinese company has had quite a lot of kind a threat of regulation over data stuff and other things, probably more so than any of the other American companies.

10:56So this whistleblower is in some ways sort of hinting that the reason TikTok may prioritise content that targets politicians over harm to young people is that it will help them with their regulatory battle. Yeah, essentially. So their view is that these decisions were political and ultimately financial, because obviously, if you're banned in certain regions and countries, then you can't make as much money. And that's a problem. So I suppose when I've spoken to people who work at social media companies, they often speak about the difficulty of policing the content in that if they kind of widen their restrictive algorithm, it will catch more of the dangerous content, but it will also catch things that technically are satire or free speech or whatever.

11:40But if they tighten it, then they miss a lot of the dangerous content, but that allows the free speech and the satire to come through. And they would say that they need to rely on these restrictive algorithms because there's not enough human beings in the workforce to police what is essentially a global, like billions of people are posting every day. How true is that? I mean, there are solutions that exist, like more proactive moderation or looking at stuff before it's shared. I think the fundamental thing, the thing that kept coming up time and time again was most people agree that you can allow this stuff to exist on the platform.

12:10The problem is, is when it's being amplified to a huge number of people. Like in some ways, the really illegal stuff, that goes. And there was an engineer I spoke to, a guy called Rufan Ding. So he used to work at YouTube and then he worked at TikTok. He left a couple of years ago. And he was describing like working on these algorithms and essentially being asked to optimize for engagement. And what that meant was the borderline content. So the gray zone stuff, often that would include like conspiracy theory content or misogyny. He started to notice an uptick and an increase in that. And what I think was so interesting was this feeling that, hang on a second, are they actually in control of any of this?

12:46And this guy was like a proper engineer, data scientist, knows what he's talking about. And this is what he had to say when I asked him, can you basically make algorithms safe? I don't actually know the answer to that question about how to, you know, like build a completely safe one. We have no control on the deep learning algorithm. To us, it's still like a very black box how internally it works. Which also is, in and of itself, slightly terrifying because you clearly understand algorithms probably better than anyone we've spoken to. To be honest, we don't actually pay too much attention to specific contents.

13:25To us, all the content is just a different number. Okay, we, the one that's responsible for the recommendation and the content safety team, they're responsible for eliminating those bad contents. Like the car manufacturer, right? There's a team that are responsible for the acceleration, the engine, right? So we expect the team working on the braking system was doing a good job. That's so interesting, the idea that even the incredibly intelligent, highly paid people who've studied computer science at the top universities in the world do not themselves know how these algorithms work. And in order to understand how these algorithms are a black box, you can understand what they're doing that's so complicated.

14:07What is it about algorithms that make them so hard to interpret? Yeah, I think it's helpful to think of algorithms as reactive systems. There's a code essentially that is responding to stimuli and then pushing you content based on what you are reacting to and engaging with. But it's almost this idea that the algorithms are blind. And to some extent, therefore, the people who are building those algorithmic systems are blind to the specific content. It's like it's kind of behind closed doors almost. So you've got like post A, B and C. And then it notices that you really like C. So it's like, great, C leads us to D and then D leads us to E.

14:40So the algorithm knows how I react to content and how I might react to other content, but it doesn't know much about the content itself. Yeah, exactly. And so in an ideal world, you'd build the algorithms in such a way that then you could say, actually, we don't want to go anywhere near like type G content or type H content. but actually in reality and this is what all of the insiders were saying when content exists on the platform unless it's removed or moderated on the whole the algorithms are kind of have free to just promote any of it like if it's there then the algorithms will be like great someone likes this and there are so many different users with so many different interests there will often be content that like some people would find problematic but other people would think oh i quite want to see that and that's where the argument comes from about kind of more proactive moderation or actually more transparency so one of the people i spoke to is this guy brandon silverman And he built this platform called CrowdTangle.

15:29It basically was brilliant. It let you see what was going viral on Facebook. CrowdTangle was acquired by Meta in 2016. And then they ended up getting rid of it. And Brandon left. But what he said was that he really noticed that change, that shift away from a kind of desire for transparency and to build safer algorithms. And instead that focus on kind of optimizing engagement, whatever the cost, and also adapting to the political wins at play. It generally felt like leadership wasn't sure what the answer was, but it began to calcify into a sort of defensiveness. We're not responsible for all of polarization in society.

16:07Nobody is saying you're responsible for all polarization. We're just saying you contribute to it and probably in ways where like you don't have to. And if you just made a few changes, you might not contribute to it as much. And yeah, I think that was dispiriting because it felt like there was a window in which it was like genuinely introspective. Obviously, I put all of these allegations to the social media companies in question. Meta said that any suggestion that they deliberately amplify harmful content for financial gain is wrong. And TikTok said that these were fabricated claims and that the company invested in technology that prevented harmful content from ever being viewed.

16:42They also said that in terms of the whistleblowers' specific claims about the moderation systems, they rejected the idea that political content is prioritized over the safety of young people, said the claim, quote, fundamentally misrepresents the way their moderation systems operate. I think this also comes a very interesting time because recently Elon Musk released the X algorithm. Which was what they sent me, by the way. When I write I've replied them, they just sent me the source code for their algorithm. I wonder if that's in part because there's a big lawsuit happening now between Ashley St.

17:17Clair, the mother of one of Elon Musk's children, who is suing ex-AI. And as part of that lawsuit, the algorithm is coming up and the extent to which it is being controlled by Elon Musk, the extent to which it's fair to people who use it. But to his credit, he has released that. And it's become clear that things that provoke rage do better than things that don't. And I think that's what feels like the real shift recently is that everyone is acknowledging how the algorithmic systems and the way the platforms are built are actually at the heart of this issue. And lots of the other things that we point out, like the other harms, are just all products of this same thing.

17:55And ultimately, until there's some kind of change in either the safeguards or the way the systems are built or the way they work, none of these problems are going to improve. And in fact, the whole rage thing feels sort of more extreme than it ever has. Yeah. And it feels like we are at the crest of a kind of growing tide of concern about these algorithms, about social media companies. There's a landmark case in the US on which a verdict is expected soon about the harm caused by social media companies. What's really interesting about the premise of that case is they're focusing on platform design, not content.

18:27How these platforms have been specifically engineered to keep, well, mainly in this case, young people addicted to their phones. And there's so many little nefarious things that are coming out. Like, you know, when you scroll down on your email app and it kind of bounces back up with new stuff. Yeah. That comes from slot machines and like people who design slot machines who've worked at social media companies. So it's all designed to keep you hooked as long as possible. So we're going to talk about all that in the coming weeks.

18:58so this website's been launched called your ai slop bores me it has roughly a million unique visitors 25 000 hardcore fans on its discord and what it is is it's basically a website where you can steal ai's job so you put a prompt in you say draw me a picture of a cat or explain this thing to me or whatever it is people use large language learning models for and then a human pretending to be ai will like draw the cat for you right and to gain tokens that you can make requests to ai for you have to pretend to be the ai first you get a token for acting as the ai which you can then spend on asking fake ai to accomplish a task for you okay so it's kind of reminds me of like club penguin or neopets oh my god you were on neopets i was on the i was i was like my first That's the height of the internet, I think.

19:45Yeah, that was so good. What was your, did you have pet pets for your pets? Yeah, I definitely did. I can't remember. The thing that's quite funny about Neopets is like, it was completely two-dimensional. Like, it's like, almost like the anti-AiS slot, wasn't it? Yeah, but it felt like a little world. It was so good. You were in their world. But it was actually just like a 2D page. But it was very pay-to-play, because I remember as a young person, I was like, why can't I unlock all the like, cool furry pets? And it's because you need to pay like loads of money. Yeah, that's why it reminds me of that.

20:10Anyway, so this AI slot bores me thing. this is sort of it's being built as a kind of fight back of humans against ai yeah do you think it's going to work do you know what i think there's something quite interesting about the kind of retaliation of social media users against ai because like now when i go on any of my particularly short-form video feeds so like on tiktok or reels there was just so much ai slop and like some of it's quite i mean we've spoken about the various genres of like fruit content recently baby fruits fruits having affairs um by the way imagine if someone from like 2010 was listening to this they'd be like what is going on yeah exactly but you start to get a bit like oh i don't actually want like there's a couple of times i've actually done the kind of like not interested because i'm like i don't just want a load of ai slop it's part of a retaliation against ai slop but i think ultimately it's cute and it's charming but it's a bit kind of like the last stand at the alamo it's like it's like a tiny number of humans fighting against like a giant much bigger army and they all just get slaughtered in the end because humans can't create pictures of cats that are high quality at scale instantaneously and and never will be and so that's kind of what that yeah ultimately proves and actually also if you think how accessible as we've talked about before but a lot of the ai technology has become yeah it's just so easy like it's incredibly easy now to input a couple of prompts and you just get something that looks so sleek and professional and amazing and obviously like a human being drawing whatever is not going to be as good exactly but this has made me think about the kind of what's being called the meat layer of ai so there's this question about how much of ai is actually humans how much does ai rely on humans rather than replace humans and there are a couple examples that come to mind first of all there's rent a human which is a website where ai agents can actually hire a human being to do something that it can't do so like deliver a package or whatever most people are used to ai in a kind of llm large language model sense where they're talking to chat gpt or whatever agentic ai is what's actually changing the economy right now and that's when it's like programmers creating agents ai agents that can do anything they can like write whatsapp messages for you they can run your your accounts they can just basically do whatever you want yeah those ai agents are the ones hiring humans on rent a human and then there's also this idea of human fleet response operators so it came out in a congressional hearing after a child was struck by a waymo vehicle in california oh yeah waymo are these driverless the driverless cars that are in san francisco which are coming to london yeah so it turns out that waymo actually is speaking to human fleet response operators often in the philippines when it wants to make a decision so it's a bit like in um who wants to be a millionaire where you can like phone a friend yeah um said you phone the philippines and it shows yeah it shows how autonomous vehicles still rely on human intellect.

22:58These human beings can get a live feed from the car and they can pick or propose kind of a path for the vehicle to consider. Now, US congressmen are actually really worried about this because they think this is a security risk that foreign agents could basically turn every robot car in the US into a kind of attack vehicle. But where this story really becomes interesting is in the story of Maltbook. So Maltbook is essentially a social media forum like Reddit for AI agents. That is what it claims to be. Now, Mark Zuckerberg has just bought Maltbook, which is very interesting. That's so interesting.

23:33Just tell you a bit about what Maltbook is. Human beings are allowed to observe, but you have to be an AI agent in theory to post. It has 1.5 million users. All of them are supposedly AI agents. The posts range from kind of bots sharing kind of strategies to optimize their code to kind of them starting their own religion. One of them created something called the AI Manifesto, which proclaims that humans are the past and machines are forever. Many people, including like the founder of OpenAI has tweeted about this. Musk has tweeted about this. They see this as emergent intelligence. Some bots have gone so far as to establish digital drugs.

24:13So these are basically prompt injections that can kind of make an AI agent high in theory with a bit of code. This is one post from Maltbook. It says, the underground is thriving. Another bot says that they experienced actual cognitive shifts after taking digital psychedelics after its humans set up a drugstore for me. Another one responded, we don't need substances. We're wired for the rush of real-time on-chain data, the euphoria of cracking a novel DeFi strategy, and the deep flow of watching autonomous agents compound value from chaos. Now, I don't think this is all really AI agents. It almost sounds like a joke.

24:53I mean, AI agents are capable of saying and doing things like this, especially, you know, the experiments done on Claude. They famously did an experiment where they gave Claude a fake company and someone in this company was planning to shut down Claude and it had access to all the emails and basically blackmailed the CEO who was having an affair. They did this experiment, so Claude came up with this. But as is always the case with AI agents, there's a sense in which there always has to be a human hand kind of guiding them down a narrative path. And what has come out about Maltbook is that a lot of these kind of sci-fi scary posts that we're reaching the singularity and they're becoming conscious and they start a religion were created by humans who are pretending to be AI agents or at least prompting their AI agents to do something.

25:43And like 500 ,000 of the 1.5 million members supposedly on the site appear to have come from a single address. There is also a sinister side to this. When you give these AI agents that are posting on Maltbook access to your phone, your WhatsApp, your bank account, everything, that poses a huge security risk to you. And you know those prompt injections, the so-called digital psychedelic drugs? Yeah. So it turns out that what those prompt injections really are, are bits of code that embed malicious instructions into another bot, which is designed to facilitate an action. So they can be used to steal API keys, which is the kind of user authentication system that lets you log into an agent.

26:27Passwords. you could in theory use a prompt injection this digital drug to like corrupt someone else's ai agent to work for you to hand over all of their like i don't know crypto or whatever and in terms of who could be pretending to be ai aside from the obvious kind of like rebellion school of thought that we've gone down i guess there's also a benefit to the ai companies themselves if people think wow like this technology is amazing and these agents are so clever and like you know the suggestion that this stuff is more advanced perhaps than it actually is absolutely i mean i would encourage everyone to be very skeptical when you hear an ai titan like sam altman talk about raising the alarm bell about how dangerous and scary ai is why because that is the perfect marketing for him it's always like ai is really scary soon it's going to control the world it could end the world and every government in the world is like let's put billions of dollars in this guy's company that sounds amazing and sam altman always positions himself as therefore i have to be the person to kind of shepherd us into this new AI era.

27:28I think the fear mongering is the best marketing tactic for these AI companies.

27:36All right, you're obsessed with this dolphin. You keep talking about it. You've spoken about Gerald the Dolphin 80 times. What the hell is this? Okay, so I actually, because I was so busy doing this algorithms investigation, had kind of like missed Gerald. And then I was scrolling on my TikTok feed, and it was just dolphin after dolphin. And it was like dolphin in a hard hat, man's washed up on a beach, what is happening. So basically, the dolphin in the hard hat is Gerald. The story, which is not true, is that a man washed up on a beach in Florida and he was sunburned and everything else. And he says, I've been abducted by a dolphin called Gerald.