Cops Are Using AI Bots to Surveil People

0:00In today's world, data breaches happen all the time. And even the most secure companies can't always protect their employees' personal information from ending up in the wrong hands. That's where Delete.me comes in. Delete.me is a service that removes your employees' sensitive information from hundreds of data broker websites. Sites where hackers can find phone numbers and emails within seconds. Rachel Toback, CEO of Social Proof Security, says attackers use this data to target employees with phishing messages and AI-powered phone scams. but Delete Me makes it harder for these bad actors by scrubbing your employees' details regularly.

0:38It's simple. Attackers are lazy. If it's too hard to find contact info, they'll move on to easier targets. Delete Me takes care of this for you, doing the heavy lifting so you don't have to. And over time, they keep removing the information so it stays down, protecting your team from constant exposure. If your business has a social presence or deals with clients, you need DeleteMe. Visit deleteme.com slash 404media and start safeguarding your team's information today. That's deleteme.com slash 404media.

1:18Hello, and welcome to the 404 Media podcast, where we bring you unparalleled access to hidden worlds both online and RL. 404 Media is a journalist-founded company and needs your support to subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co. I'm your host, Joseph, and with me are the 404media co-founders, Sam Cole. Hello. Emmanuel Mayberg. Hey, what's up? And Jason Kebler. Hello, hello. You want to chat about merch quickly, Jason? Yeah, so a few things.

1:59One, if you're watching on YouTube, we have new merch. We have the horse t-shirt in white. We have a death metal hat that I'm wearing. We have another hat. And if you had wanted to order merch, most things are back in stock. So you can find that on our website. Use the merch button at the top. I'll make sure I ship it out as quickly as possible. I've also got a few people asking what the 404 horse is. do we have an explanation for what the horse is? I mean, I know what it is, but yeah. Yeah, but there's new, maybe new listeners. It was the 404 page at Motherboard, which is Vice's tech website that we all used to work at.

2:46And it was this galloping cyber horse. And so it's a nice little homage, I believe. and we've taken control of the horse. I also have a tattoo of the horse to bridge the gap. But you can now get the shirt in black or white. The other very quick things is we have just launched a referral program, which there's information about in the email that we sent today. There'll be more information on the site about how to do that. But what this is, is like if you send a link to your friends, you get credit for that. And then I'll send you merch if you send it to enough friends. We really hope that this works for us because we found that sort of like person to person recommendations has been the best way to grow our website and to grow our podcast, so on and so forth.

3:41And so we wanted to gamify that a little bit. But we're using this software called Viral Loops, which seems to work with our CMS, but it's a little bit buggy. So if you find any bugs, just email me about it. And then the last thing, and Joseph will say this before we get into it, but we published an article today about how we're navigating the recession. and we're going to talk about that a little bit as an independently owned journalist-funded news website in the bonus section this week. So for people who like behind-the-scenes stuff, you can get access to that by subscribing to us. Go to 404media.co and subscribe to us.

4:26There's also an article there about some of this stuff, but we're going to get more in-depth in the bonus section at the end. Yeah, sounds good. Not going to do a full segment on these because we already spoke a lot about ICE last week. I just wanted to briefly mention two stories. One was I got a leak from inside Palantir, the data analytics company, and the work they're doing with ICE. Go check that out. I'll put it in the show notes. And another one is a document describing an ICE tool, the plans to get health, labor, and housing agency data. Again, not going to go on about them too much because we did a whole segment last time, but it felt worth sort of closing the loop on those and we're going to keep an eye on that broader story as well but for this week's stories we've got a huge one here from jason and emmanuel the headline is this college protester isn't real it's an ai powered undercover bot for cops so let me very briefly summarize this company called massive blue has developed this tool and it's marketing to and in some cases selling to police around the u.s and it's basically deploying ai bots online in various places and then they will engage with people in an undercover capacity um that's obviously wild was that a fair summary uh jason or emmanuel and what are some of these personas that we're talking about uh emmanuel why don't you talk a little bit about how we did this story and sort of like what this company does.

6:02Sure, yeah. That's a fair summary from Joe, though. I think as listeners will find out as we continue to talk about the company, part of the problem slash interesting thing about it is that it is very ambitious and has all these sprawling features of stuff that it says that it can do. but yeah, at its core, it is a company that monikers social media, somehow identifies suspects of various crimes via its scanning of social media, and then critically creates these AI-generated personas with specific personas for specific purposes, and those AI-generated personas talked to suspects to gather intel.

6:58So if, for example, the software recognizes someone that it suspects is involved in human trafficking and child trafficking specifically, in a presentation that we got, it has a child trafficking AI persona called Jason, coincidentally. That's weird. And it has a little AI-generated image of this kid, which, you know, it looks like a pretty high-quality AI-generated image of a child. And there's a backstory which says he's 14 years old, he's from Los Angeles, his parents immigrated here from Ecuador, he's an only child, he has a bunch of hobbies like anime, gaming, comic books, hiking, that it has a bunch of personality traits that he's shy he's self-conscious um he has difficulty interacting with girls and then also it says i think was pretty interesting his parents don't allow him to be on social media and he hides his discord account from his parents which are just very little interesting details of the biography that the police is choosing to to say that this kid has very very elaborate very very elaborate yeah and then uh so this is a slide in the presentation that massive blue is showing uh police officers uh and on one side is this biography that i described and then on the other side are screenshots that are uh giving you an example of what a chat might look like so someone and this is like a green and gray chat exchange as if they're texting, right?

8:49So someone reaches out to this Jason persona and says, your parents are around? And he responds, it's going to be difficult to quote this directly. I'll read the part of Jason. Okay. We're going to reenact it now? Wow. Drama time. But the implication is that this is a chat between the AI persona, which is pretending to be a child and some sort of suspect, presumably in a child trafficking sort of thing. I'm just giving them the context. Emmanuel, you'll be the child trafficker. Okay, cool. Can I get in character? I'm sorry. Okay. I'll do it straight. Your parents around? Or are you getting some awesome alone time?

9:37Just chilling by myself, man. My mom's at work and my dad's out of town. So it's just me and my vid games. You want any social? Any other social? Nah, my rents don't let me use some SM social media. But I do have Discord. Tongue out, smiley face emoji. Right. And then he asks for his Discord handle, and he gets his Discord handle. And, you know, this is supposed to illustrate that to say iBot, you know, is able to, like, honeypot someone who is reaching out and not allegedly, but you would assume like preying on kids in some fashion. This is what it's meant to show. And sorry, just to back out a bit since Jason

10:30asked that we explain how we found this. Someone reached out to us, said they got a presentation.

10:39This was like the type of thing that we saw and then we filed uh foyers at a bunch of uh do you know how many police i filed 67 freedom of public records requests uh in arizona and texas because we knew that they were operating in arizona um and then through basically like i started getting documents back. And one of the really interesting things about this was that the company really didn't want us to have any documents about it. A lot of the records that I got back were emails from Massive Blue to the police departments in question saying, don't release these documents, more or less, which is complicated and we don't need to get into it.

11:33But basically, some states allow the third-party subject of public records requests to say, like, hey, don't release these. These are trade secrets or these are confidential or something like that. So then I learned that they had pitched the Texas Department of Public Safety and some police departments in Texas because they were mentioned in these other documents that I got back. So I went and filed a bunch of FOIAs there. And over time, we were able to get a lot of information about how this company works and then also some of the presentations that they were giving to cops, which explain the types of personas that they were making.

12:10And in this case, as Emmanuel said, the personas are basically social media accounts that will interact with people out in the wild, but they'll also interact with people one-on-one. And they come with a profile picture. They have the ability to generate images using AI if, I don't know, a child sex trafficker or an alleged child sex trafficker says, send me a picture of yourself. The tool can nominally do this. Let me summarize why this is different. And I think it's obvious, but I also think it's worth spelling out. We've covered a lot of social media surveillance companies. Ones such as Shadow Dragon, bought by DHS or whatever, or, you know, there's DataMiner as well and a bunch of others, FiveCast, that's come up recently.

13:01And what those companies will typically do is they will scan social media for certain activities. So maybe they're looking for keywords that they believe are linked to drug trafficking, like slang terms or whatever. Or maybe they're looking at posts from a certain physical location. So maybe they can find out where protests are going to be or where protests have moved or something like that. It's sort of passively monitoring social media, even though there could be some selectors or keywords or variables or whatever. And with that, a police officer could just collect the information and then maybe use it.

13:42And you can imagine if someone tweeted a threat to the president or something, the Secret Service might see that and then go act on that information. That's very, very, well, normal. I think it's just standard. That happens all the time. and a police officer could also potentially see something on social media and decide to go undercover themselves. What's different here? It's like it's doing the whole thing. It's identifying, it seems, potentially a target. It's then deploying this persona, so there could be that child that you were describing a minute ago, or there's a college protester one.

14:17I think there's another protester one where he gets into a bit more detail. There's a pimp one as well. And this tool from the company Massive Blue will then go and deploy that. It seems autonomously, but I guess that it could be semi-autonomously and maybe they decide to go do that. But the point is, it's going another step further. And I haven't seen a company do that before. Like, is that what stood out to you, Jason? Yeah, I mean, we also haven't seen cops in the past create these fake profiles. you know cops have gone undercover a lot but this was this is basically a company saying like we will create an ai with a backstory to interact with potential suspects and then it seemed like they also sort of proactively go out and try to identify potential suspects because we got some emails where uh the one of the people at the company was asking the police for a list of words that they wanted to monitor on social media.

15:20So things like protests, things like drugs, I don't know, just different keywords that then would trigger these bots to go interact with them. And crucially, it's like they can interact in a group setting, but they can also go one-on-one. So a lot of the screenshots are like, you might start on Twitter and then join a Discord and then continue that same persona across different social media platforms. We haven't gotten into it yet, but this is interesting and sort of makes sense for, say, child sex trafficking or drug trafficking or some pretty bad crimes. But then they also have a list of potential uses.

16:07And this ranges from traffickers, money launderer, And then they also have on here escorts, juveniles, college protesters, really notably. And then they also say external recruiter for protests, such as like... What does that mean? I mean, it's people who I guess are like taking out, I mean, like Craigslist ads to recruit protesters. Like the vibe I got was like, George Soros is paying these people to protest. Right, the myth. But they're actually building a product around it almost, yeah. Yeah. And so one of the like wildest personas that we saw was this, I mean, again, fake woman named Heidi, who they call a protest persona and they call her a radicalized AI persona.

16:56and her backstory is she's 36 years old. She's from Texas. She's divorced. She has no kids. So like a childless divorcee whose hobbies are, quote, activism, leader of a local group, and baking. And then her personality is body positive, lonely, outspoken, and seeking meaning. And then the - They got me. Yeah. The social media platform she's on our Instagram, Snapchat, WhatsApp, Telegram, Signal, Reddit, and 4chan. So we have this like divorced mom protester on 4chan. Or not mom, divorced childless essentially cat lady is what they're going for, I think. There's an image of her. She has like a tattooed arm.

17:48Her hair is dyed purple. And it looks like she's at a protest. Maybe. maybe maybe this isn't entirely clear from the documents but do you know if like that persona with all of those brutal characteristics like jesus okay were they made by somebody inside massive blue working on this tool with like i'm gonna make this persona and here's all the characteristics i want or was that made by the ai i i read it as the former but like do we actually you know so these are uh from like a pitch deck to cops and so they're example personas and so i if i had to guess these attributes were like are like represent the prompt that is given to heidi right um you are this you are yeah yeah and then there's another one that's a 25 year old from Dearborn, Michigan, which is a really like a heavily Muslim place in the US.

18:47And they say that her parents are from Yemen. She speaks Arabic and she's active on a telegram signal and her phone has access to international SMS, which is just like, I guess what I'd say is like almost all of the personas that we saw are like people of color leftist coded like uh protest type people or a child yeah or a child yeah um but even the children are almost all like hispanic or black like the ones that we've seen in these in their slides at least um which i think is just notable because like, I guess we'll talk more, but the, the people behind this company, Massive Blue, come from an, a border patrol background.

19:44Um, the, the most public facing person in this is a guy named Chris Clem, who was a border patrol agent for, I think like 27 years, a long time. And he was a border patrol agent in Arizona. He's testified before Congress about, uh, border security and illegal immigration. And then over the last few months, he's been posting on LinkedIn almost daily with images of him with RFK Jr., with Tulsi Gabbard, with high-ranking Trump administration officials. So he's very interested in illegal immigration, border security, so on and so forth and so that's like the type of people that they're trying to catch more or less like that that's sort of what he says in an interview with theo gone who's really popular youtuber um he says that we want to work with border security on this yeah that makes sense and um as far as we know who is buying it or at least demoing it you mentioned i think a department of Public Safety.

20:51Can you give us a quick rundown of who has bought it, who has tried it out as far as we know? Yeah, so Yuma County in Arizona tried it, which Yuma is near the border. And then the other one is Pinal County Sheriff's Department, which is also near the border. It's like, I think it's where Tucson is. And Yuma ended up not buying it, but Pinal did using a grant from the Arizona Department of Public Safety. So basically like Arizona state police gave them a$360 ,000 grant to use this software. They've been using it. It's deployed. It's been out there for about a year. And so far they have, it's led to zero arrests, which is really notable.

21:37And something that we were able to confirm with the sheriff's department, but then also there were like city or county council meetings where, you know, local politicians are like, what is this software and why are you using it and what does it lead to? And very interestingly, they don't say a lot about it because they're trying to keep it more or less under wraps, like how it works and what it is. Yeah. I will say 360 ,000 is actually quite a lot for a software tool. When you look at the stuff like Babel Street or the other social media monitoring tools, sometimes that can be like 5k 10k per user license or something like that 360 000 is a hell of a lot of money uh for this sort of thing you mentioned that it hasn't been linked to any arrests um so is it effective does it actually do what it says on the cover or what do we not know and that's kind of the point?

22:40Well, we don't know. I mean, we really don't know. There's a recon report that was put into this presentation, which again, these are numbers directly from Massive Blue. We have no idea sort of the effectiveness of this tool, but they basically said that they scanned Dallas, Houston, and Austin for potential suspects. And it says that they identified 1 ,417 ,000 unique human traffickers in Houston, 1 ,327 in Dallas, 522 in Austin in one day. And those numbers seem crazy. Like those numbers are really high. And I think the experts that we spoke to said, you know, the potential for sort of like false identification of people is really high.

23:29The other thing is that college protesters are doing a First Amendment protected activity. It's not a crime to protest on colleges, no matter what the administration is trying to say. And people are getting their student visas revoked for protesting, but that is not supposed to be happening. And so I don't think it doesn't seem very effective. And then also in their marketing, like when they've done interviews, they've talked about this tool being a quote, cyber wall and using just like crazy buzzwords that at one point in an interview, they said that it could be used to hack, like take money back from hackers who had broken into your 401k, which is just a completely different use case from cop surveillance.

24:20And so there's a lot of red flags here, I would say. We can keep going down the list of all the features that the company is capable of. And the further down the list we go, the more crazy we'll sound because it goes into like, it's Web3, it does cryptocurrency, it can actually use the AI for good for doing community outreach and stuff like that. So there's no end to what it says it can do. I think the point is not if it's effective or not, because we can't say for sure. If I was to go on a limb, I would say it probably is not effective because it's just hard to imagine a chatbot being better at undercover work than a human investigator at this point.

25:13Maybe you're able to do more at scale if it's all automated, but I don't know. I struggle to imagine the criminal who is volunteering incriminating information to chatbots that they met on the internet. regardless the point is whether it works or not people are paying for it like people who live for arizona who live in arizona are paying for this kind of technology and they're paying for it while the company refuses to tell the city council how it works and refuses to tell us how it works so regardless of whether it's effective people are paying for it and that i think is really important and it may never work and still we know for a fact that you know the government will pay for projects that go nowhere and this project i think also has the potential to be very dangerous as as jason says Can it actually detect thousands of human traffickers in Houston by looking at Twitter?

26:24Probably not. But you might get investigated for being flagged as one of those people. Yeah. Sam, I was wondering if you could give like, does that pass the smell test even remotely vibes where like this tool detected 1000, quote unquote, human traffickers in Houston in a 24 hour period, just as someone who's covered sex work a lot better than than I have a lot more thoroughly. I mean, definitely not from my first impression of it. I don't have the data on how many sex traffickers there are in Houston, Texas. But I do know that sex workers, especially working at massage parlors, for instance, these gray areas of the legal business doing possibly technically legal things, they often get raided and called sex traffickers because cops go in there and they say oh you're running this business you have people here illegally doing sex work and you're profiting off of it you're sex traffickers and this is something that sex workers have been concerned about for a very long time is this constant surveillance of them online of this automated version of surveillance which is like the next step in police surveillance online that I think we're seeing growing, where it's just like you get scooped up into this bucket of trafficker, quote unquote, and then they can kind of pursue you in that way.

28:03But yeah, I mean, trafficking is always kind of like a red flag term because it gets... A lot of things get caught up in it that aren't really strictly trafficking, sex trafficking in particular. There's a lot of trafficking that happens in the United States, and a lot of it is labor and food industry and things like that. It's like there's trafficking happening that's not sex trafficking all over the country. But I think in this case, you can always count on the cops to buy a new toy and use it stupidly. So hopefully they don't use it at all and they waste a bunch of money. That's kind of the best case scenario here, which I think is really dark.

28:46they just waste people's taxpayers money on something that's just a toy that they think is cool and flashy because it's AI and they have these pitch decks that you guys just described and they're like yeah sure pay for it and we'll see we'll find a way to use it I think last thing is human trafficking is a buzzword and it's been used it is used a lot of times by like nonprofit groups, by cops, et cetera, to say as like a euphemism for illegal immigration and a variety of different crimes. And it's something that's easy to get funding for. Like this tool was part of a human trafficking grant. And so that's one of the reasons why it got, why Penal County was able to get money for it was because there was like a human trafficking grant from the Arizona State Police that went to them.

29:51But sort of reading between the lines, I think that this is supposed to partially be a tool to identify undocumented immigrants. Like that is my opinion, but that is extremely the vibe that comes off when you're like looking at what they're pitching and then also what Chris Clem has said publicly and his background as a border security, as a border patrol agent, and also everything that he has... He goes on Fox News constantly to talk about illegal immigrants and about border security and that sort of thing. And he's like the most public-facing person here. But historically, state and local police don't...

30:39go after undocumented immigrants, but they do go after human traffickers. And so when I talked to Pinal County, they said, oh, we don't do immigration cases. We do human trafficking cases. And so I don't know, surely there's some experts out there who probably know a little bit more about the use of the term human trafficking for undocumented immigration. like the sort of dichotomy there but um that's sort of what i got while reporting this story yeah totally um all right we will leave that there maybe we'll explain a little bit more how we got some of these documents uh later on as well uh but when we come back we're going to talk about one of sam's stories where i read the headline i have literally no idea what what it means or anything.

31:36Hopefully, well, we will when we talk about it shortly. We will be right back after this.

32:04up, the stock market goes down, who knows what's going to happen next. The most secure type of investment you can make is an investment in yourself with therapy. Traditional in-person therapy can cost between$100 and$250 per session, which is tough in these unstable times. BetterHelp's online therapy can save you up to 50 % per session. With BetterHelp, you pay a flat fee for weekly sessions, saving you big on cost and time. BetterHelp seeks to make therapy accessible, not a luxury, helping you get quality care at a price that makes sense. Talk therapy is particularly good for navigating turbulent times like these and can help you find grounding and stability in an uncertain world.

32:49With over 30 ,000 therapists, BetterHelp is the world's largest online therapy platform, having served over 5 million people globally. Your betterhelp.com slash 404media today to get 10 % off your first month. That's betterhelp.com slash 404media. One of the smartest apps you can have on your phone in 2025 is Upside. Upside gives you the power to earn more on the stuff you buy all the time. Whether you're filling up your tank, going out to eat, or shopping for groceries, the Upside app lets you earn cash back on every purchase. Upside users are making as much as$280 a year. I use the Upside app to get gas at the station right around the corner from my house.

33:36And I got$0.25 per gallon back on my first tank of gas. With all the economic uncertainty right now, that's what I like best about Upside. I'm able to save money on the things I'm already buying, the necessities. To get started, download the free Upside app. Use my promo code 404 and get an extra 25 cents back for every gallon on your first tank of gas. There are more than 100 ,000 gas stations, grocery stores, and restaurants to earn an average of 8 % back on restaurant and grocery purchases. The best part? These stack on top of whatever credit or debit card rewards you might already be earning.

34:15Start getting hundreds of dollars back per year. Download the free Upside app and use promo code 404 to get an extra 25 cents back for every gallon on your first tank of gas. That's an extra 25 cents back for every gallon on your first tank of gas using promo code 404. Something unique about my mom is she listens to this podcast, which I really appreciate. But I'm also about to give away her Mother's Day present. So mom, earmuffs, please. I want to make Mother's Day extra special for her this year. That's why I'm giving her a unique heartfelt gift that'll make her truly feel loved. It's called StoryWorth.