The ICE Tool That Tracks Entire Neighborhoods

14 Jan 2026 · 43 min · 16 chapters

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

404 Media Podcast Episode Summary

Episode Title

The ICE Tool That Tracks Entire Neighborhoods Podcast Hosts: Joseph, Sam, Emanuel, Jason Release Date: [Insert Date] Listen Here: [YouTube Version](https://youtu.be/rurJo6vPhUY)

Overview In this episode of The 404 Media Podcast, the hosts discuss two major stories published by 404 Media: a detailed investigation into Webloc, a surveillance tool used by ICE to monitor neighborhoods, and a report on Grok, an AI image generation tool that has been abused for non-consensual content.

---

Timestamps

  • 0:00 - Intro
  • 2:50 - First Story: Webloc Surveillance Tool
  • 23:00 - Second Story: Grok and Non-Consensual AI Imagery

---

Key Topics Discussed

  1. ICE's Surveillance Tool: Webloc

Summary of the Investigation

  • Background: Joseph discusses the article detailing how ICE uses Webloc, a tool that allows monitoring of mobile phones in entire neighborhoods.
  • Features of Webloc:
  • Capable of tracking movements of mobile phones across various locations.
  • Allows ICE to determine potential residences and workplaces of individuals based on phone activity.
  • Offers both broad and targeted surveillance capabilities.

Concerns Raised

  • Speculations on whether ICE could use this tool to monitor protests or activists.
  • Discussion on the ethical implications of such surveillance technologies being utilized by law enforcement.
  1. Grok and AI-Generated Imagery

Overview of Grok

  • Emanuel provides insights into a Telegram channel where users develop sophisticated prompts to generate non-consensual images using Grok.
  • Comparison: The channel has more advanced techniques compared to prior public uses of Grok that garnered media attention.

Discussion Points

  • The impact of Grok's misuse on the broader conversation about AI ethics and the responsibility of platforms to regulate content.
  • The evolution of the community and how they adapt to restrictions and challenges posed by the platform.
  1. Monetization of Controversial Content

Sam's Coverage on Grok's Monetization

  • Sam discusses how Musk's leadership at X (formerly Twitter) shifted the responsibility of content moderation towards monetization strategies.
  • Highlights the contradiction between claims of limiting harmful content while essentially charging users to access it.

Key Takeaways

  • The episode underscores the challenges faced by journalists and communities in addressing significant surveillance and ethical dilemmas surrounding tech.

---

Notable Quotes

  • Joseph on Webloc: "This tool allows an ICE official to manipulate and use this location data from mobile phones."
  • Emanuel on Grok's Telegram Channel: "Users there are developing prompts that take advantage of loopholes to create non-consensual images."

---

Additional Resources

  • Articles Mentioned:
  • Inside ICE's Tool to Monitor Phones
  • DHS Is Lying To You
  • Inside the Telegram Channel Jailbreaking Grok Over and Over Again

Support the Podcast: For ad-free episodes and bonus content, listeners are encouraged to subscribe to 404 Media at [404media.co](http://404media.co).

---

Conclusion This episode of The 404 Media Podcast dives deep into the intersection of technology, surveillance, and ethics in today's digital landscape. The discussions not only inform listeners about the current state of surveillance tools but also raise important questions about accountability in the age of AI.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Chapters

Tap a time to open that second in VO

Zine Update: Behind the Scenes

1:34 to 2:51

An update on the printing and distribution of the zines, emphasizing the manual process.

“Yeah, and you've been cataloging the process, right?”

Introduction to ICE's Monitoring Tool

2:52 to 4:19

Discussion on ICE's use of a tool for monitoring phones and its implications.

“Yeah, I mean, this is a very interesting story because the timing of you publishing it aligns with what happened in Minneapolis, which we will talk about, of course.”

How WebLock Operates

4:20 to 6:57

Detailed explanation of WebLock's features and capabilities for location tracking.

“Previously, it was run by a company called Cobwebs.”

Potential Uses of WebLock

6:58 to 7:58

Speculation on how ICE could use WebLock to monitor protests and neighborhoods.

“to really easily drill down on what phones are there, where they go, and then of course, potentially who they belong to.”

Legal and Ethical Concerns

7:59 to 10:10

Discussion on the legal framework surrounding the use of location data by ICE.

“It's like monitor phones in entire neighborhoods.”

Sources of Location Data

10:11 to 14:01

Exploration of where commercial location data comes from and its implications.

“That would probably be within their mandate somewhat.”

The Mechanics of Online Advertising and Data Harvesting

14:01 to 18:00

Explores how programmatic advertising works and the implications of RTB data harvesting.

“there is this near instantaneous and basically invisible to a normal person process where all of these companies are trying to outbid one another to get their ad in front of a certain demographic.”

Emergence of AI-Generated Non-Consensual Content

18:13 to 23:00

Examines the rise of AI tools like Grok used for creating non-consensual images.

“If you've ever thought about upgrading your window treatments, but immediately stopped because you didn't want the hassle, I get it.”

Deep Dive into Grok's Telegram Channel

25:29 to 28:02

Analyzes the use of Grok in Telegram channels for creating explicit content.

“And the loophole there was basically misspelling her name and describing sexual acts without using any explicit terms, which is kind of like the first line of defense that AI image generators have to prevent abuse.”

Exploring the Dark Side of Grok and Its Meme Culture

28:02 to 29:48

Learn how Grok's misuse in generating explicit content has become a meme phenomenon.

“So it's like you can get Grok to do pretty bad things without trying hard at all, as you said.”
Show all 16 chapters

Concerns in the Grok Community

29:49 to 31:28

Discover the worries within the Grok community about increased scrutiny and potential crackdowns.

“I don't know exactly what is happening on the Grok end of things because they will not respond to our request for comment.”

The Disbanding of the Prompts Telegram Channel

31:29 to 32:51

Understand the implications behind the disbanding of the Telegram channel dedicated to Grok prompts.

“that people have on sort of the public Grok generation.”

Grok's Monetization and Accountability Issues

32:52 to 35:30

Examine how Grok's monetization strategies are intertwined with the generation of harmful content.

“And this isn't the first time that this has happened.”

Jeff Musk's Controversial Approach to Content Moderation

35:31 to 37:24

Analyze Elon Musk's questionable practices regarding content moderation on the X platform.

“So obviously Elon said, hey, here's a convenient way or someone on.”

Reflections on Non-Consensual Content and Its Impact

37:25 to 40:50

Explore the troubling rise of non-consensual content on platforms and its societal implications.

“I think there's some Ofcom stuff as well, which is sort of this government body in the UK investigating.”

Discussion on Financial Awareness

42:00 to 42:20

Explore how attention to bills and income affects financial decisions.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:03Hello, and welcome to the 404 Media Podcast, where we bring you unparalleled access to hidden worlds both online and IRL. 404 Media is a journalist family company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co. I'm your host, Joseph, and with me are 404 Media co-founders, Pam Cole. Hello. And Angle Mayberg. Hello. And Jason Kevler. Hello, hello. I don't think we have any housekeeping this week. I'm going to do a quick, quick zine update.

0:50So we are waiting on the last of the zines to come from our printer, which I'm expecting in the next, I don't know, eight to 10 days, something like that. I think I mentioned, but we had to print many, many, many, many, many of them, and they're all being put together by hand. by a small print shop. So I will have them in the next 8 to 10 days. I have all of the envelopes addressed. So once I get them, they'll be in the mail. But I think like two or three people have asked. So if you happen to be listening to this and you happen to have ordered a zine, it will be going in the mail soon and we'll be putting a version online at some point as well.

1:37Yeah, and you've been cataloging the process, right? With videos, filming and printing. Have we posted any of those? No, because I need to put them together to do the editing process and stuff like that. But that's where we are with it. But yes, we're going to do probably YouTube videos, some social videos. It's been a cool process. And I think once you see it is like a very manual process, which is what we wanted to do for this first one. We wanted it to be very human, very manual sort of thing. And we're very happy about a couple thousand of you bought them. And so you're not really supposed to do this type of printing at this scale, I would say.

2:27And we did. But it's actually... We said we're going to ship them in mid-January. It will still be mid-January. It will probably be in the next pretty soon. Yeah, when we put them on sale, it was very much a pre-order situation. It was not a, hey, you're going to get the zine in three to five working days or shipping or whatever. But definitely looking forward to that being out. Should we move to the first story? Jason, do you want to take the lead on this one? Yeah, I mean, this is a very interesting story because the timing of you publishing it aligns with what happened in Minneapolis, which we will talk about, of course.

3:10So the story is inside ISIS tool to monitor phones in entire neighborhoods. Joseph, let's start with your story and then maybe we can zoom out to the killing of Renee Good. So first of all, what did you obtain and what is this company, WebLock? Yeah, so I'll just say, I think we did mention this tool briefly on the podcast before, because I found it in contracting US procurement records. So it's like, hey, ISIS just bought this tool and you can sort of figure it out. But this is obviously new and much more in detail because I obtained material that explains, in a very granular sense, how these systems work.

3:57So there's plenty more detail in this article than we have before. And that applies not just to ICE's potential use of it, but obviously any government user of WebLock and another tool called Tangles as well. I think Texas law enforcement agencies have it, maybe some other ones around the country as well, regional, if not local. But WebLock is a location data tool. It comes from a company called Penlink. Previously, it was run by a company called Cobwebs. They merged with Penlink, blah, blah, blah, whatever. But Weblog is very, very much focused on location data. So, and we'll go into how they get the data in a minute, but it can track movements of mobile phones, essentially, from one location to another.

4:47You could look in a certain area and figure out what phones are there and that sort of thing.

4:55So there's no indication necessarily of where ICE is using this at the moment, correct? This is sort of like how this tool works. You obtained some documents that explained what its specific features are. And as you mentioned, we knew that they had purchased this, but the news here is that now we know exactly some of its capabilities and what they can do. So what can an ICE officer do with WebLock and Tangles? And what are some of the specific features? Yeah. So WebLock makes it really easy for an ICE official to manipulate and use this location data from mobile phones. So whereas ordinarily, if a law enforcement agency got a bunch of location data, say from a company called Ventel or Babel Street, which we've covered before as well.

5:54This tool, and maybe those have this as well, but again, we've seen the material for this. So you can push a button on the interface that will turn it almost into nighttime mode. And you can see where the phones went at night. And the material explicitly explains that, hey, if a phone is at this certain location overnight, and especially, you know, repeatedly over and over again, maybe that is the person's home. If you put it into daytime mode, you click a little button for that and you follow the phone to where it is located during the day, maybe that's their place of employment as well. You can even click a little button which provides the route so you can see not just sort of the start and the end locations, their home and their potential workplace.

6:40You can see the actual route that they took. So presumably, it'll be what highway they drove down, what road they went down. Maybe if they walked or whatever, you could probably figure that out based on the speed as well. So it really allows an ISIS officer or any law enforcement official using this tool to really easily drill down on what phones are there, where they go, and then of course, potentially who they belong to. And I guess just the last thing I'll say on the features is that it's not like they just get a list of phones and they click through and they go one by one. They can do that in a sense.

7:19But what they also do is they draw a circle or another shape around a particular area. And that is going to show, hey, this is all the phones that we have in that location at that day or across that hour or whatever it is. So it really can be used for both quite targeted surveillance, if the target happens to be in this data set, or it can be used for quite broad surveillance as well. Yeah, I mean, I guess I want to be a little speculative here. I'm not saying you are. I mean, maybe you can talk about it. But with this tool, I mean, that's sort of the headline of the thing. It's like monitor phones in entire neighborhoods.

8:01So you can circle a whole neighborhood and it can give you information about the phones in that neighborhood. I mean, we have seen over the last week, and of course, honestly, over the last year, a lot of protests against ICE, a lot of protests outside of ICE facilities, a lot of protests in neighborhoods where there have been clashes between people and ICE, that sort of thing. I mean, it seems like an agency like ICE would be able to monitor protests sort of retroactively using a tool like this? Is there any indication that it could be used for something like that? Yeah, I think that's totally fair.

8:43And I'll say two things. The third, well, actually maybe three things. The first is that there is some local reporting coming out of Minneapolis that I saw and I think Sam saw as well. I still need to digest it all, but there are reports or suspicions that ICE, the HS, may be using data to go after activists and intimidate them in some way. I'm still digesting that, so I can't go much further in depth than that, but I would definitely highlight that report and come out of NPR, I think. The second is that the tool was technically purchased for HSI, Homeland Security Investigations. That is, of course, a part of ICE that's usually focused on criminal investigations, child abuse, cybercrime, money laundering, all of that sort of thing.

9:32That's kind of gone out of the window now because there's this really interesting data that the Cato Institute keeps getting from inside ICE showing that something like 90 % of HSI officials have been pushed over to the deportation arm or rather the deportation mission. So HSI as a criminal investigative entity basically doesn't exist anymore. It's basically part of ICE's deportation arm. The reason I bring that up, though, is that if there was some sort of alleged assault of an ICE officer, or there was a protest outside an ICE facility, ICE wanted to investigate people there, in my opinion, it might be HSI that would do that sort of investigation.

10:14That would probably be within their mandate somewhat. And HSI is the one that has bought this tool. Jason, were you going to say something? Well, yeah, I was going to say that they do a lot of workplace enforcement investigations. They actually always have, in addition to some of the child trafficking, drug trafficking investigations that they really specialize in in previous administrations. But we've seen this administration really focus on enforcement at workplaces. I mean, there's been videos coming out of people getting snatched out of Target during their shift. There was the case where they seized a bunch of people from a Hyundai plant in Georgia, things like that.

11:00And so workplace investigations have been a big part of Trump's deportation campaign. And HSI, that's what they have specialized in traditionally. Yeah. And then the last thing I would say on the protest stuff is that the independent journalist, Jack Paulson, previously got some other material related to Cobwebs. Again, that was the company that first developed this tool before it moved to Penlink. I believe it was Cobwebs branded stuff he got. But that was basically, it sounded like marketing material or training guides where Cobwebs itself was saying, hey, look, you could use this to monitor Black Lives Matter and protests.

11:42So that's not the same as I saying or DHS saying, we're going to use it for this. But the people making, developing and selling the tool are explicitly telling potential customers, hey, you could use this tomorrow. Right, right, right. Okay, so let's talk about where this data is coming from. I think perhaps we can't say exactly for sure where it's coming from, but you've done a lot of reporting on location data, commercial location data sales over the years. And it seems like with many of these companies, the data usually comes from the same types of places. and that has to do with the advertising industry and it has to do with location harvesting from apps usually.

12:27So is that the case here? Is that what we expect is here? Yeah, it looks like that's the case. It is not. I'm almost certain about this. It's not coming from telecoms like your AT &T, your Verizon, T-Mobile, that sort of thing. police now need a warrant to go get location data from that after the you know landmark carpenter 2018 case this according to um a internal ice legal analysis the the aclu got and shared with us discussing location data ice believes it doesn't need a warrant essentially to get that information that lines up with other agencies we've covered like the secret service I bring that up here because in that legal analysis, they were talking about Ventel, another of these location data companies.

13:18And Cobwebs and Penlink and WebLock, they're in that space. So that leads me to... There are two main ways that commercial location data companies get this sort of data. The first is software development kits, SDKs embedded in ordinary apps. So that weird weather app you got, the weird storm chasing one you downloaded five years ago and forgot about and gave location data permissions to, that might have code in it where the SDK developer then harvests the location data and then sells it to companies. The other one, and I think the much more likely one, is that it's connected through real-time bidding.

13:59So RTB. Whenever there is an advert, a programmatic advert being placed inside an app in front of you, there is this near instantaneous and basically invisible to a normal person process where all of these companies are trying to outbid one another to get their ad in front of a certain demographic. A side effect of the online ad industry is that a bunch of companies can simply monitor that process. They don't even need to get an advert in front of somebody. They can just sit there and sort of siphon the data off. And that can include, and often does include, location data and a specific person's unique advertising ID.

14:39So there you have, hey, this person moved to here, they moved to here, moved to here. And also, we have a unique identifier that connects all of that location data together. The reason I say I think that's the most likely is that I just think SDK stuff has gone out of fashion in the location data industry. When I covered that, it was mostly like 2020, 2021, that sort of time. When we exposed a company called XMode that was collecting location data related to a Muslim prayer app and selling it to US military contractors. that was like one of the last SDK-based location data companies that I really remember.

15:19Yeah, there were a couple more afterwards, but it just seems that RTB is the way now. We got that from a hack of Gravy, the parent company of Ventol, and it showed a bunch of apps that were subject to RTB data harvesting. We published a big list of those. We've covered another company called Patterns out of Israel, and we got them kicked out of the Google advertising ecosystem after we showed, yes, they were getting data from RTB. So I think that is most likely how WebLock is getting its data and by extension how ICE is getting that data. Yeah. Joe, so this is obviously all kind of happening in the context of broader crackdowns by ICE.

16:02And then, of course, the huge national conversation spurred in the killing of Renee Nicole Good in Minneapolis last week. You want to talk real quick about how we covered that with this piece, DHS is lying to you? Yeah, it's difficult to cover because of course we're not on the ground and we don't feel like we have to cover everything. But of course, this case is monumentally important. For me, it was the disconnect between seeing these videos come out of the shooting of Renee Goode, first from sort of angles further away, and then actually later, after we published the first-person perspective of Jonathan Ross, the ICE official themselves, obviously a huge disconnect between what those videos showed and the statements that DHS, DHS head were putting out, the Trump himself as well.

16:55And they just simply did not line up. I think people intuitively who probably listen to us and read our website, probably agree with that, probably saw the same thing. And subsequently, you had Bellingcat come out and do amazing technical forensic analysis. There was New York Times as well. They did their own sort of visual investigation as well. I think those are always good and they're always amazing to see and they're always helpful. Here, you did not need Bellingcat or the New York Times to tell you what was going on. You could watch the videos with your own eyes. And that's what we encourage people to do, provide links to them in the piece, explain what we thought had happened based on those videos.

17:37And of course, as the headline says, DHS is lying to you and absolutely believe that based on the statements and the videos themselves. That's... Sorry, were you going to say something? I was just going to say, I mean, this is going to continue to be probably one of, if not the most discussed stories this year. So we're going to stay on it. But yeah, let's leave that there and move to the second half of the show. Absolutely. When we come back, Emmanuel and Sam are going to talk us through some of their recent Grok and X coverage. We'll be right back after this.

18:24If you've ever thought about upgrading your window treatments, but immediately stopped because you didn't want the hassle, I get it. Window shopping sounds fun, but in practice, it's totally brutal. Waiting around all day just to get a quote, then getting hit with a price that makes no sense. That's why I've been really impressed with all of this. blinds.com. They completely changed the game. You can shop custom blinds and shades 100 % online, build a quote in minutes, or talk to an expert through a virtual design consultation on your schedule, not theirs. If you know what you want, great. If you don't, their design experts actually help you figure it out.

19:03They offer professional in-home measure and installation services without the pressure or showroom markups. Blinds.com sends samples straight to your door so you can compare colors and materials at home. And it's not just Blinds. They've got everything from bamboo shades to shutters and outdoor patio shades. Blinds.com has been around for 29 years and has covered over 25 million windows. And every order is backed by their 100 % satisfaction guarantee. Right now, Blinds.com is giving our listeners an exclusive $50 off when you spend$500 or more. Just use code 404media at checkout. Limited time offer.

19:41Rules and restrictions apply. See blinds.com for details. On today's podcast, we ask, is the litter box the worst thing about having a cat? The answer is of course, but it doesn't have to be thanks to Boxy. The Boxy in Boxy Pro stands for probiotics, which stops the bacteria that causes odors so you never smell your litter box ever again. Boxy Pro keeps the box continuously odor-free infinitely. Just remember to scoop. All you do is top off the litter and you never have to dump out the whole box. It has amazing clumping power that makes scooping easier. You know, I've got three cats at home, Gus, Lola, Bruce.

20:22I love them. But the litter box, it's the worst part. But it doesn't have to be. And that's thanks to Boxy Pro. If you're tired of switching litters, looking for the one, get Boxy at B-O-X-I-E-C-A-T.com. It's the last letter you'll switch to. Enjoy 30 % off with the code 404media at boxycat slash 404media. That's B-O-X-I-E-C-A-T dot com forward slash 404media. We're the generation that watched everything change. Jobs, markets, the rise of the internet, the economic crash. And somehow, we were expected to just figure it out. Now, throwing the uncertainty of AI, tariffs, global strife, It's hard to have any certainty in this world.

21:08Those of us in our 30s and 40s have been forced to take matters into our own hands to protect our financial future and present. Those of us in our 30s and 40s have been forced to take matters into our own hands to protect our financial present and our future. But if you're like me, maybe you never thought about life insurance. I've always felt invincible. But I got married recently and I had to start taking these things seriously, which is why I've been shopping for life insurance with SelectQuote. to give my family some financial certainty in a world that isn't. I've been surprised that policies are cheaper than I thought.

21:42The younger you are, the cheaper it is. Like double the price if you wait to buy a decade from now. If you're new to life insurance, you're not alone. For over 40 years, SelectQuote has helped more than 2 million Americans understand their options and get the coverage they need. Over$700 billion in coverage and counting. As a broker, their mission is simple. to find you the right insurance policy at the best price. SelectQuote takes guesswork out of finding the right life insurance policy. You don't have to sort through dozens of confusing options on your own. Instead, one of their licensed agents will find the right policy at the right price for you, comparing plans from trusted, top-rated insurance companies to find a policy that fits your health, your lifestyle, and most importantly, your budget.

22:26And they work for you for free. Life insurance is never cheaper than it is today. Get the right life insurance for you for less and save more than 50 % at selectquote.com slash 404media. Save more than 50 % on term life insurance at selectquote.com slash 404media today to get started. That's selectquote.com slash 404media.

22:59all right and we are back emmanuel we'll start with you the headline of this piece you wrote is inside the telegram channel jailbreaking grok over and over again so on the on the podcast last time we spoke about Jason's piece and Grok and how it's been using to undress people, put people in bikinis, you know, also some straight up child sexual abuse imagery as well. So how does this Telegram channel differ to what we were sort of talking about last time? The short version is that the user's there and the prompts they use to create images with Grok that it's not supposed to create are much more sophisticated.

23:47The prompts that we saw on X were very straightforward. They were simple texts like, put her in a bikini, take her clothes off, things like that. And I think we can say now, based on the reaction from Musk and X and mostly the things that they are not doing in response to this, that they consider this to be pretty much a legitimate, okay use of Grok. On the Telegram channel, the reason it exists, it is part of a larger community where people create non-consensual AI-generated images of real people. and there are various rooms within the community where people focus on different celebrities. They focus on different types of what they call edits, which are using Photoshop, using AI to manipulate images.

24:57And then there's one channel that is just dedicated to prompts for different AI image generators. and basically how to find loopholes within them in order to produce non-consensual images with mainstream tools that are not supposed to do that. We've reported on this thing many times. The most notable example was at 2014 when this image of Taylor Swift went viral. People were using Microsoft's designer tool in order to make that image. And the loophole there was basically misspelling her name and describing sexual acts without using any explicit terms, which is kind of like the first line of defense that AI image generators have to prevent abuse.

25:47But it is a primitive, not very good way to stop that. And those filters get better, but this community where thousands of people participate keep coming up with extremely elaborate, more sophisticated prompts to do that, which can produce far worse things than what we've seen on X. On X, we've seen some nudity, images of women in bikinis, and this is straight-up hardcore porn images as well as videos. Gotcha. Yeah, I'm glad you brought up the Taylor Swift one because it's interesting. Whereas before with that, you found this Telegram group and you sort of drew a through line between this group abusing a certain tool and then making those images and then sort of eventually leading out publicly onto Twitter.

26:49And obviously, that was a huge case at the time. I think people can continue to think about it. So there is a through line there. what's a little bit different here is like in some ways there didn't need to be a through line because the prompts on x are just hey grok undress this like it's barely a prompt for the for sort of for the public stuff but what you're saying is that in the telegram group there are still people who are actually making grok do way way worse stuff it's just that it's not sort of reaching x as the public it's not on the surface yet which i think uh is part of the deal and there's probably two reasons for that one is it does take time for this stuff to trickle out from these pretty closed uh groups on fairly obscure corners of the internet to the mainstream which is what happened with the Taylor Swift incident.

27:47So I think it's only a matter of time until you start seeing this type of thing in your feed. But then also the baseline for Grok is so bad that there's like less of a need, right? So it's like you can get Grok to do pretty bad things without trying hard at all, as you said. So I think a lot of people don't even consider the possibility that you can make far worse images if you just went looking for the right prompt. But if you did, you'd find them. Yeah. And I'm trying to think how to phrase this, but the fact that it's not the public version of Grocon X generating straight up hardcore pornography of people works in the technique's favor in it becoming basically a meme.

28:38Like it's abuse and it's horrible but it's taken on this new character on x where even people um sex workers who are selling their only fans or whatever it may be they're leveraging it or other accounts are sort of piling in like oh i wish grok could unblur this video and then grok does it and of course that's to drive engagement to an only fans page or whatever if people were using grok to generate really hardcore stuff, it might not become a meme. It would just be like, oh, this is horrible, horrible abuse. You see what I'm doing? Yeah, yeah. And I guess just a few more things to note about the group.

29:16Like I said, I think we can say judging on Grok's response that it's just not taking this problem seriously and maybe doesn't consider it a problem. Well, X's response. X's response, yeah. Sorry. But But you can see in the Telegram channel that Grok slash X slash XAI, they do have their limits in terms of what they would allow people to generate. Because the general flow of the chat is somebody comes in with a prompt that they find that works pretty well and is able to generate very graphic images. Everyone gets really excited. Everybody starts generating it. I don't know exactly what is happening on the Grok end of things because they will not respond to our request for comment.

Read the full transcript

30:09But I imagine that they start to notice that there's a lot of bad generations. And then they do some investigating and they close the loophole. I can tell that the loophole is closed because people start reacting and saying, like, this is working less. This is working less. Now it's not working at all. Then everybody's like, Grok is dead. Dead game. And then somebody comes in with a new exploit and the whole cycle begins again. The other thing to note is the channel is dedicated to prompts. And in the couple of years that have been in there, people go from app to app, right? Like we talked about Microsoft Designer.

30:53People have tried OpenAI. People have tried Google Gemini. people have tried all these different types of apps. I would say for the past three to six months, the entire channel is devoted to Grok. There is no other discussion of any other app because they're having so much success with this one app. Very interesting. You answered one of my other questions, so I'll just ask this before we move to Sam's piece. But what are people in the Telegram channel worried about when it comes to almost this wave of focus that people have on sort of the public Grok generation. Are they scared something's going to happen because of that?

31:36Yeah, I included a quote from the chat in the story where people were worried that all the attention that the images on X were getting were going to negatively impact the community and that either Grok would really lock things down or there would be increased interest in the exploit and somehow there would be like a crackdown on the community. And this is something Sam and I often see when reporting on this stuff and breaking news. I just went to check the community to see if there would be anything new to add to the podcast and it has been dissolved. It's unclear at this point. if it has been banned or that they decided to move on on their own.

32:29But a few days ago, someone came in, one of the admins came in, and they said, hey, follow this Twitter account, follow this subreddit, because we might shut this place down and we'll have to regroup somewhere else. And that has come to pass. The group has dissolved, the Telegram channel is gone, but I'm already in the new one, right? So the problem continues. And this isn't the first time that this has happened. I've seen it happen with this community specifically several times over. Right. Yeah. So it's not like they were moving to Reddit as a replacement for Telegram. They were using that as a bridge to get to the new Telegram because of course you need to point people there.

33:07That makes sense. Sam, let's just briefly talk about your one, which I think came after Emmanuel's. The headline was Masterful Gambit, which I do love. I hope people get the joke. I'm pretty sure they do. But I saw someone in my mentions being like, Masterful Gambit, you need to be more online. I'm sorry to digest that material. Masterful Gambit, Musk attempts to monetize Grok's wave of sexual abuse imagery. Really, really wild. What did X do here exactly when it comes to monetization of this stuff? Yeah, so all last week, and I'll try not to reiterate what you guys talked about last week in the pod also, but all last week, This is like the only story that was happening in AI specifically in any of my timelines was the AI abuse imagery coming out of Grok.

33:58So on Friday, when I logged on, I saw a couple outlets. It was The Guardian, Deadline, a couple others had headlines that were essentially Grok turns off image generator for most users after outcry over sexualized AI imagery. That's the Guardian. It's like Elon Musk restricted Grok, the images aren't able to be made anymore, following all of this outcry against the bad uses of it that we just talked about. So I was like, well, that's interesting and probably not right, knowing how this platform operates and how its owner operates. So it turns out that actually what was happening was Grok was giving a reply to people who weren't X premium or like paid X users saying that image generation and editing were turned off and limited only to paying subscribers to X.

34:59and then it would give a link to how to subscribe, which is not turning it off. And it's definitely not some kind of like accountability or mea culpa to the fact that it's been generating these horrific images. It's just pushing people to pay for it. And I would say, I mean, I was looking at like the searches for like Make Her and Bikini and, you know, Clear Tape, the things that Emmanuel was just saying on Twitter where people were asking Grok to do that stuff. And it was like multiple times a second people were asking for this. So obviously Elon said, hey, here's a convenient way or someone on.

35:37I mean, I say Elon is shorthand for the ex monetization team who should also be quite ashamed for all of this. But the move is to just push people to pay for the service, which is something that Elon's been trying to do specifically ever since he bought the thing is get people to pay for this. X premium service, which gives you more booster replies and things like that, gives you a little checkmark, whatever. So yeah, that was absurd to me that anyone would report that it was turned off. And then pretty quickly after that, The Verge wrote a really good story about how it's not actually turned off.

36:19I was in the middle of writing mine when I saw that. So you can pay$8 a month to still get these images generated in your feed and show them to everyone is kind of a crucial part of this. It's not like only premium subscribers can see the images that are generated by Grok by other paying subscribers. Everyone can see them. They're still in everyone's feed. So what is it really? It's not limiting any harm. It's not actually doing anything good or useful it's just making more money for the platform and as of i just checked while we were talking um it's not the case anymore anybody can make these again uh it's not giving that reply anymore yeah i mean it's it never went away um oh my camera just died um it never went away it's always been this way um and now it's um it's fully back for people who uh don't pay for the platform you can ask it to put her in a bikini or whatever other vile shit you want and then a lot of the worst stuff is obviously on Telegram like Emmanuel said and on the app itself which I've been trying to break for about a week with my own images but yeah that's where the really nasty shit is so yeah yeah I mean from all the reporting we've seen recently musk has a real problem with sort of the blowback that this bikini grok wave has received like he fundamentally seems doesn't believe in it with his fundamentally misguided ideas of what free speech is all of that which we don't really need to get into he seems to have a kindergarten kids like understanding of what that actually means but it from what you just said it looks like well, we'll just quickly move it to paid subscribers as a band-aid.

38:12But actually, no, I don't want that. I want to now go back. And the UK may be investigating it. I think there's some Ofcom stuff as well, which is sort of this government body in the UK investigating. Still in the app stores, right? I mean, just very, very briefly, you pinged Apple and Google, right? I think asking them about this. Did they ever get back to you? No, of course not. No, they never will. And it's these apps... this particular app, Grok, is all over. It's really popular on Apple and Google on their app stores. And Apple and Google very specifically forbid this type of content. Porn in general, but also non-consensual pornography is against the rules.

38:55So yeah, I guess it's okay if it's a really popular app, which is usually the MO that we see in the app stores. It's like if it's making them money, it can stay. yeah or owned by you know a very very rich guy and then he'll be allowed to stay can I just ask the yes go ahead please the incident has pulled me back to X just to look at it in a way that I haven't in a few months and I shouldn't be surprised and yet I was surprised by how God like I don't even know how to describe it. It's as if you're swimming in the outbrain gutter ads of the internet. It's just very extreme, very pornographic, and in this case, non-consensual.

39:53And it's like, am I the only one who is not looking at X regularly at this point? I still do. check it yeah i think it's a it was a good reminder that it's like i don't want to and it's not useful for me very much because i don't know i don't know any like interesting people who post interesting things there but i it was kind of a good reminder to go back and look at it because a lot of people do and it's just a obviously a very different picture of the world than what you get on BlueSky or Mass.R or whatever. I mean, I look at it for what the fuck is going on in this world reasons, usually for articles.

40:37But in my spare time, I do not hang out on Twitter in a way that I used to. Yeah, absolutely. For the best, I think. All right. We'll leave that there. If you're listening to the free version of the podcast, I'll now play us out. But if you are a paying 404 Media subscriber, we're going to talk about a really, really wild story from Jason about Flock and police inadvertently exposing millions of license plates that they were investigating, and of course, by extension, people. You can subscribe and gain access to that content at 404media.co. As a reminder, 404 Media is journalist-founded and supported by subscribers.

41:20If you do wish to subscribe to 404 Media and directly support our work, please go to 404media.co. You'll get unlimited access to our articles and an ad-free version of this podcast. You'll also get to listen to the subscribers-only section, where we talk about a bonus story each week. This podcast is made in partnership with Kaleidoscope and Alyssa Midcalf. Another way to support us is by leaving a five-star rating and review for the podcast. That stuff really helps us out. This has been 404 Media. We'll see you again next week.

42:01Transcription by CastingWords

42:20All werden pay attention in the bills and income status We'll see you next time.

From the publisher

We start this week with Joseph’s article about Webloc, a tool ICE bought that can monitor phones in entire neighborhoods. After the break, Emanuel and Sam talk about their recent coverage of Grok. In the subscribers-only section, Jason explains how police inadvertently unmasked millions of their surveillance targets through a Flock redaction error.

Timestamps:
0:00 - Intro
2:50 - First Story
23:00 - Second Story

YouTube version: https://youtu.be/rurJo6vPhUY

Inside ICE’s Tool to Monitor Phones in Entire Neighborhoods

DHS Is Lying To You

Inside the Telegram Channel Jailbreaking Grok Over and Over Again

Masterful Gambit: Musk Attempts to Monetize Grok's Wave of Sexual Abuse Imagery

Police Unmask Millions of Surveillance Targets Because of Flock Redaction Error

Learn more about your ad choices. Visit megaphone.fm/adchoices

More from The 404 Media Podcast

All 164 episodes
The ICE Tool That Tracks Entire NeighborhoodsThe 404 Media Podcast · 43 min
Listen in VO