In short
The 404 Media Podcast: Episode Overview - Here’s What Palantir Is Really Building
Podcast Description The 404 Media Podcast is a weekly show featuring hosts Joseph, Sam, Emanuel, and Jason, who discuss the latest stories published by 404 Media—an independent, journalist-owned digital media company. The podcast explores how technology affects our world through investigative journalism, breaking news, and insightful discussions.
Episode Summary Title: Here’s What Palantir Is Really Building Key Discussions:
- Palantir's ELITE Tool for ICE
- Overview of ELITE
- A tool developed by Palantir for ICE (U.S. Immigration and Customs Enforcement) to identify neighborhoods for raids.
- It compiles data to create lists of individuals (targets) in specific areas for potential detention.
- Includes a geospatial interface allowing ICE agents to visualize and select targeted locations.
- Confirms the connection between Palantir and ICE’s operations, emphasizing the role of technology in immigration enforcement.
- Sources of Data
- Addresses are collected from various government databases, including the Department of Health and possibly Thompson Reuters' CLEAR service.
- This information is crucial for ICE to locate undocumented individuals.
- Case Study: Oregon
- Discussion of a legal case involving a woman (MJMA) detained using ELITE, showcasing real-world implications of the tool.
- ICE officials testified about using ELITE to locate densely populated areas with potential undocumented individuals.
- Broader Implications
- The episode emphasizes how ELITE is part of ICE's broader goal of mass deportation through advanced surveillance technology.
- AI Influencers and Celebrity Defamation
- Introduction to AI Influencers
- AI-generated personas are creating fake scandalous content featuring celebrities on platforms like Instagram.
- Examples include AI images depicting celebrities in compromising situations, leading to potential defamation.
- Monetization of AI Influencer Content
- Creators capitalize on viral AI content by directing users to platforms like FanView, where users can purchase AI-generated adult content.
- The phenomenon reflects a hustle culture, where users generate income through AI-created personas.
- Audience Reactions
- Mixed reactions from users: some recognize the content as AI-generated, while others engage with it without skepticism.
- The conversation highlights the growing awareness among internet users regarding AI content.
- Legal Considerations
- Discussion around potential legal action by celebrities against creators of AI-generated content, referencing previous cases like LeBron James's cease and desist.
- The episode speculates on the likelihood of lawsuits as a means to combat this emerging issue.
Key Takeaways
- Technology's Role in Surveillance: The episode underscores how Palantir’s technology is actively being used in immigration enforcement, illustrating the intersection of tech and policy.
- Emerging Ethical Concerns: The rise of AI influencers raises significant ethical questions regarding consent, defamation, and the impact of technology on personal reputations.
- Potential Legal Repercussions: As AI-generated content becomes more prevalent, celebrities and public figures may explore legal avenues to protect their likenesses and reputations.
Conclusion This episode of The 404 Media Podcast provides a deep dive into the implications of surveillance technology in immigration enforcement and the rise of AI-generated content that can lead to defamation. The discussions highlight critical issues surrounding technology, ethics, and the law, making it a thought-provoking listen for audiences interested in journalism and social commentary.
Additional Information
- Listen to the Episode: [YouTube version](https://youtu.be/b-QHWpqjD-E)
- Support 404 Media: Subscriptions available at [404media.co](404media.co) for ad-free content and bonus episodes.
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Chapters
Tap a time to open that second in VOHost and Co-Founders Introduction
1:10 to 1:43
Meet the host Joseph and co-founders of 404 Media.
“And I'm actually going to be doing a little bit of reporting while I'm here.”
Jason's Conference Experience
1:43 to 2:18
Jason shares his experience at an AI and Journalism conference in Kenya.
“I've met a lot of really great Kenyan journalists, including a lot of people who focus on the same things that we do, like surveillance and privacy and things like that in a much different context.”
Introduction to Palantir's Elite Tool
2:18 to 4:27
Discussion on the Palantir app called Elite and its use by ICE.
“the Palantir app Ice uses to find neighborhoods to raid.”
Understanding Elite's Features
4:27 to 6:51
Explaining how Elite functions, including geospatial features.
“administration and on ICE's activities were about some legacy contracts that Palantir had with the Department of Homeland Security to build these big databases for the government.”
Testimony from the Oregon Case
6:51 to 10:42
Details of a specific case involving ICE's use of the Elite tool in Oregon.
“you can then click on an individual person, bring up their dossier, and that will have their name, their date of birth, a photo of them, if that's available, and their current or known address.”
Investigative Insights and Challenges
10:42 to 14:02
Insights on sourcing and challenges faced in investigative journalism.
“So what is the case in Oregon and what did you learn through that testimony?”
Understanding Palantir's Role in Tech
14:02 to 18:10
Learn about Palantir's involvement in enhanced leads identification for enforcement.
“for these court transcripts, which is a lot of money.”
AI Influencers and Celebrity Scandals
22:42 to 28:02
Explore the rise of AI-generated influencers and their impact on celebrity culture.
“Emmanuel wrote this one, as I said, and the headline is Instagram AI Influencers Are Defaming Celebrities With Sex Scandals.”
The Rise of AI-Generated Adult Content
28:02 to 29:39
Explore the monetization strategies behind AI-generated personalities on platforms like FanView.
“They say that and then drive them to an OnlyFans or an OnlyFans equivalent.”
Current Trends in Social Media Metas
29:41 to 31:16
Discuss the latest trends in social media engagement and how they impact visibility.
“And the story is really what we call a story about the current quote-unquote meta in the same way that people discuss the Twitch meta.”
Show all 14 chapters
Audience Reactions to AI Influencers
31:17 to 34:17
Analyze how audiences perceive AI influencers and their content on social platforms.
“Emmanuel sends me terrible things all the time.”
Investigating Responses from AI Influencer Followers
34:18 to 36:47
A look into attempts to reach out to individuals engaging with AI influencers and their motivations.
“One of them is, I would say, thank God, people saying, clearly this is AI.”
Legal Implications of AI-Generated Content
36:48 to 42:04
Examine the legal ramifications surrounding unauthorized use of celebrity likenesses in AI content.
“If that's your dad, please email me because I want to know what that's going to be.”
The Legal Landscape of AI and Likeness Rights
42:04 to 42:56
Explore the implications of AI on likeness rights and celebrity responses.
“from stealing people's likenesses at the very least.”
Transcript
Automatic transcript. May contain errors.0:03Hello, and welcome to the 404 Media podcast, where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist founded company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. gain access to that content at 404media.co. I'm your host, Joseph. And with me are all of the 404 Media co-founders. The first being Sam Cole. Hey. Emmanuel Mayberg. And Jason Kebler.
0:40Emanuel:Hello, hello. Jason, do you briefly want to explain why your background looks a little different? Why there's a mosquito net behind me? Yeah, I'm on Lamu Island in Kenya. which is not where I normally record. I was invited to take part in this AI conference, AI and Journalism conference, and specifically how to keep humans relevant in the age of AI. So I've taken a very long trip, very lucky to be here. And I'm actually going to be doing a little bit of reporting while I'm here. Kenya is like a hotbed of data labeling, which is like content moderation for ChatGPT and AI tools, as well as training a lot of these tools.
1:29Emanuel:So I'm meeting with some folks here. We'll probably record some podcast material for the future. But this is... It's been a really very fun trip for me. Very lucky to do it. I've met a lot of really great Kenyan journalists, including a lot of people who focus on the same things that we do, like surveillance and privacy and things like that in a much different context. So hopefully we'll be able to have a little bit more info about that on future episodes and talk to some of them about some of the work that they're doing while I'm here. Yeah, that sounds really, really interesting. And do you want to take us through this first story?
2:12Emanuel:Yeah, so let's first talk about this story. It's called Elite. Elite is in quotes. the Palantir app Ice uses to find neighborhoods to raid. This is kind of the latest on our reporting on the surveillance tools that Ice uses and specifically the latest in a series of stories that you've been doing about Palantir. Do you just want to give us some of the background about how you got started on this story before we jump into specifically what Elite is? yeah so as you say we've been covering ice and its technological backbone and palantir stuff for a while you know months ago we first revealed that palantir was getting much closer to ice with a leak through palantir and some public procurement records i looked at with this story i was hearing about something called elite and i didn't really know what it was there were just rumblings of it And then eventually, I got this user guide, which explained in much more detail what it was, what it was capable of.
3:23And of course, we'll get into that a little bit shortly as well. And then also this testimony from Oregon, which, again, we'll get into more specifics in a minute, but ICE and Customs and Border Protection officials talking really explicitly about not just this elite tool, but also mobile Fortify, the facial recognition app that we've covered, and some other stuff as well. and then of course I found the Palantir connection as well which frankly when I figured that out it was oh I have to write this quickly because this is uh this is taking on a new sense of urgency that not there wasn't important before of course but as soon as you realize Palantir is involved I mean I think people are interested in that uh for better and for worse sometimes it seems sort of this omnipotent sort of bogeyman that can do anything.
4:14But I think it's really important to know what Palantir is doing during the second Trump administration. So this story sort of brought all of that together.
4:24Emanuel:Yeah, I mean, I think some of our first reporting on the Trump administration and on ICE's activities were about some legacy contracts that Palantir had with the Department of Homeland Security to build these big databases for the government. soon after things started there was this reporting about something called immigration os and something that was never really clear to me at the time was like what was or what is immigration os like we had these contracts we had this name immigration os we knew that it probably had something to do with aggregating data and providing it to ice but with elite which you know there's all these names floating around like i I don't actually know if Elite is part of Immigration OS or if Immigration OS has become Elite or whatever.
5:12But we now know about this program called Elite.
5:17Emanuel:So that is being used by ICE right now. What is it and what is its purpose? Yeah, it's a dedicated tool for ICE to basically acquire lists of targets or people, obviously. And I should say that maybe as a force of habit, I'll use the word target just because that's sort of the terminology that Ice uses in the user guide. But of course, I'm talking about people, individuals, human beings. So I apologize if I lean on that term. But I remember when you were editing, Jason, you were like, it's okay to use target a couple of times. But do I not use it during the entire copy? Which I did because I was just referencing the material.
6:00But just keep that in mind. So it allows us to draw a list of people from a particular neighborhood, drill down on them, and then export these lists of people that they want to target or detain. It's meant to be basically the entire lifecycle from, well, let's find a neighborhood. Okay, well, let's find who lives there. Well, now let's get a list of those people. Now let's have a supervisor approve this operation and then go out. And to make it a little bit more tangible, one part of it is a geospatial interface, or rather it's a maps interface, right? So ICE can, kind of like the tool we spoke about last week, which was different, to be clear, they can draw a shape on a map, you know, a circle or a rectangle or whatever.
6:47The map will be populated with all of the people that ICE may wish to target there. you can then click on an individual person, bring up their dossier, and that will have their name, their date of birth, a photo of them, if that's available, and their current or known address. And this is the most important part, I think, because it also comes with a confidence score. Let's say it will be 88.2 out of 100 or 79 out of 100. And that is based on where the data is coming from and its recency. And I've mentioned this before, but as we keep covering ICE and its technology and its data sharing agreements, one thing comes up over and over and over again.
7:38It's addresses. ICE wants them. It is like gold for them where they can go, we believe this person is at this location or this apartment block or this particular flat or apartment or whatever. that is what they're trying to get. We've seen that where they've hired bounty hunters and PIs to verify addresses. So this elite tool essentially brings all of that together. Now, I don't mean all of the data. We'll get into where some of it comes from, but I don't know where it's all coming from necessarily. But it's to turn that data that ICE has going into this tool into something they can actually act on.
8:18It's not like an abstract technology product over here doing something that we can't really visualize or think about or sort of touch or understand. This is what it seems ICE is using to actually plan where to go and who to pick up.
8:37Emanuel:Yeah, I mean, that's part of the headline where it's like the Palantir app ICE uses to find neighborhoods to raid, which I guess we'll talk a little bit more about this case in Oregon in a minute. But was there anything else from the user guide that you sort of wanted to highlight before we get into that? Yeah, I think it would just be some of the data sources. So again, with the addresses, there was a little bit in the user guide that said, hey, the address confidence score is this new feature in Elite. And those addresses come from, it says, the Department of Health, which is pretty interesting.
9:15A lot of people aren't going to assume that data from the health department is going to be used for immigration enforcement. But obviously, this is a different world now. There's mentions of USCIS, which is part of DHS as well, right? And then there's also a mention of CLEAR, which it is hard to report when companies don't respond to requests for comment. We don't think it's CLEAR, as in the system lets you go through the airport quicker, right? It seems to me that it's probably Thompson Reuters data product called Clear. Reuters has sold tons of data to the government in the past. They regularly contract with ICE.
10:00We've covered that before. So it looks like they gain data from that as well. That being said, I don't know if that's all of the data sources. Like, oh, remember all of that stuff about the IRS giving addresses to ICE? Could that be fed into here? I don't know. like potentially but those were the ones that were mentioned in the user guides at least yeah so um
10:24Emanuel:i mean i i know you had sort of like sourcing on this and that sort of thing but the government actually has talked about the use of elite uh in this case in oregon and you got testimony from that case from i believe it was an ice officer or maybe a dhs official. So what is the case in Oregon and what did you learn through that testimony? Yeah. So the case in Oregon is about a woman whose initials in the court records are MJMA, if I'm recalling correctly. And she was detained along with around 30 other people during an immigration raid in Oregon in October, I think. She was briefly detained, taken to a facility then released with no conditions by ICE in the testimony she said that she entered the U.S.
11:18repeatedly on a tourist visa she's now seeking asylum so that's sort of the context of this case and she's suing the government because to protect her constitutional rights I believe that's all well and good that's the context but there's a ton of interesting stuff in the testimony of that case from ICE officials, from officials from Customs and Border Protection as well. And there are mentions of elite in there where they say, this is the ICE official, I believe, yes, you use elite to find densely populated areas. Like you're not going to go to a house that just has one pin on it. You're going to, say, go to an apartment complex with a load of pins on it because there's probably going to be more people to arrest or detain there.
12:07And in text messages... Sorry, so when you say density, though,
12:11Emanuel:when you say density, it's not density of just like people living there. It's like density of suspected, undocumented immigrants. It's like density of potential targets, right? Yes, density of potential targets. That's absolutely it. Now, you know, there are, of course, some questions it raises such as, well, how reliable is the data? that ICE would have on undocumented people. If they're getting data from, you know, DHS and USCIS, isn't that going to be people legally in the country? And it's going to be very case by case. And frankly, it's quite messy. But they have a lot of data here to go on.
12:51And that's apparently what they're using it for. So that case, MJMA's case, It was actually reported briefly by local Oregon media in December, because there was a hearing that month. And there was a very brief mention of the mobile Fortify app being used on that woman and returning the wrong name twice. You know, we picked up on that and fleshed that out with some additional reporting as well. And of course, it tied into the elite Palantir piece here as well. and I should just say that you know we left Vice well because the company was falling apart obviously and it was going bankrupt but one of the catalyzing events for me to you know leave and then make this with all of you was that one day I tried to get some court records for my job at Vice looking up some court case which you do every single day and it's supposed to cost 10 cents and couldn't do it because Vice wasn't paying his credit card bills, apparently.
13:59In this case, I email the court, I get a quote for$650 for these court transcripts, which is a lot of money. That's just how much transcripts cost. But we could just do it because we own the company now. If we're like, it's probably worth paying$650 to get this very, very important transcript, which has already resulted in two really, really important stories. It's just refreshing that we're able to do that. I just wanted to mention that.
14:28Emanuel:And we published the transcript too, so you don't have to pay$600, which is nice. Yes. So you mentioned Palantir a lot. Palantir's in the headline. We know Palantir made this. But the user guide you have actually doesn't say that, I don't think. So you had to sort of establish that link. Can you talk how you did that? Yes. So in the testimony and when I heard about Elite, it was just spelled either as a single word or an acronym or just by itself. But this user guide spelled it out. And it's a really, really complicated one. I'm actually pulling up the Wikipedia. Somebody has added our article to the Palantir Wikipedia because I can't access the site right now on this screen.
15:17But it stands for Enhanced Leads Identification and Targeting for Enforcement. That is the spelled acronym that was in the user guide. And I thought, well, what exactly does that relate to? So I googled it, literally that phrase. The only result on the internet at the time was an additional piece of information in a Palantir contract, where it said Palantir was being paid$29.9 million for work on a few other things, but including configuration of enhanced leads identification and targeting for enforcement. So it is clear that Palantir is working on this in some capacity. Now, did Palantir originally design the system?
16:05Did Palantir build it from the ground up? Are they just setting it up and then somebody else is going to take over? That isn't clear. It just says they provided some sort of continuing configuration on this system. But for me, that's an incredibly strong link where I've never seen the acronym anywhere else except in this Palantir contract. I mean, I think the connection is pretty clear.
16:28Emanuel:Yeah, I guess just to round this out, what comes next? Like, what's your takeaway from this? And then, I mean, presumably we're going to continue to follow the use of this and other related tools. But yeah, what do you think this shows? I think for what comes next, presumably they'll put more data in there. Part of the user guide says it's integrating new data sources. So it sort of leaves open the possibility of adding more. But my main takeaway is that I kind of touched on it a little bit earlier. but we've been covering for months that there's all of this abstract technology being built and it's sort of happening over there in the background or in the shadows.
17:16And even when we've got leaks from inside Palantir describing the company's work with ICE, it's always been quite abstract and obscure. And you can't really put your finger on it because it's just dressed in all of these very confusing acronyms and technical jargon and all of that. This story, I think, is the clearest example yet of what Palantir is building and what ICE is actually doing on the ground. It is clear that this tool is part of ICE's mass deportation effort, because that is literally what it's designed for. And now we've seen that there is a through line between the tech that Palantir is making and what is happening on the ground.
17:59It's a combination of all those things. It's the user guide. It's the testimony from Oregon. It's the public procurement records. And yeah, I just think this is the clearest link between the company and what is actually happening.
18:10Emanuel:All right. Let's leave that there. Yeah. Yeah. And when we come back after the break, we're going to talk about one Emanuel worked on about... Well, maybe just wait and see. We'll be right back after this.
18:35It's funny what people will do all to hide the smell of the litter box in their house. I've seen some very strange purpose-built contraptions in my time. I knew a guy who'd built a whole cabinet system out of wood just to contain his litter boxes and keep the smell from getting out of there. I don't have that problem. You know why? Because of Boxy. The PRO in Boxy Pro stands for probiotics, which stop the bacteria that cause odors so you never smell your litter box ever again. Boxy Pro keeps the box continuously odor-free infinitely. Just got to remember to scoop. All you do is top off the litter.
19:16You never have to dump out the whole box. Amazing clumping power that makes scooping easy. I've got three cats, and I didn't have to build a special cabinet like my buddy because of Boxy Pro. If you're tired of switching litters looking for the one, get Boxy at B-O-X-I-E-C-A-T dot com. It's the last litter you'll switch to. Enjoy 30 % off with code 404media at boxycat.com forward slash 404media. That's B-O-X-I-E-C-A-T dot com forward slash 404media.
19:53Emanuel:January is short, and February is even shorter. If you're going to resolve to change your gear, you better start now. That's why now is the time to start that business you've always wanted to with Shopify. Shopify has made it easy for us to set up and manage the 404 Media merch store. We're one of millions of entrepreneurs who have made the leap. And with Shopify's easy-to-use tools, you can get a professional, well-designed store up and out the door in just minutes. Shopify gives you everything you need to sell online and in person. You can build a beautiful store with customizable templates and even create email and social campaigns from the same dashboard.
20:29Emanuel:As you grow, Shopify grows with you. Handle more orders, expand to new markets, add new products, and handle the backend all from the same dashboard. In 2026, stop waiting and start selling with Shopify. Sign up for your$1 per month trial at shopify.com slash media. That's shopify.com slash media. Here you're first, this year with Shopify by your side. At 404 Media, we know your personal information is everywhere online. Data brokers crawl the web to build profiles, selling your home address and phone number to anyone with a credit card. Trust us, we know this better than anyone else. This exposure leads to risks like doxing and identity theft.
21:12Emanuel:That's why we recommend Delete Me, a hands-free service that removes your personal info from hundreds of data broker sites. Their experts find and remove your information, providing a detailed privacy report within seven days. Delete.me monitors these sites all year long to ensure your data stays removed. We feel like we're at increased risk because of the types of stories we cover and the public nature of our job. But the truth is, identity theft, threats, and harassment can happen to anyone. I've been doxxed. In fact, lots of us have been doxxed. And each time, DeleteMe has helped us pick up the pieces, get our data removed from shady websites, and otherwise clean up our online presence.
21:50Emanuel:I've been using DeleteMe for years, and every quarter I look forward to seeing what new, interesting, and horrifying websites they've had my data taken down from. It's kind of a perverse little game. Take control of your data and keep your private life private by signing up for Delete Me. Now at a special discount for our listeners. Get 20 % off your Delete Me plan when you go to joindeliteme.com slash 404media and use promo code 404media at checkout. The only way to get 20 % off is to go to joindeliteme.com slash 404media and enter code 404media at checkout. That's joindeliteme.com slash 404media, code 404media.
22:42All right, and we are back. Emmanuel wrote this one, as I said, and the headline is Instagram AI Influencers Are Defaming Celebrities With Sex Scandals. Honestly, I had to read the lead like a couple of times just to get my head around it because I'm sure it's very clear to you, Emmanuel, very simple, but I've been out of the loop of what these AI influencers are doing for a minute and this kind of spun me out. To start, can you give us some examples of some of these images that are being posted? Then we'll get into the how and the why. But what are we looking at when we're talking about this?
23:19Yeah, that's fair. It is confusing. I kind of had the same experience when I tried to write it down. uh we all look at instagram and i think jason and i especially get a lot of ai generated content and ai generated influencer content and i look at it every day so i'm steeped in it and it all seems very obvious and regular to me until you try to describe what is actually happening and then it's extremely weird and convoluted. But to your question, I think a good example for one of these images is, or the post rather, is it's an Instagram reel. You will see a very realistic looking image of a woman courtside at a basketball game taking a selfie with LeBron James and it will say how it started and then it will play a song and cut to an image of the same woman and LeBron James in bed, under the cover, with their shirts off, all sweaty, hair tussled, obviously seeming like they've just had sex.
24:43And again, very convincing image. Sometimes it's an image, Sometimes it's a slightly animated video. And that is just one example of many. Like any celebrity that you can imagine, there is a video of them with the exact same format. And they are all over Instagram Reels. Gotcha. But just to clarify, obviously, the Mike Tyson or the LeBron James or whoever the celebrity is, they're not real. obviously that is an AI generation of that celebrity but also the other person who is allegedly taking a selfie with LeBron or whoever is also not real I think that is as you say you know somewhat obvious to all three of you where you've looked at this AI stuff a lot but maybe people who are just coming to this might be scratching their heads being like wait the the influencer is not real either can you just briefly explain that yeah in the vast majority of cases, I think there are outliers and we can maybe talk about those, but in the vast majority of cases, the personality behind the account that is posting this content is itself an AI generated woman, female character, almost certainly created by a man.
26:12And Jason and I have done a ton of reporting about the people who create these accounts and looks very realistic. But yes, entirely AI generated. Gotcha. Well, on that, who is making these AI images then? Is it grifters? Is it people trying to make money? Is it people just looking for cloud? What's a profile of somebody who is making this? Yeah, so I think it was a year ago now, Jason and I wrote this article and have done a few follow-ups since. But I think we're due for a refresher or a couple new stories about what's happening in this space. But it is a community of people who are kind of like in this hustle bro culture, get rich quick scheme culture who are just into this latest fad in the same way that they are into drop shipping or cryptocurrency or what have you.
27:21You can plug into this whole community that exists on Instagram and Discord and other places and learn how to AI generate a personality, gain traction with that personality on Instagram, and then funnel users via that attention to other places online where you can monetize that same AI generated personality by selling AI generated nude photos of them. So that's the play. It's to basically entice people with, oh, look, I have a sex tape with Mike Tyson or whatever. I know they don't say sex tape necessarily, but you see what I'm getting at. They say that and then drive them to an OnlyFans or an OnlyFans equivalent.
Read the full transcript
28:11Is that the play? Yeah, that's how you monetize the attention. It's like you funnel people to specifically FanView, which is an OnlyFans competitor. OnlyFans so far has pretty strict limitations on AI-generated content. They do not want it on the platform, which I think is good and surprising, but we'll see how long that lasts. And FanView basically offers the same service as OnlyFans, but has a more permissive policy, which essentially boils down to you can have a totally AI-generated personality with AI-generated nudes. You just have to disclose that it's AI-generated. So the flow is you're on Instagram, you see an AI-generated woman, you're interested, you go to the bio, you find a link to FanView, you click through, and then you can buy nudes or videos from this AI-generated character.
29:15and it will say on FanView that they are AI-generated and I think that's in response to our reporting because we flagged this happening to FanView a few times where those people weren't disclosed as AI. So now that tag is there, but it's at the bottom. It's in the fine print that it's an AI-generated personality and I think it would be very easy to miss. But yeah, that's generally the flow. And the story is really what we call a story about the current quote-unquote meta in the same way that people discuss the Twitch meta. It's like people who stream on Twitch or in the Twitch culture talk about what are streamers doing right now in order to get attention on Twitch.
30:05And it's the same thing on Instagram. And we've seen several metas. And this is just the latest one. And it is extremely common and extremely viral. I just had that classic thing where you see one of these posts. And then because of the hashtags, because of the music that people use on the posts, and because of my engagement with it, I just got an endless stream of these and seeing all these celebrities. I was just up one night looking at this on my phone. and we do this thing on Slack where we send ourselves messages just to remind ourselves stuff to blog in the morning. And I just kept messaging myself like 20 Instagrams in a row of this exact thing.
30:51Yeah. That will bring me to my other question. But before that, I just want to ask Sam and Jason, have you... Either of you come across this organically? I feel like Jason's answer may be skewed because he's probably in the really bad bits of Reels. But Sam, have you come across... Is it like broken containment and come across your feed at all, Sam? Not really, which is surprising considering I assume the algorithm is basing my algorithm on my connections algorithms. Emmanuel sends me terrible things all the time. So it hasn't caught on to that yet. I don't know why. I haven't seen it just in my normal personal scrolling.
31:31So, yeah.
31:33Emanuel:Jason, did you see it? Are you talking about the celebrity stuff in particular? Yes. So I hadn't seen this until Emmanuel sent it to me. But a huge amount of my reels in general are AI-generated influencers. I feel like at this... Because I follow them, I buy their products. No, I mean, it's because I've written a lot about them. But I feel like there's all these niches within the AI influencer space. like Emmanuel has written about the ones where they've been altered to look like they have Down syndrome, for example. But I am now getting a lot of like how to make your own vibes. Like the hustle bro thing.
32:27Emanuel:And so it's like, I'm getting a lot where it's like starts as an AI generated woman and then the woman morphs into the man who made her. which is like I get a lot of those jarring yeah very creepy creepy and I think I mean Manuel mentioned it but I think like that will probably be our next follow up is sort of the like world that has I don't know of like men on yachts standing next to their AI influencer who they are like this woman doesn't exist but she made me rich yeah so the celebrity ones they have gone viral uh you know they're pretty maybe even popular is the wrong word they're just viral i'll just stick with that term right and a lot of people seemingly are seeing them do you think emmanuel people actually believe it like do they actually believe that oh my god lebron james did xyz or that like does it even matter whether whether people believe or not So they're definitely viral.
33:35I gave a few examples in the story, and I just checked on it while we were talking. And yeah, many of these posts have millions and millions of views. I think the largest one I've seen is 20 million views. That was on a John Jones video showing John Jones in one of these scenarios. John Jones being an MMA fighter. I don't know why that one took off, but very popular. whether people believe it. Sam and I have discussed this a few times. It's something that we've attempted to report out. It's a more complicated question than you might imagine. If you go into the comments, you see broadly two types of posts.
34:20One of them is, I would say, thank God, people saying, clearly this is AI. And I think that shows that, generally speaking, internet users are becoming more educated about this type of content and they can just sort it out they can see that it's not real but then the other bucket of posts is people just like posting fire emojis and saying oh my god you're so beautiful uh please post more i love you stuff like that and like are those bots are those actual people uh is it like sock puppets that are just engaging with the account in order to boost its numbers it's it's hard to say i i would i would i would guess that there is a non-negligible number of people who do believe this stuff but i i have we have not been able to like quantify and scientifically prove that's the case um sam maybe just briefly on that because as emmanuel said i remember this as well like you've tried to reach out to people, right?
35:27The people who look like old men who are replying to an AI influencer, like, wow, you're so beautiful, you're so gorgeous. I think you tried that. Did you ever hear back in general? I sent so many DMs to so many people's dads that I gave up. It was crazy. And I think a lot of them probably were also bots. It's hard to tell, right? Because it's just like a picture of this guy who just looks like a normal kind of cringey guy and he doesn't have a lot of organic posts, but is commenting on every AI model's image on the platform. That could be a real person and probably a lot of them were real people.
36:14I didn't get anywhere with that. And it's something that I occasionally pick up. It's like when I see one of these floating to my feed and then I see a bunch of people commenting, I'm just like shooting off DMs to these guys. Maybe my DMs are going to like their requests or something and they don't know how to find them. Yeah, the most like actual response that I got from anyone was a guy just started going through my profile and liking all of my posts and didn't answer me at all. And just liked every one of my posts from the past like three years. And then I made my profile private. So I don't know.
36:47It's like, I guess if you know someone who's really into replying to AI models posts in real life, ask them why and send them my way. If that's your dad, please email me because I want to know what that's going to be. Calling all uncles, please reach out. Calling all child dogs, please email us now at 442.co. The one thing I'll say that it's good evidence that somebody is buying into this is that it is an ecosystem, right? It's like there are so many of these accounts, even if it's just a handful of people who are making a ton of these accounts. And as we've discussed, they all funnel eyeballs to places where they can be monetized.
37:33And if you look at FanView, those accounts do have subscribers. People do buy the images. I can't tell you for a fact that those people don't know or don't know that the images are AI generated but somebody is buying the images you know what I mean? And this wouldn't exist as a phenomenon on Instagram if that wasn't the case Yeah Maybe just the last thing I was going to ask is what do you think might happen now? And I bring that up Because I think when you posted the article on social media, maybe you mentioned something about you reckon these celebrities might start finding someone to sue and that could be meta.
38:20Maybe I'm misremembering what you posted. But do you think lawsuits might happen here? Because the celebrities didn't consent to this, obviously. Yeah, that's a very good question. The thing that actually caught my eye and eventually made me want to write this because I've been looking at it for a while is that I saw that post with LeBron. And as Jason broke the news a while ago, there was a very notable case where LeBron's lawyers reached out and sent a cease and desist to one of these groups that's creating AI-generated videos of him. In this case, it was like, I apologize. It always gets really stupid when we talk about this stuff.
39:03But it's like, videos of pregnant LeBron in jail with P. Diddy and stuff like that. And I think that probably trickled up to him. And he was like, okay, enough is enough. And they sent a cease and desist to that group. And then I believe Jason was in that Discord and they actually stopped making those videos. So I was like, okay, let's highlight that this is happening still. I'll bite in another format. And yeah, I think it's just a question of how spicy and annoying it gets for these celebrities until they do something. But I definitely think they have standing to do something. Whether they go after individual users or Instagram as a platform, I don't know.
39:48But it's like it's pretty like it's not nice and it's unambiguous. You know what I mean? It's like the accounts are not flagged like this is parody or I'm joking. It's like literally they're lying about having affairs with celebrities in order to gain clout and sell content that could be defamation. It's pretty bad. Jason, were you going to add to that?
40:15Emanuel:I mean, I was just going to say that, yeah, we have seen when there's even the tiniest bit of pushback from some of these celebrities in the form of a legal letter that they've stopped. and you kind of like hate to put the onus on the victims of this, but like John Jones and LeBron James and like, I don't know, the people who are being put into this are not... Like what's happening is not like the most graphic type of NCII that we've seen. Like it's not good. But at the same time, it's like I do feel some of these celebrities could make quite a difference by making a thing out of this. And I wish that they would because we've seen...
41:10Emanuel:We've seen women who've been deepfaked talk about this and talk about how it affects them. Celebrities have finally gone on the record and talked about this. And we know...
41:26the really horrible things that some less famous women have gone through when they've discovered
41:34Emanuel:deepfakes of themselves on the internet. Sam's done a lot of great reporting on that. But if LeBron starts threatening to sue these people, if Jon Jones starts threatening to sue these people, I think it would make a real difference, not just in terms of they would stop making deep fakes of LeBron James. But perhaps maybe the platforms would start thinking about actually moderating this sort of thing. Maybe some of the communities would shy away from stealing people's likenesses at the very least. I don't think this industry is going to go away unfortunately but i do think uh just like the vaguest of legal threats might actually do something and it's weird that we've only really seen that one from lebron james like a couple months ago but not much else yeah and as emmanuel alluded to like it's it's bordering on defamation or you know well i guess under u.s laws it probably wouldn't be um libel right but i can definitely see some celebrities not responding to this very well and jason totally agree with you that maybe it could be an avenue to some other change for frankly normal people uh as well we'll leave that there if you're listening to the free version of the podcast i now play us out but if you are a paying 404 media subscriber we're going to talk about a pretty big change around AI at Comic Con.
43:10You can subscribe and gain access to that content at 404media.co. As a reminder, 404 Media is journalist-founded and supported by subscribers. If you do wish to subscribe to 404 Media and directly support our work, please go to 404media.co. You'll get unlimited access to our articles and an ad-free version of this podcast. You'll also get to listen to the subscribers-only section, where we talk about a bonus story each week. This podcast is made in partnership with Kaleidoscope and Alyssa Midcalf. Another way to support us is by leaving a five-star rating and review for the podcast. That stuff really does help us out.
43:48Or please just tell a friend about us as well. This has been 404 Media. We'll see you again next week.
44:02Emanuel:Thought Sweetgreen was just salads? Think again. There's a new way to do sweet green. Wrapped and ready. These handheld wraps pack bold flavor and 40 plus grams of protein into something hearty, satisfying, and built for life on the go. From craveable sauces to satisfying textures, they're designed to keep you going without slowing you down. So put that fork down. Try the new wraps today in app or at order.sweetgreen.com. Available at participating locations only.
From the publisher
We start this week with Joseph’s article about ELITE, a tool Palantir is working on for ICE. After the break, Emanuel tells us how AI influencers are making fake sex tape-style photos with celebrities, who can’t be best pleased about it. In the subscribers-only section, Matthew breaks down Comic-Con’s ban of AI art.
0:00 - Intro
2:16 - ‘ELITE’: The Palantir App ICE Uses to Find Neighborhoods to Raid
22:45 - Instagram AI Influencers Are Defaming Celebrities With Sex Scandals
Subscriber's Story: Comic-Con Bans AI Art After Artist Pushback
YouTube version: https://youtu.be/b-QHWpqjD-E
Learn more about your ad choices. Visit megaphone.fm/adchoices
