A Massive Breach Reveals the Truth Behind 'Secret Desires AI'

0:03Hello, and welcome to the 404 Media Podcast, where we bring you unparalleled access to hidden worlds both online and IRL. 404 Media is a journalist family company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co. I'm your host, Joseph. And with me are 404 Media co-founders, Sam Cole. Hello. And Jason Kebler. What's up? as we're talking, I'm getting notifications that Jason is telling Emmanuel, I don't know how to zoom in on my new camera.

0:46But maybe we'll save that for another time. Yeah, okay. Joe's been setting up his camera for like three weeks now. I do it bit by bit as in I'm slowly improving the podcast. He's reading the instruction manual on how to turn camera on. Well, we got that far. It looks really good. This is not it. This is not it. This is not it. The audio is new. This is just my standard webcam. You're just looking sharp today. He was showing me the inside of his nose moments ago. Oh, well, I missed that. Well, that's the pivot. We're pivoting now. Subscriber's content is inside of Joe's nose. Exactly. One bit of housekeeping before we get to the stories.

1:26We have a reader slash listener survey. There isn't a fancy link like a bit.ly, but I will put a link in the show notes. Jason, do you just want to explain why we're doing this and sort of why it's important? Yeah, we are on Ghost, which we've discussed before, but is the architecture that our website runs on. And it has no cookies, which is very good. It's very privacy-focused, privacy-centric, which is important to us because we care about these sorts of things. And we don't want to track our readers all over the internet. That said, we don't know very much, like really anything about the people who subscribe to us and what things you like, what things you don't like.

2:13So this is going to help us as we decide sort of what to do, which projects to prioritize going into 2026. We have questions about like, do you like our email newsletter? Do you want it to be different? Do you want to see us cover different things? would you be interested in a print product like a magazine things like this and then i mean the other thing and just like full transparency to be honest uh when we try to sell advertisements on this podcast and in our newsletter uh the prospective advertisers are like well what can you tell us about our about your audience and right now we're like well literally nothing we have no idea.

2:56So we're asking people to sort of give us like general information about the types of jobs that you have or like the types of products that you like, things like that. So yeah, it's voluntary survey, it would be very helpful. It's anonymous, of course, and trying to just like get a sense of what types of people listen to this podcast and read our website and subscribe to our newsletter and that sort of thing. And then yeah, on the ad front, it's like, if you are paying subscriber of 404 Media, you will not see ads. There's no ads, including on this podcast. Yeah. So a link to that survey is in the show notes.

3:39It'll take literally a minute, I think. And we would really, really appreciate it because it would help us grow sustainably and figure out, well, where exactly do we grow and where should we invest our time and resources? So let's get to the first story of the week. This one is written by Sam. It's a really, really crazy one. The headline is, Massive Leak Shows Erotic Chatbot Users Turned Women's Yearbook Pictures Into AI Form. Sam, first of all, what is this service exactly? It is called Secret Desires AI, what is it? So it's really similar to a lot of the erotic roleplay chatbot slash image generator apps that are out there right now.

4:28There are a ton of these. You've probably seen them being advertised on Instagram or YouTube. They're pretty popular, all of them. They all have a ton of users. this one is really similar to like character ai uh chubb ai is another one um it's similar to the meta chatbots if you recall those from when i wrote about um like the the therapy chatbots that were on meta basically you can go on there and say i want to customize my own chatbot uh usually i mean it's called secret desires with like a heart so it's the main thing that they do is like quote-unquote spicy AI chatting is what the website says.

5:10The site says, build your perfect AI partner, customize their looks and personality to bring your fantasies to life. And it's like, it's, I would say, 99.9 % erotic or like sexualized. It's a sex chapel. Stuff, yeah. I mean, if you go through the homepage, you can sort it by like 18 plus. It's all like porn categories, basically. It's like you can choose like by ethnicity, ages, realistic, anime, both. You can search by like kink. It's just like anime girls and fantasy women is probably 80 % of the homepage, I would say. And then there's like ripped men. Like eight packs are also part of the offering, which I do want to, I mean, And I do want to say off the bat that women use these services, I would say, maybe not just as much as men, but there's a big demographic of women using erotic chatbots and erotic roleplay because a lot of it's text-based.

6:13And I think it's a lot of the same, not the same people, but the same kind of draw as a romantic-y novel, except you're interacting with the character. so yeah it's not it's not like it's just men doing this but it is on the home page like mostly hyper sexualized anime women and what do you see and again we'll get to the leak and stuff in a minute but when you use secret desires ai so it's a text interface you're interacting with this ai persona but there are generated images as well is it like what it's all part of the same product or what is yeah pretty much so if you go through like the so you can pick from the stuff on the homepage that I just described.

6:56There's like a bodybuilder lady who's like basically nude. There's like a sassy 50-year-old pharmacist who is tender and nurturing and definitely not wearing anything work appropriate. So you can pick from those or you can make your own, which I think is what a lot of people do. So in the character creation part of it, you can choose gender, you can choose style, like anime or realistic, you can do like identity. so you're doing like their um like their ethnicity basically which there is not a lot of options to choose from there um you can do appearance but it goes through all of this and it creates the personality of the chatbot based on what you choose it's like you choose like their career their traits uh their lifestyle you can type in your own prompts but then you get to a point where you can then generate an image of your ai partner um you make it basically yeah you're you're describing it you're doing prompting just like you would with like um sora or nanobanana or any of these other um sealed diffusion any of those other generative ai platforms where people also use you know to we've talked emmanuel's talked a ton about civetai so it's all part of the same universe or you upload your own image as well right?

8:19well not anymore I mean that's a feature that they used to have was they called it face swap but as far as I can tell you were uploading an image of someone and it was generative AI it wasn't like deep fakes where it's a picture of someone and you're putting them in a scenario that already exists in real life it's generating a whole new fantasy see a whole new person like who looks like the person that you are the target person basically yeah it's um i mean i feel like obviously readers of 404 and listeners of the podcast will know now but like we are a million miles away from deep fakes at this point like deep fakes are this quaint um thing at this point where we're not talking about really face swapping we're talking about using generative AI to make entire new scenarios and that sort of thing.

9:13Sort of like the Taylor Swift videos, which we might get to in a minute. And people still call all that like colloquially deepfakes because it's a person in a scenario that they were never in, I think is now the definition. The definition changes every year. I've given up trying to keep up on what specifically makes a deepfake. But I think most people are when they say deepfake, what they mean is like all of the above it's a huge umbrella term of like people generating wholesale new images like these fantasies that people are making on secret desires and other platforms on civetai things like that um and then also uh the face swapping that you're talking about so that was with like generative adversarial networks which is a different sort of uh family of the same product or the same alp i guess without getting too like technical about it's um So that's what we're dealing with.

10:05So that all exists. Secretdesires.ai offers that. You then get this tip, I think, via email. What exactly was this tip? What did it say? So, I mean, there are websites out there that track open, like exposed containers. And by container, I mean, this is something that like, it's storage. It's cloud storage being used by a company or by even an individual. but usually it's companies because this stuff is expensive. And then they're drawing from that container to... The inputs go in, they draw out of it. It's cloud storage. AWS buckets, that sort of thing. Yeah, exactly. And so this specific container was Microsoft's Azure Blob Storage, which is such a funny name every time I come across it.

10:53That is what it is. It's a blob of data. And then the blobs are... There are different blobs and different containers of related data. So someone emailed me and was like, Hey, I came across this on one of those sort of websites that tracks these things. He was like, this seems like a big deal because it's... Well, first of all, it was like 1.8 million images. A lot of those are duplicates, but individual images, 1.8 million. and a lot of them were pictures of people, real people generated like AI characters and companions all coming from secret desires that was there the container belonged to them so this person flagged it to me and was like this is something that you cover this is probably not good or at least interesting I don't even think they said it's not good I was like yeah that's super interesting this is like on a saturday or something it's like yeah let me spend like the afternoon dragging these images out of this container to see what's actually in here yeah and just to clarify again this is publicly exposed information uh like it shouldn't be obviously it's basically been put in a database which is publicly accessible from normal internet so you know journalistically uh we are able to see the information and use it.

12:18And we've done tons of stories about exposed buckets over the years, not just here, but way back at Vice and Motherboard as well. So you go in and you get these images and it sounds like they're split into two sets, which is sort of the input where people are taking images of people and then the output, like this AI generated non-consensual intimate imagery or however you wish to describe it, the output. First on the input images, what was in those? Is it like porn stars, celebrities, normal people? Like what were you seeing in there? So yeah, and also just to your note that it's public. If you were responsibly storing this data, if you were responsibly using Microsoft's blob storage, you would password protect it, you would encrypt it.

13:07you would be doing things to make sure people can't just access it from the open web, which is something Secret Desires obviously did not do, which is why I was able to just click on a link that took me straight to a file full of this stuff. So what was in it? I mean, so yeah, I guess that's true about there are two different sorts of images. And then there's the input and then the output. That's kind of the broad categories. But then there were also specifically named containers. So one of the containers was named Face Swap. And that's where a lot of the images that were kind of the crux of the story were being held.

13:49So a lot of those were just images of, from what I could tell, completely like random people. people um people would know like internet and social media footprint um a lot of them are old images like images from clearly from like 10 years ago 20 years ago even because they were like taken with flip phones flip phones in mirrors or with like a blackberry with the sliding keyboard and they looked like deep fried low resolution like myspace picture so somebody's gone and got MySpace pictures and fed them into this AI. Yeah, and that's the name of the container is why we how we figured that out because they're called FaceSwap.

14:33FaceSwap is a tool that Secret Desires used to have where you could upload these pictures of someone, anyone and then it would, I assume this is how it works just based on what we saw in the containers. Based on how it works. Yeah, how that step works. People would then upload the images, it would go to the face swap container, which was a public, again, exposed. And then that's where they were stored and then Secret Desires was using its generative AI algorithm to turn those into whatever fantasy prompting that the user described. So if someone was like, I want to see my... And this is something they advertise publicly as of recently.

15:16If you want to see this girl that I saw at the gym, that's a fantasy that you can... fulfill by it used to be by uploading a picture of this girl at the gym maybe you know her like Instagram or like you took a picture of her without her knowing or whatever maybe she sent you a picture doing you know whatever you want so like I think we can infer based on the other containers which were full of porn AI generated porn that people were making porn out of these people's images So yeah, it was like regular people. It was also a ton of influencers, a ton of just like screenshots of influencers and also like sex workers and like porn influencers and adult content creators, normal, regular, like non-adult, non-safe-for-work influencers, celebrities.

16:09there was one probably the one that like really deserved me the most was and this is all it was all pretty disturbing just to go through like these people's images who had no idea clearly that they were in this but um one of them was like a local um like a representative like a like a lower level politician who was she was the image was from where she was speaking at like some kind of like town council sort of event city council thing um and someone uploaded that and then you could go to the other container which said like what were the container names specifically anyway they were i think they were called like live photo or like um characterization or something but you could go to the next one and see where their faces looked a lot like the people who were in the ai porn so i could i found one of her looked just like her um in like a really crazy like non um not humanly possible type so yeah it's like and someone actually did end up telling me after the story came out that they were a tester for the face swap tool and they said that secret desires never let people upload those images from face swap to the public feed but you could make them you You can put them wherever you want.

17:32Just because you can't put them in the same ecosystem in the feed doesn't mean you can't just share them in many other forums that these people also share things to. So it's a huge ecosystem. It doesn't just stop at you can't upload it to the same website. That point on public, I think, is the really interesting bit about this piece, which is that, of course, we see AI-generated porn of people either on tube sites or it will happen with the Taylor Swift videos that go viral on X or whatever. But there are people doing this by themselves, basically using these tools. And of course, those are supposed to be opaque systems where I'm going to go to this website, I'm going to use this app, I'm going to upload imagery, I'm going to make AI porn of this person without them knowing.

18:21And maybe they don't distribute it. Maybe they don't send it to the person. They just keep it to themselves on the hard drive. And of course, nobody gets visibility into that. I don't think a lot of these people are going to want visibility into that, even though they're doing this to these other people. And this provides a snapshot of, oh, holy shit, this is what people are doing. And yes, there are a lot of responses going, are you really surprised? Like, no, of course not. No, that's the point. So we found an actual use case of this happening and now there is a really significant data point.

18:57to show you. Just very briefly, before we move on to the next bit, the headline does mention yearbook photos. And I think a manual came up with the headline. I think it was very, very smart to include yearbook because people instantly go, oh shit, that's a really horrible recontextualization of a photo or a piece of data. How did you figure out they were yearbook photos? I mean, they were literally pictures. You would see yearbook. It was someone graduating. They had class of 2000... I think it was 4 or 5 or something. They were holding their little cap and it was like the life-touch background.

19:37I don't know how to describe that, but like blue speckly. I think people can visualize. Yeah. You know what a yearbook... And also, I don't know if people know that most... I think a lot of... Especially public school yearbooks are publicly available. That's all data that's out there anyway. but this is like a specific person that obviously had access to someone's like yearbook pictures and maybe it was their classmate it was just like some random person but yeah and then there were others that were like there was a woman like standing in front she was like graduating from university and she was like standing in front of like the sign for the school um just like a very normal like wholesome picture and someone took it and turned it into porn um yeah i don't know that's that's not get too specific on the porn that was made but i think can you give people a very general sense of how explicit or hardcore was this pornography yeah um yeah i mean so when we've written about this before um and it's something that like is not new in ai but with generative ai especially so we're talking again about the difference between like old-fashioned deep fakes where you're putting someone into a porn scene that already exists which is already shitty and sucks because you're stealing someone else's content and also someone's face but now it's like you can create scenarios and have people do things that just don't exist in real life um a lot of it was like pretty i don't want to say violent but like it was pretty violent because it was like not real people it's like no one in this is like the people are not real and also the person whose face it is is not consenting to like this being out there of them um there were quite a few very young generations like like generative ai of like very young people um or what look or what looks like a young yeah like clearly the prompt and a few of these prompts were you could see the problems in some of the file names but people were prompting for like 17 year old looks like this and that celebrity and it's like that's a minor but like i'm not even talking about 17 i'm talking about like child um it felt like a real mix because sometimes like with emmanuel's civetai coverage that has been specifically focused on child sexual abuse imagery c sam and then this it sounds like had both potentially c sam but then also definitely adults and uh it It was like everything.

22:26And I was kind of taken aback by how explicit some of the images were. Right. And AI-generated child sexual abuse material in many states is treated the same. So generating porn, like a sexual abuse imagery of a child with AI doesn't mean it's not a crime in many places. And experts have talked about how this is not only not helpful to actual investigations and tracking down people making this stuff, but also it actually damages investigations that are trying to find real abuse victims out there. So I don't know. It's like that. It opens up a whole other can of worms about like the conversation about AI and CSAM.

23:13But yeah, I mean, it's as far as like the hardcore porn stuff, it was just like, like crazy, crazy shit. I don't even know. I would say the stuff that I saw when editing this piece was extreme, like extreme, almost all of it. And like, I don't like gangbang stuff. I don't know, really just quite extreme, which is in the piece, but this is not like a standard nudify app where it's like taking the clothes off of a shirt, off of someone who's wearing clothes. It is like for hardcore porn primarily, at least what we saw. Yeah, and Jason, I was going to bring you in now because you did edit it, And I feel like when Sam got the tip, I think you were the first to respond saying something like, well, this is insane and wild.

24:06What did you think of this while you were editing it? Like, what's new about this story? I mean, we knew that this was the case, but it is another example of the fact that these tools are being used primarily to non-consensually put women into porn and sexual situations. and it was not okay when it was deep fakes for either the porn performer whose body was being used or for the celebrities who were having their faces swapped onto it. It was not okay then, but I think it's pretty alarming to see that quite clearly what's happening here is just random people are being put into these situations where they're definitely not public figures and they're definitely not, I don't know.

24:57It's just like, it's pretty gross to see. And I think that we've done a lot of reporting where it's like, this is what these tools are being used for. But I think to see this scale at which this occurred and the explicitness and all of that was quite jarring. And then, yeah, again, it's like, this is not even a big one. I had not heard of this app before. And the fact that there's more than a million images on this app is like, well, there's many, many apps like this. And I guess it's just like, I hate to be hyperbolic, but it's one of those moments that makes you lose a little bit of faith in humanity, I think.

25:42It's just like walking around, it's like, people are doing this. People are doing this. That's not good. And we're not really doing anything to stop it. and it could be you or your partner or a friend or something because apparently people are just harvesting photos that and that they can they're harvesting any photos they can get a hold of essentially is what it looks like um i guess just to wrap up this section sam what was your takeaway for me it was sort of the scale and it was being done to ordinary people like what was your takeaway after doing this um i mean not to like keep belaboring the point but like it's It's just something that we know people have been doing with the technology because, first of all, it's wildly popular.

26:29We know that erotic roleplay is hugely popular. And we know that generating real people is massively popular. And we know this from lots of other reporting that we've done on this. And we know it from listening to and talking to the users of these tools. They say it out loud all the time that this is what they're doing with it. they're in discord talking about it and making these generations it's like that's that's whatever that's fine um you know like fantasy is fantasy porn is fantasy you know i don't like i don't love that use of ai but go nuts it probably shouldn't be illegal to make fantasy ai porn but i think with the face-wopping stuff um i think it was really like you said very telling that this is like a huge part of that was a huge part of the site it was also very telling that they took it down and that it was not a tool on the site for very long um it went down in i think april and a lot of people were mad a lot of people were like i'm canceling my subscription uh people were posting on reddit talking about how they don't want to use secret desires anymore and they were like oh it's my favorite tool but now i'm about to go somewhere else because they don't let me do face swap anymore um so like there are a lot of things going on there it's like first of all people really want this feature um it's in demand and also it was in demand and the company still took it down which is interesting to me because usually these companies are only driven by like what did they find what did they decide like what um the take it down act went into effect earlier this year.

28:06So like maybe they were like, oh shit, we have to like preemptively make sure no one's violating federal law. So I don't know. It's like the company hasn't replied and said anything. They shut down the buckets though, or they closed them? They shut down the buckets immediately, like in less than an hour. They like read my email and then shut down the buckets, which I sent them all the links because I was like, you need to shut this down because it's, violating these people's privacy. Or, you know, I was like... And you can't cover it. Yeah, you can't cover it before it gets shut, basically. Yeah, we can't, like, highlight something that's, like, an exposed...

28:42This is something that's, like... It's with any kind of, like, security or, like, breach reporting. It's like, you want the company to close the breach before you cover it because you don't want to draw more attention to it. So they shut it down, but, like, they didn't reply to me. And I would think it would be very easy to say, like, we found out that, like, it wasn't... We weren't into it. It's not what we're about. We don't like that use of it. and we found out people were making like... But it is what they're about because that's all their ads are like... AI women can't say no. You can generate the lady at your gym.

29:12That is what they're very much about. So I'm very curious. It's kind of like an open question about why they took it down. That's my... My takeaway is like a... What's going on there? So I don't know. I mean, maybe we'll find out down the line. Maybe someone will come forward and say, I worked in Secret Designers and I told them to take it down or something. Yeah. If you're listening and you do know more, of course, please reach out to Sam. We'll leave that one there. When we come back after the break, we're going to talk about one of Jason's pieces about some very funny shit that happened on X, but there's much broader implications as well.

29:51We'll be right back after this.

30:04Are you always searching for that one hoodie you'll actually live in? The one that feels just right on the couch, on a plane, or out on a hike? I found mine with Paka, and now it's the only one I reach for. Paka makes performance apparel from Alpaca Fiber, one of the world's most sustainable natural fibers. Their best-selling hoodie is softer than cashmere, warmer than wool, and breathable. This hoodie is built for real life. thermoregulating, odor resistant, durable, and made to last. It's lightweight, but still cozy, doesn't stretch out, doesn't pill, and somehow keeps me warm when it's cold and cool when it's hot.

30:39Basically, it adapts to wherever life takes you. Over 100 ,000 people have already picked up the Paca hoodie. What makes it even better? Knowing it's made sustainably and ethically from traceable alpaca fiber, while supporting the communities and artisans in Peru who bring it to life. Little known fact, it actually gets cold here in LA. Most nights, actually. My Paka hoodie has become my go-to around the house at happy hour drinks and running errands. The only problem is I have to keep telling people where I got such a great hoodie from. If you've been thinking about leveling up your hoodie game, this is your sign to do it now.

31:10And right now, during Paka's Black Friday slash Cyber Monday sale, you can get 30 % off with the purchase of three items or more. Head to go.pakaapparel.com slash 404media. That's go.pakaapparel.com slash 404media to get 30 % off with the purchase of three items or more With Aura Frames you can relive your holiday traditions every day We've all got them those weird family traditions that make the holiday feel magic For my sister and me it really isn't Christmas until we put our childhood ornaments on the tree We've got a lot of great pictures at that moment and Aura Frames is the perfect way to show them off Aura Frames is a digital picture frame you can load with unlimited free photos and video.

31:55You can preload it with photos before it ships so you're ready to relive your holiday traditions the moment it's out of the box on Christmas morning. And the moment my sister and I put our ornaments on the tree this year, we can grab a picture and send it right to the frame from our phone. That way, when our mom opens it up, she'll remember. Memory keeps these traditions alive and Aura Frames is the perfect place to store those memories. You can't wrap togetherness. but you can frame it. For a limited time, visit AuraFrames.com and get$45 off Aura's best-selling Carver Matte Frames. Name number one by Wirecutter by using promo code 404media at checkout.

32:34That's A-U-R-A frames.com promo code 404media. This exclusive Black Friday slash Cyber Monday deal is their best of the year. So order now before it ends. Support the show by mentioning us at checkout. Terms and conditions apply. I remember how shopping for glasses used to be. Stumble into a stuffy doctor's office, spend hundreds and hundreds of dollars on glasses I didn't really like, wait a month to get them, then feel dizzy because they weren't fitting right. Warby Parker changed all that. Warby Parker uses only premium materials in each frame. They design the frames in-house, and their collection includes different silhouettes, colors, and fits to suit every face and every fashion style.

33:15I love that it's a one-stop shop. They offer everything you need for happier eyes Eyeglasses, sunglasses, contact lenses, and eye exams You can shop with them online and in stores Warby Parker has over 300 retail locations across the U.S. and Canada Their glasses start at just$95 and include prescription lenses with anti-reflective, scratch-resistant coatings Add a pair and save 15 % when you purchase two or more prescription pairs of glasses or sunglasses This offer is available both online and in stores free shipping and 30-day returns. I've been going to Warby Parker for years now. I actually don't think about shopping anywhere else for glasses anymore.