Inside the Fight Against Non-Consensual Deepfake Porn

22:14And the claim that I like the best is that we sued them for public nuisance. There was just no reasonable, safe use of this product but to harass people publicly. So that's where it stands now. Yeah. Well, we had to give XAI statutory notice that we were filing for a temporary restraining order. And they immediately sued Ashley in Texas, claiming that she had breached the XAI terms of service by even threatening to sue them in her home state of New York. So now we are litigating the issue in two different venues. They're also trying to use the X terms of service to get the XAI case transferred to New York.

23:00And the judge initially last week said that XAI can rely on the X terms of service to transfer the case. To Texas, you mean, to Texas. So even though X is not a defendant in our case, only XAI is, the judge still let them borrow from a different Elon company, which I think is really scary. So is that where it's going? Well, we filed an emergency petition for mandamus to try to get the stay held up. But I don't know. We don't have the outcome on that.

23:32Kara Swisher:We'll be back in a minute.

23:45Kara Swisher:Support for this show comes from Quince. A nice linen set lets you get away with a lot of outfit repetitions in the summer. You could wear the same one to a laid-back barbecue or upscale seaside restaurant and still feel classy. For elevated summer wardrobe pieces, check out Quince. They offer just that, and they use premium materials like European linen, organic cotton, and washable silk without the traditional retail markup. Their 100 % European linen pants, dresses, and tops are the pieces you'll reach for all summer long. They're lightweight, effortless to style and start at just$32. They also have beautiful 14K gold jewelry that adds a subtle finishing touch to any outfit.

24:25Kara Swisher:And it's not just clothing. Quince offers everything from home to travel to everyday essentials. Everything is affordable too. All of Quince's items are priced 50 to 80 % less than similar brands. In fact, I have used my Quince suitcase at least four or five times this summer. It's simple to use and it's very durable. I also use their shirts and t-shirts a lot this summer. They're very breathable, easy to use, and very comfortable. Make your summer wardrobe easier. Go to quince.com slash cara for free shipping on your order and 365-day returns. Now available in Canada too. That's quince.com slash cara for free shipping and 365-day returns.

25:07Kara Swisher:Quince.com slash cara.

25:12Kara Swisher:Support for this show comes from Pebble. If you're a founder or HR leader, building a great team is the fun part. But nowadays, that search for great people is global. It's just how business is done these days. Pebble is a hiring platform built for modern teams. Their platform takes delays and guesswork out of going global, and they do it with embedded compliance and AI-driven workflows. With Pebble, you can hire in over 185 countries in minutes. And when it comes to compliance, Pebble has built a decade of expertise directly into the platform. Pebble brings everything, contracts, payroll, benefits, and compliance together in a single platform.

25:49Kara Swisher:Your global team can actually be set up for success with the support they need to thrive wherever they are. Bottom line, anywhere is possible with Pebble. It's global hiring simplified. And here's a special offer for our listeners. Pebble is normally$399 a month per employee. Already a no-brainer for what you get. But right now, there's a limited time offer on their site that makes it even easier to get started. Go to highpebl.ai before it's gone. That's high, H-I-P-E-B-L dot A-I. Terms and conditions apply.

26:25Kara Swisher:Support for this show comes from Gusto. Be honest, there's probably one task on your list you constantly push to next week because it's just so tedious. For a lot of business owners like myself, that task is payroll. Gusto is here to take that entirely off your plate so it becomes an easy part of your job instead of the dreaded one. Gusto is an online payroll and benefits software built for small businesses. It's all-in-one, remote-friendly, and incredibly easy to use so you can pay, hire, onboard, and support your team from anywhere. Actually, Podium Media uses Gusto to operate. Automatic payroll, tax filing, simple direct deposits, health benefits, commuter benefits, workers' comp, 401k, you name it, Gusto makes it simple and has options for nearly every budget.

27:09Kara Swisher:Unlimited payroll runs for one monthly price. That means no hidden fees and no surprises. You can save time with built-in automated tools, offer letters, onboarding documents, direct deposit, and more. It's quick and simple to switch to Gusto. Just transfer your existing data and get up and running fast. Try Gusto today at gusto.com slash Cara and get three months free when you run your first payroll. That's three months of free payroll at gusto.com slash Cara. Again, that's gusto.com slash cara.

27:45Kara Swisher:So let's get to the tech infrastructure that supports the creation and dissemination of deep fakes. Analysis from Media Outlet Indicator found that nudify apps often use infrastructure from tech companies. VS, explain the relationship between deep fake porn sites, tech companies, and the social media platforms, and to what extent do these sites and nudify apps rely on big tech infrastructure? This was an issue when I was interviewing people who were creating all manner of nonsense around the insurrection on January 6th. And companies moved quickly to take some of it down, but they were clearly the way people got to these different sites.

28:22That's a broad question, Cara. So let me try and break it down into pieces. The first is, depending, if you're generating deepfakes at scale. So think the kind of thing Laurie was talking about with Mr. Deepfake. He's trying to generate thousands, millions of deepfakes. And so that requires computational hardware, what are called graphical computing units or GPUs, which today are pretty scarce and expensive. And that scarcity and price doesn't look like it's coming down very much anytime in the near future. Those resources are heavily supported by cloud infrastructure providers. Again, I'm not naming any specific companies, but there are a number of companies that offer cloud services.

29:12So part of the onus is placed on them if somebody is trying to use their service to generate this kind of stuff at scale. If somebody's trying to generate just a handful of deep fakes, they're probably not going to figure it out. But if somebody is generating millions and millions of them or thousands and thousands of them, they may have some idea of what's going on. But that's not easy either, because, you know, there are many challenges for some of these firms in figuring out what's a deep fake and what's not, what's consensual, what's not. And so as a first issue, you know, the bad guys are continuously evolving.

29:56So that means that the actors in question are changing. The technology used to generate this stuff is changing. and it turns out, having studied deepfake detectors quite a lot and created some on my own with my lab, what we know is that small changes can make a big difference. So to give you an example, if I'm drinking this cup of coffee and there's a deepfake of me with part of my face obscured as I'm drinking that cup of coffee, it's harder in many cases for many detectors to figure out that video clip of me saying something which I probably never said is not true. Likewise, the definition, Carrie, I'm not speaking from a legal perspective, but for a normal person, what constitutes an act that's sexual in nature can vary a lot.

30:52A sexualized act versus an actual act of some kind of sex, very different. Also, the relative positions in which a sexual act is carried out, where certain body parts may be obscured, but other things may be clear, all that causes machine learning algorithms to detect deepfakes to have a lot of problems. So I think, you know, those are some of the challenges from a technical perspective in figuring these things out.

31:22Kara Swisher:So one analyst has said that some deepfake sites now offer APIs to people creating non-consensual images and video generators. Explain what that means. So when you go to a website, you're typing stuff in, a prompt or a query, or clicking on some buttons, and you get a response. An API is an application program interface, and it replaces you with a piece of computer code. And that piece of computer code is interacting in a different kind of language with the server behind the website you're going to. And so if somebody is using an API to generate deepfakes, that suggests that they are generating it at scale.

32:06That suggests that they have a piece of code that is shipping requests automatically through that piece of code to the server, and that server is sending back whatever they requested, in this case, some kind of sexualized image. That suggests somebody who's trying to do it at scale, who probably has his or her own databases of people they want to propose in these situations that they will never begin with.

32:35Kara Swisher:Indicators analysis also estimates that the nudifier economy may be worth up to$36 million a year. Lori, in your reporting on Mr. Deepfakes, you noted the financial incentives through the community members posting in the forum. Tell us about the incentives and talk about the money making, Lori, here, and for the platforms that host this content, too. Although, for people who don't realize, Google says it doesn't allow apps that contain sexual content. Apple says the company's app store prohibits overtly sexual content. Both companies disabled searches for Nudify in their app stores. We'll get to the workarounds in a second, but talk a little bit about the incentives, the financial incentives.

33:13It's interesting because when we were looking at this site, that wasn't a site, that was a platform. It was a whole ecosystem that enabled this site to exist. And that included platform liability from a lot of these different companies. So, for example, you know, the incentive was you can go on, not only can you view, you know, thousands of videos of sexually explosive deepfakes of your favorite celebrity, you can make your own. And this is why we believed Mr. Deepfakes was so dangerous. He was dangerous because he was creating an army of Mr. Deepfakes, because now there are all these creators on there, which is kind of pay to play.

33:46And there's a freemium model. And then there's this premium model where you can pay to have specialized deepfakes of public figures who didn't consent made. And so people were probably making more money than the guy behind Mr. Deepfakes off of this platform. It was creating a whole ecosystem. But what else enabled that ecosystem? First, I would go from a regulatory standpoint. The laws just hadn't caught up. And we saw this with non-consensual pornography. Oftentimes, tech moves quickly when there aren't the correct guardrails. Women and young folks are the first ones to feel that pain. But also, there were payment providers that were accepting payments on this platform.

34:22And so it was all of these things, and we talk about these small steps that actually helped enable this ecosystem to exist. And one of the coolest things we saw, because I think so many folks would say, well, it's a game of whack-a-mole. You go after this guy, and then there are all these other ones that are going to pop up. But actually, what ended up happening is we did go after this guy. We found out who he was, and I I think maybe most alarming was he wasn't a technical guy. This is a guy who was, I would say, in his 30s, worked at a pharmacist at a hospital helping people, was newly married, had a young child, and he had been able to create a website that enabled a whole new wave of deepfake abuse because he had an interest in technology and deepfakes.

35:07And he saw an opening because when Reddit banned sexually explicit deepfakes, we were able to track where his username was talking about seeing an opportunity that could be profitable and getting other people to build on this. And so all of these things together created, I would say, an ecosystem that allowed it. And all of these things also, once people started chipping away at this, once the laws started slowly but surely catching up, once Google deranked the platform, there were all sorts of small steps that actually created a friction that made it harder to exist.

35:41Kara Swisher:So let's shift to the current laws and regulations surrounding deepfake porn, including the Take It Down Act, the law that makes it a federal crime to publish non-consensual explicit deepfakes. The federal government began enforcing it in May. Carrie, in your pursuit of tech platform accountability, you've come up against Section 230, as have many, the law that protects platforms for being held legally liable for content that users post. You mentioned product liability law as a way to stop deepfake porn and get justice for victims. So talk about using product liability law. Section 230 was intended to basically just immunize platforms for content that third parties created and posted.

36:20But when it comes to deepfakes, especially like in the situation of Grok, we're saying that Grok was the one who created this material and also published it. And therefore, Section 230 shouldn't apply at all. But even if it does, by suing them under product liability, we're suing them not for publishing the content, but for, you know, basically harming people through its defective design, defective manufacturing. These were all foreseeable uses. There's no reasonably safe use for a nudifying product. And so that's just like a second way to overcome Section 230. Now, XAI, they haven't fully exposed what their defenses are going to be in our case.

37:11But when they were opposing the temporary restraining order, they were claiming that they wanted, I mean, so far this would be the only AI product that is saying that Section 230 should apply to it. And that they're not responsible for what people type in as prompts. But even more alarming is that they're claiming that they should have the First Amendment protections and that Grok should basically be protected under free speech, which is a pretty alarming idea, you know, that AI and a chatbot would have free speech rights. They don't, you know, so far we're not at a point where we're giving constitutional rights to a chatbot.

37:54But again, the arguments haven't yet hit the stage where they've been fully briefed yet.

38:00Kara Swisher:So, Vias, talk about workarounds, because a lot of these companies are very clever in terms of workarounds. Porn people have always been very clever. How easy is it to do workarounds here from a technological point of view? You know, I was thinking about this when Lori was speaking earlier, Cara. You know, she talked about a website where somebody goes in and says, I want to create a deep fake of such and such person doing such and such thing. And so that's a classic example of a place where that website is using the APIs that you asked about earlier to access a cloud provider where this code running to generate the notification of the sort desired by whoever asked the query.

38:47But now think about it this way. Let's say the platform has the mechanism. mechanism, let's say they have a perfect mechanism that doesn't exist today, but suppose they did, to detect a deepfake video of some form of sexual activity. Then the workarounds for the porn guys, there are some simple ones. So one of the classic methods to generate deepfakes, regardless of whether it's for good or bad, is something called stable diffusion. So in stable diffusion, what you do is you, let's say, throw in an image. I'll use an image rather than a video as my example. But you throw in an image of somebody, and that image gets converted to some very coarse representation.

39:32You know, think of this as something that's a very skeletal version of that image. And then what ends up happening is that skeletal version goes through the nudification process, but at the skeletal level. So that skeletalized, nudified image doesn't look anything like what a real nude might look like. And so you can think of it as some kind of computationally weird representation of a person who's about to be sexually abused via a deep fake. And then they could do all this on platform on a cloud. And then without ever passing the source image or the final image, they bring back this coarse representation of the nudified image and perform the last step back on their own servers where it generates the final nude.

40:27So that's an example of one way in which somebody with just limited computer science knowledge, who's familiar with the code, who has access to all this code, can generate something while avoiding the guardrails of some of the platforms, even if those guardrails are close to perfect.

40:48Kara Swisher:We'll be back in a minute.

41:02Kara Swisher:Support for the show comes from Grooms. Summer can disrupt our routines with busy weekends that are full of travel and events. And with everything going on, it can be hard to remember to stick to your wellness habits. While Grooms is one of the wellness habit you don't have to think about, it can be the easiest, tastiest part of your busy schedule. Grooms are a convenient, comprehensive formula in one delicious daily pack of gummy bears. This isn't a multivitamin, a greens, gummy, or a prebiotic. It's all of these things and then some at a fraction of the price and it tastes great. They're able to pack in 20 vitamins and minerals in a daily pack with 60 ingredients, all of which include nutrient-dense and whole foods.

41:36Kara Swisher:A Gruen's pack also contains 6 grams of prebiotic fiber. To help you visualize it, that's equivalent to more than 2 cups of broccoli. Gruen's ingredients are backed by over 35 ,000 research publications. You can't fit the amount of nutrients Gruen's does into one daily gummy, which is why it comes in a pack. Save 52 % off with the code CARA at gruns.co. That's code CARA, K-A-R-A-G-R-U-N-S dot C-O. Teens share everything. That may include the bacteria that can cause meningococcal disease known as meningitis. Even if your teen's been vaccinated in the past, they could still be missing meningitis vaccinations.

42:14Kara Swisher:Ask your teen's doctor or visit meningitis.com today. Sponsored by GSK. Traditional home security only alerts you after a break-in. And that's too late. SimpliSafe is changing that. Stop. This is SimpliSafe. Police are on the way. We don't just alert. We stop crime before it starts. SimpliSafe. Plan starting around a dollar a day. Save 50 % on your new system with professional monitoring at simplisafe.com slash Spotify or with promo code Spotify. Outdoor deterrence requires a SimpliSafe Active Guard outdoor protection plan starting at$49.99 a month. Visit simplisafe.com slash licenses for alarm license information.

42:47Tennessee 2012.

42:50Kara Swisher:The Defiance Act, which is aimed at allowing victims of non-consensual deepfakes to sue for damages passed in the Senate and is currently stalled in the House, even though some states have criminalized deepfakes, enforcement remains low. Lori, what extent are these laws effective when it comes to stopping or at least curbing the spread? It's happened before with credit card companies and certain businesses that, you know, things tend to slow things down. But in this case, how important is it to have the Take It Down and the Defiance Act? I'll answer, and I'm curious for Carrie's response, too, but I think it's incredibly important.

43:24You know, the Take It Down Act, you know, forced tech companies to pay attention and quicker. It gives them 48 hours to take down nonconsensual abuse. But oftentimes, when it comes to victims, it's really difficult to find recourse for this, you know. And so the Defiance Act would create civil penalties that would enable them to actually have the ability to go after the folks who have done this and have civil recourse, which I think makes a big difference. And when we started out on our search for Mr. Deepfakes, which was, let's just be honest, really a search for asking why is this allowed to happen, you know, there weren't a lot of state laws.

44:01And I will say a lot of the state laws and a lot of the victims and survivors we spoke to, unfortunately, the onus in many ways becomes on these survivors to speak out about it and help change laws. And they did in the state level, which, you know, the state laws changing, all of them varying in scope, civil criminal penalties actually were where a lot of these different victims initially had recourse, which is also scary to watch as folks go after regulation at the state level because oftentimes the federal government is just slow to move. I would say I hope that the Defiance Act passes. It can get through.

44:34I think this is a bipartisan issue because it's really a child safety issue as well.

44:38Kara Swisher:Right, which is how they're saying it. So, by the way, a Wired investigation found that Grok is still being used for non-consensual explicit content. This is despite regulatory scrutiny and multiple lawsuits. And after Musk's ex-AI said it would add restrictions. In response to an NBC investigation, the ex-safety account posted, Do we strictly prohibit users from generating non-consensual explicit deepfakes and from using our tools to undress real people? Explain how some of these legal dynamics apply to XAI in that regard, since you're in a lawsuit with them, when it keeps showing up that they're doing the same thing you would accuse them of.

45:14Well, I mean, that statement that they say that it's against their regulations and they're basically passing the blame to users. There's nothing in that statement that suggests that they are modifying their product so that deep fakes are no longer easy to create. And the problem with things like the Take It Down Act, it's great. You know, suddenly now there's a 48-hour window of time that tech platforms are supposed to remove illegal content. But it has no teeth because victims can't enforce it. victims can't soothe the platform and say, hey, you know, it's now been 48 days and that content is still up.

45:53So the problem with all regulations and all these options is that the victims cannot enforce them. And so when we're talking about who's the bad guy, VS has talked about the bad guys in these situations. In my view, the worst guys are the platforms. And so until we have regulations that put the victims in the driver's seats to not just, you know, get criminal justice against the individuals, but to actually get justice against the platforms. You know, I think the only recourse we have is to use our courts and make these companies pay. So every episode,

46:30Kara Swisher:we get a question from an outside expert. Here's yours, and you're all going to answer it. So let's listen to it first. Hi, I'm Georgia Wells, a tech reporter for The Wall Street Journal. I just finished a deep dive into the issue of teens using deepfake nudes to harass each other. And so the big question I would ask is that now that the government has just started enforcing the Take It Down Act, how might this change the harassment landscape for young people today? Thank you. Laura, you go first, then Carrie, then BS. Yeah, I do agree a little bit with Carrie on saying, you know, there's only so much you can actually kind of enforce when the onus is on the victim.

47:06I do think from a cultural standpoint, it is very different than it was a couple years ago. A federal law, take it down act, again, state laws are at least sending this message that the stakes will be high and that young boys, young teenagers can get into real trouble for doing this and can get charged. I mean, I think that at least from a perspective of schools being able to speak to students, parents being able to speak to their children about this and having some structure will be helpful. Again, though, I'm not sure the Take It Down Act is going to be the thing that's going to actually change this.

47:42I also agree with Kerry on platform liability and creating higher stakes for the tech companies and the whole ecosystem that enables this to happen.

47:50Kara Swisher:Kerry? So I think they'll get the memo that this is illegal. I mean, having criminal laws was really, really effective when we were dealing with the scourge of image-based sexual abuse over the last decade. We went from having three states to having 50 states with those laws. And my firm saw such an enormous and quick plummet of offenders creating images like that. So that's great. But we still don't have an apparatus that targets the platforms themselves and prevents them from basically monetizing this kind of humiliation. And until we have laws that victims can enforce themselves, like it's not really harming the who I consider the true offenders.

48:38Right. Which are the platforms themselves. All right. V.S. You know, I have very mixed feelings about criminalizing a 13-year-old, all right? But at the same time, we need to protect perhaps other 13-year-olds or 14-year-olds or 10-year-olds from the consequences that accrue to them because of some 13-year-old putting out deepfake porn of that kid. So I think a first step has got to be something around education. A lot of our teachers in schools are simply not teaching their kids about deepfakes. And it's one thing if kids are using deepfakes or generative AI to create reports for their class projects.

49:28I mean, that brings up a whole bunch of other ethical issues. But it's another thing if they use deepfakes to explicitly cause damage to a kid and one of their classmates. So I think the ethics around this has got to be clearly articulated in class and by schools. And some of our schools may have done a good job of this. Others may not. And, you know, that education and the consequences of those actions in a school context are clear. I think it'll be challenging. You know, Vias, just to respond, if it's helpful, just to respond to that. I think the frustration is so many times, I agree. How do we expect 13-year-old boys to know that this isn't okay when they're getting served up notification apps on social media?

50:17You know, I think it's an and. Like, two things can be true. And so there's a huge gap between education and being able to educate folks on this and what the platforms are allowing and what they say they're allowing and what's coming through. it creates a really difficult environment. You know, I worry generally about the broad statement or the broad platforms moniker. I think there's a wide variety in what different platforms are doing. Some platforms are going to a great amount of trouble spending a tremendous amount of money to try and perform the best innovation possible so they can put some of these problems to bed.

50:58And others are doing much, much less. So there's like a huge range of what platforms are doing, some doing relatively little, some making every effort to try and get there. Now, I want to add that, you know, again, even if platforms were doing their very best, there's a famous saying due to a statistician at the University of Wisconsin, all models are wrong, some are useful. All right. And so whatever models are being used by platforms to find these deep fakes, they're never going to be perfect. And so the question is, are they making the effort? Some are making what I would consider a very good faith effort.

51:40Some.

51:41Kara Swisher:Can you name who is doing a good job? I'd prefer to stay out of that, but I don't want to talk about specific platforms, but yeah. So let's end by looking at what individuals can do as deepfake technology continues to evolve. Carrie, now that the Take It Down Act forces tech companies to remove deepfake nudes, how do you request a takedown if you're targeted? Well, part of what the Take It Down Act required was that there be a flow for people to easily request content removal. And so that kind of infrastructure, I really appreciated that being incorporated into the laws. So ideally, you know, a victim within the platform itself can now report it, you know, whether it's through a URL, or if it's like Instagram or Facebook, then the image itself would ideally have a method to request removal.

52:32Kara Swisher:When digital forensics professor and deepfake expert Hani Farid was on the podcast in January, he expressed concerns that agentic AI will supercharge the creation of deepfakes in the coming months. VS, how do we make sure the new laws and policies don't block positive uses of these tools? So I think, again, we don't want to block the technology. We want the technology to evolve because, again, as Lori said much earlier in the podcast today, you know, deepfakes do have the capacity to help do things like better understand disease. I'm not going to call this a deepfake, But the generative AI is being used to create new kinds of proteins that'll help cure diseases.

53:14Generative AI is being used to create materials that'll degrade and perform in ways that we would like, that would protect the environment and more. So I think what we have to do is to look at intent. You know, what is the intent of the individual who is creating a deep fake? And in the case of the 13-year-old boy or 14-year-old boy who's creating deepfakes of one of his classmates because he's pissed off about something, I think there's intent. Whereas if somebody, you know, there was this guy who created a deepfake of the Pope, Pope Francis, in puffy robes and a very dapper looking version of him, you know, that I think is something that's laughable.

54:00And I find that, at least I find that somewhat entertaining. I doubt very much if the Pope was, you know, horribly offended. And so, you know, I think we want to allow artistic freedom and expression, scientific inquiry, but at the same time regulate so that it's the uses that are banned or that lead to.

54:22Kara Swisher:The individual intent one. Yeah. Lori, in the end, you tracked down Mr. Deepfakes. What ultimately brought down the site and where is he now? Great question. Where is he now? I would love to know. So eventually we did track him down with the help of, you know, investigative journalists, the Internet, and some of our cybersecurity friends who did some pro bono work for us because they have kids and they viewed this as a real threat. And we were able to track him down. And, you know, I'll never forget confronting him on his way into the office. And I do remember Carrie, who I've known because I've covered nonconsensual pornography before, I called her before going to his parents' home to make sure that we did everything correct just in case, you know, we didn't want to create any legal liabilities.

55:05But I will never forget him looking at me. He knew who I was with so much disdain. And I know this is a delusional part of me, but there was a part of me that thought maybe we would have tracked this guy down and we could show up and say, do you understand the real harm this site is causing? You're a father. And I remember he walked right into the hospital. He wouldn't say a word to me and the doors closed. And I remember thinking at that point, well, we're going to have to go talk about this to everyone who will listen to us. And that has to change because we have to change culture as a part of this.

55:37This can't be where the story ends. He can't just take himself off social media, which he did, and continue running this site. And so a lot of things happened, which I think were really exciting. I mean, we, I think, were one of the first to show up and threaten his anonymity. Say, you cannot do this type of abuse and remain anonymous. So that was friction. That exposure was friction. A law started changing. When in the UK, they started talking about different types of laws that would make it harder for folks to access Mr. Deepfakes. They cut off access to folks in the UK. That created friction.

56:08A very valuable server went down for them. The Take It Down Act passed. The CBC published their investigation as well. Mr. Deepfakes finally went offline for good. And I think it was such a great lesson in all of these players that actually impact how we build out a better future. And the headline is friction. It just created more and more friction. Now, what's happened to him, I think, is actually just as important of a headline, which is nothing, really. I mean, he lost his job at the pharmacy. But we've talked to many folks, law enforcement. I actually spoke to a couple different folks in Canada from the law enforcement perspective.

56:48There's no open investigations into David Doe. That's his name. And there's thoughts that maybe he's left the country, but that's it. And I think that in and of itself tells its own story.

57:00Kara Swisher:So the last question, D-Fape Foreign obviously isn't going away. So what's the best possible outcome? Each of you, what is your most optimistic case for how it pans out in the next five years and how we get there? Carrie, you go first, and then VS, and Lori, you get the last word. Carrie? Okay. Well, it should probably come as no shock to you that my best case outcome is that the companies that unleashed these harms and profited significantly from them should have to pay, and there should be disgorgement and compensation for all the victims. And it should be financially penalizing for these companies.

57:36So that would be your best. And is that a possible outcome?

57:41Kara Swisher:Do you feel like that's a possible outcome? I mean, we're on that road with litigation. We'll see how it works. I also, you know, I do condemn the platforms like the App Store and Google Play, who for many, many years were offering and selling nudifying apps. And they just kind of were ostriches with their head in the sand, pretending like they didn't think these things were harmful. I'm really sick and tired of platforms not doing anything until something becomes actually criminal. You know, they deny that it's harmful until there's an actual criminal law. V.S.? You know, that's what I would like in place.

58:21Some kind of criminal law that says, well, look, here are the kinds of behaviors and even intended behaviors or intended outcomes that should be criminalized. Here are the penalties that you will face if you engage in these acts. And this needs to be something that's at least in the U.S. nationwide. But more importantly, many of these actors are going to be overseas. And so what we need is not just U.S. law, but law in various countries and some broad international agreements, perhaps enforced by an organization like Interpol, which polices things like child trafficking already. They should be authorized to do and investigate things like this.

59:05Kara Swisher:That's a very good point. That's exactly where it goes. It's broad. What happens is immediately it goes abroad. Laurie, last word. Look, I agree with what everyone here has said. So I will end with saying, you know, my gripe with Silicon Valley, and I have a lot of gripes with Silicon Valley, is they're sitting there talking about AGI and what happens when AI becomes smarter than us. And it doesn't matter. It really doesn't matter because here's what's happening here on Earth. Over here in reality, we have children who are ending their lives because of deepfake sextortion and people are sleeping on it.

59:39my hope is that I don't have to shout into the ether for years and say, this is real harm, this is real harm, that we can get better education, that we can change culture to understand that when new technology comes quickly without the correct guardrails, the folks who are impacted oftentimes are the ones that don't have the voice, and we can make some changes from there. Now, it doesn't take, to Kerry's point, oftentimes it takes criminal law. If I'm sitting in my seat, oftentimes it takes public pressure in investigations. I'd love to get to a world where Silicon Valley maybe has some different folks in it.

1:00:13And this isn't the last thought as people are having dinner and talking about AGI after taking a cold plunge. That's it.

1:00:20Kara Swisher:That's a very good point. But you're going to wait a long time for that. Just so you know. But I can manifest it. I'm manifesting it right here. Go ahead. All right. Sure. Why not? Why not? Anyway, I really appreciate it, all three of you. It's a really important talk. And we're going to keep coming back to it again and again and hopefully see something happen by raising awareness of it for sure. Thank you so much, all three of you. Thank you, Cara. Pleasure to be here. Thank you so much. Thanks. We've reached out to XAI and X for comment. They did not respond. Today's show was produced by Michelle Alloy, Catherine Millsop, Madeline LaPlante-Duby, and Kaylin Lynch.

1:00:57Kara Swisher:Special thanks to Lissa Sowep, Anika Robbins, and Julia Sharp-Levine. Our engineers are Fernando Arruda and Rick Kwan, and our theme music is by Trackademics. Nishat Kerwa is Vox Media's executive producer of podcasts. Go wherever you listen to podcasts, search for On with Kara Swisher and hit follow. Thanks for listening to On with Kara Swisher from Podium Media, New York Magazine, the Vox Media Podcast Network, and us. We'll be back on Monday with more.