Elon’s “Nudify” Mess: How X Supercharged Deepfakes

22 Jan 2026 · 54 min · 21 chapters

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Podcast Summary: On with Kara Swisher

Episode Title

Elon’s “Nudify” Mess: How X Supercharged Deepfakes

Episode Overview In this episode, Kara Swisher discusses a disturbing new in-app tool from Elon Musk's social platform X (formerly Twitter), named Grok Image Edit. This feature allows users to alter images of others, particularly with sexualized deepfakes, raising significant concerns about consent and safety. Joined by experts Renée DiResta, Hany Farid, and Casey Newton, the episode critically examines the implications of Grok, its failures to safeguard users, and the broader issue of accountability in the tech industry.

Key Guests

  • Renée DiResta: Former Technical Research Manager at Stanford's Internet Observatory, expert on online disinformation.
  • Hany Farid: Professor of Computer Sciences and Engineering at UC Berkeley, known for work on digital image forensics.
  • Casey Newton: Founder of the tech newsletter Platformer and co-host of The New York Times podcast Hard Fork.

---

Key Discussions

The Grok Image Edit Tool

  • Launch Date: Christmas Eve; rolled out with minimal safeguards.
  • Functionality: Users can create altered versions of others' photos, resulting in sexualized and non-consensual deepfakes.
  • Impact on Victims: Victims receive notifications for every interaction involving their altered images, intensifying humiliation.

Scale of Harm

  • Volume of Abuse: Reports of Grok generating around 6,700 posts per hour containing inappropriate content.
  • Regulatory Response: Governments globally have begun investigations; however, initial backlash from Musk included mockery of concerns.

Accountability and Regulation

  • Failed Promises: Musk's claim that removing child sexual abuse material (CSAM) is a top priority has been undermined by Grok’s features.
  • Current Actions: Some countries have taken drastic measures, such as banning Grok, while regulatory frameworks are still evolving in others.

Discussion on Free Speech

  • Incoherent Defense: The episode critiques the use of “free speech” arguments to justify the creation and distribution of harmful content.
  • Moral Responsibility: Emphasis on the need for accountability and proactive measures from tech companies to protect user safety.

Technical Challenges

  • AI Generated Content: The tools to combat CSAM are struggling to keep up with the rapid developments in AI-generated imagery.
  • Need for Guardrails: Discussion on the necessity for robust guardrails in AI systems to prevent misuse and harm.

---

Key Takeaways

  • Normalization of Harmful Content: Grok has led to the normalization of creating deepfakes, previously confined to underground communities.
  • Failure of App Stores: Apple and Google have not acted decisively against Grok despite clear violations of app store policies.
  • Call to Action: Experts urge for aggressive regulatory frameworks, alongside public pressure, to hold tech companies accountable.
  • Future Concerns: The potential rise of agentic AI poses new threats, as cybercriminals can exploit powerful AI tools without human oversight.

---

Closing Remarks The episode highlights a critical moment in the intersection of technology, privacy, and ethics, emphasizing the urgent need for accountability in an era where digital abuse is growing. The discussions reveal a complex landscape of challenges ahead, involving both technological advancements and societal responses to ensure a safer online environment for all users, particularly vulnerable populations such as children.

---

For further engagement with this topic or to listen to the full episode, follow "On with Kara Swisher" on your preferred podcast platform.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Chapters

Tap a time to open that second in VO

The Rise of Grok Image Edit and Its Implications

0:46 to 2:55

Exploration of the Grok Image Edit feature and its consequences, including deepfakes.

“For example, Grok could take a photo of a woman on X and create a realistic deepfake that looks like the original image, except the woman is now in a bikini and has, quote, donut glaze on her face.”

Guest Introductions and Their Expertise

2:56 to 3:59

Kara introduces guests with relevant expertise on the topic at hand.

“Renee DiResta is the former technical research manager at Stanford's Internet Observatory.”

Understanding the History and Impact of Nudification

5:14 to 10:10

The guests discuss the longstanding issue of nudification and its evolution.

“Renee, Hani, and Casey, thanks for coming on on.”

Regulatory Challenges and Responses

10:11 to 12:31

Discussion on the regulatory landscape surrounding Grok and its implications.

“So we're seeing this up and down the stack.”

Censorship and Free Speech Debate

12:32 to 14:03

Renee discusses the debate around censorship in relation to AI-generated content.

“So no, there really are not meaningful safeguards in the way that I think most regulators would want to see here.”

Censorship and Content Moderation Debate

14:03 to 16:50

Learn about the evolving definitions of censorship in the context of online content moderation.

“And I thought, really, is that where we're going here?”

AI and the Challenge of CSAM

16:50 to 20:10

Explore the complexities AI introduces in combating child sexual abuse material (CSAM).

“So, Hani, you develop technology that can flag previously identified images of CSAM.”

Legal Challenges for Deepfake Victims

24:16 to 28:00

Discuss the legal recourse available for victims of deepfake technology.

“So Casey, when it comes to a lot of images, we're actually in a legal gray area in the U.S.”

Elon Musk and Censorship Dynamics

28:00 to 28:58

Explore the implications of Elon's influence on censorship and regulation.

“When Elon complains, the world hears about it, right?”

Regulating AI and Chatbot Vulnerabilities

28:58 to 30:46

Discuss the challenges of regulating AI, particularly in combating CSAM.

“But the other thing I'll say about this, though, is that the other the realistic point, though, is that the integration of Grok and X is a challenge.”
Show all 21 chapters

The Moral Implications of Deepfakes

30:46 to 32:30

Analyze the ethical concerns surrounding nudification and deepfake technology.

“But because of the ways the laws are written, anyone red teaming a chatbot for CSAM might actually be breaking law.”

Women and Online Harassment

32:30 to 35:15

Examine the systemic issue of online harassment targeting women and marginalized groups.

“So let's take a step back and analyze the larger implications for the debacle.”

Hypocrisy in Free Speech Discussions

35:15 to 37:18

Critique the inconsistencies in free speech arguments related to online harassment.

“One is while women are the brunt of a lot of this, it's also all underrepresented groups have been targeted for years and continue to be targeted, including children.”

Impact of Funding Cuts on CSAM Research

37:18 to 40:03

Investigate the effects of funding cuts on research dedicated to combating CSAM.

“In the process, they've gutted organizations that research ways to fight CSAM and deep fakes, like your former employer, as you noted just earlier, the Stanford Internet Observatory.”

Privacy vs. Safety in Child Protection

40:03 to 42:01

Delve into the balance between privacy and safety in the context of online child protection.

“Give us your best-faith version of your critics' argument and then explain the trade-offs between privacy and child safety online and why you think they're worth it.”

The Blurred Lines of Online and Offline Worlds

42:01 to 43:03

Discussing the interplay between physical and digital realities in privacy issues.

“because this is an issue of privacy for everybody and a safety for everybody.”

Accountability in the Age of AI

45:31 to 46:34

Examining the lack of accountability in tech companies regarding harmful AI outputs.

“Let's wrap up talking about accountability or lack thereof, really.”

The Future of Online Safety

46:34 to 48:35

A look into potential solutions and the role of outrage in driving change.

“show you how unsuccessful it is compared to its rivals.”

Envisioning a Safer Online World

48:35 to 50:45

Discussion on improving online environments and user agency.

“regulation is going to get us out of this just because it takes so long.”

Anticipating Challenges Ahead

50:45 to 56:00

Exploring the potential dangers of unregulated AI and upcoming elections.

“there's TikTok, which didn't ever actually get banned, right?”

Recognizing Technological Threats

56:00 to 56:39

Learn about the importance of recognizing the evolving threats from technology and their implications.

“We should recognize it as a threat and yet not overreact.”
Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:01Kara Swisher:It's on!

0:12Kara Swisher:Hi everyone from New York Magazine and the Vox Media Podcast Network. This is On with Kara Swisher and I'm Kara Swisher. On Christmas Eve, Elon Musk announced a new AI feature called Grok Image Edit. It's an image editing tool built into X, the social platform formerly known as Twitter. The tool allows users to create AI-edited versions of other users' images and then post the AI version as a reply to the original post on X. But because X rolled out Grok Image Edit with almost no safeguards, strangers flooded X with sexualized deepfakes of real people. For example, Grok could take a photo of a woman on X and create a realistic deepfake that looks like the original image, except the woman is now in a bikini and has, quote, donut glaze on her face.

0:57Kara Swisher:It could do the same thing with images of children, and it did, creating countless deepfakes. To add insult to injury, because those images were often posted as replies to the original tweet, the people who were victimized got notifications every time someone interacted or replied to the sexualized deepfake image. So in essence, Elon built a tool for creating and distributing sexualized deepfakes, and he did so in the most humiliating and degrading way possible for the victims. And X has arguably become the tool for creating and distributing AI-generated child sexual abuse material, or CSAM, what used to be known as child porn, and frankly still is.

1:40Kara Swisher:No surprise, governments across the globe have begun investigations into Grok, and after publicly mocking the controversy and accusing his critics of censorship, XAI has started putting guardrails on Grok with mixed success. It's deeply ironic considering that when Elon bought Twitter, he vowed that getting CSAM off the site was the, quote, priority number one and, quote, will forever be our top priority. But it's not surprising given that Elon has positioned Grok as the, quote, spicy and anti-woke chatbot, and it's not the first time Grok has made news for generating non-consensual deepfakes.

2:16Kara Swisher:Also, Elon, as he's proven time and again, always takes the heinous position on any subject. And this, I can't believe I'm saying this, is his most heinous. The backlash against Grok has been swift and widespread, thankfully. But so far, the consequences have been minimal. I think it's important to talk about this because I remember when everyone agreed that child porn was wrong. The fact that we're debating it right now is sickening. As a parent, as a reporter, as a citizen of this world, it's grotesque what these people are doing and benefiting from it financially and otherwise. My guests today are Renee DiResta, Hani Farid, and Casey Newton.

2:59Kara Swisher:Renee DiResta is the former technical research manager at Stanford's Internet Observatory. She studied CSAM for years and is one of the world's leading experts on online disinformation and propaganda. She's also the author of Invisible Rulers, the People Who Turn Lies into Reality. Hani Farid is a professor of computer sciences and engineering at the University of California at Berkeley. He's been described as the father of digital image forensics and has spent years developing tools to combat CSAM. Casey Newton is the founder of the tech newsletter platformer and the co-host of the New York Times podcast Hard Fork.

3:36Kara Swisher:This is a difficult but important topic. It means a lot to me. So please stick around. And to all the tech people who continue to resist doing anything about safety, especially of children, we are not going to stop until you lay down and change the situation.

3:58Kara Swisher:It's not just something you made. It's the privilege that you get to work with your hands. It's building something that serves a purpose. Proof that you have the grit to keep going. At Timberland, we understand you take your craft seriously, and we do too, which is why our products are built to the highest quality. We put in the work so you can perfect yours, with purpose in every detail and crafted with intention. Timberland, built on craft. Visit Timberland.com to shop. Here's something most founders don't think about until it's too late, their business address. When you register your company, that address shows up everywhere.

4:35Filings, public records, customer documents. Stable gives you a real business address you can use everywhere so your company looks professional and your personal address stays private. And Stable digitizes the mail that comes to that address. Sign up at ustable.com slash pod and get 50 % off your first three months of a grow or scale plan.

4:53Kara Swisher:The worst advice I have ever gotten is stay in your lane. I'm Robin Ersan, athlete, executive, founder, and staying in my lane would have kept me small. Don't protect other people's comfort at the cost of your growth. This week on Project Swagger, my strategies for embracing your multi-hyphenate existence. Tune in now at Project Swagger wherever you get your podcasts.

5:22Kara Swisher:Renee, Hani, and Casey, thanks for coming on on. Hey, Kara. Hey, Kara. This is a topic I really wanted to do a show about. I'm sort of incandescent as a parent and as a person who's watched this go on. And so I have feels about it. So just get ready. But thanks for being here. It's good to be with you. Grok has been in the news for weeks because it's facilitating the creation and distribution of deeply disturbing sexualized images. Some of them are non-consensual deepfakes and potentially even CSAM. But CSAM deepfakes and revenge porn are serious problems that they've plagued the Internet for years.

5:54Kara Swisher:This is not a new thing. So before we dive into the issues with Grok, let's talk about the scope and scale of the problem. Let's hear first from Renee, then Casey, then Hani. Okay. So as you note, this has been an issue for a very, very long time. Grok is not the first model to be capable of doing this. One of the things that I think we should talk about, first, it began to happen on Grok late December. So maybe we're almost a month into this issue. It began producing what you might call nudified images. So, nudification, the ability to do what's called inpainting, to change an image that already exists, is a technology that, again, has been around for a while.

6:33It's got a lot of other legitimate uses. It became something that on Grok, you could see it done very visually in the public eye. So this was something that had previously been done much more in platforms and communities that were dedicated to this kind of thing, right? So there were always, you know, communities that you could go and find, Discord servers, places where users would do this. Then there was a spate of apps that began to pop up that the app stores were playing whack-a-mole with where kids started doing this to harass their classmates. So there was this sort of small-scale little apps where this was happening.

7:12But Grok really made it something that was very front and center. People were being harassed in their own replies. And it began to be something that took that harassment out of the dark corners of the web, which was the threat model that a lot of people who have studied this for a very long time. That was how we had previously seen it manifest. And it took it and became something where it was happening at around almost 6 ,700 posts per hour was the stat that some of the researchers who tracked it took it to. So they essentially just amplified it. Yeah, it became the place where you could create, distribute, and honestly normalize.

7:48It became something that all of a sudden was normalized. You could just go and you could do it. And that was what was different about it. Okay, Casey? Yeah, I mean, you know, as Renee notes, nudification has been a problem going back many years. In the recent past, though, you couldn't get one of the frontier models to generate a photorealistic image of a naked person. You would kind of have to hack your way around it and deal with a subpar image. But nonetheless, there were tons of unscrupulous app makers who were buying ads on Meta and who were buying ads on Apple and Google's Play stores, getting these apps into the hands of thousands of people who were using them to terrorize especially young women.

8:31And it was a bad problem. And it did lead to things like the Take It Down Act, which did pass here in the United States. But as Renee notes, what has happened on Grok has just taken it to an insane new degree where you don't have to go scrounge around on the dark web to find one of these apps or use a subpar model. You can now use one of those powerful models that we have and harass people directly in their own replies in public.

8:55Kara Swisher:Right, in public. And so this is this spicy mode, Hani? Is that what it explained that? So first, I think Renee got it exactly right. What Grok did is they centralized the creation, the distribution, and the normalization of this content, which now dates 10 years back. In fact, the very term deepfake comes from the moniker of a Reddit user who used the then nascent technology to create the first non-consensual intimate imagery. So this is neither unintended or unexpected. And it was also preventable. So your point, Cara, about the spicy mode is OpenAI's ChatGPT, Google's Gemini have pretty good guardrails.

9:32And they have guardrails on the semantic inputs, what you type in. They won't allow certain types of prompts. They have semantic guardrails on the output. They will analyze the output image to determine if it shows a child, if it's sexually explicit. And Grok decided we don't want to do that, hence the spicy mode. So this is neither, you know, sometimes we have technology where people misuse it in a way that is unintended or unexpected. This is neither. This was we knew exactly what was going to happen. Elon Musk knew what he was doing and he allowed it to happen. And it's been awful. And Renee also made a really good point here is that this stuff has been around for a long time.

10:06Kara Swisher:Yeah, talk about that, the larger, broader decade. Yeah, let's talk about that. So we're seeing this up and down the stack. I mean, there's hardly a day goes by where I don't see an article about some school kids creating this type of images of their fellow classmates and then, of course, sharing it and distributing it. people are using it to weaponize against reporters, against academics, against politicians, against people they just don't like. And it is, you know, there's multiple facts, parts to this. So there's obviously the people who are creating the content. There's the tech they're using.

10:39There's the platform that empowers them. But then we shouldn't also forget, there's an entire ecosystem that is enabling this. Advertisers are still advertising on X. Apple and Google are still hosting these apps. Ads are being delivered by Google and Facebook. There's an entire ecosystem that is monetizing and profiting from this, not just the technology companies that create the content. And we shouldn't forget that entire Silicon Valley ecosystem is profiting quite handsomely from the creation and distribution of this content.

11:08Kara Swisher:Including of CSAM. Including CSAM, which is, you know, look, maybe reasonable people want to have a conversation about non-consensual intimate imagery. I don't think there's much of a conversation, but there's no conversation around CSAM, right? It's illegal and it's awful and it's dangerous. So this is not, you know, we're not talking about constitutional rights. We're talking about illegal content. Right. It's stuff we've agreed on a long time ago. And Casey, a lot has happened in just a few weeks. Foreign governments have blocked GROC. Other ones are investigating. So is California's Attorney General.

11:36Kara Swisher:The UK is working on passing a new law to make it legal to create non-consensual intimate images. But first, Elon mocked the controversy. Then XAI made Grok's edit a premium feature, meaning you had to pay for it, I guess. Now, XAI says they've instituted guardrails to stop it from nudifying images with limited success. Give us an update. There is an article in Bloomberg by Kurt Wagner saying that there is still a trickle of images that the public Grok account is still producing. So I think whenever Grok says that guardrails are in place. That is a statement we should view with deep suspicion.

12:12It continues to be the case that the standalone Grok app, so this is just something that you can download onto your phone and is rated for children 13 plus in the Apple App Store, you can use that to make nude images of women. And if you go onto Reddit, there are now thriving subreddits where people are just competing to see who can make the most explicit porn using Grok. So no, there really are not meaningful safeguards in the way that I think most regulators would want to see here.

12:40Kara Swisher:And where are regulators right now? Well, you know, as you know, we have started to see regulators in a number of countries step up. The countries with the strictest anti-porn laws have been the fastest to act. So Grok has now just banned wholesale in Indonesia and Malaysia. But if you look in the European Union, they're planning to take action. The UK is. Even the United States Senate passed a law designed to prevent some of the harms that we're seeing from Grok, although we'll see if it passes in the House. So this has managed to get the attention of regulators around the world. But, you know, an ongoing question with Elon is what sort of regulation, what kind of fine, what kind of threat actually results in the harm being addressed?

13:23Because I think it's just been the case over and over again that that's really hard to do.

13:25Kara Swisher:He just ignores it. Now, Renee, you've studied CSAM online for years. In 2023, you contributed to a paper that predicted that AI-generated CSAM would be a problem. Looking back, what did you get right and what else didn't you see coming besides Elon's heinous personality? Well, I think the willingness to fight about it actually was what has surprised me most about this recent debacle. I posted about this on Substack and I got some comment in notes It's like, well, this is all just a plot to censor X. And I thought, really, is that where we're going here? We're using the censorship, like the mental stop word censorship is where we're going with this.

14:09But yes, I mean, that is in fact where we're going with this. So I think that it is that term that he has chosen to use for any regulation is censorship. Any barrier of any kind is censorship. Any restriction is censorship. And now we are using that term to apply to the generation of images of underage girls, right, is censorship. anything that throttles that. And it works very well, unfortunately, because I think that term, that word, you know, it has a couple different meanings, right? It refers to legally, it refers to like kind of three different things, right? There's factual claims about speech suppression, saying like, this content was taken down by a private platform.

14:48That's one meaning of the word. It's a legal claim about US law, right? A violation of the First Amendment, which doesn't apply to private platforms at all, but nonetheless, they use it in that way. And then it's a normative claim about free speech values. And that is the expansive way that Elon is trying to use it at this point to normalize what is happening on his platform. And that is how it is being used. It is the idea that it is an affront to your free speech rights to not be able to generate nude images of other people. And that's what surprised me about it, actually. Including children.

15:22Including children. Yes, that's kind of a Nambla kind of argument.

15:25Kara Swisher:Yes. So seeing that that expansion is where I was like, man, I really you know, we used to have these fights about content moderation, right? The people who would scream at you that content moderation is censorship. But then they would say, but not but not CSAM. OK, that's that's that's not censorship. That's fine. And there would be this hard line where they would say, like, OK, but we'll concede on this point. And now I'm like, we're not that that's off the table now. Right now, that's censorship, too. We just we just took that off the table. We've expanded it to include that now also. And that's because they have to keep expanding that term to encompass any possible conceivable restriction.

15:58And that's what you're seeing here. As Ofcom has gone after, Ofcom being the British communication safety regulator, as they have gone after Elon, naturally he is reframing this as, oh, they just want to censor free speech on X. And they're mad at me for this, that, and the other thing. So that has been what has surprised me. Nothing related to the actual content. I mean, you know, when Stanford Internet Observatory existed, we used to go and we would present at the Crimes Against Children conference and we would see examples of inpainting, examples of all of the different ways in which, you know, the actual technology has manifested and changed over the last, you know, even just five years, like the advances in which this kind of content creation has shifted.

16:34So nothing about the actual technology is surprising here, nothing about the ways people try to get around guardrails, but the mainstreaming of it and then the application of censorship as a normative claim in this way and the aggression has been very surprising to see.

16:50Kara Swisher:So, Hani, you develop technology that can flag previously identified images of CSAM. You've developed tools to identify AI-created images. But if Grok and X help users create and distribute sexualized images of minors, there's obviously no technical solution for that. Even if tech companies make good faith effort to prevent their chatbots from creating sexualized images in the first place, AI will inevitably exacerbate what was already a very serious problem. Talk about how these AI tools complicate the efforts to fight it. For example, deepfakes can be used to extort minors. Someone could attempt to make an authentic piece of CSAM look like it's generated from AI in order to avoid scrutiny from authorities.

17:27Kara Swisher:There's all manner of problems here. Yeah. Okay. So there's many sort of threads to pull on here. So let's see a few things. So first of all, when the National Center for Missing and Exploited Children, NCMEC, has previously identified and categorized CSAM, we extract what's called the perceptual hash. And then we can prevent the re-upload of that content on things like Facebook and Instagram and TikTok and YouTube. That's called photo DNA. That's the work that I worked on with Microsoft back in the day. Right. And all the companies have been cooperating on that. Yeah, many of the companies, I mean, I would say, you know, it was kicking and screaming.

18:00They were cooperating. Now most of the big tech companies are on board, including X, by the way. But it was not an easy ask because as Renee was just saying, nobody wanted to open the doors of any content moderation because the thought was, oh, if we open it for CSAM, well, what else are you going to come for? So it was a fight back in the day to do this back in the early noughts. But, okay, here we are. The problem, of course, with AI-generated content is that it's not been previously seen. And in fact, you can produce it. I can produce hundreds of images on my laptop every minute. And so there's this flood of content that is coming on the Internet that we simply don't have the volume or the capacity to look at every single piece of content and flag it.

18:41That's number one. Number two is that, OK, maybe if the tech was very, very good, we could deal with the public facing version of this, things that are posted on X and Facebook. But what you are seeing, as you pointed correctly, is that bad guys are taking innocent photos of children, nudifying them, sending them in things like Snap and Discord and private messaging that we're not privy to, and then extorting them. And those kids are taking their own lives, destroying their lives. And it is horrific what we are seeing. And this is happening from organized cyber criminals from around the world.

19:18And we have seen story after story after story. And here's the thing is, you know, when you look at these technologies, everything we always see, and this has been true of Silicon Valley, as you know, for over 25 years, is safety is always an afterthought. We always try to backfill safety in.

19:33Kara Swisher:It's not even a thought. It's not even a thought. I'm being generous here, right? And you can't backfill safety. You can't do what X is trying to do now and be like, okay, we made a mess. We've normalized this. We've given everybody a taste of it. Now we're going to try to sort of backtrack it. It's not going to work. I'll add one more person, by the way, if I may, to the list of folks that are going after X, which is Australia's eSafety Commissioner, Julie Inman Grant, has opened an investigation, and they're doing really, is consistently doing really good work to try to hold these tech companies accountable for this.

20:02But this is a mess that was both preventable and not unexpected.

20:07Kara Swisher:We'll be back in a minute.

Read the full transcript

20:19Kara Swisher:support for on with Kara Swisher comes from Groom's if you're looking for a health goal that you can actually stick to you might want to check out Groom's Groom's is a simple daily habit that deliver real benefits with minimal effort their convenient comprehensive formula packed into a snack pack of gummies a day this isn't a multivitamin a greens gummy or a prebiotic it's all of those things and then some at a fraction of the price and bonus it tastes great Groom's ingredients are backed by over 35 ,000 research publications, while generic multivitamins contain only seven to nine vitamins Gruens have more than 20 vitamins and minerals and 60 ingredients, which include nutrient-dense and whole foods.

20:58Kara Swisher:That includes six grams of prebiotic fiber, which is three times the amount of dietary fiber compared to the leading greens powders, and more than two cups of broccoli. It's a daily snack pack because you can't fit the amount of nutrients Gruens does into just one gummy. Plus, that makes it a fun treat to look forward to every day. Kick off the new year right and save up to 52 % off with the code Kara at groons.co. That's code Kara, K-A-R-A, at groons, G-R-U-N-S dot C-O.

21:34Kara Swisher:Support for On With Kara Swisher comes from Samsara. Companies have to get product from point A to point B, and that means hiring drivers, which in turn means vehicle insurance, dash cams, and accepting the fact that accidents happen. That's exactly where Samsara can help. It brings AI dash cams, vehicle tracking, and asset visibility together in one simple platform. Samsara can help you protect your drivers, cut costs, and operate smarter. Their AI dash cams capture real-time video that prove when your drivers aren't at fault, protecting them from false claims. Their data shows companies have reduced fuel costs and improved driver retention thanks to better visibility and coaching.

22:13Kara Swisher:Here's a case study for that. DHL's focus on safety and more effective coaching through Samsara help lower driver turnover by 50%. According to Samsara's data, their AI helps reduce crash rates by nearly 75%. It's trusted by over 20 ,000 customers worldwide, including major companies across transportation, construction, and logistics. don't wait for the next accident to take action go to samsara.com slash kara to request a free demo and see how samsara brings visibility and safety to your operations that's samsara.com slash kara samsara operate smarter

22:53Kara Swisher:support for the show comes from iq bar iq bar protein bars iq mix hydration mixes and iq Joe mushroom coffees are delicious, low sugar brain and body fuel you'll need to win your day. Listen, we all need a snack now and then, but not all snacks are created equal. All IQ bar products are clean label certified and entirely free from gluten, dairy, soy, GMOs, and artificial ingredients. I've tried their bars and their mixes and they're quite good. I really like them. I use a lot of energy bars and I mostly don't like them. They don't taste good. they taste like cardboard. These are actually really flavorful.

23:29Kara Swisher:I've tried their mixes, very good, and they're giving me all the nutrients I need, and they're very easy and on the go. With over 20 ,000 five-star reviews and counting, more people than ever are fueling their busy lives with IQ Bars, brain and body boosting bars, hydration mixes, and mushroom coffees. Their ultimate sampler pack includes all three. Again, I've tried them all. They're all excellent, and some of these things, as people know, aren't. And right now, IQ Bar is offering our special podcast listeners 20 % off all IQ Bar products, including the ultimate sampler pack, a good way to test the stuff out, plus free shipping.

24:04Kara Swisher:To get your 20 % off, text CARA to 64000. Text CARA to 64000. That's CARA, K-A-R-A, to 64000. Message and data rates may apply. See terms for details.

24:20Kara Swisher:So Casey, when it comes to a lot of images, we're actually in a legal gray area in the U.S. at The Take It Down Act makes it illegal to publish non-consensual intimate visual depictions, both authentic and deep fake, but it doesn't take into effect until later this year. More importantly, what qualifies as an intimate visual depiction isn't clear cut. The Senate just passed, as you said, the Defiance Act for the second time, which allows victims of non-consensual explicit AI images to sue. But you run into the same issues you do with the Take It Down Act. It's not clear if the AI platforms themselves are liable.

24:52Kara Swisher:realistically. Talk about the legal recourse of victims of Grok's deepfake binge have right now. Well, I mean, it's terrible because it places the onus on the victim to go out and get justice for themselves, right? Like they have already been attacked in public and, you know, suffered in who knows what way. And now starting in May, there will at least be some avenue through the Take sort of justice done. But my fear here is that this is going to be yet another horrible customer service experience from a tech platform where you go onto a website and there's a form that you fill out and you send it in and you just pray that someone answers it, right?

25:37So I actually right now do not have a lot of confidence that this is going to feel like it is getting justice for any of the victims. And if you think about the sorts of people that are often targeted in these attacks, It is often women. It is often women who are, you know, politicians or leaders of some other type. And these sorts of attacks are used in an effort to silence them and to sort of drive them out of public life.

26:00Kara Swisher:Speaking of which, Ashley St. Clair, the former conservative influencer and mother of a child with Elon, has sued X over these deep fakes of her, and it's countersued saying she violated the terms of use by actually suing for her abuse. Yeah, well, in particular, because she didn't file in the Northern District of Texas, you know, which is just a frivolous, you know, legal action designed to make this, you know, more expensive and horrible for her. But my point is here, by the time that it comes to go fill out the form on the website that says, hi, like, I've been a victim of a horrible attack, the damage has really already been done here, right?

26:33And my fear is that this is just going to drive a lot of women in particular off of social platforms, which, of course, serves the interests of the fascists who run X.com. Okay. Can I add one more thing, too, is that the Take It Down has a 48-hour window. That's an eternity on the internet. The half-life of a social media post is about 90 seconds. The 48-hour window is completely absurd. The content has been online. It's been online for two days, and it's spread wildly. There's no taking it down at this point. So the whole thing, I think, is a fool's errand.

27:03Kara Swisher:So now Google, as you mentioned, Renee, Apple and Google could have kicked X off their app stores for violating their guidelines so far. Tim Cook and Satya Nudela have done nothing. They've said nothing. Renee, you wrote on threads that the app stores wouldn't do anything. Let's have you read what you posted. Okay, here we go. It's too big to regulate. It's not a two-bit nudify app, and Elon will howl censorship about second-order consequences. He uses the word even when Europeans fine him over blue check fraud. Similarly, congressmen that talk a big game about protecting kids with Zuckerberg are afraid of Elon.

27:32Josh Hawley and Marsha Blackburn should be speaking up. Talk about why they aren't and why they're upset about Elon in particular. So as we were talking about a little bit earlier, with the small 2-bit Nudify apps, there is one use for those apps. We all know what they are for. They are for harassment. And they are often made by kind of indie developers. And so when the mole is whacked, nobody complains. Or if they complain, you're not going to hear about it. Nobody really cares. When Elon complains, the world hears about it, right? There is an entire army of people who will come out and will have censorship right along with him.

28:09As I mentioned, again, we have now seen the expansion of that term to cover your perceived right to generate news of other people. As we've seen, Ashley St. Clair had the audacity to complain. She had her blue check revoked. She was demonetized, right? It was immediately made clear that that is the cost of complaining. When you do things to Elon's companies, all of a sudden you get tariffed, right? There are people who get their visas revoked because they have the audacity to implement fines on X for dishonest business practices, right? When X doesn't adhere to the terms of the Digital Services Act, that becomes an international issue.

28:49So the dynamics around regulating X are fraught to begin with. Because Josh Hawley and Marsha Blackburn have been very vocal on this. Right. So I'll get to them in just one second. But the other thing I'll say about this, though, is that the other the realistic point, though, is that the integration of Grok and X is a challenge. Right. So Malaysia and Indonesia block Grok. The challenge, though, is that people use X to communicate. It is a speech platform. And when you block X, you are blocking a lot of other types of speech that have nothing to do with nudification. And that is realistically one of the reasons why when I say it's sort of too big to block, it has a lot of other issues that go along with it.

29:27And so when you block the nudification piece, you have a lot of other kind of second order effects. But you also have Marsha Blackburn, who if you've heard her speak in tech hearings, she gets very, very upset about two things. One, AI doing bad things, right? You've heard her complain about, you know, where they say things like they make up accusations about conservatives and things like this. You'd think she'd be very upset about them nudifying women, right? She's a strong proponent of defending women. Two, defending children. These are people who support COSA. Josh Hawley has a very strong Christian populist sentiment in his kind of ethics underlying why he wants to see tech regulated, constantly writing letters to Zuckerberg asking him to account for himself.

30:07And the two of them were entirely silent about this. The person who spoke up was Ted Cruz, who spoke up but then also promptly posted a photo of himself with Elon, Right. So you have this this tension here, right, which is he knows this is wrong, but also is very, very afraid of being seen as challenging Elon, who is a major Republican Party donor. Right.

30:27Kara Swisher:He's just given 10 million dollars to Canada and Kentucky, for example. Krak isn't the only AI chatbot with vulnerabilities around D-Pix, obviously, or C-SPAN. Obviously, XAI seems to care less. But creating robust guardrails for a chatbot image generators is legally perilous. In order to fix any of the vulnerabilities, you have to find them. That means red teaming the chatbots, in other words, trying to get them to actually generate CSAM images in order to identify loopholes and close them. But because of the ways the laws are written, anyone red teaming a chatbot for CSAM might actually be breaking law.

30:59Kara Swisher:Honey, you've testified about this issue in CSAM. What would you like to see to help create a more robust infrastructure to fight it? Because that's one of the things that they, that's the problem here. Just like Ashley St. Clair suing and then being sued for suing. Yeah. Because she's not supposed to sue under the terms of service. So I do think, you know, we should add another component to this, that as somebody who has worked in the space and has viewed this content as part of my work, it is horrific. You don't want people looking at this. Even the fake stuff is gruesome and is the thing that nightmares are made of.

31:29But I do think it is necessary and we do need robust red teaming and we need mechanisms to make sure. And you're 100 % right that, you know, we continue to follow the path of move fast and break things. And, you know, look, some of these things are really hard problems, you know, and nobody's expecting perfection here. Nobody is saying you can never, ever, ever do this. But when you make it so easy, it is really inexcusable. And I think there are many, many things we can do from the guardrails on the training data. This is work that Renee has done on semantic guardrails on the input, semantic guardrails on the output.

32:05really vigorous red teaming. But the thing, of course, that you know is that none of this is good for profits, right? And so what we need here is really good either liability or regulatory overhead to create an incentive for the companies to do better. Because what we know is left to their own devices, they won't do better. And so it's on us as the public and as regulators and the court to force that upon them. Yeah.

32:28Kara Swisher:Now, they've also created a media narrative. So let's take a step back and analyze the larger implications for the debacle. Elon explicitly created Grok to be, quote, anti-woke. And the issue with the Grok image edit is, besides making it exceptionally easy to create these sexualized deepfakes, it's made it equally easy to use those images to humiliate and harass victims, as you've all noted. This is part of a much larger problem of bullying and denigrating women online. Casey, you've mentioned this is part and parcel of the larger trend of using harassment to, as you said, silence women, push them out of the public discourse.

32:58Kara Swisher:I'd love to hear E.G. talk about how this Grog Finesco fits in this larger systemic issue of harassment and silencing. Casey, you start, and then Renee, and then Hani. Well, I mean, look, women have never had an easy time online, right? Like going back to the earliest days of Twitter, you know, women were also having a terrible time. And, you know, under Jack Dorsey and Dick Costolo, the company also massively underinvested in tools that would just, you know, keep women not just safe, but also just feel comfortable participating. I think the fresh shock here is just that the company itself makes a tool that can strip a woman down to her underwear and shove that into her mentions.

33:37And the entire executive leadership team is just laughing it off, right? I mean, it's like a level of moral depravity I just didn't think we would get to in my lifetime, but now here we are. Mm hmm. Renee? I think I would just co-sign that. I think, again, it's where leadership has gone. It's the moral degradation on that front. I think we had seen a trend from 2015 with, you know, Gamergate and attitudes around how should we think about harassment and the fact that if you think about free expression, there was a balancing act, right? a recognition that certain types of speech push other types of users out of the conversation.

34:20And so you wanted to try to maximize that equity to create that environment that allowed the greatest possible number of users to continue to contribute. So saying we're going to allow these people to express themselves by creating nude images of those other people, right, means that those other people are going to exit and go somewhere else. They're going to cease contributing. Their speech is going to be stifled. They are going to be chilled. But it's almost like we've sort of reverted back 10 years into that view of equities and conversation as we've just decided that there's a kind of new sheriff in town and this one just doesn't care.

34:56Kara Swisher:Yeah, doesn't care. More than doesn't care. Thinks it's hysterical, honey. But remember, Kara, remember, you're not allowed to create an account on X that posts the recent whereabouts of Elon's plane because that would make him unsafe and it would make him feel like he can't speak. But if you wanted to strip down any women on X, that's fine. Fair point, Casey. Honey? Yeah, I'll add two things here. One is while women are the brunt of a lot of this, it's also all underrepresented groups have been targeted for years and continue to be targeted, including children. And so that umbrella is actually quite large of people who are being harassed online.

35:29The second thing, and I want to sort of reemphasize something Renee said, I always like it when people like Elon Musk talk about free speech because what he really means is free speech for me and people who agree with me. If you are a true believer in free speech, you want a platform that brings all voices, not just the voices of people who agree with you. And by allowing this type of content, you are in fact attacking free speech for lots and lots of people. And so if you want to be a free speech person, fine, be that. But let's at least be consistent with that worldview.

35:58Kara Swisher:Oh, you've noticed the hypocrisy? Shocking, isn't it? Shocking. It's been that way forever, actually. I know. And I know I'm stating the obvious. But it's, you know, I sort of really, that hypocrisy in particular really frustrates me. Because, first of all, I think there's lots of restrictions on free speech, first of all. But this is not a free speech issue. This is absolutely a red herring that is used to silence critics. That is correct. So the right obviously has also been obsessed with pedophilia. I was walking across Union Square Park this weekend and the QAnon people are still there, still there, still protesting something.

36:33Kara Swisher:Pizzagate and QAnon were about supposed pedophile rings. Obviously, so is the Epstein story, though that one is grounded in reality. And then there's the fixation with the drag queen story hour and calling LGBTQIA people groomers. And yet we see this casual attitude towards Elon and Grux exploitation and harassment of minors online. It's never been about pedophilia. It's never been about this. That's the point, right? We don't like some people. We're going to tag them with something that seems like it's impossible to ignore. And then when your own guy gets tagged with it, we are silent. This has never been about protecting children.

37:06Right.

37:06Kara Swisher:This is what Elon did to Yoel Roth. And as you all know, Yoel was the head of safety at Twitter before he just quit. And then the minute he quit, Elon attacked him for being a pedophile, which that's his go-to. Pedophilia is Elon's go-to. Trans or pedophilia is his go-to. Renee, you and many other researchers who study online hate, disinformation and misinformation and C-SPAN have essentially been harassed out of a job by MAGA lawsuits, congressional letters, funding threats, and now deportation threats. In the process, they've gutted organizations that research ways to fight CSAM and deep fakes, like your former employer, as you noted just earlier, the Stanford Internet Observatory.

37:44Kara Swisher:What's the current state of play when it comes to research on CSAM? Well, I mean, losing SIO is a really big deal. Well, when I got this, I got a text from a reporter, gosh, it might have been like New Year's Day, I think, talking about some of what was happening and saying, hey, I've got this, I have this tip, this person saying they think they have AI-generated CSAM. I don't know what to do with it. And I said, well, you know, don't look at it. Right, right, right. But then there's this question of like, OK, where do we send it? And, you know, I was kind of DMing with my former colleagues trying to figure out, all right, well, who can we send this to?

38:23We used to have special mechanisms for dealing with it. And obviously there's like NICMEC and a bunch of other places. But as far as like researchers who are still in the space, this question of who are the research groups that are still equipped to do certain types of work on this. I think there are different groups that had different specialties that are no longer operational in part because of the funding cuts and other things where, you know, saying the rather gleeful dismantling of research centers because maybe they studied misinformation or maybe they had the audacity to say the 2020 election wasn't rigged and stolen, you know, which it wasn't.

39:03Right. So it doesn't exist. Right, exactly. Well, I mean, SIO shut down as a result of that. I think I'm heading into year three of the litigation to court cases against us. And Jim Jordan sent yet another letter to Stanford complaining about researcher speech. You know, they had a dinner apparently with a funny of Julian McGrath and a few people, which Michael Schellenberger and Jim Jordan reframed as some sort of censorship cabal. So they sent another letter demanding emails to try to get to the bottom of that. And so the harassment continues. The stupid conspiracy theories continue. And meanwhile, the question of who does important work to actually solve real problems, you know, the centers that used to really specialize in that get shut down.

39:54Kara Swisher:Do not exist. And they continue to ask someone like you, especially that ever idiotic Michael Schellenberger. Honey, your work has been somewhat controversial, though, in the industry. People that are not Michael Schellenberger, like Mike Masnick, the founder of TechDivac, feel you're too willing to compromise the privacy rights of everyone who uses the Internet in order to try to catch people who create. or distribute CSAM. Give us your best-faith version of your critics' argument and then explain the trade-offs between privacy and child safety online and why you think they're worth it. Well, first, let's acknowledge that there is tension here between privacy and safety.

40:28And that's not a strictly online thing. It's an offline thing, too. When I go to the airport, they get to look through my bags and my person. So there's a trade-off we have to make. And so one position to say is absolute privacy for everybody and safety is a second, third thought. The other is privacy for nobody, and that's a problem also. So I think reasonable people can disagree here. My position has been this. When we talk about privacy, we should talk about privacy for everybody, not just you and me. Because when I talk about safety measures for children, that is a privacy issue for the children.

41:05And so I reject this notion that I'm throwing privacy under the bus. Right now the argument that i've heard that is go something like this that if you do this Go searching for child sexual abuse material. Well, then you will enable people to do xyz the slippery slope argument And the reality is that when we built photo dna back in the day We heard those same arguments and it just was not true People said to me in my face you are going to destroy the internet you will destroy speech and it was not true I reject the notion that we can't have reasonable safety measures and reasonable privacy. This notion that everything absolutely has to be end-to-end encrypted and nobody gets to do anything, I think sort of defies the sniff test when it comes to the physical world.

41:53Nobody would say that when you go to the airport, nobody's allowed to look in your bag because it's a privacy issue. So what I would like is for this to stop being a conversation about privacy and safety because this is an issue of privacy for everybody and a safety for everybody. And I think as we have done in the physical world, we have to find a compromise. And I think that that is not a conversation that has been particularly healthy in the last few years.

42:15Kara Swisher:Yeah, I don't know why it shifts so drastically when it comes to online child pornography. I find it strange. I find it strange, too. And here's the thing. I mean, maybe 20 years ago we could talk about an online world and an offline world. There's no distinction anymore. There's no online and offline world. There's the world. It's one world. And we have to start coming to grips with that. We'll be back in a minute.

43:04Kara Swisher:dairy, soy, GMOs, and artificial ingredients. I've tried their bars and their mixes, and they're quite good. I really like them. I use a lot of energy bars, and I mostly don't like them. They don't taste good. They taste like cardboard. These are actually really flavorful. I've tried their mixes, very good, and they're giving me all the nutrients I need, and they're very easy and on the go. With over 20 ,000 five-star reviews and counting, more people than ever are fueling their busy lives with IQ bars, brain and body boosting bars, hydration mixes, and mushroom coffees. Their ultimate sampler pack includes all three.

43:37Kara Swisher:Again, I've tried them all. They're all excellent. And some of these things, as people know, aren't. And right now IQ Bar is offering our special podcast listeners 20 % off all IQ Bar products, including the ultimate sampler pack, a good way to test this stuff out, plus free shipping. To get your 20 % off, text CARA to 64 ,000. Text CARA to 64 ,000. That's CARA, K-A-R-A to 64 ,000. Message and data rates may apply. See terms for details.

44:08Kara Swisher:Support for this show comes from Harvey AI. Law is a craft forged through repetition, sharpened by judgment, perfected in the details most people never see. Practice Made Perfect is Harvey's belief in relentless refinement, in turning complexity into clarity, pressure into precision, and experience into advantage. Legal work isn't just about knowing the law. It's about applying it carefully, consistently, and under real stakes. Harvey is an AI platform built specifically for the legal practice, helping teams analyze large volumes of documents, draft with precision, and collaborate securely across complex matters.

44:45Kara Swisher:It doesn't replace expertise. It strengthens it, making teams faster, more exacting, more certain. Today, more than half of the AM Law 100 use Harvey as part of their workflow. Learn more at Harvey.ai. Kayak gets my flight, hotel, and rental car right so I can tune out travel advice that's just plain wrong. Bro, Skycoin. Way better than points. Never fly during a Scorpio full moon. Just tell the manager you'll sue. Instant room upgrade. Stop taking bad travel advice. Start comparing hundreds of sites with Kayak and get your trip right. Bad advice? You talking to me? Kayak. Got that right. Let's wrap up talking about accountability or lack thereof, really.

45:34Kara Swisher:There may not be any accountability around this whole ordeal. It boosted Grok's downloads, and Secretary of Defense Pete Hegseth announced that Pentagon will be integrating Grok into its workflows. And nothing happened with Apple and Google around not doing anything. They didn't get any—they didn't get shamed. They were not—they never responded. So what's the takeaway from this whole episode? So it'll be just another grimy chapter of the story of X and Grok, like the Mecca Hitler tweets or explicit Taylor Swift deepfakes going viral on X. Is there any possibility of real change? I think that to the extent there's going to be real change, it's just going to come from outside the United States.

46:12I do think that X's existence is just going to become more complicated in more countries around the world. It's going to become more of a headache for Elon. So maybe we'll see some measure of accountability there. If you're looking for any sort of silver lining here, though, I would just say that the fact that Grok has to stoop to these levels just show you how unsuccessful it is compared to its rivals. The introduction of its porn chatbot last year was just explicitly a way to try to boost engagement for what is essentially the fourth or fifth place AI company. So to me, this is just a sign of increasing desperation over there to get something working.

46:55And yeah, so that's what I would say about that.

46:58Kara Swisher:And then dress it up like free speech, essentially. Yeah. So, Honey, a review by Wired Magazine shows Grok is creating violent sexual images and videos with minors in them that is much worse than what's being posted on X. We assume the malicious actors are going to find ways to use AI to create illegal deepfakes in CSAM. What sort of AI tools can you imagine would be useful in combating the distribution of these images while respecting these privacy rights you talked about and minimizing false positives that can tar an innocent person's reputation, for example? Yeah, yeah, yeah. Well, first, like everything in cyberspace, this is an adversarial system, right?

47:31Like spam, malware, and ransomware. It's a give and take, and it's going to be a back and forth. There are, I mean, if you go to OpenAI's ChatGPT or Google's Gemini, and you try to put in the same prompts that you put into Grok, it won't work. So we have an existence proof that we can have reasonable measures in place. And this is the way it works. You solve 80 % of the problem relatively quickly, and then you start chipping it away with a combination of red teaming and responsible reporting mechanisms. And we get to a steady state the way we have with spam. It's a mostly manageable problem these days.

48:04But that takes effort, and it takes years and years and years to do that. And it takes will and it takes leadership. And my thing, if I can come back to the question that you asked Casey is, I think there's, there's one positive thing that has come out of this, which is the really loud outrage from the media about what is happening. And I think it, I article after article after article, just really hammering. And I think that's good to see that there is some consequence to this. Number one, number two is I don't think regulation is going to get us out of this just because it takes so long. And it's so imperfect in the way it's crafted.

48:41And of course, the lobbyists are going to beat the hell out of the regulation before it sees the light of day. I think where this starts to get traction is with liability. So putting aside how Elon Musk is threatening people. But if we start suing these companies back to the dark ages, they will internalize that cost and they will make safer products. And so I'm looking to the courts for relief here because I think that's the fastest way to get relief.

49:02Kara Swisher:You're absolutely right. You have sued the bastards. That's what I said. But Renee, we're talking about very dark, disturbing trends, and the underlying assumption behind your work is that we can have a better internet. So paint a positive picture. Let's imagine a world 10 years from now where there's a momentum towards creating a safer online world, something I've talked about a decade ago, speaking of which, one of less hate and harassment. What does it look like? What would you need to do? What are the pieces that you need to put in place? I mean, some of it is, as Ani mentions, there is the courts, right?

49:32There's the FTC also, which I haven't heard anybody mention. But I know. Oh, please feel free. Well, Baron Soka and some other folks are writing about consumer protection powers. I don't know if this FTC will do it again because of the unfortunate, you know, integration of Elon into the Republican Party and the political tensions there. But but that that's another potential option. Brandon has has an enormous crush on him.

49:53Kara Swisher:It's a little bit untoward, frankly. But one of the things that we saw. I may create an image or something. I don't know. There's the exit versus voice phenomenon, right? Users have voice. They can say, we don't like this. We don't want it. You should, I would hope, see more conservatives on the platform saying this isn't appropriate. This isn't right. This isn't normal. That's what I would have wanted to see instead of people falling in line behind the censorship bullshit. But the other thing is exit. You can leave. There are other platforms. You can go there, right? Threads is a little weird, but it exists.

50:29Blue sky is a little weird in some ways, but it exists. There are alternatives and users can vote with their feet and at some point they're going to have to. And just to say, this isn't normal and that's the other option. And that's where you just say, there are other places to go. There are other places to be. There's Instagram, there's TikTok, which didn't ever actually get banned, right? There's so many different options. It's not that they're necessarily good, but there are alternatives. And that But user outcry in the United States in particular has done more to move platforms in various directions with their policies than any regulation because we haven't had any regulation here.

51:10And that is, I think, the last lever really available to say that this isn't OK. We don't want it. It's not moral. It's not ethical. And we should be doing something different.

51:18Kara Swisher:Yeah. Yeah. I do my best by calling it a Nazi porn bar, as you know, all the time. But it seems to have caught on a little bit. So if we take a step back, this episode with Grok is just the tip of the iceberg when it comes to issues created by unregulated AI. I'd like each of you to finish up by talking about what else is coming our way, what worries you most. What is the thing you're thinking about next if you had to anticipate? I mean, I think we have anticipated that Elon will do the very worst thing given the choice. But what worries you the most and what do you think is coming our way? Casey, then Hani, and then Renee.

51:52To me, the worst aspect of Elon's underinvestment in content moderation and basically just like rejection of it as something that anyone should do is that it has just lowered the bar for every other player in the industry. Like, Meta is so much worse now than it was before Elon bought Twitter. And it's just because they feel no pressure to do even half the stuff they used to because they look at what Elon gets away with and they think, well, you know, what does it matter if we're going to get 10 % of our revenue from literal scams this year, right? Or you look at Roblox and all the issues that they've had with child predators, and they just look at it as, ah, that's just a compliance function.

52:33Let's go ahead and do the bare minimum. So to me, that's the real issue here is that when there is someone who is so publicly able to get away with the worst things imaginable on their platform, everybody else follows suit because it's cheaper.

52:47Kara Swisher:That's a very good point. Is there something coming that worries you? I mean, the thing that worries me is just that we're already in a world where people have trouble distinguishing fact from fiction. The underlying models continue to get more powerful. There's a huge contingent, particularly on the left, that thinks that AI is fake and a bubble and is all going to collapse tomorrow when, you know, open AI can't pay a bill. And so I think a lot of us are just sort of sleepwalking into a future where the media environment becomes ever more challenging and dangerous. And it doesn't seem like many people in power want to do much about that.

53:20Kara Swisher:Good point. Hani, then Renee. Sure. I mean, a lot of things keep me up at night these days. But here's Bill Gates had this great line many years ago. He said we tend to overestimate what tech will do in one year and underestimate what it will do in 10 years. And that's been true for quite a while. I think in the age of AI, that 10 years is probably closer to five years and maybe three years. I think the world is going to look very different in three years. And here's the thing I'm looking at, particularly for 2026, which is the age of agentic AI is upon us. So everything we have talked about for the last 45 minutes or so has been humans in the loop.

53:56Humans are powering this in terms of creating the content and then sharing it and liking it. But soon what we are going to see is fully deployed agentic AI, state-sponsored cyber criminals, individuals will have agents doing the work for them. Fully automatic, 24 hours a day, no sleep, no moral, no fear of arrest. And that's going to be a very weird world that we are entering in. And I think cybersecurity in general is going to explode with attacks from agents that are working 24-7 and adapting very, very quickly. And we've already started to see attack from AI-written malware and ransomware. And I think the world of agentic AI, more likely than not in the next 12 to 18 months, is upon us.

54:41And I think that is going to change an awful lot.

54:43Kara Swisher:So supersize this. It's already difficult and a huge problem. Okay. Renee? Yeah, I would concur on that. I think it's the prevalence of fraud. I think the financial industry is beginning to realize that systems it had built, using voice as an authenticator, insurers have been using voice as an authenticator. A lot of systems that were a lot of money invested thinking that this was going to be secure, the realization it is not secure, and efforts to try to find things that are. The privacy tradeoffs with things like biometrics that people are very uncomfortable with, particularly in the hands of governments, and yet at the same time the recognition that there have to be mechanisms for credentialing in some way in part to mitigate fraud.

55:23Right. So what are those tradeoffs going to look like? What is that? What is a privacy protecting identity layer look like? Again, recognizing that people don't want the government to control it. And yet the private sector options are not great either. So that's an area that I've been looking at a lot lately and wondering, you know, what on earth happens on that front. And what worries you the most in this area? You know, we've got another election cycle coming up right now. I think those questions around trust and legitimacy are going to play out very heavily in the race. You know, I've never been in an AI election doomer.

55:58I think there's always been that it's going to matter in some way. We should prepare. We should recognize it as a threat and yet not overreact. And so, again, yet another cycle coming up, gearing up for that because we have an incredible, you know, real fractured trust environment here. And we've seen the technology advance a lot over the last two years. And we're going to see what that looks like going into this next cycle.

56:24Kara Swisher:All right. But we can be sure Leland will do something repulsive. Anyway, never underestimate his repulsiveness. Anyway, goodbye and thank you so much, Renee, Hani, and Casey. It's a really important topic. I'm going to keep coming back to it all year around chatbots, around kids, around suicide, around everything because they're not going to do anything about it. As you said, the media certainly can. I really appreciate it. Thanks, Kara. Thanks, Kara. Thanks, Kara. Thanks.

56:56Kara Swisher:today's show was produced by christian castor-russell michelle eloy megan burney and kaylin lynch nishat kurwa is vox media's executive producer of podcasts special thanks to katherine barner our engineers are fernando aruta and rick kwan and our theme music is by trackademics go wherever you listen to podcasts search for on with kara swisher and hit follow thanks for listening to on with kara swisher from podium media new york magazine the vox Media Podcast Network and us. We'll be back on Monday with more.

57:31Kara Swisher:Support for this show comes from Harvey AI. As AI reshapes professional services, law firms and in-house teams are rethinking how complex work gets done. Harvey AI is an AI platform built specifically for the legal practice, helping teams analyze documents, draft with precision, and collaborate securely across matters. Today, more than half of the the AM Law 100 Use Harvey. Learn more at harvey.ai. Tires matter. They're the only part of your vehicle that touches the road. Tread confidently with new tires from Tire Rack. Whether you're looking for expert recommendations or know exactly what you want, Tire Rack makes it easy.

58:14Fast, free shipping, free road hazard protection, and convenient installation options. Go to TireRack.com to see tire test results, tire ratings, and consumer reviews, and be sure to check out all the special offers. TireRack.com, the way tire buying should be. Rinse knows that greatness takes time, but so does laundry. So Rinse will take your laundry and hand deliver it to your door expertly cleaned, and you can take the time pursuing your passions. Time once spent sorting and waiting, folding and queuing, now spent challenging and innovating and pushing your way to greatness. So pick up the Irish flute or those calligraphy pens or that daunting Beef Wellington recipe card and leave the laundry to us.

58:58Rinse. It's time to be great.

From the publisher

On Christmas Eve, Elon Musk’s X rolled out an in-app tool that lets users alter other people’s photos and post the results directly in reply. With minimal safeguards, it quickly became a pipeline for sexualized, non-consensual deepfakes, including imagery involving minors, delivered straight into victims’ notifications. 

Renée DiResta, Hany Farid, and Casey Newton join Kara to dig into the scale of the harm, the failure of app stores and regulators to act quickly, and why the “free speech” rhetoric used to defend the abuse is incoherent. Kara explores what accountability could look like — and what comes next as AI tools get more powerful.

Renée DiResta is the former technical research manager at Stanford's Internet Observatory. She researched online CSAM for years and is one of the world’s leading experts on online disinformation and propaganda. She’s also the author of Invisible Rulers: The People Who Turn Lies into Reality.

Hany Farid is a professor of computer sciences and engineering at the University of California, Berkeley. He’s been described as the father of digital image forensics and has spent years developing tools to combat CSAM.

Casey Newton is the founder of the tech newsletter Platformer and the co-host of The New York Times podcast Hard Fork. 

Questions? Comments? Email us at on@voxmedia.com or find us on YouTube, Instagram, TikTok, Threads, and Bluesky @onwithkaraswisher.

Learn more about your ad choices. Visit podcastchoices.com/adchoices

More from On with Kara Swisher

All 139 episodes
Elon’s “Nudify” Mess: How X Supercharged DeepfakesOn with Kara Swisher · 54 min
Listen in VO