Why nobody's stopping Grok

25:35Nilay Patel:Support for this show comes from LinkedIn. For small businesses, every hire matters. But the time and resources required to hire right are limited. Luckily, LinkedIn Hiring Pro is built for that reality. It's your hiring partner designed to help you hire with confidence by surfacing only the right candidates without turning hiring into another full-time job. Posting a job isn't always the hard part. It's finding, connecting with, and screening the right candidates. Hiring Pro streamlines the entire process from drafting your job to shortlisting candidates and conducting AI-powered interviews for initial screenings.

26:13Nilay Patel:Conversational interface lets you describe what you need in plain language. No recruiter jargon needed. Nearly 60 % of hirers find a candidate to interview within a week. With Hiring Pro, you spend less time searching and more time connecting with the right talent. Hire right the first time. Post your first job and get$100 off towards your job post at linkedin.com slash partner. That's linkedin.com slash partner. Terms and conditions apply. Support for this show comes from Framer. If updates to your site are feeling harder than they should, Framer is the shortcut you've been looking for. Framer is a website builder that can transform your dot-com from a formality into a tool for growth.

26:59Nilay Patel:They've helped thousands of businesses from early-stage startups to Fortune 500s build better websites faster. Framer is an enterprise-grade, no-code website builder used by teams at companies like Perplexity and Miro to move faster. With real-time collaboration, a robust CMS, and everything you need for great SEO and advanced analytics that include integrated A-B testing, your designers and marketers are empowered to build and maximize your dot-com from day one. Learn how you can get more out of your dot-com from a Framer specialist or get started building for free today at framer.com slash decoder for 30 % off our Framer Pro annual plan.

27:40Nilay Patel:That's framer.com slash decoder for 30 % off. Framer.com slash decoder. Rules and restrictions may apply. Support for this show comes from LinkedIn. For small businesses, every hire matters, but the time and resources required to hire right are limited. Luckily, LinkedIn Hiring Pro is built for that reality. It's your hiring partner designed to help you hire with confidence by servicing only the right candidates without turning hiring into another full-time job. Posting a job isn't always the hard part. It's finding, connecting with, and screening the right candidates. Hiring Pro streamlines the entire process from drafting your job to shortlisting candidates and conducting AI-powered interviews for initial screenings.

28:30Nilay Patel:Conversational interface lets you describe what you need in plain language. No recruiter jargon needed. Nearly 60 % of hirers find a candidate to interview within a week. With Hiring Pro, you spend less time searching and more time connecting with the right talent. Hire right the first time. Post your first job and get$100 off towards your job post at linkedin.com slash partner. That's linkedin.com slash partner. Terms and conditions apply.

29:05Welcome back. I'm talking with Brianna Pfefferkorn about content moderation. For the break, she mentioned something called the Paxton case. Before we really get into it, that's Free Speech Coalition versus Paxton, a case the Supreme Court decided last year, where Justice Clarence Thomas wrote in the majority opinion that it's constitutional for states to make websites and online services do user age verification. We covered this deeply on The Verge. We'll link to our coverage in the show notes. Okay, let's go. I'm glad you brought up the Paxton case because it actually helps me think about what's going on with Grock in a very specific way.

29:37I didn't think the Supreme Court issued a great decision in the Paxton case. But a lot of the rationale was, hey, we left this alone for a long time. But now all the kids have these devices. And it's not out of bounds for us to say you got to try harder. So the states are allowed to say you got to try harder. And all of these concerns about privacy, about data security, you guys are smart. You figure it out. Right. Like that was the thrust of that case. Like it's different now. And so all these precedents from the 90s when everyone was on desktop computers, whatever, like all the kids have phones now.

30:10You can get porn so easily asking to verify your age, figure out the privacy problems. Like you're so concerned about – figure it out. Like just solve it. But there's a compelling interest here to do age verification. I'm not saying that I thought the specifics of that argument were great by the Supreme Court. I'm saying that that's what that felt like to me. Like it's different now. The scale is different. The scope is different. The access is ubiquitous. We need to make some rules. Like it's time for some rules. And I know a lot of parents who generally agree with that. Parents who grew up with the internet and access and are like, I don't want my kids to have the experiences I had on AOL in the 90s.

30:45There should be a lot. I'm putting that right next to Grok where it's like, yeah, I can just put anyone I want in a bikini in a sexually suggested pose and no one seems to be doing anything about it. I can't square those two impulses. Can you? There's just something really weird about that. I agree with you that I am not a fan of the Paxton decision. And that underlying rationale of like, well, set aside the security and privacy issues. You just got to start like making sure that you can collect everybody's like biometrics or their ID or whatever in the name of keeping children from accessing pornography.

31:20It's like, OK, well, we've seen everybody's easily able to circumvent those, whether it's with a VPN or with using a realistic looking like game character to fool like an age verification tool. and so you're not keeping people from accessing the content you're trying to keep them away from and on the flip side like the moment we started seeing these laws actually come into effect there was like a massive data breach at discord of like the third-party vendor that they were using for doing age verification so great you don't get the upside that we were promised with these age verification laws and you also get the downside that everybody warned you was going to happen it happened immediately we didn't even have to worry about being called slippery slope uh you know hysterical zealots, it happened immediately.

32:04And everybody took the saddest, I told you so, victory lap, because of course, nobody wants to be in this situation where the laws are only impeding, you know, British people from accessing Wikipedia or whatever, and not actually doing the thing that they were promised to do. And if anything, I think age verification has gotten such a degree of cultural influence of like, now this is the thing we're going to use to fix all of these problems that I see it being deployed in the context of non-consensual deep fake pornography. I was giving a lecture a few weeks ago about the research that I've done on this area.

32:41And one of the questions that came up during Q and A was, well, if you want to deal with like nudify apps and making inappropriate imagery of children, isn't age verification the answer? And And no, the problem isn't whether it's a 14-year-old or a 24-year-old who's making non-consensual deepfake imagery of somebody. It's that any deepfake imagery without their consent is being made of somebody. That's the problem. And so it was this weird sort of window into when you have an age verification hammer, now everything starts to look like a nail. And that is just not the case. not having adequate like you know controls over who can view and make the non-consensual like bikini imagery is not the place to be focusing on and so it remains to be seen whether this is the thing that breaks everybody out of that mode of thinking that like if we just pass age verification in all 50 states you know the internet will somehow be a better place to be and instead we're just seeing like now not only are we seeing the failure of age verification for the benefits without all the drawbacks that it was purported to have, but also we're seeing what used to be with regard to nudification apps kind of confined to this like seedy underbelly of the internet as like escape containment.

33:59And now is integrated, you know, vertical integration directly into a major online social media platform. Let me zoom out on age verification. I'm not really asking about the technique of age verification. I'm more asking about the policy trade-off, the posture we seem to be in, where everyone knew the trade-offs of age verification. They were not unclear to the industry. Everyone attempted to make it as clear as we could to policymakers and the Supreme Court. And the answer was, well, we have to do something. It's so bad we have to do something and we did something and then immediately all the negatives happened and maybe there's no upside.

34:36But the sense that this was such a huge problem, that something had to be done and we should do this thing because it was a solution that was proposed, was pervasive. Like I heard it at every level. I heard it from our audience. I heard it from policymakers. We heard it from the court. This is a problem. Kids are getting exposed to too much bad stuff. We have to do something. And the first thing we can do is just figure out who's an adult and who's not, regardless of the trade-offs that are obvious to everyone. I'm looking at that. This is so bad, we have to do something. And that's really the idea I'm centered on.

35:07And then I'm looking at Grok. And it doesn't seem to be so bad that anyone has to do anything. And I just can't square those two things existing in the same universe as part of the same storyline. And I think what I'm hearing from you is, well, age verification might be the thing that excuses Grok. Because now you're saying, well, at least they're all adults or at least we did the thing and now this exists and it can be excused. That seems just as bad. But I still can't quite put these two ideas. the internet is so bad for kids, we have to create the opportunity for huge breaches of privacy and data leaks.

35:42And we should just let Elon do whatever he wants to pictures of women and children on the internet. Yeah, I mean, if anything, my hope is that the Grok incident is revealing that age verification is not this one size fits all solution to like the ills of the internet. And in terms of like what to do about it, like I have some fear that this will be interpreted by some regulators around the world as, see, this is why we need to expand bans on being on social media entirely to people below the age of 16, like they have done in Australia, which is being challenged legally by a group of teenagers themselves.

36:17I'm afraid that we'll see the total ban approach of just like, don't even let teenagers get on the internet at all, or at least on some defined subset, always a weirdly defined, both under and over-inclusive subset of online services until they hit some magic number of an age, that this might further that kind of request. Not only are they not happy with age verification in some regulatory like quarters of the world, but also that they will want to push to exclude young people from even larger swaths of the internet, irrespective of the upside of what you might be able to find on certain services.

36:55I'm not going to sit here and say that there's a lot of good stuff to be found on X anymore in these days. But surely, you know, we have so many stories from kids of how important it is for them, however marginalized or isolated that they may be, that they can find community in the spaces online that regulators are trying to ban them from. You know, we've seen some countries have at least temporarily decided to block Grok entirely under their own nation's laws. But I am afraid that this is going to feed into even more thirst for similar under 16 social media type bans like the one that we have seen in Australia.

37:30You will not find a lot of big fans of Section 230 among the people fighting against NCII. I personally am a big fan of Section 230, but I understand why the anti-NCII folks who have been trying to fight this thing that really harms real people in their lives and can follow them around for years, why they are not big fans. The Section 230 piece of this is fascinating on a number of dimensions. Section 230 is, I'm sure the Decoder audience knows, is the law that says platforms on the internet are not legally responsible for what their users post, except in a handful of circumstances. And it has basically allowed the platforms to exist at the scale they exist.

38:09The interesting thing about Grok is that it is generating the images. It is doing so in response to user queries, but XAI, which owns Grok and which owns X, is the party that is generating and publishing the images. Does that change the 230 dynamic here? Are they responsible for the thing that they were publishing? So a few things. One is Section 230 has never barred federal criminal enforcement of the law, and both with respect to whatever content counts as CSAM under federal criminal laws and whatever violates the new law, take it down now, defines a federal crime of publishing non-consensual deepfake pornography of adults and minors alike.

38:48So for those particular, whatever images qualify as one of those two categories, nothing stops the DOJ from taking action against XAI or X in that regard for something that violates federal criminal law. That's just the DOJ. It doesn't get at civil liability or like state AGs, like the one that California has now opened an investigation. What you're mentioning is something that actually there's also this other portion of Section 230 that says like it has to be information provided by somebody else, right? Some other third party. Usually that's a user. And it doesn't bar liability, including civil liability, where it is the platform itself that's contributing to what it is that makes the content illegal or where the platform is generating the content itself.

39:36We have seen statements from Ron Wyden, the senator who has been around in Congress long enough that he helped write Section 230, saying that AI output is not meant to be covered by that. It is not supposed to be immune by Section 230. My understanding is that it's not yet a settled question in the courts. And my prediction is that this is the situation that's going to give rise to some amount of litigation where that question will be squarely presented. Is the output of an AI, a generative AI tool covered by Section 230 immunity or not? I think we're going to find that out in court opinions probably sometime this year.

40:16And in fact, we're recording this on January 15th. I am surprised that I haven't seen a class action complaint filed against X and XAI in federal court yet. I would have thought that some enterprise and plaintiff's law firm would have filed that immediately already with ripped from the headlines quotations. Hasn't happened yet. Maybe it's because things are moving as fast as they are and people are trying to just make their best case. But we've seen, you know, even what's called the cottage industry of law firms that have focused in recent years on bringing litigation against platforms over things like social media addiction allegations for teenagers.

40:51And over, we've seen some lawsuits about the outputs of chatbots allegedly culminating in suicide, unfortunately. In one of those chatbot cases, you know, the defendants didn't raise a Section 230 defense. It's always incumbent upon the defendant to raise that. I think they raised something with regard to the First Amendment. That case just settled pretty recently. So we don't really have an answer yet in that regard. What does Section 230 do with regard to civil liability for the output of generative AI tools? But based on both the general weakening of Section 230 over the last decade or so, where courts have seemed to be much more willing to bend over backwards to find a way to say, no, it doesn't apply.

41:31and the success that plaintiffs have had here and there in terms of pleading around Section 230 to say this isn't about user content, this is about defective design features and having some success, not 100 % of the time, but some success there in not getting Section 230 to apply combined with, you know, I think there's been Ninth Circuit oral arguments, trial and so on in some of these social media addiction cases. We are in a place where 10 years ago Section 230 might have been expected to form a much more robust defense as opposed to here where we are now in a place where both the kinds of arguments that plaintiffs have realized that they can make and sometimes make successfully and the willingness of courts to not be as eager to apply Section 230 might mean that there may be more likelihood for plaintiffs to have some success if they want to take on this grok standal in court.

42:27Again, though, we also should be continuing to pay attention to like, where is the DOJ on this? We should not be letting them off the hook either. It can feel, I think, kind of hopeless to say, wow, the world's richest man is in cahoots with the federal government who allegedly they care about child safety and like protecting women online as evidence from signing the Take It Down Act. But where is the Department of Justice? Are they investigating XAI? And X, if not, why not? But we also are in a weird situation where, going back to take it down, eventually platforms, including X, are going to have to remove this imagery when victims file a complaint and want it to report.

43:08It has to come down within 48 hours. The enforcement of that is going to be the Federal Trade Commission. There's two guys left on the Federal Trade Commission because Trump fired the Democratic ones. And the ones that are left are like far-right anti-porn zealots. You know, like literally one of them was a fellow with the Heritage Foundation whose Project 2025 says pornography should all be illegal. It should be wiped off of the Internet. The people who produce it should be put in jail. And platforms that distribute it should be mixed out of corporate existence. And oh, by the way, pornography includes trans and queer people existing at all.

43:42So what happens once those takedown provisions go into effect and it is for the FTC to oversee whether platforms, including X, including Grok, are taking this material down when victims depicted it ask for it to be taken down? Are we going to see Elon Musk at the zenith of his power be able to influence what either DOJ does or what the FTC does? Or are we going to see the weird far-right zealots who are now the only people left minding the store at the Federal Trade Commission saying, actually, this helps us with our overall anti-legal porn agenda? One of the interesting things about sort of MAGA Republicans dealing with this, you know, there are many women in the MAGA Republican ranks and I've seen some of them come out and say, this isn't the time for law enforcement.

44:29We trust X to do the right thing with new technology. And I have no idea where that's coming from, where that belief that they have a shared understanding of what the right thing is and that Elon wants to actually execute the right thing. Because everything I see from Elon, he's made a number of public statements now, seem to suggest he's very irritated he's not allowed to run this image generator any way he wants. And that any government saying what he should do with it is censorship that he should fight, even as he sort of reluctantly says he's complying or X says it's complying. The dynamic right now is there's pressure from the UK.

45:07X announces that the system won't work in the UK. a Verge reporter in the UK tests the system and it still works just fine with the most trivial of workarounds, right? You can see the company does not actually want to give up this functionality. It wants the appearance of giving it up to make the pressure go away. And I just wonder, you know, inside the politics of this all, at least in this country, there is still the sense that you shouldn't do this. And the method by which you arrive at not doing it is, well, you have to either believe that Elon doesn't want to do it and he's trying to fix it, or you have to actually pressure the company into not doing it, using the tools of the state.

45:43And I'm not sure that MAGA is going to get to tools of the state. I had not yet thought about the particular Project 2025 chapter that Andrew Ferguson, the head of the FTC, has written. But it's true that there is that element of the Trump administration that is in its way ascendant. Have they said anything yet about any of this besides take it down? Does the FTC have any say here whatsoever? I haven't seen any statement by the FTC. The only statement that I've seen coming out of DOJ was to reiterate that AI CSAM is illegal and that they will go after people who produce and possess it, but that's still focused on the end users.

46:22So the Department of Justice, for its part, has said that they will go after people who produce and possess AI-generated CSAM, but that's still only focusing on the end user, not on X or XAI. Now, historically, it has been uncommon for the DOJ or the FBI to comment on whether an ongoing investigation is open into anybody in particular. That's a norm we've seen fall by the wayside whenever Pam Bondi thinks that it is convenient for her to somehow totally, like, manage to undercut her own investigations. But I digress. So, you know, I think it's worth asking, like, what are they doing about it? Are they going to investigate anything?

47:02And just, you know, see what they say. You never know. Lindsay Halligan has shown us that people are willing to run their mouths off when they think that they're not being recorded. We don't necessarily know if the DOJ is investigating. We don't know how the FTC will plan to enforce. And I think that those sorts of dynamics of like nobody is coming to save us mean that there might be a greater like appetite amongst average people to say like, OK, well, what can we be doing about this? Should our lawmakers be changing the laws? We saw a very quick passage in the Senate for a bill that would open up private right of action for suing those who make non-consensual deepfake imagery of you as something that was a pretty noticeable hole in Take It Down that only gives enforcement to the FTC.

47:50And if people are looking around saying, well, this thoroughly corrupted DOJ is the only people who can go after platforms for federal criminal violations. And if the FTC is the only one who will be able to enforce violations of takedown requirements under take it down, shouldn't there be some other changes to allow greater accountability? Maybe that's something that will drive change, even potentially here in this Congress. We'll see. Because, again, ostensibly, victimizing women and children is something that both sides of the aisle allegedly care about. Allegedly. It's funny. As we are speaking, a headline just came across the transom.

48:29The Wall Street Journal just published, Influencer who had child with Elon Musk sues XAI over deep thick images. Ashley St. Clair, who had one of Elon's many, many children, seeks a restraining order to stop the Grok chatbot from undressing Earth AI. This idea that there's a private right of action against the platform, new-ish, these are the sorts of lawsuits that have typically hit the 230 wall. The Defiance Act has not passed yet. Do you think she's got a chance? I'd have to see what her lawsuit says. Her allegation is that Grok is unreasonably dangerous as designed. So there you go. You're going to get the design defect type of allegation.

49:05And that is not surprising to me at all. As I was mentioning, we've seen in social media addiction type cases that plaintiff's attorneys have argued it is not about the content of users' posts or whatever, and therefore Section 230 should not apply. We're talking about defective design. And sometimes those claims do get around Section 230. We'll see. Now we'll see if XAI and X want to actually finally bring up Section 230 in response or whether they will only invoke the First Amendment or will they choose to just settle this quietly and make it go away. It remains to be seen. But like I was saying earlier, it is a little bit of guesswork involved in depending on what court you get, what judge you get, what arguments get made, what will come of these types of claims.

49:54But we have seen in the social media addiction context that some of those have gone pretty far so far and haven't been thrown out immediately on Section 230 grounds early on. And so we can expect, you know, if Elon chooses to fight this, which is nothing if not litigious, that we may expect a concerted legal battle coming out of this.

50:15We have to pause here for another quick break. We'll be right back.

50:26Nilay Patel:Support for the show comes from Anthropic, the team behind Claude. If you've spent any time lately trying to get an AI to do something useful, not just sound impressive, but generally help you think through a hard problem or task, you may have heard the name Claude. Claude is made by Anthropic, and they've got a few things going on right now that are worth paying attention to. First, Cloud Code. It's a command line tool for developers that understands your entire code base, run tests, iterates on solutions, and then handles complex tasks end to end. Developers have been using it for everything, not just coding.

51:02Nilay Patel:And that's exactly what led them to build something called Cowork, which takes the same agentic power and brings it to everyone else. No terminal required. You point it at your files, set a task, and Cloud works through it autonomously. in the background while you focus on other things. Less back and forth, more deep, sustained work getting done. Anthropic is committed to keeping that conversation ad-free. That means no sponsored suggestions, no third-party influence on what it tells you. Claude's only job is to help you keep thinking. Try Claude for free at claude.ai slash decoder and see why problem solvers choose Claude as their thinking partner.

51:48Nilay Patel:Support for today's show comes from Upwork. The fastest growing businesses aren't doing more. They're delegating smarter. Upwork helps you bring in expert freelance help fast so you can delegate and keep moving. Upwork is a one-stop platform to find, hire, and pay expert freelancers across web and software development, data and analytics, marketing, business operations, and more. They help grow your business by giving you fast access to specialized talent across 125 plus categories. So you can fill skill gaps, launch projects faster, and scale support up or down without committing to full-time headcount.

52:25Nilay Patel:And with Business Plus, you can access the top 1 % of talent on Upwork. And with AI-powered shortlisting, you'll get matched to the right freelancer in under six hours. No endless searching required. You can visit upwork.com right now to post your job for free and connect with top talent ready to help your business grow. That's U-P-W-O-R-K dot com. Upwork dot com. Support for the show comes from Shopify. Starting a new business, it could be a lonely endeavor, especially in the beginning. And when you're starting out, it's more important than ever to make sure you have the right tools at hand. Shopify is the commerce platform that millions of businesses around the world rely on to sell their products online.

53:10Nilay Patel:If you're asking yourself, what if people haven't heard about my brand? Shopify helps you find your customers with easy to run email and social media campaigns. And if you get stuck, Shopify is always around to share advice with their award-winning 24-7 customer support. Best yet, Shopify is your commerce expert with world-class expertise and everything from managing inventory to international shipping to processing returns and beyond. It's time to turn those what-ifs into with Shopify today. You can sign up for your$1 per month trial and start selling today at shopify.com slash decoder. Go to shopify.com slash decoder.

53:54Nilay Patel:That's shopify.com slash decoder.

54:06Welcome back. I'm talking with Stanford's Rihanna Pfefferkorn. Right before the break, we're talking about the mess of laws, regulations, and courts that might enable someone to do something about Grok. That someone, by the way, includes Ashley St. Clair, the mother of one of Elon Musk's children, who is now suing over the fact that Grok has undressed her. But in the worlds of apps and websites, regulators and lawsuits are just one piece of the puzzle. So I wanted to know who else could act here and why aren't they? We've talked a lot about the sort of legal frameworks here, what courts and governments might do, what individual actors might do using those laws.

54:42Earlier, I mentioned that there's kind of a universe of actors here, right? There are payment processors and app stores and all kinds of other pressure points that might serve as gatekeepers or guardians of norms. I'm not sure that there are norms anymore in 2026, but that's kind of the idea, right? You don't always have to go to the government to solve these problems. You've built these systems, and the system should have some points of pressure to make sure things are operating well or safely for people. The ones that come to mind for me the most here are the app stores. Apple and Google have become empires.

55:15They've become some of the richest companies in the world on the back of having total control of the applications on their platforms, charging absurd rents to those app developers for the transactions that take place. They have endured antitrust lawsuit after antitrust lawsuit. And their argument is always, we need this control in order to keep our users safe. It is the bedrock principle of their defense, particularly for Apple. And in this case, they have done nothing. Like literally nothing. They won't even comment when we write stories about it. As far as I can tell, they've commented to no one.

55:51Why do you think that is? Why do you think there's such silence from the two biggest and baddest gatekeepers of all? There's other reporting in The Verge just calling Tim Cook and Sunderpatriac cowards. There's a letter from multiple U.S. senators, Senator Wyden, Senator Markey, Senator Lujan, basically saying X and XAI, these apps are violating your own terms of service against non-consensual sexual imagery of people. Well, why aren't you taking these down? You know, the way you did with IceBlock and Red Dot, the perfectly legal apps for doing the perfectly legal act of observing ice when they're kidnapping our neighbors off of the streets.

56:31And it's hard to discern any, certainly any like moral or ethical principle in refusing to take action here. Again, you know, we're not operating on a blank slate. If you remember a year ago, at this exact time a year ago, everybody was freaking out over the impending banning of TikTok from app stores, something that a group of senators had all decided that needed to be done, that the Supreme Court had upheld as like the power to do. And everybody freaked out. And then it just kind of went away. TikTok came back to be available. Again, despite the fact that you would think any of these app stores would think there's potentially ruinous liability under the law that Congress passed and that got signed into law and the Supreme Court upheld if we allow it to still be in the app store going forward.

57:16and then it just kind of went away. And I think you're pointing to how we are in a different realm now from where it used to be that any kind of jawboning by government actors was supposed to be completely unacceptable. We even had a Supreme Court case against that. Now we have a letter from three senators saying you should remove these apps from the app store because of violations of your terms of service. That's jawboning. But I don't think a lot of people are necessarily pissed off about that. And at the same time, we're now in a place where actually it turns out you can just jawbone companies if you are Pam Bondi, et cetera, and threaten them.

57:55And the people who used to be mad about jawboning are nowhere to be found on that. And so it seems like we are past just sort of normal expectations around the rule of law or around principled enforcement of your own policies when it comes to apps that are doing harm to people every single day. and it might just be that we're going to find out whether Elon Musk companies are just too big to fail. When I've talked to people about like, well, but aren't the app stores at least removing Nudify apps? The people I talked to who researched this will say, you know what? I can still do a search for Nudify apps on the app stores and I'll still get a lot of results.

58:31So there isn't even necessarily consistent enforcement against things that exist only for non-consensual deepfake pornography that are not in any way a general purpose tool for image generation or a major social media platform. And if they can't even or won't even take action against all of those apps, and meanwhile you have a highly litigious head of apps that are in major mainstream use, it's hard to see how it can be justified if you are Tim or Sundar. But it's unfortunately kind of easy to understand why cowardice is the easy way out. We didn't write that headline and I stand by it. The thing that grabs me, I do.

59:11This to me feels like if you have spent the better part of a decade insisting that you are the only party that can keep users on your phone safe and then you lie down now, you're doing it for some other reason. And if you can't even articulate that reason, I do think that's cowardice. The thing that gets me though, that kind of boggles my mind, this undoes their big antitrust defense. Like Apple has an antitrust case coming up. The Department of Justice has under the Biden administration filed that case. The Trump administration is pursuing it still. We'll see how it goes. But here in this moment is the single greatest piece of evidence that Apple doesn't actually care about those rules.

59:53Right? And I just wonder, are they cutting off their nose to spite their face? Like at some point, you're going to end up in court and it's going to come up that they have selective enforcement of their rules. and that selective enforcement undoes the primary reason that they claim they need to maintain their monopolistic control of the application environment. And I just, I can't square that. It feels like pissing off Elon in the short term is fine if you can protect your app store monopoly in the long term. Right. And, you know, it doesn't just suggest that like maybe the government is in the right here that like, well, you're just misusing your own monopoly power to pick and choose which apps you think deserve to be online.

1:00:33If it's monitoring ICE, then it doesn't deserve it. But if it's, you know, the non-consensual deepfake porn machine, you're willing to leave that up because of other considerations, then I agree with you that it seems like it does draw into question the sincerity of those claims. I mean, I'm sympathetic to the privacy and security, especially the security review type arguments for why we need a walled garden that charges 30 % for the privilege of being in the walled garden. But it becomes a lot harder to defend, as you said, when they are picking and choosing their own enforcement. I also think that 30%, and this is also something that they got sued over, is worth examining right now because for as long as payment processors and the app stores take a cut of the paid service for making the non-consensual deepfake nude machine go burr.

1:01:25It seems like the payment processors and the app stores are also should be asked, why are they taking a cut of the victimization of women and children? That is the next turn of the story. And it's actually the next thing I wanted to ask you. There are the payment processors, right? The visas, the MasterCards. We see them in response to things like Tumblr, like revoke their services. There's a lot of very legal erotica on the internet that has been taken down or demonetized because of payment processors. We see this all the time. And they also seem to be doing nothing in this case. Is this colorable?

1:02:00Can you say because you provide payment services to X, which provides a huge range of services, you can narrowly look at this one problem and say, we should pull our services? Because maybe that's too far. Maybe there's too much abstraction for the payment processor. It's hard, and I don't want to really make a forecast about what should happen there, because anything that you go different layers at the stack, I think, has very different arguments compared to when we're just talking about the app itself. There have been various arguments in the past about whether Section 230 comes into play. Again, if you go one level down, we are starting to see now efforts to try and target layers down the stack with anybody who's providing services to non-consensual like deepfake pornography services.

1:02:46California passed a law that didn't get a lot of notice last year that not only outlawed non-consensual deepfake pornography services, it also allows for liability for service providers to those services. Once they are put on notice that they have a deepfake porn service as a customer, if they don't then drop that customer, they can be held liable. And that's the kind of further down the stack liability that I think we will see more of. I've seen the UK government talking about this. Minnesota made a kind of hand-fisted attempt at writing a bill to do the same thing that would have inadvertently outlawed all AI-generated porn, not just non-consensual ones.

1:03:23But opening up the realm of further down the stack liability to try and go from like, well, okay, well, if the user material doesn't get taken down, let's go after the platform. If the platform isn't responsive, let's go after the AWSs and Cloudflare and Stripes of the world. I think we might see potentially more of that also in response both to the general problem of like nudify apps that are still out there and bringing in millions of dollars a year and in response to Grok acting more like one of those deep fake nudify apps as opposed to like a big major good faith online service. Good faith is sort of an interesting place to wrap up.

1:04:00It's been on my mind. I don't think X or Grok are acting in good faith. But a lot of these systems presume good faith actors. They presume the good intent of the platforms, of most of the people on the platforms. And I look around and I say, well, the era of content moderation policy that was based on good faith, that was based on the idea that, I don't know, Twitter was going to cause democratic revolutions across the world or whatever we thought it was going to do, that's all gone. In fact, the era of content moderation itself might be gone. I look at Instagram. Instagram is overrun with sexualized deepfakes of celebrities all day long.

1:04:41You want to watch Trump dance without a shirt on? You can do that on Instagram all day long. It's not great, but you can do it all day long on Instagram. Maybe you can't prompt it into generating it for you on demand, but Meta's designs and AI certainly suggest that you will get to that point. Meta is moderating racism, overt racism. It's comments less and less. I see it every day. I see the same thing on YouTube. YouTube has long evaded all scrutiny. There's more and more pretty gross content all over YouTube and YouTube shorts all day long. You just see it. The platforms have pulled back from the heights of 2020 content moderation where they thought they had a big responsibility.

1:05:19And now they have seeded all that responsibility. And now X and Grok might be all the way at the end of that road. Where not only are ceding responsibility, they're actively thwarting that responsibility or anyone that might impose that responsibility. Is this just the pendulum swinging back and forth or is this an actual reset moment where we should think about the platforms and their ability to act in good faith with good intent very differently? I mean, it has laid bare how like craven the motives are for major platforms that even before the inauguration, right after the election, already we were seeing announcements from online platforms that they were pulling back from content moderation.

1:06:00Either they were going to say, we're going to put that on to end users to do community notes instead of having fact checkers ourselves, or we're going to have AI tools do the moderation for us, or just straight up backtracking on things like, oh, yeah, let's just say dead naming is okay again. We're not going to bother ourselves too much with that. I think you're right that there will be a natural ebb and flow to how much effort companies put into content moderation, depending on political factors, social factors, business factors. But I think it is now much more clear that people cannot rely upon these companies to view their trust and safety teams as something that is overall benefit and boon for the company rather than a cost center to try and be minimized wherever possible.

1:06:46And I think this also feeds into why we have seen such a great increase in interest in, you know, defederalized social networks, in the rise of middleware, and giving more tooling that is built by, for, and about the communities that are most adversely affected by noxious content online to try and put tools back into the hands of users to tailor their experience. which, ironically, going back to the roots of Section 230 to begin with, was one of the things that was envisioned to happen coming out of 230. Not just that you would have immunity for taking stuff down or leaving stuff up, but that we would see a greater ecosystem of content moderation tools that would be encouraged to flourish, to restore that level of control and customization and curation in the user's hands.

1:07:38Instead, I think a lot of us got used to just relying that it would probably be in the hands of these large centralized platforms to do that for us. And now, as we're seeing, we have an increasing need for alternative ways of doing that, whether that is getting off of out of the Nazi bar or whether that is at least building up greater tools made by the people who understand the use case and the need for them. It's always dangerous to ask this question when something is moving as fast as the situation seems to be moving. And especially when I know we're going to publish this episode several days after you and I are having this conversation.

1:08:12But I got to ask, what do you think happens next in a Grok situation? What do you think the next big turns are? What should people be looking for? So we already have seen it took them three weeks to finally announce that they were going to disallow the generation of bikini pictures. Turned out that didn't work, but it took them three weeks to get there. it's difficult to predict exactly what will happen but I will kind of be unsurprised if the outcome of this really does end up being outright saying it is censorship to tell us we can't generate imagery of 14 year old girls like that has felt like it was always where the content moderation is censorship discussion was going to go and this would be the logical culmination of that I think kind of as you were hinting at your eye that we might just see further public statements about changes being made, which do not actually work in actuality, whether because their heart isn't in it or because, as I've discussed elsewhere, safeguarding AI models really is hard.

1:09:06One thing I don't think is going to happen, unfortunately, is seeing them pull Grok's image generation or image editing features offline because, frankly, there's just too much money in it. That is an odd motivation for the richest man in the world, and yet it seems to be a motivation. Rihanna, this was an illuminating conversation. I hesitate to say it was great because I find myself deeply disturbed by all of it. But thank you so much for coming on and explaining all this to us. It seems like you and I are going to have many more conversations in the year to come. God, I hope not. Not because of you, just because of the dumpster fire.

1:09:38Yeah. That's true. Yeah. I hope we have different conversations. Thank you. Yes. I'd like to thank Rihanna for taking time to speak with me and thank you for listening. I hope you enjoyed it. If you'd like to let us know what you thought about this episode or really anything else you want to discover, drop us a line. You can email us at decoderattheverge.com. Like I said, we really do read all the emails. You can also hit me up directly on Threads or Blue Sky. We're also on YouTube. You can watch full episodes at DecoderPod. And we have a TikTok and an Instagram. They're both at DecoderPod as well, and they're a lot of fun.

1:10:05If you like Decoder, please share it with your friends and subscribe. We're going to get your podcast. Decoder is a production of The Verge and part of the Boxing Me Podcast Network. The show is produced by Kate Cox and Nick Stat. It's edited by Ursa Wright. Our editorial director is Kevin McShane. The Decoder music is by Breakmaster Cillan. We'll see you next time.