OpenAI Admits AI is Killing the Internet

0:00Emanuel:This episode is brought to you by PayPal. You know how a mom's bag has everything? Sunscreen? Snacks? A stapler? The new PayPal app is like that, but for your money. Shop, pay, manage your account, and earn rewards all in one place. And with purchase protection on eligible items, biometric security, and pass keys, you're protected at every step. Download the new PayPal app to get started. See paypal.com slash protection terms. Microsoft did this study that after they started adding AI to their search results, clicks to news sites and the New York Times fell by more than 90%.

0:58As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co. Also do remember to subscribe to our YouTube channel where you can watch all of our episodes of all of our fancy graphics, including this one. Subscribe at youtube.com slash at 404media.co. I'm your host Joseph and with me are all of the other 404 Media co-founders, the first being Sam Cole. hey emmanuel mayberg hello and jason cabler hey what's up all right so before we get to the stories a tiny bit of housekeeping something of a request i will put a link to this in the show notes but as i'm sure listeners and readers will know we've obviously been covering flock a lot and a lot of this is through public records requests to sheriff's departments, police departments, cities, all of that sort of thing.

1:59And we ask them to provide the network audits of Flock. And this shows, wow, this cop is using the Flock system to look up, you know, license plate for this reason. It might be, hey, I'm looking up for ICE. I'm looking up to find a woman who self-administered an abortion or anything like that. Now we are doing much the same thing with Axon, the law enforcement contracting giant. This is because a bunch of cities are ditching Flock and they're just replacing the cameras with Axon ones like we covered a while ago. What we're asking our listeners and readers to do, if they're interested and if they want to sort of be engaged in their own community on this is file a very specific public records request with their local agency if they use Axon obviously to get back basically the Axon equivalent of the network audit go to link in the show notes you will see a copy of the request there and hey if you're interested file it find out how license plate reading cameras are being used in your community and if you do find anything interesting please feel free to send it to us or email it to me or Jason or whoever, and then we'll look into it.

3:12All right, let's get on this week's stories. The first one by Jason is, quote, doom loop, end quote. Open AI and Microsoft admits LLMs are destroying the web and built on theft. This isn't in the headline. Oh my God, he admitted it. Obviously, I had to say that. Let me read the quotes just at the top. There's a couple of them and then we'll drill into where this comes from and everything. But you say in the lead that these companies are omitting that LLMs are based on, quote, an astonishing theft of unprecedented proportions, the largest theft of labor in human history. And there's an internal document that we'll get into as well that says AI products have created a doom loop that is killing the entire web.

4:03Jason, first of all, where do these comments come from exactly?

4:07Emanuel:Yeah, so all of this comes from the New York Times versus OpenAI and Microsoft lawsuit, which we've talked about before, but has been at least a year, probably. This is a huge, huge copyright lawsuit. Of the copyright lawsuits against AI companies, I would say this is probably the biggest one. And I think the reason for that is because the New York Times is like big mad about it. Like it's been now multiple years in litigation. The accusation is essentially that OpenAI scraped the New York Times website by ignoring its robots.txt. So ignoring its wishes, going around its paywall, and then ChatGPT for a long time was directly reproducing New York Times articles, more or less, in some cases, whole cloth.

5:05Emanuel:And then in other cases, it would be like 85 % of the article would be the same and there would be like some different words in there. But for the most part, it was the same. And so this lawsuit has expanded out to include Microsoft, which obviously is a huge funder of OpenAI. They have that special partnership. And so this case has been winding its way through the court for like years at this point. And both sides have now filed for summary judgment, which is essentially where their lawyers take all of the evidence that has come out during discovery, which is when the court lets opposing attorneys look at all these internal documents.

5:47Emanuel:They let them depose important people at both sides. So basically, all of these OpenAI and Microsoft executives have been interviewed by the opposing counsels. And the summary judgment filing is based... It's kind of like a closing argument document, more or less, where the lawyers on both sides are like, judge, here's what you should rule and why. And so this is like, I think it's like a 40-page document. And it's a summary judgment filing by the New York Times' lawyers. Yeah, and it's interesting because we don't get to see those depositions, right? Like we can't see the recordings of those themselves.

6:25We can't see these internal Microsoft documents because they are part of discovery, right? But what we can see is that the New York Times lawyers have found a way or able to do it in such a way where they can summarize sort of the evidence that they've been able to access and gather and then put it in a public court record, right? And then we can digest it. So what we're talking about here is sort of that summary of it. We don't have the internal Microsoft documents or the open AI sort of depositions or anything. But, you know, there's no reason to doubt sort of whether this is real or not. It's in an official court record.

7:03Emanuel:That's a really good point. So just before we get into what was said, a lot of this case has been filed under seal, meaning you can see who was deposed, but you can't actually see what they were saying because basically Microsoft and OpenAI were arguing that these were trade secrets and things that needed to remain confidential. And interestingly, this summary judgment was filed and then it too was under seal and then a redacted version was released where huge parts of it were redacted at the request of Microsoft and OpenAI's lawyers. So big black boxes over the text. And that came out like two weeks ago.

7:47Emanuel:And that document was interesting. But then the court went and unsealed this entire document, the unredacted version. And so we can see quotes from these depositions. We can see quotes from these documents that came out in discovery that the lawyers chose to quote from. But we can't see the underlying source. And this is actually somewhat unusual. It's like usually discovery documents you can obtain unless there's a compelling reason to keep them sealed. And in this case, that reason was basically like Microsoft and OpenAI saying that this is confidential information. And so it is notable that even parts of this have become unsealed.

8:31Emanuel:And as you said, it's like, there's no reason to doubt these because they're all in the court record. And they're all cited, as in like the lawyer says like, oh, you can go find this in the docket, but those documents are sealed. And so the court has access to them, but we do not. So there's a few things that this document pulls from. And one is an internal Microsoft document where I believe Microsoft executives discuss this idea of unprecedented theft. What did they say specifically about the theft stuff? Yeah, so they say this in a couple different places. There's this quote, basically like the training of LLMs is, quote, an astonishing theft of unprecedented proportions.

9:17Emanuel:And they also said that it's the largest theft of labor in human history. This quote came from Microsoft's director of applied science, who is basically like a Microsoft executive that came from a deposition. and kind of like throughout the case or throughout these quotes, the executives at these AI companies are admitting that one, they were trained on copyrighted material, which we've known forever, but it's still notable that they are kind of saying it in court. And then the other thing that they're saying is that this is a competitive product with the New York Times and with journalists in general.

9:56Emanuel:They're trying to make a fair use argument and the fair use argument, And some of the things that you consider when something is potentially fair use under copyright law is whether it's transformative, meaning whether it's substantially different from the original source, which these AI companies have argued that LLM training is. and then also whether it competes with that original source, like whether it destroys the market for that original thing that you're stealing or taking or using or remixing. And what these executives are saying is that, yes, ChatGPT and LLMs and AI in general is making work that is substantially similar to what the New York Times is doing, that people are using ChatGPT in a way that they used to use the New York Times, meaning they're getting their news from it.

10:50Emanuel:And then interestingly, there's a ChatGPT engineer in here saying that they have tried adding links to the output from ChatGPT and people don't click. People don't go to the original source. Satya Nadella was in here. The CEO of Microsoft was in here saying that it's a competitive product. And also Microsoft did this study that after they started adding AI to their search results, clicks to news sites and the New York Times fell by more than 90%. So I mean, just like a huge, huge decrease. And all this aligns with what we've seen in real life. News sites are not getting clicks from Google anymore because of AI overviews like that.

11:34Emanuel:Traffic has cratered, so on and so forth. As you alluded to, it's different to hear them say it. And of course, this isn't a deposition of Google, but the same would apply most likely to Google AI summaries as well. And we've seen other reporting on that as well. So they definitely know that they are sabotaging the news industry, which ironically is the thing that they are pulling all the reporting from to then present to their own users, right? I guess just on that briefly, because you brought up sort of the media stuff, there's a bit where open AI admits to getting around the New York Times paywall.

12:15Now, I don't think this is probably going to be the most sophisticated thing in the world. I think anybody on the internet probably knows how to get around certain paywalls and that sort of thing. But what did they say and what is the significance of that?

12:29Emanuel:Yeah, they didn't explain how they did it. I mean, it would have theoretically been possible just to make a bunch of accounts, to buy a handful of subscriptions. Obviously, there's various ways to get around paywalls in general. But it basically quotes these OpenAI internal chats where someone at the company told OpenAI co-founder Greg Brockman that they had created, quote, a hack to get around New York Times paywall. And then Greg Brockman said, oh, nice. Um, so it's just like evidence in there that they knew that they were doing something wrong. There's also parts in this document that I didn't quote from where Satya Nadella was like, I would have hoped that we would have understood that scraping from paywalled content is like not a good thing.

13:20Emanuel:I didn't include that quote because there was like a lot of context to kind of explain how he was saying that. But basically, there was various executives being like, Like, hmm, maybe we shouldn't have taken that paywalled content. I mean, the document that I thought was most interesting, and which we've kind of mentioned, but haven't gotten into what it is, is this one that's in the headline about Doomloop. It is an internal Microsoft document that basically assesses what LLMs are doing to the industry and specifically what their strategy is doing. And so the quote is, quote, our AI content strategy has started a doom loop that will hurt the performance of our models and the entire web at the same time.

14:10Emanuel:It is highly unusual that an end product threatens the economic foundations of its essential suppliers. But that is the situation we have created for our LLM business with respect to its content supply chain. It's basically like they're saying, we are making something that is siphoning from another thing, these suppliers, these people who write articles and publish articles and books and art and all that. And what we're doing is destroying that. And there's an OpenAI document that also kind of touches on this from their policy director, Jack Clark, where he said that the company was, quote, creating systems that substitute for the labor of the people who define the culture of society.

14:54Emanuel:And then Microsoft says that it could significantly disrupt the employment of the very people who generate the data on which the foundation model was trained. LLMs are a product that destroys its supply chain. OpenAI called itself an existential threat to news publishers and an OpenAI software engineer testified that no matter how prominently we show the links, users won't click. Is there a date for those? There's not a date. I mean, a lot of this stuff was happening around GPT-3. GPT-3 was many, at this point, generations ago. But that's kind of the moment that ChatGPT became a thing and where it had this amazing growth and people started using it.

15:35And then the depositions have taken place over the last year.

15:40Emanuel:So a lot of this was said within the last year, but then a lot of the court case has to do with the initial training of these models years ago. But this nods at the idea of model collapse or the idea that you destroy all the journalists, you destroy all the writers, you destroy all the human labor, than you just have AI training on AI or AI training on really low quality stuff. And that hasn't happened yet, but it sort of is. As in, if you go search for stuff on ChatGPT or on some like Gemini, etc., a lot of high quality websites won't provide information to those models. And so you end up with really shitty content and shitty low-quality websites being cited in these models.

16:35Emanuel:And that's part of the whole generative engine optimization game where it's like, we'll just create our own websites that will then be cited by AI and so on and so forth. And it's like, that is definitely starting to happen. The entire collapse of the system isn't happening, although there's all these papers about how it might and how in more controlled situations it is, that sort of thing. but it's pretty bleak. Yeah. So, I mean, listeners and viewers of this podcast probably kind of already know all of this, right? And we obviously know it and other people in the media industry do, other people who just follow AI.

17:14None of these are new ideas to them. But it is obviously significant that we're hearing this from Microsoft and OpenAI people. Just to round us out, why do you think it is significant that we're now learning that they actually, they had internalized a lot of these criticisms as well. It's only now that we're learning them, even though they tried to not have it come out.

17:37Emanuel:I mean, I think at this point, these companies, there's too much riding on them economically for the entire country. Like the entire US economy is being propped up by them right now, bubble or not. Like that is where all the growth is happening. That's where all the excitement is. That's where all the attention is. And so I don't know what a fair and just outcome of a lawsuit like this would be. But I think that there is... One, it's like a lot of the other copyright lawsuits, even when they found fault from the AI companies, like the authors sued Anthropic, that case was settled. It's like a bunch of authors, like a handful of authors got a bunch of money.

18:26Emanuel:Anthropic got a slap on the wrist, paid a bunch of money and moved on with its life. And a lot of the entities that have been suing these companies, I could be wrong, but a lot of them are like individual authors, individual artists, like people who are not, don't have the ability to take a court case to its conclusion because lawyers are really, really expensive. And it's like, this court docket is like thousands and thousands of documents long. So it's like millions and millions of dollars are being spent fighting this case. And I think that the outcome in a lot of these other ones has been like a slap on the wrist and a settlement at most.

19:10Emanuel:Whereas this is a really big entity. The New York Times is a big company. It's not as big as a tech giant, but it is billions of dollars company. that's fighting very hard in this case. And so I think the idea that copyright law is going to save us here or is going to kill these companies is kind of delusional at this point. But there could be a pretty strong outcome here. There could be... I don't know what it is. I'm not sure if it's just the New York Times gets a lot of money or if there's actual consequences for Microsoft and OpenAI. But I do think it's important that this is adjudicated in the court system and that someone say what they did was wrong.

19:58Emanuel:One of the comments on this piece was like, it's so crazy that we watched kids who use Napster and pirated MP3s had the book thrown at them. And then here you just have these companies admitting to huge, huge, huge theft from other really big companies and nothing is happening. And we've seen that with Disney doing lawsuits and stuff like this. It's like they always end up striking some sort of partnership because these big companies are like, oh, well, maybe AI can help us also. Maybe we should get in on this versus actually trying to hold these companies that stole our shit accountable. And I just hope that...

20:39Emanuel:Hope against hope that there's some nod to like, this was fucked up. This was wrong, what they did. and I guess we'll see. I don't know. It's going to make the world's most boring movie at some point where it's just like Sam Allman being like, yeah, scrape it. Scrape it. Download it. It's coming out. They're making it. It's almost out. The movie. Didn't it get killed or moved? No, no, no. Artificial. It's almost out. Thrilling. Did you know the Social Network sequel is coming out next week? I thought that this was coming out years from now. They really sped through that. They fast-tracked it. Yeah.

21:20Yeah, 100%. Yeah. I didn't know it was that soon, though, but I'm sure I'll watch that. All right. We'll leave that there. When we come back after the break, we're going to talk about, you know, a massive hole in how music is digitally distributed and how Emanuel was able to use that to essentially hijack a band's Spotify page. We'll be right back after this.

21:48Emanuel:This podcast is brought to you by NordLayer Browser. NordLayer Browser is an enterprise browser that secures online work without getting in the way of productivity. It gives IT teams full visibility and control over every web-based AI tool, SaaS application, and download across the organization, while making sure employees can still enjoy a seamless browsing experience. Designed for today's distributed workforce, NordLayer Browser helps companies protect access to their internal resources and prevent accidental data exposure. Go to nordlayer.com slash browser slash 404podcast and use the coupon code 404podcast10 at checkout to get NordLayer Browser at 10 % off today.

22:27Emanuel:That's nordlayerbrowser.com slash browser slash 404podcast and use the coupon code 404podcast10 at checkout. NordLayer.com slash browser slash 404 podcast. I've reached the point in my traveling days where I refuse to travel with a bag full of charging cables. Got phone cables, headphone cables, watch charger, USB-C cables. And somehow, despite packing all of them, honestly, the number of times that I bought a new cable at the airport, I'm very embarrassed to talk about. That's why I've been kind of obsessed with this thing from Ridge. Ridge. This is Ridge's 5-in-1 Travel Power Bank. I have it in the Neon Tiger colorway, which is pretty awesome.

23:11Emanuel:It's the Ridge's 5-in-1 Travel Power Bank. And the genius of it is that it's got all the cables and chargers you need already built in. It's got MagSafe compatible wireless charging, an Apple Watch charger, Lightning, and USB-C all-in-one device. So instead of finding three outlets at the airport and turning my backpack into a cable drawer, I just grabbed this one power bank. It's got 20 watts of power and a 10 ,000 milliamp hour capacity, giving you up to three full phone charges. There's even an integrated kickstand so you can prop up your phone while you're charging it and use it. I've been traveling a lot lately.

23:47Emanuel:And honestly, before this thing, my bag looked like a bird's nest, like just cables everywhere. It was a nightmare. Now I've got this. It solves a huge annoyance for me that I had basically accepted as part of traveling, and now I don't want to go back. And like everything Ridge makes, it's built to last. You get free shipping, a 99-day risk-free trial, and a lifetime warranty. This might genuinely be the last power bank you ever need. One thing to pack, five ways to power. You can find Ridge's power bank at Best Buy, or our listeners can get 10 % off at ridge.com by using code 404media at checkout.

24:24Emanuel:Just head to ridge.com and use code 404media and you're all set. After you purchase, they'll ask you where you heard about them. Please support our show and tell them that 404 Media sent you. So for fun, and it wasn't actually fun, I googled myself last week and found my home address, my phone number, and a relative I'm fairly sure that I've never met, all on directory sites I never signed up for. That's the data broker economy for you. They scrape your info, package you up like a trading card, and sell you off to whoever's paying, which is exactly how identity thieves go shopping. Bad look when your entire job is not getting doxxed.

24:59Emanuel:So that's why we use Incogni. Instead of you spending hundreds of hours filling out opt-out forms across 40 browser tabs, Incogni gets to work by sending legal removal requests to over 420 harmful data broker sites. It's independently verified by Deloitte, resubmits requests every 30 to 60 days as brokers try to quietly relist you. And you can track every single request right from the app. On the unlimited plan, the exposure scanner digs up the weird stuff across the wider web, like old forums, random blogs, or obscure directories. It surfaces all your exposed profile links right in your dashboard, where you simply select which links you want to take it down.

25:38Emanuel:And Incogni's team of human privacy experts will handle the manual corporate takedowns for you. Take back control today. Go to incogni.com slash 404media and use code 404media for 60 % off an annual plan. That's incogni.com slash 404media, promo code 404media. Risk-free with a 30-day money-back guarantee. They can't harm you if they can't find you.

26:05All right, and we are back. Emmanuel wrote this one, but I have a question for Sam as well. The headline is, I hijacked a real artist's Spotify with AI music. It was disturbingly easy. So I think let's just do this chronologically because it's like a little bit complicated. So first, you made an AI track, Emmanuel, like a piece of AI music. Let's play a little bit of that now.

26:55So how did you make that exactly, Emmanuel? I went to Udio.