AI Slop Summer

0:03Hello, and welcome to the 404 Media Podcast, where we bring you unparalleled access to inner worlds, both online and IRL. 404 Media is a journalist-founded company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co. I'm your host, Joseph, and with me are all of the 404 Media co-founders, the first being Sam Cole, Emmanuel Mayberg, and Jason Kebler. hello hello uh i'm traveling this week so i'm on a different mic so if i sound different that is why oh thank you for bearing with us if it sounds weird yeah please uh bear with us so let's jump straight into it jason you've just published a couple of stories in very very quick succession based on something that went viral and we'll all get into that and you've got a ton more tons more information about it and actually sort of symbolizes a lot that's happening in the media industry and with AI as well but the first story the headline was Chicago Sun-Times prints AI generated summer reading list with books that don't exist so this thing starts blowing up on social media definitely Tuesday potentially Monday as well correct me if I'm wrong what was this thing that was blowing up on the face of it yeah so the chicago sun times is a newspaper in chicago that's been around for a long time there's a chicago tribune and then the chicago sometimes like the second newspaper in chicago and it still exists but basically they ran uh this special insert in their Sunday paper this past Sunday called, uh, heat index, um, your guide to the best summer.

2:08And this was a 64 page special section of the newspaper. That was like, what you should be doing this summer. We can talk more about like what was in it. Yeah. It was just like a year. Yeah. Like have fun in the sun. like, here's books you can read over the summer, here's things that you can do, so on and so forth. And in that, they had an article called Summer Reading List for 2025. And in that reading list, they suggested a bunch of books that do not exist. So they were real authors like Isabella Allende, Andy Weir, who wrote The Martian. but they suggested reading The Last Algorithm by Andy Weir.

2:53And then the text of the article says, quote, or like the blurb in the article says, quote, this time the story follows a programmer who discovers that an AI system has developed consciousness and has been secretly influencing global events for years. This book doesn't exist. Andy Weir has not written this book. Isabella Allende did not write a book called Tidewater, which is what they said to go read. So, you know, this is noticed by actually a book talker on threads of all places. And then it got picked up on Blue Sky and it went like mega, mega viral because it was an example of like a printed article that was clearly AI generated in some way.

3:34Yeah. And I guess that's what makes it a little bit different because obviously long time listeners of the show and readers of the site will know there's tons of AI slop everywhere all the time. AI-generated articles or press releases or whatever. But this is different because it's, as you say, the Chicago sometimes is basically like an institution or at least an incredibly long-running newspaper and media company, right? So it's a little bit different almost now to have this physically printed AI slop. Or I suppose I should back up a little bit. And was it just clear from the get-go that this was probably AI-generated just because it was getting stuff wrong?

4:14like was that the implication uh i mean that's what people were saying like that was how it was shared on blue sky and so i went and i bought a digital copy of the newspaper from the chicago sun times and i scrolled through it and i downloaded this heat index section which again was 64 pages long and i started scrolling through it and uh my origin story in journalism is that i i first worked as an intern at washingtonian magazine which is a magazine in washington dc and my job was to fact check articles like this or like sections like this i mean it was all sorts of things but basically like every local newspaper or magazine does this thing where the like summer is coming up here are outdoor movies you can see here's new restaurants you can check out here's concerts you can do and they source this by like having a bunch of interns go to the websites of every music venue in a city and like compile this like that's how it used to be that's how i did it and very specifically it will be like go see this concert at this chicago music venue like that that's what this section probably should have been and something that i noticed right away was that it was super generic like they didn't mention chicago even one time in the entire section uh the entire section starts off with like a hundred things to do on your summer bucket list.

5:43And it's like, go snorkeling in the crystal clear waters near your home. It's like, yeah, Lake Michigan is not crystal clear. I don't know if you've ever checked it out. Um, and the Chicago river, uh, I don't know. I mean, it is like a fun place to be in the summer, but it's not crystal clear snorkeling waters uh and so i was just like this is not like at the very least is incredibly incredibly generic uh and then i i saw that you know none of these books existed and there was actually no byline on the book list but was that common when you did this as well like when you did this work was with the intern's name be on there or was it also no byline well it's funny uh often it was not on there because it would just be like a calendar listing so it was like it wouldn't say like jason kevler compiled this list of concerts uh usually our name would be like in the front of the magazine saying that like we worked at the magazine and then to get a byline you had to actually like write something so you would have to write some blurbs and sometimes you would get your like initials in there and sometimes you would get uh your actual name and it was actually like a big to-do, whether you got a byline or not.

7:04But I noticed that like every single bylined article in the entire section said Marco Buscaglia. Like it was all done by one person. And so I emailed that person and I was just like, what's going on here? This is going viral. Did you use AI. And he pretty much immediately responded to me and was like, I did use AI and I'm very sorry. And I'm like, I got caught. I'm embarrassed. He said, and then he called me or I called him rather. He gave me his phone number and I called him. And he said, quote, I do use AI for background at times, but always check out the material first. This time I did not.

7:45And I can't believe I missed it because it's so obvious, no excuses on me 100%. And I'm completely embarrassed. Uh, and then I talked to him on the phone and he sort of reiterated the same stuff. He was just like, honestly, like visibly shaken. Um, I could tell he was pretty upset, uh, and just saying like, he, he's, he knows that this is a big deal that he's really embarrassed and, you know, knows that there's probably going to be consequences for him. Yeah, which sucks because it sounds like a genuine mistake, right? So you get that information, you identify the author, you speak to them for a little bit, you publish this story.

8:31You then get some more information because, of course, well, there's still the remaining question of, well, who actually made this as in who produced this? because, as you say, one part of it being really, really generic is almost an indicator of it potentially being AI-generated, but there's actually more of a business reason because it turns out this inlet wasn't just in the Chicago Sun-Times, right? I think it was in a Philly paper or something as well. So then you published a second story called Viral AI-Generated Summer guide printed by Chicago Sun-Times was made by magazine giant Hearst.

9:10How did that come about? Yeah, so I talked to people at the Chicago Sun-Times. I talked to executives there, and they basically explained how this happened. And they bought this entire package from Hearst, which is a huge magazine company. William Randolph Hearst is one of the first media executives in the United States ever, like very, very famous. And they own a ton of magazines, but they also own this company or this subsidiary called King Features, which does syndication. And they syndicate things like Car Talk and Hints from Heloise, which is like how to clean your kitchen. It's like an advice column.

9:54They also do some horoscopes. They do some crossword puzzles. And then they do a lot of comics. They also weirdly do a column by Dr. Oz And so like newspapers around the country will buy this content and put it into their newspapers because it's cheaper to do that than it is to like do real journalism. And this is like a, this is not a new thing. Like this has been going on forever. Like newspapers will not have their, I mean, some of them have their own cartoonists, for example. but very often like the journalism in a newspaper will be produced by the journalist at the newspaper, but then the comics will be nationally syndicated.

10:37And so like all of the, you know, they'll buy Garfield from this company and put it in there. They'll buy a crossword puzzle and put it in there. And so as part of this business, King Feature sells these inserts, which are just separate, like basically mini newspapers that go into the newspaper. and the Chicago Sun-Times was like, we actually don't even read those. We just bought it, put it, like sent it to press and it got in there. And now we are upset because clearly we should have been reading it, but we did not. And now this is very embarrassing for us. That was the saddest part of the story, in my opinion.

11:18Like obviously the AI component of it is something we report on all the time and it's bad, but I'm aware of it. the fact that somebody at the paper was like, we just straight up said, we don't read what goes in our own paper, and maybe we should, is so messed up and not what I expected. I also started out writing in printed newspapers. I hate to sound this old, but that's just where the entry-level jobs were back in the day when I was trying to get into this industry. and I wrote for these type of inserts. A thing that I would do is, there was a weekend edition or a holiday edition of the paper and they wanted a gift guide type thing.

12:04And I wrote about video games. So I was like, here's 50 things you can buy for your kid. And I definitely got edits. Even though it was an insert and it was sponsored and it was definitely looked down upon, which is why I was able to get that type of work. I got edited by a real newspaper editor before they printed it because it was in their newspaper. And I thought that was the standard. And the fact that this isn't the standard at the Chicago Sun-Times is frankly shocking to me. And I just want to add, yes, it's an old newspaper. I don't know the ownership history. I'm sure it changed hands multiple times.

12:42But it's currently owned by Chicago Public Media, which also manages WBEZ which is their public radio station which is a major it's like one of the biggest public radio stations in the country that's where this American life comes from it's like this very well respected organization as well and they have some news sites that I think are well respected locally so it's not like some fly-by-night operation that's flipping AI-generated websites for money. It's a real news organization that is doing this. And just a really sad state of affairs for printed media. And I think possibly

13:31where

13:35hollowed out new media companies might be headed because newspapers have been gutted and kind of stumbling along for decades at this point, which I think is maybe how you get such low standards. Well, yeah, I think that's really important. And we're not going to do a history lesson about the Chicago Sun-Times here because I would get something wrong.

14:03But it was failing for a long time. The paper was in a really precarious situation for a long time and it changed hands multiple times in the 2010s when a lot of local newspapers died and like the fact that um chicago public media bought it was seen as this like really big deal because they bought it and they're like we're gonna focus on local journalism and a non-profit model and um you know it they sort of like took it back from this like investment firm that owned it. And so there had been a lot of trust lost already over the years because the newspaper had been like super hollowed out. And over the last few years, like under this public radio nonprofit situation, and also as a unionized newsroom, like they focused a lot on reporting on Chicago news again.

15:08And so then for something like this to happen, it really like undermines a lot of that, like really hard earned trust. Like you can lose that very, very rapidly. And I feel really bad for the journalists at this newspaper because they had nothing to do with this, like at all. And the union gave us a statement saying, We didn't even know that this was happening in any way, shape, or form because why would they? They're busy writing articles and doing reporting and publishing things. And then a business person somewhere has run a calculation that if they buy this package from this third-party company and then load it up with ads, they can make more money than it costs to actually buy the content within there.

15:55and clearly like the newsroom just doesn't have the resources to review this sort of thing which is not me saying like this should have happened but it's like a really it's just a really sad situation in my opinion yeah and i mean emmanuel touched on it with sort of the shock that this stuff isn't even edited but jason just to go back a little bit when you have this conversation with chicago sometimes um executives what was the vibe uh in that conversation because you've spoken to the author now you're speaking to the actual paper like what was the vibe of the second conversation i mean they were kind of like a hard day hard day for us like we're going through this I do think that, I don't know how to say this, and it's in the article, but he was basically like, we are creating guidelines for how our journalists use AI in the future.

17:01And that they don't have a public version of it yet because they're still working on what those will be. but it's very clear that the rule at chicago sometimes is not going to be don't use ai they're like we already use ai for some data processing and things like this and i mean we talked about a million times on this podcast but it's just like one mistake like this where you invent quotes you invent a person like ai invent some numbers or research that doesn't exist and you publish it, it's like you lose your credibility immediately. And so, I mean, I'm curious what y 'all think, but like, as you mentioned at the top of this show, Joseph, we have talked about AI slop endlessly on this show and at 404 Media.

17:54And some of them get a lot of attention, but like this story went extremely viral. Like everyone is talking about this story. And I'm curious why you think this captured so much attention so quickly. Well, Sam, I feel like it's funny because whenever Sam or any of you say, whoa, this thing is really big on my timeline. I go to my blue sky. I'm like, I'm not seeing this at all, but I think I follow like a hundred people and I don't really scroll. But Sam, I feel like you have a better grasp on what is going viral on Blue Sky. Any idea why this particularly popped off over there? Yeah, I mean, I think your Blue Sky in particular is like InfoSec, OpSec people, right?

18:42Mostly. Mine is like a lot of like AI ethics research people and also just like normal people who are interested in the stuff that we're interested in. blue sky really really hates ai like as a as a group i think i can like generalize and say that like that platform specifically when something chaotic or catastrophic or even just like embarrassing happens in the world of ai and especially slop um people point at it and say this is what we're talking about this is what we've been warning people about this is what we're mad about when people use AI in general because then you start to see the cracks show in the form of print media.

19:25And I think maybe it being print hit different for people. I think people still see print journalism as like this kind of sacred thing, which is funny because it's so desperately underfunded and disrespected. But if something happens in the digital media world, I think it's different to people than it getting sent out to your house and hitting the printers and things like that. Did we all start at newspapers? I also started at a print newspaper and watched the printers put my stories out there. We're all old as hell, I think is what we've learned i don't think i've ever worked for a newspaper like washington is like a glossy magazine okay which is like notably not a newspaper just because they were like oh we only do this once a month like take your time call like i remember i used to have to call restaurants to ask them what their phone number was so i would call the restaurant and then they'd be like okay uh is this the right restaurant okay is your refrigerator running i used to read the Washingtonian.

20:37I was a fan of it. But yeah, I mean, it's... Yeah, I think people still have this idea, this lofty idea of print as infallible or something. And in a way, it's like once it's out there, once it's printed, you can't just make an edit, you can make a correction as part of another issue, but you can't just go in and fix the errors and have it reflect, the record reflect that. It's like, once it's out there, it's out there. So I think that's why people were... they saw this a little bit differently. Also, I think there's, you know, there's obviously this snowball effect of things going viral. When someone's mad about something, everyone loves to be mad about something.

21:12So that happens a lot on social media in general and probably happened here. But it ended up being like a really interesting path to follow. So I'm glad Jason did the, the like work on tracking, tracking the guy down and also tracking down the executives responsible for this. Yeah, I think, the last thing I want to say is like, we wouldn't do this um as in like i i'm not i don't want to defend sort of like anyone involved but i think there's like a really big pile on at the moment of the fact that this occurred and i think that the issue here like i don't want to defend anyone who who did this But the issue here is like a really, really bad, horrible, horrible systemic issue where the economics of local newspapers are not making any sense.

22:13And a lot of the people who are running them, not speaking about any specific newspaper, but broadly, the people who are running them like don't really know how to figure it out. And so they are like throwing shit at the wall. And that includes like, oh, we will just like syndicate this random section from this random company and put it in our newspaper and not even read it because we had to lay off all our copy editors because there's only like two people who work here now or whatever. And that's happening like all over the country. And then also, I don't want to defend the person who did this.

22:52Again, I don't use AI in this way, and I don't think that people should use AI in this way. And I think it's bound to keep blowing up in people's faces when they do this. But a lot of companies are forcing their employees to either hit productivity levels that are impossible. They're being told that they must use AI or they will be replaced. I don't know what happened here. Like as in, I don't think that anyone told this person to use AI. But at the same time, like putting together an entire 64-page summer guide, there's like 30 articles. And he wrote, or like his name is on every single one of them.

23:40He was assigned all of those. Yeah. Yeah. And it's just like, I mean, I doubt he was paid very well for it just because of the nature of what it is. And I just like, I think that we're in like a super, super, super bleak situation. And so like, if it wasn't this person, it would have been someone else who did it for some other newspaper at some point. And it's like, this is a fight that people are wanting to have right now. And I think for a very good reason, but like, you could easily replace these characters with any other person. Like, this is going to happen again. We've already seen it happen a few times to online publications.

24:20And it's a symptom of the disastrous economic situation that journalism is in, that newspapers in particular are in, that local news in particular are in. And then also the fact that in order to quote-unquote fix this problem, they're like, oh, we'll just use AI. It will make us a lot more effective and efficient and blah, blah, blah. and that's happening in every industry right now. And it's like leading to disastrous, like public trust issues, disastrous economic issues. Like it's just like not a good situation. Yeah, totally. I'm sure it will happen again, as you say. All right, we'll leave that there.

25:04And when we come back, we'll talk about more about trust in, I guess, public institutions, but more education. is all about ChatGPT and AI in schools and a bunch of documents that Jason got as well. We'll be right back after this.

25:35One of the smartest apps you can have on your phone in 2025 is Upside. Upside gives you the power to earn more on the stuff you buy all the time. Whether you're filling up your tank, going out to eat, or shopping for groceries, the Upside app lets you earn cash back on every purchase. Upside users are making as much as$280 a year. Whenever I need to fill up my gas tank, I use Upside. Whether I'm near my house or out on the road, there's always a nearby gas station on Upside where I can seamlessly earn cash back. Use Upside to get cash back on the things you're already buying. To get started, download the free Upside app.

26:14Use my promo code 404 and get an extra 25 cents back for every gallon on your first tank of gas. On Upside, there's more than 100 ,000 gas stations, grocery stores, and restaurants, where you'll earn an average of 8 % back on restaurant and grocery purchases. The best part? These stack on top of whatever credit or debit card rewards you might already be using. Start getting hundreds of dollars back per year. Download the free Upside app and use promo code 404 to get an extra 25 cents back for every gallon on your first tank of gas. That's an extra 25 cents back for every gallon on your first tank of gas using promo code 404.

26:55This show is sponsored by BetterHelp. Thankfully, talking about our mental health is no longer taboo. but there's still progress to be made. 26 % of Americans who participated in a recent survey say they have avoided seeking mental health support due to fear of judgment. When people hesitate to get help, it's not just bad for one person. It can impact their social circles and communities. This Mental Health Awareness Month lets encourage everyone to take care of their well-being and break the stigma. The world is better when people are healthy and happy. It's worth continuing to normalize therapy because it can help us deal with a really stressful and anxiety inducing world.

27:32It's not just for people who have experienced major trauma. Therapy can help everyone. BetterHelp has over 10 years of experience matching people with the right therapist from their network of more than 30 ,000 licensed therapists. And their flexible online platform makes it easier to fit therapy in your busy schedule. We're all better with help. Visit betterhelp.com slash 404 media to get 10 % off your first month. That's better help H-E-L-P dot com slash 404 Media.

28:27That gave birth to 404 Media. Since then, we've stopped the spread of AI books in public libraries, triggered class action lawsuits against AI companies, got Congress to pressure big tech in various ways, and we've even shut down surveillance companies. This real-world impact is only possible because of our paying subscribers. As a journalist-owned business, they are the engine that powers our journalism and where the vast, vast majority of our revenue comes from. So please consider signing up today for$10 a month or$100 a year at 404media.co slash membership and get bonus content every week and access to all of our articles.

29:07Thank you and enjoy the rest of the podcast.

29:22all right and we are back with the uh second set of stories here or the singular story the headline is american schools were deeply unprepared for chat gpt public records show So Jason, you wrote this one as well. It's not really in response because you did this in parallel that you'll get into, but there was a really deep New York magazine article recently that I think all of us read or at least saw people talking about. Everyone is cheating their way through college. And some of the anecdotes in there are just insane, like people going to college and university or whatever and basically becoming completely reliant on chat GPT and sort of just breezing their way through education and not really learning anything.

30:16I'm sort of summarizing, obviously, at a super high level, but that's basically what the takeaway was. In parallel, or I guess before, Jason, you had done a bunch of public records requests, like FOIAs, but to state and local agencies what were the requests for and who did you file them with yeah i actually filed these back in uh december 2022 i'm so what yeah they're they're really really old i thought you're gonna say 2024 jesus okay no i filed them a really long time ago and the reason that i filed them was ChatGPT came out in November 2022. And soon after that, there was a scandal, I believe in Florida, where it was like, students cheat on essay question by using ChatGPT.

31:08It became national news. This became national news that a couple of students used ChatGPT to write an essay. And so at the time, I was like, oh, what are school districts? How are they thinking about this? How are they teaching their teachers to think about this? So I filed 50, I felt like 65 public records requests. They were all identical. I filed with every single state. And then I filed with New York City, Los Angeles, Unified School District, like a few in Maryland, because that's where I grew up. So I was curious what they were up to. And just some like other school, like local school districts to see if they had anything mentioning chat GPT, which is really funny because just the way that public records work, as we talked before, is like it took in some cases years to get answers back.