The Rise of AI Book Ripoffs

0:03Hello, and welcome to the 404 Media Podcast, where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist-founded company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co. I'm your host, Joseph, and with me are 404 Media co-founders, Sam Cole. Hello. Emmanuel Mayberg. Hello. And Jason Kebler. Hello. Well, Jason and Sam, we have an event coming up that both of you are going to be at.

0:46What's the rundown? What's happening? Yeah, we're throwing a party at South by Southwest on Monday, March 10th. We're going to have a kind of like live panel podcast thingamabob about AI Slop featuring me and Sam. and our friend Brian Merchant, who runs Blood in the Machine. And then we're going to have a party after that. So to register, you can go to our website and there will be a post by the time you read this because... We don't have one yet. I know, but the URL for it is really long and difficult. It's in the show notes. Show notes, show notes. That as well. It's l.lu.ma slash hffhixixx.

1:35and yeah click it in the show notes and all the posts which we will now write i love a live air url read that's really good pod people like it speaking of ai slop let's get to our first story this one is by joe the headline for the story is a slop publisher sold a ripoff of my book on amazon uh joe in case you don't know though i i can't believe you don't know because he's done a very good job of promoting his book, but he wrote a very successful book. And you, I think, probably were looking up the book on Amazon and other stores, and what did you find while you were doing that? Yeah. Were you buying your own book again?

2:22Yeah, I actually buy tens of thousands of copies like Trump Jr. style. Didn't he do that to get it to the bestseller list? I believe so. Yeah. I have not done that, to be clear. No, I haven't bought any copy even though people do that to game the system. But I was, as I sometimes do embarrassingly, just checking it on Amazon to see what the ratings are and that sort of thing. It gives some indication of, I don't know, how much people like it or hate it. And as I was doing that, I came across this other listing on Amazon. To be clear, you don't need to buy the book on Amazon. You can do it anywhere.

2:59It's just they have their metrics and their star rating, whatever. but um i'm looking through and i see this listing for a summary of joseph cox's dark wire which is the name of my book and i'm like huh that's pretty strange i wasn't expecting that and then i go through and um immediately i'm interested because of the crappy quality of this listing like there's a really generic orange cover. The description seems pretty generic as well. And just going through the other summary books uploaded by this publisher called Slingshot Books, it just seemed like a very low effort churning out, I would say, of these book summaries.

3:50So I decided to spend $4.99 of our subscribers' money on this piece of shit, basically. So you buy it, you read it with great interest, what do you find? I have read your book, but, I mean, would I be better off just reading this instead? It sounds like possibly. Yeah, you just read the summary. It's fine. It gets the basic gist. I mean, on one hand, it kind of does summarize the book, but literally what it's doing is that it's breaking down every single chapter, and I think there's like 30 or something chapters in the book, and it's just, rather than them being, I don't know, 20, 25 pages for I can remember, with all of this narrative detail, it's like three or four pages of just the normal plot points.

4:43but it well the example i give in the in the article that i wrote was that my opening of the book is about this drug traffic called owen hansen and there's sort of all this detail about him and i spoke to him and you know i've followed his escapades and his case for years and you go to the summary book and it's just two pages of hansen did this hansen did that hansen did this and it's I guess, the rise and fall of him. He does fall, you know, he's arrested by the FBI, but it just misses out all of the rich detail, essentially. So to answer your question, it was just like a really bland overview.

5:23Now, maybe that's the point of a summary book, you know, that we have cliff notes. I think we'll talk about that maybe a little bit later on in this segment as well. Like I'm not against summary books and I've used them when I've been studying as well it's different when you see your work and your intellectual property um basically summarized on like a digital piece of toilet paper basically you know um and then i i took it and i and i showed a screenshot to you all i think in our group chat emmanuel what did you think when i showed you the screenshot because i don't know whether it's ai or not you know like That's one thing we can't really determine.

6:02But I showed you the screenshot, I think. What did you make of that? Yeah, so I've been looking and reading a lot of these over the past couple of weeks for reasons that we'll get into soon. But also, I believe we talked about on the pod a couple of weeks ago, I wrote about AI-generated books, making it into libraries. And a lot of these books are the summary books, so I've read them. And there's just a certain style to them, which is like you described which is very plain and then correct well-written english but with no stylistic flair or feeling for place and narrative and all of that um it just feels uh kind of soulless so i definitely had that style um and immediately seemed ai generated to me and while we don't have a smoking gun here um there's a lot of contextual clues that we can get into later about why we're pretty confident that it's ai generated yeah and i also showed it to jason and sorry jason to put you on the spot because i didn't actually include this in the article but i just remember you responded to it by saying like there's no way this isn't ai generated due to like the sheer number of book summaries that this publisher was churning out what do you mean?

7:23Yeah, I just mean, if you think about it for one second, they're publishing dozens of summary books, and it's some publisher you've never heard of before. There's not a human being reading and analyzing these in a formulaic way in the year 2025. It's just like Occam's Razor. it's so easy to get an AI to summarize. And it's so easy to get Amazon to publish something that there's just like no business model where this makes any sense whatsoever for a human being to be doing this work. And by that, I just mean like, it's the type of AI slop that we've seen over and over and over again. And if you were to enter this market right now as a human being saying like, oh, I'm just gonna read Joseph's book and then I'm going to read some completely unrelated book and I'm going to do this multiple times per day, publishing multiple formulaic summaries that are bad.

8:29I don't know why you wouldn't have an AI do it. Yeah, it's just way more likely to be AI, especially this year. And there's actually one comment left on the article from a subscriber, Spencer Bauman, which I'll just read out now. Quote, there's no way it isn't AI. Something about the way the chapter title is written screams chat GPT to me. Similar things of articles that are individual paragraphs slash bullet points with a bolded headline at the start of each one. The bullet point is a very good clue. And I just want to run through, if you're listening to this and you see a book on Amazon and you're not sure, the contextual clues that we usually use are, it's the cover, right?

9:15you'll either see that it's a clearly AI-generated cover. If it's a book about someone, it's really easy to tell because they don't really look like themselves. But also you can see just artifacts in the images. It's the content of the book, which has the same style that the commenter just mentioned. And then I think probably the best one, as Jason mentioned, is you click on the author or you click on the publisher and you see, A, do they have any online footprint? A lot of these AI publishers do not. They just exist on these storefronts. And then the other one is just the amount of books that they've published.

9:53So usually you'll click on one of these publishers, and they've published 300 books in the last year. And usually it's the same author, and that is just humanly impossible for someone to do. And I think this was all true about your book summary. Yeah. I'll just read a tiny bit of Amazon's statement, because I reached out to Amazon for comments, and they removed that summary of my book, but then also dozens of others. When I went to go write the article, there was only one summary left. And I don't know whether that was just created after the ban hammer or somehow it dodged it. I'm not entirely sure.

10:30But Amazon said, we limit the publication of summary books that are about other titles in our store and the title you brought to our attention is no longer available for purchase. We have content guidelines governing which books can be listed for sale and we have proactive and reactive methods that help us to detect content that violates our guidelines whether ai generated or not um interesting in that the statement is is more on we don't want summary books rather than we don't want ai books which uh i think is interesting because sam you previously wrote about ai generated foraging books um i think specifically on amazon um what happened there because we i mean this story is just about my book and it was funny because it happened to me but like that story you wrote about could have real world consequences yeah i mean it's just a similar thing but with a different topic i read about this also with um knitting and crocheting tutorial books um but the mushroom one was interesting because that's actually something that like if you take the wrong information from you could die from.

11:36It's like they were identifying, these books were identifying mushrooms that were potentially poisonous as safe to eat, like good to cook with. They're very slight, like subtle differences in, you know, like a death cap mushroom versus one that you can like actually eat, put on pizza. So, you know, if you don't know what you're looking for, you just say, oh, that's a wild mushroom. But obviously the consequences are more serious than that. So yeah, it's, you know, it's people preying on beginners, people preying on people who don't want to spend more time, you know, researching something deeply.

12:17They just want the quick answer. It's just, I don't know. It sucks. It's sad to see it happen. It was sad with the crocheters too because that's such a, like a handmade, obviously, craft that real humans have to be involved in. So, you know, It's kind of eroding all these knowledge systems that I guess we've taken for granted, but I never would have imagined that they would be under threat like this five, ten years ago. I think when it's a summary of any other book at this point, you can pretty safely assume that it's AI generated unless it's from Sparknotes. And even then, I bet that Sparknotes is...

13:00I have no information to base this on, but I bet they're utilizing AI in some way, shape, or form at this point. And it's just like, I'm not even sure what the utility of a book summary is at this point. I believe you could probably ask ChatGPT at this point, give me a summary of Joseph's book, and it probably would give you something. I don't really even know. it's a it's a weird type of content that has like suddenly popped up on amazon and all over the internet but i don't think that it's like popular or i just i don't know yeah i mean you're right in that you could just go to chat gpt and get the summary that way if you have you know paid access or free access to chat gpt that's basically what the books are doing anyway right you're just paying some publisher five bucks to basically enter the prompt for you i mean look i don't know how this publisher or other ones got my book, obviously you can just pirate it.

14:00Like, I'm not going to dance around it. That's absolutely possible, obviously. Now, did they pirate it or something like that? I don't know. But you could probably even, or ChatGPT could probably still build a summary just based on the public media coverage they had as well. So, I mean, there are different ways of going about it. And not to put it all on Amazon, as fun as that is, it is also on Apple Books. And I'm looking at it right now and the exact same summary is on there. And it hasn't been taken down. Not that I'm trying to go on a crusade to remove every summary of my book. It's more funny than anything.

14:36But Apple didn't respond, whereas Amazon did. There's no utility to you, Jason, the reader, but the librarians I talk to are pretty confident that it's a numbers game where if they blast this summary across enough platforms, they're going to catch some minuscule number of people who are searching for joe's book and get the summary on accident and make enough money to make it all worthwhile like that's what the scheme is in general from from what i understand it's just like getting an errant click and and getting the the money for sure that makes as as like an ai content slop factory it makes sense but as a like curated human being summarizing these books, I don't think that it makes a lot of sense.

15:26I do know there is that startup that gives you audio summaries of books that advertise on a lot of podcasts. I don't remember the name of it and I never tried it. But that was popular for a while because it was like, you can read these self-help books in seven minutes and learn what the four-hour workweek says. But I've never tried it and I don't know how popular it is. Also, you can just ask Chad GPT for a summary of Joseph's book. I've just learned. Oh yeah? Can you read us a tiny bit? It says it's shit. It says it's great. So I asked for a summary of Dark Wire by Joseph Cox and then it just said like, you know, a three paragraph summary and then I said, can I have a chapter by chapter summary?

16:16And then it said, now it reasons because deep seek reasons so you can like see what it was thinking and it first said am i allowed to do this and then it said uh summaries are permissible so i'll proceed without including direct content so i guess it's not quote quoting anything from you but it says chapter one the encrypted underworld focus introduces the reader to the ecosystem of encrypted phone networks discusses how certain companies market specialized devices to criminal clientele emphasizing privacy features and anonymity and then it like kind of goes on and on and it never says that's wrong that's not what happened that's not what happens in chapter one that's interesting yeah uh it never produces the word anon which is kind of interesting which is the main company in the book which is what the book is about yeah huh yeah I don't know it's chapter 5 called The Covert Takeover do you remember no it's not called that okay well dude I'm I'm on the slot publisher side now at least they like got it right I don't know yeah it's not a very good summary the summary of mine is pretty spot on unfortunately oh yeah go on yeah yeah I mean it's just it's they're pulling from a lot of the sources are online media coverage of the book.

17:37So it's pretty accurate, even without having access to the book, probably. But yeah, it's pretty... It's correct. It's very specific. It's like you could get a lot of summary out of the title, which is How Sex Changes the Internet and the Internet Changes Sex. It could bullshit pretty easily off of that, but it's very specific about... It talks about post-board systems and OnlyFans and... And sources? Are they mentioned? The people you spoke to? The main source is nextbigideaclub.com, which is a blog that I gave an excerpt to. But what it's pulling from is the subheadings from that blog, not necessarily the chapters.

18:16But then again, it's like my book is on Libgen, free to download, that people torrented. So it's like it could access the whole thing if it wanted to probably, but I don't know if it has that capability. I'm sure mine is as well. That's how you know you made it. It reads like fifth grader did a book report. That's Chad GPT, man. Emmanuel, you touched on some of your library stuff, but to close the loop on that, close the loop, the Hoopla? No, that doesn't matter. Briefly remind us what the library story was and then what has happened since. Because we did talk about it on the podcast, but there's been some developments.

18:57Yeah, so Hoopla is a service that provides public libraries with access to e-books. So if you want to get an e-book from your public library, you're probably going through Hoopla or this other service called Overdrive. To illustrate how common the AI-generated summary book issue is, Hoopla responded to the story eventually. They emailed a bunch of librarians and addressed a bunch of the issues that I mentioned in the story. They are kind of working on new policies on how they curate and filter out AI-generated books. They removed a bunch of AI-generated books. But I thought it was very telling that Hoopla said that it is just removing all book summaries from its platform as a rule.

19:49Like no more book summaries, with the exception of HMH Books, which is the publisher of Cliff Notes. and maybe SparkNotes as well, but they haven't mentioned that. But they did call out CliffNotes as being the one publisher that's allowed to have summaries. But all of them are gone. So for example, I mentioned this one publisher called IRB Media that had hundreds of summaries up on Hoopla. Those are all gone. So, and I mentioned CliffNotes briefly in my one as well, because I think it's the one that everybody knows. But is the, to speculate slightly, is the reason that they're kind of exempting cliff notes it's just because well they're legit you know what i mean and like there's more there's more to it they're study guides rather than subways exactly i was gonna say i have i've never used spark notes so i don't really know what those look like but i have used cliff notes in my life and what i remember is like i don't know if you if you get the cliff notes on moby dick it's not just like here's what happens in every chapter it's like oh like it is commonly interpreted that the whale signifies yada yada yada and it gives you like historical context, you know, it's just like it tells you something.

20:58There's some analysis being done and presented to you about the book. So you know what to tell your teacher when you pretended to read it, you know. Cliff notes the original cheating with chat GPT. I mean, I don't know. I never use them. That makes sense. All right. Let's leave that there. When we come back after the break, we're going to be talking about another AI startup that is dehumanizing factory workers. This time is a lot more explicit, I would say. We'll be right back after this.

21:43Hackers and cybercriminals have always held this kind of special fascination. Obviously, I can't tell you too much about what I do. It's a game. who's the best hacker? And I was like, well, this is child's play. I'm Dina Temple-Ruston, and on the Click Here podcast, you'll meet them and the people trying to stop them. We're not afraid of the attack. We're afraid of the creativity and the intelligence of the human being behind it. Click Here, stories about the people making and breaking our digital world. AI machines. Satellite. Engine ignition. Click Here. And lift up. Click Here, every Tuesday and Friday, wherever you get your podcasts.

22:33All right, and we are back. This is one that Sam wrote. Why Combinator supports AI startup dehumanizing factory workers. We'll play the video in a second for our listeners. But Sam, maybe just give us an overview of what is this video exactly and where did it come from? Yeah, so this is something that was going around on Twitter last night and this morning that I caught there. And you know something is pretty bad when it's getting panned on Twitter, even though it's a Y Combinator startup about AI. Twitter at this point is very much like the tech bro boosterism world. and even those guys were upset about this.

23:20So, you know, it's bad. So yeah, it's a startup called Optify.ai spelled super dumb as these startups are. O-P-T-I-F-Y-E. Optify. By the way, that is the worst F-I I've ever seen. Spotify is one thing. This is crazy. Just adding letters where we don't need them. The F-Y-E, I could not understand it. My eyes were going all cross-eyed when I was trying to read the title. It's bad, and I keep calling it the wrong name in my head and forgetting what the actual name is. I'm like, Sync-dify? Like, I don't know. It's Productify? I don't know. But, okay, so it's a startup that, quote-unquote, it's for AI performance monitoring system for factory workers is their proposal here.

24:09In a very small nutshell, they are using machine vision to track workers in a factory and see what their productivity levels are, how many per items they're making, what their output is, their efficiency quotas, things like that. It's like this whole dashboard that they're advertising bosses can see. And then it shows you exactly who on the line is going too slow or isn't being efficient. and that's the idea and the video in it the founders of this startup are kind of doing like a skit and one of them is a boss, one of them is the worker and we'll just play the clip we'll play that now and then listeners will have some understanding of what we're talking about and then we'll come back oh it's workspace 17 workspace 17 is the bottleneck the worst performing workspace here.

25:10Hey, number 17, what's going on, man? You're in dread. I've been working all day. You've been working all day? You haven't hit your hourly output even once today. And you're at 11.4 % efficiency. This is really bad. It's just been a rough day. Rough day? More like a rough month. Yeah, so basically, as you can tell, this is um productizing bosses being assholes to workers is the real kind of innovation here as emmanuel put it when he was reading and editing the story it's not that this is like a new technology that surveillance on the line is something that has never happened before we see it happen with amazon we see it happen um all over the place with different warehouses and also just with computer jobs and remote work.

26:03But the shocking thing about this isn't the technology. It's the blatant normalization of turning around, calling one of your workers by a number, and mocking them about having a rough day, a rough month, and being an ass to your workers. and being this cruel overlord while they're working on your production line. Yeah. And we'll bring the Y Combinator stuff in in a second, but just while we're talking about the video itself, is this video a bit? Or what do we make of that? I don't think it's a bit. I think if it were a bit, it would be more overt. I think good satire often toes that line, but it's not if it's satire it's not good satire because it just is not over enough for people to say this is this is a send-up of this type of ai and i don't think it's a bit because y combinator is celebrating it congratulating them online um their website says they're backed by y combinator y combinator is like it's a venture capital firm um that you know creates you know makes companies is huge.

27:24Some of the biggest companies in the world have come out of Y Combinator. So I don't think it's fake. I think it's real. I think these guys are just really out of touch for various reasons that we don't have to get into. But it's just... It's a bad look for everyone involved, basically. Well, I think also, it's not just a venture capital firm. It's a startup accelerator that picks ideas that it thinks are not represented in the market yet. And so they end up being like the market doesn't have this yet so we're going to help you incubate this idea and like turn it into a reality and so i don't know it would be very odd for y combinator to to like it's just it's not satire like it's it's just not it's it's like a promotional video for their idea um well speaking about y combinators support slash backing um they deleted some posts, right?

28:24By they, I mean Y Combinator. What happened there exactly? What did they delete? So Y Combinator had posted on Twitter, and we know this because of screen shots, and also you can see it in like, just you can find the URL through deleted posts, people replying to Y Combinator. But they were saying, they were introducing Optify.ai as building AI performance monitoring for factory workers and then they link to um a link about their launch and then they say congrats on the launch and then they tag the founders on twitter and on linkedin they also made a post which i found the linkedin post through just like googling y combinator and optify and it comes up in the cache um which you can still see what the what the linkedin post part like the first sentence of it was or whatever and it was a similar thing it was like you know congratulations on launching uh to optify and they're gone now they're deleted now any idea when they deleted because the because the the promo video which was in very very poor taste that was going viral yesterday did they delete it like after that or do we not really know they deleted it at some point i'm pretty sure it was after that because that's what is all over twitter like the replies to that tweet that's now deleted or people being like are you fucking kidding me like this is right disgusting um so i think they got a lot of backlash and they were like oh like instead of you know like hiding replies or closing comments or whatever they were just like just nuke the post and we can pretend we did not see it or you can pretend that we never supported this company which is dehumizing people by calling them a number yeah i mean we'll see i mean i like i reached out to uh y combinator obviously to ask you know what's what's going on with this why did you delete the posts are you backing this company still um what's going on there and i haven't heard back yet but uh we'll see it's also all over the optify's site that they're backed you know quote unquote backed by my combinator which is obviously a big like stamp of approval for the industry it's like oh that's a big deal like jason said it's like that's you know that's kind of a mark of like oh you're serious uh serious people think you're serious so yeah you know this is worth looking at it legitimizes startups in the eyes of other startup founders and investors and all that sort of thing um emmanuel while i think you edited this correct me if i'm wrong but while we were talking about it in our group chat i think you did bring up that there was a video from a few years ago or something where somebody did do a bit about this right like again i'm not saying but we know this one's real it's bad by y combinator it's just funny that this is you can't really tell exactly and then there was a bit earlier what was the deal with the the earlier one was it somebody just mocking this sort of thing it was a i don't think it was i think it was it was either a joke or it was a one of our favorite um you know local dipshit you know like local artists makes a statement videos but it was a video of a coffee shop and it was using machine vision that looks almost exactly uh like this actual startup to kind of track the productivity of the baristas in there.

31:40And that went extremely viral because it freaked people out. It seemed so dystopian. And it's one of those videos that kept going viral, even though it turned out that it wasn't a real technology from a real company. It's not something that people are actually using. So it's funny that... I mean, it's funny slash horrible that this thing that was not real and everybody thought was horrific is now actually a real company. And I don't know, to me sort of seems inevitable. As Sam said, this type of thing exists in many workplaces, whether you're a white-collar and people are tracking what you're doing on your computer and your mouse movements and your keystrokes.

32:19Or you're at an Amazon warehouse and people track how many packages you're handling in an hour and how long your trips to the bathrooms are, etc. It's just the idea that you can use machine vision to track whether you're tracking it well or not. the idea that you can track any worker's productivity with just putting a camera in there. I don't know if this company will survive. I don't know if this will become a real thing. But I feel like the idea is just so attractive to bosses that there's no way that we're not going to see something like this eventually. Yeah, I mean, it's already here, like both of you have already said.

32:57I mean, it kind of has its origins in this idea called scientific management, which Edward Angueso used to write about a lot for us at Motherboard. But it's based on management practices from the 1880s and 1890s that has largely become algorithmic management on Uber, DoorDash, etc. And then, yeah, it's just a difference of degree, not of kind, really. but I think that the video here was so dehumanizing and so fucked up that it went viral but as you both said like a lot of people work under conditions that are like this to some degree where they're being managed by a computer software or their output is being um like their worth as a worker is entirely based on numbers and whether a computer thinks they're doing a good job or not yeah i i guess just uh last question sam to wrap it up like what does this show us is it just this one company sort of said what all the other startups are thinking and probably want to say and they just like fucking went for it and broke through or is it you know a mask off moment for labor and ai i know it's it seems like to me that a lot of companies would want to do this but they would not all be as stupid to make this video right um yeah like again these are these are i don't want to like shit on these guys too much but like they're they're computer science students at duke university they're in an extreme bubble as far as interacting with normal people, I'm sure.

34:44They have little life experience to speak of. Their families come from, they own, they've said, they say in their bios, their families own manufacturing plants. Oh. Which, yeah, I mean, yeah, it's like, and that's a whole complicated conversation that we're definitely not going to get into here. But it's something that we see over and over again in the AI startup world, this sort of thing is very much applauded, like efficiency numbers. It's like timing your bathroom breaks. And even if you're the boss, people impose that on themselves. Like they say, I slept three hours. I slept on the floor.

35:28I slept at the office. I worked my efficiency rate, my heart rate, whatever. It's quantifying of your work and your worth actually has really damaging effects on workers, whether that's yourself or your employees and your factory workers. Honestly, it's like this is not going to go well no matter what. It doesn't go in a good direction. And a few people have pointed out just like on social media, and I think I had the same thought. it's like it would be way more efficient just to talk to that worker and be like instead of being like hey you know number 17 what's your deal what's your problem and then he says i'm working i'm working all day and you just say well it looks like you've had a bad a rough month or whatever it's like just talk to people and say hey what's what are the inefficiencies on your section what are the inefficiencies that you're seeing um without threatening them with being fired probably is a really efficient way to cost effective way to get to the bottom of what's going on at your own company.

36:35But instead, people want to distance themselves from their own workforce, from the actual labor being performed so that they don't have to think about it. And they can just look at a graph instead of a person making sweatpants below them, how they view it. So, yeah, I mean, it's just... And yeah, like you said, it's that they're saying it out loud. It's just very blatant in this case because they are so young and they come from that society where it's just like they've they've seen it play out their whole lives so they don't have that kind of filter to know oh it's not cool to just put this out there you have to put it behind a few layers of like politeness or whatever uh pretend it's productivity focused when really you're just like how hard can i abuse people before they break sort of mindset i don't know it's a huge It's a huge complicated thing stemming out of a very goofy startup, frankly.