Is Criticizing AI Ableist?

0:00CodeWord doesn't want to take 30 seconds away from the podcast you love. We're fans of 404 Media too. Hell, we're even throwing a party together. So here's 20 seconds back. Love your favorite tech marketing agency, Codeword. Find us at codeword.agency.

0:23Hello. That was my Joseph Cox impression. Hello. Keep it in. And welcome to the 404 Media podcast. um emmanuel maverg and i are filling in for our colleagues joseph cox who is still away from keyboard and jason kepler who is stricken very suddenly with covid may he rest in peace um just kidding jason's fine but um he is not here today it's just emmanuel and i um this is for for many podcasts as i said we're going to go through some of our stories that we have looked into in the last week week and a half um yeah Emmanuel hey how's it going uh I'm great I want to say I want to be honest we're down bad we're down bad we've been down bad for a couple of weeks now um we had travel we had vacation and you were sick I was sick I had COVID and everybody laughed at me ha ha ha Emmanuel finally got COVID after four years but then Jason got COVID as well.

1:28And I'm sure you'll all get COVID soon, you bastards. But yeah, thank you for bearing with us. We'll be back in full formation hopefully next week. And until then, you're stuck with me and Sam. And we're going to start by talking about Sam's story, which has the headline, NaNoWriMo says condemning AI is classist and ableist. um fam let's start with what is NaNoWriMo and do you think of pronouncing that correctly I in my head I've never said this word out loud as we do many of the words on this podcast but I say it in my head as NaNoWriMo but maybe that's wrong um who cares but yeah so NaNoWriMo is I learned just through writing the story that it's been around since the 90s I thought it was a more recent thing than that, but it started as just like a little writing challenge started in San Francisco as like a local writing group challenge to write a 50 ,000 word manuscript, which is like a book book length in one month.

2:41And usually that month, or I guess always that month has been November. And then in 2005, NaNoWriMo became its own nonprofit organization. So it formed into something bigger than just a casual challenge. And nowadays it has a writing board. It takes donations, runs fundraising campaigns. So it's this kind of big, this beast of a thing now. I would say it's like it's basically the idea is a lot of people have ideas for novels but then it's like when am I going to start this thing how am I going to like get through the hump of writing the first page of like a 50 ,000 word novel and this is kind of like a trick to get people to get started like that's what I've heard from a lot of people who did it is like maybe you don't finish maybe you don't do it the whole month maybe you don't hit the goal but it's like if you participate anticipate by the end of it you have like a good start on a novel and an excuse to start it um this is a bit of a tangent but have you ever considered doing this have you ever thought about doing this i feel like for a while like when i was probably in my teens and 20s i every year i would be like oh i should try that you know because it's like it's the peer influence it's like you have like accountability basically to write it accountability buddies accountability is um so every year i was like oh I should try to do that and then never ever did um and then I did write a book and it took way longer than one month how long is your book how many words is your book uh I filed it at 100 ,000 and I think they made me cut it to like 70 I see yeah I can't imagine doing it doing 50k in a month I've considered doing it I've thought about doing it but also I've been basically paid to write my whole adult life.

4:41And I'm like, I don't know, man, I need to get paid to put in this kind of work. I was thinking like, maybe we do this every month anyway. I bet we get close. Oh, yeah. Yeah. But we get paid, right? It's like, that's the thing. It's my job. It's like, I'm not going to do it for fun. Okay. So what happened this weekend? What did this organization announce that made you write this blog? Yeah. So I guess it was Monday. I logged on. It was the long weekend. It was Labor Day weekend. So I looked at Twitter for the first time in a couple of days and people were really pissed off about this new post that NaNoWriMo published on their blog that said the title was, what is NaNoWriMo's position on artificial intelligence?

5:27Which, good start. It's going to be good no matter what um and the organizers who wrote the you know the nanorima organizers who wrote the blog said that uh nanorima as an organization does not explicitly support any specific approach to writing nor does it explicitly condemn any approach including the use of ai which pretty tame start um no big deal there but then it goes on to say that in our, this is a quote, in our belief, the categorical condemnation of artificial intelligence has classist and ableist undertones. And that questions around the use of AI tied to questions around privilege. Um, you know, I could, I could kind of read just from this, but it's also in the blog.

6:22Uh, it goes on to say, not all brains have the same abilities and not all writers function at the same level of education or proficiency in the language in which they are writing. Some brains and ability levels require outside help or accommodations to achieve certain goals. And what they're talking about here is basically using like ChatGPT or something similar. They don't name ChatGPT or any other LLM or platform or anything, but they're talking about using AI to write your book instead of writing it yourself and kind of using this like disability advocacy speech to justify that position. And obviously people really hated that because among like indie publishers and independent creators, AI is seen as this, it's like the thing that's destroying their livelihoods.

7:18Yeah, so let's, like you see, the reason you see this on Twitter is people got really pissed, right? It's just like classic online beef has emerged because of this announcement. And let me try and map out like the elements in the beef stew, in my opinion. And you tell me like what you think of these various actors or, I mean, I'm not going to ask you to like make a judgment call about like what is the right position to have here, but like what your read on it is. So it's like, obviously there is like the anti-AI crowd, which over the last couple of years has gotten pretty militant about the use of AI for any creative endeavor or any endeavor at all.

8:02There's a spectrum, right? And how people feel about it. But the people who are tweeting about it and are getting really mad about it are very staunchly anti-AI, especially in creative endeavors. Then there's the AI apologist slash AI booster crowd, which is like, hey, this is empowering me. I'm able to make art that I wasn't able to make because of these tools. That's great. And then I would say the Joker element in here is the trend that is not limited just to this controversy, but maybe weaponization is not the right word, but I'm going to go with it. but like the weaponization of language around ableism and privilege.

8:55So that is kind of like, that's a lot going on in the beef stew here. What do you think, like, where have we netted out? Are people still mad or is there a truce? What's going on? I think people are still mad. I think a lot of people have been mad at NaNoWriMo for various reasons over the years that we're not going to get into today. But there are people who are skeptical of the community who are still in it, and they're definitely still mad. I think it's just like, I don't know. I mean, you exactly mapped it out correctly that there are these various things coming together that have created just the perfect storm for a Twitter panic.

9:41But underlying this is the idea that AI is like being able to be creative is like some kind of right or that having ideas alone is in itself enough to be an author, writer or an artist. you know having like this the the right to be able to produce whatever you want by whatever means and we see this playing out in court right so like in the last couple years it's like all pretty famous authors who you know wouldn't just like sue sue companies for the fuck of it are getting into this fray so it's not just like oh people are mad on twitter it's like people whose money is on the line are actually also pretty pissed off about AI.

10:36There was a group of writers last year that sued OpenAI, which runs ChatGBT, but they're seeking class action status with the claim that ChatGBT is copying their works without their permission because it's ingesting all of these books that they've written without permission. And then just last month, so August, a group of news outlets filed a lawsuit against open AI claiming that the company was using their copyrighted news articles without permission or payment. So it's all, it's like the stance that NaNoWriMo is taking, you know, forcibly behind AI is just so interesting. And to me, it's such a forced own goal.

11:24It's like, it's like, why would you do that? Why would you even come out and say something, not just say, we welcome the use of AI in this challenge, but if you don't, you're classist and ableist. It's so spicy to say that. I don't know why you would. It's like, I personally, just to put my cards on the table, I don't like the stated position that they put out there like many of the authors who got mad. I just think it's a really wild position to have on AI. But even giving them benefit of the doubt and accepting that argument, this seems like the worst venue possible to make this argument because there's an element to this whole organization and writing a novel in a month, which is almost like an endurance and athleticism aspect of writing, where it's just like, we're going to write a gigantic number of words in one month.

12:29And that is why this whole organization exists. And it's a bit stunty, right? It's almost like, oh, forget about the quality. It's just like, are you able to produce 50 ,000 words of prose in one month? And it would be as if like, you know, we're having a marathon and I'm showing up in a car and I win the marathon. And then I'm like, it's ableist to say that I'm not able to participate in this marathon in my car while everyone else has to run. It's just a wild venue to make that argument, in my opinion. Yeah, it's like the point of it is the effort of it. and they're saying, well, you don't really need to do the effort if you don't want to or you're not able to.

13:16You can literally, with LLMs, produce 50 ,000 word novels basically as fast as you can write a prompt or click a mouse or whatever. It's kind of, yeah, it's a really bad place to make this argument. So, do you want to talk about some of the specific famous authors or the one that you kind of like mentioned in the story that was part of the organization? Yeah, so Daniel Jose Older, who has written some of the Star Wars canon and has written a bunch of fantasy series and things like that, is pretty prolific. He tweeted that he's stepping down from the writers board for NaNoWriMo because of this whole debacle.

14:13Maureen Johnson, who's also a bestselling YA novelist, also tweeted that she's stepping down from the writers board of the Young Writers Program and said, I want nothing to do with the organization from this point forward. very forceful statements from some of these authors um adam christopher who is also an author um came out and said you know nanorimo is anti-writing anti-art anti-creativity anti-craft um which again pretty pretty forceful and yeah it's it's just like a lot of these people are saying we're stepping down and we encourage you to also basically boycott this organization because we're so against what they've come out and said and it's interesting because last year they actually kind of soft launched nanorimo soft launched the idea of being pro ai um they they wrote in a blog post that was very short but it was basically like you know use whatever you want, including AI.

15:19And then they did like a gimmicky, like, here's a chat GBG response to the idea that you should use, you know, AI in your writing. And they left it at that. And people didn't really, like most people didn't really notice. Definitely not as many people got so upset about it last year, but it's this particular statement that people really reacted to and that took off the way that it did. And I've had a couple people email me and message me and say, you know, you also should mention that the sponsors or some of the sponsors in this year's challenge are AI companies, which is definitely interesting, definitely adds a new twist to this uh pro writing aid which is uh it's like a like an ai powered tool suite for writing like you know grammar checking and things like that and um writing like quote unquote writing assistance which is text generation you know you're adding like details and dialogue and things like that through this tool um so yeah people are kind of like well they're taking money from AI companies.

16:35So of course they're pro AI and against anyone who criticizes them for using AI, which I thought, you know, that's a good point. I don't know. I don't know enough about these companies to really dig into what they are specifically about, or what Nanarima is doing with that data. But, you know. Yeah, I think one of the sponsors is Scrivener, right? which is a very popular tool for scriptwriters and authors. Did you use it? I know Jo used it. I tried. I was like, that's too much. Yeah. It's awesome. But it's like, there are many tools that people can use when they work on a long project. Like sometimes Google Docs is not enough.

17:20And I get that. And I'm sure that some of them have AI elements to some degree. But, sorry, I forgot the name. But the one you mentioned. Oh, ProWritingAid. Yeah, that one, I was looking at its site and it explicitly promotes AI as part of its brand. So understandably, people latched onto that and it got really mad. Another tangent, and we don't have to get into it, but another reason this is a really terrible venue, you mentioned YA. I assume that a lot of the people who participate in this are into YA or aspiring YA authors. and by definition are like early career authors and um i noticed because my wife is a big reader and also not to brag but she's a book critic for the new york times um and it's like as a fandom as an audience um it is the most vicious audience that i can think of like people talk about gamers and how mad gamers can get it's like do not fuck with like the ya readership um so it's like that is in the mix here as well and another thing that makes it a real terrible...

18:30Yeah, they're very protective of their own fandom and very vocal and very mobilized when they don't like something. Yeah, I assume that some of these writers are like they hold their positions earnestly but also I think there's probably a pressure to distance yourself from this or anything AI because you don't want to be associated with that given your readers and the people who support your work. Yeah. So... where has NaNoWriMo left this off? What is the latest? I mean, I assume they saw the extreme explosion that they created.

19:11They put out, or they updated the original post and added some phrasing to it. I wonder if they wrote it or if AI wrote it. But the edition says, we are troubled by situational abuse of AI and that certain situational abuse clearly conflicts with our values. We want to make it clear that AI is a large umbrella technology and that the size and complexity of that category, which includes both non-generative and generative AI, among other uses, contributes to our belief that it is simply too big to categorically endorse or not endorse. I don't know. They're trying to have it both ways, which never, ever, ever works.

19:52And they're not even doing it like a nuance. That's not even like adding nuance. that's just kind of like, well, sometimes it's bad, sometimes it's good. But we're going to stand by our statement that if you say it's bad, you're ableist and classist. Yeah. Closing thoughts, books, not even once. That's what I have to say. Blogs only. Blogs only, right. You can collect your blogs and publish them as a book later if you want. Otherwise, no thanks. Okay, let's leave that there and go to break. And when we come back, we'll talk about something much better than books, which is Doom. So we'll be right back.

20:54okay we're back with a story by emmanuel and the headline is this is doom running on a diffusion model uh i love to see doom running on weird shit doom runs on our very website if you can find it it's a little easter egg um so i guess i see a lot of like this like doom like doom running on blank has become a meme it's like doom running on your fridge you're running on your roomba which are things that doom has run on but putting doom on things has become its own genre slash joke so yeah I saw this and I was kind of like okay Doom running on something else great but it actually is way more interesting than that and it has some interesting implications within gaming itself so yeah Emmanuel do you want to explain what is game engine letter N G E N But pronounced like you said, pronounced game engine.

22:00Yeah. Yeah. So there are a lot of generative AI products and gimmicks and papers. I would say way more than we can track. If you're keeping an eye on like preprint papers, which I do, there are dozens and dozens a day. But this one went really viral. so we wanted to cover it also and what it is basically is a group of researchers from google and deep blind which is google's ai outfit and some researchers at tel aviv university which i believe has some overlap with these google researchers they put out this paper um where they present this software called Game Engine. And basically what they did is they trained a stable diffusion model.

23:02Stable diffusion is an open weights AI model that we have covered a lot, primarily because it's open weights. People can tweak it and make it generate different kinds of images. Sam, you and I are obviously very interested in that because people mostly use that to generate non-consensual images. But in this case, the researchers trained the model on a bunch of Doom gameplay. And what Game Engine does is, based on the frames that come before and based on a player's input, it predicts the next frame. Um, it is probably more complicated than that. Um, if you get into the weeds, um, but to simplify that even further, what is basically happening is doom is a game that runs on a traditional game engine.

24:07Um, I believe, believe it's called it tech maybe is like what the branding was for the engines back then. and this is not running on that. It's trained on that, but it is running entirely on the stable diffusion model. It's not going through the game engine. It's just like looking at the frames that came before and then looking at the player input and saying based on that statistically, which is what AI is basically, it's always statistically producing the next word or frame or pixel that most matches the prompt or that it thinks would most match what you want, which is a pretty wild development.

24:50It is very far from being a practical product, and we can talk about that more later, but it is a proof of concept, and the proof of concept is essentially like maybe in the future we're in a place where we don't need game developers to build game engines, which are these huge investment, like hugely complicated tasks that game developers need to accomplish before they develop and ship a game. They're just going to be able to kind of use generative AI in the same way that we produce images and video and text and kind of generate video games from these AI models entirely. Yeah. So, I mean, why is Doom just simple enough to make this work?

25:45Because like the original Doom, it's a pretty simple game. You know, it's a maze with monsters in it. Is that why they chose Doom? Or did they choose it for the meme ability? Like, what's the reasoning behind that? First of all, how dare you? Doom is a beautiful work of art. I'm not denying that. So, to answer your question, they definitely understand. They did it for the reason that you said, right? It's just like, it's a very... And the virality of the study has proven their point. The joke is... I mean, let's get into it for a second. When Id made Doom, John Carmack, who is actually working on AGI right now, but he made the Doom engine, and he kind of ideologically believes that stuff should be open source.

26:36That was kind of the scene back then in the 90s for shareware and stuff. A lot of their early work was based on tweaking other existing work. So he really believed that Doom should be open source, and it is. And that is why people are able to run it on everything. That is why the joke exists, is because people can tweak in the code and put it on anything and modify it so it can run on a smart toothbrush or whatever. And that's why they did it. But I don't think they would have been able to do it with, I don't know, whatever. This is how little I keep up with video games at the moment. I can't even think of what is the hottest new video game.

27:24I'm playing Horizon, right? So we wouldn't be able to do it with Horizon, I think, because the visual complexity and the density of details would be really hard for the model to predict the next frames in. So it's a meme, it's visually simplistic, and the input are simplistic as well, right? It's like Doom is a 3D game, but you're kind of moving on one plane. You're not looking around and down and moving in and out of objects like modern games do today. so it's easy to replicate through a diffusion model for that reason. Also, I would note is a lot of the visuals are kind of static, right? So it's like the level is pretty static.

28:16A lot of the 2D textures on the walls and the doors and stuff is pretty static. the diffusion model runs at I think it's 20 frames per second which is very bad for an old game but it's functional and when you see video of it all those static elements of the game are recreated pretty well where there is a lot of animation and fast changes in animation, which is when you would shoot an enemy and the enemy reacts to that, that's where you start to see the AI quote-unquote hallucination stuff pop in, which is just like you see some visual deformities. And I think that's probably because, I don't know, the level of detail and change that happens there is kind of difficult for the model to train on properly and recreate properly.

29:16But yeah, it's a meme and it's simple enough to, simple enough for a diffusion model to handle, it seems. Yeah. I mean, I noticed in the video, like watching the video, the demo that they published, you don't see a lot of demons on purpose. You know, it's like, I'm sure they left a lot of those out because they don't look that good when you're generating them in real time. I also wonder, like, I mean, this kind of feels like an iteration of what we saw with No Man's Sky where that was like the procedural generation but maybe this is a totally different level of like using AI in video games but you know that was what like 2015, 2016 2016 yeah so we weren't really talking about like AI replacing devs at that point it was more of a novelty yeah it's interesting you brought that up I didn't think of it but because like we said people are very mad about ai and its various forms um across any medium right um but people obviously use ai and video games all the time whether it's like enemy behavior or procedural generation is really um I would say like the most common and used, you know, application of it.

30:46But I would, it's, it's probably, it's different, right? Cause it's like what you're doing is like you're procedurally generating a world and that is created and then you play through it. But all the assets and art and design that stuff is, is human made. So I don't know, maybe it's just because people got used to it but people don't get mad about that obviously or at least not for these ethical uh creative and copyright copyright reasons yeah for sure okay um yeah any more any last thoughts on on doom doom guy demons um i think i mean okay so So one reason I wanted to write the story is that a few months ago, the CEO of NVIDIA was at a conference.

31:38He's giving a lot of talks right now because everybody wants to know how he created one of the most successful companies ever. But somebody asked him if he thinks that we're headed to a future where every pixel, no matter where you see it, is AI generated. and he basically said yes um and a few months ago when he said that that seemed really outlandish like the idea that we would ai generate video games instead of using traditional video game engines uh it just seems really really wild and to be clear like this is not a peer-reviewed study yet um it has no impact on the way video games are being made right now it's like very speculative uh very experimental it doesn't mean that anything is changing right now in the way that video games are being made but um it is a proof of concept it is really impressive and i some people got mad at the story because they didn't think I sufficiently dunked on all the limitations that are obvious in this proof of concept.

33:02And that's fair. And I mean, I thought I was pretty clear on it, but maybe I should have done a better job to outline all the possible issues with this. But I think it's important to track these proof of concept papers and take them seriously because in the past two years, we've seen how a lot of the flaws with AI that people use to dismiss it are fixed pretty quickly. The thing that I keep coming back to is I feel like in 2022, when we were seeing a lot of AI images online and social media, people were really making fun of the hands and how they were all mangled and people had like 12 fingers.

33:53And the meme was like, oh, I'm sure that this will replace all visual artists any minute now. And then they would, you know, show AI generated hands being completely mangled. And that was totally a meme. And you could still see those deformities in a lot of AI generation. but that is basically a solved problem in my opinion. Like if you're really trying to AI generate hands and you do the proper training and use the latest technology, it's just like you're going to get realistic looking hands. Like problems like this can be solved. So something that is a proof of concept one day could become very practical just a few months from now.

34:36And I wouldn't dismiss a lot of these generative AI developments based on the limitations that we're seeing currently. Does it mean that game engines are a thing of the past? Definitely not right now. I'm very skeptical. But again, I just think it's important to track these things because they move pretty fast. And it's kind of shocking how fast all of the problems can be solved. That doesn't solve the ethical problems. It doesn't solve the labor problems, all the problems that we report on to no end with training data and so on. But if this is something you're worried about, you also got to keep an eye on it.

35:17Yeah. I, for one, can't believe that people got mad at you for not addressing every single possible theoretical angle in one six-underword blog post. That's just crazy to me. but yeah it's also crazy that these these things are moving so incredibly fast that they're hard to keep up with but we try okay we're gonna I think wrap the pod there if you're a subscriber stick around and we're going to talk about a story Emanuel wrote about face swap apps and the Apple store. And otherwise we will see you next week, hopefully with the full cast here next time. See you then.