CZM Rewind: The Case Against Generative AI (Part 1)

0:00This is an iHeart Podcast. Guaranteed Human.

0:31is choose an identity that other people can't have. I knew something had happened to me in the middle of the night, but I couldn't hold on to what had happened. These are just a few of the moving and important stories on my 13th season of Family Secrets. Listen to Family Secrets on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Whether it is getting swatted or just hateful messages online, there is a lot of harm in even just reading the comments. That's cybersecurity expert Camille Stewart-Gloster on the Therapy for Black Girls podcast. Every season is a chance to grow.

1:06And the Therapy for Black Girls podcast is here to walk with you. I'm Dr. Joy Harden-Bradford, and each week we dive into real conversations that help you move with more clarity and confidence. This episode, we're breaking down what really happens to your information online and how to protect yourself with intention. Listen to Therapy for Black Girls on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. What are the cycles fathers pass down that sons are left to heal? What if being a man wasn't about holding it all together, but learning how to let go? This is a space where men speak truth and find the power to heal and transform.

1:47I'm Mike Della Rocha. Welcome to Sacred Lessons. Listen to Sacred Lessons on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

2:21After a few three-part episodes, I had an idea. What if I did a four-parter? In all seriousness, I know that this is a little bit long, but the topic we're about to explore demands quite a bit of depth, and it isn't something I could really do justice to in a one-parter or two-parter or I guess even three-parter. But let's get into it. Over the last few months, we've felt the vibes shift downward in an aggressive way, with both Mark Zuckerberg and Clammy Sam Altman saying that we're in a bubble. In the latter case, said warnings of a bubble are always couched in rank hypocrisy, is it's always implied that whoever it is and the companies they represent aren't part of that bubble, but rather it's other people and other companies making unfortunate decisions.

3:00The thing is, there's really no escape for either of these guys. Not for Zuck and definitely not for Sam Altman. And over the next four episodes, I'm going to make a comprehensive case for the fact that we're in a bubble and condense everything I've been talking about into one series. And I know I've been all over the place and I get a lot of people saying, oh, well, where did you talk about this and where did you talk about that? And that's kind of fair when you put out as much as I do. But I'm going to break this down in four episodes. I'm going to give you a comprehensive argument against the bubble.

3:29Well, I mean that for a bubble, I guess, but against generative AI in general. But in this episode, I think it's good to start from the beginning and work our way forward. To track the thread from the origins of ChatGPT to the billions burned building data centers all over the world. And the weak business justifications for burning in nearly a trillion dollars to keep this hollow industry alive. Now, in 2022, a kind of company called OpenAI surprised the world with a website called ChatGPT that could generate text that sort of sounded like a person using a technology called large language models, LLMs, which can also be used to generate images, video, and computer code, or at least would eventually.

4:05Large language models require entire clusters of servers connected with high-speed networking, all containing this thing called a GPU, graphics processing units. These are different to the GPUs in your Xbox or laptop or gaming PC. They cost much, much more, and they're good at doing the processes of inference, the creation of an output of any LLM, and training, feeding masses of training data to the models, or feeding them information about what a good output might look like so they can later identify a thing or replicate it. These models showed some immediate promise in their ability to articulate concepts or generate video, visuals, audio, text, and code.

4:38They also immediately had one glaring obvious problem. Because they're probabilistic, meaning that they're just guessing whatever the right output might be, these models can't actually be relied upon to do exactly the same thing every single time. So if you generated a picture of a person that you wanted to, for example, using a storybook, every time you create a new page using the same prompt to describe the protagonist, the person would look different. And that difference could be minor, something that a reader could shrug off, or it could make the character look like a completely different person.

5:09Now, none of this, by the way, is me validating or saying that any of this stuff is good. I'm just describing it. Moreover, the probabilistic nature of generative AI meant that whenever you asked it a question, it would guess as to the answer, not because it knew the answer, but rather because it was guessing on the right word to add in a sentence based on previous training data. As a result, these models would frequently make mistakes, something which we later refer to as hallucinations. And that's not even mentioning the cost of training these models, the cost of running them, the vast amounts of computational power they required, the fact that the legality of using materials scraped from books on the web without the owner's permission was and remains legally dubious, or the fact that nobody seemed to know how to use these models to actually create profitable businesses.

5:50These problems were overshadowed by something flashy and new and something that investors and the tech media believed would eventually automate jobs that have proven most resistance towards automation, knowledge work and the creative economy. the newness and hype and these expectations sent the market into a frenzy with every hyperscaler immediately creating the most aggressive market for one supplier i've ever seen nvidia has sold over 200 billion dollars of gpus since the beginning of 2023 becoming the largest company on the american stock market and trading over 170 dollars as of writing this sentence only a few years after being worth 19 and 52 cents a share now there's a stock split that happened there but it works out that way.

6:31Now, while I've talked about some of the propelling factors behind the AI wave, automation and novelty, that's not really the complete picture. A huge reason why everybody decided to do AI was because the software industry's growth was slowing, and SaaS, software as a service company, valuations were stalling or dropping, resulting in the terrifying prospect of companies having to under-promise and over-deliver and be efficient. You know, gross things like running sustainable businesses. Things that normal companies, those whose valuations aren't contingent on ever-increasing, ever-constant growth, don't have to worry about because they're normal companies.

7:05Suddenly, there was a new promise of new technology, large language models that were getting exponentially more powerful, which was mostly a lie but hard to disprove because powerful can mean basically anything. And the definition of powerful depended entirely on whoever you asked at any given time and what that person's motivations were. The media also immediately started tripping over its own feet, mistakenly claiming OpenAI's GPT-4 model tricked a TaskRabbit into solving a capture, it didn't, this never happened, or saying that, and I quote, people who don't know how to code already used bots to produce full-fledged games.

7:37And if you weren't wondering what the New York Times was referring to when they said full-fledged there, it meant Pong and a cobbled-together rolling demo of Skyroads, a game from 1993, likely because a bunch of that training data was fed into the models. Now, the media and investors helped peddle the narrative that AI was always getting better, could basically do anything, and that any problems you saw today would inevitably be solved in a few short months, or years, or at some point, I guess. Not really sure when that point is, but damn do they think it's coming. And LLMs were touted as a kind of digital panacea, and the companies building them offered traditional software companies the chance to plug these models into their software using an API, thus allowing them to ride the same generative AI wave that every other company was writing.

8:22The model companies similarly started going after individual and business customers, offering software and subscriptions that promised the world, though this mostly boiled down to chatbots that could generate stuff and then doubled down with the promise of agents, a marketing term that's meant to make you think autonomous digital worker, but really means broken digital chatbot of some sort, or just broken digital product. It really depends how you're feeling that day. Throughout this era, investors in the media spoke with a sense of inevitability that they never really backed up with data. It was an era based on confidently asserted vibes.

8:55Everything was always getting better and more powerful, even though there was never much proof that this was truly disruptive technology, other than its ability to disrupt apps you were using with AI, making them worse, for example, suggesting questions on every Facebook post that you could ask Meta AI, but which Meta AI couldn't answer. And I mean, on memes, on just random posts. It's really not useful in any way, shape, or form. AI became omnipresent, and it eventually grew to mean everything and nothing. OpenAI would see its every move lorded over like a gifted child. Its CEO Sam Ullman called the Oppenheimer of our age, even if it wasn't really obvious why everybody was impressed.

9:33GPT-4 felt like something a bit different, but was it actually meaningful? The thing is, artificial intelligence is built and sold on not just faith, but a series of myths that AI boosters expect us to believe with the same certainty that we treat things like gravity or the boiling point of water. Can large language models actually replace coders? Not really, no, and I'll get into why later in this series.

10:10And as the number one podcaster, iHeart's twice as large as the next two combined. So whatever your customers listen to, they'll hear your message. Plus, only iHeart can extend your message to audiences across broadcast radio. Think podcasting can help your business? Think iHeart. Streaming, radio, and podcasting. Let us show you at iHeartAdvertising.com. That's iHeartAdvertising.com. For 25 years, I've explored what it means to heal, not just for myself, but alongside others. I'm Mike DeLaRocha. This is Sacred Lessons, a space for reflection, growth, and collective healing. What do you tell men that are hurting right now?

10:51Everything's going to be okay on the other side. You know, just push through it. And, you know, ironically, the root of the word spirit is breath. Wow. Which is why one of the most revolutionary acts that we can do as people is just breathe. Next to the wound is zero gifts. You can't even find your gifts and they should go through the wound. That's the hard thing. You think, well, I'm going to get my gifts. I don't want to go through all that. You got to go through the wounds of your life. Listening to other people's near-death experiences and that's all they say. In conclusion, love is the answer.

11:22Listen to Sacred Lessons as part of the My Cultura Podcast Network, available on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. On the podcast Health Stuff, we are tackling all the health questions that keep you up at night. Yes, I'm Dr. Priyanka Wally, a double board certified physician. And I'm Hari Kondabolu, a comedian and someone who once Googled, do I have scurvy at 3 a.m. On Health Stuff, we're talking about health in a different way. It's not only about what we can do to improve our health, but also what our health says about us and the way we're living. Like our episode where we look at diabetes.

12:00In the United States, I mean, 50 % of Americans are pre-diabetic. How preventable is type 2? Extremely. Or our in-depth analysis of how incredible mangoes are. Oh, it's hard to explain to the rest of the world that, like, your mangoes are fine because mangoes are incredible, but, like, you don't even know. You don't know. You don't know. It's going to be a fun ride. So tune in. Listen to Health Stuff on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Hey, I'm Kelly, and some of you may know me as Laura Winslow. And I'm Telma, also known as Aunt Rachel. If those names ring a bell, then you probably are familiar with the show that we were both on back in the 90s called Family Matters.

12:47Kelly and I have done a lot of things and played a lot of roles over the years, but both of us are just so proud to have been part of Family Matters. Did you know that we were one of the longest-running sitcoms with the Black cast? When we were making the show, there were so many moments filled with joy and laughter and cut up that I will never forget. Oh, girl, you got that right. The look that you all give me is so black. All black people know about the look. On each episode of Welcome to the Family, we'll share personal reflections about making the show. Yeah, we'll even bring in part of the cast and some other special guests to join in the fun and spill some tea.

13:25Listen to Welcome to the Family with Telma and Kelly on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

13:54But every few weeks, a story runs where everybody tries to pretend that AI is replacing workers with some sort of poorly sourced and incomprehensible study. Never actually saying somebody's job got replaced by AI because it isn't happening at scale. And because if you provide real world examples, people can actually check if they're true. Now, I want to be clear. Some people have lost jobs to AI. Not just, not really white collar workers, software engineers, or really any of the career paths that the mainstream media and AI investors would have you believe. Brian Merchant has done excellent work covering how LLMs have devoured the work of translators, using cheap, almost good automation to lower already stagnant wages in a field that has already been hurting before the advent of generative AI, with some having to abandon the field and others pushed into bankruptcy.

14:38I've heard the same for art directors, SEO experts, and copy editors, and Christopher Mims of the Wall Street Journal covered these last year. These fields all have something in common. Shitty bosses with little regard for their customers who have been eagerly waiting for the opportunity to slash labor. To quote Merchant, the drumbeat marketing and pop culture of powerful AI encourages and permits management to replace or degrade jobs they might not otherwise have. Across the board, the people being replaced by AI are the victims of lazy, incompetent cost cutters who don't care if they ship poorly translated text.

15:08To quote Merchant again, AI hype has created the cover necessary to justify slashing rates and accepting just good enough automation output for video games and media products. Yet the jobs crisis facing translators speaks to the larger flaws of the large language model era, and why other careers aren't seeing this kind of disruption. Generative AI creates outputs, and by extension defines all labor as some kind of output created from a request. In the case of translation, it's possible for a company to get by with a shitty version, because many customers see translation as, what do these words say, even though, as one worker told Brian Merchant, translation is about conveying meaning.

15:44Nevertheless, translation work has already started to condense to a world where humans would at times clean up machine generated text, and the same worker warned that the same might come for other industries. Yet the problem is that translation is a heavily output-driven industry, one where idiot bosses can say, oh yeah, that's fine, because they ran an output back through Google Translate, and it seemed fine in their native tongue. The problems of a poor translation are obvious, but customers of translation are, it seems, often capable of getting by with a shitty product. The problem is that most jobs are not output-driven at all, and what we're buying from a human being is a person's ability to think and do.

16:20Every CEO talking about replacing workers with AI is an example of the real problem, that most companies are run by people who don't understand or experience the problems they're solving, don't do any real work, don't face any real problems, and thus can never be trusted to solve them. In the era of the business idiot, which is a piece I wrote a while ago, I talked about how this was the result of letting management consultants and neoliberal free market sociopaths take over everything, leaving us with companies run by people who don't know how the companies make money, just that they must always make more without fail.

16:50And when you're a big stupid arsehole, every job that you see is condensed to its outputs, and not the stuff that leads up to the output, or the small nuances and conscious decisions that make an output good as opposed to simply acceptable or even bad. What does a software engineer do? They write code, right? What does a writer do? They write words, right? What does a hairdresser do? They cut hair. Yeah, that's of course not actually the case. As I'll get into later in the series, a software engineer does far more than just code. And when they write code, they're not just saying, what would solve this problem with a big smile on their face?

17:21They're taking into account their years of experience, what code does, what code could do, and all the things that might break as a result, and all of the things that you can't really tell from just looking at the code, like whether there's a reason things are made in a particular way. And a good coder doesn't just hammer up the keyboard with the aim of doing a particular task. They factor in questions like, how does this functionality fit into the code that's already there? Or, if someone has to update this code in the future, how do I make it easy for them to understand what I've written and make changes without breaking a bunch of other stuff?

17:50A writer doesn't just write words. They justle ideas and ideas and emotions and thoughts and facts and feelings into a condensed piece of text. They sit up late at night typing thousands and thousands of words that drives them insane. It's often quite emotive, or at the very least driven or inspired by a given emotion, which is something that an AI simply can't replicate in a way that's authentic or believable. And a hairdresser doesn't just cut hair, they cut your hair, which may be wiry, dry, oily, long, short, healthy, unhealthy, on a scalp with particular issues at a time of year when perhaps you want to change length, at a time that fits you and the way you like it, which may be impossible to actually write down, but they get it just right.

18:27And they make conversation, making you feel at ease while they snip and clip away at your tresses with you never having to think for a second, fuck, does this person know what they're doing? Are they going to listen to me? This is the true nature of labor that executives fail to comprehend at scale, that the things we do are not units of work, but extrapolations of experience, emotion, and context that cannot be condensed in written meaning or bunches of trading material. Business idiots see our labor as the results of a smart manager saying, do this, rather than human ingenuity interpreting both a request and the shit the manager didn't say.

19:00Now, what does a CEO do? Well, I did look, and a Harvard study said that they spend 25 % of their time on people and relationships, 25 % on functional and business unit reviews, 16 % on organization and culture, and 21 % on just strategy, with a few percent here and there for things like professional development. them. Hmm. That's who runs the vast majority of companies. People that describe their work predominantly as looking at stuff, talking to people, thinking what we do next, and going to lunch. The most highly paid jobs in the world are impossible to describe. Their labor described in a mishmash of linked inspiration, yet everybody else's labor is an output that can be automated.

19:43As a result, large language models must seem like magic to these dickheads. When you see everything as an outcome, an outcome you may or may not understand, and definitely don't understand the process behind, let alone care about, you kind of already see your workers as LLMs. You create a stratification of the workforce that goes beyond the normal organizational chart, with senior executives, those closer to the class level of CEO, acting as those who have risen above the doldrums of doing things to the level of decision making, a fuzzy term that can mean everything from making nuanced decisions with input from multiple different subject matter experts, to, as ServiceNow Bill McDermott did in 2022, and I quote, make it clear to everybody in a boardroom of other executives that everything they do must be AI, AI, AI, AI, AI.

20:27And that's five of those. The same extends to some members of the business and tech media that have, for the most part, gotten by without having to think too hard about the actual things the companies are saying. Look, I realize this sounds a little mean, and it's not a unilateral statement. And I must be clear, it doesn't mean that these people know nothing, just that it's been possible to scoot through the world without thinking too hard about whether or not something is true, just because an executive said it. When Salesforce said back in 2024 that its Einstein trust layer in AI would be transformational for jobs, the media dutifully wrote it down and published it without a second thought.

21:02It fully trusted Mark Benioff when he said that agent force agents would replace human workers, and then again when he said that AI agents were doing 30 to 50 percent of all the work in Salesforce itself, even though that's an unproven and nakedly ridiculous statement. Salesforce's CFO, by the way, said earlier in this year that AI wouldn't boost sales growth in 2025. One would think this would change how Salesforce was covered or how seriously one takes Mark Benioff, but it hasn't because nobody's really paying attention. In fact, nobody seems to want to do their job in this case.

21:49ad-supported streaming music from Spotify and Pandora. And as the number one podcaster, iHeart's twice as large as the next two combined. So whatever your customers listen to, they'll hear your message. Plus, only iHeart can extend your message to audiences across broadcast radio. Think podcasting can help your business? Think iHeart. Streaming, radio, and podcasting. Let us show you at iHeartAdvertising.com. That's iHeartAdvertising.com. For 25 years, I've explored what it means to heal, not just for myself, but alongside others. I'm Mike DeLaRocha. This is Sacred Lessons, a space for reflection, growth, and collective healing.

22:28What do you tell men that are hurting right now? Everything's going to be okay on the other side, you know, just push through it. And, you know, ironically, the root of the word spirit is breath. Wow. Which is why one of the most revolutionary acts that we can do as people is just breathe. next to the wound is their gifts. You can't even find your gifts unless you go through the wound. That's the hard thing. You think, well, I'm going to get my gifts. I don't want to go through all that. You got to go through the wounds of your life. Listening to other people's near-death experiences, and that's all they say.

23:01In conclusion, love is the answer. Listen to Sacred Lessons as part of the My Kutura Podcast Network, available on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Hi, I'm Radhi Dabluke, and I am the host of a Really Good Cry podcast. This week, I am joined by Anna Runkle, also known as the Crappy Childhood Fairy, a creator, teacher, and guide helping people heal from the lasting emotional wounds of unsafe or chaotic childhoods. We talk about how the things we went through when we were younger can still show up in our adult lives, in our relationships, our reactions, even in the way we feel in our own bodies.

23:37And Anna opens up about her own story, what helped her notice the patterns she was stuck in and how she slowly started teaching her body that it is safe now? So when I got attacked, it was very random. Four guys jumped out of a car and just started beating me and my friend. And they broke my jaw and my teeth. I was unconscious. Then I woke up and I screamed. And I screamed because even though I didn't know who I was or where I was, something in me was just like, hold on, wait. They could kill me and I'm not going to let that happen. I'm not going to let that happen. I'm going to get through this.