Monologue: LLM Code Is Already Breaking Big Tech

0:00This is an iHeart podcast. Guaranteed human. Run a business and not thinking about podcasting? Think again. More Americans listen to podcasts than ad-supported streaming music from Spotify and Pandora. And as the number one podcaster, iHeart's twice as large as the next two combined. Learn how podcasting can help your business. Call 844-844-iHeart. What if mind control is real? If you could control the behavior of anybody around you, what kind of life would you have? Can you hypnotically persuade someone to buy a car? When you look at your car, you're going to become overwhelmed with such good feelings.

0:32Can you hypnotize someone into sleeping with you? I gave her some suggestions to be sexually aroused. Can you get someone to join your cult? NLP was used on me to access my subconscious. Mind Games, a new podcast exploring NLP, a.k.a. neurolinguistic programming. Is it a self-help miracle, a shady hypnosis scam, or both? Listen to MindGames on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

1:05Hello and welcome to this week's Better Offline monologue. I'm your host Ed Zitron.

1:16I heard something really worrying the other day about a major hyperscaler. According to the source, said hyperscaler was allowing and even encouraging non-technical workers to deploy code to consumer-facing products, specifically those who cannot read or write code, vibe coding their own projects using generative AI, with their code at some point theoretically reviewed by an actual software engineer before it gets pushed to production. The cold comfort of that review is that it assumes that software engineers Or at least the ones reviewing that code Are actually adept at code review Even if they are that they have sufficient time to look over the overly verbose code that LLMs spew In some cases I've heard Management is actively encouraging and even mandating These non-technical workers to use LLMs to make these features Creating a mutation of tech debt Where somebody who cannot code Uses a machine that doesn't think to create code with no intention that nobody really understands, and does so at such a velocity that it burdens the actual technical workers with constantly having to monitor and fix it.

2:17LLMs do not understand anything, nor do they think, which means any solutions they build or theoretical bug reports that they may make are immediately questionable. Their hallucinations are such that even features you believe are part of your code, after all you can't read it, might not be there, or might be poorly designed, or might have some sort of unforeseen problem that neither you nor the LLM are aware of because its training data is based on code that already exists versus any ability to solve novel problems. Also, the idea of having any number of non-technical people ship code is fucking insane and indicative of an overwhelming ignorance on the part of management.

2:52Even a few years of having overwhelming amounts of code written by LLMs, even by engineers who know what it says and have some intention in the prompts they do, is going to create a situation where most of the code is written without any intention, making it much harder to debug, because nobody really knows why it was written that way, because LLMs don't think. I know I'm repeating myself already, but this situation is chilling me. Even outside of the vibe coding, there's a larger problem of developers writing code with LLMs that they barely review. In some cases, because they don't feel they need to and just skim read it, and many, many more because their bosses are demanding they ship features faster than is responsible or safe.

3:31Remember, many many tech companies are mandating llm use harassing their workers checking how much they use llms i've heard this from multiple companies and really it's not just for them but it's especially hard on software engineers adding a layer of code written by people who quite literally do not understand what it says or does guarantees future situations where major services simply break and the more of this nonsensical code that's allowed to be stood up on these services the harder it will be to fix. Code isn't just something you write once and leave forever. It needs to be maintained by other people.

4:08Sometimes years in the future, especially when people keep being laid off, which becomes much harder when there's lots of code to go through, written by an LLM piloted by somebody who doesn't know what they're actually doing, and relies on the LLM which doesn't know anything to tell them what's going on. Again, I realize I'm repeating myself, but LLMs do not have thoughts or feelings or knowledge. They are generating based on the parameters of their training data, which means that all of their code is, at best, an abstraction of somebody else's back and forth with the chatbot. The code is not written efficiently or with any consideration of who might have to work with it in the future, or indeed what other things might be involved.

4:46LLMs only know what they are fed or what they're connected to. They don't know the nuances of code. They don't know the nuances of software engineering or architecture, which means that they only know so much based on their training data and the environment around them, kind of. They don't know the nuances of how a service, let's say Meta or Microsoft's, has been built over decades. And indeed, the more of this code that's used to build those services, the less that these companies know about how their actual fucking software works. This is setting up the software industry for disaster after disaster.

5:20and it's already started to happen. To quote the information from this week, according to internal metacommunications and an incident report seen by the information, a major security alert occurred last week after a metasoftware engineer used an in-house agent tool similar to OpenClaw to analyze a technical question that another employee had posted on an internal discussion forum. After doing the analysis, the AI agent posted a response in the discussion forum to the original question, offering advice on the technical issue, according to internal communications. The agent did so without approval from the employee.

5:53It's so cool that this is happening. It's so cool. It's great. It's actually brilliant. How fucking insane. I'm kindly going to assume that the person using this knew what they were doing, but the idea that we have, and I think this is what's happening with open source too, we have people with LLMs who are like, yeah, well, the LLM tells me I'm good, so it must be. Let me just run this LLM past your problems. It's why we're getting all these junk pull requests on GitHub on open source projects, people that think they're competent because an LLM told them to, are fucking up the entire software world.

6:26And according to the information, Meta systems storing large amounts of company and user-related data were accessible to engineers who didn't have permission to see them. And this was marked as SEC1 incident, the second highest level of severity on an internal scale that Meta uses to rank security incidents. And again, that's quoting the information. The incident follows multiple problems caused at Amazon by its Kiro and QLLMs. I quote Business Insider's Eugene Kim. On March 2nd, customers across Amazon marketplaces saw incorrect delivery times when adding items to their carts. The incident led to nearly 120 ,000 lost orders and roughly 1.6 million website errors.

7:05Amazon's AI tool Q was one of the primary contributors to trigger the event, according to an internal review. On March 5th, another outage caused a 99 % drop in orders across Amazon's North American marketplaces, resulting in 6.3 million lost orders, one of the internal documents stated. One key factor was a production change that was deployed without using a formal documentation on the approval process called model change management. Very cool. I also want to be clear that it appears that these incidents were created by use of these tools by actual software engineers People that ostensibly know how code and software architecture works Reliance on large language models, especially at a time when executives are putting more pressure on engineers to deliver more features and ship more code Means that software engineers are being incentivized to be sloppy and to ship slop itself There is nothing inherently good about automating code, nor is there any inherent value in shipping a lot of it fast LLMs convince you that what you're writing is good and stable and does the thing you want it to, and if you're skim reading the outputs, or of course unable to read them at all, it's easy for you to assume that because you asked a model that does not have thoughts where it thinks you got something right, that you actually did so, and that it got it right.

8:20To be explicit, allowing an LLM to write all of your code means that you are no longer developing code, nor are you learning how to develop code, nor are you going to become a better software engineer as a result, nor are you solving actual problems. You are just handing work over to something and taking dog shit out. I'm not saying that all coders using LLMs are inherently bankrupt or anything, but I hear these stories about writing all the code, and they give me the willies. And I know what I'm saying sounds like an insult or hyperbole. I don't mean it in that way. If you are just a person looking at code, you're only as good as the code the model makes.

8:58And as Mobitar recently discussed, these models are built to galvanize you, glaze you, and tell you that you are remarkable, as you barely glance at globs of overwritten code that, even if it functions, eventually grows to a whole built with no intention or purpose other than what the model generated from your prompts. I'm sure there are software engineers using these models ethically, who read all the code, who have complete industry over it, and use it like a glorified autocomplete. I can see the value. I'm also sure that there are some that are just asking it to do stuff, glancing at the code and shipping it.

9:30It's impossible to measure how many of each camp there are, but hearing Spotify's CEO said that its top developers are basically not writing code anymore makes me deeply worried, because this shit isn't replacing software engineering at all. It's mindlessly removing friction and putting the burden of good or right on a user that it's intentionally gassing up. And ultimately, this entire era is a test of a person's ability to understand and appreciate friction. Friction can be a very good thing. When I don't understand something, I make an effort to do so, and the moment it clicks is magical. In the last three years, I've had to teach myself a great deal about finance, accountancy, and the greater technology industry.

10:08And there have been so many moments where I've walked away from the page frustrated, stewed in self-doubt that I'd never understand something. I eventually did. It took time. It really took time, and really that luxury of time is important, and sadly many software engineers face increasingly deranged deadlines set by bosses that don't understand a single fucking thing about their job or the software industry itself, let alone what LLMs are capable of, or what responsible software engineering might be. The push from above to use these models because they can, and I quote, write code faster than a human, is a disastrous conflation of fast and good, all because of flimsy myths peddled by venture capitalists and the media about LLMs being able to replace software engineers.

10:50It's fucking stupid. It's a disgrace. And there are real problems that are going to happen as a result. The problem is that LLMs can write all code, theoretically. They can just put the code out that you might have written yourself. It doesn't mean the code is good or that somebody can read it and understand its intention or that it works or that it will work in the future or that you can build any kind of sustainable or, I don't know, like stable in any way organization on top of it. Or even that having a lot of code is a good thing both in the present and in the future of any company built using this generative code.

11:27Adding the variable of code written by people who quite literally do not understand it guarantees something severe and calamitous in the future. Though I'd argue that was the case without their influence. Increasing the volume of code contributed to a company Naturally increases the amount of time needed to read it And the amount of effort needed to maintain it Which naturally encourages people to use LLMs to summarize it And then Well, you have to rely on the LLMs to tell you what good looks like And they don't know a single fucking thing And it also creates a new burden on the technical workers To have to clean up the slop in their day-to-day lives Generative code is a digital ecological disaster One that will take years to repair Thanks to company remits to write as much code as fast as possible And use LLMs as much as possible too Every single person responsible must be held accountable Especially for the calamities to come As lazily managed software companies see the consequences Of building their software on sand I'll see you all next week

12:38What if mind control is real? If you could control the behavior of anybody around you, what kind of life would you have? Can you hypnotically persuade someone to buy a car? When you look at your car, you're going to become overwhelmed with such good feelings. Can you hypnotize someone into sleeping with you? I gave her some suggestions to be sexually aroused. Can you get someone to join your cult? NLP was used on me to access my subconscious. Mind Games, a new podcast exploring NLP, aka neurolinguistic programming. Is it a self-help miracle, a shady hypnosis scam, or both? Listen to Mind Games on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

13:18This is an iHeart Podcast. Guaranteed human.