Yep, We're At Peak AI

0:00This is an iHeart Podcast.

0:30Thomson Reuters and specialized bikes have since they upgraded to the next generation of the cloud. Oracle Cloud Infrastructure. OCI is the blazing fast platform for your infrastructure, database, application development, and AI needs, where you can run any workload in a high availability, consistently high performance environment, and spend less than you would with other clouds. How is it faster? OCI's block storage gives you more operations per second. Cheaper? Better? OCI costs up to 50 % less for computing, 70 % less for storage, and 80 % less for networking. Better? In test after test, OCI customers report lower latency and higher bandwidth versus other clouds.

1:12This is the cloud built for AI and all your biggest workloads. Right now, with zero commitment, try OCI for free. Head to oracle.com slash strategic. That's oracle.com slash strategic. Wells Fargo announced a new$20 million program in 2025, teaming up with nonprofits to support small business owners. That's how Wells Fargo is helping strengthen small businesses and communities. Wells Fargo, the bank of doing. Learn more at wellsfargo.com slash say do. So I love Square. They make paying for stuff at a store really easy. I just use them at Court Street Grocers to buy a sandwich here in Brooklyn, New York.

1:51Their stuff's reliable, easy to use, It just works for both businesses and their customers. Square keeps up so you don't have to slow down. Get everything you need to run and grow your business without any long-term commitments. And why wait? Right now, you can get up to$200 off Square hardware at square.com slash go slash better offline. That's S-Q-U-A-R-E dot com slash G-O slash better offline. Run your business smarter with Square. Get started today. This is Jacob Goldstein from What's Your Problem? When you buy business software from lots of vendors, the costs add up and it gets complicated and confusing.

2:26Odoo solves this. It's a single company that sells a suite of enterprise apps that handles everything from accounting to inventory to sales. Odoo is all connected on a single platform in a simple and affordable way. You can save money without missing out on the features you need. Check out Odoo at odoo.com. That's O-D-O-O dot com.

2:55Hello and welcome to Better Offline. I'm your surly yet lovable host, Ed Zitron.

3:11Today I'm going to kick off by reading something I wrote in March 2024 and talked about in the episode PKI. What if what we're seeing today isn't a glimpse of the future but the new terms of the present? What if artificial intelligence isn't actually capable of doing much more than what we're seeing today, and what if there's no clear timeline when it will be able to do more? What if this entire hype cycle has been built on hot air, goosed by a compliant media, ready and willing to take career embellishers at their word? Reading that back, well, I think I might have been right, and that's kind of what I'm going to get at today.

3:44I don't want to scream mustard, I'm not going to get smug about it, but this is what we're getting into today and in the next episode that will come out on Friday. Now, I'll be linking to some articles, so check the episode notes if you want to read them, but I'm going to get a lot into the spoken word. So I warned you in February that Generative AI has no killer apps and had no way of justifying its valuations. I also warned you in March that Generative AI had already peaked, and I pleaded with the tech industry in April to consider an eventuality where the jump between GPT-4, which is the most current model, well GPT-4-0, to GPT-5 was not significant, in part due to a lack of training data, one of the more obvious things.

4:22I shared more concerns in July that the transformer-based architecture underpinning generative AI, things like chat GPT, was a dead end, and that there were really not many ways we'd progressed past the products we'd already seen back then, in part due to the limits of training data, and the limits of the models that use said training data. In August, I summarized the pale horses of the AI apocalypse, events, many that have now come to past, I'm afraid, that would signify the end, well, being nigh, though it's not quite here yet, and it's not obvious when it will be, but this can't last forever.

4:55But I also added that GPT-5 would not change the game enough to matter, let alone add a new architecture to build future and more capable models or products of any kind. Now, throughout the things I've written and the things I've spoken, I've repeatedly made the point that, separate to any core value proposition, training data drought or unsustainable economics that I've gone over quite a lot, generative AI is a dead end due to the limitations of a probabilistic model that hallucinates. Now, Now, just to be clear with what that means, it's guessing what the next thing might be. And it's quite good at it, but quite good is actually kind of shit.

5:26And hallucinations, of course, are where they authoritatively state things that aren't true. Like when ChatGPT tells you something like, I don't know, there are two R's in strawberry. The hallucination problem is one that is nowhere closer to being solved. You may remember a few months ago when you had every tech executive, you had Tim Cook saying it's Sachin Adela-Sondar Pichai. We'll deal with the hallucination problem. It'll be all right. But I want to be clear, they have not solved it, they have not really mitigated it, and there's no fixing it, at least with the current technology. It's not going anywhere, and it makes all of this stuff kind of a non-starter for many business tasks.

6:03I have, since March, expressed great dismay about the credulousness of the media about this, and their weird acceptance of this inevitable way in which generative AI will change society, despite the fact there's not really a meaningful product that might justify any of this bullshit. This environmentally destructive nonsense, led by a company that burns more than$5 billion a year in big tech firms that are spending$200 billion on data centers for products that people don't want or even potentially use. And you're going to need context for everything I'm saying today. So it's worth going over how these models work and how they're trained.

6:38And I must be clear, the reason I'm repeating myself on so many levels here, that it's just really important for you to know how obvious the problems of generative AI have been since the beginning. It's really important. Let's go over how they work real quick. A transformer-based generative AI model such as GPT, which is the technology behind ChatGPT, generates answers using inference, which means it draws conclusions based off of its training, which requires feeding it masses of training data, mostly text and images scraped from the internet. And both of these processes require you to use high-end GPUs, graphics processing units, and lots of them.

7:16Tens, hundreds of thousands of them. Well, over 100 ,000. I'll get to that next episode. Now the theory was, and might still be, that the more training data and compute you throw at these models, the better they get. And I've hypothesized for a while that we'd have diminishing returns, both from running out of training data and based on the limitations of transformer-based models. And wouldn't you know it, I was bloody right. I'm not going to do many of these, but this one really, this one I'm right on. A few weeks ago, Bloomberg reported that OpenAI, Google, and Anthropic are struggling to build more advanced AI, and that OpenAI's Orion model, otherwise known as GPT-5, did not hit the company's desired performance, and that, and I quote again, Orion is so far not considered to be as big a step up as it was from GPT-3.5, to GPT-4, its current model.

8:03You will be shocked to hear that the reason is that it's become increasingly difficult to find new untapped sources of high-quality human-made training data that can be used to build more advanced AI systems. Something that I said in March, I said it would happen in March, I'm pissed off that people said I was a pessimist. Well, who's a pessimist now? Me, I guess? I don't know. But they also added one other thing, which is that they believe, and I quote, that the AGI bubble is bursting a little bit, which is something I said in July. AGI isn't coming out of this shit. Let's just be honest. And I also want to stop and stare really hard at one particular point, and I quote again from Bloomberg.

8:41These issues challenge the gospel that has taken hold in Silicon Valley in recent years, particularly since OpenAI released ChatGPT two years ago. Much of the tech industry is bet on so-called scaling laws that say more computing power, data, and larger models will inevitably pave the way for greater leaps forward in the power of AI. the only people taking this as gospel have been members of the media unwilling to ask the tough questions and ai founders that don't know what the fuck they're talking about or that intend to mislead you generative ai's products have effectively been trapped in amber for over a year it's been blatantly obvious if you fucking use them i'm pissed off i shouldn't swear so much there have been no meaningful industry defining products out of this because and i quote darren samoglu the economist at mit back in may more powerful models do not unlock new features or really changed the experience.

9:28Nor what you can build with transformer-based models is really a worthwhile product. Or, put another way, a slightly better white elephant is still a white elephant. Despite the billions of dollars burned and thousands of glossy headlines, it's difficult to point to any truly important generative AI product. Even Apple Intelligence, the only thing that Apple really had to add to the latest iPhone. It sucks. It's not useful. I can make a special emoji now. I now get summaries of my texts that are completely or vaguely incorrect or just summarize a giant meaningful paragraph into a blob of a sentence.

10:05It's so stupid. And just as a side question, what the hell is Apple going to put in the next iPhone? I buy one of these every year. I'm a little pig. Oink, oink, oink. But still, I don't even know why I'd upgrade again. The camera is already about as good as it's going to get. Anyway, there are people that use chat GPT, 200 million of them a week, allegedly, losing the company money with every prompt, by the way. But there's little to suggest that there's widespread adoption of actual generative AI software. The information reported in September that between 0.1 % and 1 % of the 440 million of Microsoft's business customers were willing to pay for its AI-powered copilot.

10:44And in late October, Microsoft claimed that it was on pace to make AI a$10 billion a year business, which sounds really good until you think about it for roughly 10 seconds. First of all, Microsoft does not have an AI business unit, which means that this annual $10 billion or$2.5 billion a quarter revenue figure is split across providing cloud compute services on Azure, selling Copilot to dumb people with Microsoft 365 subscriptions, selling GitHub Copilot, and basically anything else with AI on it. Microsoft is cherry-picking a number based on non-specific criteria and claiming it's a big deal when it's actually pretty pathetic considering that Microsoft's capital expenditures will likely hit over$60 billion in 2024, with no sign they're going to slow down.

11:31Note that sticky word revenue, not profit. Those are two very different things. How much is Microsoft spending to make$10 billion a year? OpenAI currently spends$2.35 to make a dollar, and Microsoft CFO Amyhood said that OpenAI would cut into Microsoft's profits in their last earnings call, losing a remarkable$1.5 billion, mainly because of the expected loss from a company that has only ever lost money. Now, a year ago, in October 2023, the Wall Street Journal reported that Microsoft was losing an average of$20 per user per month on GitHub Copiler, a product with over a million users. If this is true, by the way, this suggests losses of at least$200 million a year.

12:12They have 1.8 million users, allegedly. This is based on documents I've reviewed. It's not great either way. $200 million is a lot of money to lose. I would personally like to make$200 million rather than lose it. Don't ask me, though. I don't run Microsoft. Now, Microsoft has still yet to break out exactly how much generative AI is increasing revenue in the specific business units they have. Generally, if a company's doing well at something, they take great pains to make that clear. Instead, Microsoft chose in August to revamp its reporting structure to give better visibility into cloud consumption revenue, which is something you do if you say, anticipate you're going to have your worst day of trading in years after your next earnings, as Microsoft did in October.

12:53It's all very good. It's all going well. Now, I must be clear that every single one of these investments and products has been hyped with the whisper that they would get exponentially better over time, and that eventually the$200 billion in capital expenditures would spit out this remarkable productivity improvement, this crazy new product that would change our lives, fascinating new things that consumers and enterprise would buy in droves and talk about how much they loved. Instead, big tech has found itself peddling increasingly more expensive iterations of near-identical large language models and shitty products attached to them, a direct result of all of them having to use the same training data which they're now running out of.

13:29But if you're running out of stuff and you can't find stuff to buy, I really recommend the following advertisement. I'm sure it will totally gel with my beliefs, the things I'm talking about right now, won't be embarrassing at all.

13:49Parking shouldn't slow you down. ParkWiz gives every driver a shortcut. Book ahead, save up to 50 % and skip the hassle of circling the block. Park smarter, park faster. ParkWiz. Download the ParkWiz app today and save every time you park. So I've shopped with Quince before they were an advertiser and after they became one, and then again before I had to record this ad, I really like them. My green over shirt in particular looks great. I use it like a jacket. It's breathable and comfortable and hangs on my body nicely. I get a lot of compliments. I liked it so much I got it in all the different colors, along with one of their corduroy ones, which I think I pull off, and really that's the only person that matters.

14:25I also really love their linen shirts too. They're comfortable, they're breathable, and they look nice. Get a lot of compliments there too. I have a few of them. love their rust-colored ones as well. And in general, I really like Quince. The shirts fit nicely and the rest of their clothes do too. They ship quickly, they look good, they're high quality, and they partner directly with ethical factories and skip the middleman. So you get top-tier fabrics and craftsmanship at half the price of similar brands. And I'm probably going to buy more from them very, very soon. Keep it classic and cool this fall.

14:53With long-lasting staples from Quince, go to quince.com slash better for free shipping on your order and 365-day returns. That's Q-U-I-N-C-E dot com slash better. Free shipping and 365 day returns. Quince.com slash better. Stop settling for weak sound. It's time to level up your game and bring the boom. Hit the town with the ultra durable LG X Boom portable speaker and enjoy vibrant sound wherever you go. Elevate your listening experience to new heights because let's be real, your music deserves it. The future of sound is now with LG XBoom. And for a limited time, save 25 % at LG.com with code FALL25.

15:36Bring the boom! XBoom. This week on a very special episode of Health Discovered, we're taking a closer look at a condition that affects hundreds of thousands of men each year, prostate cancer. I first found out about my cancer at the age of 45. Anything with cancer, you just think death sentence. In this episode, we'll explore the science behind detection, along with the practical steps men can take to protect their health. Listen to Health Discovered on America's number one podcast network, iHeart. Open your free iHeart app, search Health Discovered, and start listening. Hey guys, it's Erin Andrews from Calm Down with Erin and Carissa.

16:19So when the pressure's on, you remember the people who helped you get this far, whose knowledge guided you and whose care kept you grounded. For me, it was my dad, my mentor, who believed in me when I was just getting started in journalism. A real-life example that showed me when you work with someone who knows a lot and cares even more, you're unstoppable. At Truist, that's exactly how they approach banking. Truist, leaders in banking, unwavering in care. See what knowledge and care can do for you at truest.com slash unstoppable.

16:58And we're back. Now, there's another assumption that people have about these so-called scaling laws. That's been that by simply building bigger data centers with even bigger, more powerful GPUs, the expensive power-hungry graphics processing units that use to both train and run these models, and throwing as much training data at them as possible, they would simply start doing new things. They'd have new capabilities, despite there being little proof that they would do so in any way, shape, or form. Microsoft, Meta, Amazon, and Google have all burned billions on the assumption that doing so would create something, you know, a thing, a good thing, like a human-level artificial general intelligence or a product that made more money than it cost, that people liked.

17:43It's become kind of obvious that that isn't going to happen. As we speak, members of the media who should know better are already desperately trying to prove that this is not a problem. The information, in a similar story to Bloomberg's, attempted to put lipstick on the pig of generative AI, framing the lack of meaningful progress of GPT-5 as fine, because OpenAI can now combine its GPT-5 model with its O1 reasoning model, which is the one that can't count the number of ours in the strawberry, by the way, which will then do something. Something good. Something's going to happen. Like Sam Altman said it could write a lot more very difficult code.

18:22You know, Sam Altman, the career liar who intimated the GPT-5 may function like a virtual brain in May. Like these people are liars. They're liars. They're lying to you. They were lying then. They're lying now. Now, I couldn't possibly leave out Chief Valley Cheerleader Casey Newton, who wrote on Platformer a few weeks ago that diminishing returns in training models may not matter as much as you would guess, with his evidence being that Anthropic, who he also claims has not been prone to hyperbole, do not think the scaling laws are ending. Now, the original scaling laws paper, partly written by Dario Amadeci of Anthropic, important to know.

18:56And to be clear, in a 14 ,000-word op-ed that Casey Newton for no reason wrote two pieces about, Now, Anthropics CEO Dario, he said that, and I quote, AI-accelerated neuroscience is likely to vastly improve treatments for, or even cure, most mental illness, which is the kind of hyperbole that should have you tarred and feathered and put in a jail. I'm not seriously saying you put him in jail, but why are we trusting these people? Why are we listening to them? Why are we treating them as if they're telling the truth, or even that they know what's going on? But let's summarize. The main technology behind the entire, and I say this in quotation marks by the way, artificial intelligence boom is generative AI.

19:38Transform-based models like OpenAI's GPT-4 and soon GPT-5. And said technology has peaked, with diminishing returns from the only ways of making them better, feeding them training data and throwing tons of compute at them, suggesting that we may have, as I said before, reached peak AI. Generative AI is incredibly unprofitable. OpenAI, the biggest player in the industry, is on course to lose more than$5 billion this year. With competitor Anthropic, which also makes its own transformer-based model, Claude, on course to lose more than$2.7 billion this year, they just raised another$4 billion. Every single big tech company has thrown billions of dollars, as much as$75 billion in Amazon's case in 2024 alone, are building the data centers and acquiring the GPUs to populate said data centers specifically so they can train their models and other people's models, or serve customers that would integrate generative AI into their businesses, something that does not appear to be happening at scale.

20:29And these investments could theoretically be used for other products, but these data centers are heavily focused on generative AI. Business Insider reports that Microsoft intends to amass 1.8 million GPUs by the end of this year, costing it tens of billions of dollars. Worse still, many of these companies integrating generative AI do so by connecting to models made by either OpenAI or Anthropic, both of whom are running unprofitable businesses and likely charging nowhere near enough to cover their costs. As I've said before in my article, The Subprime AI Crisis, in the event that these companies start charging what they actually need to, their real costs, I hypothesize that it will multiply the cost of their customers to the point that they can't afford to run their businesses, or at the very least will have to remove or scale back generative AI functionality in their products.

21:14it's just it's such a waste the entire tech industry has become oriented around this dead end technology that requires burning billions and billions of dollars to provide inessential products that cost them more money to serve than anybody ever would pay their big strategy has to been to throw more money at the problem until one of these transformer-based models created something useful despite the fact that every iteration of gpt and other models has been well iterative. And it's weird. You think at some point that goes, shit, do we actually have the ability to build products with this? What are the products?

21:51Maybe we should work out the products first before we throw all the capex at it. But wait, no, over yonder, I couldn't possibly not do this because the other big tech company that also has no ideas, they're doing this. And if I don't do this, my investors are going to be angry at me. And then what will I do? Oh no. Oh no. What could I possibly do if the investors, I don't fucking know. That's your problem. Why waste this much money? It's just, there's never been any proof other than these benchmarks that are really easy to game and also only show just this vague power of these models. It's been obvious that GPT or other models wouldn't become conscious, that they're not going to do more than they do today or three months ago or even a year ago.

22:40Hesitate to give Gary Marcus credit, but in 2023, he was saying this, if not earlier. Many people have as well. And it's just really, really, really, really frustrating. Better Offline isn't even a year old, but when we put out our PKI episode, I got so much flack. I got so much shit for being a hater that I didn't really understand things, that my fly was open in my Instagram picture, that I didn't get it, and that in mere months I would be proven wrong. Well, here we are. How wrong am I now? What happens next exactly? Where do all these hundreds of billions of dollars go? What happens to OpenAI when it collapses?

23:11What does Microsoft do with all of these GPUs? Because you can't just move them into other shit, you know? From what I hear, they don't really have a plan. And that's the scariest thing. Because what happens to a stock market that's dependent on big tech companies for growth when the big tech companies can't work out a way to grow anymore. And in fact, their big path to trying to grow more was to burn a shit ton of money on things that people hate that destroy our environment. I know, I know I'm angry. I know I should calm down. I should. But as I said in the Rot Society, this money could go elsewhere.

23:50More things could be done. It would enter a fallow period of tech. But we don't just have to burn all this money. We don't have to do that. Why not make the products you have already better? Because stapling generative AI on them, I think it makes them worse. But there are more problems ahead. There are problems around the infrastructure. And in the next episode, I'm going to break down these worrying problems. And I'm going to kind of tell you what happens next as best I can. I really appreciate your faith in me and there are many people who also contacted me and said no you're bang on keep going I'm glad they did I'm very grateful for you audience I love you all much like he said in the menu

24:35Thank you for listening to Better Offline The editor and composer of the Better Offline theme song is Matt Ossowski You can check out more of his music and audio projects at matasowski.com, M-A-T-T-O-S-O-W-S-K-I.com. You can email me at ez at betteroffline.com or visit betteroffline.com to find more podcast links and, of course, my newsletter. I also really recommend you go to chat.wheresyoured.at to visit the Discord and go to r slash betteroffline to check out our Reddit. Thank you so much for listening. Better Offline is a production of Cool Zone Media. For more from Cool Zone Media, visit our website, coolzonemedia.com or check us out on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.