In short
Podcast Summary: Better Offline - Episode: How DeepSeek Showed That Silicon Valley Is Washed
Overview In this episode of Better Offline, host Ed Zitron discusses the emergence of DeepSeek, a new generative AI model that poses a significant challenge to established players like OpenAI and Anthropic. The episode explores themes of inefficiency, the stagnation of innovation in Silicon Valley, and the implications of DeepSeek's success for the future of the tech industry.
Key Concepts and Arguments
- DeepSeek's Disruption
- DeepSeek introduced a generative AI model that is vastly more efficient than existing models from established companies.
- Zitron highlights that despite the considerable resources of companies like OpenAI, they were unable to innovate or improve efficiency in AI model training and execution.
- Existential Threat to Major Players
- The rise of DeepSeek represents an existential threat to OpenAI and Anthropic, which struggle with profitability and efficient model development.
- Zitron argues that the inefficiencies in these companies stem from their focus on growth at all costs, leading to unsustainable business practices.
- Hubris and Complacency in Silicon Valley
- Zitron criticizes the complacency and lack of curiosity among major tech players, suggesting that their focus on maintaining the status quo has led to a lack of real innovation.
- He emphasizes that the failure to anticipate competition from DeepSeek reflects a broader issue of stagnation in Silicon Valley.
- Financial Implications
- The financial models of OpenAI and Anthropic are under scrutiny, with Zitron pointing out that both companies are unprofitable.
- OpenAI's need for continuous funding and its inability to pivot in response to competition raises questions about its long-term viability.
- Commoditization of AI
- DeepSeek's efficient approach may lead to the commoditization of large language models, changing the competitive landscape.
- Zitron posits that if DeepSeek's models can be built and run at a fraction of the cost, it undermines the narrative that large investments are necessary for success in AI.
- Critique of Growth-at-All-Costs Mindset
- The episode critiques the pervasive "growth at all costs" mentality in Silicon Valley, arguing that it results in wasteful spending and prioritizes expansion over genuinely useful innovation.
- Zitron calls for a reevaluation of business practices in the tech industry to promote more sustainable and responsible growth.
Key Takeaways
- Innovation vs. Efficiency: The tech industry must focus on developing efficient models, not just large ones, to remain competitive.
- Market Realities: As competition increases, particularly from efficient players like DeepSeek, the business models of legacy companies may become increasingly untenable.
- Industry Reflection: There’s a critical need for reflection within Silicon Valley regarding its direction and priorities, especially in the context of sustainability and innovation.
Pivotal Moments
- Zitron articulates a moment of revelation regarding DeepSeek's success: the realization that traditional players were not even trying to improve efficiency.
- The episode ends on a note of concern for the future of Silicon Valley, indicating that the current trajectory may lead to significant industry upheaval and layoffs.
Additional Resources
- Links:
- [Better Offline Links](https://www.tinyurl.com/betterofflinelinks)
- [Newsletter](https://www.wheresyoured.at/)
- [Reddit Community](https://www.reddit.com/r/BetterOffline/)
- [Discord Server](https://discord.com/invite/QUUQUP9szv)
Conclusion In "How DeepSeek Showed That Silicon Valley Is Washed," Ed Zitron provides a critical examination of the current state of the tech industry, emphasizing the need for innovation beyond size and expenditure. The emergence of more efficient models like those from DeepSeek may herald a new era in AI, challenging established giants and prompting a necessary shift in industry practices.
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Transcript
Automatic transcript. May contain errors.0:00This is an iHeart Podcast.
0:30Thomson Reuters and specialized bikes have since they upgraded to the next generation of the cloud. Oracle Cloud Infrastructure. OCI is the blazing fast platform for your infrastructure, database, application development, and AI needs, where you can run any workload in a high availability, consistently high performance environment, and spend less than you would with other clouds. How is it faster? OCI's block storage gives you more operations per second. Cheaper? Better? OCI costs up to 50 % less for computing, 70 % less for storage, and 80 % less for networking. Better? In test after test, OCI customers report lower latency and higher bandwidth versus other clouds.
1:12This is the cloud built for AI and all your biggest workloads. Right now, with zero commitment, try OCI for free. Head to oracle.com slash strategic. That's oracle.com slash strategic. This week on a very special episode of Health Discovered, we're taking a closer look at a condition that affects hundreds of thousands of men each year, prostate cancer. I first found out about my cancer at the age of 45. Anything with cancer, you just think death sentence. In this episode, we'll explore the science behind detection along with the practical steps men can take to protect their health. Listen to Health Discovered on America's number one podcast network, iHeart.
1:56Open your free iHeart app, search Health Discovered, and start listening. Run a business and not thinking about podcasting? Think again. More Americans listen to podcasts, then add supported streaming music from Spotify and Pandora. And as the number one podcaster, iHeart's twice as large as the next two combined. Learn how podcasting can help your business. Call 844-844-IHEART. Call Zone Media. Chosen by God, perfected by science, I'm Ed Zitron, this is Better Offline.
2:35Better Offline. And as I've written about many, many, many, many times, and argued on this very podcast just as often, the large language models run by companies like OpenAI, Anthropic, Google and Meta are unprofitable and unsustainable, unsustainable, and the transformer-based architecture they run on has peaked. They're running out of training data, and the actual capabilities of these models were peaking as far back as March 2024. Nevertheless, I'd assumed, incorrectly by the way, that there would be no way to make them more efficient, because I had assumed, also incorrectly, that the hyperscalers, along with OpenAI and Anthropic, would be constantly looking for ways to bring down the ruinous cost of their services.
3:13After all, OpenAI lost$5 billion last year, and that's after$3.7 billion in revenue too, and Anthropic lost just under$3 billion in 2024. And in the last episode I told you a little bit about DeepSeek, by the way. In this one we're going to get into, well, how fucked things might actually be. But what I didn't wager was that potentially nobody was actually trying to make these models more efficient. My mistake was, if you can believe this, being too generous to the AI companies, assuming that they didn't pursue efficiency because they couldn't, and not because they couldn't be bothered. But then, as I just hinted at, a little-known Chinese company released a product that was broadly equivalent to OpenAI's latest reasoning models, but cost a fraction of the cost to train and run.
3:58And now the conventional understanding of how generative AI should work has been fundamentally upended. You see, the pre-deep-seat status quo was one where several truths, and I say that in the loosest sense of the word, allowed the party to keep going. So the first one is that these models were incredibly expensive to train. GPT-40 cost$100 million in the middle of 2024, and future models, according to Dario Amadei of Anthropic, might cost as much as$1 billion or more to train. And training future models, by the way, as a result of this, would necessitate spending billions of dollars on both data centers and the GPUs necessary to keep training these bigger, huger models.
4:35Now, another thing was these models had to be large, because making them large, pumping them full of training data, and throwing masses of compute at them would unlock new features, such as an AI that helps us accomplish much more than we ever could without AI, which is a Sam Altman quote. And in the words of Sam again, you'd be able to get a personal AI team full of virtual experts in different areas working together to create almost anything we can imagine. I don't know, mate. You ever try creating a functional fucking business? Dipshit. Anyway, here's another one. These models were incredibly expensive to run.
5:09They has to be this way. But it was all worth it, because making these models powerful was way more important than making them efficient, because once the price of silicon comes down, and this is a refrain I've heard from multiple different people as a defense of the costs of generative AI, we would then have these powerful models that were cheaper somehow, because of silicon. Now you may think, Ed, that sounds like a, not a real argument, that just sounds like something someone said once. And it is, it is something someone said once. Anyone who knows anything about chips know how hard it is to make a new chip.
5:41Remember one of the CES episodes when I asked Max Cherney about this? You should go back and listen to him. Anyway, another thing, another part of this, was that as a result of this need to make bigger, huger, even bigger models, the most powerful ones, these big, beautiful models, we love them, we look at the big, beautiful models, we would, of course, need to keep buying bigger, more powerful GPUs, which would continue the American excellence of burning a bunch of money on nothing. And by following this roadmap, everybody wins. The hyperscalers get the justification they needed to create more sprawling data centers and spend massive amounts of money.
6:17And OpenAI and their ilk get to continue building powerful models. And also NVIDIA continues to make money selling GPUs. Remember I've said in the past that things were kind of a death cult? This is what this is. It's a capitalist death cult. It runs on plagiarism and hubris. and the assumption that being, that at some point all of this would turn into something meaningful. Now, I've argued for a while that the latter part of the plan was insane, that there was no profitability for these large language models, as I believe there simply wasn't a way to make these models more efficient. In a way, I was right.
6:48The current models developed by both the hyperscalers, so Gemini from Google, Llama from Meta and so on and so forth, and the multi-billion dollar startups, if you can even fucking call them that, OpenAI and Anthropic, they're horribly inefficient. And I just made the mistake of assuming that they tried to make them more efficient, and they couldn't. But what we're witnessing right now isn't some sort of weird China situation. This isn't China being Chinese and doing scary Chinese things to us. No, what we're witnessing is the American tech industry's greatest active hubris. It's a monument to the barely conscious stewards of so-called innovation, who are incapable of breaking the kayfabe of the fake competition, where everybody makes the same products, charges about the same amount of money, and mostly innovates in the same direction.
7:37Somehow nobody, not Google, not Microsoft, not OpenAI, not Meta, not Amazon, not Oracle, thought to try or was capable of creating something like DeepSeek. Which doesn't mean that DeepSeek's team is particularly remarkable or found anything super new, but that for all the talent, trillions of dollars of market capitalization and supposed expertise in American tech oligarchs, not one bright spark thought to try things that DeepSea could try, which appeared to be, what if we didn't use as much memory and what if we tried synthetic data? And because the cost of model development and inference was so astronomical in the case of American models, they never assumed that anyone would try to usurp their position.
8:17This is especially bad considering that China's focus on AI as a strategic part of its industrial priority was really no secret, even if the ways it supported domestic companies kind of is. In the same way that the automotive industry was blindsided by China's EV manufacturers, the same is happening with AI. Fat, happy and lazy, and most of all oblivious, America's most powerful tech company sacked back and built bigger, messier models powered by sprawling data centers at billions of dollars of GPUs from NVIDIA, a bacchanalia of spending that strains our energy grid and depletes our fucking water reserves, without, it appears, much consideration of whether an alternative was possible.
8:55I refuse to believe that none of these companies could have done what DeepSeek has done, which means that they either chose not to, or they were so utterly myopic, so excited to burn so much money on so many parts of burning the earth, boiling lakes, and stealing from people in pursuit of further growth that they didn't think to try. This isn't about China. It's so much fucking easier if we let it be about China. No, no, no, no. It's about how the American tech industry is incurious, lazy, entitled, directionless, and irresponsible. OpenAI and Anthropic are the antithesis of Silicon Valley. They're incumbents, public companies wearing startup suits, unwilling to take on real challenges, more focused on optics and marketing than they are on solving actual fucking problems, even the problems that they themselves created with their large language models.
9:44By making this about China, we ignore the root of the problem, that the American tech industry is no longer interested in making good software that actually helps people. DeepSeq shouldn't be scary to Silicon Valley, because Silicon Valley should have come up with this first. It uses less memory, fewer resources, and uses several kind of quirky workarounds to adapt to the limited compute resources available, all things that you'd previously associate with Silicon Valley. Except now Silicon Valley's only interest, like the rest of the American tech industry, is the rot economy. It only cares about growing, growing at all costs, even if said costs were really things you could mitigate, or if the costs themselves were self-defeating.
10:25To be clear, if the alternative is that all of these companies simply did not come up with this idea, that in and of itself is a damning indictment of the Valley. Was nobody thinking of this stuff? If they were, why didn't Sam Altman or Dario Amadei or Satya Nadella or anyone else put serious resources into efficiency? Was it because there was no reason to? Was it because there was, if we're honest, no real competition between any of these companies? Did anybody try anything other than throwing as much computing training data at the model as possible? It's all just so cynical and antithetical to innovation itself.
10:59Surely if any of this shit mattered, if generative AI truly was valid and viable in the eyes of these companies, they would have actively worked to do something like DeepSeek has done. Don't get me wrong. It appears DeepSeek employed all sorts of weird tricks to make this work, including taking advantage of distinct parts of both CPUs and GPUs to create something called a digital processing unit, essentially redefining how data is communicated within the servers, running training and inference. And just as a reminder, inference is the thing where when you type something in, it infers the meaning.
11:30Just could have specified that earlier. DeepSeek had to do things that a company with unrestrained access to capital and equipment wouldn't have to do, and it often used impractical and quirky methods to do so. Nevertheless, OpenAI and Anthropic both have enough money and hiring power to have tried and succeeded in creating a model this efficient and capable of running on older GPUs. Except what they wanted, what they actually wanted, was more goddamn growth and the chance to build even bigger data centers with even more compute that they would own. OpenAI is as much a lazy, cumbersome incumbent as Google or Microsoft, and it's about as innovative, too.
12:07The launch of its operator agent was a joke. A barely functional product that's allegedly meant to control your computer and take distinct actions, like ordering stuff off of Instacart, you know, things you could do with your hands. But just to be clear, it doesn't work. You'll never guess who was really into it, though. His name is Casey Newton. He writes a blog called Platformer. And he's a man so gratingly credulous that it makes me want to fucking scream. And of course, he wrote that Operator, when he used it, was a compelling demonstration that represented an extraordinary technological achievement that also somehow was significantly slower, more frustrating, and more expensive than simply doing any of these tasks yourself.
12:47Casey, of course, not to worry, had some extra thoughts about DeepSeek, that there were reasons to be worried, but that American AI labs were still in the lead, saying that DeepSeq was only optimizing technology that OpenAI and others had invented first, before saying that DeepSeq was only last week that OpenAI made available to ProPlan users a computer that can use itself. This statement is bordering on factually incorrect. It is fucking insane that Casey is still doing this. I do not want to, I don't know what to do with this guy. This guy, just, that's a fucking lie. This, the computer can't use itself, this shit can't, just to explain what Operator is, You're meant to type in something like, hey, order me some milk.
13:26Order me some milk off of Instacart. And when Casey tried this, it tried to find milk in Des Moines, Iowa. Just fucking insane. Just, this is how these companies have got big. It's people like Casey. It's people like Casey who are just like, anything they showed are like, God damn, that's the most impressive thing I've seen in my life. It's a fucking farce. But let's be frank. These companies aren't building shit. OpenAI and Anthropic are both limply throwing around the idea that agents are possible in an attempt to raise more money to burn. And after the launch of DeepSeek, I have to wonder what any investor thinks they're investing in, other than certain ones I'll get into in a bit.
14:02And to be clear, an agent is meant to be this autonomous thing, which you say, hey, go and do this action, go and sell things for me, and go and email people for me. They don't really work. There are some that kind of do that are really expensive, but large language models are not built for this kind of thing. But let's be honest. DeepSeq, and as I said in the last episode, they've built a more efficient reasoning model, so like OpenAI's 01. And you'd think, well, okay, couldn't OpenAI simply add on DeepSeq to its models? Not really. First of all, with the way these models work, you can't just plug it in.
14:39It's just not how it works. They could train a new model using DeepSeq's techniques, but the optics of that aren't brilliant it would be a concession it'd be in the middle that open ai slipped and needs to catch up and not to its main rival pretend rival i mean anthropic or to like another big tech firm but to an outgrowth of a hedge fund in china a company that few had heard of before december and like really not that many people had heard before january 25th it's very embarrassing. And this in turn I think will make any serious investor think twice about writing the company a blank check. They're going to have to dip into some very bothersome pockets.
15:20And as I've said ad nauseum, this is potentially fatal as OpenAI needs to continually raise money, more money than any startup has ever raised in the history of anything, and it really doesn't have a path to breaking even, even if they copy what DeepSeek did. Because we still right now, though DeepSeek is 30 times cheaper than O1, we don't know if that's profitable. We don't know. We haven't found out. And if OpenAI wants to do its own cheaper, more efficient model, it's likely to have to create it from scratch, like I said. And while it could do distillation to make it kind of more like OpenAI using their own models, by the way, DeepSeek taught itself using OpenAI's outputs, like I mentioned in the last episode, it's kind of what DeepSeek already did, it already has been fed OpenAI bullshit.
16:05Even with OpenAI's much larger team and more powerful hardware, it's hard to see how creating a smaller, more efficient, and almost as powerful version of O1 benefits them in any way, because said version has, well, already been beaten to market by DeepSeek, and thanks to DeepSeek, we'll almost certainly have a great deal of competition for a product that to this day lacks any killer apps anyway.
16:32Parking shouldn't slow you down. ParkWiz gives every driver a shortcut. Book ahead, save up to 50 % and skip the hassle of circling the block. Park smarter, park faster. ParkWiz. Download the ParkWiz app today and save every time you park. This week on a very special episode of Health Discovered, we're taking a closer look at a condition that affects hundreds of thousands of men each year, prostate cancer. I first found out about my cancer at the age of 45. Anything with cancer, you just think death sentence. In this episode, we'll explore the science behind detection, along with the practical steps men can take to protect their health.
17:14Listen to Health Discovered on America's number one podcast network, iHeart. Open your free iHeart app, search Health Discovered, and start listening. So I've shopped with quints before they were an advertiser and after they became one, and then again before I had to record this ad. I really like them. My green over shirt in particular looks great. I use it like a jacket. It's breathable and comfortable and hangs on my body nicely. I get a lot of compliments. I liked it so much. I got it in all the different colors, along with one of their corduroy ones, which I think I pull off. And really, that's the only person that matters.
17:44I also really love their linen shirts, too. They're comfortable, they're breathable, and they look nice. Get a lot of compliments there, too. I have a few of them. Love their rust colored ones as well. And in general, I really like quints. The shirts fit nicely and the rest of their clothes do, too. They ship quickly. They look good. They're high quality and they partner directly with ethical factories and skip the middleman. So you get top tier fabrics and craftsmanship at half the price of similar brands. And I'm probably going to buy more from them very, very soon. Keep it classic and cool this fall with long lasting staples from Quince.
18:14Go to quince.com slash better for free shipping on your order and 365 day returns. That's Q-U-I-N-C-E dot com slash better. Free shipping and 365 day returns. Quince.com slash better. Run a business and not thinking about podcasting? Think again. More Americans listen to podcasts than ad-supported streaming music from Spotify and Pandora. And as the number one podcaster, iHeart's twice as large as the next two combined. So whatever your customers listen to, they'll hear your message. Plus, only iHeart can extend your message to audiences across broadcast radio. Think podcasting can help your business?
18:51Think iHeart. Streaming, radio, and podcasting. Call 844-844-iHeart to get started. That's 844-844-IHEART.
19:04It's just, it's very frustrating to me. It's very frustrating to me. It drives me a little insane. Reading all this stuff makes me feel crazy. I think you hear it in my voice. You hear the sanity stripping away, but I'm here to podcast and don't worry. But seriously, though, anyone can build on top of what DeepSeek has already built. Where is OpenAI's moat exactly? and where's anthropics what are the things that make these companies worth 60 billion or 150 billion or oh my god as we'll discuss in a bit 340 billion dollars what is the technology they own or the talent they have that justifies these valuations because it's kind of hard to argue that their models are particularly valuable anymore celebrity celebrity cult of personality altman samuel he's he's an artful bullshitter and he's built a career out of being in the right places at the right times, having the right connections, and knowing exactly what to say, especially to credulous tech media ponces without the spine or inclination to push back on his more stupid claims.
20:05And already, Altman has tried to shrug off DeepSeek's rise, admitting that while DeepSeek's R1 model is impressive, particularly when it comes to its efficiency, OpenAI will obviously deliver much better models, and also it's legit invigorating for OpenAI to have a new competitor. Yeah, mate, sure. I'll bet you're loving this. Altman ended his tweet where that came from with, look forward to bringing you all AGI and beyond, something which I add has always been close on the horizon in Altman's world, but it's never really materialized and the timeline keeps moving and there's no actual proof they can do it.
20:40AGI is not fucking happening. And if it's possible in any way, it's not coming out of probabilistic models. I'm fucking sick of this. And OpenAI can't even lean on its relationship with Microsoft, which on Wednesday, January 29th started offering DeepSeq's models through its own cloud services. OpenAI hasn't got shit. DeepSeq has commoditized the large language model, publishing both the source code and the guide to building your own. Whether or not someone chooses to pay DeepSeq is largely irrelevant. Someone else will take what they have created and build their own, or people will start running their own DeepSeq instances, renting GPUs from one of the various cloud computing firms.
21:17They don't give a shit. They'll take the money. And while NVIDIA will always find other ways to make money, Jensen Huang is amazing at this. It's going to be a hard sell for any hyperscaler to justify spending billions more on GPUs to markets that now know that near-identical models can be built for a fraction of the cost with older hardware. Why do you need Blackwell, which is the latest NVIDIA GPU? The narrative of this is the only way to build powerful models doesn't really hold water anymore. And the only other selling point is that what if China does something? Well, the Chinese did something, and they've now proven that they can not only compete with American AI companies, but doing so is possible in an efficient way that can effectively crash the market.
21:58While there's been a recovery, this is still very worrying. I also want to address something real quick. Few people on Twitter have been suggesting that talking about DeepSeek positively in any way is some sort of Chinese op. If you believe this, you're a fucking moron. I really must be clear. Take your weird xenophobia and go eat your own shit. I don't fucking care anymore. Yes, there are problems with China. Yes, China does something to America. This is an open source thing. This is an open source thing. And you can remove China from the equation because it's open source. Someone else is going to use this.
22:32If your only defense is the sneaky Chinese are doing something, go to therapy and talk to the therapist about you being paranoid or racist because it's one of them. I should also be clear, concerns about China are very realistic. The Chinese government has tried to interfere with America. It's happened many times. Even if China is funding deep-seeks models, the fact that they are open-sourced means that anyone can run them and anyone can build their own. They can look under the hood. We don't have the training data, but that is it. You cannot win on a xenophobic argument here. You can have realistic concerns about another foreign power.
23:09I'm not saying not to. But what I am saying is that you have to look at this realistically, and you have to take this seriously, and dismissing this as Chinese magic is stupid. It's very goddamn stupid. But like I said earlier, it also isn't clear if these models are actually going to be profitable. It's unclear who funds DeepSeek, like I just said, and whether its current pricing is actually sustainable. But they're likely going to be a damn sight more profitable than anything OpenAI is currently selling. After all, OpenAI loses money on every single transaction, even their$200 a month ChatGPT Pro subscription.
23:47And if OpenAI cuts its prices to compete with DeepSeek, its losses are only going to deepen. And as I've said again and again, this is all so deeply cynical, because it's obvious that none of this was ever about the proliferation of generative AI, or making sure that generative AI was accessible. Putting aside my very obvious personal beliefs for a second, it's fairly obvious why these companies, the big hyperscalers, and OpenAI and Anthropic wouldn't want to create something like DeepSeq, because creating an open source model that uses less resources means that OpenAI, Anthropic, and their associated hyperscaler Findom clients would lose their soft monopoly on large language models.
24:25Now, what does that mean? I'll explain. Before DeepSeq, to make a competitive large language model, like GPT-4O, as in one that you can actually commercialize, required exceedingly large amounts of capital, and to make larger ones effectively required you to kiss the ring or the arse of Microsoft, Google, or Amazon. While it isn't clear what it cost to train OpenAI's O1 reasoning model, we know that GPT-4O cost in excess of$100 million, and O1 as a more complex model would likely cost even more. We also know that OpenAI's training and inference costs in 2024 were around$7 billion, meaning that either refining current models or building new ones is quite costly.
Read the full transcript
25:04The mythology of both OpenAI and Anthropic is that these large amounts of capital weren't just necessary, but the only way to do this. While these companies ostensibly compete, neither of them seemed concerned about doing so as actual businesses that made products that were, say, cheaper and more efficient to run. You know, they made more money than they cost. Because in doing so, these companies would break the illusion that the only way to create powerful artificial intelligence was to hand billions of dollars to one of two companies, and build giant data centers to build even larger language models.
25:36This is AI's rot economy. Two lumbering companies claiming their startups, creating a narrative that the only way to build the future is to keep growing, to build more data centers, to build larger language models, to consume more training data with each infusion of capital, GPU purchase and data center build-out, creating an infrastructural moat that always leads back to one of a few tech hyperscalers. Open AI and Anthropic need the narrative to say, buy more GPUs and build more data centers, because in doing so they create the conditions of that infrastructural monopoly. Because the terms, forget about building software that does stuff for a second, were implicitly that smaller players cannot enter the market because the market is defined as large language models that cost hundreds of millions of dollars and require access to more compute than any startup could ever reasonably access without the infrastructure that a public tech company delivers.
26:27Remember, neither Anthropic nor OpenAI has ever marketed themselves based on the products they actually build. Large language models are, in and of themselves, fairly bland software products, which is why we've yet to see any killer apps. This isn't a particularly exciting pitch to investors or the public markets, because there's no product innovation or business model to point to. And if they'd actually tried to productize it and turn it into a business, it's quite obvious at this point that there really isn't a multi-trillion dollar industry for generative AI. Look at Microsoft, and their attempts to strong-arm Copilot into Microsoft 365, both personally and commercially.
27:04Nobody said, wow, this is great, when they demanded you use Copilot in Word. Lots of people, however, asked, why am I being charged significantly more for a product that I don't want or care about? OpenAI only makes 27 % of its revenue from selling access to its models, so allowing people to use their models to build products. Around a billion dollars of annual recurring revenue, by the way, with the rest of their money coming in about$2.7 billion last year, coming from subscriptions to ChatGPT. If you ignore the hype, OpenAI and Anthropic are actually deeply boring software businesses with unprofitable, unreliable products prone to hallucinations, and their new products, such as OpenAI's Sora, cost way too much money to both run and train to get the results that, well, they suck.
27:46They're not good. Even OpenAI's push into the federal government with the release of ChatGPTGov is unlikely to reverse its dismal fortunes. Seriously, think about it. I'm sure some of you are going to say, well Trump will just give them money these motherfuckers need way more money than Trump is going to give them and why would Trump bet on a loser why would Trump be like oh yeah I'm going to give more money to this company that does the same thing he doesn't understand any of this shit and he probably just looks at Sam Allton and goes nah that's a kind of new money I don't like but to make this more than a deeply boring software business OpenAI and Anthropic needed larger models.
28:22And they needed them to get larger, generally in perpetuity, and for the story to always be that there was only one way to build the future, and that the future cost hundreds of billions of dollars, and that only the biggest geniuses who all happened to work in the same two or three places were capable of doing it. Post-DeepSeek, there really isn't a compelling argument for investing hundreds of billions of dollars of capex in data centers, or buying new GPUs or even pursuing large language models as they currently stand. It's possible, and DeepSeek through its research papers explained in detail how, to build models competitive with both of OpenAI's leading models, and that's assuming you don't simply build on top of the ones that DeepSeek released.
29:02It also seriously calls into question what it is you're paying OpenAI for in its various subscriptions, most of which, other than the$200 a month Pro subscription, have hard limits on how much you can use their most advanced reasoning models. One thing we do know, though, is that OpenAI and Anthropic will now have to either drop the price of accessing their models, and potentially even the cost of their subscriptions, too. I'd argue that despite the significant price difference between O1 and DeepSeq's R1 reasoning model, the real danger to both OpenAI and Anthropic is DeepSeq V3, which competes with GPT-40, which is their general purpose model, by the way.
29:39And as I'm recording this episode, by the way, news broke that Alibaba, a behemoth out of China in its own right, has created its own model that outperforms DeepSeek. I'm yet to fully dive into it, but if it's true, it's only going to pile on the price pressure. Though I kind of wonder what they could possibly do. Is it going to be cheaper? Because if it's more powerful, that's just like, I don't know, it doesn't really change shit. Anyway though, DeepSeek's narrative shift isn't just about commoditizing LLMs at large, but commoditizing the most expensive ones, run by two monopolists backed by three other monopolists.
30:13I mean, the magic's died. There's no halo around Sam Altman or Dario Amadei's head anymore, as their only real argument was we're the only ones that can do this, something that nobody should have believed in the first place. Up until this point, people believed that the reason these models were so expensive was because they had to be, and that we had to build more data centers and buy more silicon because that's just how things worked. They believed that reasoning models were the future, even if members of the media didn't really seem to understand what reasoning models did or why they mattered, and that as a result they had to be expensive, because OpenAI and their ilk were just so fucking smart, even if it wasn't obvious what reasoning meant or what it allowed you to do or what the products were.
30:55It's just very annoying. And now we're going to find out, by the way, because reasoning is now commoditized, along with large language models in general. Funnily enough, the way that DeepSeq may have been trained using, at least in part, synthetic data, also pushes against the paradigm that these companies even need to use other people's training data, though their argument, of course, will be that they need more training data, always. We also don't know the environmental effects, by the way, with DeepSeq, because even if it's cheaper, these models still require those energy-guzzling GPUs to run, and they're running at full tilt.
31:26In any case, if I had to guess, the result will be that the markets are going to be far less tolerant of generative AI and the idea that generative AI is the future.
31:45Parking shouldn't slow you down. ParkWiz gives every driver a shortcut. Book ahead, save up to 50 % and skip the hassle of circling the block. Park smarter, park faster. ParkWiz. Download the ParkWiz app today and save every time you park. This week on a very special episode of Health Discovered, we're taking a closer look at a condition that affects hundreds of thousands of men each year, prostate cancer. I first found out about my cancer on my birthday at the age of 45. Found out my cancer has spread to my pelvic bone. And from there, life just changed. About one in eight men will be diagnosed with prostate cancer during their lifetime, and the risk increases with age.
32:32Anything with cancer, you just think death sentence. And the only thing I could think about was, who's going to take care of my family? You have to go out there and build your support system. You got to build your team. In this episode, we'll explore the science behind detection, along with the practical steps men can take to protect their health. Listen to Health Discovered on America's number one podcast network, iHeart. Open your free iHeart app, search Health Discovered, and start listening.
33:05Run a business and not thinking about podcasting? Think again. More Americans listen to podcasts than ad-supported streaming music from Spotify and Pandora. And as the number one podcaster, iHeart's twice as large as the next two combined. So whatever your customers listen to, they'll hear your message. Plus, only iHeart can extend your message to audiences across broadcast radio. Think podcasting can help your business? Think iHeart. Streaming, radio, and podcasting. Let us show you at iHeartAdvertising.com. That's iHeartAdvertising.com. Jan Marselech was a model of German corporate success. It seemed so damn simple for him.
33:45Also, it turned out, a fraudster. Where does the money come from? That was something that I always was questioning myself. But what if I told you that was the least interesting thing about him? His secret office was less than 500 meters down the road. I often ask myself now, did I know the true Jan at all? Certain things in my life since then have gone terribly wrong. I don't know if they followed me to my home. It looks like the ingredients of a really grand spy story. Because this ties together the Cold War with the new one.
34:23Listen to Hot Money, Agent of Chaos on the iHeartRadio app, Apple Podcasts or wherever you get your podcasts
34:39OpenAI and Anthropic no longer really have moats Unless, well there is another idea what if there was a huge fucking idiot with a lot of money? How about billionaire dipshit Masayoshi-san, the CEO of SoftBank, a multinational investment firm that's rumoured to be investing anywhere from$15 to$25 billion in OpenAI in a round that values the company at an astonishing$340 billion as part of a round of up to$40 billion. Now you may think, damn, this is a sign that OpenAI is going to make it, but I must remind you how bad SoftBank is at investing. They put$16 billion into famously awful real estate company WeWork and managed to lose, I think,$800 million on the DoorDash IPO and$1.8 billion on their investment in Uber, and in both cases did so because they were desperate to bandwagon on to supposedly surefire bets either just before they crashed or way after an investment made sense.
35:38According to the Wall Street Journal, SoftBank would lead this insane $40 billion round in the company and would, and I quote, help assemble investors for the rest of the round. In doing so, SoftBank would also become OpenAI's largest investor, replacing Microsoft in, yes, SoftBank was the largest investor in WeWork before it went tits up. It's also important to remember that OpenAI has pledged to put$18-19 billion to fund the Stargate Data Center project, along with, you guessed it, SoftBank will be committing the same amount. This is on some level SoftBank handing money to itself to invest in data centers to prop up an industry that's dying.
36:15Now, this is a developing story, but it's hard to imagine any serious contribution from any respectable investor at this point. My money is on a few VC firms desperately scraping at the bottom of the barrel. Quarter of a billion here, quarter of a billion there, maybe NVIDIA chucks in some. And on the subject of barrels of stuff, I also expect money from the Kingdom of Saudi Arabia and its associated venture arms. You're going to see a few, maybe Andreessen Horowitz gets involved, they don't think much. It's just very fucking silly, and I don't know how this works out. OpenAI burns money, and even if they somehow make more efficient models, the actual total addressable market of generative AI is actually pretty small.
36:55Microsoft said in their recent earnings they made$12 to$13 billion of ARR, on ai just to be clear that's not profit and that's not a business unit there's no ai business unit which means that that is just spread across delivering cloud compute for ai co-pilot on microsoft 365 products which by the way no one likes they're having trouble selling and other associated co-pilot products they sell i don't think and like 12 to 13 billion dollars across four quarters that's not actually good at all it's just very silly all of this is so silly and when to think about it too hard to feel a little crazy open ai makes 3.7 billion dollars in revenue and they do so as i mentioned primarily from chat gpt subscriptions even if that somehow and it won't by the way turns into 3.7 billion dollars of profit or even 10 billion dollars of profit a year that's less than the profits in a single quarter of any given hyperscaler it would be respectable sure but a$340 billion valuation, I guess, makes sense if it was profit.
37:58It doesn't make sense if it's not profitable, though. It also isn't obvious how OpenAI would actually provide any liquidity to investors, by which I mean allow them to sell their stock, beyond selling shares of people that work at OpenAI, like people who work there who have been given stock runs, selling that to another investor, a really dumb guy maybe. And as an aside, SoftBank bought$1.5 billion of stock from OpenAI employees in the tender at the end of November 2024, just a note for you. They could also take the company public, but with the unit economics of this fucking company, which boiled down by the way to, our products lose billions of dollars and are extremely commoditized, I'm not really sure what the plan is here.
38:40The fundamental problems that OpenAI has are not solved by throwing more money at the problem. This hasn't worked before and it won't work this time. They're burning cash. And in SoftBank's case, it isn't obvious what it is they're getting from OpenAI. Is it the chance to continue an industry-wide con? The chance to participate in a capitalist death cult? I don't know. Maybe it's the chance to burn money at a faster rate than WeWork ever could dream of. Will this be the time that Microsoft, Amazon, and Google just drop OpenAI and Anthropic and make their own models based on DeepSeq's work? What incentive is there for the hyperscalers to keep funding OpenAI and Anthropic?
39:15They hold all the cards, the GPUs, the infrastructure, and in the case of Microsoft, non-revocable licenses that permit them unfettered use and access to OpenAI's tech. And there's little stopping the hyperscalers from building their own models and just dumping them entirely. In fact, Microsoft might actually be a little glad to see SoftBank become the biggest investor and pick up the tab for OpenAI's expenses. I can imagine Satya Nadella texting Sam Orton being like, no, don't take that money, lol. Don't do it. Oh, I'd hate that. I'd hate it if this was someone else's problem. And the Stargate thing, by the way, is an attempt to, the up to 500 billion thing, it's just bollocks, whatever.
39:54It's an attempt to remove themselves from Microsoft. And Microsoft actually allowed OpenAI to alter their deal so that they could get cloud compute from others. Now, at the time, people were like, yeah, man, this is a good thing. This show's OpenAI will be independent. No, it doesn't. It just means that they're going to be under Masayoshi-san now, the funniest, dumbest man in investing. I love Masayoshi-san. I think it's nice that we have an insane guy who isn't instantly murderous in our lives. Anyway, anyway, though. As I've said before, I believe we're at peak AI. And now that Generative AI has been commoditized, the only thing that OpenAI and Anthropic have left other than a pile of cash is their ability to innovate.
40:37And I don't think they're capable of doing so. And because we sit in the ruins of Silicon Valley with our biggest startups all doing the same thing in the least efficient way possible, living at the beck and call of public companies with multi-trillion dollar market caps, everyone is trying to do the same thing in the same way based on the fantastical marketing nonsense of a succession of directionless rich guys that all want to create America's next-top monopoly. It's time to wake up and accept that there was never any kind of AI arms race, and that the only reason that hyperscalers built so many data centers and bought so many GPUs is because they're run by people that don't experience real problems and thus don't know what problems real people face.
41:19Generative AI does not solve any trillion-dollar problems, nor does it create outcomes that are profitable for any particular business. DeepSeek's models are cheaper to run, but the real magic trick they pulled is that they showed how utterly replaceable a company like OpenAI, and by extension any LLM company, really is. There really isn't anything special about any of these companies. They have no moat, their infrastructural advantages moot, and their hordes of talent are relatively irrelevant. What DeepSeek has proven isn't just technological, it's philosophical. It shows that the scrappy spirit of Silicon Valley builders is dead, replaced by a series of different management consultants that lead teams of engineers to do things based on vibes.
41:59You may ask if all of this means generative AI suddenly gets more prevalent. After all, Sachin Adela of Microsoft quoted Yvonne's paradox, which posits that when resources are made more efficient, their use increases. Sadly, I hypothesize that something else happens. Right now, I do not believe that there are companies that are stymied by the pricing that OpenAI and their ilk offer, nor do I think there are many companies or use cases that don't exist because large language models are too expensive. AI companies took up a third of all venture capital funding last year, and on top of that, it's fairly easy to try reasoning models like O1 and make a proof of concept without having to make an entire operational company.
42:37Sheer OpenAI barely has one. I don't think anyone has been on the sidelines of generative AI due to costs. And remember, few seem to be able to come up with great use case for O1 or other reasoning models anyway. And DeepSeq's models, while cheaper, don't have any new functionality. As a result, I don't really see anything changing beyond the eventual collapse of the API market, which is the way you plug these models into things for companies like Anthropic and OpenAI. Large language models and reasoning models, they're niche. The only reason that ChatGPT became such a big deal is because the tech industry has no other growth ideas, and despite the entire industry and public markets screaming about it, I can't think of any mass market product that really matters.
43:19Even if DeepSeek doesn't land the fatal blow, it could set the foundations for another company to drag OpenAI's carcass out behind the barn and hit it with a big stick. One way in which this entire farce could fall is if nasty Mark Zuckerberg decides he wants to simply destroy the entire market for LLMs. Meta has already formed four separate war rooms to break down how DeepSeek did it, and apparently, to quote the information, in pursuing LLAMA, which is their large language model, CEO Mark Zuckerberg wants to commoditize AI models so that the applications that use such models, including Meta's, generate more money than the sales of the AI models themselves.
43:56That could hurt Meta's AI rivals such as OpenAI and Anthropic, which are on pace to generate billions of dollars in revenue from such sales. and lose billions. Fucking hell, I love the information, but can you add the most important bit? But I could absolutely see Meta releasing its own version of DeepSeq's models. They've got the GPUs, and Mark Zuckerberg can never be fired, meaning that if he simply decided to throw billions of dollars into specifically creating his own deep discounted LMs to wipe out OpenAI, he absolutely could. After all, a few weeks back, Mark Zuckerberg said that Meta would spend between 60 and 65 billion dollars in capital expenditures in 2025, and this was before the DeepSeek situation.
44:37And I imagine the markets would love a more modest proposal that involves Meta offering a ChatGPT beta simply to fuck over Sam Altman. And that's the thing. ChatGPT is big because everybody's talking about AI, and ChatGPT is the big brand in AI. It is not essential, and it's only being treated as such because the media and the markets ran away with a narrative that they barely understood. DeepSeek pierced that narrative, because believing said narrative also required you to believe that Sam Altman is a magician versus an extremely shitty CEO that burned a bunch of money. And I don't believe that, even before DeepSeek, that Altman's peers really bought into the hype.
45:17Sure, you can argue that DeepSeek just built on top of software that already existed thanks to OpenAI. Thank you, Casey, by the way. But this begs a fairly obvious question. Why didn't OpenAI build on top of software invented by OpenAI? And here's another question. Why does it goddamn matter? In any case, the massive expense of running generative models hasn't been the limiter on their deployment or their success. You can blame that on the fact that they, as a piece of technology, are neither artificial intelligence nor capable of providing the kind of meaningful outcomes that would make them the next smartphone or cloud computing.
45:54Honestly, it's all been a con. It's been a painfully obvious one. what I've been screaming about since February, since when I started this podcast, trying to explain that beneath the hype was an industry that provided modest at best outcomes rather than any kind of next big thing. Without reasoning as its magical new creation, OpenAI really doesn't have anything left. Agents aren't coming. Large language models aren't going to build them. AGI isn't coming either. There's no proof it's possible. All of this is fucking flimflam to cover up how mediocre and unreliable the fundament of the supposed AI revolution really was.
46:28All of this money, all of this energy, and all of this talent was wasted thanks to markets that don't actually do anything, markets that don't make for good companies just growth hogs, and a media industry that fails to hold the powerful to account. And it looks like everything got broken by some random outgrowth of a Chinese hedge fund. It's so ridiculous. It's so sickening. I can't believe it. Well, I can totally believe it. I'm actually surprised I didn't come up with this idea myself. Just the idea that someone could do this cheaper. It makes me go insane. And what's more insane is that OpenAI is still going to be able to raise that round.
47:04But I think we're approaching the end of days. I'm not calling the end of the bubble yet. I refuse to do that. I'm not going to do that. What I am going to say is it's deflating. And I am going to say I have no idea how they reinflate it. A bunch more money isn't going to change anything. These companies are washed. Sam Altman's washed. He's the Mark Sanchez of the tech industry. and he's so sickening all of them are so sickening imagine if this money had gone anywhere else imagine if it had gone into batteries imagine if it had gone into climate stuff imagine it gone somewhere useful imagine if instead of spending billions on this dog shit they actually fix their problems they actually fix the products that they've made worse but the problem is that the rot economy is in control that the growth at all costs mindset is all that you see in the tech industry.
47:52And Silicon Valley needs to repent. Silicon Valley needs to change its ways. Because when the bubble bursts, and I really think it will, the destruction that follows will be horrifying. And it will hit workers. It will hit tens of thousands of tech workers. And it will affect the markets. And after that, the markets are going to realize something. The tech industry doesn't have anything left. They don't have another growth market. They're out. They're all out. and I look forward to telling you how I look forward to talking about it when it happens and I'm so grateful for you listening Thank you for listening to Better Offline The editor and composer of the Better Offline theme song is Matt Ossowski You can check out more of his music and audio projects at mattosowski.com M-A-T-T-O-S-O W-S-K-I dot com.
48:47You can email me at ez at betteroffline.com or visit betteroffline.com to find more podcast links and of course my newsletter. I also really recommend you go to chat.wheresyoured.at to visit the discord and go to r slash betteroffline to check out our reddit. Thank you so much for listening. Better Offline is a production of Cool Zone Media. For more from Cool Zone Media, visit our website coolzonemedia.com or check us out on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
49:32We'll see you next time.
49:46Smarter, park faster. ParkWiz. Download the ParkWiz app today and save every time you park. Did it occur to you that he'd charmed you in any way? Yes, it did, but he was a charming man. It looks like the ingredients of a really grand spy story, because this ties together the Cold War with the new one. I often ask myself now, did I know the true Jan at all? Listen to Hot Money Agent of Chaos on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
50:46Listen to Intentionally Disturbing on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. I'm Dr. Scott Barry Kaufman, host of the Psychology Podcast. Here's a clip from an upcoming conversation about how to be a better you. When you think about emotion regulation, you're not going to choose an adaptive strategy which is more effortful to use unless you think there's a good outcome. Avoidance is easier. Ignoring is easier. Denial is easier. Complex problem solving takes effort. Listen to the Psychology Podcast on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.
51:23This is an iHeart Podcast.
From the publisher
In this episode, Ed Zitron explains how the emergence of a much-more-efficient generative AI model has become an existential threat to the future of OpenAI and Anthropic - and tells a dark story about the death of innovation in Silicon Valley.
---
LINKS: https://www.tinyurl.com/betterofflinelinks
Newsletter: https://www.wheresyoured.at/
Reddit: https://www.reddit.com/r/BetterOffline/
Discord: chat.wheresyoured.at
Ed's Socials:
https://www.instagram.com/edzitron
https://bsky.app/profile/edzitron.com
https://www.threads.net/@edzitron
See omnystudio.com/listener for privacy information.