The AI Slowdown Debate | Diet TBPN

0:01We're back. Did anything happen over the weekend in tech or was it kind of mellow?

0:06John Coogan:It was a crazy weekend. There was a lot of news, a lot of back and forth about pacing the AI debate. I was in the hospital because I got rhabdo. It's brutal, but I am recovering. Explain rhabdo because it sounds really scary and it is bad. It can be very bad. Fortunately, I did okay. I got an IV, got some medicine, broke a fever, got through it, and I'm healing up. But it happens, and it's the dumbest, dumbest illness you can possibly get. It comes from working out too much and not being hydrated. Very avoidable. Hopefully, AI can find a cure. If there's even a 1 % chance that AI can find a prevention for rhabdo, I think it's worth any amount of risk.

0:45Pushing forward.

0:45John Coogan:Yeah, we must push forward. No, lots of debate over how fast we should push forward. We have never missed an episode for being sick. Yeah. And I realized that because someone was asking me the other day, what do you guys do when you're sick? We just keep going. I mean, I did talk to the doctor. They said, do you want a doctor's note for work? And I said, no. I kind of laughed. But I do think that this job can be done while sick because you're just sitting in a chair having a conversation. But I might be a little bit slow today. So bear with me if I'm not the sharpest on any of the questions. Before we get into the pacing AI debate, what happened with the U.S.

1:25John Coogan:Open? Give me a little tennis update. What happened? It's on the cover of the Wall Street Journal, so you know it's important news. Tennis' U.S. Open climaxes in back-and-forth duel, but there was something interesting. So Ben Shelton, the American, lost. Okay. And what was notable was that when he actually hit the winning shot and won, he went back and was getting ready for the next point. And he starts looking around and everyone is standing. You can see people in the crowd are standing up and cheering. And he slowly realizes that he won by checking to finally look at the scoreboard. And, yeah, it just became very obvious that he was so focused on the next point.

2:13He was actually not looking at the scoreboard at all. It was just like flow state, just playing. That's awesome. And I'm sure a lot of venture capitalists out there were seeing that. And they're like, yeah, all that matters is your next investment. You shouldn't even know when one of your companies, your seed companies, are going to IPO. You should be so focused on the next.

2:31John Coogan:I mean, I think a lot of people that don't like VC Victory Labs would appreciate that too. It's like, we get it. There was an acquisition. Let's focus on what you're doing right now. I also like the cameraman work. I saw a clip of the scoreboard was counting down. There was zero, zero, seven seconds left. Who do they cut to? Pierce Brosnan, 007 himself. I thought that was nice camera work. Shows you that there's levels to the game of live production. Yeah. You know, you got to, those little touches, clearly that was just spur of the moment. That's not, that's not something that's scripted. That's just someone being creative in the moment with the tools that they have.

3:06John Coogan:The switcher, basically. Anyway, the pacing AI debate. I wanted to start with some history here and then we can go through the timeline of what actually happened since we got off stream Friday. And then all the news breaks, of course. But the pacing the frontier goes back to July 28th of this year. So there was a pacing the frontier statement, an open letter. It was signed by employees and leaders from OpenAI, Anthropic, DeepMind, Meta, and a bunch of other places talking about this idea of pacing the frontier. Then the same day, Sam Altman publicly said, we may have to pace the rate of AI development.

3:45John Coogan:Two days later, OpenAI tells Axios that OpenAI shaped the petition's language, and Altman had discussed pacing with White House officials. So stuff was happening in D.C. This was back in basically the very beginning of August, very end of July. Then August 18th, OpenAI publishes something called Pacing and Model Development in an Era of Cyber Critical Capabilities. OpenAI said it actually slowed scaling and paused a major RL run. There was that two-week training pause that happened. On August 31st, official Anthropic Post explicitly discusses pacing the frontier, so the idea is spreading, and calls for coordinating pacing mechanisms.

4:26John Coogan:And then on September 12th, it was actually Saturday morning that Dario dropped the essay, We Must Pace the Frontier, turning the concept into a much bigger public Anthropic campaign that everyone's talking about today. So the rollout of this was pretty, pretty quick. And there's a few key things from Dario's essay. It hit the timeline at 7 a.m. Pacific. He was up early, posting. Perfect time to get a lot of likes on a banger. You've got to feel something. Everyone else, there's no fundraising news happening. Drop your essay early Saturday morning. Let people digest it with their morning coffee.

5:00John Coogan:Elon was clearly engaging. By 8 a.m., Elon endorses it. 9.30 a.m., Sam endorses it. Saturday, 4 p.m., Demis generally agrees with Dario's point and argues that it needs some work. Remember, Dario put out something pretty similar, talking about pacing and AI development and the need for oversight generally. Months ago, before transitioning out of his role as CEO of DeepMind and stepping up to, I think, chairman. Then later, David Sachs, on Saturday, but at 8.15, has had time to process everything. He asks some questions about Dario's post, goes back and forth. Sunday morning, the Financial Times reports that Donald Trump has rejected calls from tech bosses for an AI slowdown.

5:44John Coogan:Later that day, Sunday morning, Gavin Baker rounds up the weekend. So we can kind of go through some of this and go through those posts. But what actually happened in Dario's essay, We Must Pace the Frontier? He has the three-step plan on how he proposes Number three will shock you. basically. Number one, give independent third-party evaluators access to AI companies so they can verify safety practices. He calls out Meter, which drew a lot of attention because people were debating how independent Meter is. There's some Meter employees who have worked at OpenAI, have worked at Anthropic. There's people that work at Anthropic who worked at Meter.

6:22John Coogan:It's all very much a revolving door with these AI safety groups. And then the AI labs, you go back and forth depending on what you're interested in. So not as independent as like, oh, yeah, we're going with an accounting firm who doesn't really have an opinion about the AI future. These are all people who are deeply embedded in the discourse, which could be a huge benefit because they take the stuff really seriously, could also lead to some people flagged as like conflicts potentially, but still some optimism from the community around what that would look like. Two is Frontier Labs in democratic companies should work together with the government's help to establish AI safety standards.

7:07John Coogan:And this was the one that people were going back and forth on. Do the labs actually need government approval from this? Dean Ball was talking a little bit about the Sherman Antitrust act, there are very clear rules about companies agreeing to do really anything in concert with each other. This is the number one reason that you don't want monopolies and cartels forming, is that if all of a sudden, you know, oh, you know, American Airlines and Delta and United all say, yeah, we, you know, for safety reasons, we should have fewer flights. It's like, okay, well, what would that do to the market? Probably drive up their margins because there's less flights, charge more for them.

7:47John Coogan:They have fixed costs. So you could very clearly see that collusion in that market would lead to more profits for them and more harm for the consumer. That's where the Sherman Antitrust Act came from. We flagged this a week or two ago that even though it seems like, oh, well, if Demis and Dario and Sam and Elon all agree, this should just happen. That's not a problem. There are actual legal and federal regulatory rules that might need to be adjusted or waived for something like this to happen. So I think that's why he's calling for that specifically. He's saying we need an exception to the rules so that we can actually do this because it's great to all write blog posts that we loosely agree on, but to have anything formal, we need government approval.

8:31John Coogan:And then third, this is the one that will shock you apparently, the U.S. and other democratic countries should try and coordinate with authoritarian governments. So he wants the U.S. and China to coordinate on AI pacing. Of course, China still wants to catch up, and there's some tricks there. What will happen in equilibrium if both countries have the exact same capabilities? Is that the best outcome, or do you always want to be in the lead? If you always want to be in the lead, it's very hard to negotiate with someone and say, oh, well, why don't you just have half as many nuclear weapons as we have?

9:04John Coogan:That'll be great. We'll pace you. And so you go back to nuclear nonproliferation and all the difficult debates there. We obviously wound up with a multipolar world with everyone wanting at least some capabilities there. So Elon chimed in agreeing with Dario. Then Sam and Demis also agreed. The timeline, on the other hand, had some questions and concerns about Dario's post. Who will these third-party evaluators be? Why wait for Washington to regulate the frontier? If Anthropic believes AI is advancing too quickly, why not simply slow down on its own? David Sachs raised some questions on X. and we can go to David Sachs' post, which I believe here is in the timeline.

9:41John Coogan:He said, Dario has written that we need to, quote, pace the frontier and Sam has agreed. People may be surprised by my response. Go ahead. You guys are the frontier by any reasonable metric, market share, revenue growth, model capability. The two of you have a duopoly on frontier intelligence. You've also claimed the lead is widening because of recursive self-improvement. I don't think what you see, I don't see what you see in the lab if the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible. But stop pretending you need anyone else's permission.

10:12John Coogan:Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending meter is independent when it is intertwined with Anthropics investors and staff. Stop pretending that you need those same evaluators to police competitors who aren't even at the frontier. And so, yeah, maybe that's a little bit of an issue. Although, yeah, I mean, And the regulatory capture angle is always like if the safety rules become super cumbersome such that you can't start a instinct-like product, right? It's like instinct is a small team.

10:51John Coogan:They're not building a frontier model. They're not training their own models, at least not yet. But they're like doing stuff with AI if all of a sudden they have to spend a year negotiating with a bunch of third-party evaluators, have a bunch of people embedded, and then go through some sort of review process and the government's not getting back to them, like you can't have a breakout consumer product. And that could be true for the long term. And it doesn't feel like there's been any type of effort from anywhere to try to control or regulate like application layer companies. And Greg was on Odd Lots.

11:23This episode came out this morning talking about not trying to – doesn't believe there should be any sort of limits to –

11:30John Coogan:Application layer stuff. Yeah, but basically, like, if you just want to train a model at home for your own use cases, you know, there shouldn't be limits on that. Yeah. Trump came out yesterday. He was pretty active over the weekend. He said the only control or, quote, unquote, guardrails that AI needs is a strong and smart, high IQ president. And the USA has that in spades. Trump. That is a wild post. the trump administration has stopped ai people from doing bad or potentially bad things like dario what are these graphic who is now pretending to be a perfect little angel and we will continue to do so we already have tremendous criminal and regulatory power over these companies there is a sick conspiracy going on against ai and data centers and the only one that is happy about it is china whoever wins ai wins we are leading china and all others and we'll continue to do so.

12:24Conspiracy theorists, treasonists, traitors, and leakers, beware. Thank you for your attention to this matter. It is funny to accuse the labs of being in a conspiracy against AI.

12:36John Coogan:Yeah, that really, really doesn't follow. I don't, yeah, it does not make sense. But he also followed up just 40 minutes ago, said concerning AI, when in the history of business, capital H, capital B history of business. Did anyone see the leaders of an industry call for regulation that, if strongly implemented, would drive them into oblivion and bankruptcy? AI is taking over the world, destroying humanity, and all other things bad is a hoax, no different from Russia, Russia, Russia, Ukraine, Ukraine, Ukraine, impeachment hoax one, impeachment hoax two, and all the other hoaxes and scams that America was forced to endure.

13:12President Xi of China just announced that China will be doing absolutely nothing to stand in the way of AI or its future. Google has recently stated that they want to build a massive plant in Finland, all because they are finding permitting too difficult in the United States. I'm not happy about this and want them to change their thinking. AI and data centers will be the greatest economic development engine in history. Bigger than oil, gold, diamonds, or even the internet. Bigger than diamonds.

13:40John Coogan:Bigger than diamonds. Like, of all the technology. The internet, the transistor, electricity, fire, the wheel, diamonds. These are equivalent things. It will not be stopped by brilliantly run destructive forces during the term of President Donald Trump. Okay. Well. Anyways, I'm wondering what the, so a bunch of hedge funds are paying, you know, they pay to get access to this, let's assume like a minute early. something like that how did they trade this because it's very long two meters long long diamonds yeah maybe diamonds are are underappreciated i think diamonds are actually in a really rough spot i think people are moving to lab grown stuff really aggressively so it's a it's actually a tough tough period of time for the diamonds and i'm sure the diamond industry is calling for regulation uh for sure for sure um we got to keep going because the president was on a roll this weekend.

14:45Let's pull up this video. We take it a lot of their cars. Car tariffs. This week, again, there's been so much concern about AI. Have you heard directly from any of these AI tech leaders? And you seem to downplay some of the concerns that we've heard. But it's going to be more good than bad, but by a lot. And I've said it from the beginning. Whoever wins AI and we're leading by a lot. Whoever wins AI wins. Have you ever used it yourself or have you been? How do you use AI? But I would say, you know, for the most part, you can use AI for a lot of things. What do you use every word? Right away. Speaking of AI.

15:24John Coogan:The funny thing is that, like, there's AI generated slop on his social, true social account like constantly. Like the slop videos are obviously AI. Like both sides of that debate or that discussion should have known that, I guess. but I guess it's like people on his team are creating like the AI slot videos because like there will be like a new vibe coded like like uh you know some sort of simulator that they put up on White House stuff all the time so but I guess that's not him it's probably people on his team that are like I made this AI video yeah but I don't know Gavin Baker had a roundup of what happened let's go through what's going on and lots of people just sort of like processing this going back and forth people were expecting Elon to be sort of against this maybe anti-Dario uh he of of course, came out in favor and has been talking about how, like, he was working with Nick Bostrom in 2014 on AI safety, has been very concerned about this the whole time.

16:16John Coogan:Although, at various points in time, Elon has been less of a doomer and more like, we got to build, we got to go, we got to, you know, actually compete here, working both alongside labs, co-founding OpenAI, founding XAI, growing the model capabilities there. But he's like firmly in favor of this. So let's see what Gavin Baker had to say. He said, wow, 24 hours for AI and lots of different proposals have been made. TLDR, the only tangible new fact is that OpenAI and Anthropic are going to have embedded third-party evaluators from unknown organizations with Dario floating meter as a possibility.

16:56John Coogan:Having third-party evaluators is smart as there's no Section 230-style liability shield for model outputs, and showing a duty of care will be important in future litigation. Several Internet companies might have gone bankrupt without Section 230, so limiting liability really matters. I think there are minimal investment implications from this single new fact, but I do think that anyone who wants, quote, a smoother for a longer cycle than most constraints are good. Wafers, watts, real rates, and spreads. I was thinking about how a lot of the safety folks are worried about just the capability build out from the semiconductor supply chain.

17:34John Coogan:Like they would like to restrict the amount of chips that are created. And I'm wondering if should we not be pacing electricity generation? Does everyone agree that we should be building more solar, thermal, wind power right now? Is that uncontroversial? Is there anyone who says, like, no, we shouldn't be doing that? Because if we wind up in a world where you're pacing frontier AI development, current models are sort of diffusing and there's some economic impact there, but you're not, you know, blitzscaling the data centers and the frontier intelligence. can you apply some of that capital, some of that human talent to just getting nuclear power online, getting solar power online, re-industrializing the American energy supply chain, battery, relieving grid tension?

18:21John Coogan:Because everyone who complains about higher energy prices would be happy with that. I don't think the AI safety folks would be upset about that because you could use it for a lot of things. Worst case, it's like, okay, you wound up with like, okay, we can never build you know, AGI or ASI because it's too dangerous. At least we have a lot of cheap energy. Is there any reason why you wouldn't, why you would want to pace the build out of the American electrical grid? I can't see why. What do you think? I mean, at some level, like, yeah, it makes sense because it is like an input, but it's like, seems like way too diffuse.

18:55Yeah. Because like, yeah, it's like, if you build a lot of chips, like sure, you can like, you know, make more data centers that run YouTube or whatever, but like, there's kind of a narrow use case for chips.

19:06John Coogan:Yeah. Where power, you can kind of do whatever. I wonder if you, yeah, at what point is there some sort of like AI safety valve that's like, we need to regulate that company in South Korea that makes toilets, that also makes films for semiconductors. Because if we can restrict them, then we can slow down the pace of AI progress. Going really, really deep in the supply chain for safety reasons, potentially. To summarize the events, Gavin Baker says, Dario made the most maximalist proposal of the weekend. embedded third-party evaluators, national regulatory regime for models beyond a certain capability ingredient threshold, a broad international regulatory pact between democracies, stricter limits on compute distillation for China, and then a different international regulatory regime that encompasses China.

19:52John Coogan:Before there's ever a national regulatory regime, he wants a Sherman Act waiver so that Anthropa can safely coordinate with OpenAI and other frontier labs without antitrust fears. To be frank, this latest proposal is much less maximalist than some of his prior proposals like policy on the AI exponential, where he advocated for an FAA for AI. Now people are talking about FINRA for AI, which is more non-governmental regulation, but industry focused. I believe he is sincere in his beliefs. And despite all the protestations, all of this would also probably be good for his business over the long term.

20:26John Coogan:Yeah, there's a lot of tricks here where people, where something, if there are multiple quadrants of like actions you can take, and it's like it's either good or bad for your business or it's good or bad for safety you can land in the good for your business and good for safety quadrant and everyone will accuse you of being cynical and doing it only because it's good for your business and so yeah it gets really difficult to to say okay yes like the thing that i'm advocating for is good for my business now rune had another take which is like no if you're placing the frontier that's going to compress margins because everyone else is just going to catch up and it's going to become more oligopolistic So if you hold, you know, like, yeah, there's this duopoly right now for the true frontier amongst OpenAI and Anthropic.

21:08John Coogan:But if you hold that constant, like Watermelon from MSL is going to catch up, DeepMind is going to catch up. Like you're going to have three or four Grok is going to train to that level. And so you're going to have four, five, six, seven really capable competitors. What was interesting is like R. Karazian was saying that AI spend declined a little bit in August. and that was mostly because of price cutting. It wasn't actually people using less AI. So it's not like people are pulling back on their use of AI, but there's more price competition. So OpenAI is discounting and people are being, we saw like the Fable adoption was a little weak and a lot of it's like, hey, are we sure we want to use the million dollar model for checking the weather?

21:52John Coogan:Like let's be a little bit more judicious about this. And so if you wind up with a world where there's much more price competition, that could compress the frontier's margins, which is not good for business. So it doesn't really put you in that top quadrant that you have to be so cynical about. The interesting thing is that he said that he doesn't think that it's open source Chinese models that are actually putting pricing pressure on the frontier. It's more of the duopoly dynamic. And I think that's because, and I think he sort of alluded to this, that there's just a lot of companies that are like, I am comfortable doing business with an open AI, an Anthropic, a Google, of course, something that's on AWS, something that's on Azure.

22:31John Coogan:Yes, I could just download the weights for DeepSeek, but I'm not really comfortable with that. I don't really want to deal with that. I don't want to deal with the scaling and uptime of that. And so, yes, there are APIs out there that are reasonable, but none of them are as mature as the Frontier Labs, not in terms of the intelligence, but in terms of the actual, if I implement this in my business, will it reliably give me the right results? Is there a high-quality SLA? Will the service stay up? Will uptime be good, et cetera? So there is a world where you could get to a place where this is bad for the frontier companies business-wise.

23:07John Coogan:And Sam has talked about that in an interview with Fortune, saying, like, if it's bad for our business, like, we have said to our investors at every step for, like, a decade that, like, this is kind of a nonprofit. There are a whole bunch of different ways in which this is not purely aligned to shareholder interests like other companies. PBCs, yeah, public benefit corporations. So I think both Sam and Dario feel very confident about being able to go to their investors and say, hey, we're doing something that's going to slow down revenue growth or tank margins or just reduce the market cap of the company.