Ben Thompson: Anthropic, the Pentagon, and the Limits of Private Power

0:00Ben Thompson:You might not be interested in politics, but politics has an interest in you. What is politics? War by other means. You might not be interested in that. It is going to have an interest in you. If we're going to analogize it to nuclear weapons, as Dario Amadei has done repeatedly, you have to think through what would happen in a world where a private company developed nuclear weapons. It's a tension that's been brewing for years, which is, are you an American company subject to American law and even beyond law, just morally compelled to support the U.S. military or not? A private company built something powerful enough that the government threatened to destroy it for not cooperating.

0:47Jordi Hays:That's not hypothetical. It happened last week when the Department of War designated Anthropic a supply chain risk after the company refused to remove safeguards against mass domestic surveillance and autonomous weapons. Ben Thompson's response wasn't to defend either side. It was to point out what almost no one was saying. If AI is as powerful as its builders claim, the people with guns are going to want to say. Whether that means the U.S. government compelling access or China deciding to act because America is getting too powerful? These are no longer theoretical questions. In this conversation previously aired on TVPN, John Coogan and Jordi Hayes speak with Ben Thompson, founder of Stratechery, about his essay, Anthropic and Alignment.

1:36John Coogan:We have Ben Thompson in the Restream waiting room from Stratechery. Welcome to the show, Ben. How are you doing?

1:43Ben Thompson:I'm good. Hopefully I have the right microphone turned on this time.

1:46John Coogan:You do, and it sounds fantastic. Thank you so much for joining on Short Notice. Thank you for writing Anthropic and Alignment. It is a fantastic piece that I think covers all of my questions. But I want to start with just how did you process the weekend? How did you get to this particular place? And then what is your key thesis with Anthropic and Alignment?

2:09Ben Thompson:I mean, this is one of those ones, I don't know if it's good or bad, that it came out sort of at the end of the week. So I had a lot of time to think about it. Ultimately, I think it was good because I'm not sure anyone as explicitly made the point I did. And maybe it was bad because I feel there's a lot of like caveats, maybe in retrospect, I should have put in the article that would have addressed a lot of the points that people are upset about.

2:34John Coogan:Yeah.

2:35Ben Thompson:Basically zooming out, this was not a normative article where I'm saying what's happening is good or bad. And that's really the one caveat I really wish I would have put on there. I mean, I'm being out there accused by like a Neil Apatow, the full-throated fascist endorsement of fascism or something like that. And it's like, relax. Okay, can I get some credit for the last X number of years? basically the and there is a deep-rooted concern that i've had for a long time about and i'm now hesitant to even use sort of ea as a term because it's kind of now politicized thanks thanks to the events of the last week but a failure to grapple with a world of guns is basically the long and short of it and i actually think elizor has been the one guy who's been honest about this, where he wrote that Time article about potentially bombing data centers someday.

3:30Ben Thompson:And that's actually a point worth bringing up, which is all this stuff is right now in the digital realm with robotics and potential other applications. And it's obviously being used for military operations. It's crossing over into the physical realm. But if AI is as powerful as people say it's going to be, then there are going to be real world reactions to that. And if we're going to analogize it to nuclear weapons, as Dario Amadei has done repeatedly, you have to think through what would happen in a world where a private company developed nuclear weapons. What would the government's response be?

4:12Ben Thompson:And that's not to say that the government response in that case is good or bad, or does it follow sort of constitutional principles or whatever it might be. Obviously, I want them too. On the surveillance point, I've been concerned about the application of computers to our surveillance laws for years. So many things in our society assumed a certain level of friction in doing things that computers already obviated and AI is going to just do that on steroids. I do think we need new laws. I think all this stuff is correct. And I think the idea that AI being applied to these commercially purchased data sets, for example, is a huge problem that I don't want to happen.

5:01Ben Thompson:The concern I have is that if this technology is as powerful as it is on pace to be, unilaterally imposing restrictions, even if those restrictions are good, isn't just an issue as far as who rules us, the democracy issue, that sort of Palmer Luckey, I think, very eloquently raised. It's inviting very bad outcomes for those asserting that in general. And I feel there's been a lack of awareness of this. That's why I brought up the Taiwan-China thing. This has been a frustration I've had with Anthropik generally. They talk about, you know, Amadei's been very outspoken in terms of opposing selling chips to China for, in a narrow, you know, aspect, very, very good reasons.

5:56My pushback has always been what happens if we get super powerful AI

6:03Ben Thompson:and China doesn't, what are they going to do?

6:06John Coogan:Sure.

6:07Ben Thompson:The optimal thing would be to just bomb TSMC out of existence because suddenly that becomes optimal, even with all the costs that that does. And then what? Then what are we going to do? Like we're entering this. Like I don't like getting into political posts. It's not fun at all. I'm not having fun with this. It's not enjoyable. I can promise you this. And some people are like, well, you should have just made the post private. And I'm like, no, I actually, I really want Anthropic and people associated with this to read this because people have theorized for a while about what's going to happen as AI becomes more powerful.

6:47Ben Thompson:And now it's starting to happen for real. And I guess over the weekend, part of it was just, I felt compelled to say this and girding myself to do so. And even then, I still wasn't, I haven't waited in this for a while. It's no fun, but it is what it is.

7:05John Coogan:Can you unpack a little bit more of that tweet that you posted where you did the find on the Dario article for Taiwan and saw that it wasn't mentioned?

7:14Ben Thompson:Oh, I mean, I've just got to, I've sort of griped about this in general. I think that.

7:18John Coogan:So do you just think he should be, he should be talking about the Taiwan issue more deliberately? He should be messaging that? Why is it important that, why is it significant that he doesn't mention Taiwan?

7:32Ben Thompson:Well, I think the position about not selling chips to China is a totally legitimate one. I understand the argument. I could make that argument if I needed to. I have advocated the opposite, that number one, not only should we be selling chips to China, and a generation or two behind, which has always been sort of our standard practice with chips, we should also be allowing Chinese companies to fab with TSMC. That is a restriction that has come down. Now, these Huawei chips are somehow manufactured by TSMC. Let's not look too closely at it, but we should explicitly be allowing it. And the reason for that is I think it is a safer equilibrium to have China dependent on Taiwan than to try to cut them off from Taiwan while we are dependent on Taiwan.

8:23Ben Thompson:Taiwan is 70 miles off the coast of China. It's not an ideal position in the world for us to have a dependency on it and China to not have a dependency on it. So this is the problem. All this stuff has, everything going forward has massive trade-offs. The implication of letting China fab with TSMC or the implication of letting them buy NVIDIA chips is that they gain these incredibly powerful AI capabilities that is driving this entire debate. That is, in a vacuum, not a good thing. But nothing's in a vacuum. Everything is a trade-off. And in that specific area, I think that just it's repeatedly, again and again, being absolutist about the chip issue when I am frustrated to not see any public comment about the, that's not quite fair.

9:19Ben Thompson:He has made comments about, oh yeah, that would slow down sort of the adoption of AI the long run if Taiwan got bombed. I'm like, that's, in my mind, that's an insufficient consideration of the possibility of Taiwan getting bombed. Now again, I'm biased in that regard. I lived there for nearly two decades, but it's just the, the reason I brought it up in this context is if AI is what it is, the people with guns are going to want to have a say, whether that be domestically, whether that be internationally, that might be in the context of the U.S. government just taking it, trying to kill your company because they feel you're not cooperating, or it might be the context of China deciding it has to act because the U.S.

10:00Ben Thompson:is becoming too powerful. And it's not a fun debate. I do think the nuclear angle is a good one. It has echoes of the proliferation, question of mutual assured destruction, all those sorts of things. And that's just going to be the reality of the debate going forward. And again, it's not very fun, but I think it's also irresponsible to sort of run away from it.

10:25Jordi Hays:How much attention or what kind of factor do you think the information asymmetry between the Department of War and Anthropic played last week? It felt like in hindsight, Department of War knows they're headed into a major, what is now looking like a drawn-out conflict. Anthropik sitting there thinking, hey, we have this arbitrary deadline. Why do we need to renegotiate this now? And then if going off of Emil Michael's timeline, it sounds like they were still in the final hour trying to make a deal happen. And according to Emil, Dario was in a meeting and was busy and wasn't really respecting the deadline, which maybe he felt was kind of artificial.

11:11Jordi Hays:But in hindsight, now looks like it was a significant because the Department of War was, you know, taking the country into a conflict and wanted to know, hey, can we lean on one of our AI partners?

11:27Ben Thompson:I don't know. I mean, I think that seems pretty arbitrary to have cut. I mean, I'm hesitant to speculate. I don't know what was going on. I don't know the angles. I think, And that's why I didn't sort of delve too deeply into it. And I also think some of the specifics like this supply chain risk probably overbroad. And almost certainly the way it was stated in the tweet is definitely overbroad if you actually go and read the statute. I think the goal that I was – and again, this is where I wish I had sort of put more caveats to say, like, I'm not actually talking about all that stuff. I don't really care.

12:07Ben Thompson:I do care, but that's not the point of this article. The point of this article is there's all this talk about alignment. That's why I put that in the headline. And on one hand, alignment is aligning AI with humanity generally. But for the foreseeable future, and you could have a philosophical argument about the long-term viability of nation states in the age of the internet, much less the age of AI and whatever that might be. That certainly is a more pressing conversation than probably ever before. Anthropic exists in the context of the United States. And that's why I put that quote, you might not be interested in politics, but politics has an interest in you.

12:50Ben Thompson:What is politics? War by other means. You might not be interested in that. It is going to have an interest in you. And there's, like I said, a certain longstanding frustration of not fully grappling with that fact, having dorm room theoretical arguments about AGI. You go back to that post over Christmas about like AGI in like 100 years and no one having any jobs or being worthless or pointless or whatever, which included some implicit assumptions around property rights existing in 150 years as they exist today. Newsflash, if that happens, property rights as they exist today are going away. All these rights, this is a philosophical argument.

13:32Ben Thompson:That's why I started with the international law concept. All these rights, all these laws are subject to the agreement of those governed by them to follow them. And the final say is those who successfully inflict violence. And again, this isn't fun to think about. It's not pleasant. You would like to assume we operate in a world of laws that everyone follows them and goes by them. but to the extent AI is as impactful and powerful as it is, the more these questions, fundamental questions that we thought have been settled for hundreds of years, if not thousands of years, are going to be raised. And this is just the first of several episodes where I think that's going to happen.

14:18John Coogan:I grew up in sort of like post-Cold War, no ducking cover, didn't have a lot of fear of nuclear Armageddon, But Dario Amadei is a fan of this book, the making of the nuclear bomb. And it seemed like he sort of predicted that if AI becomes super powerful, the US might take a similar approach that they did with regulation of nuclear weapons. And as I was thinking about that, I feel sort of good about the way nuclear weapons are regulated. Like, I feel like we got the good ending and we haven't had nuclear weapons drop in 70 years. And it seems like things are going well there as well as they can, considering that there's this amazing, or tremendous, dangerous technology that exists, but it hasn't been deployed.

15:09John Coogan:It hasn't actually bombed anyone. But how do you think he's processing that book? How do you think we should be processing that idea of the government running the same playbook that they did with nuclear weapons.

15:25Ben Thompson:It's pretty interesting. I mean, on one hand, just from sort of a physical perspective, dealing with weights and software is very different than dealing with fissionable material. Or I guess the super bombs are like, they're actually like fusion devices, right? And that is trackable. It is interceptable. You know when Iran, to take a pertinent example, is trying to build enrichment facilities, all of which makes the problem easier to solve. So that's difference number one. Difference number two, and I really wish I would, I had this included and I cut it so the article would be tighter, but there is a very interesting point in technological history, which was the early days of Intel.

16:10Ben Thompson:And Bob Noyce made the decision that we will sell to the government, but we're not going to design chips for the government. And the distinction there was you had guaranteed orders, which was great. The government would take your IP and there was, and in his mind, the more important thing is there was limited volume. And the way that he foresaw correctly that this was going to be a very upfront capital intensive process of designing, you have to design them, you have to do the equipment, all of which is in the billions of dollars today. Back then it was in the tens of millions and hundreds of millions, is you need to find the largest possible market, which was the consumer slash business market, you design for that, that will accelerate your improvement and your capabilities so much that you will end up having better devices than the government could have ever requested or made for itself.

17:02That is at stake on steroids with AI.

17:06Ben Thompson:People, like I was talking to someone, they're like, why doesn't the government just get someone to make their own model? It's like, because it's got, like you talk about government contracts, we're like single digit billions. We're talking about for the amount that's going into CapEx, the cost of these models. We're talking hundreds of millions of dollars for the models and hundreds of billions of dollars approaching a trillion dollars a year in CapEx. That is only sustainable and viable if you're selling to everyone. But that introduces the entire new dynamics where the government built nuclear.

17:42Ben Thompson:It started there, and it started with a lot of assumptions because it was a government program. We are necessarily for economic reasons because of all the upfront costs entailed, starting with private companies of which the government is one of many customers. And that introduces the assumption that, well, it's a private company with private property rights and all those sorts of things, all of which I want to be true. Again, I don't like how this is going down at all. The point here is to say there's a good reason why it's not going down that way. And there needs to be cognizance that even though this is a private company that is building the model, general purpose, and for very good reasons, wants to put restrictions.

18:30Ben Thompson:Again, I think the surveillance one is a very powerful argument that I agree with. The problem is that you just need to be aware of, yes, the government is a small customer. the government is also the entity, again, not to be blunt, with guns. Like, you know, like, why do I pay taxes? Because the law says to pay taxes. No, at the end of the day, I pay taxes because, you know, if you really want to distill down, if I don't, someone with guns will come to my house and throw me in jail, right? Like, we don't think about that. But at the end of the day, where do these assumptions and laws and rights flow from?

19:05Ben Thompson:And as long as that is still the case, that it needs to be a decision-making factor for these companies.

19:13John Coogan:How do you think this plays out for Anthropic? It's such a small contract, but it's so important in the zeitgeist. There's a lot of people that are rallying around Anthropic because of this. There's a lot of people that are pulling away from Anthropic because of this. It feels like there is a business to be built that doesn't work with the government, but delivers coding models and knowledge retrieval systems and a whole bunch of really valuable products and technology, and it winds up being fine. But at the same time, you don't want this hairy relationship with the government, adversarial, to go on for a long time.

19:50Ben Thompson:I would like them to sell to the government, and I would like Congress to pass a law addressing these digital surveillance issues. Yeah. And a lot of people are like, that's unrealistic, which I'm amenable to. But at the end of the day, if you don't have it's legal or not legal as your guiding standard, the only alternative is someone has to decide. And the implication of that not being a sufficient justification is that means a private executive is deciding. And if AI is what it is, I think that's going to be, I use this word intolerable. I didn't mean intolerable to me. I meant intolerable to those with power to have a private executive making those decisions or not.