FULL INTERVIEW: Ben Thompson on Why Anthropic is Wrong

0:00John Coogan:We have Ben Thompson in the Restream waiting room from Surtekery. Welcome to the show, Ben. How are you doing?

0:07Ben Thompson:I'm good. Hopefully I have the right microphone turned on this time. You do, and it sounds fantastic.

0:11John Coogan:Thank you so much for joining on Short Notice. Thank you for writing Anthropic and Alignment. It is a fantastic piece that I think covers all of my questions. But I want to start with just how did you process the weekend? How did you get to this particular place? and then like what is your key thesis with anthropic and alignment i mean this is one of

0:33Ben Thompson:those ones uh i don't know if it's good or bad that it came out sort of at the end of the week so i had a lot of time to think about it yeah um ultimately i think it was good because i'm not sure anyone very as explicitly made the point i did yeah and maybe it was bad because uh i feel there's a lot of like caveats maybe in retrospect i should have put in the article that would have addressed a lot of the points that people are upset about. Basically zooming out, this was not a normative article where I'm saying what's happening is good or bad. And that's really the one caveat I really wish I would have put on there.

1:09Ben Thompson:I mean, I'm being out there accused by like a Neil Apatow, like the full-throated fascist endorsement of fascism or something like that. And it's like, relax, okay? Can I get some credit for the last X number of years? basically the and there is a deep-rooted concern that i've had for a long time about and i'm now hesitant to even use sort of ea as a term because it's kind of now politicized thanks thanks to the events of the last week but a failure to grapple with a world of guns is basically the long and short of it and i actually think eliza has been the one guy who's been honest about this, where he wrote that Time article about potentially bombing data centers someday.

1:53Ben Thompson:And that's actually a point worth bringing up, which is all this stuff is right now in the digital realm with robotics and potential other applications. And it's obviously being used for military operations. It's crossing over into the physical realm. But if AI is as powerful as people say it's going to be, that there are going to be real world reactions to that. And if we're going to analogize it to nuclear weapons, as Dario Amadei has done repeatedly, you have to think through what would happen in a world where a private company developed nuclear weapons. What would the government's response be?

2:35Ben Thompson:And that's not to say that the government response in that case is good or bad, or does it follow sort of constitutional principles or whatever it might be? Obviously, I want them to. On the surveillance point, I've been concerned about the application of computers to our surveillance laws for years. So many things in our society assumed a certain level of friction in doing things that computers already obviated and AI is going to just do that on steroids. I do think we need new laws. I think all this stuff is correct. And I think the idea that AI being applied to these commercially purchased data sets, for example, is a huge problem that I don't want to happen.

3:24Ben Thompson:The concern I have is that if this technology is as powerful as it is on pace to be, unilaterally imposing restrictions, even if those restrictions are good, isn't just an issue as far as who rules us, the democracy issue that sort of Palmer Luckey, I think, very eloquently raised. it's inviting very bad outcomes for those asserting that in general and i feel there's been a lack of awareness of this that's why i brought up the taiwan china thing this has been a frustration i've had with anthropic generally they talk about you know amide's been very outspoken in terms of opposing selling chips to china for in a narrow you know aspect very very good reasons my pushback has always been what happens if we get super powerful ai and china doesn't what are they going to do sure the optimal thing would be to just bomb tsmc out of existence because suddenly that becomes optimal even with all the costs that that does and then what then are we going to do like we're entering this like i don't like getting into political posts it's not fun at all i've i'm i'm not having fun with this it's not enjoyable i could promise you this yeah and some people are like well you should have just made the post private i'm like no i actually i really want anthropic and people associated with this to read this because people have theorized for a while about what's going to happen as ai becomes more powerful and now it's starting to happen for real.

5:14Ben Thompson:And I guess over the weekend, part of it was just I felt compelled to say this and girding myself to do so. And even then, I still wasn't, I haven't waited in this for a while. It's no fun, but it is what it is.

5:27John Coogan:Can you unpack a little bit more of that tweet that you posted where you did the find on the Dario article for Taiwan and saw that it wasn't mentioned?

5:37Ben Thompson:Oh, I've sort of griped about this in general. I think that...

5:41John Coogan:So do you just think he should be talking about the Taiwan issue more deliberately? He should be messaging that? Why is it significant that he doesn't mention Taiwan?

5:55Ben Thompson:Well, I think the position about not selling chips to China is a totally legitimate one. I understand the argument. I could make that argument if I needed to. I have advocated the opposite, that number one, not only should we be selling chips to China and a generation or two behind, which has always been sort of our standard practice with chips, we should also be allowing Chinese companies to fab with TSMC. That is a restriction that has come down. Now, these Huawei chips are somehow manufactured by TSMC. Let's not look too closely at it, but we should explicitly be allowing it. And the reason for that is I think it is a safer equilibrium to have China dependent on Taiwan than to try to cut them off from Taiwan while we are dependent on Taiwan.

6:46Ben Thompson:Taiwan is 70 miles off the coast of China. It's not an ideal position in the world for us to have a dependency on it and China to not have a dependency on it. So this is the problem. All this stuff has everything going forward. has massive trade-offs. The implication of letting China fab with TSMC or the implication of letting them buy NVIDIA chips is that they gain these incredibly powerful AI capabilities that is driving this entire debate. That is in a vacuum, not a good thing, but nothing's in a vacuum. everything is a trade-off and in that specific area i think that just it's repeatedly again and again being absolutist about the chip issue when i am frustrated to not see any public comment about the that's not quite fair he has made comments about oh yeah that would slow down sort of the adoption of air the long run of taiwan got got bombed i'm like that's in my mind that's an insufficient consideration of the possibility of Taiwan getting bombed.

7:54Ben Thompson:Now, again, I'm biased in that regard. I lived there for nearly two decades. But it's just the reason I brought it up in this context is if AI is what it is, the people with guns are going to want to have a say, whether that be domestically, whether that be internationally, that might be in the context of the US government just taking it, trying to kill your company because they feel you're not cooperating or it might be the context of china deciding it has to act because the u.s is becoming too powerful because that you know and um it's not a fun debate it it does i do think the nuclear angle is a good one it has echoes of the proliferation question of mutual assured destruction all those sorts of things and that's just going to be the reality of the debate going forward and again it's not very fun but i think it's also irresponsible to sort of run away from it.

8:48Jordi Hays:How much attention or what kind of factor do you think the information asymmetry between the Department of War and Anthropic played last week? It felt like in hindsight, Department of War knows they're headed into a major, what is now looking like a drawn out conflict. Anthropic sitting there thinking, hey, we got this like arbitrary deadline. Why do we need to renegotiate this now and then if if going off of emil michael's timeline it sounds like they were still in the final hour trying to make a deal happen and according to emil dario was in a meeting and and was busy and wasn't really respecting the deadline which maybe he felt was kind of artificial but in hindsight now looks like it was a significant because the department of war was you know taking the the uh country into a conflict and wanted to know hey can we lean on

9:46Ben Thompson:one of our one of our ai partners i don't know i mean i i think that seems pretty arbitrary to have cut i mean i'm hesitant to speculate i don't know what was going on i don't know the angles i think and that's why i didn't sort of delve too deeply into it and i also think some of the specifics like this supply chain risk probably overbroad. And almost certainly the way it was stated in the tweet is definitely overbroad if you actually go and read the statute. I think the goal that I was, and again, this is where I wish I had sort of put more caveats to say, like, I'm not actually talking about all that stuff.

10:27Ben Thompson:I don't really care. I do care, but that's not the point of this article. The point of this article is there's all this talk about alignment. That's why I put that in the headline. And on one hand, alignment is aligning AI with humanity generally, but for the foreseeable future, and you could have a philosophical argument about the long-term viability of nation states in the age of the internet, much less the age of AI and whatever that might be. That certainly is a more pressing conversation than probably ever before. Anthropic exists in the context of the United States. And that's why I put that quote, you might not be interested in politics, but politics has an interest in you.

11:13Ben Thompson:What is politics? War by other means. You might not be interested in that. It is going to have an interest in you. And my, there's a, like I said, a certain longstanding frustration of not fully grappling with that fact, having dorm room theoretical arguments about AGI. You go back to that post over Christmas about like AGI in like 100 years and no one having any jobs or being worthless or pointless or whatever, which included some implicit assumptions around property rights existing in 150 years as they exist today. Newsflash, if that happens, property rights as they exist today are going away.

11:52Ben Thompson:All these rights, This is a philosophical argument. That's why I started with the international law concept. All these rights, all these laws are subject to the agreement of those governed by them to follow them. And the final say is those who successfully inflict violence. And again, this isn't fun to think about. It's not pleasant. You would like to assume we operate in a world of laws that everyone follows them and goes by them. but to the extent AI is as impactful and powerful as it is, the more these questions, fundamental questions that we thought have been settled for hundreds of years, if not thousands of years, are going to be raised.

12:36Ben Thompson:And this is just the first of several episodes where I think that's going to happen.

12:40John Coogan:I grew up in sort of like post-Cold War, no ducking cover, didn't have a lot of fear of nuclear Armageddon, but Dario Amade is a fan of this book, The Making of the Nuclear Bomb, and it seemed like he sort of predicted that if AI becomes super powerful, the US might take a similar approach that they did with regulation of nuclear weapons. And as I was thinking about that, I feel sort of good about the way nuclear weapons are regulated. I feel like we got the good ending and we haven't had nuclear weapons drop in 70 years. And it seems like things are going well there as well as they can, considering that there's this amazing, tremendous, dangerous technology that exists.

13:29John Coogan:But it hasn't been deployed. It hasn't actually bombed anyone. But how do you think he's processing that book? How do you think we should be processing that idea of the government running the same playbook that they did with nuclear weapons?

13:48Ben Thompson:It's pretty interesting. I mean, on one hand, just from sort of a physical perspective, dealing with weights and software is very different than dealing with fissionable material. Or I guess the super bombs are like, they're actually like fusion devices, right? And that is trackable.

14:05John Coogan:It is interceptable.

14:08Ben Thompson:you know when Iran, to take a pertinent example, is trying to build enrichment facilities, all of which makes the problem easier to solve. So that's difference number one. Difference number two, and I really wish I had this included, I cut it so that sort of the article would be tighter, but there is a very interesting point in technological history, which was the early days of Intel. And Bob Noyce made the decision that we will sell to the government, but we're not going to design chips for the government. And the distinction there was you had guaranteed orders, which was great. The government would take your IP and there was, and in his mind, the more important thing is there was limited volume.

14:54Ben Thompson:And the way that he foresaw correctly that this was going to be a very upfront capital intensive process of designing shapes you have to design them if you have the equipment all of which is in the billions of dollars today back then was in the tens of millions and hundreds of millions is you need to find the largest possible market which was the consumer slash business market you design for that that will accelerate your improvement and your capabilities so much that you will end up having better devices than the government could have ever requested or made for itself yeah that is at stake on steroids with AI.

15:29Ben Thompson:People, like I was talking to someone, they're like, why doesn't the government just get someone to make their own model? It's like, because it's, like, you talk about government contracts, we're like single digit billions. We're talking about for the amount that's going into CapEx, the cost of these models. We're talking hundreds of millions of dollars for the models and hundreds of billions of dollars approaching a trillion dollars a year in CapEx. That is only sustainable and viable if you're selling to everyone. But that introduces the entire new dynamics where the government built nuclear.

16:04Ben Thompson:It started there and it started with a lot of assumptions because it was a government program. We are necessarily for economic reasons because of all the upfront costs entailed, starting with private companies of which the government is one of many customers. and that introduces the assumption that well it's a private company with private property rights and all those sorts of things all which I want to be true again I don't like how this is going down at all the point here is to say there's a good reason why it's not going down that way and there needs to be cognizance that even though this is a private company that is building the model general purpose and for very good reasons wants to put restrictions again i think the the same variance one is a very powerful argument that i agree with the problem is that you just need to be aware of yes the government is a small customer the government is also the entity again not to be but with guns like they you know like why do i pay taxes because the law says to pay taxes.

17:14Ben Thompson:No, at the end of the day, I pay taxes because, you know, if you really want to distill down, if I don't, someone with guns will come to my house and throw me in jail, right? Like, we don't think about that. But at the end of the day, where do these assumptions and laws and rights flow from? And as long as that is still the case, that it needs to be a decision-making factor for these companies.

17:37John Coogan:How do you think this plays out for Anthropic? It's such a small contract but it's so important in the zeitgeist there's a lot of people that are rallying around anthropic because of this there's a lot of people that are pulling away from anthropic because of this it feels like there is a business to be built that doesn't work with the government but delivers coding models and knowledge retrieval systems and a whole bunch of really valuable products and technology and it winds up being fine but at the same time you don't want this like hairy relationship with the government adversarial to go on for a long time?

18:13Ben Thompson:I would like them to sell to the government and I would like Congress to pass a law addressing these digital surveillance issues. And a lot of people are like, that's unrealistic, which I'm amenable to. But at the end of the day, if you don't have it's legal or not legal as your guiding standard, the only alternative is someone has to decide. And the implication of that not being a sufficient justification is that means a private executive is deciding. And if AI is what it is, I think that's going to be, I use this word intolerable. I didn't mean intolerable to me. I meant intolerable to those with power to have a private executive making those decisions or not.

18:59Ben Thompson:And if you think about if power, if we're going to have this very sort of brute analysis that power flows from or laws flow from power, AI is a source of power. Yeah. So it's not just that. And I think this is where the supply chain, again, which I'm not endorsing, but I think that's where the motivation is coming from. The goal isn't to find we just won't use Anthropic, I do think the goal is to hurt Anthropic. And if you're not going to be subservient to us, you're not going to be allowed to build a power base, period. And again, I'm not endorsing all this. It's just a matter of, it's not a surprise this is happening.

19:46And this needs

19:48Ben Thompson:to be just a real risk factor that has to be considered in all these decisions.

19:54John Coogan:putting on my dario hat i'm thinking about a different way to achieve the goals with maybe less acrimony and i threw out this idea that maybe the better solution is like work with the government but then lobby for a surveillance act and actually try i mean i wish the white house

20:17Ben Thompson:would come out and say yeah there's a digital surveillance problem let's work on a bit like i I don't, probably another regret I have is sort of putting this all on Anthropic. That was sort of the angle I was concerned about. And that left me, I think, fairly open to the critique that this is just like defending the White House's approach. And that was, again, that was, I was trying to be a higher level that's saying, look, this is what's going to happen.

20:40John Coogan:But yeah, the, I think there should be a way to find a middle ground here. I'm just thinking of like, from the perspective of like, if the White House is like this immutable thing, I mean, you are involved in Anthropic, like one advice would be, hey, okay, instead of going and having this confrontation with the government directly, go and start a political action committee that lobbies for change in the way that you want through the democratic process.

21:10Ben Thompson:Yes, that is the ideal process. I understand why people are frustrated and skeptical about this. I used to have this debate a lot in the context of antitrust and aggregators. And one of my sort of theses about the aggregators and antitrust is that the antitrust laws are fundamentally unsuited to dealing with aggregators. Because antitrust law has historically been about control of supply and the power of aggregators flows from control of demand. And so you end up with all these solutions that I call pushing on a string. You're just trying to get people to change how they behave. And that doesn't work very well.

21:46Ben Thompson:Like Google has always been right. Competition has always been just a click away. The problem is people aren't clicking. And like, so the solutions focused on the supply angle doesn't work in a world where the supply is there, just no one's choosing it. And therefore my prescription is you actually need to pass new laws, not try to retrofit these old laws to this new use case where they don't work. And the reaction is always, that's impossible. We can't pass new laws. and okay but realize the implications of what of what you're saying i mean i saw a tweet i again i i didn't like it so i lost it forever one of the most inferior things in the world but someone's like i would definitely rather have dario amade make these decisions then and he and to this tweeter's credit he wasn't limited to trump because me this isn't a trump issue this is a any politician issue yeah he said i would rather have amade making these decisions than whoever comes out of our screwed up democratic process yeah and points for the honesty because that's the actual choice that is that is being put forward and you could say congress isn't going to do anything therefore amade should just appreciate that is giving up on the democratic process and saying we should have unelected unaccountable individuals making weighty decisions And again, I understand the sentiment.