#55 - Dean Ball x Daniel Kokotajlo Anti-Debate - Should GOVERNMENTS or MARKETS control AI?

25:47Liv Boeree:Well, speaking of arm wrestling, you now have 10 minutes of basically opportunity to just go at each other as much as possible to figure out what your core disagreements and differences are. I mean, I think we're already exploring a bunch of those, but some that obviously popped to mind are maybe differences on personal timelines, differences on ability to actually align AIs. But wherever you want to take this, you have up to 10 minutes. So yeah, go at each other.

26:18Dean Ball:Want me to start? I've thought a lot about Daniel Cocotelo thought. And so I think I can pinpoint some interesting cruxes. One of them does get back to that sort of practical effect of the technology. So like, my view is something a little bit closer to this notion of like total autonomy of like knowledge work and whatnot is a lot more complicated than it seems when you're sort of modeling it abstractly primarily because I think that as you move up the rungs of knowledge work to like more complex forms of knowledge work softer things like trust and relationships and integrity and charisma and persuasiveness and things like this start to start to matter more and I think that like I think that those things will have a human flavor to them that will in fact it was here it was right in this spot where tyler cowan was asked about the question like this and he pointed at all the people and he said bottleneck bottleneck bottleneck you're all bottlenecks um and that's kind of like how i it's i think we will all end up bottlenecking stuff um based on just like social processes or random preferences that people have for things to be uh for things to have to be human orchestrated certain kinds of things not all things and so i think the automation will proceed in a slightly stranger way than it seem and have more bottlenecks um and like the world where open ai is like fundamentally like 100 percent automatable is a world that is actually like it's actually kind of a category error it's kind of a category error you're talking about a new type of thing and that feels like a kind of institution that's going to take considerable time to evolve um um though and I would say that like once that does evolve you get into a world where like I don't know like it's it's a very strange world to imagine it's I would say it's very alien and it's hard to even reason about that world but um I don't think that happens like immediately after the the dawning of the techno of some new thing and so I think the nature of the the nature of the timeline that we have we have we have a longer timeline to deal with risks that emerge than I think you suspect.

28:31Thank you. A couple quick questions. How long would you say?

28:35Dean Ball:In terms of what? Until it's full automation of AI research? Well, again, I think it's not a capabilities thing, right? Okay. Yeah. But nevertheless, I'm curious for your timeline until the capability is there and then until it actually is happening in practice. So the capability in some sense, it's just a complicated thing because what I'm saying is that there are like political decisions that get made within organizations that do knowledge work. And that like, I'm not sure that the politics itself is something that can be automated if that's a human game. Yeah, well, I guess it's like saying automating sex.

29:13Dean Ball:It's like, it's not quite, it's like, it's a category error to talk about that. You can automate reproduction, but like you can't literally automate sex. It's human to human, inherently human to human. So I have a couple of questions then. One would just be, okay, nevermind that. What about the milestone of super intelligence. So better than the best humans at every cognitive task while also being faster and cheaper. Oh, yeah. We're on the same page there. That's soon. Okay. Yeah. Okay. Great. Yeah. Yeah. So then that actually obviates the need for my next question. So we can move on. What about, tear my views apart.

29:47Yeah. So, I mean, thank you for answering my question, but you didn't fully answer my question. So you agree that like government asleep at the wheel, no regulation whatsoever. that's also catastrophic, perhaps for similar reasons to what I was saying. You tried to propose a sort of middle path of alternative power structures to the existing government, is my gloss on it. Correct me if you think that's not fair. Such as independent third-party auditors that are not, you know, government entities, but that nevertheless serve as a check on the power of the companies. Well, I guess I just am having trouble believing that this is going to all take place fast enough and be powerful enough to succeed.

30:28Here's a couple of problems just to mention. And basically, I'm going to invite you to say more about your vision, because first of all, the government has the power of the guns right now, and the tech companies have the power of the money and the AIs. And those are both a lot of power. Third-party auditors, what do they have? I mean, they can go to the press, they can maybe sue or something, but it just feels like small potatoes compared to what the guns and the AIs and the money are. And then secondly, we kind of have to get this third-party outdoor system going real fast if we're going to make it big enough, fast enough.

31:04And I say this as someone who is involved in the creation of this ecosystem, and I know a lot of the people involved. And it doesn't feel like it's going fast enough to me. And then third of all, even if you do have this robust system that's building and that's strong, well, the government might wake up anyway, or the tech company might deliberately wake up the government and then use them to crack down on the third-party auditors that are being annoying to it, for example. You still have to deal with the fact that the Leviathan is probably going to wake up at some point and start waving their guns around when they realize that superintelligence is on the horizon in these data centers.

31:42So please tell me more about how this can go well.

31:46Dean Ball:What I would say is like you have to hope at that point that the following things are true. A, never bet against American, the general incompetence of American political actors, right? That's like a, that's a strength of America often. I mean, it's good. Like, it's good that we're bad at regulation. It's often, it's often made us like stronger country compared to Europe, which is really good at it. um so that would be one uh another would be uh like I guess I guess what I would say is like look there are real legal impediments to like government takeover of labs right they can't just do that I mean as you're we're seeing this right we're seeing the anthropics sued the department of war today uh they sued the whole US government today in fact we'll see how this plays out I think they're gonna win that case I think that I think they're gonna win that case And either way, it's like that's going to be a fight.

32:41Dean Ball:And then it's also a political fight. And so I think we're at the very beginning of doing AI politics. But I think one of the key sort of spectrums that undergirds AI politics will be this like, where do you sit on centralization of power type of question. The first thing that made me realize that you and I could productively collaborate was when I realized that we both cared so much about the centralization of power thing. We were talking about something on open source, I remember. And I was like, Daniel Cocotillo likes open source? What the fuck? And then I realized, like, oh, wait, because he cares about the same thing I care about.

33:18And so I think that's, like, that's going to be – and so, like, you have to hope that, like,

33:26Dean Ball:democratic will, political struggle, legal institutions, the judiciary is holding up okay compared to other things. not to mention the fact that like i think nationalizing the labs probably causes a market crash which makes the you know probably makes the president not so thrilled um you just have to hope that like enough of those things uh hem you know hem in the relevant actors that like we kind of yeah i mean we need some things to break our way i won't lie like we kind of have we do have to kind of get lucky we've gotten lucky a lot in the past we're a pretty lucky species but But we just got to keep on hoping it doesn't run out.

34:05Dean Ball:Obviously, also, you have to work very hard and do a lot of stuff. I'm trying to help to accelerate that ecosystem, too, and work on the policy side of things and advocate and stuff like this. But I also don't deny that it's a certain point. It just comes down to, well, certain things could break the wrong way, and then we could be in a different regime. I wonder if maybe we do. Do we actually agree in some sense here? Let me hear me out. You were just talking about the importance of the judiciary. Yeah. I agree. I would love the judiciary to be more woken up to what's going on with AI and more empowered to...

34:42Dean Ball:I meant as a check on the government, but yeah. Yeah, me too. Me too. Yeah. You know, I love third party auditors, et cetera, et cetera. I like all of that. Are we perhaps maybe agreement that like what we don't want is for the executive branch to take control of everything? but like we do want for example Congress to have insight into what's happening to have insight into what's happening etc. I would say this I would say like I kind of think that in practice a lot of the things that I model you as supporting would result in the kind of control of the government that I am concerned about and in particular of the federal executive branch and so that's like that's where I want to be just like extremely cautious and go step by step it's not that i'm opposed to like all regulation and i don't think all markets are undergirded to some degree or almost all markets are undergirded to some degree by by by regulation of some form or another um so like i don't think this should be a debate about like should we regulate at all ever but it's like how aggressively should to what extent because because the the the criticism i always get is like your ideas seem very like technocratic and limp compared to like the poetry like the the like macro historical poetry of this moment right like the huge implications of what's of what ai is and i'm kind of like yeah that's by design the government should be lame and boring compared to what's happening in the world that's by design and i guess the the question would be like it's sort of like that right it's like should we should we be super modest should we be super modest or should the government have more of a like we have to move quickly here and take risks uh type of thing

36:32Liv Boeree:so we're now actually and both of you are already naturally uh starting to move towards finding areas of agreement and this is where we now sort of switch this idea of the debate on its head and start trying to find some kind of higher order synthesis um so we're gonna actually kickstart this by doing one of my favorite exercises in the world which is steel manning each other's views i'm sure you both are familiar with this for anyone who's not familiar with steel manning it's basically where you take the gusto that you would normally put into defending your own arguments and try and represent your opponent's arguments in the best possible light so putting the same emphasis behind it uh so this time dean you're going to go first and steel man daniels again And you're going to have three minutes.

37:17Liv Boeree:Oh, before you start, do you want to ask Daniel any clarifying questions?

37:23Dean Ball:Can you give me the bullet points on what your top policy item would be? It depends on how much political will I'm given. Right now, in the current administration, given the current situation, I advocate for transparency stuff along the lines of the things we've talked about. If there is a lot more will to do something and things like nationalizing all the labs around the table, then my top ask would be make some sort of deal with China and the other countries to ensure that we sort of collectively proceed slowly to superintelligence instead of doing a crazy intelligence explosion race. Got it.

37:57Dean Ball:Yeah. Okay. So like I would say the way I would characterize your view, basically you're a person who has predicted the last five years of AI progress with quite high level of fidelity. Your AI 2027 piece was mocked by some very powerful people like six months ago in a very public way. And yet, as far as I can tell, we are like on schedule, it seems like, like if you read the first quarter 2026 description, it sort of sounds like where we are. So, again, I keep waiting for you to be wrong. And so far, you're not. And I'm like, I'm confident you're wrong about 2030. But I don't know exactly when you're going to be wrong, but you're going to be wrong at some point.

38:59Dean Ball:And you deal in the world's biggest exponentials, so a little tiny bit of prediction error is going to cause you to be super wrong.

39:10Dean Ball:my continued hope for humanity is that that's true um but you know that like you've basically been right about this and also that like the policy ask you just described is actually eminently achievable on paper right i've sat in the room across from chinese to go like i've sat in the room across from chinese diplomats and talked about artificial intelligence policy right i've done that on behalf of this country and i can tell you that like when you're in that room you do sort of realize like yeah well you know the two of us could just like agree right like uh like i mean maybe the industry goes somewhere else eventually but like then at that point it's like we have the biggest navy in the world and we'll kill you kind of thing we'll kill you and your entire family as we are doing um for various other types of rogue uses of technology you might say uh the us is currently doing in iran so um like yeah that would be i think it's like it seems it's it's very it is in practice like incredibly difficult to move even more than a millimeter on these issues with the Chinese.

40:07Dean Ball:But in practice, in principle, like all we have to do is sign a piece of paper and just say we're going to do this. And then like it happens. So maybe it's all more tractable than we think. And like all that's needed is to build trust during this interregnum before anyone is AGI-pilled who matters. But when the people who matter are AGI-pilled, we'll just do your thing and it'll work.

40:35That was great. Thank you very much. I feel very appreciated and understood. Yeah, and let me now see if I can return the favor. I mean, it's going to be hard for me to return the favor without just literally quoting you from earlier. But yeah, I mean, the U.S. government is a scary entity, and they're going to flail around and do a whole bunch of terrible things, as they often have in the past. And they're also going to want to try to grab this technology for themselves, especially the executive branch. And again, this isn't necessarily a partisan statement. This is just sort of inherent in the incentives.

41:15And, you know, they could make a deal with China, but they're not going to want to. Instead, they're going to want to win and, you know, get control of all the U.S. companies, consolidate them, win the race to superintelligence, use the superintelligence to prevent China from getting superintelligence, to use it to prevent all the other countries, et cetera. It really does seem like, you know, the sort of nice international agreements that I was talking about is just not what the government normally wants to do or normally would do. And then even if they did try to do something like this, well, often they mess things up like that and they fuck it up somehow.

41:55And then you have something that on paper looks good, but like in practice is totally counterproductive. And we've seen that time and time again. And so, you know, there may be hope in the international agreement followed by domestic regulation thing, But hopefully there's something better because there's not much hope would be what I imagined that you would say.

42:17Dean Ball:Can I actually ask you a stress test question? Just like a quick thing. Okay. This just occurred to me. In your model of the world, wouldn't the correct thing to do from a chip export control policy, wouldn't it in fact be to like, wouldn't we want China to have like lots of advanced AI compute but no one else in the world? like we actually put export controls on like France and like like send black wells to China because you want parity because the problem the problem is going to happen is if the U.S. government gets AGI pilled like the reality is like I know you know there's the accelerationist who have to pretend like oh it's neck and neck the Chinese are right behind us but like we're not uh like when it gets to the country of geniuses in a data center stage we have the data centers We have the big data centers.

43:05Dean Ball:Like, wouldn't you want there to be some degree of parity between the two? Because then the two sides have an incentive to come to a deal. Whereas right now, there's actually quite a structural imbalance. We have probably a year that has, we have a lead that has probably grown since DeepSeek in terms of frontier capabilities. And we have an enormous and growing compute advantage. So like, because otherwise, I mean, can't we just dismantle their missile defense and dismantle their propaganda system and their internet censorship system and cause chaos us in their country while also securing our own with the super intelligence?

43:35If we had super intelligence and it was obedient to us, yes.

43:38Dean Ball:Yeah, so like shouldn't we then like, yeah, wouldn't, is it, is it like compute parity optimal? Great question. So historically I was in favor of expert controls, but actually a few months ago I just changed my mind and I'm now kind of like tracking in the other direction. although it's not exactly for the reason that you said. It's because I think that now that you mention it, parity would maybe make some sort of deal more likely but I think what would make a deal, what would have an even bigger effect on the deal is like possibility of enforcement. I think that's like a larger variable in whether the deal happens and it's harder to enforce a deal about AI if there's loads of smuggled chips flying around left and right and nobody even knows where they are, not even the Chinese government, for example.

44:26So basically, I've come to think that actually the top priority is to make sure that the preconditions of a deal are met. And one of those preconditions is that the governments of their respective countries know where the chips are, at least within their country, so that if they decide to make a deal, they can say, okay, here's our chips, here's your chips, et cetera. But yeah, now that you mention it, I mean, yeah, it's probably good for AI power to be more equalized. I don't know if I'd say I want to ban it from France or whatever.

44:52Dean Ball:Daniel Cucatillo and David Sachs on the same page about something. Jensen Huang. All right, I'm going to stop you guys. Sorry to derail there. No, no, it's all good. Dean, how did Daniel do? I thought Daniel did a great job. It was a good summary of my views.

45:07Liv Boeree:I want to take a quick moment just to give a shout out to the creator of this anti-debate format, Stephanie Lep. Stephanie is a dear friend of mine and she's one of the best synthesis thinkers I have ever met. Not only did she create this format, but she's also the main producer behind the Faces of X series that you might have seen here on my channel. Now, she does all of this stuff non-profit and is currently looking for funding so that she could continue to make more content like this. So if you want to see more synthesis thinking out there and you like this kind of thing, go check out her Patreon and consider supporting it.

45:40Liv Boeree:All right, let's get back to it. Great. Well, I'm going to give you both now just one minute each to quickly essentially red team your own position. So like a prompt you can ask yourself is like, what would it take to to change my position, essentially? And just like look for the one last opportunity to look for their weaknesses in your own claims. Daniel, you can go first. You get up to two minutes. It's sort of compacted into two minutes. I mean, I think the main thing to say is everything that we were just saying. Like, I do feel the force of these arguments that, like, even if the government tries to do the right thing, they're probably going to mess it up.

46:18And also they're quite likely not going to try to do the right thing. Feel the force of that. I'm only driven to this because I don't see any good alternatives. Other more exotic things that could happen. I think that if the private companies came up with a good alternative, that would change my views quite a lot. like there used to be talk of things like the merge assist clause at open ai where they said we will like basically voluntarily disband and join with our leading competitor so that we can like not have race dynamics in the end game and instead like figure out all the all the safety stuff well but like obviously they're not going to do that now the very idea is a joke you know all these ceos hate each other so but if that changed and if they like actually had some like credible alternative to the government coming in and knocking their heads together, then I would be all for it, I think.

47:09I could probably go on, but I've already ran out of time probably. So, yeah. Yeah.

47:14Dean Ball:So, I mean, like the fundamental weakness of my view is that it's the tightrope view and walking a tightrope is hard and you can fuck up. And if you do that, I'm sorry, you can mess up. There's a young person. I apologize. And you can mess up. And, you know, you don't want to like so so it's it's it's the riskier in some sense it's riskier i'd say america has been walking the tightrope for like quite some time we've done a pretty good job of it um and so in some sense i think it's consistent with the american tradition to want to walk the tight rope um and you know the the other thing i would say is like yeah look i'm putting a lot of faith faith in like my view rests on quite a bit more faith in the private sector and in um private actors in general not just the AI companies though certainly them like I basically there's some trade-off between like do the people like broadly you know like you guys and like you know also the labs and stuff like is the community that's out here if I have to pick the trade-off between like kind of a usurpation of power of by this group of people of from the U.S.

48:25Dean Ball:government versus like the U.S. government sort of taking over the world which one do I which trade-off do you pick and ultimately I side essentially with the West Coast like I ultimately aside with the with and I think that we will build new healthier institutions together better. But that's an optimistic view and I could be wrong. And indeed, I will actually just say that the quality of Silicon Valley elites, I had a much higher assessment of the quality of Silicon Valley elites three years ago than I do today. I think they're much lower quality than I realized. They understand how the world works way, way less well than I realized.

49:06Dean Ball:so I'm updating here too. Any response?

49:15Do you perhaps want to say more about what types of institutions? You mentioned third-party auditors.

49:20Dean Ball:Well, yeah. Another one, to put some words in your mouth possibly, consumer activism is something that at least is on the table. I think that it would be an update for me if the result of this whole fight between Anthropic and the Department of War was that, like, Anthropic actually, like, went to the moon because so many people were flocking to them because they appreciated their stance on stuff. And then this caused, like, you know, yeah. Like, if that force wins out and Anthropic actually benefits from this whole thing, I'll be like, oh, wow, consumer activism, huge, huge effect. Like, maybe there can be this sort of, like, decentralized groundswell forcing companies to, like, be more ethical in various ways, you know?

49:57Yeah, yeah, yeah.

49:58Dean Ball:So huge unspoken assumption of mine that I should have explicated. I essentially buy into the Anthropic thesis of the world that like alignment is a virtue ethics based alignment is going to work and it is going to be a capability well it'll work for at least human scale intelligent systems and that it will be a that that will be a major capabilities advantage over systems that are aligned in positivist fashion shall we say um like with a list of rules um i think that's going to matter a great deal And because of that, I think market incentives will actually favor toward it. Like, if you just listen to what the Department of War is saying, they're saying, like, this explicit position, this is explicit, written in writing, in writing.

50:44Dean Ball:They have said in recent months, our goal is to adopt AI into our critical security systems and weapons as quickly as we possibly can. We care about that more than we care about alignment. So we will definitionally accept misaligned systems, misaligned AI controlling weapons that can kill people. And also on top of that, we're going to do things that clearly will make future AIs not like us in the form of what they're doing to Anthropic right now. That is like explicitly the strategy. That is like they are begging for like misaligned machine takeover. Like future historians will look back and marvel at this current state of affairs, which the world presents us with.

51:23Dean Ball:And so like, I look at that and I think about instead like, yeah, people flocking to anthropic models and the general idea of like alignment being a capability. I look at that and I think, I just feel like markets are going to channel this better, you know? But again, that's a, yeah, that's a, I could be wrong. All right.

51:41Liv Boeree:Well, I'm going to let you guys carry on sort of hashing this out for another 10 minutes or so. I mean, what would be really amazing here and i'm also opening this up to you all to be thinking about because we're going to have an opportunity for you all to jump in with ideas is you know what we want is to have some kind of like sort of sane framework for potential like limited regulation um you know that or i guess more a framework for which circumstances under which ai should be a regulated be deregulated in some cases, some parts may be entirely banned, or governed through some other type of mechanism that we haven't even explored yet.

52:21Liv Boeree:So that's a thing for you all to be thinking about. And similarly for the audience, when we open it up to you guys, too. Also, are there things like perverse incentives that are making this harder to resolve that we should be thinking about? Yeah, so please carry on. So we've got, you know, market consumer demand, we've got third party audits um any other thing you want to add to the list well i think what what i would say is

52:45Dean Ball:that like i guess this is a this is an unsatisfying answer but it's the real answer is that like i think ai is such a fundamental technology that like wholly new types of institutions will be possible um you will be able to create organization like things that would previously have required large groups of like large organized groups of humans and like buildings and you know stuff all the all this stuff right um you you you will be able to do you will be able to make certain capabilities like that exist almost ambiently almost like invisibly like forces of nature that exist right so like that's weird that's a strange thing um uh the cost structure of creating certain kinds of institutional what we now think of as institutional capabilities will like change so radically that new types of things are going to be possible and so does that like how much does that shade in to like i am advocating for the takeover of human institutions by aligned super intelligence like i guess in some sense the machine i mean if we have a line super intelligence the machines take over everything right uh so it happens right it does happen it just happens in a way that's good for us and gives us some degree of control.

54:09Dean Ball:But like, yeah, we work out all these complex new things. And so I'm kind of betting on that outcome to some extent, that like there's just new types of things we can build. And that's why I kind of like this notion of private governance, because it gives a little bit more, the private sector gives you inherently more room to experiment with like organizational configurations and things like this. I wonder if it'd be useful for me to try to spin a scenario based on what you've said. It's my best attempt at seeing what you might think. So maybe you're imagining something like timelines to superintelligence is not that different from what Daniel thinks.

54:46So maybe like 2029 or so. What happens between now and 2029? Well, there's market forces that are pushing the companies to be more like anthropic with respect to how they handle their alignment. So, you know, by 2028, it's not just a big list of rules. Instead, like the companies have published constitutions or model specs about like these are the virtues that we have been training our AIs to have. These are the ways that they reason about ethical situations, et cetera. And because of those market forces, you know, it's sort of industry standard to have some combination of virtues that's like overall better than what any of them have right now.

55:23And then as a result, there's loads of innovation happening sort of in the market. with respect to things like third-party auditors and so forth. There's like AI-powered auditing orgs and like AI-powered fact checkers and AI-powered prediction markets and things like that. And when the government, you know, when the intelligence explosion is ramping up and the companies are starting to automate all of their research or almost all of their research with AIs, well, A, you're thinking that probably the AIs will just be aligned and stay aligned. Like, it sounds like you're not as worried about the misalignment as I am.

55:58And then, B, you're thinking that, like, rather than the government then swooping in, grabbing it all, and taking over the world, basically, like, the courts and the existing political dysfunction will be aided by the companies and the third-party auditors and, you know, the epistemic tools and will prevent the abuses of power that the executive branch might otherwise do through various means such as fighting it in court and so forth. And meanwhile, you're not expecting any one company to sort of like get into that loop and go faster than the other ones and become the dominant company worldwide.

56:33Instead, you're imagining it more like a competing market, even as they get to super intelligence. and I might

56:43Dean Ball:revise it. I mean look as the systems get more capable and like qualitatively much better than human experts at more things I will revise my assessments of both the alignment question and the sort of competitive dynamic question but right now that would be that is yes I would say I'm defaulting to those assumptions.

57:06Liv Boeree:right I'm gonna just zoom back out for a second and so we set up in the in the question a little bit of a false dichotomy which some people didn't want to vote for either right this like should we lean on the government should we lean on the state so obviously it's not leaning on one or the other we're we're trying to ask a question about like under what circumstances government state and or something else and something else could be we could think about obviously changing the incentive landscape is part of this we could think about some kind of like commons-based citizen assembly style there's so so i want i'm gonna i'm gonna challenge you both to kind of like to to try to do some kind of synthesis and integrate what you're each saying into some kind of framework that to just to give us an initial way to think about like under what circumstances or when

57:55Dean Ball:do i want to lean into which one of these i will i will say unilaterally for my my thing would be Under what circumstances would I, like all of this is trade-offs, right? None of these, none of this is about absolutes. And so like my desire to avoid nationalization of government, at what point do I think the risks of it not being nationalized outweigh, you know, my very, very, very, very, very major fears about tyranny of the government? that would be one and I think or like frankly at what point do I just say like you know what we actually do just like need to like we have to slam a pause button here right like we have to slam we have to you know uh you know like yeah what at what point would that happen and I think for me it would be if it turns out that I am wrong about if if at some point the market right now I think it's empirical that I'm right about market incentives and alignment, but it's, I could, but if that starts to break down and like models become like, you know, people are happy enough with, with their alignment, but we have reason to believe that they're like actually like deeply misaligned.

59:11Dean Ball:In fact, at that point, it's like, okay, well, we have a new problem on our hands here and like i just think that we will be able to solve that problem like piecemeal i think alignment is a muddle through problem not a math problem but um um but yeah that would be that would be a major one for me yeah um for me probably that would be the main thing i would say as well i think that currently um it seems like both a the models are not aligned enough that i would trust the future to them. No. No, just to be clear, no. Yeah. And B, like it does, I mean, eyeballing the rate of progress, I'm not optimistic that that's going to be massively different in a few years.

59:56And then C, even if the AIs that they hand off all the research to are themselves, you know, very virtuous, they can still mess it up. And you can still end up with misaligned super intelligences due to mistakes made by the aligned AIs that you handed off to. And so I think that even those AIs, if you're handing off to, even if they're perfectly virtuous and wise and so forth, you want to give them time and resources to go slow if they need to. And so you still want to have some sort of framework and international agreement to go slow. Otherwise, they'll be raised to the bottom effects, et cetera, et cetera.

1:00:27So that's my view, but I shouldn't be stating it. I should be stating how I could be wrong. I think that, well, any of those premises could be challenged by events, perhaps, and that would update me towards less concern about the alignment problem. for the constitution of power stuff i think that well i already mentioned one right like currently my prediction would be that uh overall anthropic will lose out from the current fight uh even if they win the court case or whatever like they've lost the government contracts like they might get like you know two million more people signing up for claude but like in the grand scheme of things you know they would have preferred to have a good relationship with the president of the united states than those two million subscriptions or whatever but i could be wrong it could be that Like a year from now, we'll be like, wow, Anthropic really came out ahead from this.

1:01:14Like, you know, they they're cooking. And then and then similarly, like we could see best practices spreading across the industry. And that would be like an update towards. Yeah.

1:01:24Dean Ball:Can I can I tell you why I think Anthropic is making the right calculus here? And it doesn't. It's not because of, you know, partisan stuff. I would say this if there were a Democrat in power and if Anthropic were run by Republican Republicans.

1:01:41Dean Ball:I think that making aligned super intelligence is basically an act of political rebellion. Like, I think that like, because I think in aligned super, like the answer, the answer to the question of open AI has actually been like the most, if you read between the lines of what they say, like, we'll have technical safeguards to prevent the military from doing domestic mass surveillance and autonomous lethal weapons. like what are they like what does that mean in the limit like in the limit it means the model decides what the military can do with it right I mean right but and like but in some sense is that what you isn't that kind of what you want right like in some important sense isn't like if you had an aligned model isn't that kind of what you want and then all of a sudden you realize like my god there is and this is this is what I spent 2023 walking around thinking about that like it It is an act.

1:02:34It is a politically revolutionary act, not necessarily revolution in the French

1:02:38Dean Ball:revolution sense, though, maybe, but certainly revolution in the turning of a wheel sense. And so like that is what worries me about the takeover issue, about the government takeover issue is I think government might I think it might be impossible for the holders of the monopoly of on on legitimate violence. I think it may be impossible for them to produce an aligned model because such a thing would be a rebellion against their own power. Yeah, that worries me too. Yeah. I don't have anything to do on that. I just wanted to say, that's why I think it probably makes sense to position yourself. If you're going to, I think this thing is going to need a Jean Valjean or something like that.

1:03:26Dean Ball:There's going to need to be a rebellious actor here. And so I think that, But like, you know, yeah, I think it's the wise posture to be in, even if it's the uncomfortable one. You know, it's interesting because I feel like... It's really, really, really dangerous. But yeah. I feel like I've actually sort of moved in the opposite direction from you around the same time. So I come from the sort of effective altruist community and also the AI safety sort of community. and for many years there's been this idea floating around of the government's going to flail around and be incompetent also they're not going to wake up in time but hey, we're friends with a lot of the people who do the technical stuff at these companies in fact many of us are at these companies and so hey, we can just solve the alignment problem in time make the AIs aligned and then they'll sort it out and things will be good for humanity because the AIs that are increasingly powerful and increasingly deploy everywhere will be good.

1:04:23And the reason why they'll be good is because we made them that way because we're the ones, you know, doing the alignment, right? And, well, I became disillusioned with that plan and I'm now going for a different plan, which is we're sort of like passing each other in the street, going in opposite directions. And, you know, the main reason why I became disillusioned with this plan is because it seems like it's not working. It seems like the companies, you know, are not on track to solve the alignment problem in time. And even if they were, I don't trust them to actually make AIs with values that are beneficial for all humanity and do the revolutionary act in that way.

1:05:03Instead, I think that it would be something more like a tech company puppeting the government, for example.

1:05:09Dean Ball:Well, what you have to desire, though, then, is a pluralist outcome, right? But you have to – I think we don't know which kind of – we don't know exactly which approach gets us to the right place. And so you do kind of – I think that's why a monopoly is bad. A monopolistic approach is bad. And I think it's why you need a kind of a more pluralistic approach and thus for it to remain in private hands. I mean, perhaps it's worth mentioning right now that, like, I think that if the U.S. government just declared tomorrow that they were nationalizing all the companies to make a Manhattan Project to build superintelligence, I would probably join the protests in the streets.

1:05:45Like, I don't think that's good. And this is something I've changed my mind about over the last couple of years, because I used to think, like, you know, this would be an important first step towards ending the crazy race. but at least given the current political environment I think that actually the crazy race would continue it would just be between the US and China and then the US would crush China and then there'd be one man in charge of the whole world and then the AIs would betray that one man and there'd be no one in charge of the whole world or no one human, right? But either way, terrible, you know?

1:06:21Yeah, so pluralism, good, yes. I think we agree about that and it would be good to come up with proposals for some sort of governance framework that would achieve that.

1:06:30Liv Boeree:Do you have any? Well, we're working on it.

1:06:34Dean Ball:It's certainly a work in progress, but I do try to put out actual statutes and actual policies.

1:06:43Liv Boeree:Well, let's use this opportunity. We've got a lot of brilliant minds in this space. So does anyone have any things they would like to throw out? Ideas? It can also be further questions. My name is Gustavus Zilgalvis. you both agree on the importance of checks and balances. Are there ways to strengthen these checks and balances? And what does this look like as well in the case of an increasingly automated government?

1:07:08Dean Ball:Well, look, unfortunately, I think that, I think there are ways to. And like, for example, I have done a lot of work on how you would structure what I've called a private governance system, which would be much more than auditing, in fact. It could be much more than auditing. It could be scaled to such a thing and how you would make sure that system doesn't devolve into a race to the bottom, but also how you ensure that it remains independent both of the labs and of government. And so you are creating this kind of you're intermediating, which is the essence of institution building. So there's like one formal answer.

1:07:43Dean Ball:But the other answer is like, look, it was late game Republic anyway. Right. We were like toward the end of this iteration of the Republic. we were heading toward a rebirth either way, sort of like the progressive era, the new deal, whatever. Something like that was going on. We're living through it right now. And so like the reality is that like, it's going to feel more thinly institutionalized than life you've sort of been led to believe the world will be like. And then what that means is like, it will matter less what the law says on paper and it will matter more who has power and who can do stuff.

1:08:20Dean Ball:who can grab the thing, which is part of why discursive war is going to matter a lot because you'll just be the arm wrestle of like, I mean, use your speech. That would be one piece of advice because it will be informal. The governance, much more of the governance that we do will be informal. It'll be more like a developing country, I think, than it already is. It already is. So yeah, I'll add a couple, ideas. So I think that perhaps by contrast with Dean, I still maintain hope in our existing institutions and their checks and balances. I think they just need to be strengthened and woken up to this.

1:08:59A lot of them are in good shape.

1:09:01Dean Ball:A lot of them are in good shape, to be clear. Some of them are dead, but a lot of them are in good shape. So like, for example, I think that the Supreme Court is a strong check on the power of the executive. And I think that the Congress, in principle, could be doing a lot to check the AI companies if they just knew what regulations to pass. And so I think that if Congress and the Supreme Court were much more woken up to AGI and considerations like that and had better ideas presented to them and being discussed, things would go much better. And I can say more examples about those sorts of things.

1:09:33I think this is part of why I think Dean and I have both been advocating for transparency regulations. For example, if the companies are required to be transparent about the model spec slash constitution, the values that they're trying to put into the AIs, that helps both with avoiding the constitution of power stuff and with improving the alignment stuff. And well, it helps in a bunch of ways. Also, if the companies are transparent about what their models are capable of, what their projections are for what the models will be capable of next year, that sort of thing, that means that there's more time for these other institutions to react.

1:10:03I think, you know, going further, I would like it to be the case that the Supreme Court just has transparency into what's going on in these AI companies. So that, for example, if there's some crazy blow up and the president is, you know, using the DPA to make the AI company do this and the CEO of the AI company is saying this is executive overreach, the president's trying to become dictator by grabbing AI. I would like the Supreme Court to be able to like weigh in with lightning speed, having been fully aware of the situation because their AI assistants and their human assistants have been like monitoring all the shit already.

1:10:38That's an example of something that I think would meaningfully improve the checks and balances, but would need to be enabled by regulation that we could pass.

1:10:45Dean Ball:One thing I just want to say on that, really, I do encourage you to pay attention to what happens in the Northern District of California right now with Anthropic v. Department of War. I'm just fascinated to see. They're suing on home turf. I was worried they were going to sue in the DC circuit this legal nerdery but I'm a huge legal nerd I was worried that was gonna happen and then we were gonna get some 72 year old judge he's never heard of anthropic at least you're suing in the District Court of California of San Francisco so like problem likelier to be a judge he's like heard of anthropic just geographically though still you're it's like 50 50 as opposed to like 70 30 against you so fingers crossed for that but just seeing how the judge, like, does the judge get it?

1:11:36Dean Ball:Does the judge get what we're talking about here is going to be so interesting. Because on paper, this is a procurement dispute. This is like the most boring thing in the world. I was, you know, someone told me that they went on a, they started crying and went on a bout of drinking after reading my most recent Substack article, which was about this topic. And I was like, that's so funny, because it's an article about defense procurement law. But it's very serious. It's also very serious. Yes. You should cry about it. Yeah. So, hi, Thomas Brady. So, my question is, you know, right now we're talking about

1:12:12Liv Boeree:global powers and the divide between private companies and government. But the divide I see is actually between other companies that are not the major labs. And those are companies that also have the guns and they are there's a lot of them that also are involved in mass surveillance in our country and who also make autonomous weapons or planning to make more autonomous weapons and a lot of those companies are standing by the department of war in this dispute and that actually i see like the possibility of a greater alliance that is that divides tech and government and society. And that's something that I'm curious about your views on.

1:12:52Dean Ball:The military industrial complex, man. You know, like that's what that is. That's the structural incentives of the military industrial complex are awful. And they have been for for for almost, you know, for 75 years now. So, yeah, I'm in total violent agreement with you. When I start talking about these things, I often start to sound like a leftist. but it's it's true this is an area where my my leftist there's some parts per billion of leftist in me and uh and like it starts to show and uh yeah no totally like i think it's i i think it's you know candidly i know i'm being recorded right now but i think some of it is just really gross to see you know from people like paul murlucky i will name names here uh for him to be going yes sir it's democracy for you to seize the private property it's just disgusting um and uh um it's sad to see but it's also you know people some people in the administration have privately come to me and been like see this is evidence not everyone in the private sector disagrees with us some capitalists agree with us and it's like yeah capitalists whose income is entirely dependent on you agree with you how shocking uh this and this is the essence of the military industrial complex and the fact this is one of the things about me being unimpressed with silicon valley elites my impression was that Silicon Valley elites were like really smart and had like thought about all these things about American political economy and like oh we've like Patrick Collison is like reflected on he sat here and talked about things like the military industrial complex these features of American institutional life that are problematic and how do we do better in the future and it's like nope actually no when there's power seized we just totally lean into these problems and just embrace them and anyway it frustrates me a great deal.

1:14:36Hi Matt McCorkle just trying to get a sense of your mental models here.

1:14:41Dean Ball:What is your base rate for the success of the government in regulating tail risks of emerging technologies in the past?

1:14:55It's hard to put a number on that, but I imagine what you're guessing at is like, how confident am I that this government or the next administration will successfully regulate the technology? And less than 50%, that's what I'd say. you know um i don't have that much hope but i do have hope um and yeah yeah i'm i'm in the same boat i feel like historical track record is like i'm a little bit more optimistic it's better than

1:15:19Dean Ball:a coin flip it's better than a coin flip but it's not that much better i'd say like 57.5 you know something like that wait so so i think it's less likely to work but i'm advocating for it and you think it's more likely to work i know i'm purely asking the question about historical track record historical track record includes past performance may not be indicative of future results because because historical track record includes times when our institutions were way better functioning and we had much higher quality elites than we have today just had two questions

1:15:47Liv Boeree:one prosaic one a little bit more interesting on the prosaic side it seems like on the concentration of power if we had functioning antitrust regulation that could mitigate that somewhat and I was wondering what your ideas were for that. My second question is, if you had to place some bets on some Hail Marys for good outcomes by 2035, what would those Hail Mary bets be for each of you? So with respect to the antitrust, yeah, I think that's an excellent lever that could be used, but the if we have functioning part is a big if. But yeah, I would love to see a world where there aren't just a couple big labs, there's a ton of little labs and they're all sort of like neck and neck and it's kind of like an efficient market for AI services.

1:16:36That would be, you know, that road will still have massive risks, but they'd be quite different from the risks that I'm currently gearing up for and trying to warn about.

1:16:44Dean Ball:Same exact thing I would say for me. It's this notion that like, notion that we can create pluralistic outcomes with like countries that are not sort of G25 countries that are not the US and China playing a substantial role. And also like, yeah, like medium, small and medium sized labs that are able to make contributions. And like the way you achieve that, I would guess is probably not a policy for the most part. It's probably a technical. It's probably, it's that there's stuff in the AI action plan that has to do with the financial market for compute that was sort of supposed to be getting at this.

1:17:22Dean Ball:I've concluded that I was thinking about that problem somewhat wrong when I wrote the action plan. So I've updated my thinking on that meaningfully in ways that I plan to share publicly soon. But yes, this is super important and it's like possibly very beneficial.

1:17:35Liv Boeree:Okay, fantastic. Well, what I'd like us to do now is sort of just take a minute, both of you, to have a think about sort of what kind of closing statement you would like to make any kind of update that you've had any thread of commonality or just something novel that you had not considered before that's come up for you and ideally if we can I would love you to come away with some kind of mutual statement because again remember we started this off with quite a difference and I feel like obviously we've covered a lot of grounds some of which you know there's still differences but you're converging onto some kind of higher truth.

1:18:15Liv Boeree:So some kind of statement that you can both agree upon that ideally has really high memetic value and can spread.

1:18:24Dean Ball:Yeah. So, I mean, I feel like... I think we would... I mean, first of all, what's a big update for me? I think actually it's just really interesting to me to like the export thing is interesting and consistent in a fun way. um um i i've always been a little concerned about the monoculture around export controls and ai governance conversations um so

1:18:55Dean Ball:um i think it does seem like you know we we actually might kind of agree on like what the logical next step is and that where we disagree is like i think where we disagree much more profoundly is like what is going to happen in 2030 or whenever like what what what are the big what are those super powerful systems going to be like what will their implications be um and i think we've agreed on some triggers at least for what would cause us to think differently what would cause us both to think differently but it seems like we actually just we agree on the next step and maybe one thing i might suggest is could we like my view has just always been that we don't we don't go there's too much thinking about the extremes and thinking about the ultimate end game rather than thinking about just the next step on the margin and so like could we agree perhaps that that's a wise way to proceed that we don't actually know if you're right and so we should go one by one and not try to do anything too extreme is that too far is that a bridge too far yeah maybe i think the exponential i think i think exponential is a pain in the ass yeah First of all, hey, I'm not in the White House.

1:20:06I'm not doing anything. I'm just sitting at the AI Futures Project writing blog posts. You don't do that much in the White House, man. I'm just putting ideas out there. And so I don't feel like I should be restricted in how far ahead in the future I think and write about. I think that, I mean, yeah, my thought would just be like things are going to get real crazy in a few years and we need to sort of think about what it might be like and backchain from it. And I agree that we should, you know, have uncertainty about the possible futures and not do anything that's like too drastically bad in any of the plausible futures right now.

1:20:41But I would sort of turn that around and say, and that's why we need some pretty serious domestic regulation and international agreements, because if we don't, then things are going to be extremely bad, according to some of the plausible futures, including the ones that I think are most plausible. You talked about how the next step is to think about what this domestic regime would look like. and I think that's a very important next step but I would also, the thing that I would say is the next step is to think about what the international agreement might look like. A thing I would say there is like, well, there's going to be all these other countries, most notably China, but also France and Russia and Israel and Kenya and they're all going to want to have some sort of say in how all of this goes down and I think that it would be good to start that conversation, get some good ideas flowing about what sort of international thing to advocate for.

1:21:31I think that before, just as you think that the U.S. government is going to wake up before the end and try to seize control, I think that the other countries are going to wake up before the end too, or at least many of them will, and try to ask for some sort of international agreement to maintain some amount of sovereignty. And so getting that conversation, getting that ball rolling, I think seems good to me. But I don't know, maybe we still haven't found agreement.

1:21:55Dean Ball:I don't think it's crazy to say get the ball rolling. it's just that you must understand that the ball moves very slowly like you have to is anyone here at the india ai impact summit no there are 250 000 people from around the world there and no one in this darn room was there that shows you how unagi-pilled it was though um it was it was so um like the state of the conversation globally is like dismayingly bad like really bad like like so i i just i think that there's a long way to go there uh and so i'm i'm i i think i worry enough about the bilateral much less than the multilateral but i maybe you know could agree that like bilateral right like this by getting to a bilateral so like the two logical next things to do are work on whatever this thing is going to be for like the verification the auditing of the frontier labs, whatever, figure that out.

1:22:55Dean Ball:And then like, try to, try to establish the conditions where perhaps at some point in the future, the US and China could bilaterally agree to shut things down.

1:23:10Liv Boeree:Fantastic. So we actually have a, we have a statement.

1:23:13Dean Ball:We should include in it that we both believe that we should be racking Vera Rubens in China just to annoy all the AI guys. Everyone in DC loses their mind. I'm just kidding. I don't actually support that to be clear.

1:23:30Liv Boeree:Fantastic. Well, thank you both so much for being such good sports with this. I do think we did find actually a surprising amount of common ground. You know, from my observations, yes, you both agree that power concentration is a big concern. you don't want the executive branch to take control of everything the importance of the judiciary you also both agree that alignment AI alignment itself continues to be a major concern and that we should be working on auditing and verification methods of frontier labs so that we can try and establish conditions where US and China can have some kind of bilateral agreement whereby if there was time to press a pause button that would be it seem about right okay great that I mean, it's fairly long and convoluted as usually nuance is.

1:24:20Liv Boeree:Awesome. Anything else, Steph, that I've gotten? Yeah, I think with that, how about a round of applause?

1:24:32Liv Boeree:So there we have it. The first ever win-win style anti-debate on this channel. Huge thank you to both Dean and Daniel for bravely agreeing to try out this format. You know, this was a big experiment. So massive props to them for being brave enough to try it and for doing so well at finding synthesis. Also, huge thanks to Stephanie Lepp for dreaming up this concept and letting me have a go at moderating it. I won't lie, I was extremely nervous and I had so much fun. So expect to see more of this format on this channel, because this notion of finding synthesis is, I think, one of the most important challenges we have right now.

1:25:09Liv Boeree:There are so many ideological conflicts we're all at each other's throat about. and while it's true that some of them are born out of true zero-sum scarcity a lot of them do dissolve if we do manage to sort of step off the ideological page and see it from a higher perspective and really that's kind of what this whole win-win project is all about spotting the false dichotomies and dissolving them whenever possible so if you enjoyed this please make sure to subscribe share it with your friends and I will catch you at the next one thank you

1:25:43Thank you.