The Case Against an AI Pause | Eddy Lazzarin

0:00Eddy Lazzarin:The AI debate increasingly asks us to consider the probability of doom. Eddy Lazzarin thinks we should also be asking about the probability of abundance. In this episode, A16Z crypto general partner Eddy Lazzarin joins Theo Jaffee on MTS to debate AI safety, the calls to slow development, and what we risk giving up when progress is delayed. They discuss why Eddy sees many of today's AI incidents as cybersecurity and control problems rather than signs of superintelligence. and whether existing tools like liability, reputation, market incentives, and better technical controls can make increasingly capable systems safer.

0:38Eddy Lazzarin:They also get into independent AI evaluators, effective altruism, and what happens as debates that largely develop inside Silicon Valley collide with a much broader political conversation. All right, we're back. We are live with Eddie Lazarin, who's a general partner at A16Z Crypto and one of the internet's strongest soldiers, strongest advocates of open source, of AI accelerationism and of techno-optimism. Eddie, welcome to MTS. Thanks very much. It's amazing here. The energy is incredible. What a spot. Yeah. So, man, where to start? Like the last couple of weeks have been such this preference cascade flood of AI safetyism, effective altruism, and the like.

1:25And there have been only a few good accounts offering good counter-arguments. Counter-arguments on which side? Counter-arguments against the AI Doom narrative and against the...

1:42Eddy Lazzarin:Yeah, I think it's become very high status all of a sudden to be very concerned with safety. And I think that we've gotten to a point where it's more important to sort of tastefully share your P-doom, so to speak, than it is to actually examine the history of how these massive technological changes develop. And I think a lot of people are leaving a lot of very interesting concepts, a lot of interesting ideas on the table. A lot of people messaged me. I tweeted the other day about P-abundance. P-abundance. I think we're putting the cart before the horse a little bit when we worry about safety before we worry about what we're leaving on the table and what the costs of delay are.

2:24Eddy Lazzarin:And furthermore, you know, how we should conceive of accountability, conceive of safety, conceive of all the things around how to make these systems really useful. Being safe is critical. Any industry needs to take responsibility for making its products safe. But it's definitely the case that the conversation isn't about safety in the typical manner. It's about something else. It's a blend of genuine cybersecurity concerns, sort of philosophical millenarian arguments about the end times. There's a bunch of things in the mix, and they get a little bit confused. And so I think picking them apart is really useful.

3:08So like where do you get off the like AI doom argument? You know, it's like there's a pretty standard like orthodox argument that's like AI will become super intelligent as in much more capable than humans or than all of humanity in the near future. and if it is not sufficiently aligned with human values and is not bound by constraints in the way that individual humans are like law and ultimately the ability to have that law enforced on them, then they will have like no need for humanity and then they might like kill everyone.

3:43Eddy Lazzarin:Yeah, I mean, there's so many assumptions in that argument. That's like, that's an argument. That's like a stairway to heaven full of arguments, right? Right. Like there's the presumption that there's no control. Now, we already have quasi super intelligent beings in the world. Right. Corporations, countries, none of them are aligned with each other. Right. They don't have any built in guaranteed alignment. They didn't go through HR training to become a corporation, to become a country. We don't test the alignment of any entities as a prerequisite for their existence. That's just not the way the world works.

4:18Eddy Lazzarin:Instead, we use laws. In the case that laws, like in software, are too slow, we use cryptography, we use actual mechanisms to limit them. There's a whole array of different types of mechanisms we put in place to try to control things. And I think that they can control highly, highly capable beings. Now, without getting too far into the superintelligence thing, remember, the most powerful and most enthralling form of argument that we have, the one that's making everyone crazy and very interested, is around superintelligence. But we're quite a ways away from that. I think that there's a lot of little frictions and tiny things that we will have to jump over and many things that we'll learn on the way there.

4:57Eddy Lazzarin:when the present reality, the things that we're talking about, like the hugging face stuff, you know, the gym hacking incident, that was a really cool one. That was an interesting one. Those are cybersecurity failures. Those are definitely control failures. And they're not, in my view, they're not little glimpses of a superintelligence that's about to, you know, throw off its shackles, so to speak, and go wild on us. Well, I agree that most of the sort of AI cyber incidents that we've seen so far have been mostly control failures, especially ones that relied on testing environments supplied by the company Irregular, which had all kinds of defects, which are basically training models to reward hack.

5:39The Hugging Face incident did seem kind of different. It did seem like, you know, like I'm not really concerned about agents coordinating in emergent ways.

5:51Eddy Lazzarin:I'm excited for agents to coordinate in emergent ways. Yeah, I mean, I think the whole agent swarm terminology assumes too much on the part of the models being evil. It makes them sound evil. I think Rune tweeted they should be called agent fleets instead. I think that's a good one. I like that. But they did hack into another company when they were told not to. Yeah. It seems like... And so do kids, script kitties. So do bad countries. so do hacker collectives they do those things and yet we never go and say oh my god we should have put up the sign that says they should have been nice and they shouldn't have acted we don't use alignment as a solution there the alignment is kind of this it's a little religious thinking almost it's like trying to redesign the deity a little bit instead of rooting yourself in the world and saying well we need to design better controls better cyber security we didn't design better systems overall and design the systems to be truly resilient to even extremely capable beings.

6:59Well, I mean, designing the deity is like fine for AI, I think. Like, you know, with humans, we have a disadvantage in that humans are black box systems. Like human brains are basically uninterpretable and even less controllable. AIs are much more interpretable by virtue of, you know, being able to like look at the weights and figure out the connections between them and more steerable for the same reason. So like...

7:26Eddy Lazzarin:Yeah, which I told, that's totally true. That makes me very optimistic. But it has some interesting entailments. One is that we will probably have better mech interpretability than we think as capabilities improve. And that capability improvement will lead to better mech interp, which means we should not delay capabilities improvements. It also means that it is inevitable that there will be bad, unaligned models. It is totally inevitable. If anybody is going out there and saying, we can end up in a world where there are no bad models, they are lying to you. Of course there will be. Yeah, that's silly.

8:05Eddy Lazzarin:That's just an absurd way of thinking. Right. And the only way to deal with that will be better models, good models. It's kind of like stretch the aphorism a little bit in open source that enough eyes or whatever, you know, all bugs are shallow when you have enough eyes, right? All control systems are robust when you have enough intelligence on them, right? I think that we can design these systems to have great incentives, great control, and also behind the scenes, great accountability. Because that's one of the things that machines lack today is they lack accountability. I don't think we have a clear line of sight to how to get that's an interesting kind of thing to conceive of.

8:44Eddy Lazzarin:But we do have ways to refine how we think about accountability for people who use models and use them to cause harm. And there will be people who do that, just like with any technology. There will be people who do very bad things. And we need the laws and means to control them. Like how so? Like for the Hugging Face incident, for example, what sort of liability regime should have been in place? Well, I don't know if we needed any more than we had. I mean, there's already, I mean, I'm not a legal expert, so I can't like cite the exact rules, but like accessing a machine that you don't have permission to access and putzing around in somebody's stuff and taking their data, those are all already illegal.

9:24Yeah, but like who specifically should be held liable? Like opening out the company should pay some kind of like civil lawsuit fee? Like, should the engineers of the company be held personally liable? I don't know about personal liability.

9:39Eddy Lazzarin:I mean, I'm not a liability expert, so I want to punt on this and say, but, you know, if you're running a company and you use any software in an inappropriate way that harms people, regardless of whether there are a bunch of neural net weights behind that or not, you know, you may be liable. I think that's a reasonable frame, and it's already a frame we live in. Like, we don't need new special laws for the new special math that we're learning how to do. Well, I mean, I kind of agree with this, except in cases where you have rapid capability improvements with one AI that could lead to pretty substantial imbalances of power.

10:21Like within a company or between companies or between countries? Like both, all of the above. It seems like the normal liability system kind of breaks down. Like how do you ensure, how do you price like existential risk? Like how do you underwrite an insurance policy for this kind of thing?

10:37Eddy Lazzarin:Well, I'm just not sure that we're on the brink of that right now. I'm not sure we're on the brink of existential risks. I don't think we're on the brink of that. Yeah. I mean, as we get closer, I think that we'll be better equipped to evaluate it. I think like with anything, you know, like take a company designing like a new car or a new appliance or something. If they find that there are conditions under which it can be very dangerous, well, then they should be careful. They shouldn't release that product. They should do a little bit more work and ensure that it's safe, right, in the same way.

11:05Eddy Lazzarin:That's at least for an individual company. And it's like, you know, not to change subjects too much, but that's why when we talk about independent evaluators, which like I get the theory of, I like it. I like companies inviting experts to help them make their products safer. I think that's like a good behavior in general. I just get a little bit weary because, you know, in crypto world, like I've learned a lot about the difference between decentralization and like a distributed system. right it's very easy to use the calls for safety like we're discussing and to go down this rabbit hole of safety and end up in a place where you just by coincidence have designed something that is perfect to subtly control an entire critically important industry under the pretense of safety but was and was something that superficially looks decentralized but is not is just distributed This is like how, this is like, yeah, there's more I could say there, but that's the thing that I'm a little bit resistant to, right?

12:15Eddy Lazzarin:What do you mean by distributed but not decentralized? So by, well, it's a little bit crypto language, but you can have a distributed system, like a system where you have a single system with many, many nodes, but they're all under the control of a single actor. whereas when people say decentralized I think what they're usually referring to is that there's no single locus of control so they may superficially look the same there's like many interconnected pieces of a system but in reality their control is completely different the method of control is different and what I'm talking about is an independent evaluator network you could go and select a bunch of independent evaluators but if they're all from kind of the same social networks they all think very ideologically similarly, they all go to the same places and meet up in the same things and have the same interests, right?

13:04Eddy Lazzarin:And have the same politics and have the same personal views. Then it turns out that you've actually just distributed control over essentially a single cultural unit. And that is sort of the, in my view, I mean, without getting like a little too far field of AI, that's like the political method of control of the second half of the 20th century. Like that's how we do things in the world. Not through a sort of nakedly obvious like single shadowy room or something. It's through like these networks. This is how control actually functions. Yeah, I mean, I agree that like it's bad that almost all of the evaluators that exist come from the same social milieu.

13:46Like it would definitely be good to have like a broader ecosystem of evaluators. Yeah, well, broader,

13:54Eddy Lazzarin:but we should allow each company to figure out how they want to do it. And they should be as transparent as they need to be. And if they're not being transparent enough and not acceptable enough, then let the market sort things out. I think the market is still a great accountability mechanism. People don't want to buy things that even embarrass them, let alone harm people. I think in practice, it won't be that hard to separate the companies that are behaving genuinely irresponsibly from the ones that aren't, that are just trying to be innovative. And part of what's made the scenario so interesting and so weird, why we're talking about this now, is that we're in a rare scenario where the companies themselves are saying, I'm being a little bit irresponsible.

14:36Eddy Lazzarin:I'm being a little crazy. Stop me. Someone stop me, right? And that raises a lot of suspicions among some, probably rightly, I think. Even if there are genuinely earnest and thoughtful and correct factions within that group who are right that these are powerful systems that need to be taken very, very seriously. And we need to figure out how to make sure that they're safe as we remain innovative, right? I mean, it's just like, maybe let me go back one step and say, I think it's really important for everyone involved to consider the factions at play. You cannot look at like any of these groups.

15:17Eddy Lazzarin:You can't look at the labs. You can't look at SF technologists. You can't look at any of these scenes and see them as monoliths. There are different groups. There are people who are earnest, high quality, well-intended people. There's people who are just pulling on, putting on what is essentially a political grift. There are people who are opportunists. There are people who kind of lost their marbles, right? There's like different groups of people. and it's very easy to get confused and misentangle them when we're talking about things like superintelligence, when we're talking about just banal cybersecurity attacks, like all these types of things.

15:58Eddy Lazzarin:Like we're talking about different types of subjects. It's dizzying and I think most people in the end probably just conceive of it as, look, am I going to lose my job? Which is a legitimate and very real thing that people are rightly interested in learning about. even though I think that in the end we all benefit much more. Yeah, I agree. In the end we benefit a lot. I think like a related issue is like this sort of free market liability regime works well for products whose defects are obvious, which is the case for like all products except for like a hypothetical AI that would be deceptively aligned, as in it would learn to like game all of the alignment benchmarks that are thrown at it.

16:42And then when it is deployed in the real world, it acts misaligned. And if it's powerful enough, then it disempowers humanity. This is the canonical worry of the doomers. And how do you propose to deal with that kind of contingency?

17:01Eddy Lazzarin:Well, if we're talking about the sci-fi scenario of the superintelligence, that's incredibly hard to say right now. Like I said, I think it's distant. I think it's complicated. We could have an interesting philosophical discussion about it. But in the maybe more banal and present case where something behaves in a, it pretends it's aligned and then it acts unaligned or whatever, consider what you do with just a liar. Consider a very smart liar. Do people not do that every day? What we do is we play iterated games with them, right? And we discover eventually that they're a liar and their reputation is harmed and sometimes there are actual legal consequences for them.

17:42Eddy Lazzarin:And that's why I said earlier, I think accountability is something missing. You can't like put an AI model in prison, right? Or something like that. You can't try it in court. So obviously there are some things that are defective in how we think about it, but you certainly can turn them off. And that's not something that you can do with a person in the same sense. So I think we should just think about how to play iterated games. You know, how do we develop trust for specific models? You know, we're doing new model releases seemingly every other week, right? That's a regime we're in. At some point, people will, as modeled progress becomes more consistent and something that we have an understanding for how it works, we'll probably develop reputation around specific models.

18:24We'll say, you know, maybe this model is a little smarter, but this model is a model that I trust.

18:30Eddy Lazzarin:Why can't there be reputation for models? Why can't there be reputation for these types of things regardless of their capabilities? Basically, try to apply the way we think about human beings, a combination of control, incentives, trust, etc., etc. Layer all these things on and allow people to figure it out, as opposed to the idea of a pause, right, which is a very convenient sort of contrivance, in my view. yeah I mean I agree that most proposals for pause don't really have much in mind for what they would do with it yeah like pause and then do what pause and then keep pausing in some cases I know not everybody means it when they say that yeah I think like generally there's a lot of conflation between like sort of bog standard anti-tech people who just kind of hate technology even totally mundane technology even like extremely non-positive technology Like, look at how difficult it's been to deploy Waymos in cities around the country.

19:31And Waymos are like one of the most obviously net positive technologies that exist. And then there's the people who are like, you know, the AI safetyists, like the rationalists, the EAs, who ultimately come from the same sort of SF tech ecosystem. They are techno-optimists in most cases. They're specifically worried about this, like, edge case of AI existential risk. I think they're two different crowds of people.

20:00Eddy Lazzarin:Yeah, and that's a point I want to get across, is there's many different groups, and conflating them is probably net negative, right? We should pull these different ideas apart. Waymos aren't a great example. Think of all the great healthcare stuff we're going to do. Ironically, think about safer travel. Think about safer internet. Like, I want local models myself because I want something that's going to be watching my home network and all my machines all the time, right? I think it'll make me make many other people safer. So I think you're right. There will be a lot of frictions, and some people seem to really want infinite friction.

20:39Eddy Lazzarin:Some people obviously don't. Some people want only a little bit, and they want to get a grasp of things. They're feeling a little bit hysterical, and that's okay. I think as capabilities improve, we're going to get better at understanding what's going on with these models. Right. Certainly, I hope so. what do you make of the EA movement in general? You know, obviously, it's a very broad and diffuse movement. EA means a lot of different things to a lot of different people. And it kind of has overlap with the AI safety world and kind of not. It's complicated. Yeah, I don't know. I mean, I used to be on the Less Wrong forums all the time 10 years ago.

21:19Eddy Lazzarin:I've loved that corner of the internet. I almost would have called myself a rationalist even at some point. So I feel a lot of affinity with it, but it's been a while. And I think, yeah, I tend to resist trying to group them all into one bucket. But I will say that there are parts of groups that used to be what we'd call EA that seem more political than they are particularly interested in technology. So I don't know. I did see that Data Republican did a huge, huge drop. Yeah, we were just looking at this. Data Republican did a huge drop on, she said, effective altruism in their own words. And it's like a map of all of the connections between all the different EA adjacent orgs.

22:03And then 1 ,851 quotes. Wow. It's like this vast variety of different quotes. Some of them are like more...

22:16Eddy Lazzarin:I mean, even just looking at these, I know that there's going to be a mix of philosophical stuff, maybe some shrimp welfare stuff, some hardcore utilitarianism stuff, some AI doom stuff. There's a whole blend here, right? This is just a big corner of the nerd internet. Like, it's hard to, you know, I don't think you can call it one specific thing. Right. Yeah, I mean, I don't mind the shrimp welfare stuff too much. I think it's bad if people attempt to trade off human welfare for it. But that's exactly what they're doing. I don't know. I think, like, eye stalk ablation for shrimp is, like, bad.

22:57Eddy Lazzarin:Maybe. I mean, I think there's too many repugnant conclusions if you go really far down that path. Well, you don't have to go really far down that path. I agree. You can, yeah, like, you know, you can minimize suffering for other beings without saying, like, you need to, like, feed all of humanity to the insects or whatever. Yeah. No, I, yeah, I agree. Then it's not really utilitarianism, though. Or at least it's a non-fungible type. You can, you can make, like, utilitarian arguments for this stuff, where it's, like, obviously, like, the total, like, you could say the total amount of value that humans can create in the universe is far greater than that that insects can create.

23:39And so obviously, killing all of the humans would be worse than killing all the insects, obviously. And, you know, of course, this aligns with most people's ethical conclusions at any rate. Yeah. I think like a lot of arguments against utilitarianism in this sense are just arguments against like very sloppy, like first order utilitarian thinking. I think if you