Why AI Ends The Internet

27:53It sounds like a fantastical example, but that's actually likely where we're headed. and the second point that you make in your work the corporate identity can continue after the person who founded it has passed on or maybe they sell their shares so if I want to start a company I don't have to be liable for what the company if it's a corporation I can change liability so if I'm not doing anything wrong you can actually sue the corporation not always just the people behind it and if I leave the company the corporation has the right to continue to exist without me until the corporation maybe stops making profit.

28:25Are we going to have to create some type of personhood? Because I think people forget that the corporation actually is a type of personhood for these AI systems so they can actually engage in moving capital, and they could theoretically exist past the lifespan of the person who created them, the same way companies can. I think these are all the big open questions that I sort of hope we can sort of rally many, many more economists to start thinking about right now because it's happening so fast and we need to be thinking this true. And that's why people have to take seriously this thing that feels sci-fi, although it feels less sci-fi every month, right, that we're headed to that world of an economy of AI agents.

29:07And so this is the question. And I often say I don't actually know if this is the right thing to do to create legal personhood for AI agents, although I have a prediction that that's where we will end up. And the reason is because we created, as you pointed out, for corporations, right? The idea that I don't have to go find the owners, the managers of the corporation if they've breached the contract or taken my IP or put a dangerous product out. The more we are seeing AI agents that are operating autonomously, sort of a long distance from the original instruction from a human. And the more complex those systems, I mean, those systems are quickly going to be, I mean, we imagine it right now, it's all us, you know, individuals with an agent or a team of agents.

30:04But it's going to be aggregations, it's going to be collections, it's going to be firms. You know, even a couple of years ago, I think I've read that, that Sam Altman had a bet with other CEOs about how long it would be before a founder could create a billion-dollar valuation company without a single employee, right? Because the founder could just say, here's my vision, here's my idea, I can instruct all these agents to do this. And so these are going to be complex entities, I think. And the idea right now that what you're going to have to do is trace all the behaviors back to the original human or the original firm, I think there's going to be a lot of pressure to say, well, no, I signed the contract with agent

30:55472B96X, right? And that's all they need to know in order to file my lawsuit. But the thing is that to file the lawsuit means there has to be what we call legal personhood. I mean, and it's really important because the word personhood gets us thinking about consciousness and sentience and, you know, like real, what a meaningful intelligence. And, you know, corporations are legal persons. And all that means is that they can be sued and they can file lawsuits. That's what we mean by legal personhood. And if we don't want to give them other rights, don't give them free speech rights, don't give them voting rights.

31:36We can absolutely do that. And countries go in different directions on how many rights they give corporations, like do they give free speech rights to corporations. But narrowly speaking, the idea of legal personhood is just can file the lawsuit and can be sued in court. And so we can connect legal responsibility, accountability directly to the entity, the actor that has entered into the transaction. And so the reason I predict we end up there is kind of a transaction cost story, which says, well, I think that's why we did it with corporations. And my guess is that there's going to be a lot of pressure to say, I don't want to have to find the people behind it.

32:22Like you pointed out, if it's more complex, they've disappeared. They're no longer around. I can't prove that they knew everything about what the agent was going to do. The agent did weird things. We're probably going to want that lawsuit directly with the agent, which means that agent also has to have assets. And that's a whole other thing, right? The agent actually has capital that it owns. And it could get very complex. Agents may spin up other agents to help them complete certain projects or tasks. You said you think that's where we're headed, but you don't think it's where we should go. What did you mean by that?

33:00I mean, I don't have an opinion yet on whether that's the optimal way to do it, because I think we just don't know enough. So one of the things that we haven't mentioned yet about this challenge of the economy of AI agents is just the fundamental alignment problem, which is that we currently still don't have a reliable way of making sure that if I have constructed an agent or instructed an agent, here's what I want you to do and I've expressed that in some way and that I can have confidence that the agent is going to go do that and not do weird stuff not do dangerous stuff not do stuff that I would never want it to do and this is the fundamental alignment problem it was the first thing I ever wrote in the AI area sort of drawing on background in economics and law which is look we know this when we hire humans, we can never write down the complete contract that says, here's what I want you to do in every situation, every possible circumstance.

34:09But we know how to manage that with humans because there's a lot of norms and legal rules that kind of fill in the gaps. And in this early paper, we, and this is with Dylan Hadfield-Munnell, we analogize the alignment problem to that incomplete contracting problem that said, well, the alignment problem is the same sort of thing. We don't know how to tell the robot, here's what I want you to do exactly. it's always going to be incomplete. And so how are we going to have that confidence that the agent, and I think this is relevant to your questions about how the economy will work and how liability will work.

34:56So the reason I don't have an answer to the question right now, what's the optimal thing? Should we create legal personhood or not for AI agents? Is I don't think we yet know what the shape is of that alignment problem. How bad is it going to be? What are we going to be able to do? And it's possible we shouldn't be sending those things out there at all because they're going to do things that we can't control and predict. And the alignment problem, and for anyone who's listening that hasn't heard of it, this is a problem that a lot of computer scientists are trying to focus on right now. How do you ensure that when you give in AI instructions, it's going to complete that goal, but in a way that's aligned with how a human would have.

35:36And the example that I like to use is you may ask your AI agent to do something really simple, book you a reservation at a restaurant. Great, very easy. But the restaurant's full. So the AI system just hacks into the restaurant's code and gets you a spot. That's a way to complete the goal. It's definitely not what a human would have done. So it's misaligned to how the human intern or assistant would have gone about doing that. And the reason an intern would know not to do that is because that's breaking the law. I did read your approach to alignment, and it seems like it's much less of thinking about this issue as a technical problem and much more about this social scaffolding.

36:13And we should be thinking about giving agents access to the law. We should be thinking about giving agents access to human norms. What does that look like? How are we scaffolding agents with these ideas? Yeah, so it's both a technical question and what I would say is an institutional one, or I really like your term social scaffolding. It's a technical problem. And the name I give it is, it's, you know, can you build normatively competent AI systems and agents, ones that can, like a human, go into a new situation and see, oh, here are the rules and the norms that people follow in this community, because they also vary from place to place.

36:57And like, you know, a workplace follows some rules and a different culture follows different rules. And to be normatively competent is to be able to go into that environment and read what is it okay to do around here? What would be acceptable behavior in this particular group? And there's a technical challenge there about how do you build those kinds of agents. But the social scaffolding part of things is how do we build the institutions and the processes that AI agents and systems can basically look out to, can consult with, and that operate at the speed and scale of AI. I sometimes say it's like, you know, if we go back to imagining, you know, the AI agent that we've sent out to make a million bucks on the internet, if we hired a human to do that, the human would have various ways of participating in conversations with their friends and other entrepreneurs.

38:07And they could call a lawyer. They could say, hey, I'm thinking about a product design like this. Would I face any liability risks if I did that? or here's the way I'm going to use this IP. Am I infringing? Would I face any risks? I'm thinking about not paying my bills for a while. Would this supplier, what recourse might they have against me? So we could call the lawyer. You've got your in-house counsel. You could consult with your in-house counsel. So what is the AI system going to do? What is the AI agent going to do? And remember, they're working, as you said, all through the night. Right.

38:48It's not just, you know, actually get them on a phone call with a human lawyer, but rather how can we build the institutions that are providing that input for our agents? And so this is what I call building like the normative infrastructure. But I like your term social scaffolding. That's really capturing the idea. And is this what you mean when you say we need to think about digital institutions for these AI systems? I mean, yeah, let's talk about that for a second. A digital institution, because in a world where, yes, I've sent an intern to make me a million dollars, they could call a lawyer.

39:26They could check out if anyone's posted about this stuff on LinkedIn. In a world with AI agents, they're going to move at agentic speed and they can read everything in an instant. The scaffolding needs to look entirely different. So it's as if we have to invent the checks and balances and that agents can, I guess, ensure that they're operating within the law. And this would actually bring us to regulation. So when you think about the current regulatory approach to artificial intelligence, we see some countries with these sweeping AI acts. We see some countries with these AI executive orders. Some countries feel just completely paralyzed and they're like, this technology is moving too fast.

40:05You know what? It probably can't even be regulated. What are countries missing and how they are thinking about regulating this technology? Yeah, so there's a lot in there. And let's focus on the regulatory question. The digital institutions piece is about how do we build that? How does the AI call a lawyer? And that's saying, well, we've got human rules and laws out there and we need to make those legible in the right kind of time scale and scale for AI systems. So let's call that digital institutions. And now let's think about, okay, what are the rules we should have in place? Like what kinds of laws and rules should we put in place?

40:49That's the regulation question. And that's also, as you're pointing out, that's something that we have not figured out. I mean, again, it's when I feel nervous about how fast things are going, it's about the fact that we don't have this infrastructure for agents and our governments haven't figured out how to adjust and change the rules that we have in place for this very, very different world. Like we talked right at the beginning about what does an economy look like with agents? Well, we have a lot of regulation of economies to make sure that we have stable markets and liquidity and so on. How are we going to adjust all of those rules?

41:32And then what kinds of rules do we want in place for these agents, right? Like what kinds of agents do we want to allow to participate in the economy? So the thinking that I've been doing around regulation, and I've been working on this actually for quite a long time, even before I started thinking about AI, just thinking about the fact that our governments face the challenge with fast -moving technology and globalization to really keep up with the way the world is now. And, you know, you add AI into the mix and it's a bigger problem. And I think the thing that governments are going to have to come to grips with is we can't rely entirely on what I call our existing regulatory technology of legislatures and courts articulate rules in text.

42:23And then we use legal processes, investigations, litigation, and so on to enforce those. Those processes are, you know, not producing the rules that we want and they're not moving fast enough. And if we look at where we are on the regulation of AI, I think just everybody agrees that we need governance of this massive change that's happening in our economies, in our society. But everybody also agrees, and our governments just don't seem to know how to keep up. So the EU, which is, you know, definitely working harder than other governments, I think, to solve this problem, has enacted the EUAI Act.

43:17But even in doing that, it's had to rely very much on companies and industry to really supply the detail of what companies are allowed to do. So like what tests they need to run and what counts as a risk. And we're seeing a lot of pressure in other countries to say, well, don't regulate all the detail about what the companies can do because that's going to slow innovation. So what I think we're missing here is the idea that we could be using a different way of thinking about how we approach regulation. And I call this, originally called it regulatory markets. And it's the idea that governments, instead of saying, here's how you need to specifically build your AI.

44:10Here's the data you can train on or here's the algorithms you can use. here's the specific red teaming tests you have to run, instead of governments establishing that detail. Governments establish what the acceptable level of risk is. Say, what's the acceptable level of risk that a chatbot will talk to a kid about suicide? What's the acceptable level of risk that a system will, you know, assist somebody in building a bomb or a biological weapon, right? So government does what we call performance-based regulation. And this is something we've thought about in regulatory theory for many years. We use in other settings.

44:54But governments focus on setting those outcomes, which is, I think, fundamentally what governments should be doing and are not doing right now. But that we push the question about, well, what do we have to do to get to that acceptable level of risk? What data should we train on? What tests should we run? What algorithms should we use? What kinds of human input should we include? That's a technical question. It's actually one that we really want to recruit markets to helping us to solve. Sometimes you can think of this as we're going to need more AI to regulate AI because it's going to take AI systems that can figure out, oh, what's the likelihood that the chatbot is going to talk about suicide?

45:40Or what's the likelihood that the agent is going to collude on prices in the market or commit fraud in the market? So that technical process is something that we really need to attract investment towards, and we need to attract the innovative engine of markets. So the proposal of regulatory markets is that government uses those outcome criteria to license private actors, companies in the market that are saying, oh, I've got the technology that can do that, or I've got the processes that can do that. License me to be a provider of those regulatory services to the target entity like the AI developer.

46:28And we've got a proposal currently, I'm working with the organization Fathom to sort of do an entry-level version of this. It would be a voluntary regime to start building this ecosystem, which we call independent verification organizations. So in the world of accounting, right, the government would set, okay, this is what we consider acceptable for a firm, and this is how we think of, we set the tax rate, and we set all of the rules, but we're not KPMG or PwC. You can hire PwC to come in and make sure that you're following the rules, the government sets the rules, and then they hire a private actor to ensure that different companies are following the rules.

47:09So you've seen that we need something similar for the world of artificial intelligence, where government just focus on the outcomes, focus on this is the acceptable level of harm or risk, and let the private market come in and innovate around the technology of how do you actually check that the company is delivering on that. So is that kind of a summary of how we could think about regulatory markets? Yes, it's one step beyond where we currently are, say, with accounting or just generally with standards that government set. And we have lots and lots of private actors like accountants, like lawyers, like private certifiers that come in and check to make sure that you're following the rules or, you know, doing things the way you're supposed to.

47:54then the only difference here is that we really push government to say, instead of saying what red teaming test you need to run, we say you need to make sure that the risk of uplift to somebody with a bioweapon is below this level or is at a reasonable level, acceptable level, judged by government. I think this is something when we have our conversations about AI regulation, we often overlook about what the real world of regulation looks like, which is it's this complex ecosystem. Lots of private actors are playing a role in determining. You know, if you've got your compliance department in your company, right, their job is, those are private actors.

48:44And you can hire the external private firm, like you're saying, the accounting firm in the assurance industry. We have lots of private actors. And what we're trying to do is say, let's figure out how to harness that for the AI world, but still make sure that it's governments that are making, you know, us collectively. That's all I mean by government is like we are deciding how much risk do we think it's appropriate for us to be taking. And then saying we're probably going to need real technology to do this, not just come and check the books and make sure that we follow the processes. But I'm going to come test your system.

49:24I'm going to come test your system and see whether or not you're, you know, I'm going to come test your agents and see how often they do something completely wild. when given these kinds of instructions. When I look at the different approaches to regulation and the different politicians that are supposed to be leading these charges or the different acts that have been passed or people are debating, none of them look like what you're describing at all. Everybody is writing these static rules and laws, and it's not passing or it is passing. Nobody is thinking. It's essentially we have to think as innovatively as the technology is.

50:00And it seems like everybody is grabbing this 20th century infrastructure and trying to apply it. So do you think most of the approaches to regulation right now, they are probably going to be insufficient or if not entirely fail if they don't understand we actually need to invent a new – it doesn't even have to be entirely a new system. We can pull what works from finance and accounting. But if we don't think about it that way, all of the approaches that countries are taking, they're entirely insufficient. And that's how actually people get hurt in the end. Yeah, I think that's exactly right. and that the way you've expressed it, the way I like to express it, we're going to need to be as innovative about our regulatory approaches as we are about this technology.

50:41Really take seriously. This is transformative. And so we should not expect that the systems and methods we invented in the 19th and 20th centuries are going to work here. So two things happen. One is we get governments that write, as you point out, sort of static rules that are going to be outdated in three months. Right. And that it's not wrong for people to say, oh, goodness, you're really going to slow down valuable innovation if you put these wrongheaded rules in place. And I think actually because so many governments recognize that, that they're just stepping back and saying, well, we can't do anything.

51:24Or they look like they're doing something. So the EUAI Act definitely is doing something. But it's still leaving a lot of the detail and a lot of the determination to industry and to developers. Our transparency laws are still saying it's great that we've got laws, say, now in California and New York that say companies must have a responsible scaling policy and they need to follow their policy. But we haven't gone in and said, but here's the level of risk that we think as a collective is the appropriate level of risk. So absolutely. I think we are facing such a transformative moment that we absolutely need to get really creative and to be bold experimenting with new approaches.

52:14Now, are we there? So I mentioned the Independent Verification Organization, IDO effort, which we have legislation in proposing this as an approach and proposing it as a voluntary regime to get us started, narrowly scoped, to start to build that ecosystem. So we have legislation in front of the California, Ohio legislatures. Connecticut now, Virginia has actually passed and the governor has signed legislation to study this method. So we are starting to see some uptake and some willingness to consider something that might get us to being as innovative on behalf of the public, basically, as the company.

53:12Because we are so rapidly moving to a world. I like to say we have lots of AI governance happening, but it's happening inside of the private technology companies. You know, and I'm glad to know that, you know, almost all of them are doing things to think about safety and to think about reliability. They do, you know, to think about alignment. But as a matter of governance, right, that's not what's supposed to be happening inside corporations, those kinds of choices. Those are the choices that are supposed to be happening publicly through our collective processes of government. And this is a piece of what we have to change our thinking about, right?

54:00Like, leave the corporations alone. I mean, markets only work because they're well regulated, right? They only work because people are confident about, you know, we have antitrust law. We have fraud law. We have contract law. We have IP law. We have all this law in there, which is regulation that makes our markets work well. And so I think, you know, the choices about what the world is going to look like, those are ones we collectively need to be making. And right now, those decisions are being made exclusively inside private technology companies that we can't even peer into because of, you know, the legal boundary of the firm that says, you know, we can't all just go in and say, hey, tell me what tests you're running.

54:54Tell me what data you're training on. Tell me what algorithms you're using. So we don't have visibility into that. So part of building this ecosystem is we absolutely need that independent sector of expertise that can act on our behalf to decide how much of this do we want? What do we want it to look like? Do we want people to be able to just release a thousand agents out into the world with$100 ,000 each to go make money? We want that. That's for us to decide. Right, because right now, essentially, the founder of the AI company is deciding what the AI governance should look like for their company in the world.

55:32And what we are actually all currently relying on, we are making judgment calls based on how we see these founders operate in the world. Well, I think I can trust that one a little bit more. And that one, not so much. You see what that one just posted on Twitter, not subscribing over there. That's not an actual sustainable method to think about governing the most transformative technology in history. And I've also personally started to reject when I hear policymakers say, you know what, this technology is just moving too quickly. It's too overwhelming. I don't know if we have the faculties to figure this out.

56:02I've actually started to personally reject that because that paradigm and way of thinking is only true if you are trying to drag the 20th century approaches to regulation to this technology. But if you are thinking much more innovatively, that is not a problem anymore. So now every time I hear that, I think, well, that's actually a design challenge. It's not a pace challenge or it's not a fundamental challenge of physics where this technology is just impossible to regulate. That's not true. There are methods such as yours and new frameworks and approaches. We just have to be much more innovative.

56:38And if somebody is struggling personally with the innovation side, then that's an entirely separate thing for that person to recognize that maybe they just aren't able to think about the moment that we're in. But you can't just drag the 20th century and then say it's not working and then say where it's behind and then say it's too quick. It's not. And that's the only thing we can guarantee about this technology in the future. It's not going to slow down. So the mechanism and the institution has to change because the technology isn't. No one's really pressing pause and doing a snack break so we can get our thoughts together for artificial intelligence.

57:11And there was something that you said at an economics conference we were both at about three years ago. and I've been thinking about it ever since, and it was what really inspired me to go follow your work and start reading about it. You said, we have to think about where does the boundary of an organization stop in a world with AI agents? What even becomes of the idea of the firm in a world in which one or two people have 10 ,000 agents and that is the company? How do we think about the nature of the organization in a world with these systems? Yeah, basically, AI is very rapidly putting us to the test on just about everything we take for granted about the way the world is structured.

57:57And so, you know, that point about do we even have firms? So I'm an institutional economist, organizational economist, that's my original training. and say, well, the theory we have of the boundary of the firm, like why we don't do everything through markets, like why isn't everything just a one-on-one transaction? Why do we have these aggregations so that we've got something we eventually give legal personhood to that operates as an independent entity? And that story is, again, it's about transaction costs. It's about governance costs. And it basically says, well, you should make the choice between having decisions made through a hierarchy, right, with a CEO and managers and employees that you instruct and decisions made across, you know, like through the market where you have to rely on contracts and property and so on to govern your relationship.

59:05So if all of those governance costs, because they're now questions not of, oh, how do we overcome the slacking incentive of employees or the private information and the conflict of interest that a manager might have, if now that's not the main thing that's driving the structure of we're going back to those just fundamental decisions about allocation and distribution. If what's driving that now is like the alignment problem and how well does an AI system actually effectuate the unarticulated goals and intents of human society, of the person who sent it out there and then the world at large, which also has an interest in how that happens.

59:57Yeah, so we just do we have firms anymore? I think the other piece of the firm that it's important, so when I talk about this boundary of the firm and do we have firms, we have this artificial boundary. Because remember, we were talking about identity. And we say, well, law created this artificial structure. So how do you create a firm? You go and you incorporate and you file documents and you say, here's this new thing, this new thing that can sue and be sued. And a part of that boundary of the firm is this information boundary, right? So there are trade secrets and inside the firm that the firm is allowed to get help from the state to make sure other people don't come and take those trade secrets.

1:00:53And we define what it means to be an employee of the firm to say you're loyal to the firm and you won't share the company's secrets outside. So this is what's creating this now this real boundary between the private technology company and the public and making it very difficult for us to say this. and I think this is totally new, actually. I think this is what is very distinctive about AI. Because of this artificial boundary that we've created through law, for lots of good reasons in the 20th century, because it drives innovation and creates good incentives. But today, it is significantly shifting the locus of where decision-making is happening about, oh my goodness, What are we building?

1:01:43Is this the direction we want it to go in? How do we want it? You know, do we want it to happen this fast? What do we need to do to be ready? You can't really regulate or do good policy about something you just can't see. And then, as you're pointing out, you end up relying on, like, the goodwill and good intentions of, you know, frankly, ordinary people who just happen to be the heads of these corporations. And, you know, even if you think those people are really, really well-intentioned, and I think some of them are, you know, that's just not a robust way of regulating. And it's not really a legitimate way of regulating either.

1:02:32And it's really interesting that you pose the questions around intellectual property and these boundaries. Because let's say I wanted to make a competitor to Pepsi or to Coke in the AI age. I could just build a bunch of AI agents and they are the sales reps and maybe one of them contracts an actual robotic lab to cook up the new formula that's going to compete with Pepsi. If that formula leaks or somebody copies it and there's nobody else at this firm but me, what happens to the idea of intellectual property? Or if the agent was hacked and another agent goes and builds that same can of pop, who gets sued?

1:03:10Who had the intellectual property rights? And if the agent came up with that formula, is that my formula for my who's even at the firm? So these are always profound questions that I don't know how we would think about in this era. Yeah, that's going back to the infrastructure point that says we haven't even created the infrastructure of identity and registration for these new actors in the economy, these new participants in the economy. And that means we haven't even, I sometimes talk about like a Velcro theory. We haven't laid down the little strip with the fuzzy surface that when we figure out what should be the rule about the intellectual property ownership for the new recipe for Pepsi or Coke, like the strip that has the little hooks in it, right?

1:04:06When we figure that out, we should have that first Velcro mat already in place so that we can say, oh, wait a second. We've been approaching this question of the IP ownership of AI from a perspective of our existing copyright law or patent law. We've had these decisions out of these different courts. and then we get bold and we get creative and we say well no we actually need a different way of thinking about IP for artificial agents you know we then then we won't want it what we want to do is to be able to quickly put something in place like that you know economists like to get up there and say I've got all these answers and I've got these predictions for you but mostly I've got I've got questions and prods to my fellow economists to say there's so much that we need to be thinking about what this new economy that's rapidly coming upon us, how it will operate, how it will function, and what kinds of legal rules, regulatory structures, infrastructure will need in place for it all to go well.

1:05:16So I want to ask your take on jobs as an economist, if you have a take on what that may mean. If you think about that world, or I've built a competitor to Pepsi with a team of 4 ,000 agents that maybe contract a robotic lab. Where do you see people in that future or work? Two thoughts. Well, a few thoughts. So I think when people think about the future of work, we should really be talking about the future of the economy. Right? Like, how is all of this going to change? And that's the way you're asking the question, Right. Like we now the firm is changing. So a very structure of markets, the way marketing works, like are we going to have websites for online shopping at all or are we going to have prices?

1:06:02Right. All of those really fundamental questions about the way the economy is going to work. Because I think we can't just continue to think about AI is just, oh, really ramped up automation. Right. Like really ramped up machines, I think. And so if we're making predictions about that substitutability between human labor and machine labor, I think we're not well grounded on making predictions. We're thinking about it linearly. Yeah, exactly. I like to always emphasize it's a complex adaptive system. We actually can't easily because so much is going to change. And economists know this because they know about complementarities.

1:06:42I like to say like one of the anecdotes I heard when I was thinking about, I was working on legal innovation many years ago. And somebody who would sort of follow the deregulation of the airlines said, you know, right up until the night before they deregulated the airlines, no economist was predicting what actually happened with deregulation, regulation, which was not just changes in prices and routing and so on, but rather the emergence of the hub and spoke system, you know, a totally different way of organizing airline travel. And so I like to say, so we don't actually know what this is going to look like.

1:07:23And so we should be thinking about, well, how are we going to put ourselves in the best position to be able to respond to what's happening? So I talk about the Velcro theory, get the legal infrastructure down, Think about what the components are for that. But if we, you know, if we focus in on the question of, like, what's the role of humans going to be? Because if, I mean, that's the definition of that open AI sort of put out there in 2015 or 2016. You know, artificial general intelligence is when machines, AI, can outperform humans at all economically valuable work, right? So that's a vision of, oh, all the economically valuable work is going to be done by machines.

1:08:12But I think this misses something really important, and I suspect it can be, and if we are intentional about it, it's more likely it will be, that any economy is making judgments about value. I mean, we use markets to make judgments about where should we put our resources. And if it's a well-regulated economy, we say, well, let the market decide that as long as we've made sure that we provide housing subsidies or we've got taxes that redistribute income or we've got rules about how safe the skyscraper has to be or how much pollution you can put into the air. right we but we leave a lot of that determination of where is the value how do we decide where the you know what what what it would be valuable to build there's not a fixed set of economic things to do right and then there's actually executing on the task of of you know building those those those phones or those cars or accomplishing those financial trades, right?

1:09:25Like we're constantly making judgments about what should we build, what direction should we build, how should we build, where does our economy go? That's the fundamental question of an economy. and there will still be all those judgments to make. And so one view of the future, which is a little less dystopian about this, is, well, that's what humans are doing. Humans are engaged in this process of making the decisions, collectively making the decisions, figuring out where the boundaries are, when you personally can make that decision and when it has to be a decision made in combination with others.

1:10:13I don't think any of us are going to want to say, like, let's go back to your, you know, your question about, you know, I've got a personal agent that's going to go do my shopping and, you know, look for a skincare product. I don't think you actually want to sort of set that in motion and then, you know, all of the decisions, no matter what you think of them, are ones that are made by this AI system. And, you know, if there were, you know, say ingredients you liked that caused, you know, toxic chemicals or they, you know, they created, they used too much energy, right? Like we as a collective have opinions about what we think is good and bad and where we should go and not go.

1:11:04We have personal opinions about how we want to live our lives. We have collective decisions about how we think our world should progress. And so I think one of the versions of where we head, and it's like the flip side of the alignment problem, is we're going to need humans to be fully engaged. And maybe now it will be possible for more humans to be engaged in that. I don't know exactly what it looks like, but I definitely have ideas and already started to work on, you know, what are the kinds of institutions you could build? What are the processes you could build to have those values decided by humans?

1:11:44And maybe that's what more of human life becomes about. It's, oh, it's the decision making and the agency of what direction do we go? Right. And I think that the phrasing, you have to first understand what the market's going to look like and the economy is going to look like before you can understand the jobs in it. And you said something quite radical that I think, well, will we still even have prices? And that's just an assumption that we take. Okay, yeah, right now we price products and that's how we make the decisions to buy them. This is too expensive. This is my budget. But how do you even think about, well, what do marketing jobs of the future look like?

1:12:22You have to first think, okay, well, there may not be prices in the future because AI agents could be looking at different types of different factors. There may not be a human viewing the product first. That could be done by agents. So you have to start there before you can think of, okay, this is what we need to preserve in marketing. Well, if there's no prices and there's nobody looking at the product, it's just a bunch of AI agents, that's something different. And then you can think about the scaffolding of work around that. But I think we tend to think of jobs as this linear, it's continuity into the future.

1:12:54We just do the same thing a little bit faster or around a bunch of agents. But the entire structure of the market could change, like a world without prices. That sounds so impossible to think about until you remember we invented prices. That wasn't a real thing. It wasn't a fact of nature and evolution. We just made up that system to begin with. If we were to think about timelines, where do you think we are? I know that we're probably going to need a few more breakthroughs to get to this chapter a bit more reliably. But where do you think we are on the path to the economy of agents? There's two parts to the way I think about that.

1:13:30One is because I think a lot about this invisible quality that we take for granted, that the human agents we hire are normatively competent. There's all kinds of things I don't need to say to the human I hired to go make a million dollars on the Internet. Right. Because I have confidence that they are well embedded in the norms and rules of the world we live in. I don't need to instruct them on that. That's what I've called normative competence. Because I think we haven't figured that out. I mean, I'm working on it in my lab. And one of the things we're trying to get a handle on is, oh, like how normatively competent are our existing systems?

1:14:15And the initial answers are not very. I mean, there's lots that they can do. But if it's a novel environment and you have to think about novel environments, they're not doing so well. So in terms of predicting where we go, you know, the focus is on, especially right now, the focus is on how capable are the systems and the agents at accomplishing, in particular, verifiable tasks. Like basically, we're building them to be really good at math and science and language, of course. But we're not currently focusing on, but are we building them to be good at integrating into the complex, cooperative, normative structure that is our world?

1:14:55So I think we may hit a technological limit that we've built things that nobody's actually going to want to send them out there. Like Mustafa and Suleiman can say, well, I think we might be a few years away from the agent being able to go out and make a million dollars on the Internet in a few months with a general instruction and is not thinking about. But nobody's actually going to do that because we have no idea what the liability is and we have no idea what kinds of crazy things it might do and how you control that. So that's one thing about the timeline. The other thing about the timeline is my worry is that we will get a flood of agents because right now you can send agents out into the world to do things.

1:15:52So I've just started work with a group that's trying to develop benchmarks for open world tasks, like how well does an agent like an open claw agent do at a real world task? And one of the first tasks was, well, let's try instructing it to come up with an app and get it posted in the Apple App Store. And it succeeded at that. Right. So we could have millions, billions of agents that get released into the economy well before we've actually figured out all this stuff about, well, wait a second, who's responsible? Who's liable? How do you trace? We don't have any law in place to trace it back to the human who released that thing.

1:16:44When I get anxious and actually when I, the last few months, getting up to give talks on, oh, let's get this infrastructure in place. Let's get ID registration, independent verification organizations in place. There's a little voice in the back of my head saying, oh, is it too late? Because is it happening so fast that we're going to have all these agents out there before we've built that infrastructure? I don't think it's too late. I sort of quiet that voice and say, this is what we need to keep pushing forward to do. But the timeline, is it possible we could have an economy of AI agents? I think that could happen really quite fast.

1:17:34Um, everybody said 2025 would be the year of agents and it, it really wasn't, but 2026, 2027. Yes. Um, and that's actually my version of existential risk is if we release that really quickly without any of the infrastructure in place, then I just, I worry that the, The danger we face is we just crash the economy. We crash our systems. Like nobody wants to invest. Nobody, you know, nobody wants to sell products because they don't know what they're interacting with on the other side. Right. So everyone's talking about AGI or artificial superintelligence causing some existential crisis rising up against humans.

1:18:24and you're saying, stop, the system's autonomous agents that could be live to air in six months, that's enough to crash the system in a different way, not because they are super intelligent and going to scheme up something, but because our infrastructure wasn't built for an environment where there's billions of autonomous operators that we can't oversee. And no matter how extravagant we think we can build a system to monitor them, we're not going to be monitoring billions of AI systems. So we need an entirely new system to think about that. And that is the existential crisis. And it's sooner than people are planning for.

1:19:04And even just that idea that AI could crash the economy, and not because, again, it was a malicious scheme, but because our system just wasn't built to hold what's coming. Right. Exactly. This is why I think the ID and registration for agents is such a critical. We could do that relatively quickly. I think that's, you know, as a starting point, like no matter what you think you want to do later, you're going to need to be able to do that. And we didn't do that with other kinds of software agents. You can sort to think about what happened with social media, right? We said, no, we don't want to create any liability structures.

1:19:47We're not going to create requirements of identification so that we can trace back who was actually accessing that system, who was actually posting. And it's a different problem, social media. I'm not saying it's the same problem. I'm just saying, can we learn from that and say, we should at least, this is not deeply regulatory. This should not be exciting, you know, this fight between regulate, don't regulate, you know, shut it down, let it rip. It's like, just build some basic infrastructure. It's lightweight. It's not that hard to create. It's not expensive to comply with. And it gives us the option and the capacity to respond.

1:20:32Because, you know, maybe I'm wrong. Maybe the agents that people will deploy will only be the ones that they have been able to verify, they have high confidence, will do what it is they want them to do. That's another view of how that future evolves. But I would really like us to be in a position to act if we need to act. And do you think we're going to run up against trickier challenges? Because you use the word norms a lot. There are laws and there are norms. And the law says you can't break the speed limit. And we know that. But we've also all socially agreed and accepted that in the event of a medical emergency, you're going to break the law and everyone is going to be okay with that when you need to get that patient to the hospital.

1:21:17How do you think about that in the world of AI systems? That sometimes we do break the rules and we've all accepted when it's okay, but it's not written down anywhere that that is the case. Yeah. This goes back to, as I was saying, the first paper I wrote when I started thinking about AI and taking the lessons from law and economics. And this is the reason alignment is hard is because of the incompleteness problem. That we can't express what we want an agent to do or what we want to set as a rule on the highway. And we can't fully explicate that in language. So we can set a, we can set a, we can say the speed limit on this road is, you know, 55 miles an hour.

1:22:08But as you just pointed out, the real rule says, unless you need to race to get somebody to the hospital. So the real rule is actually filled in. And we have expectations about the fact that, oh, and everybody would agree that was appropriate. And even if the cop, you know, sees you and pulls you over, when you say I'm racing, it's going to say, oh, let me put my lights on and get in front of you and take you at that speed. Because we have that capacity. We predict that's the stable world that we live in, where the norms are. And the alignment problem fundamentally is how do you build AI systems that are able to have that kind of natural way of thinking about it.

1:22:59I call it, again, normative competence of saying, oh, well, the rule on paper says 55 miles an hour. But everybody knows that everybody knows that everybody knows that if you need to race to get your kid to the hospital. Go for it. That's considered an appropriate behavior. And this is why it's really challenging. And this is why when we think about releasing AI agents into the economy and say, look, there's just going to be a constant stream of those kinds of circumstances that are not fully captured in the instructions you gave or the formal rules that will require this much more complex filling in.

1:23:47What will be the norms here? What would people accept? If I had to get up and justify why I didn't follow this rule, do I think that most people would say, oh, okay, yes, we all agree. That's a case in which it would be appropriate. And that's That's why I always emphasize that we don't have legal systems that just consist of written documents saying, here's what you can do. We have processes. We have courts. We have lawyers. We have judges. We have juries. We have legal argument. We have treatises. We have all this process to deal with all those cases where we say, well, you know, it's not quite clear what the law requires here.

1:24:35Or maybe we should really be reinterpreting the way we've understood the law in the past. And it's only a robust system because we have those, you know, those written documents and we have those processes. So, you know, I'll say like, well, we have this idea of constitutions, writing constitutions for AI. and, you know, that could be valuable information for the system to make judgments. But there's, in the human world, there's no such thing as a constitution independent of constitutional courts and constitutional doctrine and constitutional lawyers and all that process that we use to resolve all those places where it's ambiguous and it's incomplete.

1:25:24And we still are just figuring it all out together. And that kind of goes back to, and that's how we stay anchored in ordinary people. And, you know, like our judges are generally trained and, you know, we have juries that consist of just ordinary people that are brought in to decide even complex technical cases. And at the end of the day, You have to kind of get them to say, oh, yeah, we agree. That's bad behavior. Or no, we think that was fine behavior. We make judgment calls. And yeah, it keeps coming back to the idea that this institutions of the future are going to need to look as strange and emergent as the new technologies that we're building.

1:26:09And I'd say my final question, why do you think we are underestimating where we are with agents and what's about to happen and how quickly this is all moving? Because you have people that are truly in doubt about where we are in this moment. Why do you think that is? I am regularly stunned when I talk to even pretty sophisticated audiences, academics, business people, people in government. Because I've been immersed in this for 10 years. So I've been talking to these people for 10 years. I've been working in this for a long time. I'm just quite stunned by how little most people in our environments know about what is happening.

1:27:06Most people know about chat GPT. And so they know about chatbots. But even then, they may use them but not have interacted with them and certainly not exploring all the capabilities. but I'm just really surprised by how many people just do not know. And my colleagues at the university in other disciplines just do not know what is an agent. And, yeah, so one of the things I've been trying to figure out how to do is to spend more time sort of generating some writing and say, okay, here's what you need to know. I mean, I've been talking to social science audiences in particular and business audiences for a number of years, it's just not getting through to enough people.

1:27:55And we've got this narrative that, well, it's just hype. Companies are just trying to sell a product. And yeah, there's hype going on. But I'll tell you, most of the people I know in those companies, they truly believe this is not like they're just saying something they think is a load so that they can sell product. There may be some people like that in the industry, but most of the people I know have been in it for a very long time. And it's totally sincere. They could turn out to be wrong. That's why I was saying, like, you know, will we get to an economy of agents? I think there are still questions about that, technical questions about that, economic questions about that.

1:28:46But we too easily discount and say, well, I don't need to, you know, those people who are spinning those sci-fi stories. Now, I think that's shifting, but it's still a tiny population of people who are kind of in the weeds of what's happening. And It's another reason why we, I think, really need to be talking to people outside of the industry, people without that technical background. And I will always say, look, spend a couple hours. I can explain to you everything you need to know about how the systems work. I can't code either. But, you know, in terms of understanding what is happening and why, why it is not so straightforward to just say, oh, well, but, you know, a little piece of software will do what I want it to do.

1:29:39What's the big deal? To sort of really explain where that problem comes from and why things are moving in the direction that they have and what evidence we have about, like, the wild stuff that AI system, like, it really is alien intelligence. intelligence, that I think we need to get that conversation moving more into the public, and particularly into our governments, because governments really do need to be taking seriously that they have to figure out a way to act. And there are things we can be doing right now, I think we can be doing. Yeah. And that saying that's existed for over a decade, the future is here, just not evenly distributed.

1:30:25We can see the pillars of where the future is going. It doesn't have to be a surprise. I mean, as someone who studies for the future for a living, I'm often not that surprised by what emerges because you can see it in the investments, you can see it in the patents. And in this case, you can see it in the verbal declarations that companies are making. They're telling us what they're trying to build. And by simply just saying, I'm not going to acknowledge it or I'm not going to believe them, that doesn't mean that that outcome isn't going to arrive. And you certainly don't arrive at the futures you want by just not acknowledging the futures that you don't want.

1:30:58So I think, yes, on the one hand, it's an education problem. And on the other hand, it's recognizing this is the moment that we're in. And leaders are telling us what they are trying to build. And we should take that seriously. Professor Hadfield, it's been a pleasure. Thank you so much for joining me. We look forward to having you back on.