The AI Economist: The Skill You Need to Stay Employed in the Age of AI | Ajay Agrawal

16:42Sinead Bovell:It's very hard to make those types of estimates. In other words, what you're doing is betting against the general intelligence of the market, given how, especially right now, because things are changing so fast. You know, I've been studying e-commerce technology for about a quarter century, And I haven't observed a period where there is this much, this pace of change. And so it makes it very difficult to make bets. But of course, everyone investing has to. You know, every investment is a bet.

17:16Ajay Agrawal:And so would you think it's fair to say for people, expect some ebbs and flows. Expect there to be more hype, excitement with the technology. Expect people to also get kind of disillusioned with the technology. But over the long term, this technology is still not overhyped and it will change the game. But expect the ride to not be straight linear.

17:37Sinead Bovell:That's exactly what I would say.

17:38Ajay Agrawal:And so let's say all AI progress stopped tomorrow and there's just no more advancement in the field. Do the current AI system still hold enough capabilities to fundamentally disrupt the workforce and how we live?

17:51Sinead Bovell:I would say that the capabilities we have today with no further progress certainly have the capability of having a very significant impact in every industry. In other words, we've hardly scratched the surface of deploying what we already have. And so then when you compound that with the fact that the slope of improvement is so steep, we get to this point of uncertainty.

18:21Ajay Agrawal:Because my fear is that, I mean, GPT-5 came out and it was underwhelming for a lot of people. And so there was a lot of speculation. We're starting to hit a wall with this version of generative AI with this architecture. And we're not going to see too much progress. And so I have two concerns. One is that people start to check out from thinking they need to pay attention to this technology because they see a headline. Progress is slowing. It's not going to turn out to be what people promised it will be. And then they think they don't need to pay attention for their own job. Or a business thinks, yeah, this is overblown.

18:57Ajay Agrawal:I'm going to turn away. But you're saying even if we stop today, there is still so much to grab from the current systems that it's still going to disrupt either your job or your company if you're not paying attention.

19:09Sinead Bovell:Yes.

19:10Ajay Agrawal:And do you have any bets on AGI? And AGI is artificial general intelligence, so this is an AI system that could be just as smart as anybody at everything. And that's what companies see as the holy grail. Everybody is positioned towards AGI.

19:26Sinead Bovell:Yeah. So I don't really understand the line in the sand that people have drawn around AGI. When people say smarter in every category, I think what they mean is, for example, able to answer technical questions. Like when Grok 4 was released by XAI, Elon Musk's company, one of the key benchmarks that they used was something called HLE, Humanity's Last Exam. and that is predicated on a large number of very deep technical questions like phd level questions from many different fields chemistry physics biology and so on and the idea of this benchmark is that an ai that's able to do well on this would be not just smarter than you or i but smarter even than a nobel prize winner because nobel prize winner might be able to score reasonably well let's say if they're a Nobel Prize in chemistry in the chemistry questions but not necessarily in the physics questions or the math questions or the Latin questions and my view was okay let's imagine we had that kind of oracle what would that do to us today in fact I was sitting with with a couple of my colleagues and we're saying like what if what if someone creates that genius level of intelligence who would it even replace like in other words if you go like pick any large company and say, how many Nobel Prize winners do they hire?

21:00Sinead Bovell:Like, do they employ? Like most companies employ zero. Few might employ one or two. And think about like a baker or a, you know, a retail store or a manufacturer. How many geniuses do they hire? How many do they need? And then we sat there and said, well, what organization does employ a whole bunch of PhDs? And we were thinking, and then we just looked at each other and said, wait a minute, it's ours. and ours is arguably quite dysfunctional. And I don't mean just, you know, ours, like every university. In fact, I would say one of the slowest adopters of AI in terms of how to effectively use it for teaching and education has been universities.

21:43Sinead Bovell:And so it's not at all obvious that having a bunch of geniuses, you know, packing to an organization is going to, you know, all of a sudden transform it. so for me the the AGI line is um is not so obvious what the implications are in other words it's not obvious that it's more than just a continuation of what we're already doing like the increasing capability uh that can do more and more things and all the people that have been most of the people have been talking about AGI have been talking about AGI in a box meaning like in in um dario's uh the ceo founder of of anthropic has an essay um machines of loving grace and he describes achieving this this sort of level of agi bar me that he describes as a country of geniuses in a data center and the reason he describes it that way is because they can do anything you and I can do on the computer, but they couldn't pour this glass of water.

22:51Sinead Bovell:So there's no physical instantiation. And the reason for that is just that because robotics is so far behind the intelligence that we have online.

23:00Ajay Agrawal:Speaking of jobs, I mean, the disruption is already happening. We're starting to see it in the numbers. There was a Stanford study released by some of your colleagues called Canary and the Coal Mine. And it showed that specifically for young workers, so if you're 22 to 25 and you're in an AI exposed field, so this is the finance, marketing, computer science, accounting, audit, there was a 6 % decline in their employment. And then there was another study I think was called seniority-based or seniority-biased technological change from generative AI. And it showed that for new college grads, there was a 22 % decline in employment in the workforce.

23:39Ajay Agrawal:So what is this telling us about how this AI disruption in the workforce is going to unfold?

23:46Sinead Bovell:There is a fear that the things that AIs are good at are things that new employees typically did. And so that the first place that AIs are biting with respect to displacing human workers is at the entry level. There are a few things about that Canary paper. One is that overall employment went up. Two is that although the number of employed at the youngest age went down, the wages didn't. Now that's very curious. For a labor economist, when demand falls, wages should fall. So that raised the question, well, why didn't wages fall? And part of this was potentially what that paper was measuring. It was measuring people working at reasonably mid and large firms.

24:48Sinead Bovell:And so, for example, software developers, if there was an increase in young people going to work for startups rather than the mid and large sized companies that were included in that study, they would fall out of the data set. But they're still being employed. Just they're being employed in places that aren't being captured in those data. The conclusion that people jumped to, which was this was the first evidence of our greatest fear, which was AIs were coming into the workforce and they were starting at the bottom and they were going to start working their way up. And maybe that's what's happening.

25:23Sinead Bovell:But it's still too early to tell. The fact that that paper got so much, like everyone's talking about it, is just a reflection of how little data we have. It's all so new. It's just a very short time window. um you know since effectively like this whole effect that they're measuring was a couple years before the release of chat gpt in november 22 and then a couple years after and so we're still very early and that's why they call it canary and they had a question mark at the end of the title you know canaries in the coal mine question mark it's in other words they were saying this isn't definitive and so i think we shouldn't ignore it but i would also say it's not conclusive um and it It could be that some of those people, or in fact, many of those, the people that look like they were missing or were dropping out were going to work somewhere else.

26:13Sinead Bovell:And especially young people in software development. That's not too hard to imagine.

26:18Ajay Agrawal:So all of the discussion about software development being not the best career to go into anymore because AI is going to slowly creep into that territory, you would say maybe that's actually overblown. And if somebody has studied computer science and they've just graduated, not to fear, because we're not seen in the data where these people are getting hired, but it's because they're going to be in startups, in places that just aren't in the data where we're measuring jobs. So somebody shouldn't be super fearful if they graduate as a computer scientist?

26:51Sinead Bovell:Well, all I'm saying is that's a possible explanation. So it's just too early for us to know. but what I would say to people in software or any field also all AIs are computational statistics that do prediction and sometimes it feels like there's more like there's you know a ghost in the machine that feels alive because they can communicate in natural language but all of that natural language is is produced using computational statistics so the whole thing whether you're using AIs for language, using AIs for vision systems, using AIs for control systems to control robots. It's all statistics that does prediction.

27:32Sinead Bovell:And what AIs cannot do is they have no judgment. They have zero judgment. And so no matter what field you're in, if you're in software development, in medicine, if you're a baker or an artist, the thing that we have, the machines don't have, is judgment. And so just to give listeners a mental model of judgment, I teach at a business school. We're sitting in it right now. If you came to the business school 40 years ago, the primary subject that people studied was accounting. And they would also do finance and marketing and so on, but accounting at that time was the major, the dominant field. And for accountants, most of what they did all day was arithmetic, adding, subtracting, and so on.

28:23Sinead Bovell:In fact, there was a homework assignment that people would get. They say, go to your phone book in Tarot page 47 and add up all the phone numbers. And the reason that was a homework assignment is people had to practice their adding. They would just add up all the numbers, carry the one, all the longhand form of arithmetic. they spent a lot of time building the muscle to be good at arithmetic because 80 percent of their job was doing arithmetic then all of a sudden along came spreadsheets and now it didn't matter if you were sort of mediocre at arithmetic or you were excellent at arithmetic the machine was better than everyone it was superhuman it was the the arithmetic version of agi and you might imagine if these things came along and now in an instant the machine can do all this arithmetic perfectly, never makes a mistake that we wouldn't need any more accountants or very few.

29:17Sinead Bovell:But yet there are accountants all over the place. There's so many accountants out there. And the question is why? Like what are those people doing given that the spreadsheets can do all this stuff? And the answer is they're applying judgment. They are no longer doing the addition and subtraction but they are deciding what numbers should we give to the spreadsheet and then spreadsheet does this magic and then what numbers like how should we interpret the output and so they are applying their judgment ais don't want anything they have no wants so where do they get their direction from from people a key thing a person needs to do when they set off to to build something is they have to want to build a thing and then they have to want to be able to articulate what its capability should be and how it should function and how it should interact with a person and so on.

30:09Sinead Bovell:And so, and then when it actually does its job, they have to be able to assess it and say, okay, you know, I can either make this tool work faster or it can take longer but give a more detailed answer. And so all those are trade-offs, judgment. So the thing that I would encourage every student, whether you're in computer science, software development, or any field, is as you're developing your trade, recognizing the difference between what's effectively prediction and what's judgment. And the more people develop their judgment muscle, the more they will be able to contribute to the overall production of stuff and work alongside AIs.

30:55Ajay Agrawal:So judgment becomes you have access to the best supercomputers in the world. What do you ask them? And when they tell you a result, is that good? Or which answer do you even go with? So you're basically evaluating the output of supercomputers, understanding how to apply that. And those skills will become more important potentially than anything that you're doing right now in your job. So doesn't that also mean somebody who is excelling in marketing today and they're maybe running the numbers, knowing which campaign to pursue? In a world with a supercomputer, that might not be the best person with the strongest judgment to handle how AI comes into the marketing department now.

31:37Ajay Agrawal:So there may have been someone that's not as great with the numbers, but they're better at judgment. They're better at understanding tradeoffs. And that could be the new person that's best at marketing in the AI age.

31:47Sinead Bovell:Yes, that's exactly. And that was one of the key points in our book, Power and Prediction, was that the power, who has the power in different organizations may significantly shift for exactly the reason you say. That the skills that gave you dominance before AI may not be the same skills that give a person dominance after AI.

32:10Ajay Agrawal:So you could even be somebody with great judgment in marketing, and then you could move to finance because you don't really care about the numbers, but you're great at assessing trade-offs. So you become even better in that department, which means org charts could look really different because the skills change. Do you think, so one, is the future of work more about skills than it is jobs? And two, would you say that that is the most important skill for the future of work?

32:34Sinead Bovell:Yes and yes. Yes, skills will trump jobs. And yes, in our view, judgment becomes the number one skill.

32:42Ajay Agrawal:So if the future of work is less about jobs and it is more about skills, and you're a new grad, you should be doubling down on building judgment and understanding whatever it is that you studied. Figure out how to direct AI systems in that field. Figure out how to weigh what an AI gives you. Is that good? Could you ask for more? And maybe you actually surpass people that are, because if we go back to that study, the canary in the coal mine, and you had mentioned that wages stayed the same and that employment actually grew in those same departments or in those same occupations where new hires weren't getting hired at the same rate.

33:21Ajay Agrawal:And even what was fascinating is that even if you were just 30 years old, you've been in the workforce just a few years, your employment was stable or continued to grow. So it was just the new hires. But if you come out of college and you can't get a job at some of those big organizations.

33:36Sinead Bovell:Or you choose not to get a job.

33:37Ajay Agrawal:Or you choose not to get a job. That's a whole thing, too. That's a different type of a journey. Would you recommend or advise new grads double down on judgment, start understanding what AI means in your field, and then move towards a startup or start building your own experience? Do you kind of put together your own apprenticeship in a way and start to patchwork yourself into your career?

33:58Sinead Bovell:It's a great question. I think a significant shift in education that we'll start to observe is the difference between learning from reading versus learning from doing. And when I say doing, the key thing I mean by doing is making decisions under uncertainty that have consequences where the person who's making the decisions owns the outcome. And the reason is that when you own the outcome, then you feel the pain of a bad decision. And so that creates a loop where you make a decision, there's an outcome, the outcome is either good or bad, and you own the outcome. And the reason I think that's very important is because that loop creates judgment because you start getting far more attentive to the trade-offs.

35:01Sinead Bovell:And trade-offs are the essence of judgment. So you can think of judgment as occurring at two levels. First is in just preferences, like what do I want? And so do I want to build a thing like this or do I want to build a thing like that? And then as I'm making decisions, I'm weighing different outcomes. And that weighing of trade-offs is the second form of judgment. And so the best, most salient way to develop that muscle is to actually make decisions. And so we have created an education system that's largely about reading. You and I both participated in a reading group. And a member of that reading group is Rich Sutton, who recently was awarded the Turing Prize.

35:55And he recently wrote this essay that we are shifting from an era of data to an era of experience.

36:03Sinead Bovell:And his point was that these AIs, in order to get over the next hump in terms of the next level of intelligence, is they can't get that much smarter from simply reading. They have to have experience, meaning take actions that have outcomes that generate feedback. And I think it's the same for people.

36:28Ajay Agrawal:So if you're already in the workforce, what should you be doing today? I mean, even if you think your company hasn't talked about AI, you feel pretty comfortable in your job, what does that mean you need to be building and doing to build that judgment skills because you're seeing everything that's in a book or simply on the internet, expect AI to do it because it's probably Reddit and it's Reddit more in depth and more times. But it hasn't done any of the actual doing, but people have. So if you're a marketer right now that's been working for 10 years or in finance you've been working for 10 years or a sales rep, what should you be doing in this moment to start preparing for AIs that will inevitably step into your department?

37:10Sinead Bovell:it people will need to become comfortable with with a much higher velocity and propensity to ask why you know if i asked you in terms of just let's say branding um this is called i've got questions and i'd say why do you pick that um you know why do you pick that name and like what were the two other names that you consider for this and then you would say well i picked this one because maybe it appeals to this kind of demographic or because and then i said well why do you care about that kind of demographic and i would be asking you why and the reason i would keep asking why is because in your answer you would be implying trade-offs well i wanted something that would appeal to this kind of person and then i would say well that means you're that you're making a trade between this kind of person or that kind of person uh or that you said i want these kinds of conversations I want them to be sort of, you know, the conversations to be a, a, um, representing the types of questions that my listeners will have.

38:14Sinead Bovell:And then I'd say, okay, well, that's a trade-off between, you know, having that focus versus a different, but each thing I'll be asking you why. And as you answer my questions, it will force you to be thinking about the trade-offs that you're making. And so you would have answers to all of that. People's job will be to direct the AIs by applying their judgment. And their judgment is reflected in their reasons for why they will choose one thing or the other.

38:41Ajay Agrawal:I think that is the most important piece of advice on the future of work. Because we hear everybody has to learn how to use AI. And that is that AI is going to be like a computer. When you show up at work, we expect that you can operate that. We don't ask that anymore. But what is part two post-asking an AI a question? What are the actual ways that you continue to differentiate and compete in the job market? It's not working with AI because we all are going to have to do that. It's what you have just summarized. And I think that that's absolutely vital. If we're going to, I want to understand that the structure of the knowledge economy itself, because if you are a lawyer or you are an inside sales rep or you work in finance, the modern economy was built on the assumption that the cognitive skills required to do your job are relatively scarce and now we're seeing ai systems be able to do those same cognitive tasks for pennies so what happens to the structure of the knowledge economy and to all of the people in it when the cognitive work they've built careers around that we've all built careers around can now be done by AI at a fraction of the cost and a fraction of the time.

39:56Sinead Bovell:I'm going to sort of pull a thread from your first question through to this question, which was your first question was about, is this hype? That if you take health care and the way you and I receive health care, it's really bad in terms of how expensive it is and the quality of care in many cases. Not all, but in many cases, the quality of care. there is such enormous room for improvement that it's just hard to fathom how much better it could be relative to what it is now uh like in other words i think it'd be much much much better and by better i mean cost adjusted better so that it is not just quality care better but much cheaper and therefore much more accessible so much would have to change for that to be true that, yes, we will have to reorganize the way everything works.

40:47Sinead Bovell:Yes, we will have to have a totally new division of labor between people and machines in order to be able to provide that kind of service at that low of a cost. And so the number one recommendation that I have now for organizations is to create the systems inside their companies to enable experimentation. because nobody knows. Like when you asked me about the org chart, yes, it will be different. If you had asked me the next why that I felt was coming down the line, which is, okay, how will it be different? The answer is no one knows. So nobody knows how hospitals are going to be different. Like, you know, people, and I think amongst the best at this are science fiction writers.

41:33Sinead Bovell:They are very good at imagining, okay, if we have this technical capabilities, what would the hospital of the future look like? And they, you know, have to paint the picture as a science fiction writer. The one thing we know is it should be drastically different than it is now. And so how do we get there is through a whole series of experiments. Every employee adds value by asking why. Like that is their job. And because the machine can keep doing stuff, but it never has a preference. It only does what it's been told to do.

42:00Ajay Agrawal:So we aren't, at least in the short term, looking at a bunch of layoffs and a rapid decline in employment, but it almost becomes more on a micro level. So if you aren't able to build that judgment skill in an organization and you're not great at working with AI, your job might be at risk. But overall, as the cost of doing cognitive tasks falls, we'll probably just use more of them. So it might be who's in organizations may change, but companies are still going to be needing people to drive these machines that don't have any desire and to decide the output of what the machine gives you is actually good and worthwhile and how you implement it.

42:47Yes.

42:48Sinead Bovell:And when you say cognitive tasks, in both our books, Prediction Machines and Power and Prediction, we write about two core cognitive tasks, prediction and judgment. And so while the AIs are getting better and better prediction, we have made zero progress on AIs having judgment.

43:06Ajay Agrawal:And what's an example of a prediction, say, in healthcare or in marketing that somebody would be able to understand?

43:11Sinead Bovell:So a prediction in healthcare is, let's say I have a lesion or a mole on my arm. I'm not sure if it's cancer. I can take my phone and a picture and an AI evaluates the image and predicts cancer or not cancer. The same way that a doctor would look at it and predict cancer or not cancer. Except the AI is being trained on millions of images and the doctor went to medical school and got trained on only thousands of images. and the judgment is if the doctor says well we can do this treatment here's the benefits here's the the risks of the treatment what do you want to do weighing those trade-offs is judgment and so and that might depend on my age it might depend on my uh like how much i like sports or how much I do this or how much I did my lifestyle.

44:02Sinead Bovell:And so it's up to me to weigh the tradeoffs and make a decision. Or it's up to my doc if my doc needs to be there with me to help me weigh off the trades and can ask me questions and infer from the things that I'm saying what the right tradeoff is for me.

44:21Ajay Agrawal:And then if we were to zoom in on an actual micro skill level for a particular occupation. So if you are a lawyer or a writer or even a consultant, being able to write well is a barrier to enter that field. If you can't write well, you're just not even in the running at all. So now that we have AI systems that can write better than most people, does that mean we're going to see more opportunity for people to become lawyers who can think really well, but they can't necessarily write? Or does the competition in journalism and legal fields actually become more fierce? Because you can no longer lean on being a good writer.

45:00Ajay Agrawal:The competition moves upstream to how well you can think. So now it's about how do you think through that case? Are you able to scenario plan and almost war game what your opponent's going to do? And the thinking becomes more competitive because the writing in some ways has been automated.

45:17Sinead Bovell:Yes. Don't think it'll be any any less competitive it's just the skill that becomes the the basis for competition uh shifts and the example you gave there let's say in law and writing i think is a it's it's a very good one for a broader point of redesigning the factory floor just like we talked about earlier with electricity you know there's a cartoon that's gone around the interwebs um two-frame cartoon And in the first frame, somebody says, oh, you know, I have these ideas. I'm going to use an AI to draft a three page email. And then the second frame is the person said, oh, I just got a three page email.

45:59Sinead Bovell:I'm going to use the AI to summarize it down to a couple of points. And so people look at that and they joke because everyone knows that there's some of that's going on. but it actually hides something that's really foundational for the factory floor. So step one is you just sort of type out your thoughts. The AI then takes those thoughts and predicts the essence of them. Then you receive the email. Traditionally, you would read the email. Now, if two hours later you went to talk to your producer about that email, you would convey the few points in the email you could not remember the specific word sequence that they said you you would not be able to recite the three pages that they sent you and so that begs the question we currently have a factory floor where it goes from an idea maybe jumbled that may be clear in my mind to then right now we put into an ai then the ai determines the predicts the essence, and then based on the essence, it predicts the sequence of words.

47:07Sinead Bovell:Then we send the sequence of words, then an AI reads the sequence of words, and then goes back and summarizes it. A lot of those steps can be collapsed, because in essence, the only thing that you cared about that you then wanted to talk to your producer about were the key idea. You didn't need all the extra stuff. And so when we're thinking about these very, very beginning of this discussion, you asked about the economic impact. The long-term big economic impact is going to be a result of that. It's going to be a redesigning of the factory floor, whether the factory floor we're talking about is in law and the way we conceive of ideas and communicate the ideas, whether it's in, you know, when we talked about medicine.

48:01Sinead Bovell:If you go to see your doctor, you walk in and a typical patient-doctor interaction, let's say seven minutes. And in that seven minutes, the doctor asks you some questions that maybe she puts on her stethoscope. She might, you know, listen to your heart rate or, you know, do some tests and then your seven minutes is up and while your doctor's talking she's taking a few notes at the end of the day all the patients have gone home the doctor sits at her desk and she will fill out her charts then they she sends her charts uh they probably go to india and some people there receive the charts overnight they read them and then they convert them into reimbursement codes so they write down the reimbursement codes uh of of what happened in that patient-doctor interaction so that the doctor in the hospital can get reimbursed.

48:48Sinead Bovell:Then they send those codes back to the hospital in the U.S., and then the hospital then sends out the reimbursement codes to whoever the payer is, like Blue Cross, Blue Shield, Medicare, whoever, and then they send a payment. All of that process originated from data that was created in those seven minutes. now today we are building AIs for example there's a number of companies are building AI tools for doctors they say hey you know you can use our tools so that the AI can generate your doctor's chart so rather than sitting there for two hours at the end of your day when your patients have gone home the AI can do that and it'll take it from two hours down to 15 minutes and then you take those charts and then you'll ship them to India and then in India now they've got AIs that We'll read the charts and we'll take a task that used to take an hour and make it three minutes to identify what are the reimbursement codes and so on.

49:43Sinead Bovell:But they are each AIs just increasing the efficiency of that step, but they're not changing the factory floor. But you can see that everything's just from that seven minute interaction that ultimately we will collapse all of that process. and right in those seven minutes where that information is being generated, you could imagine the reimbursement occurring at the minute the patient walks out the door at the end of seven minutes instead of that whole chain. And so those types of jobs that are in a workflow that can be collapsed, they will be reoriented towards, for example, things like audit.

50:32Sinead Bovell:Is this like a legitimate claim? And is this following an appropriate process? And so which is tied to liability. So, you know, who's liable for what? But that is a completely different emphasis of skills than the ones that are the basis of the jobs in the current factory floor.

50:54Ajay Agrawal:So humans are actually going to have to bring more to the table in a world with artificial intelligence. Because if you, for example, with the writing example with law, the competition moves upstream and it becomes more about the thinking. In a world where you were the person transcribing a doctor's notes, now you're the person deciding who could be liable here. Is this claim legitimate? All of the skills actually become a bit more challenging in a world with artificial intelligence. So when people say we're stepping into a world that's going to look like WALL-E and nobody's going to be thinking for themselves, that's actually not true.

51:30Sinead Bovell:No, there'll be a lot of thinking. I mean, applying judgment requires a lot of thinking. I suspect there will be some transition period where there will be some jobs that feel like less thinking in between the time where AIs are very good in the digital world, but are very weak in the physical world. So there'll be that window of time where AIs are doing a lot of the sort of sophisticated prediction tasks. And because there's so poor in the physical world, there'll be a lot of jobs of just sort of implementing what AIs want to do. An example of that are Uber drivers, where before you had to know the city.

52:09Sinead Bovell:You had to be knowledgeable about the city to drive a taxi. Now you can put your brain on autopilot, but that's just because we don't, you know, that the physical implementation of AIs is still far behind. Once that catches up, then we will really be in a world where our job is judgment. And, you know, everywhere we've got people is people applying judgment.

52:33Ajay Agrawal:And the examples you gave, it also means that AI could lead to less inequality in the workforce because it means somebody paired with an AI could do higher order work than they're currently doing today.

52:45Sinead Bovell:Well, maybe. Depending on the goal. It depends on judgment. In other words, what we don't know yet is will judgment be more evenly distributed than prediction was or will it be even more skewed? If it's more skewed, then there'll be even more inequality. If it's more uniform, then there'll be less inequality. And we just don't know yet.

53:08Ajay Agrawal:So what does that tell us about what we should be learning in school? So if we can easily see judgment and experience are key for the future of work, what should colleges be doing? How should they be reorganizing themselves? I mean, what conversations are you having here?

53:23Sinead Bovell:Trade-offs. So much has to do with it. Rather than learning facts, learning trade-offs, and therefore always asking why. So, you know, in other words, let's say in history, rather than memorizing the facts, was at this point in history, given these things that just, you know, had happened, was a better decision to do this or do that? Why? And every time you ask the question why, then you're forced to think about the trade-offs. everything you need an answer for because that is how the role that we'll play in guiding the AI. And so it feels very much like we're heading into a world where the reason we have to be so good at why is because the AI does all the work sort of up to that point and then stops.

54:13Ajay Agrawal:So we're all going to become executives, miniature chief executive officers of a bunch of AI systems in the same way you would ask your team, why did you pick that? Why is this your presentation? We all have to do that for each and everything we do. So when you think about college then, how easy is it for a college to be redesigned this way? Is college still worthwhile? But how and what college teaches needs to fundamentally be reorganized? Or are we going to get to a point where it's not going to be the investment that it once was?

54:44Sinead Bovell:Yeah, it's a great question. So definitely it needs to be reorganized. that feels for sure true. What's less obvious is, is the current incarnation of university the best way to teach the skill? We don't know. We don't know whether it's the best way. And one of the reasons is, if the best way to learn judgment is by actually making real decisions that have consequences where you own the outcome, universities aren't the best designed for that. um so but we don't know we don't know what the best way is of of teaching judgment uh you know i'm putting this out there as a um as an one of many ideas of how we teach judgment is through is through active decision making um but what i will say is that universities did adapt in the case of accounting so in accounting if you would have said 50 years ago to an accountant your main part of your job is always going to be asking why.

55:49Sinead Bovell:Accountants would have said that's absurd. But now that's all they do because the machine does all the arithmetic.

55:55Ajay Agrawal:We've talked a lot about the cognitive workforce, but a few years ago you were on the Sanctuary podcast and you asked a question and I actually want to rephrase it back to you to hear your answer. So you said human humanoid robots will become the largest market in history, But what will be the triggering event that will happen that will cause the penny to drop so that the rest of the world sees that too? How would you answer that? What will be that mark?

56:26Sinead Bovell:Yeah, so first of all, for listeners, Sanctuary is based in Canada, had growing in Vancouver, was one of the first companies in the world to focus on humanoid robots, what they call general purpose robots with human-like intelligence. And then a few years later, after Sanctuary began working on this, Elon began his work on the Optimus project. And Elon would tell Tesla shareholders, this is going to be the biggest market in the world. It will dwarf the automobile market. and everyone finds that very just... Wall Street seems does not process it. In other words, when you introduce Optimus, they didn't do anything to the share price of Tesla because it just feels so science fiction.

57:14Sinead Bovell:When will the penny drop? I think the penny will drop when there is a first implementation of a generalized capability in the physical world that feels similar to what people experienced in December of 22 with ChatGPT. There was something about ChatGPT because it was general, that you could ask it anything. And, you know, it would make mistakes, but it had a reasonable attempt at anything. And at that point, people sort of started to realize,

57:51Ajay Agrawal:wait a minute, this feels like we've entered a different category of thing.

57:55Sinead Bovell:Right now, the humanoids are very limited in their capability. In the videos that you see online today, they do extremely narrow tasks. So people watch them, they're curious, they're interested. But until you see one that's in a room like this, that is able to just do a bunch of different things on command, and you haven't been given a menu in advance. So if they say, oh, Sinead, you can ask it to do these 11 things, you'll ask it, but I don't think you'll be that impressed. it's when you ask it to do a thing that has not been told to you, and it just goes and does it, that all of a sudden you think, wait a minute, we've entered a new category.

58:36Sinead Bovell:And so that's when I think the penny will drop, and it has to do with generality.

58:40Ajay Agrawal:And you're confident that moment's coming. So when people look at robots and they think, it's taking five minutes to pick up the coffee mug, who's hiring this thing? You're saying, no, no, no, there will be a chat GPT moment in human robotics, and everything is going to change.

58:52Sinead Bovell:Yes, there absolutely will be. I mean, at this point, it's just now turning the crank. It's engineering. It's getting, it's just making, in other words, there's already a base level capabilities and it's just the physical part of it is slow. And that's just engineering. It's just going to get faster. The fingers will get more dexterous. They'll have more degrees of freedom. And their cognitive capacity, like their library of things they can do. You know, the way that we do language is we predict the most likely, the so-called next best token, the next best word when you're forming a sentence or a paragraph.

59:30Sinead Bovell:And so you generate a sentence or a paragraph by predicting word after word after word. And when you and I receive it, it feels like a human-generated sentence. There's all these things we have, all these jobs we do with so many different physical tasks. but those physical tasks are like you think of them like a book there's so many different books and essays and news articles and blog posts but all of those blog posts and books are a re-sequencing of effectively 30 letters and characters punctuation marks so there's a very small number of symbols that can just be re-sequenced to produce all these different books what's happening in robotics is they're training robots to do a lot of things to do a small number of verbs pick up place speak look at read so on um and once you train those robots to do those verbs then it becomes a job of predicting what's the right sequence of verbs to complete the task and there's so many issues of implementing that in the physical world that each one is a very significant job to overcome.

1:00:47Sinead Bovell:But the process is underway, and I'm sure we will hit a series of unexpected snags as we go, but I can't imagine that this is not going to happen.

1:01:05Ajay Agrawal:And how long, if you were to put a time frame on it?

1:01:09Sinead Bovell:I would say that we will have a, what I will call a factory grade general robot. So let's call it humanoid.

1:01:25Sinead Bovell:In 10 years. And by factory, what I mean is it's able to do a general, a wide range of tasks, but in a controlled environment. And then I would say we will have a generally capable robot for an uncontrolled environment, like walking down the street or in your house, in 20 years.

1:01:46Ajay Agrawal:And that's the robot that can do any physical job a human can do that we see today. It could also do.

1:01:51Sinead Bovell:I wouldn't say any, but I would say a lot.

1:01:53Ajay Agrawal:And so do you think if we're to found...

1:01:55Sinead Bovell:And again, it doesn't have judgment.

1:01:58Ajay Agrawal:Okay.

1:01:59Sinead Bovell:It doesn't have judgment. We're still, whether it's a physical robot, it's an oracle and a computer, we are still guiding these systems.

1:02:05Ajay Agrawal:We are the ones with the desires, the wants, the evaluation capabilities. And so my final question, is there a potential future at all where we could be looking at a post-work world? Do you as an economist think that's not out of the equation? We don't know when. We don't know exactly what that would look like. But I can't say with 100 % certainty humans will always be the main entity in the workforce.

1:02:28Sinead Bovell:So there is a venture firm called Bloomberg Beta, and we were hosting a conference a few years ago, and one of the partners there, a fellow named Roy Behat, I asked a question like yours, and we're talking about the future of work and so on, and he said, we're already there. so he would point to you and i sitting having this conversation and he would have said do you think in 1950 anyone would have called what you and i are doing right now work um he would have made the point that uh we are in a post-work world um and it's just we we don't notice it and it's just such it's sort of a long arc transition and so you know one of the things I think about is, did you watch that TV series Downton Abbey?

1:03:22Sinead Bovell:Okay. So in Downton Abbey, you know, there's the Lord Grantham and the family that lives upstairs. And then the servants, you know, who work downstairs. And as the series goes along, they're, you know, they're living their lives and they're doing their things. You know, they don't work. The Lord Grantham's family, none of them work. But it doesn't mean that their life isn't easy and it doesn't mean they don't compete. They are competing for other things. So in other words, there's always something scarce. They're competing for status. They're competing for affection. They're competing for power amongst their class of people.

1:04:03Sinead Bovell:They're competing for recognition amongst the charitable class and so on. So they're not being paid for it, but they're competing. And there will always be scarcity. And as long as there's scarcity, there will always be something that we would view as work. And a distinction here is, are we paid for it? And do we need to be paid for it in order to to do it. But I don't think that, you know, even if AIs are able to do all kinds of things, that we'll all be sitting around with nothing to do. We will always be, in some form, working towards whatever our objectives, our goals are, and competing for scarce resources.

1:04:56Ajay Agrawal:Ajay, it has been a pleasure. Thank you so much.

1:04:59Sinead Bovell:Thanks, Sinead.

1:04:59Ajay Agrawal:Thanks so much for joining us for this episode of I've Got Questions. If you've got questions, we'd love to hear them. Send us a message on our website. And if you found this episode interesting, we would love for you to subscribe to the channel and share it with someone you think may also like it.

1:05:13Sinead Bovell:All right, we'll see you next time.

1:05:15Ajay Agrawal:I've Got Questions was created by me, Sinead Boval. The show is produced and edited by Tara Cutts and Sandra Itainan and executive produced by Paola Piers-Torres. Artwork by Corey Vincent at Feel Studio.