Alex Imas and Phil Trammell – What remains scarce after AGI?

18:31Phil Trammell:There aren't that many institutions that have thought as hard as Jane Street about how to turn smart people into some of the most competent researchers and engineers in the world. This relies in part on an apprenticeship model, where new hires are paired with senior mentors. But Jane Street also runs a bunch of classroom-side lectures and hands-on boot camps. These courses cover a range of topics, and they go pretty deep. There's one lecture that focuses on reverse engineering systems with tools like S-Trace and GDB, and another that teaches you how to profile code down to the cache hierarchy level.

19:00Phil Trammell:Importantly, Jane Street designs these courses not just to teach the relevant object-level skills, but also to impart the relevant tacit knowledge. For example, their week-long neural net boot camp starts with general theory, but then quickly progresses to how to apply neural networks to trading. And here they cover the specific obstacles that Jane Streeters tend to encounter and the workarounds they've come up with to get around them. Jane Street takes this sort of learning incredibly seriously. Every office has dedicated classroom space, and courses are prioritized as part of regular work. If you'd like to work at a place like this, Jane Street is hiring.

19:31Phil Trammell:You can check out their open roles at jainestreet.com slash dorkesh. There's one possibility, which Molly Kinder has written something about this messy middle scenario. And there's that possibility made me think about whether it might be better to have, at least as far as wealth distribution and redistribution goes, it might be better to have much faster AI takeoff. And I want to ask you whether the following possibility is at all likely, or there's any set of assumptions that this can make it so, which is that AI makes it possible to automate jobs such that many people are losing their jobs, but it doesn't create enough wealth while the process of automation is happening to pay off, basically, the people who are getting laid off.

20:17Phil Trammell:there's like a Pareto improvement. Everybody's getting better as a result of AI automation. And of course, there's a trivial sense in which that must be true, because whatever money you're saving, whatever money the company is saving by not paying the humans instead of just paying the AIs, those resources still exist in the economy and they can just be paid off to people. But there's going to be some allocative inefficiency if the government doesn't know exactly who got laid off because of AI. There's some political problem of if the meta worker gets laid off first and they were making 200K a year.

20:47Phil Trammell:Is there a politically sustainable situation where you give them a 200K check a year when there's many people who are working who are making much less?

20:58Phil Trammell:So do you at all find this scenario plausible where AI is actually automating a bunch of things, but there isn't enough wealth creation as there is automation? I think it's - Is that plausible? Possible. To me, it does seem like a pretty narrow window. My guess is that if we have the technology to automate, so many jobs that it becomes like a new kind of political problem, then the pie will also be growing really fast. Well, unless in all of those professions that it's automating, it's just a hair more productive. So like the cost of all the capital to replace all the software engineers is just, you know, a hair less than the cost of what we've made the software engineers.

21:33Phil Trammell:And why is it implausible that it just like a company can save money by laying off a bunch of software engineers? But and in the long run, there's a Jevin's Paradox thing. And, you You know, we can't anticipate in advance what we do with more software and surely there's going to be more uses. But in the short run, the fact is just that a lot of people are laid off and they still need to figure out how they can use a million X more JavaScript tokens.

21:55Alex Imas:I think the thing that like is in either like, you know, Phil and I have been like writing about these things and we have mathematical models in the back of these things. We don't have any political economy in any of our models. Andy Hall wrote a really nice blog post about the politics of AGI. And he made a really interesting observation. If there's a 2 % increase in unemployment, the political winds completely change. Unemployment has a huge effect on what happens politically. To Molly's excellent essay, by the way, I think in some ways one of the worst scenarios is a drip scenario because of the political economy piece.

22:31Alex Imas:Right. Because like, you know, people essentially what you what you might see is like people not really being unemployed in mass, but kind of like moving into sectors that pay them less money, kind of basically getting what happened with phone operators in in in the mid century of the of the between 1920 1940 phone operators were completely automated. Right. But it took 20 years, even though the technology existed. And therefore, there was this drip. It wasn't like this giant sector just disappeared. And when it ended up happening, there's a really nice QGE paper on this basically showing that they got reabsorbed into the economy, but at lower salaries and they were mostly underemployed.

23:13Alex Imas:And I think that's the scenario that Molly was writing about this kind of messy middle where things aren't a disaster because we saw with COVID, the fiscal response can move quickly if there's an emergency. And an emergency is a quick uptick in unemployment, which could even look like two or three percent. That's like a national that becomes a national emergency if it becomes fast.

23:34Phil Trammell:The concern is that suppose whatever you're saving on those white collar workers, if that's not growing the economy, but it's just creating some, you know, saved resources that can be allocated elsewhere. is that enough to do a broad-based redistribution scheme? Because then you have like the money you've saved off a couple of people. Yeah. And unless you can figure out exactly how to get to them specifically, you have the problem of can I do like, can I do a UBI off the money I saved?

24:04Alex Imas:Yeah, so you're basically saying like, look, the pie did not grow that much.

24:07Phil Trammell:Yeah.

24:07Alex Imas:You're just basically displacing a bunch of people, but that actually didn't grow the technological frontier of what the economy can produce.

24:15Phil Trammell:And so then there's a question of like, well, maybe every time, I don't know if this is the case, maybe every time this has happened in history, the technological frontier has expanded a bunch.

24:22Alex Imas:I think that's the case. I think simply in history, the technological frontier has expanded. So it's kind of, and I think Philip made the same point. Like it's hard to imagine that sort of scenario where you are getting like intelligence that's kind of just enough to replace the software engineer, but still costs a lot of money. Like it's just a hair less expensive than the software engineer. So you're not getting this abundance effect. Right. And so where is the redistribution going to happen because the pie didn't grow?

24:54Phil Trammell:Yeah, yeah. Okay, so this is very helpful. So there's many different things that would be true for this scenario to come to pass, each of which seem unlikely. One, it has to be the case that it is possible to automate entire white collar jobs, but only in a piecemeal way. That is to say that you can only automate software engineers, but that same program can't also automate an accountant and an analyst and whatever. Where I think at least my model of intelligence is such that both of like the breadth of tasks that it requires to do something like software engineering and what intelligence is, is such that, you know, if you can really just like lay off all the software engineers, you've got enough in the bucket there that you could like automate all kinds of white-collar work.

25:37Phil Trammell:So yeah, you're saving, there's huge amounts of potential savings that have happened as a result of these layoffs. And also that AI is going to be cheaper than human labor. And if both of those things are true, this messy middle scenario where we literally don't have the wealth to go around seems unlikely. And the question is, what is the best way to tax it and redistribute it?

25:56Alex Imas:Yeah, I have some thoughts. I think it's just really important to outline the costs and benefits. Like, it's also important to know that there. So first, there's differential complexity in implementing these things. Two, they differ in the timeline of like being actually helpful. So like something like universal basic capital, that's not like that's not going to generate returns for something that happens in six months. So you probably are going to end up with a layer of things. So like, for example, like a negative income tax, like you implement it and the day it turns into law that is already you already have this sort of insurance that like, you know, there's a floor for which, you know, everybody gets a certain amount of money.

26:39Alex Imas:And then, you know, if you earn more money, you get taxed more and things like that. And but, you know, there's positives and negatives to negative income tax. With UBI, for example, I worry a lot about like the political economy implications. Like, for example, like if people are just kind of dependent on a check, it really matters who's in power. Like right now we're endowed with labor that can turn into income when that is no longer the case. And we are now at the mercy of the elected official for like basic needs. Right. So that to me feels like a power sharing arrangement that's really dangerous.

27:21Alex Imas:But wouldn't that be true of any sort of government redistribution program? So something like university basic capital where you have like an ownership share and you have property rights for capital, then you just – You're a normal shareholder. You're just a normal person.

27:34Phil Trammell:But this goes back to the question of indexing because if indexing is hard, then universal basic capital is hard.

27:38Alex Imas:And that's the problem of university basic capital is targeting.

27:41Phil Trammell:Right.

27:41Alex Imas:Right. What do you target to put into people's portfolios?

27:44Phil Trammell:Like what if Anthropik goes to zero but some random robotics company takes all the surplus? Exactly.

27:47Alex Imas:Exactly. So that's the risk of university basic capital. with the negative income tax, you have the same sort of issues with UBI, where like, you know, somebody comes into power and says like, this is, we're not going to do that anymore and people can't work. And then, you know, you have the issue of the floor being open.

28:03Phil Trammell:One concern with the wealth tax is that, you know, there's no politically sustainable equilibrium at like 0.5 % wealth tax. And, you know, I mean, this happened with the income tax, of course, right? You'd start slow, it's like for war or something, and then it slowly and slowly escalates until the marginal tax rate in the U.S. is probably on the order of income tax rate is like 40 % or something. And in certain states, upwards of 50%. With a capital tax, is there a reason to worry? Would that distort investment? Because people would just be like, why would I invest in Anthropic or Intel? The government is going to take larger and larger shares of it and dilute my share.

28:40Phil Trammell:Well, hold on. So I think it's worth separating how the revenue is raised, like what's taxed. and then how it's distributed. It could be that the government hands out shares of Anthropic to everyone by a broad-based tax and then buying Anthropic. Yeah, okay. Which would probably be the right thing to do. I mean, hopefully, some, like, populist proposal doesn't interfere with that and, like, expropriate some, like, particular company that everyone happens to know about. Yeah. But how... So you're suggesting there could be a tax that is some sort of optimal tax. We're taxing externalities or we're taxing land or we're, I guess, We probably need to tax something other than just those two things.

29:19Phil Trammell:But that tax... Yeah. Or consumption. Okay. So a consumption tax, like a European value-added tax type thing, that allows the government to go buy a bunch of stocks, and then it just distributes those stocks to everybody.

29:34Alex Imas:That's David Otter's... Yeah.

29:36Phil Trammell:Yeah. I mean, that's not going to be that different from just redistributing the stocks, but it'll be a little different. Yeah.

29:41Alex Imas:That's what social... That was the proposal for social security, by the way. That was privatizing social security, right? So it's like you turn like this sort of weird, like not weird, but it's been working. It's worked so far. But, you know, there's questions for how long it's going to keep working. Like basically privatizing Social Security was giving everybody a basket of stocks. Right.

30:02Phil Trammell:All right. I'm curious to understand people talk about whether there's a white collar apocalypse already. Is there any evidence that suggests that there is mass automation or unemployment as a result of AI already?

30:16Alex Imas:I think there's a lot of people are looking at it. So this is an area where there's like a lot of eyes and a lot of data being produced. So the budget lab over at Yale is doing really good analysis on this. They just recently released a report. And I think like you really have to squint to see anything happening. Like basically, if you want to take kind of like an approach across the entire economy and looking at even looking at like software engineering, like the most exposed serve sectors, there's just like not really anything going on. There might be a little bit of a signal about like junior developers getting jobs less than before, but that's like a less than before rather than a level shift as in there's actually an increased demand for senior manager, for senior software engineers, if anything.

31:02Alex Imas:And so if you look at trend, it's kind of like for junior managers, it's a bit below trend.

31:08Phil Trammell:So as you're saying the growth is slower than before, but there is still growth even on entry-level software engineers.

31:14Alex Imas:Yeah, exactly.

31:14Phil Trammell:And what do you think is going on with the anecdotal evidence of graduating college students saying that they're finding it harder to find CS jobs or something?

31:22Alex Imas:I think that's anecdotal evidence.

31:24Phil Trammell:You think it's always been hard to get jobs for some people and now it's getting turned into an AI narrative. Same with the layoffs where it's probably just a normal layoff and they turned it into an AI layoff.

31:33Alex Imas:Yeah, I mean, you have to be careful with all of this. I think there are these there are these public coordination devices for, let's say we get into a narrative where if you're a firm and you're not laying people off, then you're seen as not adopting AI enough. So then you're going to just get a cascade effect. A firm's just needing to keep up with the Joneses in terms of starting to lay people off. And that's super worrying where actually the firm might be worse off after the layoffs than before the layoffs, but it's just doing the layoffs to have the perception that, look, we're not behind the times.

32:07Alex Imas:We're using AI. You probably heard these anecdotal stories of these token counters that you have to maximize tokens and things like that. So again, right now, we don't really have any evidence of a white collar bloodbath.

32:23Phil Trammell:And is that surprising at all? I feel given the fact all these things that AI can do is just like, this is a story as old as time. If you automate some complimentary task, the overall bucket of things that the human labor, which complements the automation, will increase in value.

32:39Alex Imas:So this is one of the statistics that's really important for that argument is elasticity of demand. So like you take the O-ring model of jobs. So like, again, jobs is a series of tasks. Let's say the AI automates like nine out of 10 tasks. One task is not automated. If that person can now kind of focus in on that task, the job will become more productive. If that translates into a price effect where the product is actually cheaper, if the demand responds enough where it's being bought more, it's being used more, the service is being used more, that could actually lead to more hiring. And a lot of people on the internet have been making that argument very generally saying, look, we're seeing, if anything in the data, we're seeing an uptick in software engineering.

33:25Alex Imas:Right. Yeah. Yeah. Which suggests that, at least for now, given the way the jobs work, it might be elastic.

33:31Phil Trammell:But I think this elasticity of demand argument is incredibly important, both for a lot of arguments that people make, or just a lot of labels that people use, without understanding what the underlying causation is. So people often talk about Jevons Paradox. Yeah. This is this idea that as something gets cheaper, you will want so much more of it that the total amount you spend on the thing increases. And so famously, this happened to coal in Britain 200 odd years ago. But really, this only happens if the demand for something is highly elastic. There's many things for which there is not super elastic demand.

34:09Phil Trammell:If oil, for example, gets super cheap, it's not like magically... Or insulin, right? Yeah, exactly. Magically, there's going to be so many more cars that now we're going to be using way more oil than before. At least not in the short run. Exactly. So long-run elasticity is higher than short-run elasticity. But even in the long run, so agriculture famously is the example where we can produce way more food if we dedicated the same portion of the economy that we dedicated to agriculture. We're already producing more food regardless, but we could produce even more food if the same portion of the economy that was producing food 100 years ago was currently producing food.

34:42Phil Trammell:But, you know, you eat enough and then you're done. And so the claim with software is that it is not some inherent property of markets that as it gets cheaper, you'll just keep wanting more of it. Absolutely not. The claim about software is this is a particular kind of good, whereas as it gets cheaper, we'll want more and more of it. And it's also highly relevant, and you wrote an essay about this. A lot of this podcast is me summarizing your essays back to you. that there's this very viral scenario planning about the future by Citrini where they're predicting as a result of automation, as a result of very powerful AI, there will be a recession because white collar workers will get automated.

35:20Phil Trammell:Their salaries, which we're paying for a bunch of things, will no longer be available. And so there'll be a slump. Do you want to recapitulate why this might be implausible?

35:31Alex Imas:Well, I mean, so part of it is plausible. Part of it's not plausible. So the part that's kind of like within the, this is something that we started the conversation with, is the idea that there could be unemployment, a lot of unemployment. If the speed of automation is quick and things like that, people could get laid off and they may not find work very quickly. So that part of the Centrini essay about the unemployment, we can quibble about that, but that's not the issue. The issue is that they talked about negative economic growth. Right. And so what I did in the piece that actually Phil and I had a back and forth on was to say, like, let's start with the proposition that there's negative economic growth.

36:09Alex Imas:What conditions do you need on the economy to get negative economic growth? And it turns out the conditions are pretty improbable. So one thing that you need is like for the holders of capital, like rich people, basically, like basically what you have in this in those sorts of scenarios, like you have a reallocation of wealth and income from like lower income people who are working or using their label towards capital owners. So what you need is that basically demand to be bounded, like a hard bound, not even like a soft sort of like diminishing sensitivity. You need for them to eventually say, I've had enough.

36:45Alex Imas:I don't want to spend any more money. And for that money to not enter as investment. Right. Which is and then you can get negative growth, which is like.

36:53Phil Trammell:And the crucial thing is, even if we don't want more shit, the world in which there's a singularity and we don't want to invest more money is crazy. Right. Where we're not like, let's build more data centers. Let's do more fabs. Even though we have AGI, we're not like investing in more data centers to run the AGI. And that's like driving more economic growth.

37:12Alex Imas:And so I sent the essay to Phil and Phil actually wrote back being like, this is pretty dumb. Like my essay saying, you're trying to say that there's going to be negative economic growth, but these are very implausible conditions. And I was like, actually, that's the point of the essay. These are very implausible economic conditions. So that's where I think like scenario planning really shines is you have the Centrini essay, which I think is like, I think it was great that it's written because I kind of started a conversation, but it's just like, it's so intuitive, this idea that like, look, if there's demand collapse, we can get the economy to shrink.

37:46Alex Imas:But it's actually, you could get that with a depression, right? Where in the depression, the technological frontier didn't expand. Here, the technological frontier is expanding. You actually have abundance and for abundance to generate negative economic growth. That's really hard to get.

38:03Phil Trammell:Right, exactly. Google recently announced Gemini Omni and its video editing capabilities are incredible. You can upload a video and then tell Omni to do things like change the background or adjust the lighting or add or remove elements, all while keeping everything else consistent. But Omni isn't just a video editor. I got a chance to sit down with the research and product team behind Omni and I learned that it's a preview of how future Frontier models will be trained. It can take in any kind of input, whether that's text or audio or video. And while it doesn't currently do so, architecturally, it's capable of just as seamlessly outputting images or text.

38:37Phil Trammell:So it's really a bet on the multimodal data transfer hypothesis. The model becomes better at predicting one data type by seeing the others. For example, Omni is really good at accurately rendering text on video, even though Google didn't specifically target that capability in this model. And Omni is the next step towards more accurate world models. Because in order to predict the next frame of a video, you have to have a deep understanding of physics and spatial dynamics. As Omni progresses, it'll be interesting to see whether it can close a sim-to-real gap. Because it's much harder to collect data in the real world than it is in simulation, robotics progress has lagged other applications of AI.

39:11Phil Trammell:But if you have really good video models that can simulate reality, maybe that stops being the case. In the meantime, if you want to try Omni, you can check it out in the Gemini app at gemini.google or use it in Google's AI creative studio, Flow, at flow.google. we're talking a second ago about why there isn't more automation as a result of llms and one plausible mechanism could be that as you're saying with the o-ring uh so o-ring theory refers to this fact that the challenger shuttle blew up because there's there's one component that malfunctioned and it destroyed the whole thing and maybe that's a more general model of how goods are produced in the economy that you got to make sure everything is reliable and works well and you can't automate an entire job to an AI right now, even though it might be able to perform it at some probability, you need extreme reliability in order for it to not destroy the finished good.

40:01Phil Trammell:I think this is, yeah, so this might explain why there's less automation now than there otherwise could be. But I think it works in the other direction once AIs get advanced enough that integrating humans into the production flow of future goods, even beyond the arguments about how humans will be more expensive or dumber or whatever. Even beyond that, just there will be whole production flows that are organized for AI labor, where they're talking in neural ease. They're thinking many thousands of times faster. So even if there's some comparative advantage where it makes sense to hire a human, there will be like transaction costs and worries of a reliability that will actually make it hard to integrate humans into future production flows.

40:41Phil Trammell:Yeah, that seems right to me. In particular, I just want to distinguish between the point that if you automate like nine tenths of a job, then people might kind of shift over to the last tenth, but like there might be 10 times more work demanded of them from the model of O-ring automation from like Gans and Goldfarb recently, which was that if you can only automate nine tenths of the job, but you can do it to a lower standard of quality than the human could do it, you might not want to automate even those nine tenths. And that's the thing that could totally port over to, like, symmetrically, it could be a reason why we don't use a human for one part of the job anymore, because a human just can't perform it to the level of quality that the AI can perform the other parts of the job or the level of speed or whatever.

41:30Phil Trammell:And they end up pulling down the quality or speed of the finished product. By the way, the model you're talking about seems extremely plausible to me of why more lawyers or accountants or whatever are not automated. Like there are cases or even software engineers where there's a pretty good probability that the thing worked as you expect. But the thing you're paying the lawyer for is like, no, really, my company is not going to go under because.

41:52Alex Imas:You're also paying for a lot of like regulation stuff. Right. So like with lawyers, particularly, you need some entity to back up the product. You need kind of like an ownership of the product. You need somebody to be able to fire or hire like licensing issues. There's a lot of like sort of like regulatory layers that are like also going to be keeping, even if there's no relational element, human in the loop that have nothing to do with like the ability of the human to actually perform the service.

42:23Phil Trammell:Yeah, I mean, you know, all of these frictions on the political type decisions that we are accustomed to only trusting humans, you know, only having humans for, like legislation and being a judge, being a jury, or all the licensing that keeps certain professions human, that all strikes me as transitional, right? I mean, what we expect to come from a human and how we organize our politics, that's changed so many times throughout history, right? From little hunter-gatherer bands to empires to whatnot. And yeah, once an AI-run political system is much more efficient than the alternatives, then those will probably tend to outcompete the others.

43:06Phil Trammell:And, you know, so speaking of which, we've been talking about what preferences humans currently have and what impact that has on what kinds of goods will be scarce in the future. But of course, we'll have different kinds of entities in the future. AIs, right? There was a time when there were no humans on Earth, but evolution selected for agents that have specific drives and preferences because those tend to survive the most. And those preferences now basically determine how a hundred trillion dollar world economy, what it produces. And so why not expect the same thing of AIs in the future? This is not even a world with catastrophic misalignment.

43:45Phil Trammell:That is to say, they just kill everybody. But there will be evolution of, even if not individual AIs, then firms which have AIs as part of them. And what will that evolution favor? where it will favor probably firms or agents that grow, right? There's like a selection argument that things which grow will be more prevalent. And maybe just based on that, you can make some predictions about what their preferences will be. But is the kind of entity which prefers to have human intrinsic goods going to be the kind of entity that accumulates resources the most? Probably not, right? Probably like saves more.

44:21Phil Trammell:Or it has unsatisfiable demand for things like whatever the relevant resource happens to be. Compute is an obvious one.

44:28Alex Imas:And can we use that to make some prediction about the non-human preferences that will be guiding the future? Yeah, so I think if there's like an AI that's like has its own welfare and it's fully autonomous and it's like making its own decisions that are welfare relevant. To be honest, I have absolutely no prior that they would at all prefer to deal with humans. There's no reason. But let me take the other side of that argument. Will humans' preferences to be interacting with one another and to trust and empathize and all of these sorts of things with other humans versus a simulated AI? I think it's a really important question whether those will change.

45:11Alex Imas:Right? So I've heard a lot of arguments saying like, look, right now we're just not used to the technology. And at some point, like what you're thinking of relational or something like that, people are just going to see like an AI therapist as a superior product and they're not going to need the sort of like empathy or whatever that the human is providing. I think this is actually a really complicated question. Here's one argument for why it's not going to go away and that that has to do with evolution. So let's say there's two types of people. One person doesn't really have this preference. They can just interact with other AI, whatever can simulate it better.

45:48Alex Imas:The other one has almost like a moral emotion, like from using Jonathan Haidt's framework, moral emotion against offloading those sorts of social interactions to an AI. Which of those two people are going to reproduce, find a mate, all of these sorts of things? I think the answer is kind of clear, right? It's the second one that has the preference for other people.

46:10Phil Trammell:It depends on how the reproduction is happening.

46:12Alex Imas:Fair. But if we're in, you know, the world where like reproduction is still happening the way that it's happening, I think, and this is a big question. I'm not even like, I'm not making a prediction again. I'm just saying like, if we're thinking, you know, you had David Reich on the show, like his point on the last podcast was that, you know, we're buzzing with natural selection.

46:32Phil Trammell:Right.

46:32Alex Imas:So even if like you get some sort of indifference now, you might get selection to point into like an even stronger preference for other humans.

46:40Phil Trammell:Here's one way to think about it. How is the wealth of the richest people in the world instantiated? Of course, they can, as you were having a call earlier and making the point that their consumption is more geared towards relational goods. Like Mark Zuckerberg is hiring MMA instructors and dancers for his wife's birthday and so forth. but most of his wealth is just stock and meta and he as a controlling shareholder could say hey meta just give me all this income or turn all this wealth into dividend income um and i will just spend that on consumption but instead he rather would have his wealth compound um and meta to build more data centers basically so you don't even have to change humans for this to be the case it is just the case that the humans which are wealthiest and are growing wealthier because their wealth is compounding.

47:27Phil Trammell:Just have this like almost Nick Landian preference for like accelerating capital. And that does seem to suggest that, yeah, is that an important determinant of what kinds of things are produced in the future? Yeah, I could kind of just say like, there's two ways you could get the two kinds of people, one of whom prefers a human therapist and one of whom is fine interacting with the AI. If they both satiate equally quickly in capital, right? But the one who likes the human therapist just also likes having some human intrinsic services, then the marginal value, like how the marginal value of capital in the future compares to the marginal value of capital today for each of them, if they start out equally rich, should be basically the same.

48:13I mean,

48:14Phil Trammell:there could be interactions and whatnot, but basically that should be the same. If what's driving the difference is that one person just doesn't satiate in capital because they're engaged by the prospect of, you know, exploring the universe and turning their head into a galaxy brain or whatever, and the other one satiates, then the person who doesn't satiate in capital is going to have, if they're being rational, they're going to have a higher savings rate. Okay. So in the long run, they're going to have most of the well and the overall capital share will basically be the capital share of that person spending, which is going to be well.

48:49Phil Trammell:It's important that this is, we're not talking about a hypothetical future. Yeah. Elon Musk is talking about mass drivers on the moon. Right. And he's like by far the wealthiest person in the world. I mean, obviously, currently his investments are going towards humans as well as machines. But I don't think he cares particularly that his future researchers and engineers are humans versus AI. And he manages to reproduce fast as well. Yes. So anyway, so I just think it's worth drawing that distinction. Yeah, there are currently some rich people that don't seem to satiate quickly in capital. And, uh, and so maybe in the long run, they'll save the most.

49:25Phil Trammell:And right. Yeah. Um, that does seem sort of right to me. Um, and I would just also say, even if they do reproduce more slowly, like biologically, that might just not matter that much in the long run, right? If, if, uh, they can live forever and, you know, the, the living forever is key. Yeah.

49:42Alex Imas:Right. So I think, I think again, like, and we're, we're, we're scenario building here. Right. So I think if you could live forever, a lot of stuff changes for my story as well. I think it's, to your point about rich people just consuming, not consuming a lot and investing, I think this will all depend on the returns to capital. So right now, the returns to data centers are super hot. But if we get into a situation where people are satiated with capital, then the returns to accumulating capital are going to be lower. And so then these rich people are going to be consuming more, right? So because the incentive to invest is smaller.

50:23Alex Imas:So basically, you kind of think about this in general equilibrium. The general equilibrium of this sort of process, like we have gotten tremendously more richer since, you know, 1820. We've gotten many more people are investing. But you're still getting a consumption response, which keeps, you know, people employed in labor share high. And that's because - Wait, hold on.

50:46Phil Trammell:Not necessarily. I think you're probably making the same point. But I mean, they could just, it could be that their investment has to be chai-traded through actual laborers who had to go like do things for their investment to work, which like would not, in the future, only the consumption is human mediated, right? Because the investment can just be done by the robots.

51:03Alex Imas:But if the returns are, if you, so we're in the scenario with like how you can keep high labor share, right? Let's take that scenario. In the scenario with high labor share, for whatever reason, the returns to capital are going to be lower.

51:16Phil Trammell:Yeah, that's right. And I mean, to the earlier thing where we're in the messy middle, we were saying, why, this is implausible. I feel like we can do a similar thing here. Where for returns to capital to be lower, the growth rate has to be lower, right? I mean, it certainly has to be lower than what we're expecting through the period of transformative AI. You know, if there's explosive growth. Yeah, yes and no. I mean, so the capital stock could grow quickly, but the price of capital goods relative to consumption goods could be falling faster than the capital stock could grow. Oh, interesting.

51:46Alex Imas:Yeah. It's the difference between like the potential frontier of technology and like what the realized prices of these things. Because you have relative prices. That's right. So you're saying I could be putting my money towards, you know, earning 30 % interest and investing in data centers or whatever.

52:04Phil Trammell:There'll be something in the future if the growth rate is high that earns high returns. or I could, as a result of all the technological breakthroughs or some cool product that I really want to buy right now. And both of those will be compelling options. Yeah, it doesn't have to be a new product. It could be a human intrinsic product. Right. Although if it's a human intrinsic product, we would want to have it much more in the future than we want it now because sort of the thing it compares against is... So we might want it the same as we want it now in the sense that like the marginal utility in a ballerina performance.

52:38Phil Trammell:It's exactly the same as now, right? But the margin utility in a robot might just be a lot lower than now, right? So in units of robots, we want it a lot more than we want it now, right? So would the interest rate be 30 %? It depends what you mean by the real interest rate, okay? It might be that every robot now can turn into 100 robots next year, right? So in units of robots, the interest rate is 10 ,000%. But if the price of robots is falling really fast... Prices adjust.

53:06Alex Imas:I mean, that's the whole, I think that's the whole point.

53:09Phil Trammell:Yeah, but here prices are adjusting this interesting way that too many macro models don't allow for, right? So what's happening is what would be called investment-specific technical change, where, yeah, the price of capital is like falling relative to the price of consumption instead of like doing the standard macro thing of saying there's just output. It's like chimera of a thing called output, which is one for one can be allocated to capital or consumption, right? That's not going to be true in this world. Yeah. Every unit of capital next year is giving up way less consumption than each unit of capital this year.

53:42Phil Trammell:Because like the just one robot now turns into many robots next year. But the number of ballroom is the same.

53:49Alex Imas:And again, we're going to go back to the increasing varieties thing. Where like if all of those extra robots next year are actually different varieties of robots and I'm not getting satiated on those robots, then it's a very different story.

54:04Phil Trammell:Yeah, right. Right. But now we're talking about the consumption world, whereas for the investment side of things, there could be just some greedy titan of industry who keeps wanting more and more robots. And that alone would be enough? It would. To increase the marginal value of robots and therefore decrease labor share? Yeah. Okay. But why are we not expecting greedy titans of industry to keep existing?

54:30Alex Imas:I mean, greedy titans of industry historically have built libraries. But that's because they die. They all die. Everybody dies. Well, we'll see. But I mean, conditional on people dying, I think, again, you had a guest on the show who said, to understand the future, you should think about the past. And I think like I you could have new types of titans being right being born who's where their entire reason for accumulating wealth is just to accumulate wealth. But a lot of the time, you know, at least historically, I'm just talking about historically, the wealth accumulation process is part of a large social sort of like social interaction amongst peers, amongst the community where you want to be admired in some way or something like that.

55:25Alex Imas:So people end up like the stylized fact of titans of his of industry is like you accumulate the capital and then you like buy a bunch of stuff.

55:33Phil Trammell:Yeah, I mean, I guess it's sort of a historical question, but it does seem to me in a lot of cases what is happening is that as near the end of their life, they either hand it off to their children who are worse stewards of capital than they are. And they don't even manage to grow their wealth at the rate the economy grows much less faster than the economy grows, which their parents were doing. And also they're like, well, I care less about my children having it than me sort of playing this game of accumulating wealth. and so I'm just going to give it to some trust. And if people are living longer or if they can figure out some way in which to align their trust to this wealth accumulation process, it just feels like the evolution here is so strong where you just need a couple of agents that think this way for this to be the dominant thing, determining the preferences of the whole economy because this part is growing much faster than the other parts of the economy.

56:22Alex Imas:I think you just, like, the part about satiation and diminishing marginal utilities, it keeps coming up, but I think it's really, really important. Like, you know, if a person has an intrinsic preference for accumulation, that's just like, that's what they want. I think your story is totally right. But that's just like not how usually preferences work. Like you have enough, whatever, hedonics in your life. And then like the social status, all of this sort of, you know, Rousseau wrote about this. St. Augustine wrote about this. This is like a kind of like a basic part of preferences. Now, too, you guys are arguing about something else.

57:00Alex Imas:where like you could have such high concentration that you could just have a couple of exceptions to the rule and that's going to be enough. And I have nothing to say about that.

57:09Phil Trammell:Yeah. Yeah. I mean, I think the claim's a little stronger, not just like you could have some exceptions, but that it seems that historically and today we see the exceptions and they just haven't really taken over the economy historically because there've been these dissipation shocks, as they're called. So they've like given it to their kids. They've squandered it or they put it in foundations, which um uh which spent it i mean it's not a really a shock but i mean it's right people went people might have liked to uh you know fill the universe with monuments to themselves and sort of whatever live forever very wealthy and it's like a weird preference but it's not a hypothetical preference i think that's that's the thing but who knows what's going on in their heads i think um uh even without though like the kind of intrinsic preference for accumulation there are some instrumental reasons why people some people might value accumulation which is also worth bringing up so um there's a desire for uh political or philosophical or religious influence right so people get to sort of an arms race over like what you know what society looks like and what people believe.

58:22Phil Trammell:And then similarly, but differently, because it's not an arms race, there's just a total utilitarian philanthropy, right? So when I think about why it might be good to have a lot of wealth in the future as a good classical utilitarian, to me, the values, or at least one way you could have a kind of almost unsatiating utility function in having wealth in the future is to create new happy beings, right? They just add to the total welfare of the world you know i mean this this idea goes at least as far back as like bostrom's astronomical waste point that we could like put dyson spheres around the stars and turn all the energy into really happy simulations and whatnot i think the particular greediness of this optimizer doesn't matter what they're greedy for i think you're forgetting about utilitarian philosophy or whatever like just a pure von neumann probe has i don't know what the is this an accurate way to it.

59:15Phil Trammell:They just have high marginal value for like the random solar system they'll occupy because that turns into like more solar systems, it turns into more solar systems. But like a von norman probe is a thing that can exist, right? And that's like a very greedy optimizer. Yeah. I mean, if we're talking about like whether they'll dominate the economy, maybe this is a technicality, but, you know, we only count final consumption goods and investment goods as GDP, right? If there's just this phenomenon. How does a von norman probe show up in GDP? Yeah, exactly. Right. So if it's like if we recognize it as a person that like owns itself and it's like sort of, you know, optimizing on the margin between like spending a bit more on a baby von Neumann probe that colonizes another star system or like a ballerina or something.

59:57Phil Trammell:And it's just like it doesn't value the ballerina very much. But it's yeah.

1:00:00Alex Imas:Yeah. When we're talking about like AI beings or like like it just it just completely depends on how we're doing the accounting there.

1:00:06Phil Trammell:Right. Yeah. But it's just like what does the world look like in a world where like von Neumann probes are possible? Is it possible labor share is high? Anyways. Yeah, I think it's possible the labor share is high the way we usually account it. One of the biggest problems in RL right now is credit assignment, because you have these extremely long rule outs, and you need to know why they succeeded or failed. One of Cursor's researchers, Sasha Rush, gave me a Blackboard lecture on how they use targeted RL with textual feedback to deal with this problem and train Composer 2.5. I filmed on my iPhone, so apologies for the camera work.

1:00:37Phil Trammell:So we've generated this output. It's just a sequence of tokens. We're going to send those sequence of tokens to this model that's going to read it. Yeah. And then it's going to isolate a specific, say, turn that it says is problematic. Yeah. Then we're just going to do text manipulation. We're just going to take that trajectory, and we're literally just going to, like, smash in some extra tokens. After Cursor injects these hint tokens, they run another forward pass. The trajectory itself doesn't change, but the hint causes the model to assign lower probability to the error tokens. Cursor then trains the original model to match those probabilities, basically teaching it to downweight these specific mistakes.

1:01:16Phil Trammell:There's a lot more nuance that we couldn't include in this mineral. If you want to watch the full thing, I posted it on my Twitter. And if you want to try out Composer 2.5, head to cursor.com slash Dwarkesh. Do economists have any advice for countries which are not in the AI production chain? If you're not either producing the AI models, you're not producing the hardware that goes into AI models, If you're not Korea making HBM or Taiwan making with the FAS or not the Netherlands with ASML, like what is India or Nigeria? What should they be doing right now? If you're talking to Modi right now, what do you say?

1:01:52Alex Imas:I think the biggest lack of resources that we have allocated in the economic profession is thinking about middle income developing countries in the age of AI. And I mean, this is my fault. You know, this is something I fault myself with as well. there's not enough people thinking about this question. Like there are scenarios where, you know, you get like AI technology, you know, being allocated and dissipating to Nigeria and developing countries and things like that. And like that leveling the playing field, like essentially like giving them a, like a level up as far as capabilities. But there's another world where like, because they don't have enough resources, they're not making it, they're not training the models.

1:02:32Alex Imas:They don't have the hardware where they just completely get left behind. And because of automation, we can produce commodities in developed countries now. Then we don't even have the consumer market. And then that world looks pretty bad.

1:02:49Phil Trammell:Yeah. This seems to me like an extension of the messy middle case, right? One of the ways in which the messy middle might only be bad in a narrow range of scenarios isn't just that it would be easy to redistribute because it probably would be bigger, But because the interest rate would be way higher and or sort of equivalently, the price of everything except the human intrinsic goods would be would be falling really rapidly. It's sort of two sides of the same point. A little bit of savings would turn into a lot of consumption next year. Right. So things have to go really wrong for us to like just get over the threshold of, you know, capital being productive enough to automate lots of work, but not be productive enough that that the interest rate is high and or the price of capital produced goods is falling a lot.

1:03:35Phil Trammell:OK, so even without redistribution, a little bit of savings will save a lot of people. Sorry, you're saying that the developing countries have some savings.

1:03:41Alex Imas:Yeah.

1:03:42Phil Trammell:In the developed world, that will be enough to produce a lot of surplus that they can. They will now be able to consume a lot. Right. Using their savings. So, but I mean, the messy middle could be like wider in this case. I mean, they're starting from such a lower level in terms of like how much they have it and how much it's like actually indexed to the global economy. Right, yeah. And I think it's important for them to get on it now. And I don't have strong feelings about whether it should take the form of like sovereign wealth funds that invest in the right supply chains or just, you know, subsidies to their own citizens to buy a little bit.

1:04:16Phil Trammell:This is actually, I think, a crucial point. we were talking earlier about why the rockefellers or whatever the world by their descendants don't control everything if our argument about the selection of these kind of greedy optimizers hold and one argument is just that it's like very hard to index the economy and maybe they would have just decided to have their heirs index the economy and have it grow at the rate of economic have their wealth grow the rate of economic growth and they would be you know trillionaires their heirs would be trillionaires by now but it just before index funds existed it's just very hard to just get a represent.

1:04:46Phil Trammell:It's just a very small fraction of the economy going back 100 years accounts for a majority of the value created now. And if you miss those particular things, you would have basically, your wealth would have just kind of stagnated. And maybe there was a brief golden window from the creation of index funds up until, I don't know, five years ago, where actually you could index the economy and you could have your wealth grow at the rate of the economy grows. But now that we're in this world with very concentrated returns, especially to private companies, which is capital that is, as we were making a point in our blog posts, the average person has disproportionately less access to, as opposed to, you know, most of their capital is like having a random house, at least in the U.S.

1:05:27Phil Trammell:Or a part of a house. Yeah, which is, as we were saying, is sort of unique, a capital that is uniquely ill-suited to be complementary to the production of AI or the serving of AI or to robots. Or the kinds of goods that the rich will bid up the prices of. Exactly, right? Because what is the value of a house currently? It is really the land is close to other humans and modular relational stuff. That is just not going to be the main factor of production. And this is why Georgian tax

1:05:59Alex Imas:would not raise enough money for the sort of programs that we were discussing. Right.

1:06:05Phil Trammell:Stepping back, the point I was trying to make is if it gets harder to index the economy now, and that is supposed to be the main way in which both one and normal people are supposed to modular some sort of universal income. In the developed world. In the developed world, are supposed to have some leverage on or have some purchase on the wealth from AI. And it's also the way that developing countries are supposed to have some purchase on the wealth gains from AI. But it's very hard. I don't know. Does Nigeria own a lot of SK Hynix and like anthropic? I'm guessing not, right? It's not enough for them to just own the S &P 500.

1:06:40Alex Imas:So actually this brings up a really important point. Like is AI going to be like electricity or social media? Right. If it's, so think about ComEd or ComEdison, whatever the electricity provider here is. It's a monopoly. It provides a resource that everybody uses. But do we think about electricity as like generating, creating concentration of power? and it's ComEd having this huge amount of political power, social power, or something like that? No, because with electricity, a lot of the downstream benefits actually came to the users of the electricity rather than the actual entity producing the electricity.

1:07:20Alex Imas:On the other hand, with social media, it was the opposite case, right? Social media, it was everywhere. Everybody uses social media, but the rents went to the platform.

1:07:29Phil Trammell:But that's a really interesting point. The more you think, I don't endorse this take yet. I'm going to talk out loud. The more you think AGI is going to be our economy is going to be run on AGI the way our economy currently runs on electricity. That is, there's a broad fundamental transformation of the entire economy. The more it looks like electricity and the more it's like every company in the S &P of the future. Exactly. If it's going to make it to the S &P 500, it is because it has leveraged AI.

1:07:58Alex Imas:And then you're indexed again.

1:07:59Phil Trammell:Yeah, exactly. But then again, I guess it is totally, if you just look at how concentrated the S &P is over time, you know, just like these big tech companies much more so. I guess this goes to a fundamental point that it's hard to reason about, about how much of the gains from AI these individual private companies will be able to control.

1:08:19Alex Imas:And I think like the open model thing is going to be a big point here, right? So if we're in a world where it's like the open models are six months behind the frontier, nine months, then we'll hit AGI, we'll hit whatever. And in six months, everybody has access to this resource.

1:08:39Phil Trammell:And this goes to show you that every question is connected to every other because then that question about whether there's runaway gains connects to questions about recursive improvement. And even if not recursive improvement, then continual learning or online learning, which lets a model learn on the job. So if it's deployed, it gets to learn more. And these are just sort of like technical questions or forecasting technical questions, which then impact, I guess, whether Uganda will have any purchase on the returns of AGI. but it sounds like your answer really the reason I'm emphasizing the question is I think both for the messy middle and for developing countries a recommendation that is often made naively is you got to do some kind of retraining you got to do some kind of like jobs program or you got to have them build data centers in our country and I think you guys are suggesting something closer to just buy the index of AGI that's like probably a much more cleaner and much more likely to succeed strategy.

1:09:34Alex Imas:It's really good. These are the two scenarios, right? So I think there is a world where it is concentrated, in which case it's going to be really hard to index AGI. There is another world where it's electricity, then basically every company has access to AGI. So you just buy the index. So Nigeria just needs to buy the index. And Nigeria has access to AGI. Yeah. Right. Like because of the open miles.

1:10:01Phil Trammell:Yeah. So just to get back to the question of like, about whether to go with retraining or just trying to index, I would prioritize trying to index, but just given how fast AI could, you know, hit the world. But I definitely wouldn't just rely on that because like it could, the sort of messy middle type cases or just the long timelines cases on which like you, we don't get anything like AGI all that soon. We'll still, you'll just be like leaving a lot of value on the table if you could have like retrained to be a bit better, you know, like educated to how, you know, how to use the latest wave of computing.

1:10:41Phil Trammell:And yeah, so I don't think there's that much of an either or there. Like, I mean, maybe the reason to be pessimistic about this is because one of the reasons the country is poor is that it's a bad education system. And so becoming the best in the world of retraining people at using AI, it doesn't seem like a particularly promising strategy for that country.

1:11:02Alex Imas:Although there are cases where in developing countries, you had this leapfrogging effect. With, for example, mobile banking or something like that, it's much more prevalent in Nigeria than it is in Germany or something like that. Everybody's doing mobile banking. They have it on their phones. They're constantly doing this sort of thing. So, I mean, I, again, I'm not putting probabilities on this, but like with a transformative technology like AI, you could get leapfrogging.

1:11:31Phil Trammell:Yeah.

1:11:32Alex Imas:Where, you know, you skip the step in the middle and you can get like really astronomical growth.

1:11:38Phil Trammell:Maybe. Just about the ease of indexing. Can I just quickly say, I think it's definitely something to worry about a bit and keep an eye on. but as discussed in our own essay and as other people pointed out, it's already not that hard to index. So there's been a bit of an increase in the privatization of returns but it's still like well under 20 % of the total market cap of non-tiny companies in the US is private. And everyone thinks about OpenAI and Anthropic and then if that's where all the wealth will accrue then yeah, like all these questions about whether open models will stay only a little bit behind.

1:12:19Phil Trammell:Those are important. But even they look like they're going public before too long, probably. And the frictions that have been keeping companies from going public might themselves be alleviated by AI a lot, right? Just all the disclosure requirements and whatnot. They want to get access to more potential investors too. and if I had to guess, I would guess that the kind of long kind of general trend of just like lowering those frictions and making it easier for more and more people to index more and more will continue despite the recent bump in the other direction. This actually makes me hope even more so than before that the labs do get commoditized or at the very least they go public as soon as possible but hopefully they just get totally commoditized because I think AI will be much more popular and more importantly, will be much more likely to lead to broad increases in prosperity if the gains are just not particularly, it is as hard to capture the gains of AI as it is to capture the gains of electrification.

1:13:22Alex Imas:Yeah, exactly. So I think like everybody, there's no anti-electricity people out there, right?

1:13:26Phil Trammell:I mean, electricity doesn't take your job, but it takes some people's jobs. Yeah, yeah.

1:13:32Alex Imas:And I think it's, you know, this is maybe tangential to the conversation. I think like there's like a really, narratives matter. And there's this like really negative narrative around AI right now. But that's because people are not putting out the positive narrative. And there's a reason. It's more difficult to imagine something that doesn't exist that's a good thing than losing something that exists. Right, yeah. Right. So it's very easy for somebody to go on a podcast and to say like these jobs that you like, they're going away than to somebody to spin up like a utopia, which doesn't exist. Right.

1:14:10Phil Trammell:I hope this isn't too out of left field, but I think I would be remiss if I didn't point out one big cost of having commoditized frontier AI models, which is the tech race dynamic. right that like for safety purposes you might want fewer frontier companies so that each one has a buffer in case they want to slow things down to make things safer and uh the way this relates to our point before about the kind of widespread access you know of the returns um is that i think there's a lot less of a trade-off there than some people imagine where you know some people think either uh frontier ai gets commoditized and we all enjoy the benefits but there might be some risk because like it's the market's really competitive and cutthroat or things are safer because there's a big gap between the leader and the laggard but that means that the leaders get fantastically wealthy no like you could just have a relatively big gap but it's a public company ownership and it's widely distributed yeah yeah yeah more recently i have been thinking that the risk of commodification, which is that it sort of diffuses the ability to use AI to harmful ends, is worth the benefit that I just feel, I worry that not only having these concentrated labs makes it so that the sort of surplus isn't as widely distributed through society, but also it creates a very tangible, clear political target for the government to, I mean, we saw this with the Defense Production Act threat against Anthropic.

1:15:47Phil Trammell:If there wasn't one lab that is, or a couple of labs that are clearly ahead of others, this kind of threat would be much harder to make. Thank you guys for doing this. Yeah, thank you. Thank you. I feel like there's a lot of unresolved questions, but it is helpful to know what the relevant, at least like what is the first branch along all these important dimensions. Great. Thank you. Okay. well