The Unexpected Economics of AGI - with Christian Catalini, Tech Founder and Co-Creator of Libra

24:07But you can think about modes. Do you believe that like in chess, the best way to beat the best computer is to be a human and a chess computer together? Do you think that will be the case for a while? I think it's inevitable. And the paper kind of in the economics of it hinted it. At some point, the only way out is through augmentation. We have to become the thing. We have to be similar enough to the thing that there's no difference between us. Also from a safety and security perspective. You know, when you think about alignment, it's very clear if you read some of the things that people are way smarter than me, on the alignment space are thinking about, it's not going to be like a one-shot thing.

24:46It's going to be like raising a child, especially in the first iterations, and maybe they'll raise each other. But if you believe in that model, we need to be able to understand their preferences. We need to be able to understand their talents, which I think it was last week, Entropic released some interesting model activation experiments where they're trying to reverse engineer what the model is actually thinking. But at some point, the models are going to be so smart that if we don't have a brain computer interface and we cannot process things at the same speed, there's no way we're going to be in line with it.

25:18And sure, you could hope that the model has the same preferences as humans and we did a wonderful job the first time, which I think is very unlikely, or you need to be the thing. What are the highlights? I haven't read the anthropic model activation paper. Are there any interesting highlights relevant to this discussion? I thought it was really fascinating on how they were kind of using a model to try to match, you know, what came out from the internal thinking process in a way that it's almost like a verifiable trace of what happened inside the black box. This goes back to another theme of the paper, which is like verification is going to be increasingly useful also as the technology stack.

25:55And that's where maybe crypto eventually will make a comeback. Like we'll need verifiable inference. We'll need all sort of like additional tooling to make sure that when these things are running, they're running with accountability and provenance in a way that, I mean, right now nobody cares, right? Because essentially it's a race. But as these systems start taking on sensitive things or even more important parts of the economy, I think we'll look for that. Now, I want to come back. You had mentioned, we just talked about moats at the company level, but going back to the individual level, I think some folks will probably get some of this verbiage.

26:26But when you say build your own harness around the part that's valuable, if somebody said, okay, like I'm listening to Christian talking to these guys, I'm ending the podcast right now Is there a tool stack? How do you even think about building your own harness? Yeah, and look, you can get very sophisticated. So depending on your expertise, you can enter this all the way from fine-tuning your own model. But I would say the simplest entry point, and maybe it's why people are so excited about things like OpenClaw and Hermes, that's essentially the bare-bone version of your general computing platform, which is saying, look, there's these models.

27:04You can run them locally. You can connect them to the cloud. but now you're starting to train an agent or a group of agents to your preferences, your behavior, your intuition of what's right and what's wrong. Here's a really simple one. I think we've all experienced how LinkedIn is very problematic when it comes to spam, right? 99 % of messages spam every now and then there's some legitimate ones that want to get in touch for a good reason. A lot of that is a judgment call still today. And I'm sure LinkedIn could build a better tool, but you could imagine training one of these agents to just do that.

27:37Right. Based on who I am, the kind of business I'm looking for, the kind of conversation that might be valuable for me versus not, help me screen my opportunity. That's a really simple application for that, where, again, you're very fine based on your own weight. Is this message coming in something you should be paying attention to? And do I really need to read the 99 percent that's going to be told Jack? But is that an MD file? Is that a skill file? Is that a GPT? What when someone is is training that harness, where are they doing it and how are they doing it? Is it in a conversation with an AI that's a harness generator, right?

28:09What's that look like? Yeah, I think, look, the UI UX of the conversation with an agent is being kind of killer. What we haven't figured out is probably the proper memory piece, which is like conversations tend to be erratic. You jump topics. And I think someone eventually is going to crack that problem of like making that conversation as useful as possible. So some stuff goes into one term memory automatically. Other, it's like the dreaming function that, again, was released recently and many have been trying. I think that's probably the missing piece from a UX perspective, but eventually these models will be smart enough to just work with you alongside and learn what matters to you, what doesn't matter.

28:46Then again, you can get a lot more sophisticated in the skill. A skill is already an upgrade from, I mean, I would say most people probably are just having a long conversation with their LLM or multiple chats. And so the memory that comes out with, you know, Cloud or ChatGPT is already extremely useful and creates stickiness on like, oh, it writes the way I look. The next level is probably you start codifying some of these in skills and then you can kind of move up the complexity stack. I think the good news is everything is moving so fast that the tooling will get much easier for everybody. But the future is something where I do think you have this set of agents that work for you, that really understand how you would behave in certain conditions.

29:24And we'll get 95 % of those conditions right without even bothering you. And then they'll ping you for the 5%. I started a fun experiment where I take my agents and I created a network for them. This is something that we've done in the business I've been involved in, but then I've done it on a personal level now where I now have them do daily standups. And one of the things I'm trying to do is to tease out that thing. I'm obsessing about the persona MD file. Like basically, how do I codify myself and my verification system so that I can scale myself even more by basically having the agents do more in the mirror how I would do it.

30:03And so I find it to be such a fascinating new frontier. Like how do you actually go around and do that? But have you found, I was like building skills, like do you ask your agents about those things and do you codify them or do you create a taxonomy for it? Or like what's the best way to kind of think about creating this? I tried all sort of wrong ways to do it, you know, from having a transverse knowledge base and kind of more of a graph structure like the Karpati ideas on Obsidian. Like I think there's many ways to try that. But if anything, my main lesson doing that is that I do think people may finally care about privacy this time around.

30:38And crypto has been betting on this angle for a long time. But it seems like, imagine you succeed with your project, right? And this is like the perfect, distilled version of yourself. Would you want that to be, you know, fully available to a third party? And local models seem to be getting more capable. So maybe this time is going to be different. and people, because it's so personal, because it's so critical to their lifeline in business, they would want this to be closer to their home. Wouldn't Jaguarmin be like previously if Facebook took and just kind of did the Henrik skill, then they probably would have like a pretty good guess on what I might like and what I might dislike.

31:17I think now you're going even more personal and more nuanced. I think the chat conversations are probably more and more detailed than anything that we've even fed Google searches or social media networks. It's at a level of, I don't know, it feels closer. Then again, you're absolutely right. Maybe consumers won't care. And, you know, this will give you more of a moat to whoever can distill the different. For those of you that have watched Westward, there's that scene where, you know, she's going through the library and every person is a book. So that's probably, you know, your skill MD for every persona.

31:51I think at the very least, just to put a funny point on it, at the very least, making an attempt is worth doing. And there's probably a lot of folks who go, hey, whatever my LLM, whatever its memory of me is, is sufficient to my purpose. But even just asking a model, write my persona file based on, and give it to me as a downloadable Markdown file, right? And then try using that with another model, right? Like if you're typically working ChatGPT, give that file to Gemini and then test and see where does it fail And then having, I think the feedback loop of giving feedback on how the model is interacting with you is really, and then editing the file, right?

32:35I think if kind of a normal, you know, layman gets in the habit of, who am I, feed that to the model, and then make a personal commitment to updating the underlying file based on the quality of the interaction, that would probably up-level their game, you know, by an order of magnitude if they just kind of thought like that. That's verification infrastructure. I think that's the best definition that we've given so far, right? Which is essentially, if you iterate with any one of these models or multiple models, and you keep giving them nudges about what you really want, you're showing your verification stack.

33:12Yeah. It's very, very nice to talk to somebody who has a positive outlook on these things. And I would say that one of the worries that we could have is we would read documents like AI 2027, and then we'll see how well we're tracking against it. And a lot of this stuff, obviously, in many ways, because that's why how we are wired as humans kind of goes to these dark places, right? But it seems that there is not a lot of people who are brainstorming and originating on what is the good thing that could happen. It's easier to be a doomer, right? It's like, it's so many ways it can go wrong. Give us a little bit on what can go right.

33:52And you can't do the, we'll all just sit there and like chill and have AI do everything for us. That's not going to work. Okay. So that's also not going to work for a very simple reason, which is people need meaning, right? And so the age of abundance where we just get handoffs from some sort of UBI program, I just think would make people extremely unhappy. So let me take the reason why after writing the entire paper, I was still optimistic. And by the way, there was this funny moment during the write-up of the paper where we were bouncing with all the different LLMs, right? And there was a moment where Gemini was really good.

34:26I don't know what happened after it, but it seems like they did release and then they always did great. but did think was going through the paper and noticed that we had left a funny footnote. And so I read the footnote. The footnote was written for an LLM. And of course, you know, made a comment about it and then concluded, okay, what should we do next? And the footnote was all about don't turn us into paperclips. I'm like, don't turn us into paperclips. And the next response, which maybe when we release the podcast, I'll share on X, it was the moment where I was like, it was my little alpha go moment where I say, oh, wow.

35:00not only understood everything we had done until that point, but made essentially a funny joke about the entire economic model and tried to reassure me that, you know, because of that, we're not going to turn into paperclips. It was like this encounter with this AI and intelligence is really special. But all of this to say that I do think that, look, yes, are there ways this can go wrong? Yes. And I think actually, let's put this aside. The ways that this can go wrong are probably, you know, very silly. Not paperclip-like, but it could be just random side effects of like stroing unverified agentic output into society, right?

35:37And suddenly waking up to some systematic failure that's a bit like a Chernobyl. So if you park that, of course, it's a risk. The only way through, right? So first of all, there's no way putting the cat back into the box. The Pandora's box between SOTA models and even the open source stuff being like a few months behind. We just have to accept the new reality. So if you accept that, then what's the case for optimism? I think, first of all, think about talent. So if you're a young person right now, you're probably slightly depressed because a lot of the entry-level jobs are getting more scarce.

36:09The model happens to do typically a pretty good job at most entry-level jobs that a human can do, whether it's in marketing, legal, engineering. The IC4 is probably the most at risk of their entire enterprise ladder. But the good news is that if you look back, when we were younger in that stage, you can build like so many more things. You can learn about what you like. You can tinker with hardware, with software, with systems. You can launch a business. Like all these things that used to be extremely complex are at your fingertips. Back to learning. Anything you may want to learn also available to you.

36:46You can essentially have someone teach you all the steps customized for you into any domain. And so you have no more excuses. It's all about what you actually want to do. You can discover your type much faster because rather than pursuing some hypothetical career or like, okay, I do good college, then I get a good internship, and then I get a good job in a big firm, that's gone. We can probably all agree that that's going or most of it, it's going. So yeah, it's good news, bad news. But I think the good news is going to outweigh the bad news. We're going to empower like so many individuals to do a lot more, to be a lot more creative, to be able to even switch sectors, right?

37:23So if you think about careers today, they're very static. How many people, you know, I started in academia, then I went into big tech and then into a startup. I can tell you each phase of my life taught me very different things and things I like and don't like about each one of them. I think people will do a lot more of that. Now, you need to be able to cope with more uncertainty. Things are going to be more fluid, especially for a little while. I think it's going to be a lot more murky. But the upside is much higher. I think we'll be able to do a lot more with less. We're going to probably launch many, many startups and teams that will build wonderful things.

37:58Incumbents that haven't been challenged for decades can be challenged. I'll give you a very simple example from payments, right? Defeating the card network has been historically practically impossible. People love to tap to pay. It's such a native behavior, so frictionless. During COVID, I think in New York, it was like 70 % of transactions or so were tapped to pay. And after that, of course, it only increased. How do you displace the card networks? Well, as it turns out, if these agents really take over, there might be something that's even more convenient to have to pay. It's like not even think about paying.

38:28The agent will take care of it. Maybe it'll send you a verification if it's above a certain amount or anomalous. But you just go through your life and you pay for things in a seamless way. I think that technology is there today. UIUX probably needs to be figured out. But all these things that used to be entrenched incumbents can also be challenged. So look, the transition is going to be painful. Again, I've already said this. I do think some of the job estimates are probably optimistic. What people seem to say, okay, there's not going to be any impact because society is going to move so slowly.

38:59If tech is a cannery in the coal mine, people are restructuring these firms. Sure, they have bloat, but they also realize they can do a lot more with less. And there's going to be a lot less jobs of the typical type. But I do think on the other side, we're going to be much happier, more fulfilled, more creative. So yeah, I want to be an optimist on this. That's such a good point to end, I think. This is incredible. Thanks for coming and sharing your research with us. Super excited. Really, really enjoyed. Anything you felt we left out that we should make sure we get on tape? No, I think this was really fun.

39:29You guys are very dynamic. I loved it. Welcome to The Debrief. What you have not heard is for the last 15 minutes, Henrik and I have been talking about things that we thought were unrelated, but actually we constantly realize our audience would probably like to hear that. Is there anything, Henrik? Well, maybe we should go back to the Christian conversation, then we can wrap with anything that stood out to us from our conversation. Okay, let's do the Christian one. I'll do a few. I think, obviously, verification is a powerful mental model for thinking about what can you do that is still relevant in the age of AI.

40:08And the second thing you had on a personal level was how do I scale my value? And so what can I verify and how do I scale basically that? And then thinking about if I am the director of a project, how would I think about that? Like, how would I become better at that? And I think that is actually pretty topical for those of us who have seven or eight agents. We are now trying to get them to work as much as possible when we're not around. so we need to give them understanding about how we would operate so then they can do more themselves and then obviously when we come back to them we want to kind of like answer as in a we have to verify or have to provide them feedback as high quality as we can so they can get off and do their work again so i think that was very interesting i'd love of course having a little bit of the positive view on what can happen the one that i had never thought about which i think is interesting is the idea of switching careers and that never before could you really switch career because it took so long time to retool yourself.

41:16And it was often so expensive. You had to go back to college. You had to pay a lot of money. It took five years. You didn't have a way of making money out of that. I think it's kind of interesting, this concept of, well, what if you can at all time take whatever career you wanted to kind of be on? And then suddenly, you could probably get a much more meaningful life because that you could dedicate your time to what you wanted to learn more about. And so thinking of that as a possible positive outcome of this abundance kind of way of thinking, I thought was kind of interesting. I never thought about that before.

41:56Yeah. You know, and if retooling is actually easy to me, or, or, or easier than it's ever before, career changing. I think the whole, the hand-wringing right now around entry-level jobs is maybe a misunderstanding because what is retooling if not tooling? You know, what do young people need to do? They need to get tooled up. And I don't know if there's a, if it's about maturity or wisdom and an experienced person can now change careers because they've already attained maturity and wisdom required to then retool, then that seems unattainable, perhaps, to a young person who's lacking maturity, lacking experience or wisdom or whatever.

42:34However, if it's really just about learning something, I mean, to Christian's point, you now have a personalized tutor. As he said, you have no excuses, right? And maybe an experienced person is they don't think from as, I can't even actually say they can't think with as many excuses because I think actually it's probably more inertia for someone who's experienced, who's built up a lot of knowledge in a particular area. They don't want to make a change, right? as much as probably a young person is motivated to jumpstart a career and things like that. So I agree. That's a super interesting opportunity space.

43:07I think it, again, just speaks to the value of initiative and being able to not wait for instruction, not wait for an assignment, but commission yourself, as I like to say. If you're the kind of person who can commission yourself, there's never been a better time to be alive, right? is now you actually have all the resources available and all sorts of capabilities that were not available to any other generation. The only challenge is whether there's that spark of initiative. I did a workshop the other day with a group of consultants. And one of the things, there are about a hundred of them and they're incredible smart and kind and kind of very open-minded.

43:43So it was very cool. And one of the things we came up with was to say, hey, why don't we all just write a piece of software that write our proposals? and so we had 100 people basically make a proposal writing a piece of software. And of course, it only took like an hour, an hour and a half and the outcome was incredible. And at the same time, I was talking to one of the people there and she was talking about how she the other day, I asked like, what's one of the most valuable things you've done with a client recently? And she's like, you know, like there was a client who called me up, really needed like an hour or two of my time and I'd go out there for a cup of coffee and it's not something I would ever charge that person full but it was just like really interesting to understand their business.

44:24Of course, you know, over time they'll be business out. I know it sounds so obvious, but writing proposals take a lot of time. And normally when you, when I previously have been speaking with that team, when they hear it, what should we do with AI? Their inclination is that, oh, you know, it can't do what we do for our clients. They go to that AI should have like the do what they do. But then when we say, well, it goes to do rather than what can it do? Yeah. And also, where do you act as a robot right now where you don't like to act like a robot, which is proposal writing? So let's do this. But the other thing which is what I thought was very interesting was that it's suddenly daunting on me and everybody else that we have thought about software as you build one software to many because it was expensive to build software.

45:09And as I was seeing these many, many different versions of this software, it obviously dawned on me that, hey, this might not be one piece of software we make for the team. This just might be a hundred pieces of software we make for a hundred people. because they might have different ways that they prefer to write their proposals. And so this idea that software could be the inner one, that it could be a completely personalized one, was a nice articulation of this abundance story that we have, where it isn't just about software that has to be like a generic thing that has to have usability because everybody has to use the same thing.

45:42This could just be the software that is wrapped around that human so that it makes them more kind of efficient so that they can spend time on the thing that really matters, which AI cannot, which is to drive over to a client, have coffee with them. Right. Yeah, you see that in all sorts of different arenas, right? You think about a coach. Coach spends a lot of time on scouting report, which is actually quite a robotic thing to do. The robot cannot spend time on the court with the players, you know, hands on the players. Right now, if you will, I like that as a search parameter. What's the thing you do that feels robotic?

46:12Stop doing that. You know, almost. That's a great thing to automate. me. One of the things that I was thinking about for our audience of, you know, Christian's comments around build your own harness for, you know, your own verification infrastructure. I think that's actually, it's very insightful. And I think it's very intimidating for somebody who's maybe early in the learning journey. What does it look like to build verification infrastructure? I got inspired by a couple of different LinkedIn posts around, you know, there's all these kind of posts around, you know, the top 1 % of people don't prompt AI, they do this instead, or, you know, write these files and never do this again.

46:49I, and I got inspired. I actually start working with my agency. I know I've done something like this. What have I done that would be authentic to me that I could give to people? I'm working on a video right now, actually. I've got an amazing YouTube team that's helping me make this video. But I think the, the thought that we ended on with Christian of if you're at the starting line or you're early in your learning journey to say, who am I? I want to write down whatever I think it is. I mean, you've spoken a lot about the persona or soul.md or whatever, right? Write it down and then have the discipline to revise it based on how it impacts your interaction.

47:27So every time you interact with a model with that source file as a reference point, then it does require kind of metacognition. You actually have to think about the process? What was particularly, you know, rewarding, gratifying, enjoyable? What was frustrating, dissatisfying, et cetera? And then how do I edit the underlying input that influences the next interaction? But getting in that habit loop of making a guess, making a hypothesis as to a way to describe myself to a model that will help the model work well with me, being, you know, well being defined as deliver an output that I value. And then taking the human agency to actually update the underlying source code, so to speak.

48:13I think that's a very, it's a simple thing to do, but very few people are doing it. They're going, you know, they're working with an LLM and they aren't being more thoughtful about the information they're giving to the model and then reflective about how is the information they're giving to the model impacting the model's ability to serve them. how would you pose the initial question to a model that people will just put up their chat to be a cloud or whatever agent right now? Would you simply just ask, what are some of the characteristics on which you think I make decisions? Or when do I often feel that the answer you provide is not aligned with how I would pick?

48:50Or what's the, where do you think is the best starting prompt? Well, I mean, maybe it's easier actually to emulate. Maybe if we want, I can even, I can share my file structure because I think probably what somebody wants to do is say, hey, I want to make like, look at Jeremy's file. Like I could give my file as an example. Look at Jeremy's file. I want to make something like this for myself. What do you need to know about me to rewrite this file for me? Right. That's probably something I would suggest. It's very simple, right? But then it's, you're basically giving the model permission to interview you with a template as an example kind of output for the purpose of iterating the template so that it works for you.

49:29And then there are, there's kind of layers to that, right? Why don't we do this? Why don't we, if people make it to this part of the conversation and email us, we will sell both our files to them, like the files that we use ourselves. How's that for a little bit of a kicker? Ooh, kicker, hashtag kicker. I think that's a pretty good way to end. If anybody gets here and wants our soul files, we will send them to you. Awesome. Anything else we should add, Professor? You know, actually, I have one thing. We talked a little bit about different ways that you can kind of go through the paths of either optimizing your company or yourself.

50:09And I think you mentioned this interesting learning that you had the other day, which was kind of like using the optimized, accelerate, transform model. On the accelerate, do you mind just sharing and getting on tape what you did the other day of what you started to measure? Sure. Yeah. Well, I was telling Henrik in the space between our conversations with Christian and this recording of our reflection that I have found a lot of value from Section had an AI MBA years ago that I, that was how I met Greg was I went to that AI MBA years back. It's where I met Eric Perez. We were both in that program together, who he's been on the show.

50:49And I think he's coming back on soon for round two. But in that class, there's a framework they describe OAT, O-A-T, Optimize, Accelerate, Transform. And there are different things you can do to the business with AI, right? You can optimize, you can kind of do things faster, cheaper, et cetera. That's kind of optimizing existing workflows. Accelerating is kind of driving the business in a particular direction, which we'll come back to in a second. And then T is transform, which is kind of re-imagining the business. On the Accelerate, one of the recommendations for identifying opportunities is to specify what are metrics you're trying to drive.

51:24And then you can ask yourself the question, how can AI help me drive this metric? And what I was telling Henrik is, as I have reflected on, because I continue to reflect on my own life, practice, business, et cetera, I have started to realize there are things that I don't measure, but I probably should. And so one example for me in my life is I do a lot of keynotes and then I do other things like courses and advising and things like that. And I realized I don't really measure the conversion from a keynote to a deeper engagement, whether it's a class or an advisory relationship or something like that.

51:57So I just said, I just asked myself the question, how could AI help me drive conversion from, you know, a keynote event to a deeper engagement? That was a question I'd never even asked myself before, but actually there's a lot of great answers. I mean, I can share stuff like that too. But I think that, I think Henrik, your question is, what is that kind of thought process that someone could undertake? And the simple thought process is, what should we be measuring? Either what are we measuring? What's a key, you know, performance metric that we do measure and how could AI drive that? And, or perhaps more interestingly, what's something that we've never thought about measuring before, that if we did think about measuring, AI could actually help us not only measure it, but drive that metric.

52:41I guess it could also be like, what What is a new metric that we should change to that now that we have AI? I'll give you an example. At Barbox, for example, we've always been very proud on the amount of interactions we have with our customers. And other organizations have always been like, well, you should try to reduce that because obviously it is costly. But with AI, I think we now have the ability to say, no, we could actually try to see we can get more of those. So like, how do we not get people off the customer support line, but how do we get them on it? Keep them on. Yeah, yeah, yeah. That's cool.

53:15It's like the old Zappos thing, right? Didn't they have an award for the longest customer service call? I think one time at Zappos, someone was on the call for like seven hours with a customer. That's awesome. On that note, I think it's time to say goodbye. And so with that, bye-bye. Bye-bye.