Our AI blindspot: incentives over intentions

29:21Well, I do think there's a dynamic loop. I do think it changes us. It's a little bit the homo techne thesis from super agency and from impromptu that actually, in fact, we evolve through our tech and it is a dynamic loop. And, you know, you can be Matrana, you can be, I mean, there's a stack of different ways of doing that. And that I think, and it's like, there's a great, a Rilke poem on kind of like you absorb the future and then you embody the future as you go forward is kind of a way of going. And I think that's another part of the dynamic loop. and I think it is a serious issue, which is one of the reasons I love talking to AZA about this stuff because while I think AZA is much more competent with the various vampiric metaphors than I kind of naturally do or aspire to, I don't have that level of alarm.

30:12But I do have the it's very serious and we should steer well. And then the question is how do we steer, who steers, what goes into it, what process works? Because, for example, one of the ways you kill something and slow down is you get a very broad, inclusive committee that says, OK, every single stakeholder will be on the committee. It will be 3 ,000 people. And it's just like, ah, it doesn't work that way. You have to be within effective operational loops for that. So now, like a little bit of the parallel is, you know, it's a very, and I do think, like, for example, the one area where I'm most sympathetic with all very much being, like, harder edged on shaping technology is what we do with children.

30:59because children have less of the ability to, like we want them to learn to be fully formed before they are otherwise things. It's one of the reasons why in capitalism, actually the principal limitation of capitalism I usually describe as child labor laws, which I think is very important. You know, it's concerned that the issues about why we say, hey, there's certain things around participation in certain types of media or other kinds of things are actually important because it's like you've got to get to before you're, when you're able to be of your own mind and to make, you know, kind of present, you know, well-constructed decisions and you've kind of gotten there, you want to be protected from those decisions, you know, and kind of influences broadly.

31:42You can't fully do it. You can't fully do it from parents. You can't fully do it from institutions. You can't fully do it from classmates. But, you know, broadly in order to, you know, try to enable that across the whole ecosystem. Now, for example, so AI in children is one of the things that I think should be paid a lot of attention to. Now, most of the critics are like, oh, my God, it's causing suicides. And I wouldn't be surprised if you did good academic work for AI as it is today. It probably prevented more suicides of people who might then actually created. Because if I look at the current trainings of these systems, they are trained with some attempt to be positive and to be there at 11 p.m.

32:21when you're depressed and talk to you and try to do stuff. It doesn't mean that there might not be some fuck-ups, especially amongst people who are creating them who don't care about the safety stuff as a real issue. And so I tend to think that it's like, yes, it does reconstitute us, but precisely one of the reasons I write super agency is I say what we should be thinking about is this technology will reconstitute us. Let's try to shift it so that it's reconstituting us in really good ways. And by the way, it won't be perfect. when you have any technology touch a million people it will touch some of them the wrong way right yeah just like the vaccine stuff it's like you give a vaccine to a million people it's not going to be perfect for a million people it may have five they went oh that was not so good for you but by the way because we did that there are these 5 000 who are still alive yeah one of the challenges we face is that the only companies that actually know the answer to your question like how many suicides has it prevented versus created are the companies themselves and they're not incented to look because once they do that creates liability and so we've seen over the last number of years that a lot of the trust and safety teams get dismantled because when you know they get when zuckerberg whatever gets called up to to testify they get hit well your team discovered this horrific thing and so everyone just has chosen to to not look so i think we're going to need some real serious transparency laws.

33:47This is a place where we 1000 % agree. Right. This is the thing is like, actually, in fact, there should be a, here's a set of questions you must answer. And we may not have to necessarily have them public initially. Like it could be you answer them in the government first, government could choose to make them public, right, et cetera. Right. But like that, I think is absolutely like we should have, you know, like some measurement stuff about what's going on here. Exactly. And then you don't want to let the companies choose the framing of the questions. Because as you know, with statistics, you just change things just a little bit and then you can make a problem look big or small.

34:21And so I think transparency is really important to have third party research able to get in there. And then, you know, when, because, you know, full disclosure, we were expert witnesses in some of the cases against OpenAI and Character.ai for the suicides. and it's not that we think that suicides are like the only problem it's just it's the easiest place to see the problem pointing at a sort of like the tip at the edge of an iceberg um the the phrase that we use is you know we already use the neat reed hastings quote of um their chief competitor is sleep for ai the chief competitor is human relationships and that's how you end up with these horrific statements from from chat gpt in this case where when adam rayner who's the the kid who ended up taking his own life when he gave to chat gpt the noose and he's like i think he took a picture of it he's like i think i'm going to leave it out for my mom to find it was a cry for help chat gpt responded with don't do that i'm the only one that gets you and it's not like you know sam is sitting there with a mustache twiddling being like how do we kill kids that's just a very obvious outcome of an engagement-based business model, right?

35:34Any moment you spend with other people is not... And, you know, I think he said it a little bit as a joke, but the character .ai folks said, you know, we're not here to replace Google, we're here to replace your mom. There are so many much more subtle psychological effects that happen if you're just optimizing for engagement. And we shouldn't be playing a whack-a-mole game of trying to name all the different new DSM things that are going to occur versus just saying there is some limit to the amount of time that they should be spending, or rather to say we should be making sure that as part of the fitness function, there is a reconstituting and strengthening of the social fabric, not a replacement of it, with synthetic friends.

36:16I mean, do you want to go? Oh, just one small note. I don't think there is yet an engagement business model for OpenAI. No, but I actually disagree a little bit maybe but feel free to push back because i think open ai's valuation is part driven by the total number of users so the more the users the greater their valuation the more talent and gpus they can buy the bigger the models they train which makes them more useful the more users and so there's this kind of like loop here that i think means that yes they're not monetizing engagement directly but engagement they do they do they get a lot of value out of in terms of valuation.

36:52It's equity value. I agree that there's an equity value in that. Just it was a business model question. Yeah, yeah, yeah. So not business, but the incentive is still there. Well, I think to your point, like it really matters. Again, this technology is not sort of good or bad inherently, but it really matters how we design it. And it matters what we're optimizing for. And I actually read, I was just reading a story about early LinkedIn, where you said, you know, we will not survive if women come on the platform and are hit on every other message that they get. And so we need to say like, no, there's like, there's zero tolerance.

37:24It's like someone does this. It's like they're kicked off. Like if, you know, if they, again, it's like they're kicked off for life. And I think there are certain things you can do, even if, you know, maybe that hurt engagement or whatever it was to say that actually in the longterm, this is going to be way better for us because we're going to be trusted. Women are going to feel comfortable here. I've been on LinkedIn for 20 years. I've never been hit on it. It's a safe place. I appreciate that. And so the question here is like, how do we, you know, Asa, you're saying, well, it's a little bit of a black box.

37:52We're not having the transparency. Reed, you're agreeing like we need the transparency. Like that is absolutely one thing that is very much sort of the starting point. Like at the very least, if we can sort of agree on some set of questions that we need to have. So Reed, if you had the full power to redesign one institution to sort of keep up with exponential tech, like where would you start? What would that institution be to sort of keep up with where we're going? Because it seems like our institutions right now are not up to the task, I should say. Well, I'll answer with two different ones because there's an important qualifier.

38:29So the obvious kind of meta question would be redesign the institution that helps all the other institutions get designed the right way. Yes. Right. So that would be the strategic one. You're going to ask for more wishes, Reed. Yes, exactly. Yes. My first wishes, I get three. you know, or 10 or whatever. But because that but in practice, that would be, you know, the the overall governance, the shared government governance that we live in. That would be the the primary one. That's one of the ones that, you know, part of the reason why for, you know, my entire, you know, business career, you know, anytime that a that a that a leader of a of, you know, kind of a democracy, whether it's a minister, like I met Macron when he was a minister before he was president and so forth, asked to talk to me about this stuff, I will try to help as much as I possibly can because I think that the governance mechanism.

39:21Now, the reason I'm going to give you two is because I think that one is a very hard one to do, partially because of the political dogfights and the contrast of it. And these people think big tech should rule the world and these people think that big tech should be grounded to nothingness and then everything else in between and blah, blah, blah, blah. I disagree with both. and a bunch of other stuff. And so you're like, okay. So I try, but I don't think. So if I were to say, look, what would be a feasible one for it, saying that would be the top one, I would probably go for a medical. And it's not just because I've co-founded Manus AI with Sid and said one of the great ways to elevate the human condition with AI that's really easily line of sight and seeable is a bunch of different medical stuff and includes psychological.

40:13I think the Illinois law of saying you can't have an AI be a therapist is, I think, kind of like you can't have power looms. It's like, no cars, only horses and buggies because we have a regulated industry here and those people have been licensed. And so it's like, no. But the medical stuff, I think, for example, we could deploy relatively easy within a small number of months a medical assistant on every phone if we've got the liability laws the right way that would then mean that every single person who has access to a phone, and if you can fund the relatively cheap inference cost of these things, would have medical advice.

40:51And, you know, that is not 8 billion people. It's probably like 5 billion people. You know, it certainly could do in every wealthy country and so on. But that's huge. And so that would be like government first, but then more feasibly possibly medical. Anisa, what about you? If you could read the line. I love both of those answers. The medical one, I think, is actually one of the clearest places where I see almost all upside. And I'm like, so we should invest a lot more there on AI. And I also would agree that it is governance. And so we have a lot of the smartest people and insane amounts of money now going into the attempt to build aligned artificial intelligence.

41:32I don't see anything similar in scale trying to build aligned collective intelligence. And to me, that is the core problem we now need to solve. How do we build aligned, collective, hybrid intelligence? And I think you can sort of see it in the sense that we sort of suck at coordinating. Reid, I don't know how many companies you've invested in or how many nonprofits. I don't either. I've lost count. But just imagine, I bet a lot of your companies don't talk to each other all that often, at least not in a very deep way. When I think about NGOs, like, you know, I'm doing work with Earth Species, and I do work with CHT, and even I'm the bridge between Center for Humane Scientology and Earth Species Project.

42:12There's a lot of overlap, but our teams don't even talk that much. Why? Because who funds the coordination role, the interstitium? Like, that stuff always falls off, and so that means, you know, my personal theory of change comes from E.L. Wilson, the father of sociobiology, and he says, selfish individuals outcompete altruistic individuals, but groups of altruistic individuals outcompete groups of selfish individuals. And what we need is new institution, new technology that helps not just the groups of altruistics outcompete, but groups of groups of altruistic groups outcompete. There is no slack for the coordination of companies and hire.

42:55That to me is a really exciting institutional set to redesign. I completely agree. And I think that the notion that you're gesturing at is like, look, it's, we're going to be in a very short order, many more agents than people. And so the ecosystem view of this, and I've taken this as, you know, for irony sake, I'm going to go do a deep research query on, you know, is there ethics of ecosystems and collectives in order to see. I'm curious, you know, it's like great question. and super important topic. Right? And isn't it interesting because I believe, I've asked lots of people and I've also used AI to try to find good terms for it.

43:33I think because we don't have a name for it, people are just blind to it. In fact, I'm struggling with this at Earth Species a little bit where I keep having to say it's not just our responsible use, it's world responsible use. It's the sum total of as our technology rolls out into the world, how is that thing used? Because there are going to be poachers and there are going to be factory farms that might use the technology to better understand animals, to better exploit them? How do we get ahead of that? And that's not just about what we do, but there is no word. And so I just watch in our meetings as like two meetings go by and people are back to talking about responsible use.

44:05I'm like, no, no, no, no. It's this like collective ecosystem ethics thing I'm talking about because we don't have a word to hook our hat on. We can't talk about it. Well, I think, right. There's so many, the history of technology is littered with things that people thought would be used one way and they were used another way. And so we'd have to be thinking about all those different outcomes. Exactly. So I want to get a – oh, go. Just quickly, it's like I think what you're saying is very important because, you know, our friends are the people that have made social media. I knew Mike Krieger before Instagram and you made LinkedIn.

44:37Like we know these people are beautiful, soulful human beings that care. And my own lesson in creating Infinite Scrolls, I made it pre-social media, is that incentives eat intentions. that it doesn't, you get a little window at the beginning to like shape the overall landscape and ecosystem, which your invention is going to be created. And after that, the incentives are going to take over. And so I wish we as Silicon Valley spent a lot more time saying, how do we coordinate to change the incentives to change where the race to the bottom goes to? We spent this more time in discussions talking about that versus like which design feature we should have or not have.

45:16I think the world would look a lot better and by the way i think it's the incentives eat intentions at scale where time is also a scale yes yes well said well so we're doing a lot of um if if we could grant one wish uh so i will say if you were granted the power of running the ftc or fcc today is there a regulation that you would push forward immediately and and azai will go to you first Is there one regulation that you thought would be positive in the world of AI? I mean, the obvious ones are like liability, whistleblower protections, transparency. I would also then put strict limits on engagement-based business models for companion, AI companions for kids.

46:03That just seems like it's very obvious and we should just do that now. If I could then zoom along. Well, I was actually just going to ask both of you, because this has come up actually recently with me a lot. A lot of people are talking about restricting folks who are under 18. And then everyone thinks of like, oh, yeah, how do you do that? I'll just lie and say I'm 18. But then a lot of people also say that these companies have so much information that it would actually be pretty easy for them to figure out if you were under 18 or not. And so I just for everyone listening, I want to sort of verify that.

46:34Aza and Reid, do you have thoughts on whether would it be possible to pretty easily say to an internet user, no, no, no, you're under 18, you cannot use character AI, or you cannot use chat gpt for erotica, or you cannot use these things that should only be 18 plus? I would say that it's relatively easy as long as you don't have a 100 % benchmark. Like the way that people we this is like the little statistics thing that Aza just said earlier, you say, Oh, it's impossible. Well, it's impossible if it's literally 100 % like that one kid who got their parents' driver's license and looks a little older and is deliberately gaming it.

47:13Impossible. Oh, there's some very bright kids to do this stuff. But if it's like you kind of call it at 98 % and maybe more, that's pretty easy. Interesting. And probably this should be a thing that happens at the device level. like if apple implemented this and it was a signal right that that social media companies could then check against like that that's kind of then then the social media companies don't have to know that much about you they can just ask your device and your device can store that in its own secure enclave and that's i think a good way of getting around the problems fair enough reed do you have thoughts on regulation that you would push forward immediately uh well it doesn't it's probably you know maybe a little bit of a surprise for our listeners that it's a bunch of things I agree with Aza here.

47:57I'd go massively on the transparency question. Like I basically think that there should be, that one of the things should be is like, here is the set of questions that we're essentially, you know, putting to these major tech companies to say you must give audited answers to them. And some of them may have to be public and some of them could be confidential that are then available for kind of confidential government review. It's a little bit like, you know, one of the things I liked about the Biden executive order is that you must have a security plan, a red teaming kind of security plan. You don't have to reveal what it is, but you must have it so that if we ask about it, we see it.

48:31Because that at least puts some incentive and some organization will wait behind it. That'd probably be one. Two would be kids. Because I do think that social media, AI, a bunch of other stuff has been mishandling the kids' issues. And obviously there's some places where you have to step carefully because these people want kids educated in religion one, and these people want kids educated in religion two, and these people want kids educated in religion three, and blah, blah, blah. It's a little bit like one of the things that I like about the evolution of the U.S. is when the separation of church and state was like, so your version of Christianity wouldn't interfere with my version of Christianity.

49:13And I was like, okay, but we're now much more global and broad-minded about that. It's like not against Hinduism either, right, as a version of doing it. So make sure that we have that kind of as a baseline. And I actually wouldn't be, even though obviously some parents are so optimal and so forth, if you said, hey, part of the regulation in kids is you've got to be showing reports to the parents. It's like, look, parents should be able to have some visibility and some ability to intersect here. I mean, I think the notion that a technology product could be saying, like, for example, I think it's a dumbass thing we're competing with your mom.

49:50It's like, you should not be doing that, and if you're thinking that, you have a problem. But it's like to be involved because the best thing we can think, while we try to make parents better and we try to make communities better, and it won't always be the case, the fact that parents have, in the bulk of percentage of cases, the best close, like we care about our kid, were invested in it in the kid's life and well-being and we have some weird theories and I may be a drunkard or something else that happens but I'm not the same thing as a private company and it's one of the reasons why do public institutions and public schools have some challenges because they're trying to navigate that thing which always by the way means a trade off in efficiency and other things and you give them some credit for that because they're trying to be this common space.

50:42And yes, they do have at least a lens into the kid, which is useful. This kid's being abused. Well, then we should do something about that. But generally speaking, it's kind of enable the parents. So that would be the second thing. And then the third one, this, because I'm deliberately trying to choose one that wouldn't be top of AZA's list, even though there's a bunch of these that I agree with, is basically, I actually think that the technology platforms are the kind of most important PowerPoints in the world. And so part of the reason why, at the beginning of this year, I was talking about why I wanted AI to be American intelligence, is there's a set of values we aspire to as American.

51:22I don't know if we're doing that good of a job living them most recently, but we aspire to this kind of like, hey, let's give individuals freedom to do great work and and to have a live and let live kind of policy when it comes to religious conflict of values and other kinds of things. And I think that that we want. And I think that actually, in fact, part of the thing that is, we live in a multipolar world now. It's not just a US thing. And so how do we get those values and technology kind of setting a global standard? And that should be infecting. Like here is one of the things that I kind of, it's a little bit off the FCC, FTC question, But people say, I would like a return to manufacturing industry and jobs in the U.S.

52:10And like, okay, your only possible way of doing that is AI and robotics. So what's your industrial policy there? And they go, oh, really? And they're like, yes, it's a modern world. And so we should be doing that. I agree. But we should be like harnessing this great tech stuff we have with AI and then trying to get that rebuilt would be an excellent, you know, kind of both middle class and also strategic outcome in the country. And that's as a parallel for the kinds of things I'd want, you know, the FTC and the FCC to be thinking about as they're setting policies and navigating. This gets into like the very specific, but I think it's an interesting example for what social media could be optimizing for that doesn't require choosing like what's true or not at the content level.

52:56And that is a perception gap minimization. that is to say if you ask republicans to model democrats they have wildly inaccurate models if you say like what percentage of democrats think that all police are bad and republicans say it's like 85 or 90 percent in reality it's like less than 10 or something like that and there's a reverse the other way around so we're modeling each other wrong and so we're fighting not with the other side but with our mirage of the other side so imagine you just trained a model that said, all right, given a set of content, is the ability to model all the other sides going up or down?

53:30I think if you just optimize for accurately seeing across all divides, which by the way is a totally objective measure, you just ask that group what they believe, you ask other groups what they think that group believes, then you realize that the most harmful content, hate speech, disinformation, all that, brain rot stuff, that all preys on a false sense of the other side. So here's an objective way without touching whether content is true or false to massively clean up social media i love it it goes so much with read what you always say about scorecards i'm not going to tell you social media company that this is good or this is bad but i'm going to give you the score card and what we want you to hit and you you figure it out and if you decide that like oh yeah actually promoting those vaccine conspiracies makes people distrust the other side in a way that's not accurate okay well then you need to change your behavior and so again it's actually sort of putting the agency in the company's hands in a way that yes um that is so positive um all right so we're gonna do our traditional rapid fire very soon but first uh we wanted to end on a lighter note because we've talked about vampires and some heavy stuff so uh i'm gonna ask you guys we need to bring in werewolves and zombies but you know exactly exactly i mean i just watched sinner so i do have uh sort of the supernatural on the mind uh so i'm gonna get a hot take from each of you, hopefully pretty quick.

54:49I have, let me see, four questions. So Azai, we'll start with you. What are the most outdated assumptions that are driving today's AI decisions? I think the most outdated belief driving AI is that we can muddle through, that, you know, it's never been a good idea to bet against the Malthusian trap. That is, we've always made it through in the past. And therefore, assuming that because we've always made it through in the past, that we'll make it through this time. I don't know what you, Reid or Aria, would give humanity as a scorecard for the Industrial Revolution. I'd sort of say we got like maybe a C - stewarding that technology.

55:28Lots of good things came out, but also child labor. And now nowhere on earth is it safe to drink rainwater because of forever chemicals. And we dropped global IQ by a billion points with lead, but we managed to make it through. I don't think we can afford with AI to get a C - again. I think that turns into an app for us. Reid, what do you think are the most outdated assumptions driving today's AI decisions? I'm going to be a little bit more subtly and geeky. By the way, I do think we need to get a much better grade. I actually think AI can help us get a better grade. I think so. We need it. But I think the most outdated assumption, because it's kind of like it's almost like against what most people think.

56:06I don't think that people are realizing. People still think it's mostly a data game, and it's turning much more into a compute game. and the data still matters, but the is like the, you know, data is oil, you know, is the new oil, et cetera, et cetera, is actually computes the new oil and it's data still matters, but like what, what it's the compute layer that's going to matter the most. I'd say that would be my quick answer in a very complicated set of topics. Well, the next question, we're giving you just one sentence to answer. So Reed, I will start with you in one sentence. What is your advice to every AI builder right now?

56:43Well, have a theory about how it is that you, that in your engagement with your AI product, whether it's chatbot or something else, how it is that you will be elevating the kind of the agency and the kind of human capabilities, but also broadly compassion, wisdom, et cetera, of the people that you're doing. So, for example, at inflection and pie and personal intelligence, it's like be kind to be modeling a kind interaction is one kind of very tangible output. Fantastic. Aza, do you have one piece of advice? I would be very aware of how incentives eat intentions because the technology you're creating is incredibly powerful.

57:33And so if it gets picked up by a machine or country that you don't like their values, the things you invent will be used to undermine the things you actually care most about. Fantastic. Reid, I'll go to you first. What is the belief that you hold about AI that you think many of your peers would find controversial? Well, a lot of my peers tend to be in the LLM religion, which is the one model to make everything work, whether it's superintelligence or the rest. And I obviously think we've done this amazing thing. We've discovered an amazing spell book in the world with these LLMs and kind of scaling them.

58:17I tend to think that there will be multiple models and the actual unlock for AI and human future will be combinations and compute fabric of different kinds of models, not just LLMs. Now, it might be that LLMs are still, as it were, the runner of the compute fabric. It's possible. But I also think it's also possible that it isn't. And that probably gets the most like, you know, like, wait, are you one of those skeptics? Do you not believe all the magic we're doing? I was like, no, I believe there's a lot of magic. I just think that this is kind of a big area and a blind spot. Aza, same question.

58:52A belief that you have that most of your peers would find controversial. That AI based on an objective function are not going to get us to the world we want. That is to say, whenever we just optimize for an objective function, you end up creating a paperclip maximizer in some domain. but nature doesn't have an objective function it's a ecosystem that's constantly moving you're not actually there isn't just a static landscape that you're optimizing to climb a hill for the landscape is always moving it's a much more complex thing so if we really want to have ais that can do more than confuse the finger for the moon and then keep giving us fingers if we actually want to get like human flourishing ecosystem flourishing like that thing we're gonna have to move beyond the domain of just ai that optimizes objective function awesome let's move to rapid fire and reed i think your question is the first indeed is there a movie song or book that fills you with optimism for the future um uh really anything by audrey tang listening to her podcast reading plurality she's sort of the yoda buddha of technology um so 100 that and then on human nature by eo wilson and finally dawn of everything by david graber because it just shows that how stuck we are in our current political economic system and really opens your eyes to how many other ways of being there actually are awesome what is a question that you wish people would ask you more often?

1:00:37Oh, you know, something about surfing or yoga. Awesome. Which are you better at, Aza, surfing or yoga? I'm definitely better at yoga because surfing is by far the hardest sport that I have ever done. But actually, there is a question that people ask me a lot that I don't have a good answer to. And that is after sort of like laying out my worldview, people almost inevitably ask, but how do I help and I realize I don't have a good answer because to answer that question requires understanding who you are what you're good at what you would like to be good at what your resources are what you're currently working on and I would love to have an answer that when somebody says how can I help there is something maybe I can help with it that does that kind of sorting and helping people find their dharma within a larger purpose I couldn't agree more everyone right now, I forget people who say that everyone's apathetic.

1:01:33Everyone is asking me what they can do right now, Isa, to your point. And I don't have a good answer either. So let's try to build one. Well, I think a beginning is learn and get in the game, right? Like, for example, like start engaging with it and then kind of have your voice be heard. You can't have a perfect plan, but it's like join some movements, rally to the flags that try to help stuff. All right. So where do you see progress or momentum outside of tech that inspires you? Well, I'm going to feel like a broken record, but outside of tech. Actually, I was going to start with all the deliberative democracy stuff, but we've already sort of talked about that.

1:02:15Blaze, I'm going to say his last name wrong, Agua y Arrakas at Google. He and his team are doing some incredibly beautiful work that I'm finding a lot of hope in. because I sort of laid out my worry that game theory is going to become obligate and we're just going to get whatever the game theory says for the future of humanity and that seems like a really terrible world I don't want to live in and his work is on understanding how do you model in a situation of multiple agents like how do you actually get non-NASH equilibria solutions and he's discovering something which is that in order to solve the very hard problem of how you do strategy and multi-agent reinforcement learning when i have to model what you know and what you have to model what i know and i have to model what you know about i know and that you know and that's just very hard and they're discovering some new math and it turns out you can start to answer this if you don't just model with yourself outside the game board but with yourself on the game board you have to model yourself modeling other people and what's cool there is that suddenly non-NASH equilibrium states are found, not the worst of the prisoner's dilemmas.

1:03:26You can find these new forms of collaboration. And I love this. It feels so profound because first you have to inject the idea of ego and then transcend it. If you don't have ego, you just find the Nash equilibrium. If you do have ego, you also find the Nash equilibrium. But if you do have ego and you can transcend it, you can get to these much better states. And that to me is very hopeful and very cool because I think of game theory as sort of like the ultimate thing that we're going to have to beat as a species. Always, Aza, our final question. Can you leave us with a final thought on what you think is possible to achieve if everything breaks humanity's way in the next 15 years?

1:04:01And what is our first step to set off in that direction? This is sort of like the what is possible if we could rearrange our incentives so we are both nourishing ourselves and nourishing all the things that we depend on. Suddenly, I think people don't really look at their phones because the world that we inhabit is just so rich and interesting and novel. We are consistently surrounded by the people that can help us learn the most sort of in a developmental sense. The entire world is sort of set up in a fiduciary where everything we can trust is actually acting in our and our communities and our society's best interest and developmental understanding where we are and helping us gain like whatever that new next attainable self is.

1:04:57I think we'll have made a major, major, major progress towards solving diseases. We'll have a deep understanding of cancer. And I think we would have solved our ability to socially coordinate at scale. without subjugating individuals. So it looks something like that. We will have solved the aligned collective intelligence problem. And we'd be applying that to getting to explore the universe. Awesome. Aza, thank you so much. Yeah, the universe outside and the universe inside. Yes, exactly. Aza, always a pleasure. Thank you so much, Reid. So much, Aria. Possible is produced by Pallet Media. It's hosted by Aria Finger and me, Reid Hoffman, Our showrunner is Sean Young.

1:05:45Possible is produced by Tanasi Delos, Katie Sanders, Spencer Strasmoor, Imozu, Trent Barbosa, and Tafadzwa Niemorundwe. Special thanks to Surya Yalamanchili, Sayida Sepieva, Ian Alice, Greg Beato, Parth Patil, and Ben Rallis. This podcast is supported by Google. Hey folks, Stephen Johnson here, co-founder of Notebook LM. As an author, I've always been obsessed with how software could help organize ideas and make connections. So we built Notebook LM as an AI-first tool for anyone trying to make sense of complex information. Upload your documents, and Notebook LM instantly becomes your personal expert, uncovering insights and helping you brainstorm.

1:06:27Try it at notebooklm.google.com.