#29 - Peter Wang - Founder of Anaconda on Open Source, AI & Virtual Worlds

28:52It sounds like that approach is particularly good to solve for increased innovation, more secure products, etc. I don't know about security, actually. That's one of the weaknesses of it, but it does allow parallelization of innovation. Oh, interesting that you mentioned that it's... I would have thought that the programs that... or rather the software that gets many more eyes on it and of which the source code is open is going to just naturally have a higher chance of high security because of that? It can be. Transparency is one component of having greater security, but it's not sufficient. So this is an argument that actually one of the early famous people in the open source community, Eric Raymond, he had this quote that, you know, many eyes render all bugs shallow.

29:40He wrote the book, The Cathedral and the Bazaar. It was like an essay. It was pretty seminal in the 90s as sort of a piece of the open source movement. And he made this argument. And unfortunately, time has proven that's not the case. Because when you build something really, really, really, that's not sufficient. It's sort of like a reason. It's a bit of a, it's hard, right? When you build something that is of a certain technical sophistication, then relatively fewer people will really understand how to get into the details of it. It's not wrong, but it's not totally right by itself. I guess I don't want to be like all against it or anything.

30:15I do think transparency in software gives us better hope of having more audit, more verifiability, things like that. That being said, the open source development communities now are being actively attacked by state-level actors. And this is something that's now come into the foreground from just in the last month or two. Like this, you know, we've now had an attack come to light that specifically preys upon how beleaguered some of these open source devs are, how under-resourced they are. So state-level actors are doing long-term multi-year human psychological ops against targeted software maintainers and taking advantage and exploiting the open source nature of the innovation commons.

31:01What's their goal in doing that? What are they trying to do? Well, the specific thing is what I'm referring to is the libxz attack, which happened, which may not have gotten out of the technical sort of arena into the broader world. But it's a really, really shocking moment for all of us because it was a multi-year attack where there is a library called libxz. It's a compression library. And it's relied upon by a much more important library or program. that is how people use the more important program is the one that people use to connect remote servers it's a foundational piece of technology for human civilization and it has this upstream dependency which is this little library that was under maintained the maintainer was sort of like you know burned out kind of you know and all this stuff and a new person showed up just submitting helpful like fixes here and there participating a little bit and then they had some additional things they were going to add and fix some bugs and write some better tests and then someone else chimed in and said hey i want this feature merged can you can you guys merge it and the guy and the maintainer's like no i'm too busy i don't have the bandwidth and then this other person says hey well what about this person he's been helping and everything why not just let him also be a maintainer and commit this stuff and the original maintainer says okay well yeah maybe you know it's accurate this person has been quite helpful so then they merge us in turns out both of these people are acting the same team they may even be the same person and so what got merged in then was a backdoor exploit.

32:26And this was a multi-year attack to earn the trust of this one maintainer, of this one little critical piece. It's like finding an exhaust port on the Death Star, right? And it's also like a cockroach. When you see one cockroach in your kitchen, you know there's a hundred more hiding behind the stove and the fridge and everything else. So this is just a really amazing moment for the open source community because, I mean, I say it's a state-level actor. I think people mostly agree it must be a state-level actor that is paying someone in a black hat hacker sort of employment scenario to have someone do this kind of attack.

33:01And if he's attacked this one thing, they've attacked who knows how many more. So that's kind of what you pointed at, that security is not necessarily higher because if you're talking about very sophisticated software, then it ends up being the case that individual bits of it, which are critical but somewhere deep in, they don't get that many eyes anymore. They don't get that much maintenance anymore by that many people. It's in this case basically was one person who was responsible for this one critical piece. For this one critical piece. That's pretty nuts. Yeah, that's a massive attack vector, of course.

33:31And when you look across the open source software world, most things are actually held up and propped up by just a few people. You'd be shocked at how few software developers it actually takes to make something that is, or how few there are involved in maintaining some of these really popular, well-known things. And presumably that's something that commercial software is not so vulnerable to because they have more direct incentive to protect against it? No, because they starve the development. Because there's only so many people who can commit to the source code at all. But the way the commercial software works is that the only people who can update the software are the people employed by the company.

34:09So there's simply not that many people employed by the company to maintain it. Even some of the most powerful and critical pieces of, not powerful, but the most critical pieces of software for really big companies oftentimes can be maintained by just small teams of you know 5 10 20 people i heard it i heard anecdotally this could be apocryphal but i heard anecdotally that at one point in time internet explorer was being maintained by essentially one dev with a stack of product managers on top of that poor one dev but it was like down to essentially the equivalent of one dev now that could be not true but i just heard that But I do know like the core Oracle, you know, Oracle database, they're massive company, the core Oracle database engine team, not, not that many people, you know, for how much money that thing makes it's, you know, maybe dozens, maybe a dozen.

34:53It's not that many people on the core engine. Same thing for the limit, for the windows NT core kernel, the heart of the windows operating system, that core, like the thing that actually runs all the processes, not that many people. So software is unbelievably productive. Yeah. It seems like one of the massive downsides, if you have truly only one person, for example, working on the maintaining Internet Explorer, and then someone finds a vulnerability, they're like, hey, you have to update the Internet Explorer, and then the person is looking at his task list given by all of the product managers on top probably as well, and it's like, okay, I don't know when they get to it, so this vulnerability then exists.

35:29And I thought that the advantage of open source may be that basically you posted various people now have the chance to update, add some code, propose code even to fix that vulnerability. And the user also, the end users have the option to kind of, I suppose maybe they don't, they have the same option as with a closed source software to decide whether they run it in the meantime or not with a vulnerability. But yeah, is that the case that when we look back at it historically that such updates of relevant vulnerabilities occur faster with open source, with actively used open source softwares? I can't say if that's true or not because it would be hard to benchmark that against closed source software where they also do have instant response teams that are very skilled and can respond very quickly.

36:15I mean we could compare like for like Windows versus Linux for example Yeah but is it really is it an apples to apples comparison if like for the Windows release process they have to test for compatibility across all these different things and updates for all these languages simultaneously and all this like there's just the products are very different right so it's not really necessarily an apples to apples comparison if you're solving for the question if you're trying to answer the question of development efficacy right what i do know is that for most of the common kinds of exploits that you find they will patch them pretty quickly um but it's also not the case that all exploits are trivial to fix either right some things may require engineering some some kinds of bugs may require oh we have to rethink how we're doing this whole thing and when we change it now the thing works somewhat differently and end users may not want to take this update they may just want a hot fix for the way it used to work etc etc software is just of such incredible complexity that it's hard to say in generality right what is what is and isn't true for some of these kinds of things what are your views on open science because i've had to be in my bonnet against like Elsevier and some of these big publishing companies they don't pay for the research they receive you know the papers but they then charge the scientists or anyone who wants to read the papers that they receive like$32 a paper including technically the person who wrote the paper like if they want to look at their paper again they now have to download it I'm sure they keep a copy but like that seems absurd and their profit margins are some of the highest in any any industry they're like 37 or something like that and then uh quite famously this this girl from kazakhstan alexandra um el kabayan el bachayan uh created something called scihub which is basically kind of like a pirate bay of of science papers to allow people around the world to access them for free uh she then got sued etc by these big companies so um i know where i I emotionally stand on this.

38:21I think it's quite awesome what she's doing, but I'm really curious what your views are. We'd be remiss not to mention Aaron Schwartz, right? Who, poor kid, was hounded by the federal government to the point of suicide because he was trying to, you know, he was trying to basically push for this kind of open access for publications. And anyway, yeah, this also is a very, very interesting topic for for the podcast because at the end of the day you know we actually have to argue we have to we have to sort of say well what is an appropriate amount of profit margin right if because if we could answer that question this then maybe we would just be able to dovetail this question of is uh Elsevier like or Springer or any of these kind of like nature like all these guys like is scientific publishing what is the appropriate margin what is the appropriate tax for scientific publishing to charge civilization for what it does.

39:20I don't think it's intellectually honest to say that they do no work. I think there's a lot of value they add. I mean, when you have, let's say, very, very passionate academics fighting with each other about who's right, about whatever, and their careers on the line, yeah, it's useful to have a third party that everyone trusts. There's like this intersubjective, there's value in having a judge from a point of intersubjectivity, okay? I mean, they're a coordination mechanism. Yeah, it's a coordination mechanism. And we understand we have to pay a tax for that to work. But are we being overcharged that tax?

39:53And is there a way for us to peacefully revise or argue for a lower tax on that? And that is a general, I mean, I would say central banking has this issue, like all sorts of things. If you look at the profit margins and what portion of profits are captured in America by the banking and finance sector, is that an appropriate tax to pay for the advantages that we have of a market economy, right? Because that's ultimately the question of capitalism. Like, is 40 % of profits too much of a margin? Is there too much of a percentage? Is it too low? What's appropriate? So the thing with the appropriateness is that, like, I'm very near to the idea that it's not the case that there should be someone from up on high that says this is the right margin, and that's what's allowed, and you kind of would want the market to let decide.

40:42But then the market has more ability to adjust those, to allow a competitor in with lower margin, dependent on what the field is. That's why one of the things that is the good version of antitrust is when monopolies emerge, because then they now can increase the margins too high. And you can have them in particular when you have either very strong network effects that are hard to jumpstart. or when you have this inadequate equilibria, so to speak, where the prestige is just so associated with it, which is kind of what seems to be the case with Elsevier and Nature. It may be the case that someone could run a successful with only 10 % margin or 20 % or something like that because I'm a successful competitor, but how do you jumpstart it?

41:35If you can build a more efficient highway system to the American highway system, how would you even start? that competition. You could not, right? Boring. Yeah, but I would say that's probably not more efficient in the near term. That's like quite expensive. And this is not just us, I think, theorizing in the abstract. This is actually the central problem of coordination that we sort of have, what, a few modalities we can imagine. One is the top-down coordination, saying, well, we've solved it. This is how you do it. And the second one is allowing for some kind of emergent thing to happen in a market or something like this in a network.

42:13But networks and markets ultimately can settle into an equilibrium that is not the optimal, that is a local optimal. It may be even global optimal for its time, but then as the world evolves, it's no longer the optimal. How do you infuse a world seated at that little optimum to jump over its hurdles to settle into an even more efficient state, right? And that is a coordination problem that we, I think, don't even have the mythic infrastructure to describe to all the participants in that network, like what that would even look like, right? And yeah, and so all we know is, however, when you blast it with commoditization where it drops the cost of everything, then people have no choice but to go and try to re-emerge something.

43:03So, you know, in a sense, we kill it with the wildfire of commoditization and see what emerges out of the burnt forest. And that's about our best model. And it's possible that that is the only model. I don't know. But I think this is the general problem we have to solve to allow for what peaceful transition and upgrades in the punctuated equilibria when we do settle into efficient states for a network or a market yeah the thing you mentioned that it also not only is uh the case that it might be a local optimum state local most efficient state while there is somewhere a global one but it may be that it's global at the point in time which changes is actually relevant and often forgotten because markets optimize uh dynamics and prices etc given the currently available resources at the time right and sometimes those change quite a bit like we're I think an example of that is also like the the reason open AI is developing currently or everyone is developing LLMs that are initially chatbots is that we have this large database that is not large database but it's the amount of text that's available on the internet right it is if the goal was initially to say hey, we're building AI to improve medicine and all of those things.

44:28In no way it is actually the case that, oh yeah, to get there, you first have to build LLMs on the basis of the internet. It's not the direct path. It just happens to be that in the pursuit of trying to get revenue or use for AIs first, you're going to look at what is freely available somewhere. What can I use very beneficially? And then, yeah, it is an open exploration, right? And they happen to find that, oh, text-to-text translation tasks can be generalized. And holy crap, we have a thing that, you know, I think GPT-3 came out a year and a half or so before ChatGPT came out. And even then, people were like, oh, my God, this thing is amazing what you could do.

45:09But you have to build all this infrastructure around it. You have to use it as a programming library sort of to talk to it as an API. and then when they actually released chat gpt with this like one tweak with this one one weird trick they were able to produce something that captured everyone's imagination and got people started talking about like the consciousness and all this other stuff and it's like yes but let's just keep in mind it was all still a trick that is based on the invention of the transformer model for doing translation between languages right for actually doing vision models and for doing like Like what does a recurrent neural network, what could it actually distill from images and graphics?

45:45We were starting to have really amazing results in the 16, 17 timeframe. And I don't know if you remember those things of the generative adversarial network scans and style transfer where they can make a picture in the style of Picasso. That for me was the moment of like, okay, we're off on a different world. Like we're on trajectory to a completely different world right now. I'm not sure how we're going to get there, what's going to happen. But that was a vision task, right? Not a text translation task. But backing up to your broader point that, yes, when the market settles on an equilibrium, it is a point in time.

46:20But the thing I want to say about markets, though, is that for us in the West in particular, and I'm a refugee from communist country, so I very much appreciate markets, capitalism, and a lot of the nice things of the neoliberal order. But one thing that is interesting is that a market lets you diffuse the question of values. You can pretend that you don't actually have to answer the question of what does this community of people participating in a market, what are their values? If you have a top-down person who says, no, you know what? Everyone's going to wear this style of color today. Or no, you know, we're not going to build these things because those are bad.

46:56Those are unholy and against the will of God. Well, you know what the values are. And you can point to one person at the top and say, well, that's the, that's the guy. That's the reason we can't have nice things because that guy says this is unholy. Right. But when we make a market, you know, you sort of diffuse culpability about making a decision. And this is, I don't know if it's good, but it's certainly bad in many ways. It could be good in many ways too, but I see the bad in it. Right. Because the bad is that when many people make little decisions that all have some net external cost that they can't easily sum up, then we have this net externality that no one wanted.

47:31But you can't find any one person to blame. You can't say, well, I want to do that. When you say bad, do you mean worse than the top-down version? Or do you mean just it has... I said it has also negative effects. I'm sorry, I'll be very clear. Yeah, yeah, yeah. To clarify that. I'm not in any... At any point... Just starting with refugee from communist country. Right. If any of my words can be interpreted as to support an authoritarian sort of approach to the world then please you have misunderstanding me and i approach i probably have mis-expressed myself right no no it's i just wanted to clarify i think that that um that are uh a lack of culpability and a lack of kind of a clear articulation of values in a market that is part of the neoliberal trick it's like hey by simply shopping your preferences and sufficing for your own happiness maybe in the near term or even the long term uh you're absolved because you get to choose.

48:21It's multipolar. It's multicultural. It's da, da, da, you know, express yourself, be yourself, love yourself. And then it plays very nicely into the sort of hedonistic trap that then degenerates, of course, nowadays into narcissism. And then when real like bad stuff is happening in the world and we actually have to like all get together and like, like, Hey, we got to do something about this. We've lost the ability to even imagine what collective kind of coordinated value system driven action could look like. And I think that's this really weird thing, not weird, but it's kind of an unfortunate thing that's been lost here because we have a lot of these externalities.

49:01We, you know, I would say this whole, like, at the time, I thought it was quite brilliant, this articulation of think globally, act locally, right? But I think if people are not acting intentionally locally, somehow the aggregate of our local actions did not equal the output we wanted the outcome we wanted the global level you know turned out to lead to you know giant rubbish heaps in the middle of the ocean and ngos funded to the tune of billions achieving absolutely nothing in a lot of developing countries where they're supposed to be you know doing things it's just like this is this somehow and the sigma did not yield the integral you know somehow is that is it just because of the improper in transfer of information is that what it is it's just because people can't see the aggregate outcome and thus what is the moral structure of a humanity where there is a a spectrum of both uh information uh sort of uh propagation and ability to act on that information because we would like to think that if people just had the right information, then the sum aggregate of some kind of longer-term horizon like rationality will lead them to not burning up the earth or polluting in the ocean or whatever.

50:23Which is why we believe in markets as well, because they are an attempt or the best attempt we currently have so far to aggregate that information in a way that is action-relevant. And so like carbon capture and other kinds of things are... Or why I'm saying it's the best current attempt is because other ones also have externalities. And the market obviously also does. It's just that it still achieves... Yeah, it achieves a lot of things while still having... Yeah, yeah. And the question is really, I think, where we go from here, right? And if the paths we see forward to better and wiser collective action, if they require everyone to all be at this level of information, agency, rationality, et cetera, et cetera, you know if if your site only works if it's eight billion high elves then it's probably not going to work right but at the same time how do you then have a moral theory of governance and of like a practical theory maybe that you say look we give everyone the door to becoming better versions of themselves more agentic more whatever but at the same time recognizing when they're not able to make those kinds of decisions but without us being a holes that tell them what they should think think what is the right scope for them to figure out their pieces?

51:34And I think this is where the liberal theory ends a little bit in terms of once people are free to make certain kinds of choices, is there a way to go from the outside in as a boundary value problem and say, hey, you own a portion of this bad thing that happened. Here's your portion of the externality, and what are you going to do about it? And is the market still the best mechanism for communicating those kinds of things. Is there a version of markets where we can communicate the total cost, where we can actually have people surface their needs versus just buying once something is priced in after production?

52:11Are there these kinds of things that we could imagine that would respect the agency and the sort of the inviolate sort of like the primacy of the individual, but at the same time recognizing that we do have a spectrum of individuals are trying to aggregate this over. And some of them, if we just let them make the decisions they're making right now with limited information they have, we end up in really bad places. That's the big looming question. I mean, that's where the government usually is meant to step in, I suppose. Or that's one of the solutions that have, that's the solution we're currently employing, right?

52:45It's, well, that's why we're arguing, someone is arguing for a carbon tax, or that's why we're having social welfare systems, because some people can't actually participate in markets if they don't have capital or labor force available to them at present. And yeah, I think that's one solution. But in the end, the government solution is still a, okay, they have the bigger guns, and that's why they get to impose a solution. It would be kind of neat to also have one that makes the market wiser. Or we need to give a way to make the commons have its own personhood, its own agency in some way. Those are two fantastic different directions when you take the conversation because i agree and disagree with both right that i think there's something in both of those things but what i would say about the government thing maybe as a as a question is i think i think that you can really only have freedom in the classical sense if people have awareness of their environment otherwise you're just putting different colored of lollipops in front of children you know the simpsons lampooned this actually there's an episode where like it's like mr president you have a decision to make here's five options and then you know he reaches for like option a he's like you know and so you know it's like this is the this is the kind of thing that we make fun of what happens when we have ignorance at scale and and so a motivating question would be this like could we imagine would we imagine that uh the nation would be run well if we gave people from the ages of five to eight, five-year-olds to eight-year-olds, if we only gave them the right to vote.

54:25That seems like maybe not so great, right? But check this out. Why not? Okay. Well, it's because we would say they probably don't have the information to make good choices and they're easily swayed. You literally throw candy in front of them and they're like, well, I'm going to vote for the guy who gave me more candy. So, okay. So we say that the five to eight-year-old, you know by eight years old you do not have enough information to make high quality decisions about the world and so your choices then you know not that they don't matter but but this is not a good arrangement of human affairs well what about by the age of 10 or 13 you know there was a time we'd send 13 year olds into battle what about the age of 18 why is that sufficient okay why do we believe because if you think about the last 50 years how much more complexity we've added to the world how much more infrastructure and if you think about the world that's coming.

55:12We have people in Congress who barely know how computer works or how the internet works. okay that is i would say the equivalent of going to 1950 and putting an eight-year-old into congress i think we have to actually do this and i'm a physicist i would say we have to actually measure the quality of our governing no matter what your theory is you're like well democratic you know republic people vote great i don't care about the architecture what i care about is for a civilization of that that exists at a certain level of complexity is there such thing as a minimum level of information and education that a person has to have to actually be achieving personhood in the sense of being able to make valid decisions about themselves, their families, their communities, and their country?

56:05That, I think, is a perfectly valid question. And we might not like the answer we get, but the answer could be of the form that, honestly, we need to provide free education for everyone to the age of 28 because we're adding whole new centuries of human knowledge. We're adding equivalent. How many 1 ,800 to 1 ,900 equivalents of knowledge are we adding per month to human knowledge? And once we have AI going foom, what is that going to do, right? So I think this is actually the key question. Because, yeah, anyway, that's got a bit of a rant, but that's my sort of framing of this question. Yeah, I mean, it's certainly the case that in the ideal version, one would like to see everyone who's making relevant decisions that affect many other people to have the prerequisite knowledge and capability to make those decisions.

56:54But at the same time, one can't, as you increase complexity, as you said, like, well, maybe they need to learn until 28, but then it increases further until 38. So to which point do you push it and probably you reduce it, you maybe make it domain specific and maybe figure out like representative voting exists as an idea, right? Which is actually kind of neat where you on per issue, you decide that you're giving the, you trust someone to make a decision that you would believe affect you positively on that issue and you make a per issue representation of yourself and then they can further pass it on potentially.

57:30But yeah, it's an interesting question of like what amount of information is necessary to make such decisions. Do you have a guess of how to deal with the truly ever increasing complexity that we will have? I think we have to actually create collective sense-making like collectives. I think this is, you know, when you look at nature, biomimicry is not necessarily the end-all be-all of ways of knowing, but certainly nature gives us a hint that at a certain level of complexity, at a certain level of difficulty, hey, if you get together and you special, then you can specialize, but you have to really trust the interiority, right?

58:08And then your boundary surface is smaller and you can all specialize and do your thing. I think for any single individual to want to be a polyglot to be experts on law and history and finance and technology and and and all these things is asking way too much and that we're now maybe forced with this uncomfortable thing where you know we can't just trust everyone to be a generalist oh I read the paper I you know check the news now I'm informed to go trade some stocks and do some other stuff you know it's not gonna be enough so if we actually think about what a future society that's resilient to Moloch would look like, you know, Moloch sort of emerges from misplaced institutional trust, ignorance, fear, fog, all these things.

58:50People sort of dissipate energy into what then gets sucked into the engine of Moloch. That's what we're actually, we're not really quite sacrificing our children directly, but we're sacrificing our own potential futures and our agency. But if you were to actually form resilient collective tribes of sense making where actually every individual has financial stake, real stake in the decisions and what we decide to do. Now, like, okay, people have some chips on the table, right? And now you can actually maybe get together and say, well, look, you are the finance expert. I'll be the software expert.

59:22Someone else will be, you know, whatever, geopolitical expert. And we will form a high quality tribe and maybe we do some of this stuff. Like this is maybe where we end up going because Because I just, I mean, to your point, the world's only getting more complex as time goes on. And our ability to actually do modeling and sensemaking is increasing. So we're going to be competed by other groups and other human collectives that can maybe do better than just us individually. How would you recommend, because I think a lot of the viewers listening to this are the type of people who would be interested in building such a community.

59:54Have you tried doing such a thing? Like, how are you navigating the sensemaking crisis right now? Making as much money as possible. at the end of the day. No, no, that's half tongue in cheek. I'm trying to do it by, well, by thinking through things like this, having conversations with great folks like you all and others. And really looking at what is the best, you know, what is on the best discourses available and best thinking available on, you know, collapse, metamodernism, a lot of these like fun sort of

1:00:26topics. But also amassing a certain amount of personal resources to then when I do make a call, I can put a lot of chips down on it. Or if I see, okay, it's down to two or three solutions, I split my chips and put them on those. So I think that's kind of what I'm doing. And also just part of it is we are, I mean, I would say for the last six or seven years, Many of us have been talking about just being in this liminal zone where even prior to the pandemic, global sensemaking is actively being warred upon. And as an individual trying to do sensemaking, it's very, very difficult. Yeah. When you point at the last six, seven years, what are you in particular pointing at?

1:01:05In particular, sensemaking, so intellectual, actual, rational, trying to figure out what's going on, has been more and more, it's turned into a cultural mode as opposed to an intellectual mode. so it becomes more about what is your tribe and that tribe dictates sort of your filter of what comes in and has happened and there's been many different kinds of events where when you really get into it and you look at the objective like okay what are the actual facts or you know and we we can look back um in many of these events we look back through lens of history and see like okay well more facts came out or there was a trial and we actually looked at what happened and facts on the ground should i have interpreted that event to be what it was at the time right?

1:01:45And this applies to events from many of the things in BLM. This applies to things around Trump. This applies to things around COVID. This applies to a lot of these kinds of things. I'm someone who's classically left-leaning, progressive, whatnot. But I've also read a lot of libertarian and conservative Republican kinds of things as well in my time. So I don't have, I really do try to do sort of my own kind of sense-making around certain kinds of things. So when COVID happened, I was very strongly on the lab leak. And I still am quite vocal about having supported the lab leak hypothesis. And at a meta level, how quickly the Overton window snapped down on it was extremely suspicious to me, right?

1:02:25So anyway, now in the fullness of time, because what's interesting is for someone who has some visibility in the scientific community, because my software is used by many scientists, someone who's literally from Wuhan as well. So I have connections to the Chinese Wuhan community through my parents and WeChat back in the day. So through a lot of these things, it was interesting to me to see how many people, I would say in the center left, came down really hard, and in the science, sort of mainstream trust the science kind of folks, came down so hard on skeptical inquiry into this particular topic.

1:02:56Because, you know, I'm classically trained as a physicist, skeptical inquiry is sort of my calling card. Like, I get to ask questions about whatever, you know, and I'm going to ask those questions until I get a satisfactory answer. And it was interesting to see like, oh no, you're like, you know alt-right conspiracy theorist and like here's your pepe the frog thing i'm like no no i'm literally just asking some questions about how unlikely it is for this thing to have been like you know whatever a bat spit on a pangolin or something like no like really seriously people let's look at this for a second right end of 2019 there were around 39 000 wet markets in china presumably let's call it you know there's some let's say there's a hundred or so in wuhan um There's probably fewer than that, but what are the odds that the first emergence of this is in the wet market just down the road from the BSL level for virology lab?

1:03:49Which literally was funded. Working on bats, coronaviruses. I mean, I was really trying to do some legwork on my own and gathering some of the papers and looking at some of the stuff. I actually went to the Internet Archive and found a prior version of the Wuhan Institute of Virology website where somewhere like page 13 or so, there was a publication. from 2015 when they were talking about some work they were doing on like human optimized h2 receptors based on the you know like sars cove one the sars um genome and all these kind of stuff well scroll forward to 2020 that particular publication got yanked so you can go actually go look at internet archive and you can do the delta you're like huh like that's not a smoking gum by any means but it's like these people are aware no i mean i could see a world where they take that off where they're innocent right right that's um but then there's but there's lots of other side.

1:04:36We don't have to get into that particular topic. The point just being that, because the topic here, the question here is what are examples or, you know, what are some things that are hallmarks of this challenge to sense making now in the last six or seven years? And I think that's an instance of where some of it is, I mean, a lot of it's social media. A lot of it, I think, is people feeling like, well, my tribe is an online tribe. And the way I signal who I am, my online identity is, well, I have a pride flag or I have a pepper the frog thing or I've got the Wall Street bets when it was GameStop and to the moon kind of stuff.

1:05:10People become tribal at an iconic avatar level in these places. And because it is a soundbite market, who is holding who responsible or how much accountability is there for someone making a bad call? You guys are poker players. You get held accountable for bad calls instantly, right versus like on these areas of discourse they're not serious forms for discourse they're performative sort of as if discourse places and there is simply no almost no consequence for people being wrong well how high of a quality of a signal are you gonna get from that and yet because attention it doesn't one of those points that i i feel um very strongly about attention is actually a scarce resource for any individual of course i mean it's not particularly controversial but the synchronous or the joint attention of a society is a scarce resource.

1:06:02If I listen to New Source A, or I'm paying attention to phenomenon A, and you're paying attention to phenomenon B, and you're paying attention to phenomenon C, we're doing our own little thing. But if all of us together are getting three different looks at a single phenomenon Z, we're going to have, I think, better perspective on Z. But Z better be worth our joint attention. So this joint attention being this commons resource, the way that a piece of pasture land in the Swiss Alps is a commons resource, that is, I think, underappreciated in all this discourse about social media, that what it did is not only evaporated, it's sliced, thin sliced everyone's attention.

1:06:39So everyone gets their own little algorithmically compiled little bits of pieces of stuff. But when we even do have joint attention, it only shows up when it's something particularly outrageous. Oh, a black person got shot by a cop. Or, oh my God, can you believe Trump did that? Or like blah, blah, blah. And so even these joint synchronous moments are only convened in a sense when there's something outrageous, not when we have a moment to contemplate and go deep. You think about church traditions, you get together on Sunday, not to get, it's not the 15-minute hate or whatever the thing is from 1984, five-minute hate, whatever, two-minute hate, whatever it is, I should know.

1:07:14But you get together for several hours of contemplation and meditation and reflection. That's very different, right? So I think the sense-making has gone downhill primarily because of the rise and primacy of social media as a way of signaling as if sense making as if discourse instead of deep compilation so is that maybe why the super bowl i mean the market seems to still dictate that you know the having an ad and the first break of of the super bowl is the most prime time attention real estate because it is the synchronous event where actually whatever the largest percentage of America is watching a particular thing, possibly even more than a presidential inauguration at this point, because like people, you know, half the country hate whoever's getting in at this point, whereas everyone seems to love the Super Bowl.

1:08:02But it's an example of actually a happy thing. It's not an outrageous thing. People are tuning in. So maybe that's where you can do, maybe we should buy sense-making ads. Well, yeah, I mean, the synchronicity of that in a world that is more and more online and indoors and has less and less sort of bumping elbows and rubbing shoulders in the outdoor, in the real world, not IRL, in a world that's more and more online, synchronicity and time and that kind of stuff, that dictates more of what is our actual shared reality, right? I mean, society is ultimately, fictive kinship is such a critical concept, right?

1:08:43There's no way you know everyone in America. There's no way that anyone in America even knows more than a few thousand people. And so the idea that we are part of a country of 300, 400 million, it's fictitious. But it's really important we believe in that fiction, right? And it's really important we believe that we are part of a nation of several hundred million as opposed to a split nation, split between 180 million or whatever. And I think that fictive kinship and the radical destruction of that fictive kinship in this kind of evaporation of shared synchronous sort of moments, that is a deep sort of rot that's happening for us.

1:09:23Yeah, it's also interesting that actually, like as we were talking before about the collective sensemaking, the reaction to go to very tribal versions around a topic is kind of an attempt in part of that, right? Like overvaluing, like, oh, I'm just sticking with whatever my tribe believes ahead of necessarily what's there to be true. It's the reaction we just described you would have to a complex world where now you have to go into a collective so that you can make sense of it. So you will actually rely very often on the collective that you imbue with trust. So that's kind of what is happening with the world get more complex and everyone is like, I'm just sticking with my tribe here.

1:10:08And that's why it's like, yeah, it's very understandable as a reaction. It's just then it becomes the question, I suppose, between how do you trade off tribal versus truth? Or not truth, but your interpretation of the credibility of something versus the sticking with a tribe. Man, this is all of it. This is the key thing, right? Because an actual person, if you think about the brain you've got in your head, and this is something I realized the day after Trump won the election, because I saw all my friends, all my Democrat, liberal friends, they were just in free fall. There was like, the world didn't make sense to them.

1:10:55And I realized, oh, you know what? it actually is this very simple thing that the brain really wants the brain is seeking for a sane world. The brain is solving for sort of coherence. And it wants things to make sense more than it wants the sense to be true, the sense that is made to be true. And a lot of my liberal friends were just, and even in the wake of like for months afterwards, there's a lot of this talk, even now, a lot of talk about disinformation and the truth. If we just get people to understand, and I appreciate, I really understand deep in my heart where that comes from, but there's unfortunately, I think a lack of recognition of something that is true about human nature, which is that we really do want the world to make sense and we will sacrifice a lot in order to at least have that first boundary.

1:11:54Like our ground floor substrate is we at least are not insane. We should actually have some story. That story of what's happening in the world has got to actually be coherent for us. And that story is a cultural story. That story is a personal story. That's like, we have to have that as our firmware. And then that firmware has to be the substrate and it can be load-bearing or could not be. It can be very shallow. And then we have to keep going back to our cult leader to keep giving us more validation and for us to do our little things, but, or it could be very self-sustaining and we could put a lot of weight on it, but, but that is still the fundamental like thing that your brain needs, I think.

1:12:32And so when it comes to sense-making collectively, like you were saying, people are looking for social validation, especially in times of confusion. They're looking for like, am I, wait, am I the only one here? And what social media did was it connected people of vastly different regimes and locations to being part of some of these kinds of intellectual or cultural motifs, which is why all of a sudden you have lots of literal Nazis in America. Well, it turns out there were a lot of not like proto Nazis, but once you found a way from them to all find each other, now they're doing peer-based sense making like, whoa, I can't be some weirdo.

1:13:06Like all my friends tell me I'm weird. I don't even say these things in public at the bar with my friends, but I'm now connected to a bunch of friends online who are all like, they're my people now. And we're all in agreement that yeah, like being an American Nazi is like a good thing to do to save our country. So this kind of cultural-based sense-making thing is part of the operating system in our brainstems. If you have wrong models of the world, you can either update those models on the basis of the input that you've received, or you can ignore the input in part. And when we describe that you can show someone who sticks with the tribe despite new information coming in, that is kind of a reaction that is of the type of ignoring simply the input so that you don't have to update the model of the world because it is a lower stress response, basically.

1:13:55And the internet has mediated and made it easier for you to retreat into that bubble where everyone tells you you're saying. Because now the model has become even more enshrined. Beforehand, your model had less certainty. Now, if you're going into and find little conspiratorial groups that often turn out to be right, but sometimes also not, then you now calcify that model more so now the sensory input needs to be all the even stronger for you to update that model in any way i mean i do blame the algorithms and some of these kinds of things but at the same time the internet is also you can go and find what you're looking for right and this is kind of a problem because there's way more stuff out there i mean this is the whole point of being in the liminal like who is actually prepared to really encounter the liminal where do you really know is this really true is this you know it takes actually quite a bit you have to put on like a eva suit space suit with some oxygen to get into all that and be like oh am i supposed to update my models on this fundamental thing or should i not or whatever like it's not something you just do casually and this is where i do take issue with what some of my again i i seem to be crapping on my liberal friends a lot i'm actually kind of fairly liberal myself but i feel like they're the ones who are most put out because they tend to be like college educated intellectual they value intellectualism all these kinds of things but then they talk about updating models as if it's an easy thing to do and it's like well hey i got some center right and slightly maybe more alt-right things to update you on would you be willing to accept these things here's some information about kyle rittenhouse did you know this or what about this particular thing about this particular shooting which we were all very angry about but then when you look at some of the details like wait a second right so so updating one's models is not a trivial thing and and i want to this is where i want to be kind of empathetic to like we have a world where all of us can just hit a button like you type of you click a link literally hit a button and you're teleported to the edge of not just your own liminal boundary but maybe society's liminal boundary you know and um and you're like well what the hell do i do here i don't even know like trigonometry like how do i know if the moon landings are real you know so there's something about um gosh it was as you're talking there was a a thing i thought about which was the guy you remember the whole like the whatever the the pizzeria uh the the whole queue pizza gate pizza gate right and the guy literally showed up with a with a rifle with a semi-automatic rifle and what i wanted to sort of i wish i could like find a recording or something like what was the look on his face when he went there and like there was no basement because that's when the the reality comes and your models are all wrong.

1:16:27And you think about how deep of a stack of models are wrong there. Yeah, if you see the, what's the person, the pizzaiola, they are like making the pizza and they look fine and there are no Democrats there yet and they're all like Italians. You're like, okay, okay. There's a hidden trap door. Very, very good. Yeah, there's a trap door in the basement. Hillary Clinton's down to a little torture of small child. At some point, the inputs are just too strong and you can't. And this is the same thing I think some of the January 6th protesters, they were like, we're part of a huge, we're here to save the country and all this stuff.

1:16:54And then when you actually like when they're put on trial and everything, it's this whole collapse of sense-making and this cohering back to something of a consensus model of reality. And this is a question, right? Well, who's consensus? You actually talked about in a Medium post of yours, a blog, about this idea of colorful dystopia and virtuality. Can you explain how that feeds into it and what that is? Yeah, I guess it's related to this in a sense. So I use the word virtuality to just refer to the phenomena of it's us experiencing a world that is more and more manufactured, that is designed, in fact, to deliver a loaded payload into our senses.

1:17:38Maybe it's the best way to sort of encapsulate that concept that we inhabit the virtual. And this can be quite, you know, I mean, maybe the best example of this would be, or the most accessible one is when you watch The Matrix and the guy's like, well, I mean the steak and the steak tastes good to me, right? So many aspects of our world, when you learn to look for it, you realize, oh, wait, that's made up. It's Disney World, it's Vegas. It's designed to make me feel a particular way and it's just a virtual experience. Can you contrast it with a non-virtual experience? Yeah, a lot of, I mean, maybe the most accessible sort of non-virtual experiences are when people are out in nature or when people have a deep connection with each other absent a lot of the dressing, absent a lot of the, you know, all the accoutrement of what might tell us or of the social environment that tells us how we should think of each other.

1:18:32When you're not an avatar, I'm not an avatar. We're literally just people trying to have a conversation about something, like being real, right? And usually that involves actually exposing part of yourself, being vulnerable in some way, and being vulnerable not to being attacked, but to being, to updating your own models, to updating your heart, right? To actually being sensitive to something new you hadn't experienced or thought about before. And we get this when we go to the great outdoors. I don't know how much you guys like being outside, but like if you go and you go to a national park and you're in front of a mountain, the mountain is right there.

1:19:02And you feel the breeze, you might smell the grass and you hear things chirping and whatever else is happening, but you're there in it. And that presencing, that being present in that outdoors that is all there, or when you jump into the water and you're just in this thing that is definitely a full body experiencing presencing, we can do that in social environments as well, but less and less. And so virtuality is the opposite of that. It's a world, and I'm borrowing this term, there's many other philosophers and people, social psychologists, have written about this. Maybe can you contrast it a little bit more as well?

1:19:34Because I think you're pointing at a different thing than in that world where we're just hanging out and meeting each other, etc. that there's still cultural or memetic versions of we will value the things that we value due to the not some intrinsic thing, but because it was communicated to us through cultural memes. But you mean an even further step away, right? Yeah, I think maybe the thing about, the best way to talk about virtuality is the opposite of it, right? What is a way to experience non-virtual? And non-virtual experiences to me are when you are experiencing the substance of something or you're relating to something on a substantive basis based on your own intrinsic values.

1:20:17And you can update those values and you can update your models. And so much of the world, I feel like we have more and more things that are made and presented to us, ready made, ready digested for us to consume in a memetic sense or even in an informational sense. And this touches on something which I wish more people would appreciate. and we don't have to talk about right now, we can talk about later, but this idea that Gregory Bateson said this beautifully, like information is a verb. It is the act of sense making, not the sense that's been made. Because real sense making, the sense that's been made is sense to you.

1:20:50And the idea that even at a metaphysical level, when we say information as this artifact, that's this golden artifact free of context just floating here, you know, oh, if we can only get the Trump voters the information, then they would vote the right way. Like, no, that's actually the wrong way to think about all of this, because if you had gotten the information the right way, you wouldn't be telling me I was a conspiracy theorist for saying that there might be a lab leak, right? This information is not this golden. And scientists also know that science is a process for putting a bound to infinite error.

1:21:19It's not to mine some golden truth, right? But I think a lot of people who are on the outsides of that, who only know science as an institution that has produced the modern world, they can think of it as this abstract thing that could be brought down from a mountain like fire, isolated as an artifact, alienated artifact. And so this is the same thing with like, oh, I'm valuable or this is beautiful. Beauty is some abstract, some absolute thing. We would never say that. That's obviously, it's something that you experience in a kind of a relational way. So virtuality, I guess, a world of the virtual is a manufactured world.

1:21:54It's a world where we allow other agents to intermediate between our direct experience and relationing and sense-making. That raises this question of, is technology values neutral? Because a lot of people claim that, particularly technologists. That seems insane to me to be able to claim that now. I know, but they're like, I don't know, it's because they can wash their hands of responsibility or something but they're like oh you know it's it's not up to the technology the technology is not good or bad it's just how the people use it it's like well actually the medium often is the message and it's increasingly so you know we're in we are you someone goes to a view if you go to a beautiful vista these days um the majority of people will get their phone out and start recording it taking pictures of it for instagram for example 20 years ago that did not happen people would take it in maybe they would have a camera which they would then get out but the primary thing was absorb the mountain through your eyes now it's the mountain is a backdrop for this projection of this avatar of myself i put through it through social media and so that that existence of that technology is increasingly setting the tone of how we view the world so it seems absurd to say that it is values neutral i try to do my best to steel man you know arguments that are in opposition in mind so I can better refine my own point of view on things but I don't see how anyone in this day and age could argue that technology is value neutral I suppose the point here would be to say that maybe let's let's go over that uh the it's it's a technology that exists and who are we to say that it's a better or worse outcome that are now with the new phone as well as uh Instagram technology we have shifted away from experiencing to showing others what we have seen or something that the relative values that the mountain provided have shifted towards that.

1:23:51Why are we making a judgment in the first place about saying that, yeah, this is now worse and the technology therefore has worsened the value? Well, it's shifting the locus of attention away from the present to the future. Right, but the strong liberal kind of stance would be to say that... That doesn't matter. No, no, no. Yeah, exactly. That it doesn't matter or that it's in the end, you have to optimize for the freedom. Right, but the question is, are they making the choice? Because if you have grown up in a world where everyone, you're born now and you grew up in the world of Instagram, you've never seen the former.

1:24:29I think that leads us back to what we were talking about before with the market often finding a local optimum or an optimum at the time. And sometimes it's not even an optimum. It's just kind of driven into you what you want, even though for like some short-term benefits rather than... Yeah, and there's so much wrapped up in just this example of a person who goes and takes a picture of a beautiful mountain at some overlook. I mean, we build an overlook because we all sort of generally agree, this is a gorgeous place to look at these mountains, right? Hey, that's great. At least there's some consensus.

1:25:00It's beautiful view. You go and you do the beautiful view. You know, there's an interesting thing about the taking a picture thing because I like photography as well. And I certainly take a lot of photos in my camera, in my phone camera. And someone pointed out like, you know, part of the reason you do this is because it's your own life stream. You look back and say, well, this is my point of view that I had. And this was the shot. And maybe it's got, maybe it's got my kids bouncing in the corner or my wife is like, you know, hunched over digging like some snacks out of a backpack or something.

1:25:25But that's my moment. No one else can take a picture of that mountain with my family in it. Or it's my corner of my van. It was that first van trip we did with that van. You know, there's aspects of this that are a little bit personalized for me. But even if all those things were moved, the reason I take a picture is so that when I look through my curated stream, I'm like, here's what I did. And so there's the aspect of I'm collecting a memory to add to my better, to have a better memory for my own recollection. That's a different motivation than let me do my hair up and then get the selfie stick and pose just this way and do a little like Instagram thing.

1:26:00You could tell I'm not really doing any Instagram stuff because I'm like, you know, I can't do whatever the latest dance is. But that is a performative thing. That's letting the camera push my identity onto me. Or the audience specifically. It's my guess at the audience. Because if I could actually see how little the audience gives a flip about me, I wouldn't do any of it. This is the arbitrage. This is the crab trap that we were talking about earlier. This is actually what's evil and unethical about the way that an illiberal, it was a sort of an exploitation of a loophole in the liberal philosophy.

1:26:34because these things give users choices, but we nudge them, manipulate them a little bit, right? You see who's posted the most. You don't see who hasn't posted. You see how many likes there are, how many hearts. You get notification when there's likes. You don't get notification when someone really has engaged with this and thought about it a long time. What if you didn't get any, what if no like counted unless someone dwelled on your post for 15 seconds? What would that look like? See, the thing about the technologists who think technology is value neutral is I don't think people who really work in user-facing areas can really say that because the only trick, the one cool trick of most social media versus prior communication mechanisms is it was designed without a two field.

1:27:21It's a messaging system without saying two. Like when you write email, there's a two, two colon, who are you trying to talk to? Social media, all of them by definition are broadcast by default. In fact, it is sometimes not even possible to limit who you want to see this thing. In other places, it's possible, but it's a massive pain in the ass to configure it, right? So imagine going out into a town, you stumble upon a medieval town, and the only volume was 11, and literally everyone was just screaming at everyone else all the time. That's insane. That is not a human way to interact with anybody.

1:27:55That's like, whoa, why is everyone shouting literally from the pub down to the mill? Why is everyone shouting at each other? Well, we want to be, in case we have something funny to say, everyone will like what we said. You're crazy. Like, what are you doing? But that's the one trick. We went from email lists because there were chain emails and people trying to go viral with emails with like these massive forward chains. And they were kind of funny and there's some good stuff and ha ha. But then when we flipped to this broadcast by default mode, that technology user interface choice changed, fundamentally changed the nature of human communications.

1:28:31Imagine a car, imagine the next version of Tesla comes out and it only goes 65 miles per hour, minimum. How would we design roads if your car out of the garage, or maybe as soon as it comes out of your driveway, it could only go 65 miles an hour. Holy shit, like, wait, why can't we slow this down a little bit? You know, but no, they get compensated based on how many miles are traveled by the vehicle. and that's what we have for these communications technology companies and at a regulatory level we don't have policymakers who even have the language or the philosophy to back this up to say number one the joint attention of a society is a commons it must be defended as one of the most sacred artifacts number two communication technologies are part of like the infrastructure of a country the way that the nervous system is a deeply important part of a body you can lose toes, you can lose chunks of flesh.

1:29:20But when you lose nerves, you are like part of your spinal column. Holy crap, right? We need to defend our communications infrastructure. Every society needs to defend that. And it used to, it did used to, right? You couldn't just go put a pamphlet up. Like if you're Martin Luther, we know Martin Luther because he nailed a pamphlet up to the door, right? The ability to actually disseminate communication at scale was a sacred thing and was heavily policed because of the impact it would have on human minds. but now we have this thing of well laissez-faire everyone does what they want they say what they want people are making choices and who are you to say someone's making an invalid choice the only way to push back on sort of the the liberal sort of uh perspective on that is uh speaking about a commons or hey it's somehow hurting us together collectively so then this choice has to be mitigated in this way but even then people making choices young girls deciding to spend a lot of time on Instagram and having body dysmorphia issues.

1:30:14Is that a valid, like, we have to have some philosophical sort of scheme to actually talk about when people can, when we can build technologies that outrun people's choices, then are they really making that choice? A new kind of bad version of that that might happen is will the internet get polluted by a bunch of LLM text such that it can later not be necessarily used and noticed anymore what's real text versus what the generated just for getting your attention on the website text is, right? Like already, do we have any infrastructure in place to reduce that from happening? We don't, but I don't know if that's the framing of the problem that I would choose.

1:31:00because in a sense, Google and Google being an easy place as an index, like Google took over a search, like just completely dominated. I mean, I, and I remember the earliest internet, there are many choices for search. Google just cleaned house. And then because there's that one control point, it became a scarce resource. The listing on the top, not even the front page, but the top half of the front page of Google became an extremely scarce resource for society. And so when we talk about the internet, I think if you look at how people experience the internet today, there's a very, very, people experience it through apps.

1:31:34How many websites, de novo websites have you typed in? Maybe you guys type in a few more than the average person, but for the most part, you know, these aggregators, it started with things like dig and slash dot and Reddit, right? And they kind of came out behind, but then like this idea of the internet in the early days, it was not actually that anonymous. An email address was a thing. You had an identity. If you posted on Usenet, sure, there's some garbage posts and stuff, but like Usenet was a forum, a global forum for people to talk about all sorts of things. That's how I got to interface both Linux as well as the Python community was through these Usenet things.

1:32:08So people had identity. Now you could be anonymous if you wanted to, but for the most part, you would keep posting this place and people would know who you are. So speech mattered. But then it became the cesspool of anybody set up a webpage for whatever, hopefully the Google SEO engine indexes you and hopefully make a few bucks before your site's taken down. And that's not actually what a discourse of commons looks like where there's accountable speech, right? Yeah, I've heard, um, it was Daniel Schmachtenberger was talking about it, like with this idea of technology not being values neutral. So if you have, if you order your stack with the technology first, which then drives the, um, the sort of social structures, which then drives the memes, you end up with destruction.

1:32:53You end up, that's, that's Moloch basically. whereas if you flip that and you have the memes or the philosophies which then drive the social structures through which you then build the technologies in service of those that's the inverse that's like the win-win solution um so i'm curious what are there any like if you could start designing such that this stack from scratch what would be the kind of philosophies or memes you would want to imbue in it? I should have expected that question, and yet I don't have a ready-made answer for that. I think I'm most in agreement with Daniel on this kind of idea that we should use, we should harness technology in service of what we actually want to achieve and to recognize that the values come first, and we can build technology however we want to to suffice those values.

1:33:48in some cases we do do that but for the most part we are led by technologists building new kinds of you know shiny toys and we kind of just like oh well we'll move in here you know but but i would i think there's some aspects of what i don't have any ready-made solutions but i think there's some things we need to explore which is what does what is necessary given the modern technology have that we have what what does it mean to be a digital human to be a human with digital technologies that can enhance our memory enhance our perception um really if we were you know if we were all just billionaires who could just come you know have an army of software developers build us whatever or hardware developers as well build us whatever the perfect thing would be to help us be the most we that we are um what would that look like and if we start with that for each person and then say, okay, then the phase two of that question or part two of that question is, well, for us together as a group of friends or for us together as a family or for us as a community, what will we like?

1:34:51What will we want the technology to do for us? And I think you have to start with those kind of first principles because those really put, you know, I think a lot of times when people ask those questions, they come somewhat from a Luddite perspective, like let's block out the technology. We want a school where there's no tablets allowed. Well, you know what? My kids, for all the hate that I give, like a lot of the stuff, my kids learn a lot through a lot of the great content that's available online today it would be great for them to be able to engage with that content more it accelerates their learning but of course i don't just wire them up to a tablet and we'll you know set it and forget it so there is sort of this intentionality around it but i feel like a lot of the um yeah it's very much an either or thing right now where it's like well let's just do all the human stuff and then we'll figure out the tech on top of it and i think this should kind of work together um because otherwise what's going to happen is you're going to come with someone impoverished view of what the humans the the philosophy and then the socialization layers you have an impoverished view and the technology just completely blows right past all of it so you have to infuse the technology into each of these different things uh and then build that to sort of build those layers together or not not holistically you have to build those layers holistically with technology in it are there any sort of technologies that can that you think have potential to help people develop essentially better psychosecurity psychological security against manipulation um because we're in this sense-making crisis an attention crisis essentially where it's so hard to it's just like so chaotic therefore it's making people more vulnerable we need to bolster people's psychological defenses against manipulation i think that for some folks uh who are let's say older and they just find themselves like, you know, I don't want to be doom scrolling, but I'm just, what else am I going to do?

1:36:37Like I'm sort of in it. There are some apps that I see emerging where, I mean, you know, fight fire with fire, I suppose. But some of these apps that try to remind you like, hey, you spent this much time doing this thing or you can even tee up beforehand. How much time do you want to spend each day doing these things and how well are you tracking to that? Right, so simple things like that can nudge behavior because we are talking about psychological technology and behavior kinds of things. Obviously, there's little tricks of like, you know going to a monochrome screen black and white screen um setting hard rules for yourself like hey no screens in the bedroom so i'm just reading a book or i'm you know i i don't i'm not very good at that rule myself but like i can appreciate how wise of a choice that is we try and do the whole no phone in the bedroom i i call his phone that bitch because i'll sometimes turn over in the morning and he snuggled up with feel no that bitch is lying in between us i'm like oh she's in our bed again huh it's a rule that you agree to rather than we both agree to i didn't sign up for a three-way relationship there are technology tools like from apps and things like that for people who really do want to mitigate this but they just find themselves kind of sucked into the dopamine um cycle of that um i think more broadly well for younger people it's harder right because they have a little bit less control over this stuff their experience with all of these apps leads to many more connections that they impart their life is defined as meaningful as yes these things have really ingrained themselves as the substrate for their social identity like again as we go back to the kind of market mechanism which is not only a financial market but also like in life you're more likely to be in a local optimum that's harder to get out of if it has much more anymore yeah the trap is much larger it's sort of a tech cult it's like the same reason it's hard to leave a cult when all your friends are in the cult right it's hard to leave an app of all your friends and the jokes and the like hey people you actually genuinely like your way of getting to them is through this app.

1:38:26So for younger people, I think, well, and I'm a parent of both sort of a younger teenager and then a 10-year-old daughter who's just coming up into now the age where it's very dangerous for girls with technology. And I think about how important it is for us to have good connections with other parents, the parents of like her friends, and that we're all on the same page about how they as a friend group interact with technology. So this is to some extent a little bit of a worked example around collective action against Moloch. If only one parent is like, well, my kid doesn't use tech, well, guess what?

1:39:03First sleepover, your kid now addicted to really cool apps. They're going to come back and be like, I want that app on my phone. I want that app on my tablet, whatever. You actually have to get together as a group and say we're going to parent our kids in a way that actually encourages them to have experience with each other. They can be playing board games. They could be doing art together. They could play instruments. They could run outside, do sports, swim together, whatever. But we're going to really limit for their group activities. We're going to limit the screen time thing, right? And so I think that is the kind of way that for younger kids, before they get really sucked in.

1:39:34For older kids, once they're kind of sucked in, man, that's hard. That is really hard. Because you do want to give your kids, as they become teenagers, you want them to make good choices on their own. You want to respect their privacy and their agency. you also just there's a lot of other hazards you want to be able to police as a parent in a digital world um and there's really i would say not a lot of great solutions out there unfortunately um i think you have to sit down and talk to them have this level of conversation to say hey i'm sorry that you have for all the discussion about extended teenage years unfortunately you're also doing early onset um adulthood through a digital you know mechanism like there's just a lot you're going to see on the internet that you may not really understand that you're not really processing very well or other kinds of things tell us when you're having these kinds of interactions but i mean yeah it's hard because parents themselves get sucked into this right yeah you got to teach them about dopamine essentially and and regulation and it's very it's especially hard if i would imagine if they're at a phase of wanting to find out for themselves rather than listen to you about everything but yeah you without the internal motivation it's very hard to build a habit that they can defect off just all the time by themselves because they're constantly with the phone by themselves.

1:40:50So they have an internalized motivation. For those teenagers, though, I think it's part of value systems. If you can find who they look up to, some of the people that are, whether it's a pop icon, whether it's whatever, most famous people also kind of hate social media as much as they have to use social media. And so if you can curate them going off about some of this stuff, that might be a helpful thing to sort of talk about. uh i know like i in one of my blog posts i cited um you know fairly famous people just signing off twitter because it's like whether say what yeah cardi b he was like you're trying to make me into some disney person i'm not some disney star like f you like i'm off this thing um yeah there's others as well right but who are the dark side of fame being micro sliced like this and pushed on people you do have people who are who talk about what what dangers and what bad stuff this for them.

1:41:34So you might be able to find whether it's NBA stars, or as other people talking about, look, this creates a whole dynamic. The social media creates a dynamic, and I don't want any part of that dynamic. It's just sort of like, it's so over, you know, or something like that. That might be a way to connect to a more teenage kind of crowd. I don't think we've talked about how like society like grows or like innovation is improved if like you publish and you talk about stuff and people see things, etc. etc we think we're okay if we're bad at science right now but actually what if we're not what if we like have to really get our shit together in the next 50 years before the alien invasion fleet shows up another omemura whatever like do you mean agi uh no not even agi just all yeah right which is kind of yeah agi could be the the thing that actually reaches out on the em and then gets influenced maybe our agi is actually really bad it's like the right flyer and then aliens have really much better agi in the ether that then hits our shit what do we have to really get our stuff together on science before we make first con before our agis make first contact so would do you think having fully open science would facilitate that i think if we actually you know did scientific collaboration at global scale as if it mattered right now all we can do is think about when lose scarcity who makes a better ev blah blah blah some other like happy horse shit but it's like no no actually what if it's an existential thing for humanity to get really good at science like now right because part of dark forest is when you first open your eyes and realize the dark forest and you're like wait that means everything else is hiding what are the hiding from oh shit i gotta get smart really fast right there's some of that and then the decline of the west being you know we think we've peaked well the first moment you think you've peaked you've peaked right the fact that we don't see ourselves in competition oh no putin you know invaded ukraine that's all bad and everything but like your competitor isn't putin your competitor is some like alien asshole thousand times worse what about that these are the kinds of things that we should be challenging ourselves with but we don't because the west is like well everyone gets you know whatever a chicken in every pot you know a iphone in every hand like maybe you're good and we're not good right well what is your epistemic status on aliens um i'm pretty sure there's aliens out there i think it's quite naive to think that we're alone i mean now i have no evidence for it so i'm not going to say that i think people who think we're alone are idiots i think they have every reason to rationally believe we're also alone but my feeling is that basically you know how long have we known about neutrons and electrons well neutrons about 100 years you know up until 1929 was it we thought the milky way we thought that galaxies were spiral nebula like we don't know anything about anything we don't have a working theory of gravity right now why do galaxies not fly apart oh dark matter okay no seriously do we have a working theory of gravity we do not so like if we don't have a working theory of gravity how do we think we can even see anything right like we don't we don't really know anything about the cosmos out there we've only known about the milky way for about 100 years so i think we have a long ways to go before we can sort of declare whether or even our local neighborhood has stuff in it we look in a very narrow slice of the spectrum we have i mean we have a telescope now finally um kind of far away from the earth It's a nice big scope, but you know, it's really small.

1:44:48We know James Webb. Yeah, it's really small compared to like the orbit of the Earth, right? Around the sun. We could totally set up more advanced instruments that have much longer baselines, that have incredible aperture. What could we see then? James Webb just imaged a protoplanetary disk. Beautiful, beautiful, but it's very blurry. If we actually had long baseline, you know, advanced instruments. What do you mean by long baseline? So basically, if you are able to have multiple instruments, then you can actually do interferometry between them. And so you have much sort of a synthetic aperture.

1:45:22I see. Kind of like with the, what is it, the large array radio telescopes. Yes, the VLA or the square, well, not square kilometer array, but the very large baseline, whatever. Right, because you have one on this side of the Earth and one on half the Earth away. And they effectively work as one giant radio telescope. Exactly, exactly. If we were to do multiple optical telescopes, you know, I mean, having interferometry across multiple telescopes in space is very hard. But for radio, it's easier. But anyway, those are the kinds of things we can't even begin to do right now because we don't have the tech, right?

1:45:52Or just too expensive to do. But I have a fun little thing because you're obviously you're very knowledgeable about astronomy. But just for people to have a sense of how small we are and how little we've seen. If the Milky Way were the size of the United States, continental United States, coast to coast, right? California to Maine, something like that. If the Milky Way were that big, how big is the solar system? How big across the solar system? Milky Way is a solar... It's less than... It's less than one in a hundred thousand of it. It's like one ten millionth or something, I would guess.

1:46:30The size of the Milky Way relative to the solar system. I think the solar system is less than one ten millionth. Like, is it like one hundred millionth? Depending on what you define as the size of the solar system, the orbit of pluto is about one and a half inches in diameter so if you go from the set so so one orbit of like the edge of the orbit of pluto to the sun and then the same distance the other side are our um our solar system now to pluto at least is uh an inch and a half out of the entire united states that's one galaxy right um if you go out to like the heliopause or something maybe goes out to like seven inches or something but it's not it's not very big compared to the united States.

1:47:08One of my favorite analogies is if you shrink the sun down to the size of a tennis ball here in Austin, the next nearest star, our nearest neighbor, Alpha Centauri, a mere four light years away would be another tennis ball in Los Angeles. Right, right. That's fucked up. It's fucked up. And the Milky Way is 25 ,000 times that. Yes, that's right. Back to the Milky Way being the size of the United States. If our solar system is an inch and a half in diameter, Alpha Centauri is two football fields away, right? The sun is about one and a half, one and a half like billionth or trillionth of an inch.

1:47:47It's, I mean, it's beyond microscopic, right? That's how small we are. We can't see out more than a few hundred parsecs in terms of parallax distances. Beyond that, it is all using astrophysics and galactic ladder scales, things like that. to even see how far things are apart. But that's just our little star. And we're kind of not in a, we're on the spur. So we're not really in one of the dense arms. And if you think about the kind of energies and the kinds of different kinds of chemistries and electromagnetics that are available inside that kind of environment, like we, again, we are living in a very cold, small little place right now, looking out at a very big world.

1:48:26The idea is though that maybe we would have been visited, right? And then like we would have noticed that with aliens or we when looking out should identify a certain set of signatures and some of those are for example if we look at another planet like mars which has a lower habitability rating by quite a bit i suppose than earth but presently at least but if we look at a high habitability rating planet and we don't see any life there and then we look at another thousand and we don't see any life there then it's like okay now it started but we actually haven't looked at that many in so much detail right right and we don't have that much we don't know how to look for the evidence of civilization right or what we consider you can sort of say well that's potentially could give rise to life yeah what what are the types of signatures we're actually looking at to identify whether yeah something has uh life there is it that um what was it oh yeah uh the amount of energy that goes onto the planet is less of it is reflected than you would expect or yeah you could you could look for things like differences in albedo and things like that but i would say that one of the things that people do look at is like in the chemistry right so if you look at certain kinds that you would expect for an atmosphere of this kind of composition and getting this much energy you expect certain kinds of processes might be available and there's certain kinds of maybe maybe there's some kinds of chemical things that we could see that are only produced like short lived kind of things that are only actively produced by an active life process there's things like that that you look at in the chemical signature but actually looking for structures optically or maybe maybe you look in the radio frequency but i think that's really hard right that's just really really hard even we are a nuclear capable civilization and for us to put out a strong signal out to you know even 50 light years away i think would take quite a bit of doing just because radio waves attenuate and get yeah and it's like against the background and just all this kind of stuff the sheer number of stars and possible planets to look at is unbelievable when you guys come out observing with me next time i'll show you with my night vision when you look up at the stars and that's in the local neighborhood of a few thousand light years i mean that's that's all you can see the sky is just completely messy with stars so i i think what we have to look at is even imagining beyond for alien kind of right back on the topic of do aliens exist i think you know, in my flights of fancy, I think about, could there be sustainable energistic patterns, like in a place like the surface of a star, even, you know, could you actually have something that we would consider semi-stable patterns that give rise to other kinds of higher dynamical patterns?

1:50:58Like, could those exist at the energy scales of what we find available in a place like the sun, maybe inside the sun, even? I don't know, but that would be an interesting thing to model, right? Because the sheer amount of energy available in that kind of environment just makes this place look like the freezer i mean we're basically we're epsilon above pluto in terms of the energy available on earth to do anything so i i sometimes it's like in my flights of fancy thinking about the sun really this the stars just singing to each other right just beaming out with music and energy as these solar civilizations are actually what matter in the universe a friend of ours bill perkins uh who was a previous guest uh he has a theory that um stars are actually giving it's like a programming language coming out of them that they're the infant they are they are the programming that then hits hits the earth and in certain ways and um by the great simulator basically they're used as the distributors of the updates an interesting thing people don't talk about very much is there's a strong correlation between the solar sunspot cycles and life expectancy at birth.

1:52:04Do you know about this? If you were to chart life expectancy for humans and you'd also chart the 11-year sort of sunspot cycle, there's a direct correlation. It's like a lot. It's not - In what direction? So the year you were born, basically when you were gestating in the womb, depending on if there was a lot or fewer sunspots, I think it's directly - So if you're like born at a peak of a solar cycle when there's a lot of - Yeah, something like - Is that better or worse for your health? I forget now, but there is a serious - matters yeah yeah well you can look it up i mean you can't do anything about it now right well no sure but well yeah but we might we might be having children soon we're in the you know we're in yes i remember it's pretty hectic when i think when there's less sunspot activity is when you have a longer uh life cycle expectancy but but you don't don't take my word for it you have to look it up but there is this the periodicity is amazing across the whole globe so there's definitely a thing there so i don't know if it's a if it's a simulator like programming us or whatever or We're programming the simulation, but there is definitely a lot of stuff going on that we don't really talk about very much in terms of the energy and the patterns of energy coming from the sun.

1:53:06We're really kind of early in our understanding, I think, of solar processes, because again, we've only had, we've only known about neutrons for a hundred years. So like, come on, you know, like this is, we're really kind of new to all of this, right? At the same time, it, it feels like physics, you know, went through it some real glory days in the early 20th century and you know we had quantum mechanics we had relativity etc all these big breakthroughs and then things have really stagnated uh would you agree with that and if so what what's what's the main driver i think yeah we're not getting we're not getting the kinds of revolutions that we were once getting i think physics um i think in the areas of applied physics, we're getting some really amazing and interesting things happening.

1:53:51It sort of diffuses into these other areas, right? So in areas of chemistry, material sciences, and I think as we get better and better at computer simulations and with some sort of approaches of applying AI to some of these kinds of things, we'll get really interesting stuff coming out. I mean, I think about all the physicists that graduated with me in my year, I think maybe only one is still doing physics. Like everyone went to different fields, someone to software, someone to finance, others and biophysics, other kinds of areas. So if I were to, I hope I don't make any enemies with this, but I feel like when physics became sort of, I don't know, I feel like there's a lot of, physics has to be very experimental science and you have to be able to ask big, bold questions and you have to come up with really shockingly new models.

1:54:42Like physics is just one level above metaphysics. you know there's not much else between that and math and that's and then it's just in the world of pure theory and i think what happened in the 20th century is we've built off the successes the successes of the of the theoretical physics in uh nuclear in the nuclear physics area right so the approaches of pen and pencil and sort of being a theorist as a full-time job became part of the institution of physics and then physics became sort of like you know it's i don't know there's there's a lot of we can get into holo malik and institutional decay and institutional capture and all that kind of thing i think there's some of that going on i mean eric weinstein there's nothing i can say that eric weinstein has already said about this particular topic not to say i particularly buy in his theory of um geometric unity or any of that but just to say the critique of um the institutionalization and the theoretic sort of the theoretization of modern physics it sort of becomes this vortex right and so a lot of the people i know who you know in the proto stages if you go into academia you sort of have to go down these paths um but we make breakthroughs by asking weird off-the-wall questions and actually supporting people doing those kinds of explorations so i think someday you know assuming i you know make my millions i would like to fund some of that stuff myself and just put money behind people doing really interesting weird work in the edges of experiment and and uh and all that so so obviously there's a big debate playing out right now about whether or not AI development should be open sourced.

1:56:16I mean, I don't think the answer is binary. It's probably some kind of spectrum. And in some cases, it's good to do. And in other cases, it's probably not. But I'm really curious to hear your thoughts on... As an expert in open source software, do the same arguments apply to AI? Yeah, and I would say we should sort of look at the fundamentals of what makes open source software a good thing or a powerful thing, and what are some of the ways that it ceases to then manifest those values when certain other kind of dynamics come into play. And in the realm of AI in particular, though current implementations, let's just make sure we sort of put some context around this.

1:57:00When we have these big models being built around primarily transformers, LLMs, run on lots of GPUs by these big companies with lots of funding, that particular model of AI is different than software in that the artifact itself is a combination of, well, freely available source code. Most of the source code is well-known. Sometimes it's readily available. Well, other times the algorithms are pretty well known. Anyone can sort of implement one. And then training data. And so you run this code on a lot of what's called training data on a large number of machines. And so then you have to have a lot of infrastructure to coordinate those machines.

1:57:37You put some parameters into the training run and out comes this artifact, which is what's called a model. Really, when we call a model, those are the model weights. And that's what takes$10,$100 million to generate is those weights from all this training data. So there's multiple different pieces we could talk about when we talk about open and we talk about source. Okay? So it's no longer just here's a recipe, here's a cake. There's a lot more involved here, right? So is the source code itself available and open? In some cases, yes. Some cases, no. And then if that's yes, then what about the training data?

1:58:09Is the training data available? Some cases, yes. Most of the time, no. Okay? Are the different scripts and the parameters you use to invoke the scripts run for this long on these things and all that stuff. Is that available? Most of the time, no. Sometimes, yes. And sometimes it doesn't make a difference. A naive script could still actually probably train something successfully. And then at the end of the day, the weights that come out, which is literally just a giant multidimensional matrix of numbers, is that freely available? Most of the times, no. But in some cases, yes. So you actually have this multiple different aspects.

1:58:44So it's no longer just, can I look at a text file that has source code and can I use commonly available compilers to then turn that into an executable that I can run? That's the old school open source. New AI, when we talk about open source AI, there's almost no effective, I think there's no effective, you know, sort of leading or state of the art kind of foundation model that's actually open source by that metric. Because in the most, they will make some of the weights available, but they won't make the training data available. So it's free to use and you can look at the weights but good luck making sense of the weights it's a pile of numbers you don't know what training data they used it's probably then helpful as well to uh point at let's start with it like what were the initial values and benefits that um society and individuals received on the basis of having open source software rather than closed source software yes and then like looking at um yeah how can we get those and do we need actually all of these components or maybe not right right um In a sense, it's sort of like, yes, the value to society was that many people could make software, but they were sort of gatekept by the cost of getting the tools to make software.

1:59:56Compilers or SDKs, software development kits, right? Things like that. As more and more of the world, as more of the open source community made these tools, then people could then make their own. It's sort of like, sorry, this gets a little complex, but if you want to have carpentry, before you can have a really good carpenter's workshop, you have to actually have jigs and you have to have lathes and saws that are accurate and all these things. So the very first projects that were done in the open source world were things like this. The compiler itself was open source, the GNU compiler, GCC, GNU C compiler, and some of these basic tools.

2:00:33So then once you had an open source factory floor with all these tools, then you can make other open source things and everyone could always build on the open and improve it. And that's the open source software world. So the value, the benefit to society there was that many people could come in and improve things. And then as an individual, when you buy a computer, you could then be empowered and enabled to do whatever you want on your computer. It's like if you buy a car, you could drive it everywhere. Now, of course, that's our expectation as the buyers of cars, because we know how to drive.

2:01:03But you could imagine a world where, oh, you know what? you only lease the car from the dealership. And the car, you know, the basic subscription only lets you drive to certain grocery stores or certain restaurants. And if you want the premium subscription, that lets you then drive to certain other restaurants. You know, that'd be kind of like a crappy way to have a car. That would suck, right? But if the roads themselves were all toll and you could only, your car was only licensed to drive certain roads, you know, that's sort of the way that the world has kind of closed over on software because to use certain APIs and servers, you have to then pay a subscription or to do, you know, so there's a sort of a rent capture network value extraction way that the world has evolved to even on the basis of an open source, open development software ecosystem, because the software is no longer where all the value is.

2:01:51It used to be that very few people could write software. And so the few people who could do it in the open were adding a lot of value to everyone. Once everyone could sort of write software or once software ceased to be the valuable place to capture economic value, it was the user's eyeballs. So the source code for all of the source codes for things like Instagram or Reddit are not really available, the source code isn't the value. It was the network of users. It's all the content that was uploaded. The value was in the content and the connection to the phone and everyone having that on their home screen and using it all the time out of habit.

2:02:21So the source code and the software, even though it started as a way of enabling user freedom, the users walk themselves right back into the crap trap. And now they don't really care. They just don't care. You can actually run tools to extract all your data from Facebook. Roughly 0 % of Facebook users ever do that, right? Because they don't care. They're using Facebook. And so I think with AI, the reason that dovetails in open source AI kind of stuff is all the training data that goes into making these LLMs effective, a lot of that is scraped through the open internet. Some of that is proprietary data that the companies pay for to get access to.

2:02:52But the data, yeah, one of them has paid. Yeah, but that data is actually where the value is because actually what produces the valuable AI model in the LLM weights, it's really just a reflection. It's a condensation, a compression, and sort of a structuring and feature extraction of what's in the training data set. So if you have crap data, you get a crap model. If you have really high-quality data, you get a high-quality model. And this is an important thing for everyone to understand, that even though AI is made using software, the value in AI mostly comes from the training data going into it.

2:03:27And most of these companies, even though they produce free models that are available, they're not really open source because they're not giving you that training data. Well, in some cases, I imagine they also then can't give you the training data if it was the case that they licensed data from the New York Times because they may hate to deal with the New York Times. They can't then afterwards post the New York Times data, right, just publicly. And also there's legal liability if they admit to using some of the scraped data. Yeah, exactly. But then also, yeah, how you process the data is also very important, right?

2:03:59Like if it starts with kind of like just open internet, a lot of crap data, and then you kind of want to bring it down to a better data set that you're actually going to use to train the model, that's a very relevant effect. And I think there's probably some secret sauce that they also don't want to give up in some cases. And I think like with open source AI, they're actually currently still debating and figuring it out and run workshops to trying to understand what should be the standard that will be defined. Right, right. So I have been in dialogue with OSI on some of the stuff and in some of those workshops.

2:04:30And the unfortunate thing is that, yeah, when you really look at it, most of the models don't comply with all of those openness definitions. And they have a little bit of, I think, and I hope I'm not speaking out of turn, but I think there's a little bit of a political problem in the sense of if you come out with a definition of open source and literally none of the models out there can fall under that definition, it's sort of like, well, do you want to be that guy? Say, hey, all of y 'all are actually not doing this. It would be helpful if there was at least one model that did all of these things.

2:04:57And so anyway, I think there's tremendous value to society if we could build one that was fully transparent and all these things were available to show, well, here's what a totally available, all these things available, all these things transparent kind of model could look like. Like, maybe the performance is not as good as one of the vendor-produced, somewhat open or freely available models. But it's important for them because otherwise they would suffer the critique, I think, of like, well, you're creating an impossible definition. No one can meet this bar. But then again, that's why I wonder about which benefits do we hope for openness to put out, give to society, right?

2:05:30So maybe it doesn't matter that in a puristic sense it doesn't apply if it still has the benefits. if, say, Lama still ended up fostering a community of tons of people that were able to contribute and develop their own AI projects and just help jumpstart a bunch of developers to look into it, which it did, right? Right. And so this, I think, gets to the crux of it, which is that the openness is more of a consequence and not, I think, the core value. The core value was about freedom enablement, was about this idea that individuals, we have a technology so powerful. A personal computer is an incredible piece of technology, man.

2:06:14Holy crap. I mean, your average, even a five-year-old laptop is an unbelievable piece of technology. And we have, rather than leaning into educating and leveling up a society of individuals who know how to harness that to the maximum of its potential to help themselves, we instead end up with basically a captured society of just use they're just users um and and so how does this reply uh apply to open source and and you know openness of ai um the again coming back to to this idea that open source emerged as an enablement argument right that individuals who buy as a user i buy a computer just like i buy a car i should be able to drive it anywhere right it's like hacking the map system of that captured car to say no screw you i bought the car i get to drive wherever and i don't care what the dealership people say or the lawyers say i'm going to drive this car because i bought it i bought a computer i get to use it for whatever there were a lot of arguments in the early internet days like in the late late 90s early 20 early 2000s a lot of arguments against drm digital rights management and the people who are for it against it like streaming video mp3s all these things were technical capabilities that existed on the machines and then the copyright holders really came down hard on this and they got in sort of in cahoots right with the um the the telco providers and isps to make sure that hey you know what you can use your computer for anything but not for pirating content because we've got to make sure we pay hollywood and got to make sure we pay whatever so i think with ai it's the same kind of thing that if it's about enablement if it's ultimately about i buy a computer it's got 32 gigs of ram it's got high-end hardware to be able to do all sorts of machine learning I should be able to use it to do whatever I want, right?

2:07:55And if I can do that, then great. But if I can't, then why not? So for instance, can I use machine learning on my machine to automatically navigate through my Facebook and extract the most meaningful posts that I might care about and offer me a new interface, right? Because all these web apps, all these Web 2.0, software as a service, B2C consumer apps, they all are based on renting eyeballs. Okay. But if I use an LLM, whether it's copilot technology from Microsoft or the browser, if I use these things to actually scrape all of these different apps and present to me a really cool, wonderfully human attention serving, completely stripped of dopamine addicters kind of interface to my information environment, that's a very humane technology.

2:08:42And you know what? People like me and a million other people like me want to build that kind of technology. Now, here's the question. If I use an open model from a company who makes its money, literally billions and billions of dollars every month from renting out those eyeballs, how long do you think I will be able to continue using that model for that purpose? I mean, Lama has a restriction on the use by saying that, I mean, it's pretty high, but they say if over 700 million users start using something that is based on your work with Lama, then at that point, Facebook Meta gets to choose whether they license it to you or not.

2:09:24But I'll bet you by the time that you get even 10 million Western, because Western users of Facebook, Instagram are worth a lot of money, right? By the time you get even 10 million of those users to change their browsing behaviors as a result of this, you're going to have a different conversation yeah i mean presently they're just trying to block google exactly yeah the ones that already have the distribution yeah and this by the way this is nothing against any of the people who work at meta or the facebook team or anybody who's wearing these things like they're just doing their jobs too i get it right and i have good friends who work there and i the work by the pie torch team is fantastic there's a lot of good people there doing good things for good reasons but i'm just sort of saying like we exist in a capitalist world moloch puts its tendrils everywhere.

2:10:06And if we actually use these kinds of open technologies to cut off one of the strongest growth roots, one of the tap roots of Moloch is network extraction of attention economy. If we chop that thing off with LLMs and AI, you're going to have a lot of people coming down on what you're building. Like just realistically speaking, you have to model that kind of response, right? So you said that one core value that open source tried to enable was enablement of people and freedom that come with it. I thought there was also a large part that we touched on before around collaboration and this collective work together.

2:10:45Is that just a means to enable the empowerment or is there something else more about this collaborative way of building a community together that then ends up contributing to a project and edits and modifies it constantly? Yeah, I think it doesn't have to be either or. right? So many great projects are just one, maybe two people building something very interesting. Actually, Clay Shirky wrote a book about crowdsourcing, right? And he's very well known thinking about crowdsourcing. And he talks about all sorts of crowdsourcing, starting with step one being me first collaboration. So you're actually building something for yourself.

2:11:21So you are the best end user product testing person for your own need. And when you use something like the internet to connect to other users who have a similar, or other people who have the same need, now you've built a great product for all of them, And so I think there's something about the network effect of being able to find lots of other people who have the same needs you have, who then can adopt what you've built. And then they can make incremental improvements. So that network effect is also real and makes it valuable for you to give and make your thing open rather than just having it be close to yourself.

2:11:48But you don't have to do that in order to be effective at building a thing for yourself. On the topic of open source AI, one point that's often also raised is that, well, if we look back at the past, then historically open source has also contributed to making systems more secure in some instances and like intelligence agencies partially run i think some of their tech stack on open source software and their point then is if you care about ai security ai safety then you should also have it open source because open source historically led to it the thing that i then as i thought about open source more and i wonder if you agree is that it seems to me that the benefit of open source software security improvement was from noticing vulnerabilities and reducing those so improving defenses basically whereas the thing when we worry about ai safety is more about the ability of the ai to attack or someone else to use it to attack someone so it seems like a category error actually to apply the same thing because it is not the case that if someone puts out, sorry to use the example, but like Lama 5 and Lama 5 has incredible virology or like pandemic making abilities and we notice that and now like, oh, let's patch that by fine-tuning it.

2:13:19It's like, okay, cool. You fine-tune it and the next version that meta releases is now fine-tuned and doesn't have that ability anymore if you were able to do that. But that attack vector now exists because someone downloaded probably the LAMA 5 pre-fine-tuned version or could even fine-tune it away. So you're not democratizing security. With OSS, you're sort of democratizing the ability to add security. But with AI, you're democratizing the ability to attack. Well, at least that's, there's also this other factor where using AI models as part of your tech stack will also create some vulnerabilities and others noticing that will like make you able to reduce them.

2:14:04But the thing that often I think people in the kind of like, hey, there are risks around AI worry about is more its ability to be used as an attack vector. And I think that I don't see how open source can actually help that very much by putting out a piece of software, in this case, like a model that has this attack. Now everyone can have access to it. Yeah, I don't see how security is improved. Many different intersecting concerns here, right, in this topic. But you're absolutely right in that when people talk about AI safety, they're generally talking about the use of even a proper or a non-specifically designed to be harmful AI model.

2:14:44Like if someone with the best intentions builds an AI model, Could you still, does it have latent information in it that could be abused for some purpose? So I think a lot of the AI safety is around that. But also, could you embed, could you sneak some side channel thing in there so it does something nasty down the road? That is the topic of AI safety, I think, is those kinds of things. And that's a very different set of concerns than when we typically talk about software security with open source. And this comes to a very crucial distinction between these things, which is that with software, ultimately when you have source code, we talk about open source a lot, but really we're just talking about the word open.

2:15:20We're not talking about all the connotations of the word source. Why is source such an important term in there? It's important because the transformation from source to binary is a well-defined procedure that we use a compiler, we use an interpreter, whatever we use. we can put tests on it. It is a well-defined and well-characterized transformation. So we care about, but it's a one-way transformation, right? Reverse compilation kind of works, but oftentimes you really can't tell what's going to happen. So we care about that one-way transformation, but it's still a well-defined transformation. So if you can see the source, you can be somewhat guaranteed that the binary that comes out is going to be well-characterized.

2:16:00So in the software realm, and this is the term to key in on, correctness is actually a sane and reasonable and coherent concept. With LLMs, that goes out the window. You can do all the correct things, but if your model has something in there about people putting glue on pizza, like your training data has that, it has like glue on pizza as a thing, your model is going to tell people to put glue on pizza, and that's just what you got, right? And even more directly than that, because these models, they're not, I want to go on a limb here, they're not intelligent in the way we typically think of intelligence and when we hit them with a prompt and they emit some string of tokens that sequence of tokens is um it's it's sort of very much in response to the prompt so like if you fine-tune this is the gemini debacle right when they first released gemini the image generator and they had a fine they had a system prompt in there to say hey be diverse when you generate pictures they had like generate pictures of black nazis right like and it was like holy crap i didn't you know I didn't want that, but that's literally what this model does because we're not really good at operating these models yet because we don't really know quite what the LLM model constitutes.

2:17:10We have some ideas now, but I think the fact that correctness is underdefined for LLMs is where a lot of the AI safety conversations and all this kind of stuff, social impact conversations, they kind of end up like spiraling around each other because of that. But that's a completely different universe than software correctness. And so we talk about the value of open source software in making certain kinds of vulnerabilities more transparent, easier to find and track down. That's all correct over there. That same stuff does also apply to the AI models, but the AI models have another dimension of other stuff beyond that, right?

2:17:45Why do you think it is that so many open source advocates are being so almost like zealotous, like religious about the same, that all AI development should be open sourced as well? Well, I may not be one of those zealots, but I'm in strong agreement that as a world, because I believe so much that we're on the brink of creating technologies that really can augment human cognition, I fundamentally believe in open access to that for everyone. If it doesn't take, you know, a civilization level effort to build just one, if it actually can be a non-scarce resource that everyone can run, which I believe it can be, then we should give many people that.

2:18:30If we give it to every school child, wow, could it accelerate their learning? If we give it to every individual and we actually have a well-designed system to help them do sense-making, holy crap, that would be a civilizational upgrade like we couldn't even imagine, right? So there's aspects of this that is, for me, a deep aspect of really fundamentally liberalism. Human freedom doesn't mean much if you've chopped off a part of the human's brain so they limit what they imagine freedom to be. But likewise, if there's the ability to put on an extra brain onto a person so they can even an even better version, even better vision of their potential, we should totally do that.

2:19:04That's like the most humane thing we could possibly advocate for doing. And I think the current incarnation, the current conception of LLMs is these giant data centers and Jensen gets us cut and it's like just a billion dollar things. I think that's completely bogus. It's completely in the wrong direction. we're going to be able to build more effective models, architectures. We're going to have cleaner training data sets that lead to extremely performant, very narrow models. We're going to build new architectures sampling from lots of small models. We have explicit chain of thought kinds of things.

2:19:32We'll have more observability, more explainability. I think that's where we're headed with all this stuff. And I think all of this will be able to run on an average home computer. And so with that vision, that's where a lot of us open source people are leaning towards is that There's no reason. This is one of the biggest anti-Moloch. This just chops his legs off because so much of Moloch emerges now from capitalism saying, I need exponentials to extract. Oh my gosh, web centralized client server architectures. I can extract exponentials from here. That has destroyed a generation or two of people's sense-making and whatnot.

2:20:05In psychology, we can go and just cut all of that off by actually empowering people to have their own tools for this. So it's very much aligned with the core ethos of open source. We need to build the defenses. Once we have built the defenses, then I'm more happy to distribute the attack vector to everyone because now they're not that effective anymore, right? So I think time actually buys us something around some things. So for example, if someone was to develop a, yeah, again, like very, like built on like a large virology related training set and pandemic capable, with like novel various pathogens that it could like design and then maybe order on a DNA synthesis printer somewhere.

2:20:48Having the capability distributed immediately versus knowing that this capability exists and now kind of, oh shit, we need to act against like to build the defenses because we know in three, five, 10 years, everyone will have it. I feel like those three, five, 10 years really might matter for some of these kind of large scale. defenses. Yeah, large-scale attacks. Taking a civilizational view on this conversation, I agree with you that there's a lot of polemics and hyperbole being thrown from different extrema. There are real dangers if you actually just give everyone the ability to figure out how to do certain kinds of things that could have a large-scale negative impact.

2:21:30And the installation against that is not to gate the knowledge necessarily, but to ask how do we design society so that people don't actually want to do that? And this gets to kind of where Western liberalism gets us to a certain point in terms of fighting against feudalism, fighting against tyranny in the basic ways. But how do we build a scaffold to support people in desiring to build positive things on the way up? That's different than trying to dig ourselves out of a hole. And maybe that's kind of an abstract metaphor. But my point is that if you think about what some of the people did with WhatsApp, okay, So they had like, this is in, I think in India, you know, there was like this genocide thing that was happening there, right?

2:22:11I guess the Rohingya. And basically they, they have like, there's one instance where someone just had a backpack full of like phones, flip phones preloaded with like WhatsApp or whatever. And they are, or maybe they're not flip, but they were very simple phones. They dropped a backpack of this stuff off in like a remote village where they'd never seen this kind of technology before. And people were like, oh, this is great. Cause we can like message each other. Oh my God, this is like alien technology to us. And then messages started coming down to these WhatsApp groups about, hey, there's a village over there, you know, where they're like, they just like beat up and raped some, you know, Indian woman, like you need to go take out those people.

2:22:44And like the ability to, when you have that level of technology, and it was not a very high level of technology compared to what we have, and you drop it on a society or microculture, micro group that doesn't have the ability to absorb that technology, then it is literally text messages from God. You know, why would they, I mean, why would they question it? And so just with text messages and WhatsApp, You can actually incite genocide. Okay. So should we ban text messages on WhatsApp? Well, no. We actually, in most places where people have WhatsApp, they also have some understanding of what the technology is and isn't, right?

2:23:16So I think the similar kind of thing with this, which is, and this is the civilizational point, maybe for us to really get to the next level, we need to have a civilization of people who are really good at using AI technologies and augmenting themselves and building better versions of themselves. To do that, we actually need to have a cultural and political sort of social infrastructure that supports people in that. And if all we ever do is we say, well, we start with a building block of a person being possibly just a sociopathic simian, and we can't give them an LLM because they will then just either kill everything or rape everything and be done with it.

2:23:52Well, yeah, you're right. We just couldn't then. That would be very dangerous. But what if the actual skill test for us here now is not just building an LLM or building an AGI, but being a civilization that can equip lots of people with augmented intelligence like this? And the governance hurdle is actually the hurdle, not the technological one, right? So I think it's worth us. What do you mean the governance hurdle? The governance hurdle. How do we keep from killing each other with this stuff? How do we build a society that is able to have a culture that is able to find people with the beginnings of sociopathy and or support people's development so they don't become sociopaths and psychopaths?

2:24:30How do we do this in a way that doesn't require a homogeneous culture for 100 million people, but actually support small groups and tribes where people can take care of each other, build real relationships and build real sort of like real culture coming from this kind of thing? it's almost sort of like we have to get past that industrial mode of relating we have to get past the the capitalist mode of production um and commoditization and alienation need to be actively thinking now collectively about how to actually protect some of these kind of infrastructure things um it's one thing to have like you know custom printed viruses and pathogens um in which case well you know we know where those labs are we kind of know what some of that like the people who do that kind of work people sell the precursors all those kinds of things like you you guys need to get your you need to level up your act right and then on things like critical infrastructure you know one of my friends was like you know what i was given a task of spending a hundred thousand dollars to take down the entire eastern seaboard's power grid and i could do it or maybe it's quarter million dollars but we let's make let's actually take that as a real task to defend our infrastructure so that some random smart dude with a quarter million dollars could actually take down the entire eastern seaboard's power grid right because we need to do that anyway right so these are the kinds of things that it's not actually about ai it's just ai being a forcing function for we need to get serious about this other stuff um and and maybe that's again a thing of like well wouldn't it be easier if we just stopped the ai from being open source in the first place right rather than that and or at least matching the timelines of securing these things which i can appreciate but here's the problem we don't have a choice because our adversaries will be more than happy to release models that have this information in them so we kind of don't have a choice to whether we do this or not.

2:26:09And so, you know, I feel like that's the hard conversation that we sort of have to have, where some of the people think like we can, especially I get, you know, I see people talking about trying to stop, you know, open source AI and whatnot here in the United States. Great. How's your meeting with Xi Jinping go? Right? Because has he stopped developing models and releasing certain versions of them? I don't think he's going to stop. You Vladimir Putin can throw a few billion dollars at some proxy actors. It doesn't take that much expertise to train one of these models. I mean, is Xi Jinping motivated to release open source models, or is he just more motivated to have...

2:26:47Well, the Chinese are already releasing quite effective open source models. Built on LAMA, I thought, in large part as well. But I don't know exactly what all they're built on. But look, half the authors on these AI papers are Chinese anyway. They're Chinese-Americans working in American labs. but clearly the country has people there that are capable of building these things if you look at the latest um they have a what was the name of the one the um the movie one uh yeah it's like it's like a sort of thing yeah right there's no shortage of talent and computation there to do some of these things if they have state level interest so i think it would be naive for us to imagine that we really are the only ones in charge of our destiny on this matter i think we have to think very proactively about what will happen.

2:27:27What do we need the world to look like at the point when basically there is distributed access to these mass casualty tools, unfortunately? And that is, though, a different perspective than to say that those mass casualty tools just won't exist, right? So I think there's a... I feel that often in this conversation there is the, oh, it's just like, it's kind of like a doomer-y perspective to think that there will be these capabilities. But my take on that is, well, you're kind of just pessimistic to what AI will be able to do with more effort by the smartest people in the world with the most funding it currently receives in the world.

2:28:08It's like, I really believe in human ingenuity and we will figure out many more capable things to come out of AI tools. So yeah, I think the question of what do we need the world to look like so that when these bio-capabilities, for example, are distributed, we are protected. And frankly, though, there is a bit of a choice there because it is still, I don't know, OpenAI has a thousand employees, DeepMind maybe now, okay, DeepMind had like 1 ,500 now with all of the other ones, many more. But it's still like an amount of people in the thousands that do this work. And whether 10 of them or 500 of them work on these kind of like defensive, biased protection mechanisms for the world afterwards, does change the trajectory a little bit in relevant ways, right?

2:28:56Like you can either choose to, okay, let's just like 100 % accelerate all dual-use capabilities because we're just hoping to balance over time. Or how about 80 % dual-use and 20 % defense-biased work or something like this to the degree that it is possible to separate them out. But in some ways it is sometimes. Well, I think the most immediate concerns that we can look at are in the area uh the most um dangerous asymmetric kinds of things where information uh in the form of an llm could lead to mass casualty kinds of things i think those are mostly in like bio and chemistry kinds of areas and if you can kind of look at those areas okay um some of the basic chemical things that can do a lot of damage you're more likely to kill yourself playing with them than you know actually chemicals are chemicals bio is really bad bio is kind of pretty bad well because it's self-replicating it can be self-replicating and it and yeah and there's a lot of like an actual living human being operates on a very like knife edge balance of equilibrium and it doesn't take a whole lot i mean a few milligrams of neurotoxin you're gone right so um that's that's where yeah there is more danger there but i think about like the to your point about the researchers on safety at places like anthropic or at open ai deep mind um we are working on, we're getting much better, I think, now tools to look at interpretability and what's in a data set and doing it, I'm sorry, what's in a, just a weight.

2:30:18If we just get some weights, we can actually evaluate and look at what does the feature matrix look like? Can we extract some kinds of things? Now, you could absolutely imagine a national effort to build a very, very, very good expert model with a lot of really good chemistry in it, informed by some of the best chemistry and bio and sort of bioweapons researchers and whatnot. And we're would have a that would be a private model that would be run in sort of secure government infrastructure to then drive evaluations of other models to look at well are these features present do we see these things mentioned and and whatnot i think that's the kind of thing that we could be building from a safety perspective right now the fact these are black boxes is part of the danger is that it you know or not danger is part of what makes it difficult to talk about this but we're coming i think to a point where we will start being able to extract some things out of them and see that there's another ecosystem dynamic which is once you're able to put open models out there that don't have these kinds of things but that do a lot of the fun stuff or help people learn geometry and help people you know reason about philosophy um that can out compete some of these other kinds of things and the people who are doing those models that have maybe more dual use kinds of things and whatnot it makes it a little bit easier to find some of those kinds of places you actually you know the gpu is available for people to fine-tune their own stuff maybe like they'll be more used by these open models there's a lot of stuff you can do once you actually have ability to say here's the problem we're whittling down this particular set of it or focusing on it so maybe these are all very hand wavy arguments and i'm not the deepest on like the biggest doom versions the biggest doomer versions of like gray goo and you know nanotech whatever or or or bio weapons kind of stuff but yeah but but i but i do feel like we are coming up with some tools that could do some evals on the stuff that could really look through it and um and we should be securing the the the actual tools anyway like the actual physical tools for biochemistry and things like that knowing the stuff is coming right but then on the conversation of openness again like if we were so one thing that is being researched on is interpretability of the black box so to speak and then also okay so now you did identify where it's like virology understanding sits potentially right um can you uh specifically only turn that capability off like in a targeted way where not that you trained it without the capability because like if you reduce the data set it's going to be probably less of a good brain in all sorts of ways.

2:32:44But after the fact that you've trained it now you potentially you can turn it off. If you released in an open way that entire process though then again someone could just do the whole thing without turning it off. So then you again gave the capability, right? So like there's I think there are just certain things where, yeah, like prior to having the defense in place, I would still like to, okay, this is one aspect that shouldn't be openly distributed. Like we don't have open access to every single technology of any kind, right? We don't have open access to nukes. I know it's always the example that people worry about, but like we also don't have open access to, even though in practice we do have it to smallpox even, I suppose, but we shouldn't have open access to smallpox.

2:33:27um yeah so i think yeah and unsurprisingly like a little bit subtlety around which capabilities are we which few capabilities do we feel good about uh turning off yeah i think certainly it's better if you can actually have access to the training data to say well it didn't learn anything about virology so it's not going to have biology content in there right doesn't have the concepts of what some of these organic molecules are, what some of these protein sequences do, just doesn't even know. And it wouldn't have to know if it's going to help me write code, right? Coding assistant, whatever. And when we are looking at putting those data sets together or people put those data sets together and then create a fine tune that does this, then yeah, I mean, that certainly is possible, I guess.

2:34:16But this is, I mean, not just, I guess, it will certainly be possible. And I do think this will be done, but that's why it's so important for us to invest in building these interpretability and model monitoring and vowels that are able to actually gauge how much knowledge does this particular model have of whatever and start tracing some of the stuff down like it that's not an un foreign concept to us I guess to regulate certain kinds of things like with guns we have certain kind of regulations with airplanes yeah you have to get rated for certain kinds of airplanes of a certain you know complexity and so for models most of the applications we can do and most of the things we can do that will make society better we could run with models that have no knowledge of virology right yeah and undoubtedly we want like given that it is a just enabling thing ai it's just intelligence that you can use for pretty much all sorts of in all sorts of ways and that you would expect will increase productivity across all sorts of domains you do want it to be distributed of course ultimately yeah and but that's just for the current incarnation which is not AGI, right?

2:35:16When you get to really superhuman intelligence and what it could do for people there, this is just all baby steps in preparation for what happens when we have access to that kind of capability. But I think we need to get good at this. We have to start getting good at this. And it starts by actually being able to have a conversation about values and technology. You mentioned superhuman abilities and you also described the version where you will have a lot of narrow AIs helping people that maybe able to run yeah what's the world you're picturing in say 10 years what do you think where we'll be well i would hope that in 10 years we've gotten into a mode where people really are solving for what's meaningful for them and that they feel supported by technology in doing that and the technology is there to help them both do that for themselves but also connect with their family groups, friends, whatever, their individual cultures and doing that kind of thing.

2:36:16I think we are, I really am hoping that we can lean on some of the emerging AI capabilities to break free from the way that Moloka sort of hijacked people's psychology in the current mode, especially with some of the stresses on global finance, some of these kind of, you know, people being more conscious of climate change and overproduction, overconsumption, all these kinds of things. And actually, even just more and more podcasts like this, like yours talking about, you know, win-win and not selling for zero-sum, more generations coming up through the social media and overconsumption and fast fashion, the hype consumption, whatever, and getting the other side of that and saying, I want something more, right?

2:36:57In a sense, we're all sort of trying to rebuild meaningful civilization from the shadows of whatever the fuck it was the boomers left for us, right? And so I'm hoping that LLMs, not LMs, but AI or some of these kinds of technologies can help us be more thoughtful, more intentional in our interactions with the world, with each other, and really how we regard ourselves. Because all of the stuff we've built, a lot of these early deployments, PC technology and smartphones and all these amazing technical feats we've built are ways of creating more black mirrors or colorful dystopias. They're ways of making us into happy little clockwork oranges.

2:37:28But really, we want to be humans expressing our full humanity. and we want technology to really, we want to put the horse in front of the car, we want to put humans in front of the technology, I think we can do that. But to do that, we have to be intentional, deliberate about doing it. It's not going to happen automatically. And it certainly is not going to happen if technologies keep pretending that technology is value neutral, which I'm just flabbergasted in this day and age. Anyone can make that argument. So it sounds like you're, yeah, I like that vision as well. And I think at the point when AI's initial uses were like advertisement improvement, et cetera, right?

2:38:00For the algorithm, it's at the point when it actually improves truly, as you previously said, for us to be a more enabled us. At that point, it really helps the collective sensemaking. One could really well see how added intelligence, where some tasks of the collective sensemaking, could be outsourced to trusted AI versions, which you can be sure of that they actually optimize correctly for the thing that you want them to optimize for you around certain topics, that would be a pretty good future. Yeah, I would say that this is the first time we really can have... I want technology that helps us discover for ourselves what's best, as opposed to this treadmill of always trying to get us addicted to what's new.

2:38:43So moving what's new to what's best and giving us agency in that decision. I think really this will be the first technology in a long time that will come out and tell us, hey, it's time to actually turn off the computer. step away from the app. Go touch grass. You need some you time. Like imagine your AI assistant, you need some you time touch grass, right? That'd be great. That'd be good. None of the current apps will tell you that, right? Definitely not. No. Awesome. Well, thank you so much. Thank you so much, guys. This has been absolutely a pleasure and a blast. Thank you. Thank you, Peter.

2:39:13That was great. So there we go, folks. Huge thank you to Peter for letting us dig into his amazing mind. If you enjoyed this, then I highly recommend you check out some of Peter's writing. I'll link to it in the show notes he's written some really incredible blogs digging into a lot of these topics in more depth he's also a regular on Twitter or X, whatever it's called so give him a follow on the link below too and as always, if you enjoyed this please share it with whoever else you think might do so too and until next time may your next two weeks be full of win-wins

2:39:55Thank you.