Andy Matuschak - Self-Teaching, Spaced Repetition, & Why Books Don’t Work

26:53So just to come back to the question, explicit memorization or explicit making sure that you can recall the thing reliably, we can test it against these things. So for the case of the creative instinct, for instance, noticing the contradiction, noticing the connection, I imagine that we will have future notebooks that will do some of this noticing with us, and that will decrease our need to be able to rely on our own sense of salience or something like that. But I guess I don't know how much. I'm skeptical that my own experience coming up with weird ideas that feel very new is that it feels very personal, if it feels very inquiet, I often haven't been able to describe textually the constituents of the thing very clearly.

27:45There's just kind of a feeling that something in this general direction is connected with something in that general direction or there's attention. And so that makes me a little hesitant. LLMs depend on our ability to externalize things and to make them legible. Back to the learning point about the rule of memory. If what you're trying to do is to understand something pretty difficult, your ability to understand that thing is still absolutely going to be bound on your memory of the constituent material. Do you think there's value in, there's pedagogical value in forgetting? So I guess some sort of anecdotal or unrelated evidence is in neural networks.

28:22Sometimes you can improve performance by pruning some of the weights. Obviously, we forget things. So, clearly, we don't remember everything when we sleep or lose a lot of our memories. Is there possible that I'm forgetting the details and only getting the gist that actually helps us better generalize the insights we're getting from techs and things like that? What do you think of that way of thinking? Yeah, it could be. Memory is very connected to attention. We can't attend to everything. One of the rules of memory is to help guide us to the things that are important. So maybe I happen to know the magnitude and energy of an electron volt.

28:58That's something I can draw on because of the memory system stuff. But I also don't want that to be front and center in my mind all the time. I want to be hyper salient the way that I don't know some very important design principles to me. So yeah, there's some role there. There's some theories that the reason we have forgetting is that our environment, our ancestral environment was very traumatic. And so our episodic memory in particular, we would like to maybe not be all of that faithful. I actually don't know the status of those theories. Probably very forget dreams as well, right? Like dreams are pretty dramatic.

29:33If you thought of them at the same as every old life experience. Yeah. So I mean, another weird thing about memory is that as far as we can determine, memories aren't lost exactly at least not completely. There's a bunch of theories of interesting experiments that people have used to demonstrate that decades later, things are still there. If you can cue them right, people can bring things back, even things that they feel are lost. And of course, you can also cue people in ways that are hallucinatory, so you need to be careful about that. I guess the reason why I bring that up is that it flies in the face of this view that there's a limit.

30:10Like one of the things that I think's kind of weird about, this memory system stuff, or memory champions, ships, or something like that, is people like, oh, Well, if you do these things, will you start to forget other normal human stuff? And what's weird is, no, I've been doing this memory system stuff for years. I just know more stuff now. And this is aligned with the experimental literature, which seems to suggest that there's probably upper bounds, but we're not close to them. And some of these memory champions have memorized, I mean, truly absurd orders of 90, maybe two orders of magnitude, more things than I have practiced.

30:50Certainly, people who are multilingual have really, really observed numbers of things memorized. So there isn't like a resource management argument. Well, if there isn't, why do we forget so many things? Is there some reason the brain just some of the things we're working with us? Maybe they're just not the we were training the intestinal environment to find certain things salient that are just on map onto books. No, it's a good question. So we're getting to a part of the cognitive science space that I'm less familiar with and also that I suspect we simply know less about. But let me just riff a little bit.

31:28One of the things that we sort of know is this idea of spreading activation. So when you go to try to look something up or when you try to deal with a particular situation, There's something almost kind of like DNS exchanges or like routing on a network or something where okay Like we start from some point that it's like a stimulus and Just speaking very informally like we kind of expand outwards from there and There are effectively like weights on those connections and by tuning those weights effectively We like route the packets on the network effectively memory is encoded in these weights at least partially So if you didn't forget things, then you might just have this weird, coughony on the network, and in particular, like, what's salient, what to do next, like, which response seems most appropriate to this question, like, you might answer those kinds of things very ineffectively because all this stuff is coming up for you that is like much less relevant.

32:30Like, one of the theories about how well we remember stuff and what circumstances is actually called predictive of utility theory. And it suggests that the probability of retrieval of a particular item in a given situation actually does correspond with basically a model of how to what extent the brain predicts it will be useful. Right. And then the prediction doesn't necessarily map on to. Doesn't necessarily. Exactly. And so when you repeatedly access something, when you practice retrieving it, the prediction of the utility of the thing goes up. Yeah, and when you do it in a variety of situations, it goes up across a broader distribution.

33:08Okay, so this is interesting. When did you start your memory practice? Presumably it was after Apple. Yeah. Okay, so let me ask you this. So at Apple, you were in charge of a bunch of important flagship features on iOS and I'm guessing other things. Presumably you didn't have some sort of practice, but since you were encountering these things day -to -day, that natural frequency and way in which problems came up. Did you have a worse understanding of this problem and the things now knowing what you do and knowing that having the practice as you do, you're able to comprehend now? I don't know the question made sense.

33:44No, that's a great question. So I mean, like, here's the fun thing. I was much better at what I was doing than I am at what I'm doing now. That's pretty funny. So I mean, it was just totally different. Let's talk about this a little bit. This feels very juicy for me. Most of what I was doing was engineering, some of the very difficult engineering, but mostly engineering, mostly on things that were fairly well understood. So I wasn't trying to decide what should be done usually. Sometimes I was from a technical perspective, but certainly rarely from a product perspective was that a relevant question for me.

34:24I was kind of like a somewhat designed -minded engineer, and I did a bunch of engineering and design -ish things on tasks which were set out for me. At that point, I had been programming for a really long time. I don't know, 13 years maybe, by the time I joined Apple longer. And programming in Apple's ecosystem for probably two -thirds of that time, three -quarters of that time. So everything was just really familiar. And like, it was mostly flow all the time every day. I was kind of like, I was just in it. I knew the stuff that I needed to know. I was very well practiced. And the space didn't change that much.

35:05You know, like engineers, most engineers at Apple most of the time are not like pushing the frontier of what is known, like trying to discover, they're like doing very difficult technical work, mostly applying things that they already know and understand quite well. To problems which are usually not always moderately well and pretty well understood, let's say. Memory was essential to me doing that job well, but I had already built most of it by the time I got there. I'd already built just tons of stuff for Apple's platform. I had to learn a lot of stuff. I learned a ton of stuff about the internals of those systems, but because I already had such a rich understanding both of Apple's platforms and of computer science and engineering in general, I had this really rich network for stuff to slot into.

35:52So learning stuff is easier when you have other stuff to connect with. It's a nice principle. Metacognitive load on me was lighter because others were figuring out what we should be doing. So just like by contrast, now I'm doing research, like I'm trying to discover things that are not known. I'm trying to make things that didn't exist. The hard questions that I answer are mostly like, what should be done or like what should I do? And that question is not just a technical one of like, how should I implement this feature that needs to get built? but what intervention on a reader should be taken.

36:25That requires synthesizing lots of different unfamiliar literatures. There's two different threads I want to go on. Maybe I'll just mention the other one. This is also related to the thing we're talking about a few minutes ago with Eleums. And there's this thing that I'm sure you've talked about yourself as well, the Swanson Linking, where this guy was just, I don't know, Swanson Linking. Oh, okay. Actually, Michael Nielsen has run a violin in his work. Okay, cool. But this guy was just somebody who read the medical literature and If you were just like familiar with a lot of esoteric results and one of the different things would come up and he would be able to Figure out what different things are connected for example I think he noticed in one case that headaches are linked to some other symptom and that other symptoms linked to magnesium deficiency and And so apparently a whole bunch of people's headaches were solved once they were given a magnitude supplement.

37:18And you notice that connection. Again, this is the kind of sort of combinatorial thing that you wouldn't notice otherwise. But on this subject itself, so listen, there's this natural way in which you're able to get up to speed in all the things that are happening in Apple. And is it possible and maybe advantageous to do similar kinds of things in other fields? For example, instead of doing an explicit space repetition system when you're trying to absorb material from books, you just read a cluster of books. And hopefully things will just come up that are relevant to get them again. Or is there a value in having explicit practice of setting up cards and so on?

37:52Yeah, right. So again, the answer is going to be it depends. I think that maybe the most familiar example of what you're talking about is immersion learning a new language. So immersion learning is like a great thing. And it's going to be more interesting and more effective than doing space -to -appetition practice is going to be integrative, it's going to be socially bit, so there's a bunch of stuff about social learning that's relevant. A problem is though is that say that you decide you want to learn Swahili today and you go down to the local Swahili community center and you're like, cool, I'm going to immerse myself.

38:28Good luck. You can't even get started. So through this lens, explicit practice is a way to bootstrap yourself. Likewise, a great way to become a better musician, and all of the best pianists at site reading that I knew in university played with churches. They were so good at site reading because they had to show up every Sunday and they're playing a different thing. New him every Sunday, right? So this is immersion also. And over time, they're learning all these cadences and these things that are really common and whatever. But like you can't show up and be the church pianist every Sunday in the first place if you don't already have like some decent foundation.

39:08So this is sort of a bootstrapping argument that like one role for explicit practice of this kind is to get yourself into a position where you can more naturalistically reinforce. I see. Okay, I got it. But there are still going to be instances where the naturalistic reinforcement isn't going to work. So for example, the linking that you brought up, one issue for doctors is rare diagnoses. So if it's only going to be once every couple of years that you see a patient that's going to present with these symptoms, that's not going to be frequent enough to naturally reinforce your memory of that, and you're going to need some out of band mechanism.

39:43And unfortunately, I think for many kind of creative leaps and creative insights, that may be closer to the Regina that we're in. Yeah, that makes a lot of sense. Where in many fields you just have the things you're regularly doing is the thing you need to reinforce. It makes a lot of sense that if you're a researcher, the long tail of events that might come up is a thing you might happen once every few months, but the regularity is not a thing that matters, right? It's sort of the fact on your work. Here's a question I actually have. So when we were doing the practice, or when we were doing the quantum mechanics textbook, it was like three hours, and afterwards I was just exhausted.

40:18And I was actually surprised that you went the entire three hours with an interruption, And so afterwards I was packing up and you were like, hey, I'm about to actually go to my piano lesson. Right. I was so confused that how you had the stamina to keep going, is the stamina just inherent in you, or is that something you did to develop? So one of the things that I think is funny about stamina, is first off, there's some kind of weird grasses always greener kind of situation where like I often feel struck by other people's stamina and feel like I have very little of it. I struggle with energy. I've actually written extensively about all my struggles with energy and ways of managing energy.

40:54I spend a lot of time thinking about it and managing the energy levels, structuring my day around it. So I think there is something where one often feels maybe lower stamina than one actually is because one misapprehends others stamina. Okay, so in that particular situation, how do I explain why three hours of studying, et cetera, first off social? So if I were alone and studying that book for three hours, and I weren't effectively trying to perform for you, Dr. Cash, it wouldn't have been nearly as energizing for me. And I definitely would have taken breaks. I still would have been able to go for three hours, I think.

41:34Part of the reason for that is that it's simply way less hard than things I normally do. In some sense, learning quantum mechanics like should be much harder and it kind of is cognitively demanding in a lot of ways. Right. It's much more cognitively demanding in kind of a direct way than what I actually do day to day. But it's much less, it's much less demanding on what William James calls the energies of men, which is something like like a life force that permits you to act according to your will or something like that. Maybe it's gumption, maybe it's willpower, maybe some Some people call it spoons, I don't know.

42:11These aren't all the same thing exactly, but sitting and staring at a page and deciding what you should do next on a research project is incredibly draining on that resource. The not knowing, sitting and not knowing is like the hardest thing that I do in my work. And so there's something, it's like a wonderful vacation to be presented with, oh great, somebody else is gonna tell me what to do, right? This is great. So although it might be less demanding than our usual work, it is definitely more demanding than the way in which I or most people approach textbooks or other material in the sense that I would just like read through and then once I get to the exercise I'm like, let's see what I didn't understand.

42:58Whereas just the attention and the intensity to go through sentence by sentence and and constantly being paying attention. Seems like way more exhausting than. Yeah, I mean, so this is sort of true. Like, and it's definitely the case that I will occasionally do some of this like before bed reading where I think like, oh, like, you know, let me just do a little bit more. You know, and it's like it's basically useless. Yeah. But I want to make the case that there is a kind of pocket that you can fall into. Maybe you call it flow, I don't know, where like the demandingness that you're bringing to bear is matched to your ability.

43:29The book is not overwhelming. Like, do you feel like you can make your way through it? And this is actually more engaging. So I occasionally, well, find myself reading as an undemanding reader and finding my attention kind of slipping because I feel like I'm just not that attached to the text emotionally. I'm kind of reading dutifully. I'm like trying to get through it. That produces sometimes like an adversarial aspect where they text is in my way, or it's kind of something to be, I don't know, like accomplished. And often, then I will find that I need to bring more gumption to bear, to kind of power through, and like, you know, make myself sit there and keep flipping the pages.

44:19Then I need, if I actually just like open my curiosity and open my attention and really start engaging with the book. There are some of the ideas that people come up with for different pedagogical tools which are mediums that give slowest connection to the reader. One is you know you have some sort of fiction account where a concept is introduced and reinforced or you have a video game with characters you care about. And as far as I know there isn't something that has really taken off using these sorts of new mediums. What do you think that is? Is this just an inherent limitation of everything about text and lectures or people would just have been given at the right content and design.

45:01Yeah, I'm fascinated by this question. Let's see, I can say a few things about it. One is that I would argue that one meeting is taken off in an absolutely enormous way in that video. People love video. People will watch Grant Sanderson spend, you know, an hour going through some explanation of an esoteric math problem. People who would never crack like a springer graduate textbook in mathematics or something like that. The issue is that like they will in general not walk away from that interaction with much understanding. But they're much more engaged. So that's cool. That's suggestive. You know, and suggest the question like, well, but like is there a version of that that I actually produced detailed understanding?

45:42Maybe one approach to producing that might be like a game. My favorite example of this is the witness by Jonathan Blow. You played the witness. Okay. I think the witness is an absolutely extraordinary work of art. So it's a game that has no text, at least no text that's relevant to the game elements. In kind of classic mist style, you wake up on an island and figure out what's going on. And the game proceeds to explain to you without using words, but just by shaping your environment. a series of extremely complex mechanics, basically, of a system that exists in this world. So you learn a bunch of stuff, and it gets to the point where it feels like you're in conversation with the game's designers.

46:34It's like, ah, he's asking me to do this here. No one's asking you, right? There's no text, but you can feel that you are being asked. And you perform some interaction in the environment, and you feel that you have answered, and the game responds in kind. This is very, very interesting. It's like it's a medium of action. Some people have tried to make educational games, games which are explicitly about arithmetic or something. Jonathan Blows Games is not about that. It's the mechanics that you learn are, they're like about the environment. I don't think anybody has yet really succeeded in doing this about explicit subjects.

47:10There are, for instance, things like a purple space program. Maybe people learn some things about project management or overall mechanics from that. Zactronics has a bunch of games that are sort of about assembly language. Roughly speaking, maybe you can learn some things about that. The issue seems to be that games are ultimately, they're in aesthetic form. The purpose of the game is to have an experience that feels a particular way. And so they're sort of serving a different purpose than grants videos or a text. Grants videos are also serving a different purpose from the text. Like the text you might pick up because you're like, I want to be able to build a robot.

47:51Like, so you pick up a textbook on robotics or something. And so is there something that you can pick up that's sort of like a game in so far as it's an active environment that you use in a similar situation to, I want to learn to build a robot. Maybe kind of, we don't quite have those yet. We have some things that are kind of like that. I don't know if you've seen like from NAND to Tetris. This is a very interesting project that's kind of along these lines. And what characterizes it like games is doing. It's active. So when I was asking all those questions of the book, that one is active learning, active reading.

48:29NAND to Tetris is naturally active. So this is a course in which you kind of start with basically nothing, you start with memory and you build a virtual computer and you build Tetris, you build a processor and stuff. And so the whole thing's active, the whole time you're making the computer go. This is doing a similar job to the question asking that I was doing except that you don't have to regulate all of that yourself. the regulation, the choice of what activity to do is in the course, is in the structure of the material. I think there is waiting to be created some kind of mass medium that is like that, but that can be applied in many, many circumstances.

49:14We have the non -mass medium version of it already, and it's a apprenticeship. Like if you want to be a good yoga teacher, you like go hang out in yoga studios. If you want to be a good surfer, you go to the beach when the other surfers are there, and you like participate peripherally, and you talk to them, and you learn about their tactics. They might give you some feedback eventually, and you'll start to participate less and less peripherally over time, and eventually you'll be part of the community. This is in a mass medium. We can't print billion copies of it. We can't with a book. What is the experience of watching George Houghtson's story in a coat of tiny grad?

49:52How does that compare to just being in an office with him? Yeah, because even if you're in an office with them, there's, you know, there's, there's constraints on his time and how much engagement there would be. So, why is individual a scalable way to increase the apprenticeship? I'm incredibly excited about streaming, actually, as a medium for this. So, we're kind of gesturing at a particular kind of learning that needs to happen. It's often called tacit knowledge. So, like one of the things that you have to learn to do as an engineer is to learn to deal with 100 ,000 different like weird situations where something is not behaving the right way.

50:29And eventually you learn a kind of pattern recognition. You learn ways of dealing with this. And much of this is like not described in any book. It's not explicitly taught. You just kind of learn it by doing it over a long period of time. So by watching George do it, I think that people do absorb stuff. They can absorb some of that knowledge. That's part of how apprentices absorb that knowledge. There's a few things that are missing, you're not getting feedback, there's a whole lot of chat there, there's a whole lot of stuff that probably isn't all that meaningful. It's also true for apprentices.

51:03Actually, I'm pretty excited about streaming videos. I've complained loudly that there aren't more designer streamers. One of the things that I think is really interesting, is that we have some disciplines like programming, where there are like a million books on courses about how to learn to program. And they don't give you everything you need. There's this tacit knowledge stuff that you need to develop. But like, if you work through these courses, if you go through the MIT OpenCourseWare for computer science, you'll be able to build some stuff. And you'll be able to leave yourself up. This is not true in all domains.

51:39You know, if you're looking at, well, in particular, design, but lots of other domains that are sort of like that, like these are composition architecture, something like this. Nope, like it's normally done in studio classes. Lots and lots of hands -on feedback. Stuff has, the feedback is highly contingent, it's highly contextual. We just haven't figured out how to communicate this. And so it's good to see lots of programmer streamers, but I really want to see the streamers in these other domains. Right. On the point of the more programming books, ironically, the reason why there's some more resources on programming is that it's just so legible, but it already makes it easier to understand in the first place.

52:20So you just have this reinforcement. But also, you know, the nantotestress is, let's say, a second video game analog to learning, maybe not just programming, but how things in the internals of a computer work. Yeah. Programming has an element where it already feels like a video game. I'm a friend who has a sort of intense sort of manic energy, and he used to be addicted to video games when he was a teenager. And now he just stays up all night in these coding vinges. It's just the same part of the brain. Are you optimistic about things like video games and fiction being able to work and feel that are not already kind of like a video game like programming?

52:57I think what makes programming feel like a video game is this sense of instantaneousness, this sense of direct contact with the environment. You're learning about a kind of a conceptual world, but that world is right underneath your hands. And as you manipulate it, you're constantly getting this feedback. The red squiggly, your pressing command are regularly. and you're seeing it fail. And that feels great. Like there's this feeling that's very common for programmers and it's laden with doom. The feeling is it's like 9 p .m. And you've been working on a thing all day and it's almost working. It's almost working.

53:44And you know, like if you just debug this one thing, then like your project will be like done and you'll be able to like go to this. So you're like, well, I'll just stay up and like, I'll debug this one last thing. And then you start debugging it and you get it and you solve it and that feels great. And then immediately you run into like one more thing, like, oh, it's almost running all the way through. It's almost going end to end. And you're like, wow, I'll just stay up a little bit longer. And you know, before you know it, it's like 2 a .m. And you keep going, cause it feels so good. Like you feel the sense of forward progress.

54:13You're not just staring at a wall. I think like for the programming problems where you are at a brick wall, it doesn't feel like this. It feels bad. So can every field be transformed into something where you can feel the sense of forward progress? You can get this rapid feedback cycle. I think that's really hard. I think some fields, like, it's not clear to me that they can be transformed in that way. I think we can get a lot closer in most cases than we're at right now. Now, so what's hard about designing a easier interface is that often there's this feeling of exploring kind of a combinatorial search space.

54:56Programming often feels like a search problem too. You have a sense that there's some right way to solve the problem. There might be some set of right ways to solve the problem and you're kind of looking for it. And you have some heuristics that guide you to, like, oh, this might be a dynamic programming problem. or this might be something that is solved well by separating concerns or something like that. Design often feels less like that. You have those heuristics, too. You have those patterns, too. Often it just feels like, nope, I just need to try 300 things. And so there's this characteristic where you'll look at a designer's figma file.

55:30And what you do in figma is you press Command D. This is the core action of figmas to duplicate. So you have an art board, you tried something. You know, like, that didn't work. You'd select it and you press command D. And what you end up with, and you look at design Twitter, and it's just all these screenshots of people's figmas with like a million artboards. They're just a giant stuff. And you don't have this feeling, or at least I don't, and I think many designers don't have this feeling of like progress. It's like, you're just kind of exploring parts of the search space. And like, you're learning that parts of the search space don't work.

56:06And, you know, eventually you stumble on one that does, but you don't have this feeling of getting closer often. Often there will be like weeks that go by without feeling like you're getting closer because what you're doing is just kind of like narrowing the search space. Interesting. Although there are people who are obsessed with design in there and what is the sort of loop that keeps them obsessed with, I process that doesn't feel intrinsically forward feeding. So to some extent, I think they are skillful. So the people that I know who are like this, it's a combination of their skillful often and like the nature of the problems that they're solving are highly tractable.

56:50So like an example of a kind of thing that designers will often rabbit hole into is like designing a poster. It actually often used to be kind of a cliche that at Facebook, there were all these posters up on the wall of the office. Very, very elaborate, beautifully designed posters for a talk that someone was coming to give at Facebook. And it's like, why did somebody put all this effort into it? Like, well, it feels really good. Because a poster is this really constrained, it's like finite, it's ephemeral. You can start it, and within a few hours, yeah, there's a search space, but you can find a decent part part of the search space pretty rapidly.

57:34And once you're there, there's this beautiful and very enticing feeling of turning the crank and making it better and polishing it and trying this or that. But when you're trying this or that, all of the options are kind of okay and you're kind of trying them out of curiosity, or maybe it can be even better. And that's very different from the kind of design where you're just like, I simply don't know how to do this. And I think it's part of why those designers loved making those posters. It's a snack, it's a treat. It's also something they get to control, whereas ordinarily they don't. Just don't tell the manager how many software engineering hours were used up in the poster designing at Facebook.

58:18Well, it's not known as software engineering. Oh, okay. It's only designers. But for the software engineers, I mean, CodeGolf is the equivalent, right? What is CodeGolf? You know, in golf, like you try to get the lowest score. So co -golf you try to solve the problem as Minimally as possible, right? Like I don't need this. I can like combine this. I can do it in three lines All right, if I use Haskell I can do it in a one line That's a that's a kind of thing programmers do that's what's like this But just endless refactor is another thing that's kind of like this. You have the thing working But it could be more beautiful.

58:50Right. Well, so it seems like the tools and the ideas you're developing see especially geared towards very intelligent and very motivated students. If they would be different, what would the tools that you would develop for a median student the education system look like, both in motivation and in other you know other traits? Yeah they'd be super different. I kind of got out of it. Educational space in part because I don't like the framing of this problem. Well, for the median student, the education system mostly wants to make the student do things they don't want to do. It's not about helping them achieve their goals more easily or more effectively.

59:34For the most part, it's about achieving goals that aren't theirs. Obviously, that's not always true. But for the median student, I think it kind of is true. I become very interested when I was at Khan Academy. I was thinking about this problem. one of the angles that I found really interesting was, so it kind of had to me we were mostly thinking about not just the median learner but like maybe the 25th percentile learner. One of the angles that felt most relevant, maybe not from an efficacy perspective but for me from like a breaking out of this, getting them to follow goals that aren't their own perspective, was to focus on inquiry learning and to focus on transforming the learning experience into something that actually is related to their goals.

1:00:20That is, we're asking questions that are authentically interesting, that they authentically want to answer, and that they can participate in in a way that feels natural. We did a lot of experiments with dynamic media, representations of things, the idea being that like, you've probably seen, you know, movies, like, plastic blocks or things that people can play with when they're kids to get an idea of numbers and number systems. Kids will play with these things and prompt it because they're fun. It's just it's a pleasure to handle them. It's a pleasure to manipulate them. When you have them in hand, it's very natural to suggest like, ah, can you make a pattern like this?

1:01:07Like, why? You can't seem to make patterns like that. Why is that? So, for instance, you can start to point out things like, so, queues and air rods is the name for a set of 10 rods that have basically unit length 1 to 10 and are all different colors. And so, you can do things like take 8 and like the rod that represents 8 and put two of the rods that represent 4 up next to it and showed that like this one you can, and you can divide into two rods effectively. But then if you take seven, there is no other pair of rods that, for the same color, you can put it next to it. So you get these different patterns.

1:01:48So things kind of naturally suggest themselves by experimenting with these materials and having conversations with people around these materials. And so one of the things we're interested in was, well, are there things that are like that that are for more advanced topics? like can we create something that's kind of like those rods, but that is about, you know, like a more advanced topic in math or are about debates in history or something like that. One of our tactics was to lean heavily on social interaction. People like talking about stuff with people if it's like a real conversation. For the same reason that I had to use less willpower to study that quantum mechanics text because you were there with me, a student who's engaged in like a real activity with a peer will need less will power as well, but also learn from their peer if he's structured things right.

1:02:48So social learning becomes interesting, but I can get a high level. I mostly have abandoned this question to others. Basically everyone in the educational space This isn't totally true, but like, you know, 90 plus percent of people in the educational space are focused on really like the bottom quartile, like not even medium. And there's a good reason for this. Like many, not not quite as many, people who are in education, they are motivated by arguments of equity and opportunity. And they want everybody to have the opportunities they had. they're very motivated by the injustices that they see in the differing access and the differing support that different people have.

1:03:32And they're very motivated by the very real disadvantages that accrue to the bottom quartile performing students. It's also true that the marginal impact that you'll have in that student's life will be much greater probably than the marginal impact on say 80th percentile performing student or so the argument goes like that student will be fine which is like properly true. But there's a big marginal difference between fine and it's super charged. Yeah, yeah, I mean that's true but anyway, I say all this to say like I understand why the vast majority of people in education are focused on what they're focused on and I think it is good and I'm glad they're doing it and I'm mostly have decided to let them do that.

1:04:19to focus elsewhere. No, I see tremendous value in focusing on realistically, the cool you should that's coming out, where's that coming from and what less the weight increase that. Okay, so it's interesting to know that the same tools might not just work across the spectrum. Part of the trouble here is that the cool shit is very likely to come from students who are performing at the 20th percentile in school because they're disaffected and bored and none of this stuff matters to them. And so part of the trouble here is that by opting out of helping these people learn, there are all kinds of interesting inventions that could probably occur.

1:05:03They aren't occurring. So I don't quite know how to contend with that. I guess basically I'm trying to bite off a piece of a problem that feels maybe tractable. Once all the tools are built, when you're at the end of your career, would But the learning process is it's supposed to feel fun. Or does it have to feel fun? Is there an element of even when all the tools are there that there's just like a level of David Goggins, you know, this is going to be miserable, but I've decided to learn this in this way and I just had to go through it. Where does misery come from? Hmm. Where does it come from?

1:05:39I guess I'm asking this honestly, not really rhetorically. Let me try to answer my inquestion. I mean, so let me say first off, like I think I am broadly speaking very opposed to to what I understand to be David Dagan's esk attitudes towards almost anything. In this particular instance, I think what I think is something like, if I ask, why is it miserable to learn a particular subject? The answers that come to mind are things like, first off, I don't care about this subject, and I think that's not what we're talking about. You're asking about a world in which these great tools exist and someone's using one of these tools to try to do something they really care about.

1:06:12So another reason why it could be miserable that I think is pretty common is that you have some idea about like You're not going fast enough or like you're failing or you're struggling and the misery comes from resisting that it comes from Feeling like you're you're doing poorly and like you shouldn't be doing poorly like it's bad that you're doing poorly And to be you're feeling fearful like like others are going to judge you or like you don't have enough time or something like that. And I think that's basically like an emotional problem that needs to get healed rather than like a practical problem with the learning.

1:06:48In the case of something like organic chemistry where like you truly do just need to learn like you know 200 names or something. One answer is that like okay that can be done very cheaply using modern memory systems So like organic chemistry students suffer through this and they don't need to but even with modern memory systems Like you're probably gonna spend a total of I don't know call it a hundred minutes across some weeks Start studying all of these Formula and that still is unpleasant so cannot be resolved and I think the answer is yes actually So I was thinking about this in the context of the Cell biology by the numbers book.

1:07:29I was telling you about where there's all of these things like the volume of a nucleotide, I said, as a nanoliter. So to study the flashcard, what's the volume of a nucleotide is not terribly present. I'm not sure it constitutes suffering exactly. It's fine. I'll do it while waiting in line. But I think there is a better version of that, which is solving an interesting Fermi problem, which involves that term. So something like, if I have a vial of the COVID vaccine, Like, how many copies of the COVID RNA are likely to actually be in it? If the vial is a milliliter large or something, that's like kind of a fun little question.

1:08:14And I can enjoy sitting and noodling on that. And in doing so, I will need to retrieve the volume of the nucleotide to help me make that approximation. So I think there's moves like that that you can use to kind of paper over any remaining stuff that feels kind of necessarily unpleasant or wrote. I'm actually surprised to hear you say that because one way in which I read your stuff is The sum of your stuff is that this is actually a way of endorsing the traditional way of thinking about education But using new tools to get the traditional ones to give you an example of what I'm talking about You know go back to like a headmaster from the 1900s and you say is it important to have the taxonomy of a subject memorized You say, of course it is.

1:08:56That's what you're going to spend a year memorizing to autonomy. And then you would say memorizing to your students is actually important so that you have a dictionary by which to proceed on the subject. So in those ways, but you have a sort of new systems where you're doing the same kind of thing. So I'm not just surprised to hear you say on this particular. And the reason in this particular case, I was expecting you to say that, know that you have to be disciplined if you decide to learn something. Is I expected that in the case of the three hours of intense learning followed by an intense piano session, you were just like really tired at the end and you're like, but no, this is something I have to do this evening.

1:09:30So, yes, I'm actually kind of surprised to hear you say that. Yeah, yeah, no, I really enjoy this tension. And I'm probably overstating my, I'm probably like reacting to the, the Goggins reference, like a bit of an over extreme, over correction or something. But this really is how I feel. And I feel this tension all the time. Like the But the histories in educational psychology that I'm most aligned with are like the most robotic, like authoritarian kind of histories. And also the ones that are most like kind of unschooling and monastery -esque. Like I really have a ton of sympathy for elements of both of these directions.

1:10:09And there's kind of a weird synthesis of this in my head that I can't, I guess, fully externalize. I just part of what I'm saying is aspirational. Like, I mean, it certainly is the case that I do in practice use willpower to make things happen. So, just as an example of something totally contrary to everything I was saying, I use a tool called B -Minder, which charges me if I don't do certain things. Right? This sounds, I don't know if it's, you know, kind of military, but it's certainly more authoritarian than this kind of free -wheeling butterflies kind of gesture I was making the moment ago.

1:10:43And I use it to make sure that I do my memory practice. Shouldn't my memory practice be so joyful? It's at the center of my research, right? It should be like the most interesting, exciting part of my day, but often it's not. And so I use this to do it anyway. There's some tension here. I think I do want to say, you know, that the reason why I'm willing to endorse this headmaster's view about this taxonomy has to do with the price. I did a bunch of memorization in high school, and it was very inefficient and it was very uncertain. So it was emotionally difficult because I wouldn't even feel confident that I'd learned the stuff.

1:11:20I didn't know what it was to learn something reliably, like to be confident that I'd be able to recall it. And it also, it was hugely time -consuming, because I didn't have techniques or tools. And now, part of why I respond so favorably to just learning the taxonomy is that for me, it's just trivial. like yeah sure whatever throw it in the deck you know it'll consume a total of 15 minutes over the next few weeks and then I'll know it you know it just doesn't cost anything so like yeah okay fine other things in learning do still have real costs and those are maybe more difficult to negotiate actually this is a good place maybe to ask you about unschooling and you're out of two words let me just let me ask I think somebody in Twitter had this question which is your kids, is there growing power?

1:12:08How are you structuring their education? Well, okay, so to be clear, I don't have kids. Right. I have a medical case. Yeah, so you're going to hear the foolish response of a person talking about what one would do hypothetically. This is very difficult. School, of course, has many purposes other than instructional. right? It has a social purpose, it has a societal purpose, it has a behavioral purpose, and it also has like a pragmatic purpose of basically babysitting. Those things can be unbundled. I think it's pretty interesting to consider that. If I actually did have a kid, I would probably consider that project pretty thoroughly.

1:12:48I think it's like pretty likely that some kind of homeschooling situation would occur. It probably wouldn't be leaving the teacher, but it would probably be the people I would hire. I have some resources. Like I'm not wealthy, but I have some resources. So that is maybe a difference. But during the pandemic, I was struck by Brian Tobal started a company, which is now defunct. And so this is a fun example to bring out. But it's called School House. And the idea was that people were getting together in pods. That was the thing we did during the pandemic. And in particular, they got together in pods with their classmates from school, maybe five or six kids, and some of these pods started hiring elementary school teachers who were not working because of the pandemic.

1:13:34And the elementary school teachers would come to the backyard of one of these people's houses, and the five or six kids would get together with the elementary school teacher and they do stuff all day. Buying this one teacher's time split five or six ways was actually really very tractable. Say you want to pay the person $50 an hour, maybe that seems reasonable for a teacher, But this is not that hard to do. It actually costs less substantially less than a private school. I think the schoolhouse costs something like a fifth or whatever, the cost of an elementary school. Once you got to older grades, you need maybe specialists.

1:14:06It's actually not clear if you do. My friend, Alec Reznik, is working on it. And a very interesting school called Powderhouse in Somerville, Massachusetts, that does something like the model I just described where you have adults who are in more of like a coaching role and they aren't necessarily domain specialists but to look to connect people with domain specialists. So anyway, I would explore something like that model. I'm sorry, this is a little bit vague. If you want to ask about specific things, let me ask you. You got a 12, this child grows up and it's 12. So at this point, you know, it's not just like, of course, you're taught the arithmetic and reading and everything.

1:14:46Are there, do you proceed and you have to learn your biology, you have to learn chemistry, or do you just say, what are you interested in or interested in Roman history? Oh, let's learn about the aqueducts. Or is there an actual curriculum that proceeds until they get to college? Yeah, this is really challenging. So one of the sort of the heroes of the Reform School movement is this philosopher named John Dewey. And he is a lovely book called Experience in Education. sort of written near at the end of his time looking back on all of his efforts to reform schooling in a kind of unschooling Ish direction.

1:15:19He was never as extreme as that, but broadly looking for freedom on the child's part. And he makes this wonderful argument that because these kids, a 12 year old, doesn't have a fully developed prefrontal cortex, certainly doesn't have a fully developed kind of sense of self to let them do whatever it is that their whim commands them to do in any given moment. It's actually not freedom, but rather is is chaining them to whatever that impulse is. It makes them the the subject of these tides of impulse. And I think that's a pretty compelling argument. And it doesn't authorize tyranny, but it also suggests that you gotta be a little bit skeptical about the planning, the plans of 12 -year -olds, I guess.

1:16:16How skeptical should bone be? I don't know. I think I would probably have stronger opinions on that if I had a 12 -year -old. But my instinct as a foolish, non -parent would be something like a kind of mix. I would be interested in exposing the 12 -year -olds. lots of topics and possibilities I would be valuable in expressing the consequences of any particular actions like if they just want to compose music all day we can talk about like well what does that mean what kind of life does that look like I would try to be non -coursive in this as much as is possible and I think to some extent the student should or the child should be allowed to feel the consequences of of their choices.

1:17:00This is complicated by the fact that like, you know, I, again, like, I'm not wealthy, but like any, any child of mine would like have chances, I guess, you know, like if they made some weird choice about a career path when they're 13 and so they didn't get into Harvard or whatever, like that would be okay, you know, like they could do, they could be 24 and finally figured out then or 32 and finally figured out then like it would probably turn out fine. And so this doesn't seem like reliable guidance. You should notice I'm feeling very confused about it. Yeah, that's not worries. Yeah, okay. So one question I have is historically and maybe even to the modern day, it seems like improving education has been a very intractable problem.

1:17:45And you did reference this earlier when we were talking about gearing towards immediate student versus the whatever percentile you're working with. But I don't know. Do you feel like there's been progress even in the percentile you're gearing your stuff towards. And if not, what is the explanation for the relative stasis? I mean, this is something you've talked about. We have so many new tools with IT. Where is the... Well, what explains the broader sort of stagnation here? Well, the fun answer to your question is actually, there's been a ton of progress. Like, actually things are pretty good.

1:18:16One thing is, I think the stat is, in 1900, 6 % of teenagers graduate in high school in the US. Now that doesn't mean that 94 % didn't have an education that we regard as a high school education, but it kind of means that. It roughly means that. Now, these people are homeschooled. It's also the case that a high school education meant something lesser than a substantial fraction of high schoolers now study AP courses and complete them in high school. And that's at the high end. On the low end, illiteracy was a very live situation 100 years ago in the US and is emphatically not now. Now it is the case that something like 10 to 15 % of adults depending on which polls you use are maybe would struggle to perform like simple kinds of number manipulation or reading or writing kind of tasks, but our bar is basically moved.

1:19:20It used to be like, can you read it all? And these tasks are like maybe a little artificial, they're maybe not relevant to their day -to -day, and that's actually why they're experiencing this. So the number of people, the fraction of the population who graduates at 17 or something, knowing a particular amount of stuff has basically moved up monotonically. And this is mostly about the bottom portion of the population. It used to be the majority were effectively uneducated past age 10 or something other than in formerly and in their trade. And really the story of the 20th century has been in part one of mass education, where part of why we have a service economy, IT economy is that basically all of our populations educated at a particular level.

1:20:14If you look at there are these national tests of fourth, eighth, and twelfth grade math and language proficiency, you'll see really like pretty slow movement in the 75th percentile and practically none at all in recent decades. But you'll see like absolutely enormous movement in the bottom quartile. And so in some sense, the story, especially the last 20, 30 years, has been closing what's often called the performance of the chief and gap where certain groups, a part of underfunded schools, or who might have households that are unsupportive or difficult, you know, we're just not having anything like the educational attainment of their peers.

1:21:00And that story has changed. One thing I'm curious about is every other part of the distribution has been moved up has the as a ceiling been raised significantly. Yeah. Well, it depends on what we mean by the ceiling. Because you can go back like hundreds of years and the most learned people around. It's incredible. You look back on how many books Thomas Jefferson read. Yeah. There's some story where Kennedy had a bunch of noble laureates and the hoes of the single noble laureates in the White House in 1963 or something. And he says, this is the greatest collection of genius and insight and wisdom that has been collected into this room.

1:21:41Ever since the time the Thomas Jefferson dined alone. Right. I think it's very hard to raise the ceiling. So the ceiling has aristocratic tutors. The ceiling has whatever family dynamics and and heritable propensities produce tremendous intellectual greatness. Early 20th century schools produced von Neumann. And it's certainly not at all clear that they are now producing more of von Neumann's or something like that. In fact, von Neumann's production seemed to have probably very little to do with any kind of mass schooling that we would recognize. As far as the very top, I think that's difficult.

1:22:29We're talking about an institution that was created for, I guess, the masses. I guess there have always been people who have been using resources outside of those kinds of systems. So the mass system doesn't seem to help those people, I guess. That doesn't seem surprising. By the way, on the von Neumann thing, okay, a mass system doesn't help him. What is the production function for von Neumann? Yeah, so lots of people have studied this. I actually am not a student of von Neumann's history I know that many of his peers the 20th century greats got something like aristocratic tutoring or came from Small Eastern European Incredible schools that there's stories about these things.

1:23:12I actually don't know them. I'm sorry I mean you have sure you've heard about that one high school. Yes. Yes Okay, interesting are are we getting worse at that the bottom -of -the -mortem production or is it just static? Well, maybe. I mean, so I don't know. So let's see. Here's the theory that seems kind of plausible. If someone was going to have aristocratic tutors in the late 19th century, would they now go to a fancy private school? And would that experience now actually be less good for them? I don't know. I think it's probably more likely that they'd go to the fancy private school and also still have fancy tutors.

1:23:49And then go to a very exclusive university where they're going to get a bunch of highly hands -on kind of interaction with professors. Although the reason that might not be the case is the opportunity cost for people who might become teachers or aristocratic tutors is much higher now, whereas a kind of person who would be, you know, your tutor can now directly be making lots of money on Silicon Valley or Wall Street. That's interesting. Okay, so that would be an argument that maybe it's not so much about the 20th century that we've gotten worse about this, but more like over history, maybe Aristotle was a tutor to Alexander the Great, and now Aristotle would be like a full professor and maybe take that job.

1:24:32That might be so. I mean, I think it's maybe the case that some tutors have been priced out of the market, but it's not clear to me that the most expensive tutors actually would be the best. There is a bunch of empirical research on tutoring and one of the questions they ask is what kind of experience level do the tutors need to have? And it's interesting how far you get in tutoring efficacy when the tutor doesn't necessarily know anything. So just having another warm body there actually contributes a very large effect. No, I mean things get better as you get an expert. And I also have a healthy skepticism of these studies.

1:25:09like I think part of the role of having airs to bottle as a tutor is communicating a world view. It's not something that would show up on a test or something that these studies would be measuring. So having an extremely inspiring individual might actually be the important component and inspiring is going to be highly correlated with expensive. I think, not necessarily, I don't know. That feels complicated. I mean, as well as you today, the material is available. What the tutor is ringing is the inspiration and the motivation. not the exclusively, but one of the large parts of their. That's right.

1:25:41They're not really responsible for instruction. I mean, I'll say also, like, I know lots of people who have postdoc tutors right now. You know, these people as graduate students, they're very pleased often to have, you know, a $60 an hour, say, tutoring kind of commission, and that's a little sad, but, you know, the pool of available postdocs to hire as tutors, I think is very large now compared to how it would have been 100 years ago. The pool being bigger doesn't mean that the top 1 % are getting more though. So I think that's undecided. There is a question of have teachers gotten better at their jobs over the last 50 years to say?

1:26:28And there are some ways in which maybe they have. There have been a bunch of projects of trying to disseminate certain research results, like ways of instruction that are more effective in other ways. Like, it's good to interleave stuff, for instance, like rather than doing blocked units where it's like, okay, like we're going to talk about the Civil War and then we're going to talk about women's suffrage. You know, it's better, I think those are somewhat far apart, but it's better to kind of weave these things into each other and not just in history of it in general. So that kind of dissemination has been happening more systematically in the last few decades.

1:27:01I actually am unaware of any kind of studies or results trying to establish anything about the efficacy of teachers now versus long ago. Well, I'm sure you've seen the claim that one of the consequences of the very unfirst circumstance of the 20th century or the mid 20th century was that one of the very few occupations an intelligent woman could pursue was teaching. and now that other options are available, which is obviously on neta, hugely good. You know, there's other competition for those same very intelligent women. Oh, that's interesting. I haven't heard that claim. Yeah, I think I'd have to think about it.

1:27:39I guess it's not clear to me how much intelligence matters. Like if you want to think of that as some kind of separable quantity. Or whatever trade is relevant to just that, you just had a population that was hostage to either housework or teaching. I guess what I'm saying is something like, if that were true and there are like a bunch of people who are now, you know, astrophysicists or something, you know, it's not clear to me actually that they would have been the good teachers. Like being a good teacher is often about empathy and effective communication and care. It's very personal, it's very intimate.

1:28:18Like you need to understand the subject, but to teach a 15 year old or something, you actually don't need to understand it. At a postgraduate level necessarily, it's very interesting to see that there's a bunch of studies of the impact of domain knowledge on teaching efficacy. I've read some in math, I'm sure they exist in all fields. And one of the things that comes up is like, if you aren't very familiar or comfortable in math, then you will struggle specifically to do inquiry -oriented classes. classes that are more about like creative ways of thinking with math or open -ended problems as opposed to like here's how to do this algorithm because to conduct those kinds of classes you have to be able to think on your feet like you pose a difficult question to which there may not be just one appropriate answer and your students will throw all kinds of stuff at you and you have to be able to take that stuff and integrate it and show how one student's answer relates to another student's answer and show how those conceptions can be built upon in order to produce some useful understanding for what you had in mind.

1:29:28Anyway, this kind of improvisation requires a mathematical familiarity and ability, but I don't think it requires anything like extraordinary ability. Yeah, but more than the extraordinary, haven't pulled out of teaching as a consequence. Yeah, I guess I'm just wondering what the correlation is. Like, if it's the case that actually the effect of teaching is mostly about empathy, then maybe it's anti -correlated. Like, the people who are going to be good at anti -particle physicists are actually like, they wouldn't make good teachers anyway. Interesting. Maybe. Why has a hypertext changed how people write more?

1:30:03You know, so often, you know, I write a blog post and I actually do wonder how much different it is with the knowledge that I can add foot notes and I can link to things. But you just hope that, you know, I'm actually a fan of how Wikipedia organizes content. It is genuinely surprising how often the best explanation of a subject is just a psych resource that is trying to explain every single subject. Because I think there's just a practice of you don't need to do exposition in every single topic, you can just hide it behind links and things like that. Anyway, so why hasn't Hypertext change writing more?

1:30:32Online writing at least. This is a really good question. So I think the reason why Wikipedia works as well as it does is that Encyclopedia entries are already forced to stand on their own. And that was true before Hypertext existed. In fact Encyclopedias were already Hypertext -ish before there was Hypertext. There are some other interesting kinds of Hypertext that existed pre -computers. There is this very interesting book called The Scentopicon from Adler that if you want to understand what classical authors had to say about a topic like the father's responsibility to a daughter, you can look that up in the Scentopicon and you will get references across Russo through the Bible and so on and so forth.

1:31:26And those are kind of hyperlinks. something that they were printed on on dead trees, but you're expected to get the books down, look up the appropriate pages. This in TopicCon wasn't that successful. I think it's in part because those concepts unlike the Wikipedia entries, they don't quite stand on their own. So, cleanly, you kind of need sinews, you need linkages. And actually, I wanna make the case that, well, Wikipedia is an astounding resource. I find it rarely to be the best available introduction or explanation of a topic. I find it often to be like a good jumping off point. Like it'll help me know the right thing to ask about.

1:32:01It's good as a reference. So hypertext is a very effective navigationally. It can help you get to a spot that you're looking for very quickly because it's about automating flipping through pages. And so for a reference, it's very effective. If what you have is like a table of chemical compounds and their properties, like a book of chemical compounds in their properties. Hypertext is going to let you navigate that book very effectively. Likewise, dictionaries have been revolutionized by Hypertext. So navigating around the sources by clicking on links to say like, oh, shaded a little bit more like that.

1:32:38It's like a much better phosphorus. So I guess I'm making the case that like there are certain kinds of text that are more amenable to Hypertext because they are more amenable to having the reader dropped in the middle of them. Incyclopedias are like that, dictionaries are like that. But most text is not like that. And I guess most concepts are not like that. I guess most ideas are embedded in something kind of holistic or richer. They require a narrative arc. They're difficult to excerpt. Not everything. But things that are not so raw and atomically informational. So there were all these dreams of hypertext novels, for instance, and some people wrote them.

1:33:19And one of the problems that a Hypertext novel has is actually could be seen in a choose your own adventure book that existed before there was digital Hypertext. And that's that the author is forced to write something like a lowest common denominator story. The page that is the destination of a hyperlink, it has to work as the endpoint of all of its reference. And so it can't establish any kind of coherent or consistent arc, unless there's a kind of sameness to all of the reference, and the more that there's sameness to the reference, like the less useful hypertext is. So a lot of people have been disappointed by this conclusion.

1:34:02I am among them. I'll say that I do find hypertext very useful in my own notes, not really for reading. I actually don't think it makes for a very good reading experience for others But I haven't been a reader you have a separate web page where you have your working notes for the audience by the way It actually is like a very cool UI in the format to explore your thoughts Thanks, it does an interesting thing for me as a writer it lets me build stuff up over time so today I was working on I was reading this This very old cognitive psychology paper on the topic of adjunct questions, which we discussed earlier.

1:34:42The effects of asking questions while you read, not on remembering the information covered in the questions, but actually just on kind of the general effect that it has on stuff that isn't touched by the questions. I have some notes on the design decisions of the nomadic medium, this quantum country thing that I was talking about earlier, interleaving the questions into the text. And those notes are kind of partial. They evolve over time. What was the impact of doing this? My notes about that, they've come from interviews with readers. They expand when I read a paper like this that's relevant to them.

1:35:20And it means that when I go to design the next system and I'm thinking about the role of questions in text, I'll have sort of a place to look. The role of hypertext in this is roughly as a kind of navigational aid. It's possible to do this without hypertext. You just end up with what Lumaan had, a giant, something like a dresser, but made of card files rather than drawers for clothing. This actually goes back nicely to the original conversation we had about why people like Tyler are able to integrate so much information. without an explicit note -taking system. And in fact, I just remembered another person who comes to my mind immediately when I think about a person like this is a Burnt Hill Park.

1:36:05And again, you have an example of somebody who is extremely prolific, a daily financial zither with just like a tremendously detailed and insightful, daily financial newsletter. So it's like, it is like a daily note -taking practice in some sense. Nothing quite accumulates for either of them, at least not in the same way. Right. It's very interesting. like they're doing the whole thing over again every day. One thing I find kind of interesting about in that Levine's newsletter is that when he's talking about a topic repeatedly, like something that comes up, like the recent bank collapse or something, he will have to explain some concept, like interest rate risk, over and over and over again for days, like every day he has to explain it, but every day he explains it a new, and every day the explanation is like colored a little bit by that day, and this is an argument against the kind of note -taking that I do.

1:36:55It's an argument for ephemerality, for like recreating the thing every day because it will change and it will become kind of inflected by what you're thinking about now and your experiences. I think it's pretty interesting. I find myself these days doing the kind of mix. I have, you know, kind of like a journal that's about today and I'll do a bunch of writing and often I'm recapitulating stuff I've written before and I have these other things that are trying to be more durable, be like a useful reference that can stand outside of time. The combination feels useful. I don't yet have a clearer model of when one is better than the other.

1:37:37Well, actually, an interesting way to tie in what you just said with the Hypertext is, with so Burns newsletter is, I've doesn't give that much context on, And often you'll find yourself lost about what is the concept being talked about here if you're not familiar with the topic. And in fact, I asked him at some point, have you considered doing narrations of your blog post like Skydoll Exander has somebody who has a podcast where they narrate his blog post? And he said, I don't think it would work out as well for mine because I heavily rely on the old blogger spears, norms around hypertext where you can add jokes and sarcasm based on when he's ample of this is he was talking to you to write up about SPF and his collapse and he said you know he had a bunch of links like you want to learn more about margin calls, read this you want to learn more about this and he goes and if you want to learn more about the psychology of utilitarian bets read this and it was just a link to the Amazon page of crime and punishment.

1:38:37So just that kind of stuff is harder to do. Yeah you're right so he's leaning more on his past explanations which is interesting because he can't update them like that format of writing a newsletter and then linking to past newsletters. Or as you say, this is sort of a former blogosphere thing to do. You have a series of six words and like each word is a link to a previous post -photography. I certainly have written stuff like that. It's kind of funny. It's approximating the durable note thing I was writing about, but without the ability to revise it over time. And maybe for many topics, you don't need that ability.

1:39:11It's certainly the case that, well, I wonder now, Now, what fraction of my notes are basically in the state they were when I did my first major revision of them. It's probably at least a third. It might be more than a half. What percentage of your notes have you published? I don't know. By word count, by note, so like for instance, my journal notes are not published and there's one of those every day, so there's a lot of them. So if we're looking by note, we're excluding all of those. I also have a note about all of the people in my life, and those are for the most part, not public, unless they're public individuals.

1:39:49So there's a lot of notes that are not public, but they're mostly not durable. They wouldn't be all that meaningful to others. The journals might be, but they're also intimate. Are they written in a way that would be intelligible to, if you were to give that to somebody else? It depends. But usually, actually, My journals are like complete sentences, complete paragraphs usually. Sometimes bullets, sometimes kind of veering and breaking and changing to new subjects suddenly. But they tend to be filled with links to the things that I'm talking about in part because I'm trying to accumulate context in those things.

1:40:27How come there and why not just short -hand? It's partially because past me is another person. It's kind of a cliche, you know. but I am routinely looking at journal entries from a year ago. This is partially like, you could view that as a failure of this note writing system. Maybe in some ideal sense, I shouldn't be looking at these journal entries because if something's important and it's going to be something I refer to a year later, it should be in some durable evergreen note. I don't know. You don't always want to do that. It feels like prepping. Maybe there's an amount of prepping that's good.

1:41:06We live in California and maybe everybody should have an earthquake kick, right? Maybe that's good. But maybe you don't need to hoard 300 cans of beans. So there's an amount of prepping that feels like a reasonable amount to do and there's an amount that feels kind of dutiful and unpleasant. As a researcher who is in these sorts of Silicon Valley circles, what is your opinion on the startup advice of do things fast, fail fast, get to users immediately with an idea of MVP as as somebody who was making products, but is also in a sort of in a different mode than a typical startup kind of making products.

1:41:41How do you think about advice like that? I have complicated feelings about this. I need different advice on different days and of course different people need different advice on different days. But when I was getting into this kind of work, what that kind of advice led me to do practically speaking is to not think all that deeply about the ideas I was exploring. and to look to basically like an idea would come up and I think like, oh, I can try that. And then I would try that. And then I'd learn something and then I'd repeat. And there wasn't this sense of building a theory of like what the problem is and what it would mean to solve it.

1:42:21Instead it was just a theory of action. Like a theory of action is supposed to a theory of change is if you imagine like you're at some point, your current position, and eventually you want to get to some coal state. A theory of action is you look around you and you say like, well, what can I do? What can I build? What do I see as possible? And a theory of change is to look at the end point and try to work backwards. Now the metaphor is imperfect because in research you actually usually don't. Exactly what the end point is, and you certainly don't know how to work backwards. But I guess what I'm saying is that following that advice historically often has led me to try things that were straightforward.

1:43:02I think the most powerful design work has ideas in it. What makes a nonlinear text editor, that is like the text editors that we all know and love. So powerful is this observation that writing is a nonlinear process. But writing with a pen linearizes it. And many, many other observations like that and on the nature of what it means to have a thinking environment is how we got that particular interface. Likewise, the way that we got powerful programming environments in so far as we have them is by people thinking very hard about what it means to specify a system and coming up with new primitives that express those ideas.

1:43:58I think the most powerful interfaces are often the expression of new ideas or new primitives that capture new ways of doing, new kinds of objects that can be manipulated. In Photoshop, for instance, you can manipulate a photo by means of a construct called a layer. This is a very strange idea. It has some precedent in dark rooms where you could potentially have kind of like sheets of film. I don't mean like the negatives. She eats of like gels that you could potentially put over the lights to affect the exposure and to make there be more exposure here and less there. But in Photoshop, they're non -destructive and they're continuously manipulable.

1:44:44And the layer is like a new primitive that is introduced into the activity of photo editing. And it utterly changed what you could do in photo editing. So I guess what I'm saying in a very long winded and confused way is that I think it's difficult to have ideas by means of building an MVP very rapidly. Now, if you have an idea that you think is interesting, it is good to test it rapidly. And so, part of why I'm confused is, in my response here, is that it's good advice. Once you have something worth testing, it's just that for me, adopting that mindset, and I've lived in it for so long, that it's very ingrained in me, it makes me not sit it in stillness and in confusion and in contemplation with the ideas long enough for them to be good.

1:45:41I mean, very concretely. So, Michael Hilsson and I made this quantum country thing. When I was trying to think about what to do next, the most obvious or natural idea was like, well, what if we just tried that with lots of other things? And that idea occurred to me, and the pandemic had just struck. So, I was feeling a little timid, I guess, creatively or emotionally. I wanted something that felt kind of safe and I knew that I could do that. Like, okay, I can build a platform that generalizes this thing that we did for this textbook. So I did. And I did it relatively quickly. I did it in a few months.

1:46:14And that wasn't the right thing to do. It wasn't really the right question to be asking. The idea wasn't that strong. It wasn't the right way to test it. Like building this highly general version of it, I would have been better building more one -offs rather than like a self -serve thing that anyone could use. And this comes down to the difference in aim. I'm not trying to build some kind of scalable thing for the world at this moment. I'm trying to build the idea. The prototype is an expression of the idea. And once it arrives at a good place, then maybe there can be some scalable solution. But it's not necessarily at that place.

1:46:52And until it's at that place, there's a lot of thinking and sketching that goes along with the building and prototyping. I think part of my confusion here is that often I still need to hear this advice. Often I will just time myself in knots in theory land. What I really need to do is to have a friend sit me down and say, is there a piece of this that you can carve off and build next week? You're hearing a lot of tension. Interesting. What was the consequence of shipping orbit out before it felt ready to scale? I mean, I learned some things. It was fine. Like, it taught me a lot about where that particular format succeeds and fails in other venues.

1:47:32It was just not a very effectively to find those things out. I built this like very general. It was an MVP in the sense that it has very few features and it's very simple, but it was highly general. I mean, it's like, it's a deployed thing that has infrastructure and has accounts. that has all this stuff that you do when you're building a real thing. And that's very different from, well, let me work with this one author and see if I can make it work with this one other book. That's very different from Poin and Country. It's a form like a specific question or a specific theory about, well, it worked for this text.

1:48:06What's the next kind of text that would be good to test with? And then to do that, I could potentially do it. I certainly could have done it much more rapidly. Why do you think that this idea of tools of thought has nerfed so many people in Silicon Malley? Well, it contains this message for technologists that they can potentially be very powerful, and that's always tantalizing for people, I guess. I think it also feels very actionable for people in a way that's actually super misleading. I mean, so I meet tons and tons of people who tell me that they're interested in tools for thought and 95 plus percent of them are engineers.

1:48:47And the problem with this is that building an interesting tool for thought is basically entirely a design problem. And their design ideas are usually not very good or are troubled in a variety of ways. And yet they can make a thing. They can make a thing that solves a problem maybe for them in their lives. and that feels very tantalizing or encouraging. It feels like something that gets their hands around it, I think. I think we in Silicon Valley are very, very interested in thought. We are like a stinky people. And people are very interested and engaged also with anything that could potentially expand our capacity there.

1:49:27And so that too is tantalizing. Like what if I could think better? It's also tantalizing because it's meta. So there's all these clichés about people, tinkering with their dot files endlessly or tinkering with their blog website Which has two posts on it, but they have to rewrite it because you know, they want to do something else and now the new one I'll have three posts on it before they rewrite it again Tools for thought also scratch that itch It's work about the work This sounds very cynical by the way. I don't I don't mean for it to be I was trying to earnestly answer the question But okay here's a more optimistic and generous response I think many of us got into computing because computers portray a sense of personal empowerment and possibility.

1:50:15We maybe remember growing up and being locked in our bedrooms at midnight or whatever, like fooling around and we have this very powerful tool at our disposal and it's opening up these worlds for us. And I think for many people here that was like a formative part of their personal development. And so anybody pointing to that and saying we can do more stuff like that is going to be pretty compelling. Okay, this was an interesting question from Matt Clancy on Twitter. What are the characteristics of a good crowdfunded research project? One of the maybe unfortunate things that I've learned in my crowdfunding experience is that there are some dynamics that seem hard to change.

1:51:04So one of them is at turn rate, you know, like any subscription kind of revenue business model, I guess that's what I have. You lose subscribers every month. In my case, it's about 2%. And it's not that large, but it does mean that I need a certain number of new subscribers all the time. And one thing I've learned that's kind of interesting is that the trend rate is surprisingly insensitive to anything that I do. I've experimented with a variety of things and it really hasn't meaningfully changed the trend rate. What does change things is getting more people into the top of the funnel, in other words marketing.

1:51:39And there are some things that have maybe affected the fraction of those people in the top of the funnel who convert or whatever. I really hate this way of thinking about it. But in summary, the thing that I've discovered that's kind of sad is that I ended up having to think about this a little bit. In particular, I realized that this project only even slightly works, the one that I'm doing, for crowdfunding, because it's understandable to others and it's interesting to others. It's already in a place where there's some results that maybe look promising or people are like, oh, more like that. But it's very easy to imagine other projects that are not broadly applicable.

1:52:19If I were doing marine geology stuff, I probably wouldn't have a big crowd of internet people not nearly as large anyway. We're excited. So that's one property. This work is very general. It applies to many, many people. It applies to people who have disposable income. So if I were doing a research project on, I I don't know, like, writing practices of disadvantaged artists. Like, I don't know. Like, I think my audience might not have as much disposable income. I have already made some progress. I think that's probably important, unfortunately. So it's probably very difficult to use crowdfunding in the very early days of a research project.

1:52:56I've already sort of chosen a research agenda or direction, like, and I can kind of express it. So I think this says that like, crowdfunding applies probably after the first few stages of research have been completed. There's probably something like the standard grant advice where like at some point here I'm going to be using this crowdfunding like to figure out the next thing and I won't be able to explain it to anybody and there certainly are seedlings like that but you have to have something in flight. I probably need to be able to say something about my progress with some kind of regularity.

1:53:31So for instance, my wife is working on this study of biological markers of age in association with a delirium in traumatic brain injury. And to do this, she is basically signing up patients who show up to the hospital, who have traumatic brain injury, and once they agree to participate in this study, taking various blood samples and things like that from them. And recruiting enough patients to get like the significance that she requires will take like, you know, two years or something like this. She can report a little bit of intermediate stuff, but like certainly not a monthly update or something like that, right?

1:54:13Yeah, weird page gun post. Yeah, I mean, I can't quite report monthly updates either, but I think like there's a cadence that's necessary. Why bother with it at all? I'm sure there's many, what wealthy individuals would be happy to single hand if they find your research. Is there any such shows crowdfunding? Well, those wealthy individuals are very welcome to reach out and offer to do so. You know, I will say I've been fortunate to have many high net worth individuals as sponsors, but I guess each of them is providing, I guess on my Patreon, a sponsorship is $100 a month. So that is what I get from these people.

1:54:55I'm certainly not getting wild offers for more. I think you're using the wrong tool going to distribution of your given the wealth distribution of your items. Maybe. There's a couple of ways to interpret your question. One question is, why crowdfund when I could appeal to high -network individuals and another version is like, why crowdfund at all is supposed to raise in grants or talking to philanthropy or whatever? Are you mostly focused on the first of those? Yes. If I'm going to be honest, it's because it has worked. The history of the crowdfunding of this project is like many things in my life, the result of goading from Michael Nielsen.

1:55:32Early on when we were working on this quantum country project, he suggested we set this up and I kind of hemmed in hot and I said like, yeah, you know, it's going to be distraction. We don't really need this right now. Let's deal with it later when we have something to show. He's like, no, no, no, no, let's just get it started. it's going to be a long time to get enough subscribers and so on. And it turned out he was right. You know, the process of crowdfunding a project is, it takes like maybe a couple of years, at least in my experience, to build up a subscriber base and starting earlier was better.

1:56:02And if that hadn't worked or if we hadn't started early, I think I probably would have just reached out and asked for individual help and I probably will if it fails on me. I'll say also like when there have been specific projects that I've wanted to do that require say hiring people, I have reached out to high net worth friends and they've helped but you know, and kind of like the low five figure or four figure kind of range. And that's great and I'm very grateful. So I guess the answer may be a mix. One of the big limitations to the crowdfunding thing is it seems pretty clear to me it can't sustain a team or an institution or anything like that, they can barely sustain me.

1:56:47I earn somewhere between a grad student and a junior faculty member. That's kind of okay, I guess. There's a variety of reasons why that's okay for me that are pretty particular to my circumstance, but it certainly wouldn't be okay for everybody. Even for me, it doesn't allow me to support others. Right, and it's even more, I guess, striking because in terms of the success of what a public intellectual is basically in some sense, you're like a public researcher in this sense of like some of your researches, or you publish your research in a public -facing way. Even in a context in which you're like pretty well known, especially amongst the kind of audience would be happy to find this kind of thing.

1:57:28If the LeBron James of independent public research is like between a grass student and Yeah, it's it's another great sign in general for that I think it's worth considering that I'm also maybe not pretty good at this like I mean so first off like I'm not that successful as a researcher like I guess like kind of objective at the LeBron James characterization It's true that I'm maybe the most successful crowdfunded researchers and stuff and that's kind of weird But like the last couple of years, you know, I've like figured some stuff out but I guess I wouldn't say I've had any spectacular hit kind of publication kind of things.

1:58:02One thing that is true of this is that when I have big publications, I get a lot of new subscribers. So I think there is some kind of market force that could be higher if I were having a more spectacular successor with my research. I think it's also true that I, is pretty systematically avoid marketing it. That's kind of a self -protection thing like I am really worried about the corrosive influence of audience and marketing on inquiry, honest inquiry. Okay. It is very easy, very, very easy to distort my work. It's almost a default to try to make it be something that people would be more likely to like rather than the thing that I actually want to investigate or to do the boring simple version of it rather than the interesting deep version so that I can publish more stuff more often.

1:59:01Like one thing that I've chosen not to do that is a choice that's definitely costment financially is to publish what academics would call like minimum some of them, viable units of paper or something like that, so that they have a pivier phrase than that. Minimum viable papers. It's very common to take any new marginal insight that is kind of above a particular bar and publish that. It's a little thing and I just haven't done that. I've written little informal letters to my patrons and I'm like, yeah, I figured this thing out this month. And like a Bower and Academic, I probably would have published that as a paper.

1:59:36And if I were a marketing -oriented, crowdfunded researcher, I probably would have done some glossy thing and promoted it and whatever. Like, look at this thing I figured out. But actually, I just don't think it's that big a deal. And I'd rather get on to the next thing. I have that choice, I guess, of waiting to publish. But that's not really what I'm worried about. Really what I'm worried about is marketing, man. Marketing, it makes it so hard to be honest with oneself, at least in my experience, not only to be honest as I said with what I think is interesting and what I think is important, but even to be honest about the results, every paper is in some sense a little marketing piece trying to make the case that it's significant, that its results are really exciting and really important.

2:00:21And that is really corrosive to discovery. I mean, it's true that you need a really strong emotional connection to the work, I think And part of that emotional connection comes from a sense of excitement of maybe being hot on the tail of something really good. But there's a temptation to kind of portray what you found in the best possible light and to kind of downplay its limitations, and to take up space and to totalize. And all of this is just, I think it's just death for discovery. It is interesting to hear that from somebody who inadvertently and without intentionally trying to do so has done a good job of spreading your material.

2:01:08You know, I've known about you for a long time, so but I do wonder if there's an element of I think if you get to a certain level of quality after that trying to mark your stuff, not only doesn't help, but probably hurts you. If you can try to think of somebody like, I don't know, Gourne trying to like post YouTube shorts of his blog posts or something like that, it would just be like, what are you doing man? Where's that thing? It's just so good that he doesn't need to promote it. Gourn is an interesting example because there's a simpler failure mode and that's that I still routinely run into people who will tell me like, oh, I've really liked your work for a while.

2:01:40I didn't know you had a Patreon and that's kind of like a simple failure of a certain kind of marketing on my part. And I think Gourn actually has this even worse. I adore Gwern. I have learned so much from him. It is the case. You can go to his Patreon page, and he actually makes public his revenue. He makes on Patreon a tiny fraction of what I do. I think this is inappropriate. Gwern is much more impactful researcher than I am. He is a much bigger audience than I do. The fact that they aren't converting into patrons, I think, is mostly a matter of the way talks about it and the way that he presents it.

2:02:23And it's not that he needs to market more people to his webpage. I expect he has plenty of traffic and a plenty large audience. I think it's much larger than mine. I think it's more just like there are a bunch of variables about the way that you talk about this membership offering and not of us really want to think about them. And I've ended up at a slightly more effective part of the space, but I'm pretty sure that there's like much more effective ways to do whatever it is I'm doing. Yeah, this is a really interesting problem because I have a sub stack where if people choose they can help contribute to the podcast and Well, it's broad enough revenue to help you first learn episodes and traveling in comparison to now that I'm going to be doing ads in comparison to well -being making in ads it's like a small fraction and Which is a some people might say it's unfortunate that you have to do ads and I I'm maybe that listeners will just be finding out for the first time that there was an option on Substack.

2:03:20But also you don't want to be in the position where you're asking listeners for money every episode, right? Yeah, I hate asking people for that. I think this is a common issue for creative people. I hate it. I really hate it. I probably need to get over this. I do want to make one point though. I had much more success with my Patreon when I recast it as like, like, oh please support me, like subscribe to support my work. Like the thing you were describing, to something that I guess feels slightly more like, I don't know, in offering, like become a member. And like when you become a member, like these things will happen.

2:03:56Where like these things are not terribly substantial necessarily, but I guess what I'm saying is like, there's a difference between a tip jar and a membership in people's minds. And like becoming a member means something, and if you could offer something small that feels membership -ish, you might get very different results. And Gourne has the tip jar vibe. And these days have kind of a member vibe. And my instinct is that if you were to move to become a member of Gourne's lab kind of thing, he would have better results. Well, he has a thing on Patreon where it's like, if you donate five bucks or eight bucks, he will read an entire book and review it.

2:04:37Yeah, this is crazy. I don't know if anybody's ever taken him up on this. Yeah, but I mean, that's like valuing his time at a dollar. Yeah, yeah. I don't quite understand this. I mean, I think it's also the case. Like you'd probably have an easier time asking for subscriptions if you had a larger audience. First, like you can build the audience for free and then kind of have some bonus offering or something that's behind a wall, maybe. I feel very conflicted about this actually. Maybe you can help me think about it. Well, I just, I have all these patron essays. It's like where most of my writing is these days, because I'm kind of waiting until I can collect enough things for the next big public piece.

2:05:18I have a couple of like big public pieces and various stages of flight. And so anyway, I mean, I'm writing a lot for patrons. And I think probably much of my audience for people out there, like don't even know that's there. So like one challenge of memory -only content is even making clear that it's there to others. And often people will try to achieve this by tweeting about or sending newsletters out about this subscriber -only content. And I just can't bring myself to do it. It feels terrible to say, like, oh, here's a link, but you can't view it. Yeah, I can't do it. I don't know how you think about this.

2:05:55Or if you think about subscriber -only material for Lunar Society. I was actually just about to mention this to you, which was, you know, I'm a patron, and I got a chance to read all your page. Thank you, you're welcome. Only, page and only essays. And they're great. And I was actually thinking while I was reading them, it's like really unfortunate that a person might not know they exist or if they're not familiar enough with their work to go ahead and sign up, it just like behind the page you're on. So it's a shame that the way to fund public work is, or run of the ways to fund public workers to make some of that work less public.

2:06:28I think it would be, yeah, there was some way to make this. I think there are better ways to do this. I think there are design solutions. So for instance, if it were the case that my work was kind of mostly all in one place rather than in these separate places, and the subset of the work that's public was kind of visually and structurally adjacent to the subset of the work that's private, it would be clear that there's this additional stuff that's available and perhaps you can see the first bit of it. Substack has this kind of stuff to get some sense of what it is that you'd be seeing, I just, I've invested like zero effort into figuring out the inappropriate presentation of this stuff.

2:07:07Right. And then also another thing to consider is to the extent that this may not be the largest thing you're here about, but it is a factor is that a big part of the impact of at least your writing work, one of the things in that equation is how many people will actually consume it. Yeah. And the expected value on that is dominated by the probability goes viral. Sure. And you just can't go viral if it's on. Like, for example, I think you're a recent post on, you had this really insightful post on, based on your experience in industry at Apple, about the possibilities of a Vision Pro and in what ways it's living up to and not.

2:07:42And I think like that would have just, Oh, thanks. I did make it public. I put it on Twitter and it was on the front page of Hackernoo. All right, okay. I think you're right. And like usually I don't want this stuff to go viral. Like it's in media's race. It's, I think that the primary value that most of it has for people is kind of opening up a window into a particular very unusual kind of creative work that they don't normally get to see the behind the scenes of. And most of it is kind of context, laden, it's not really free standing and I don't really want to write it as if it could be free standing.

2:08:17I've occasionally had the experience of one of these kinds of things getting widely distributed and then getting all these comments of people like just like being kind of I'm so angrily confused about what I'm even talking about. That's kind of discouraging. I guess all of this to say, when I want to write something for broad public consumption, I write something for broad public consumption here. Okay, I've got some questions for Twitter, from Twitter. Okay, bring it on Twitter. This is actually another question from Matt Clancy. Are there other examples of beneficial knowledge work practices that perhaps mostly work because they are a form of space repetition practice where the participants don't realize it?

2:08:52Yeah, I mean, I think this is like, this is embedded in our working world. So for a researcher, when you need to write papers regularly and you're writing those background sections and you're repeatedly explaining the history of a particular line of research and citing the appropriate sources, like that is a kind of space repetition. When you have students and you're mentoring them in conversation about like, oh, in this kind of situation you really need to remember to do X. That is a kind of space repetition. And all of this stuff is it's kind of accidental. Doctors have grand rounds when even when they're not seeing patients regularly, they're still exposed to other patients.

2:09:37And there's often a structure in this where like while the patient is being presented, you're supposed to be kind of trying to think like, well, what I think to ask, like what would my differential that you'll be before you hear it. So there's like covert retrieval happening. So I think it's like it's everywhere in our world. And it's spaced and it's repeated. The thing that differentiates the kind of formal practice that I've been exploring, I think it's mostly, it focuses on material that you wouldn't otherwise normally have repeated either because you're too early with it to have a consistent practice or because it's just not firmly tethered enough in anything in your life.

2:10:16This is a question from Ian Bonagas. What is the optimal amount of effort that should go into a personal website? And I think you might have noticed the amount of CSS that exists on dooshad .org. Yeah. Which is very beautiful. But I don't like it. Oh, no. This is what everybody says about their website, right? You know, it's three years old. That means I want to redesign it, but I will not allow myself because it feels like a distraction. Yeah. What's the right amount of effort? You know, I mean, there's no general answer to that question. Of course, that's going to be my answer. But what can I say about it?

2:10:49What's the job of the website? What's it trying to do? I think many, especially engineers, do themselves a disservice by fretting over their websites unnecessarily building vast technical infrastructure when really what they want is like a place to post a markdown files. And they're better off just like getting a ghost installation and going to, I think the main thing to think about is like, what is it that you want to put out in the world? What is the ideal form of that thing? And to try to find some way of organizing and expressing that, we have these common patterns, like the blog or a portfolio.

2:11:29And often people end up kind of forcing themselves into these patterns. People will end up using blogging software to make something that's kind of durable. And I think like very interesting personal websites often come from people who are thinking about that question, kind of the shape of the thing that they want to put out into the world and making something that speaks to it. Often once you understand the shape, making the thing is not that effortful. My website was not a not an enormous project for me, probably should have been a slightly larger one given that my income depends on people coming through it.

2:12:04The working notes with the, I mean, that was a weekend. Really? Yeah, I feel kind of bad about it, because it's made its way into tons of commercial projects now. And people are like, oh, this is like, this is the way to present network notes. And like, there's actually, I think it's not very good in a variety of ways. I spent like a couple days on it. Wow. I thought this was where the question was alluding to us. You were supposed to spend months on this. No. Wow. Oh. I mean, it is a little bit of like the, you know, the thing about the mechanic, like hitting one thing and kind of knowing the thing, like, okay, I have design intuitions that led me in a particular direction, but there's lots of things I don't like about it.

2:12:40I just haven't allowed myself to spend any more time on it, because I just don't think it's important though. I have a question about, actually, your time at Apple before I asked if I'm a Twitter question. You know, like everybody have iPhone, and from the outside, there must be so many different trade -offs and constraints. When a thing like this is being designed, you know, what is this apply of certain components and the cost, what are different consumers want, what features is the R &D team ready to put forward. And then at your time in Apple, you were responsible for a lot of these cornerstone design features.

2:13:15How is all that information integrated where guys are taking all of these constraints into account, this is the design. How does that happen? One thing that's very interesting is that it's very compartmentalized. And basically none of what you just said was relevant to me. It was all pre -specified. So the thing at Apple is you have a little domain that's your own and the boundaries of that domain are determined by every other, every else is a little domain. And so there's a person who's responsible for thermals. Actually, there's a team that's responsible for thermals. And they kind of figure out, like, okay, like how can, I guess, what is our thermal budget?

2:13:52How much can we have the CPU on during what kinds of working situations? and I basically can't argue with that. Like those are just my constraints. But aren't those constraints informed by what's differential problems? It is iterative. So we'll run into stuff where like, oh, there's a thing we really want to do. We can't pull it off because like it drains power too much. Right. So hey Siri is an interesting example. To be able to activate a voice command at any time without interacting with the device is great. And people prototype that just like having a thing listening in the background and like watching for it.

2:14:27But that requires having the main CPU on all the time, like processing audio buffers and like you simply can't do that, it trains the battery. And so that attempt led to eventually having a just dedicated co -processor that runs at a lower power and it's very limited and restricted and it can be on when the main CPU is not on and it can and listen for that sound. Is there a person who's job it is to take all things into account and be like, I decided, given the members from everybody that thermals, you guys need to work on this, you guys work on this? Not exactly. It's a little more push and pull.

2:15:06So like a given team, usually some of their priorities will be internally determined. Like the thermals team has its hobby horses and it knows what it thinks is important and some of them will be externally determined. There is an executive team that makes ultimate decisions about like, you know, the main priorities for next year's devices or whatever. Like, ah, next year we're going to do this like face ID thing to unlock the phone. And we're not going to have a home button. Like, okay, as soon as you, like, if you want to not have the home button and you want to have the screen go to edge to edge, like, this has all of these impacts like top to bottom on the device.

2:15:40So that decision creates like lots of necessary work for lots of teams. But some stuff is kind of handled at a, I guess, a more local level. looks, for instance, more locally to the iOS team rather than at the top of the executive team. The director of iOS apps might decide, like, we have this problem that because the apps were built at the same time as the system frameworks, we end up building our apps using this like weird Frankenstein, like partially internal framework, partially the public one that our developers use. And the internal one is always a little bit different and it's not always maintained reliably.

2:16:22And so we have all these problems about the skew between the two. So like a big priority for us is going to be to rewrite all the pieces of our apps to only use the public bits so that they could be distributed on the app store. And that's kind of like a more local decision. What I find really interesting about this is that it's possible for a two trillion dollar company to integrate all this information to have a cohesive hierarchy where assuming different products, so many different trade -offs are being made. Does that make you think that over time these very well -functioning tech firms will get bigger and bigger that they can't actually handle the cost of having this much overhead?

2:17:02Let me first just respond to this observation about the enormity of the company and then we'll talk about the other firms. I think the reason Apple is able to do this is because of the way that they delegate. So while there is a very strong command and control structure and for important decisions that they really are made by a small group of people at the top, the individual leaders in the various areas at all levels of the hierarchy have an enormous amount of latitude and that's the only way that any of this can work. So individual people are given very, very strong responsibility and authority within domains to make decisions.

2:17:42And that's how you can have all of these disparate products like Craig Federiggi's head of software and Apple. What does that mean? How can you be head of software? How many platforms do they have? You know, iOS, iPadOS, watchOS, VisionOS, MacOS. Also, there's an operating system running in a bunch of the cables, like on little chips in the cables, right? And like, all of that is undercreg. What does that mean? You know, like in practice, what it means is there is a set of software concerns that he's actually super concerned with and he's thinking about day to day like when I was at Apple I had Craig Federie in my office like talking about gesture recognizer heuristics with me because like that was something that was hyper salient to him.

2:18:20At the same time, he was basically completely ignoring, you know, 95 % of software related decisions and he just fully delegated those things to others. There's a really interesting Harvard Business Review piece from a few years back about Apple's management structure and about how they have a couple of different concentric rings of kinds of responsibility for any given leader. There will be, I don't remember exactly the breakdown, you know, call it 5 % of things that you're responsible for, that you have your hands on at all times and you are like directly manipulating, controlling. And then there's, you know, a ring outside of that that's a little bit bigger.

2:18:57Those are the things that you're keeping that eye on, right? So like they are salient to you. You're getting reports on them. You are checking in on them. You are thinking about them. You're coming up with ideas and sending them down the chain but you're not directly controlling them. And then there's a bunch of stuff that you've figured out how to delegate and you want to hear if there's problems. And they talk about how that structure is evolved over time. It's been now eight years since I've been at Apple and so I'm sure it's practically unrecognized to me. This is a question from Basel Halperin on Twitter.

2:19:29Is the lack of space repetition adoption a market failure or is the lack of adoption efficient? I think it's probably mostly efficient. So in places where space repetition, as it stands without substantial like novel cultural knowledge that's difficult to transmit and isolate, where all of that is valuable, we see a lot of space repetition usage. So among medical students, or highly motivated, have lots of reason to study, and the material is shaped in a way that's highly amenable to space repetition usage. There's tons of space repetition usage. In fact, the Med student -onky subreddit is like bigger than the onky subreddit.

2:20:08Likewise, among language learners, space repetition in various forms is extremely common. Duolingo has space repetition integrated into it, and space repetition is kind of naturally present in the process of immersion learning. In fact, modern space repetition tools between the lightener box and Wasnac Super Memo, they were both originally motivated by language learning. So in language learning, there's like a substantial market for space repetition. It can probably be used in a variety of more creative ways. For instance, Russell Simmons has pointed out to me that studying individual vocabulary words on flashcards often misses kind of integrative opportunities, which you really want is to kind of study lots of sentences or something like that or possibly to build up towards that.

2:20:54Do a link or do something kind of like that and people in space repetition for language learning subreddits mostly don't. Some of them do, it's kind of complicated. So there's edges at the market where you need early adopters to kind of try things that have rough edges and the early adopters sometimes they get cut and they bleed a little bit. And so that's why people aren't rushing into it. As to why space repetition isn't widely used, for instance, to like learn quantum physics, I think it's basically correctly priced. You know, I can use space repetition to learn quantum physics a bit faster.

2:21:27It doesn't make it a fail -complete or anything like that. It's not like learning anatomy where basically if you study the deck you'll be done. You need some more stuff and I'm working on some of that stuff. And also you need like an incredible amount of very unusual knowledge that's largely tacit at the moment in order to use it in that way. That's part of what motivated recording this other video is to kind of show some of that in action. So the fact that the market isn't acting on this thing that it kind of can't really act on seems pretty appropriate. That's I think that's a good place to the tie off that other collaboration and Table on this this project.

2:22:03This was really interesting. This was a much. Yeah, this is many hours of just insights and What's the food for thought? Wonderful. Thank you. Yeah, things are coming on Hey everybody, I hope you enjoyed that episode. As always, the most helpful thing you can do is to share the podcast. Send it to people you think might enjoy it, put it in Twitter, your group chats, etc. It just blitzed the world. Appreciate your listening. I'll see you next time. Cheers.