Ep. 370: Deep Work in the Age of AI

23:47And as the weather cools down, I tend to wear sort of like athletic type pants. I'm often on camera from the waist up. It doesn't matter what I'm wearing. I mean, I'm wearing bikini briefs right now because it's hot. You got to get people to want. Cozy Earth has a new product called Bamboo Joggers. So that sort of like casual pant in their super comfortable style fabric that we already know and love. You can wear their irresistibly soft jogger pants just about anywhere. They're made from that viscose bamboo fabric. These joggers are the softest staple wardrobe. You're going to, one of the softest staples you're going to have in your wardrobe, whether you're staying in or stepping out.

24:28And what about if you're going to like a slightly more formal affair? So like, you know, me going to the Lincoln bedroom to investigate their sheets. Well, they also have you covered with their everyday pant, which I have a pair of. And I also love, they, they look snazzy, but they're also like super comfortable with that fabric. So cozy earth nailed comfort with their bamboo joggers and their everywhere pant. That's next level. Everyone I know who is trying these things out says this is really comfortable. So I'm a big Cozy Earth booster. You should be too. Go to CozyEarth.com slash deep and use my code deep for 40 % off the best pants, joggers, shirts, and everything.

25:01And if you get a post-purchase survey, tell them you heard about Cozy Earth right here. Built for real life, made to keep up with yours, Cozy Earth. That's CozyEarth.com slash deep and use the code deep. This show is also sponsored by BetterHelp. We talk a lot about the different types of advice that might help you be more organized or make more progress on the things that might matter. But you know what advice is probably the most important of all? You need to cultivate a healthy relationship with your own mind. Everything else rests on that foundation. If you want to improve this relationship, therapy might be exactly what you need.

25:40A professional who can help you understand your own mind and improve that relationship. BetterHelp makes therapy easier. With over 30 ,000 therapists, BetterHelp is the world's largest online therapy platform. They serve more than 5 million people globally. It works. They have an average client rating of 4.9 out of 5 stars for a live session based on over 1.7 million client reviews. It's convenient, too. You can join a session with a therapist at the click of a button, helping you fit therapy into your busy life. Plus, you can switch therapists at any time. As the largest online therapy provider in the world, BetterHelp can provide access to mental health professionals with a diverse variety of expertise.

26:18Talk it out with BetterHelp. Our listeners can get 10 % off their first month at BetterHelp.com if they go to BetterHelp.com slash deep questions. That's BetterHelp, H-E-L-P dot com slash deep questions. All right. Speaking of questions, let's move on to our own. All right. First question is from Sam. I found that using ChatGPT questions can act as project diaries. I now keep multiple chats for projects I'm working on. It enables a shutdown and a place marker for where I need to restart work. Can ChatGPT chats act as single-purpose notebooks for projects? I mean, sure. A couple of questions here.

27:07what is a project note like this idea of like i need notebook to keep track of the progress on on my project was that something you're doing before i guess it's useful if you don't know where you left off and you want to have external notes on it but here's my concern my concern is that you're doing something like cybernetic collaboration with chat gpt we talked about this in the deep dive when you have this running list of notes and conversations with chat gpt key. I'm concerned what you're doing is transforming your work into something where you go back and forth with the chat bot. You hand off more, you know, have them do some stuff and you look at what they did and you ask a question, you go back and forth.

27:43This is more pleasant. This feels nicer. It makes work seem better. But just as the meter study found with programmers, if it's reducing your peak intensity of focus or the sustained duration of your focus and you're doing anything that's non-trivial, it also could be slowing you down and hurting the quality of your results. The key with any type of effort that requires deep work is not reducing the difficulty of the deep work. It's not reducing the intensity of focus. It's setting things up so that you can reach high intensity of focus. The focus mind is what produces value. So that's my only issue.

28:15I would be wary if you're trying to make work in this cybernet collaboration that might not be making you as productive as you think, even if it feels nicer. Last week, you You talked about leaving a narrative for personal projects. And originally, I think this question was in relation to that. But now that you talk about the cybernet collaboration, it kind of makes sense. But if you could touch on the narrative part too. Yeah. So like understanding. So what I meant by that is like understanding how to hit the ground running. So, you know, if I'm writing something, I will end a writing session.

28:47It's like, okay, I've gotten up to here. And then I have a few notes. I'm like, okay, here's what comes next. So if I could like, I'm missing this example. But if I could get this key example, you can – we could finish this section and try to get to the end or something, right? So it's a little note I left for myself so that when I load up that writing project again, I know what I'm doing. I would do this a lot with mathematics papers as well, right? I would put notes right into the document where I was typing in my results or I would email these to my collaborators and be like, OK, well, here's where I'm stuck.

29:15But I think here are like three ways I was thinking might be useful for getting unstuck. It's like a little narrative, a little note so that when I get back to that work, I'm like, OK, here here's where I was or here's where I was going to think. So I do think that's useful, but I usually just leave these notes right where I'm doing my work. It's in the document where I'm writing. It's in the in the document where I'm putting out my math equations. And I think that's useful. So, you know, where to start again and it's easier to get into it. The thing that makes me more nervous is this idea of this long ongoing interaction.

29:44I don't actually think that type of cybernet collaboration in most cases is a useful new form of knowledge work. I think it's just escaping strain. It's a fancier version of like I'm looking at my phone a lot. It sort of makes the work feel less hard, but hard is sometimes what you need. All right, who do we got? Next up is Mark. When using active recall to study a large volume of material, how soon and how often should I revisit material I've already mastered? Right now, I find that when I return to something a few weeks after learning it, I've forgotten many of the details. Well, I mean, it depends what you're trying to do here.

30:18Like, what are you learning and why are you trying to hold on to it? If you learn something that you then use, it will stay sort of active in your memory and eventually will really dig in. If you're memorizing some sort of information that you just basically don't come back to again, you're going to need to retouch on that within a two-week window. Like, if you let about two weeks or more go, your mileage will vary depending on the information in your brain. But if you let more than two weeks go without using information at all, it's going to start to lose its location. That is just like informal based on my experience as a student professor.

30:50So if it's like this is arbitrary information I want to remember, every week or so you want to keep going back and doing that active recall. Eventually, if you do active recall enough, it will lock in. But again, be sure that you need to – if you're not using this stuff, be sure you need to remember it. it's the best way to submit something in your memory is to use it to do it i you know you teach the concept a couple times you really remember it you explain it to someone else because you're using it for a project you know you really remember it active recall of course simulates that by recalling the information from scratch you're firing up all the circuits you would use if you were actually applying the the technique so it's using your own brain's own memory apparatus but yeah i would find if i memorize something using active recall and just never touched it again within about two weeks, I might start to forget it.

31:37So I don't two weeks ish, but what are you trying to remember and why, and why aren't you using it? So maybe it doesn't have to be. All right. Who do we got? Next up is Beth. I've seen articles ranging from AI is terrible for the environment to studies suggesting it's carbon emissions can actually be lower than traditional methods. Can you help put this in perspective? For example, by comparing the emissions from one person's chat GPT use to something more familiar like a plane trip and by clarifying whether AI's footprint is unique to platforms like chat GPT or similar to energy use in everyday tools like Google search.

32:10All right. I mean, I do have a hot take here. If you want like the actual answers, I don't know. I don't have the numbers in front of me, but something like a Google search is sufficiently, significantly more energy efficient than like a chat GPT query. I mean, you have to have this frontier model loaded up in memory now this thing could have hundreds of billions if not a trillion parameters defining it right so you're not going to fit into the memory of a single gpu chip you're gonna have to shard this thing over like three four five maybe six gpus that are then going to have to be running all out just to generate tokens for your answers i compare this to a google search where they use commodity intel chips most of this stuff is cached they can be dynamic of like oh there's a little downtime on this chip hey can you like go look something up in a hyper-efficient sort of cached, you know, search index.

33:00And the amount of computation they've got, it's down, you know, minuscule. Probably the answer you're looking for is in some CDN server that's like 10 miles away. So, you know, it's a lot of computational power. But here's my hot take. I might get yelled at for this one, Jesse. You ready for it? Yeah. I think right now the focus on the environmental impact of AI is in part a way for people who are critical of technology or big tech to get in on like, I don't like AI in a territory that they're much more comfortable with than the technology. So if I'm just like, I don't know, like a typical like left-leaning critique of the tech industry and AI, I'm super comfortable for talking environmental stuff.

33:44Like you don't care about the environment. I do. I saw the Al Gore documentary. You don't know science. The environment's important. Like I feel like I'm smart there. when it's me talking to other people like that's where i'm smart like i know more about this than you you're you don't know science global warming is a real thing so that's like a very secure place of critique a less secure place of critique is you know the type of stuff i've been doing in my writing recently like coming in and be like here i don't think that reinforcement based learning post-training refinements is giving you the sort of uh lost reductions that you would have expected on a more elongated power law curve like people getting into the technical details is very like it's hard and you're like i don't really know this details well but i i don't like this and i don't like the people involved and sam altman's kind of creepy and mark zuckerberg still talks like a robot whose emotion circuit board is shorted out we've covered this on the show and this is a territory that i'm much more comfortable in um but just like also really leaning into like the specific ai safety concerns over the particular words that a particular chatbot will or will not say, well, I'm comfortable there.

34:50This is inappropriate speech. But I think if we're really thinking about AI, these are kind of transient concerns because I don't think a future of these massive frontier models being queried for everything is a future that makes any economic sense anyways. Here's the right way to think about it. If querying one of these models is bad for the environment, that means the amount of computation it's using can't possibly be profitable. So they can't be bad for the environment too long because it correlates also with expense and people aren't willing to pay for the massive computational expense. I mean, it takes money to create heat because that requires electricity and electricity costs money.

35:32So my thought about this is, yes, I would be concerned if we had chat GPT used all the time, but I just don't think it's profitable. I think the future of AI is going to be it has to be systems that have much smaller models machine native models if possible meaning like this thing is running on my ipad not in the cloud somewhere on a bunch of gpus small specialized language models combined with other specialized components like policy networks for evaluating options future simulators for trying to understand which actions to take next control logic that's specific to the task at hand put together into a program that you can get your arms around that can run on existing hardware and does really good at being intelligent about a very specific thing that I'm convinced is the future of AI.

36:17It doesn't have an environmental footprint that's different than, you know, other things that we're doing right now with our computer. So there probably is, it does use too much electricity, but I just don't think these massive frontier models are going to, it just doesn't make sense. I cannot be querying a trillion parameter model that requires six H100 GPUs to even produce a token for me for everything I'm doing in my life. That would probably be an environmental catastrophe, but we're not – that's not sustainable. I think frontier models like F1 cars, it's a way of showing off your technology and proving that your company has the best technology, but it's not the car we're buying on the Ford lot a year from now.

36:53That's going to be a much simpler car, but we'll buy it from the company who had the best F1 car because that convinced us. So I don't know. That's where I am. So that's my hot take is of all the issues we have with AI, the focus on environment right now, we're not in some steady state yet. I just think it makes people more comfortable because that's territory where they're familiar being the sort of higher level of the hierarchy and the argument. And they're uncomfortable getting the tech because, like, let's be honest, my people are nerds and are kind of uncomfortable to be around. And we don't talk very normal and we know too much about algorithmic nonsense or whatever.

37:23So I don't know. That's what I think is going on. So I'm not as worried about the environment because I don't think this industry can survive in a mode that is super bad for the environment. It just costs too much money. People aren't going to pay$1 ,000 a month to pay for all their queries or things. I think smaller models can work. Did you know, for example, the model – Noam Brown's model, Pluribus, which was the first AI system to beat actual tournament players in Texas Hold 'em poker. The original way they built it was with like very large neural networks, and they used a supercomputer system center at U-Pittsburgh to train it.

37:56It was like, okay, and then they figured out like, oh, we could have a couple different networks, and what matters is like the logic of how we connect them together and the logic of our AI and the system they built, Pluribus, it can beat the professional players you can run on a laptop. Because it didn't have to – it wasn't just like we have 100 billion parameter network that we're just like learn poker. Like no, like we have a future simulator that simulates the possible mind states and cards of the other players. Then we have a smaller neural network trained just to like understand different poker positions and say what would be good or bad.

38:28And then we have some logic that we – symbolic logic we hand-programmed in about like, okay, let's calculate the value, the expected value. If we do this and this was the case, here's what would happen. Let's give that a number. What about this? It's just like straightforward mix of neural network unsupervised learning with like symbolic old-fashioned hard-coded AI and the whole thing fits on a laptop and it can beat professional poker players. so hopefully the future of ai i think is most most likely the environmental concerns are not going to be um i think substantially different than sort of the environmental footprint of the computing we have right now but we'll see all right what do we got next next up is david if you left academia would you spend your days as a writer and podcaster would you enjoy this or become antsy um yeah i would continue to write and i would continue to podcast i don't think i would become antsy so i've kind of two reactions to this no i want to become antsy uh writing and podcasting actually takes up a lot of time and it's not like i'm hurting for other things to do you know how many teams i'm coaching and my kids like schools and sports leagues right now let me I guess three, six.

39:42It's three. I'm coaching three teams right now. I've had a lot of time doing them. I have a lot of things to do. That's fine. But here's the bigger point though is writing and podcasting is like a lot of what I do as an academic right now. So I don't know how different it would be. So for those who know my trajectory, I've been a writer my entire adult life. I started writing in college. I trained as a theoretical computer scientist focusing on the theory of distributed systems. So I studied at Nancy Lynch's Theory of Distributed Systems Group at MIT and did my postdoc under Hari Balakrishnan and his Network and Mobile Systems Group.

40:17So I had a computer science specialty that was on the math behind distributed systems. Went to Georgetown. This is where like my NSF funding was all about this. My papers were all about this. My grad students. We did a bunch of – I was a pretty good distributed theoretician. Published a bunch of papers. We're on a bunch of steering committees. Somewhere around the time I was – I got tenure. Then I got full professor. So around the time I was getting full professor, Georgetown, where I work, they began making a real move for like we want to be one of the places grappling with technology and its impacts.

40:48They call the field digital ethics. Like we want to be at the core of this because we're in Washington, D.C. We have a big ethics background here. We were the university that like figured out bioethics in the 20th century, the Kennedy Center for Ethics. We want to do the same thing for tech ethics. We're here in D.C. We have all these policy centers. we have one of the biggest tech law faculties at our law school like it makes a lot of sense and i was like this is kind of what i'm doing already with my writing i write about technology and how it impacts us i want it and so like i've really been focusing on that so i was one of the founding faculty members of georgetown center for digital ethics under inaugural director of our computer science ethics and society academic program it's the first major in the country to integrate computer science and ethics in a combined major there's places where there's majors were like you're a computer science major and you throw on a little bit of ethics first integrated one uh in the country so like this is actually what i i'm doing largely right now as an academic is technology how it impacts us what to do about it and i do it in a lot of different forms and i think i think public outreach is important i think the podcast is very pragmatic but i reach a lot of people this way my writing for the new yorker now we're talking to a little bit more of a of like a rarefied crowd crowd this gets me in front of policy makers this gets me in front of like the Senate and on NPR or whatever.

42:06But it's a way, again, to work on tech and its impact. I do some academic papers on tech and its impact, which is more for like an academic crowd. And then my books fall somewhere in between. So I don't really know. My life wouldn't be that much different if I left academia to write and podcast. Writing podcasting is what I'm doing in academia. The main difference would be there'd be less time around really smart people and less time around students, both of which I like. also teaching but teaching increasingly i can teach things that like helps me think about these thoughts anyway so you know no i want to be antsy and also i don't know how different it would be do you still do any math i haven't very recently i haven't yeah been a year or so so do you still teach undergrads like traditional computer science classes i do a mix of traditional computer science and stuff that's relevant for uh the computer science and ethics program yeah and i teach less than I did before too because I'm doing running these things or whatever so that's a good question though um I think we have a case study yeah before we do the case study if folks have calls then just go to the deep life.com slash listen and submit some updated calls the deep life or the deep life the deep life the deep life.com slash listen there's a link yep and you can record the call right there from the browser yeah do that we have a lot of calls but they're getting kind of old right yeah you got a good shot a good pithy call on a topic i've been talking a lot recently you got a good shot now so i would go i would go record the calls uh we do have a case study however i love case studies this is kind of a long one but it's great case study on my theories of lifestyle centric planning i'm thinking so much about this because of my new books i'm happy to have this we have what theme music for this right yep do we i don't remember is it just to introduce this segment or do we play it the whole time?

43:51I don't remember. Oh, the two episodes we did one, but then last week you wanted to hear both. We'll play it now and see your mood at the end. Alright, let's hear it.

44:08I just want to do a deep thoughts with Jack Handy. Alright, so today's case study comes from Vin. I hold a bachelor's degree in interface design and in my 20s, first worked as a web developer and later as a UX designer for several digital agencies. I enjoyed my work, but I felt stuck in big cities and kept dreaming about a life closer to nature. So in my mid-20s, I did something that you would surely advise against. Uh-oh. I threw my career into the trash and moved to Norway to live and work on an organic farm. I had a great time there and enjoyed every aspect of it. However, I didn't have a sustainable long-term plan.

44:46I realized I needed and wanted to move back to my home country where my girlfriend, now wife, lived. Back home, I decided that my next big move would be to find a job that allowed me to work in nature. So I signed up for a nature guide program and actually managed to land a job as a guide at the very place where I had completed that course. For the last six years, I worked there running workshops and similar programs. Sounds like a happy ending, right? Not quite. Life changed. I became a father of two children and we moved closer to my parents so we could get support from the grandparents with family life.

45:20This meant I was now living almost two hours away from my workplace, constantly torn between family and professional life. On top of that, since most workshops take place on weekends, I was away from my wife and our two little ones at the very time they needed me most. It turned out that the dream of working in nature definitely had its flaws. I'm going to stop there for a second. Let's take stock of where we are. In the story so far, we're seeing a common issue when people think about the deep life. So a common issue that people have is that you fixate on a single change that you begin to believe will make everything better.

45:59It's simpler to think about a single change. We can sort of inhabit that change. The idea of doing something radical itself makes us feel excited in anticipation. and we can sort of take the imagined feeling of like the best parts of that change and sort of expand that in our mind and be like, my life is going to be better. So Savin did this with nature. He's like, I like nature, so I'm just going to focus like a laser on if I could just be in nature all the time, I would be really happy. The problem, as we often say, when we talk about lifestyle-centric planning, is that your daily subjective mood is not the result of a single decision or change, but on all of the relevant aspects of your life.

46:37call this your lifestyle all the different things that are relevant during the day add up to give you your subjective experience of the day so you might like the part where i'm giving a workshop and it's in nature and that's nice but you have other parts of your life too like the fact that you had to drive two hours to get there and that you're kind of stressed out about what's going on with your your in your grand your parents and like the kids are you're not around enough to help and it's creating tension with your wife all of your houses maybe not the it's nowhere near nature and you're actually spending most of your time in a car, all of these other aspects of your lifestyle matter too.

47:11And this is how you can make one radical change that you're excited about and yet end up less happy than you were before. Because by making this thing better, you accidentally maybe made other things worse. So lifestyle-centric planning says you got to construct your ideal vision for an entire lifestyle, all the parts of your life. How would I reorient my life so that all of the parts were something that resonated? And then how do I make progress towards that? You're going to enjoy your life more if you explicitly construct it so that all of the things that contribute to your enjoyment are being pushed in ways that are better.

47:43Let's return this fin because he recognized that, and let's see how he made some changes. Around this time, I came across your books, Deep Work, So Good They Can't Ignore You, and Slow Productivity. They gave me a framework to better understand my professional situation. I had to take a hard look at myself and realize that for the last five to six years, I hadn't built transferable career capital that could help me in a different role. After some reflection, I decided to revive my previously built career capital as a web developer. However, I hadn't been coding seriously for years and looking at job descriptions, I realized how much had changed in the web development space and I had a lot of catching up to do.

48:19All right. So he's doing something here called evidence-based planning. So he makes this decision. He's looking at his life and is like, actually, I need a job that is going to whatever the criteria here, but probably like more money, more autonomy. I don't have to commute so much, whatever. Web development is like, I have career capital there. Let me start there. But he does evidence-based planning. He actually looked at real job listings like, oh, skill A, B, and C I don't have. So I'm going to summarize a little bit here. But basically, he uses like techniques from deep work and slow productivity as well as from Scott Young's excellent book, Ultra Learning, to begin an education process of learning the specific skills that his evidence-based planning said were important, not what he wants to be true, but like I need to know A, B, and C.

49:05These employers want exactly those skills. And how do I get there as effectively as possible? I'll jump back to regular speed here. Thanks to your techniques and tools, it took me about a year to get out of my difficult job situation and back into my old career. The side effect, I now earn more than twice as much as before. My next move won't be as bold as the ones I made in the past. I want to become a valuable asset for my company, get so good they can't ignore me, and then use my career capital to reduce my working hours so I can have Fridays off to spend in nature without having to work there.

49:37So see what happened there. It's when he did lifestyle-centric planning. He's like, I like nature, but by reorienting my life around my career takes place in nature, a lot of other things got worse. And actually being in nature is not as great as you think when you're also like working they realize like oh if i could take my programming expertise make it up to date make myself really valuable i can go back to programming get good enough that i can cut my hours still make more than my old salary only work three or four days a week and then spend time in nature very intentionally but now without the commute now i've been home with my family now things being more stable um man that's a better life So lifestyle-centric planning worked out better than just putting, laying back, like, what's a radical thing I can do?

50:23I'll move to Norway. I'll get a job as, like, a nature guide. Lifestyle-centric planning was not as exciting, but it led them to a better place. All right. So we got coming up my investigative journalism. Beware. I do a little bit of data reporting. But first, before we get there, what you've really been waiting for, to hear about another one of our sponsors. Visible puts unlimited 5G data and hotspot in the palm of your hand. Powered by Verizon's 5G network with no contract holding you back. And for a limited time, you can get Visible for just$19 a month for 12 months when you use promo code SAVE6.

51:00All the features of big wireless service for half the cost. Tap the banner to switch today. Terms apply. Standard rate applies at month 13. See visible.com for planned features and network management details. But here's the thing. If you run a small business, you know there's nothing small about it. Every day there is a new decision to make, and even the smallest decisions can feel massive. Jesse knows this. I get upset whenever anyone refers to our business here as small. I know how hard it is to run this. Just the other day, for example, the nice woman who runs the gift shop down on the next block here in Tacoma Park asked me how life as a small business owner was going.

51:38So naturally, Jesse and I rolled our car into a swamp. We're going to put up with that nonsense. So here's what I'm trying to say. It's hard to run a small business. So when you can find decisions that are no-brainers, you should take them. And when it comes to selling things, using Shopify is exactly one of those no-brainers. Shopify's point-of-sale system is a unified command center for your retail business. It brings together in-store and online operations across up to 1 ,000 locations. It has very impressive features like endless aisle, ship to customer, and buy online pickup in-store. And with Shopify POS, you can get personalized experiences to help shoppers keep coming back.

52:17And they will come back. Based on report from EY, businesses on Shopify POS see real results like 22 % better total cost of ownership and benefits equivalent to an 8.9 % uplift in sales on average relevant to the market set survey. So get all the big stuff for your small business right with Shopify. Sign up for your$1 per month trial and start selling today at shopify.com slash deep. Go to shopify.com slash deep, shopify.com slash deep. I also want to talk about our friends at My Body Tutor. I've known Adam Gilbert, My Body Tutor's founder, for many years, and he is my go-to guy for fitness advice.

52:57His company, My Body Tutor, is 100 % online coaching program that solves the biggest problem in health and fitness, which is lack of consistency. They do this by simplifying the process into practical, sustainable behaviors and then giving you daily accountability and support, what you need to actually keep taking action on your plan. You have an app. You check in every day. Like, hey, here's what I ate. Here's how I did my exercise. And your coach checks it. Hey, how'd it go? If you have a question, like, you know, I'm really struggling with X. Like, great, let's fix this right away. Knowing there's someone checking on what you do every day and is there to help you if anything's not working, that's how you get consistency and getting healthier.

53:36It's not the information, the consistency. And having this online coach, it is a really, really good way to get healthy. All right. So if you're like, I want to get in better shape, this is the way to do it. Go to mybodytutor.com. Mention you heard about it on the Deep Questions podcast when you sign up, and they will give you$50 off your first month. That's MyBodyTutor, T-U-T-O-R.com, and mention Deep Questions. All right, Jesse, let's move on to our final segment. So I want to talk about an interesting article about technology and students, and I want to look a little bit deeper at it and have a bigger point to make.

54:15This article actually appeared recently in the Washington Post. It came to my attention, however, when Tyler Cowen wrote about it on his excellent blog, The Marginal Revolution. I'll pull it up here on the screen, the blog post, because the Washington Post is behind a paywall, but Marginal Revolution is not. All right, so here is Tyler's setup just summarizing this article. He says, until recently, a nearby radio telescope meant the local school could not use Wi-Fi, and how did that go? All right. So this article that he's citing here is about Green Bank, West Virginia. Have you heard about this place, Jesse?

54:51No. It comes up a lot in technology circles because it's rural West Virginia and the world's largest steerable radio telescope is there. Oh, really? Yeah. And there's a town next to it. And radio interference could mess with the telescope. So there's no cell phones and there's no Wi-Fi. So it's like a town that's kind of like off the grid because of this telescope that's in the middle of the town. So what this Washington Post article that Tyler Cowen is citing, what it was about, it was actually an op-ed. It was written by someone who has a book he's working on about Green Bank. What it was arguing is the fact that there is no Wi-Fi in the town has been hurting the students in the school there.

55:37Green Bank has an elementary, middle school, combined school, about 200-something students. He's arguing without Wi-Fi, they can't use like the modern Chromebook, online curriculums, do online testing. And this is hurting the students. So let me read here. This is a Tyler Cowen citing The Washington Post. So what I'm about to read here is from The Washington Post article. While the rest of the country rushed to bring tech into classrooms, Green Bank remained stuck in 1999. Without Wi-Fi, the school's 200 students couldn't use Chromebooks or digital textbooks or do research online. Teachers couldn't access individualized education programs online or use Google Docs for staff meetings.

56:18Even routine tasks such as state-mandated standardized testing became challenging with students rotating through a small hardware computer lab where they took the exams. Some of the teachers say this has been a problem. Here's a quote here. The ability to individualize learning with an iPad or a laptop, that's basically impossible. That was teacher Darla Huddle. Here's another teacher being quoted in the op-ed. Without the online component of our curriculum fully working, it's really detrimental to our instruction. That was Sarah Brown. So this is the issue here. They can't do this sort of modern ed tech stuff.

56:55And the argument of this article is like that's held this school back. And how do we know this? Well, there isn't data on this in the article itself, like specific data, but here's what the author says. While these discussions dragged on, so discussions about like is there some way we can get like low-powered Wi-Fi or some other way to get Wi-Fi in the school. While these discussions dragged on, students fell further behind in math and reading with Greenbrink consistently posting the lowest test scores in the county. All right. This is an interesting argument here that without modern ed tech, students are falling behind.

57:34And that's just like a natural experiment. And look, this school has the lowest scores in the county. Now, this is like a relevant time to be arguing this because there's so much discussion about phones in schools and technology in schools. And it's mainly negative. Like technology in schools is often negative. But here's this op-ed in the WAPO that's being contrarian. And Tyler, who also has a real affinity for technology, is like, yeah, come on, let's be careful about it. Maybe some of this technology is really helping. So I want to look into this. You know, I'm not an expert data journalist, but I was like, this is interesting.

58:05And I want to look a little bit closer. So what did I find? Well, first, is it true that Green Bank consistently posts the lowest test scores in the county? This turns out to be called Pocahontas County. It's kind of interesting. I'm like, West Virginia, okay. So it's Pocahontas County, West Virginia. Yes, the Green Bank Elementary Middle School has the lowest scores in the county. But there's a problem here. It's a very small county. I mean, it's a lot of geographical area, but this is not Montgomery County here in Washington, D.C. The county, there's the Green Bank Middle School and Elementary School.

58:44The county has one other middle school and a shared high school. And then there's two smaller elementary schools that are in like another town over there, much smaller, one of which which has the county's like gifted and talented program. There's only 70 kids there. Half of them are gifted and talented kids. Like we're not talking to big county. It's a handful of schools of which like two of them of the six are in Green Bank. One is shared. And then there's like one other middle school and two other elementary schools. They do – so yes, like the elementary school in Green Bank has lower scores than those two other elementary schools.

59:18The middle school has lower scores than the other middle school. All right, but so it's a small county. That doesn't necessarily tell us much. Like maybe Green Bank, for whatever reasons, that part of the county just is worse off and they just get worse scores. I don't know what's going on there. Like do we know this is from not having Wi-Fi or not? Well, what we really would need to test this is we need time series data. So we need to see test performance over time, right? Because until somewhat recently, it wouldn't matter if you had Wi-Fi or not, right? So iPads were introduced in 2010. We see the rise of Chromebooks and classrooms picks up in the 2010s.

1:00:02That's really where this thing, this happens. So before the 2010s, there would be a lot more technological parity between this particular school and the other school in the county because no one was using technology like that in school. So what we really need is like a time series that shows somewhere in the 2010s, the Wi-Fi free school begins to separate. The impact on not having Wi-Fi makes them worse because they might have always been worse. We want to see them get worse. All right. So I went looking for this data. So it's hard to find time series data for the particular schools in Pocahontas County.

1:00:39But because Pocahontas County is so small, there's two middle schools. One of them is with Wi-Fi. One of them is not. There's three elementary schools, one with, one without. And the other two add up basically to the size of the one that didn't have Wi-Fi. It's such a small county. What we can do is compare Pocahontas County to other counties in West Virginia. Because presumably if like half the students in your county didn't have Wi-Fi and that affected them, that should bring down the whole county around the time that Chromebooks, et cetera, became big in school. So there is data. I can get data on NEAP.

1:01:16These are state-mandated test scores. I can get data on this county by county. So let's see what's going on here. So let me load up some charts. This is from Education Recovery Scorecard, which aggregates a lot of this data. It's really interested in like what happened after COVID. But they have data that goes back to as far as 2009. So it's perfect for us. What I'm loading on the screen here for people who are watching instead of just listening is math performance grades three to eight. Because again, going past middle school, it's a shared high school. So like we don't learn much from that. From 2009 to 2024.

1:01:48All right. So as we see here, going from 2009, we see an increase. There's a bit of a gap in a data, but we see roughly actually math scores were getting better. Go Pocahontas County. Math scores were getting better from 2009 until about 2017. In 2017, oh, this is interesting, the score started going down. It went down, went down. Then we get the pandemic where we don't have data, but they were falling multiple years before the pandemic. And then we see after the pandemic, we begin to get a recovery, which we've seen nationwide as things fell so low during the pandemic that there's a recovery right after.

1:02:24So that's the chart we're seeing. If we look at reading scores, it's something kind of similar here. The peak is around 2015. Then it begins to go down after that as well. The timing here is compatible with the Wi-Fi hypothesis, right? That kind of makes sense. Chromebooks and all these other things that require internet begin spreading in the 2010s, it might make sense that where would you first start seeing the impact of not having this technology? Like midway through the 2010s as these things were gaining traction, that might be where you start seeing performance go down. So this data from within Pocahontas County itself is compatible.

1:03:00Roughly speaking, it's compatible with the Washington Post hypothesis of without Wi-Fi, things started to get worse. But if we want to do a controlled experiment, we have to compare this to similar counties. So what we need to do, because Pocahontas is half the non-Wi-Fi and we can't break that out. So let's find other counties in West Virginia that are similar in terms of size and demographics and socioeconomic status. Like very similar other counties nearby in West Virginia where there were no radio telescopes and everyone is allowed to use Wi-Fi. What we would expect to find if the Wi-Fi hypothesis was correct is that Pocahontas County should have a much more notable drop in performance starting in the mid-2010s than these other counties that didn't have any Wi-Fi restrictions.

1:03:52We have that data. That's what's interesting. So let me scroll here a little bit. All right. So this is – we're going back to math performance here. We have three stacked charts. So if you're listening, I'll explain to you what we see. Each of these charts has a downwards arrow and an upwards arrow. The downwards arrow, which is purple, and I'll put this up here on the screen for Pocahontas County Schools first, that is the decline in math scores during that period up to 2019. Let me get the exact dates here. 2019 to 2022, where we had the steepest sort of losses we saw in those curves before.

1:04:30And then the green arrow is the improvements that they've seen since the pandemic. So this chart is showing here for Pocahontas County Schools. It's just quantifying what we saw on that chart. In that downhill period from 2019 through 2022, there was a 0.6, negative 0.6 drop in their performance compared to, I think this is like state average. So yeah, they fell. And then we see the green, there was like this 0.36 increase. So don't worry about the numbers, but there was an increase. This is the recovery after the pandemic. All right. So I'm just quantifying what we saw on that chart. Here's what's interesting, though.

1:05:06They did the same thing for all of West Virginia counties. That's the next one. And for counties that had similar socioeconomic demographic and size, so similar population. So if you're wondering, this is Nicholas County, Hampshire County, Barber County, Tucker County, and Pendleton County. Here's what's interesting. the similar West Virginia districts that were the same in terms of demographics, but differed mainly and they didn't have the Wi-Fi restrictions. They saw a bigger drop from 2019 to 2022, and they saw a smaller recovery after the pandemic. So the similar counties that had Wi-Fi and never had that taken away did worse than the county in which like half the students are in this Wi-Fi free zone.

1:05:56And in fact, if you look at the West Virginia as a whole, that also was worse than what Pocahontas County did. So look, I don't have school-by-school data. I hear this a lot. I found a lot of informal reporting online that did blame poor performance in Green Bank by them not being able to use Internet in the school. So that might be true, and these teachers seem to think it's a problem. But if we look at the data without having time series data from specific – the handful of schools within Pocahontas County, we do not see here a data story that looks at – that looks like the lack of internet once Chromebooks became a thing and these online curriculums became a thing began to make Green Bank much worse.

1:06:37It seems like that's just a bad school. Those schools are bad, and they've been bad. And I don't know why you'd have to know about like – it's just like what this town is. I don't know what particularly is bad about that town. It could be little things, by the way, because these numbers are so small. Not to get too much into the data, but I was looking at proficiency on math and reading test scores broken down by the individual schools so I can get those numbers for the most recent times they've measured them. And they're like 50 percent better, for example, in Hillsborough Elementary versus the Green Bank Elementary.

1:07:08But Hillsborough Elementary in Pocahontas County is the gifted and talented program. There's only 70 kids in that school. 70 kids total. So all it takes is like, oh, we have 30 of our gifted and talented kids are there. You're going to get 50 % better number of people who are math proficient, right? So these are small numbers. So we have to be careful. So yeah, there's not a lot of schools there and the Green Bank one doesn't do well. But we don't have evidence that it is because they couldn't use internet connected Chromebooks. Again, the only way for this to be possible would be somehow the lack of Chromebooks and internet-connected tech really caused the Green Bank schools to fall really hard.

1:07:52But for unrelated reasons, coincidentally, the other two or three schools in Pocahontas County did unusually well during this period, and they sort of offset the fall that the Green Bank was having. And that's why that county, even though other counties with the same demographics as Pocahontas County, fell farther, that there was something special happening with the non-Green Bank schools of Pocahontas County where they offset the Green Bank losses. Maybe that's true, but we would have to hear a plausible reason why that's true and actually see school-by-school data. So I don't know. The author of that op-ed is writing a book on Green Bank, and he might have really good data.

1:08:23But knowing what we know now, I think this is a nice cautionary tale, and it's a good reminder for myself or anyone else who talks about technology trends. When there's an answer that we like, it's often easy to jump to it, find any point of data that seems to imply that and expand it beyond what the data says. And I think that seems to be what was happening here. There's a nice subtle leap from the schools without Wi-Fi are worse to the schools without Wi-Fi are worse because they don't have Wi-Fi. It's an easy leap to make, but the picture gets much more murkier once you pull even a little bit on the data story.

1:09:00So anyways, I guess this is just a back-to-school note of we all have to be careful and take with grains of salt claims that are made that sound intuitive and there's a little bit of data to support doesn't necessarily mean it's true. It's like when it comes to cell phones in schools, I have read exhaustively not just the research but the debates that researchers are having about the research and the complaints and then how the complaints are answered. I, you know, there's an hour long talk I give about the evolution of the research literature on harms from phones for kids. I feel like I know that data very well.

1:09:33And it makes me confident to say these are often a problem. The benefits aren't worth it. You shouldn't have phones before high school if you're a kid. Right. That's really different than like, hey, that school struggles and they don't have Wi-Fi. Internet's good. So anyways, there we go. I'm not investigative journalist data. Who knows? Maybe this professor has the school by school data. But I think the picture in West Virginia, Pocahontas County is more complicated than if only I could synchronize my Eureka math curriculum with like online resources, our kids would suddenly be much better. Harder reality.

1:10:05All right. Speaking of hard realities, that's all the time we have for today. So thank you for listening. We'll be back next week with another episode. And until then, as always, stay deep. Hi, it's Cal here. One more thing before you go. If you like the Deep Questions podcast, you will love my email newsletter, which you can sign up for at calnewport.com. Each week I send out a new essay about the theory or practice of living deeply. I've been writing this newsletter since 2007 and over 70 ,000 subscribers get it sent to their inboxes each week. So if you are serious about resisting the forces of distraction and shallowness that afflict our world, you've got to sign up for my newsletter at CalNewport.com and get some deep wisdom delivered to your inbox each week.

1:11:15Thank you.