The Impact of Generative AI on Critical Thinking

0:00You might not be surprised to learn this, but I use LLM's a lot. I use them in my work. I use them in my personal life. And sometimes I use them to help me with stuff that I kind of know how to do. I just want to make it a little bit easier. And it does make it easier for sure. But something that I worry about sometimes is that over the long run, I'm going to pay a price for that. I'm going to get lazier. I'm going to get a little bit dumber. And the question is, as I'm outsourcing my thinking to LLMs, am I becoming reliant on them? If they were ever to go away, would I lose my ability to do basic things?

0:43I like feeling like I'm a smart, capable person. Am I letting that slip away without realizing it just because I want it to be easier for me to think of the meal planning for the week. In today's episode, we're going to talk about a paper that came out of Microsoft Research where they're studying just this issue, trying to understand how people think critically, when they think critically, how much do they think critically when they're using LLMs versus not. You are listening to Linear Digressions.

1:34All right. So today's paper is called The Impact of Generative AI on Critical Thinking, colon, Self-Reported Reductions in Cognitive Effort and Confidence Effects from a Survey of Knowledge Workers. As I said, primarily from researchers at Microsoft Research. We also have one of the authors on the paper, is from Carnegie Mellon. So this is a survey paper. These researchers talked to a bit more than 300 knowledge workers, and they asked them about how they use generative AI, in particular when they're using it at work. For each of those knowledge workers, they said, tell me about three real tasks that you've used generative AI on, asked a bunch of questions about those tasks.

2:18So we've got about 936 real tasks, 300 workers. And this was the basis of their data set. So they're analyzing the sort of self-reported survey data, trying to find any important trends about how critical thinking increases, decreases, changes, or maybe does none of the above when people are using generative AI. So what do they actually find? AI doesn't necessarily make us think less. So a lot of the people in the survey report that they're still very critically engaged, cognitively engaged with the output of the LLM. But that cognitive engagement, that critical thinking is happening in different points of the process than it does without generative AI.

3:04So it's not quite as simple as like, I turn on generative AI and I turn off my brain. It's I turn on generative AI and my brain has to start flexing in different directions. And And so it's just an interesting and more nuanced story. So to understand what they found, we need to start by thinking about what the definition of critical thinking even is. There's a taxonomy for this called Bloom's Taxonomy. It's from education research, and it has six levels of thinking. And these kind of ladder up from the most basic to the most nuanced or complex. So at the beginning, at the foundation, you have things like knowledge, remembering facts, comprehension, which is understanding them, application, which is then taking those facts and applying them to problems, analysis, which is breaking problems down.

3:56Now we're starting to get into the more complex ones, synthesis of combining ideas and evaluation of judging quality. So we got our ladder, our Bloom's taxonomy of critical thinking. And the question is, when you're using chat GPT for a work task or generative AI, maybe you don't use chat GPT, which rungs of this ladder are you actually on? Are you going up or going down? Now, how are we going to study this? There's a problem with doing this sort of the most naive and straightforward way, which is looking at the outputs that people create in their work when they're using generative AI. And the reason why is that two outputs can look the same, but you have a very different thought process that's happening behind them.

4:42So let's say that you have chat GPT. It produces an answer in response to a question. Person A mindlessly copies and pastes that answer, and there's zero critical thinking going on. And you could have person B who takes that same answer. They think really hard about it. They're very critically engaged. They decide that, you know what, actually it is perfect. And then they copy and paste it. So lots of critical thinking. You have the same output, but totally different cognitive engagement from these two people. So you have this problem of how do you actually measure what's happening inside of people's heads?

5:15The answer here is like, well, just ask them. Use survey methodology. Talk to enough people, which is several hundred in this case. Ask them about enough different cases, which is almost a thousand. And then ask a bunch of questions about when they were thinking critically. What did that look like? How do you think about the task that you were asking the AI AI to help with versus your own capabilities. And then what they measured in the survey, kind of two facets on this. Number one, a bunch of information about the people themselves. So some of this is demographics, but also it's about which AI tools they use, their general trust levels in AI, how much they tend to reflect on their work.

5:53They also ask questions at the task level. So for each of those three tasks, what type of task was it? How confident this person felt in doing that task, how confident they are in the AI's ability to do it, and how confident they are in evaluating the AI's output. Then you take this big survey data set and you run it through a regression. And from the coefficients of the regression, you can see which of these factors are most predictive of critical engagement. And then there's a lot of good qualitative analysis as well, where they're pulling out written explanations. They're seeing these little snippets of a justification, this qualitative analysis that people are giving as well.

6:30All right. So they take this data set and they use it to frame a research project that has two big questions. The first is, when do people engage with critical thinking? And then their second research question, does AI make critical thinking easier, harder, or just different? So let's start with the first one of those. When are people engaging in critical thinking? The first is, there's still a lot of critical thinking going on. It's just happening around the periphery of what the AI is doing instead of actually accomplishing the task. So let's say you have some job where normally you would have done it yourself.

7:08You're writing some document for your work. Instead of writing the document yourself now and using your critical thinking in the actual document writing, you're thinking about all the other stuff that happens before and after. So what am I even trying to accomplish here in writing this document? How am I going to prompt the AI so that it has enough information to accomplish this well? The AI generates a candidate paragraph or page or something for the document. I'm going to inspect it. I'm going to read it. I'm going to evaluate whether it's high quality or not. If it has sources, I'm going to go verify those sources and see if they actually exist and they actually say what the AI says they have.

7:46And then last but not least, once it's finally probably cycling through this process a few times where I'm tweaking my prompts and I'm reevaluating the answers. And then finally, once it reaches the point that I'm like, okay, this is good enough, I'm going to use it. I have to figure out how to integrate it into the thing that's going to be used downstream. So I have to copy it over into the place where the document is going to go so I can share it with my colleagues or whatever it is. So all of those are forms of critical thinking. Those are all cognitive engagement. And a lot of those are significantly harder than, or they're net additive, they're net new to anything that I would have done in a pre-gen AI world.

8:23So there's still a lot of cognitive work going on. It's just in different places than if I'm generating this document myself from scratch. In particular, there's three motivators that they find in this research for critical thinking. Number one is when there's questions about work quality. So when the AI output isn't very good, it isn't good enough, that's when you're going to be really engaged with it. You're going to be trying to reprompt it. You're going to be really carefully reading the responses. You're going to be thinking about whether it's good enough now or if you need to go another round with it so that you get a good high quality work output.

8:59Number two, another motivator for critical thinking is when the stakes are very high. There's an example where a pharmacist gave some feedback on the survey. They said that when they use AI for professional development documents. There's regulation that's going on around pharmacy. Like if this gets messed up, there could be people who take the wrong medications. They could have professional consequences. So if the consequences are serious, people stay engaged, or at least the people that answered this survey. And then the third case where people stay really engaged is when they want to develop a skill, when they want to learn.

9:34So if you're trying to learn a new language, or you're trying to learn how to program a computer better, you're learning some new software library, and you're using the AI to help you through that process, then that's another place where you're going to be really cognitively engaged. You're trying to build competence. You're not just trying to get the answer. There's three barriers to critical thinking here as well. So number one is they call it awareness barriers. A better way to describe this is over-trusting the AI for quote-unquote simple tasks. So when you think there's something that's simple, that's easy, that AI should be able to handle this, like it should be able to handle rewriting this so it has a professional tone.

10:15I'm just going to let it take care of that and not worry about it too much. The AI is so good, I'm just not going to question it. And there's some danger here, which is that if you assume AI is competent for this routine stuff based on its past success, you could be overestimating it. And you wouldn't know because you're not reading it very carefully. You're not critically engaged with the output that it has. It's just like not doing that well and you're not paying attention. Number two, another motivator for not being critically engaged is motivation. You just, you're not motivated to do that or there's incentives pushing you in some other direction.

10:53So there's an example here, a quotation from someone who works as a sales representative who says, you know, they're under a quota. I have to hit my quota every day or risk losing my job. I use AI to save time and don't have room to ponder the results. So if you are just trying to do quantity over quality, probably not spending a ton of time thinking critically about everything that the AI puts out. Third barrier to critical thinking is ability barriers, which is where the people, the prompters, the workers say that they don't believe that they have the ability to evaluate or improve the AI output.

11:30So as an example here, I might say if I am asking a generative AI for some reason to translate something that I've written into Spanish. I don't know Spanish. I don't speak Spanish. I can't read Spanish. And so if it gives me, it's going to give me something back in Spanish. I'm not going to be that critically engaged with it because I just, I don't have that ability. So maybe in other cases, people actually are more capable than they think. But let's take people at their word that sometimes they're using AI to do things that they can't. then they're going to be less critically engaged because they don't have the capabilities.

12:02They don't know how they could critically evaluate or improve the answer. They're like, all right, it's just going to give me what it gives me. Something you may be wondering at this point, we've got a lot of things going on. There's all of these factors. There's person level factors and task level factors. There's different motivators to be more or less critically engaged. So what actually matters out of all of this? When you run the regression, what actually predicts from this survey data anyway, whether someone thinks critically? There's two big factors and they're pulling in opposite directions.

12:32So if you have confidence in yourself, if you believe in your own abilities, especially to do the task that you've given to the AI, you're more likely to be engaged in critical thinking on the AI output. So if you feel competent at the task, you're more likely to check the AI's work. Conversely, if you have more confidence in the AI, less critical thinking. So the more you trust chat GPT, the less you check it. And there's something interesting and I think a little bit nuanced here, which is there's potentially a bit of a feedback loop here as well, and one that could be pretty dangerous, which is that the better AI gets, the more you are willing to trust it to just take care of stuff for you, you're engaging in less critical thinking.

13:20And I think it's valid to consider that the less you're critically engaged with the AI outputs, you might start to lose some of that critical thinking ability. That critical thinking is like a muscle that if you don't use it, it starts to atrophy. When that happens, it is also reasonable to think that you're aware that it happens. And so you're becoming less confident in yourself. And that's also nudging you towards trusting the AI more. So there's potentially a pretty significant one-two punch here that suggests that especially through repeated use and through losing some of the confidence in your own abilities or overtrusting the AI, you can be in a cycle to successively trust the AI more and more, maybe even in ways that you shouldn't where you might be overestimating its abilities.

14:17So that's the findings for research question number one, when are people thinking critically? Research question number two, how is the critical thinking shifting in the tasks when you're using generative AI? When people are engaged, what are they thinking about? Is it the same work, just easier, or has the work fundamentally changed when we're using generative AI? And the answer is that it's changed. Three specific shifts. Number one, the first shift that they find is from gathering information to verifying it. So let's think about a research task. Before generative AI, this might involve a lot of Googling.

14:56You're finding different sources. You're reading them. You're gathering this information to perform the research task. Now the generative AI is doing all of that. And it's going to serve that up to you very quickly. So you're not going to go through that process. Critical thinking on that step has gone down. But you now have this task of having to verify everything that the AI is telling you. So do these resources actually exist? Do they say what the AI is suggesting that they do? You should be verifying what the AI tells you. I'm sure you all do that right now. But this is real work. It's not that you're doing, maybe you're doing less work, but you're definitely doing work in different places.

15:34This cognitive load has moved from the information gathering to information verification. You're not searching anymore, you're fact checking, and that's a different skill. Number two, there's a shift from problem solving to response integration. So before AI, you would solve some problem from scratch. Now with generative AI, you might have ChatGPT or Gemini or Claude solving the problem, but then you have to take that response and fit it back into your workflow, back into the specific context. So when AI writes, let's say, marketing copy, you're going to have to be reviewing that, editing it before it's ready for the next step in the process.

16:18You're adding a bunch of additional tweaks for tone, for content guidelines, like whatever. So there's all this extra effort that you have to put into the AI-generated content in order to fit it into the workflow. So that last mile, that's work that in general you didn't have to do before when you were generating this marketing copy yourself or you're writing code or something. It's not like you write it initially in the wrong way and then you go back and revise it. You just write it the right way to start. So when you have AI that's giving you 80 % of the solution, that extra 20%, that's net new of trying to get it fit back into the process.

16:58So there's this last mile problem that you probably are very familiar with if you use a lot of AI. Number three, from task execution to task stewardship. I think a better word here would be task management. But in any case, before you had generative AI, you would do the task. Now you're managing the AI during the task. I think this is probably happening more than anywhere else in these software engineering, these coding agents right now. where you have a whole bunch of agents that are going off and doing a bunch of tasks for you, you still have to do some stuff. You have to manage the AI doing the tasks.

17:36So they had this example, a quotation from the paper talking about image generation. Image generation requires more effort for everything except the actual image generation. I have to think of what I want, then how the AI wants it described, then correct it when it makes wacky outputs. I think there's an analogy here for those of you who've been individual contributors and you've moved into the management track. When you did that, you might have initially thought like, oh, hey, now I don't have to do as much work anymore. I'm not in charge of actually building the stuff or writing the code or running the analysis.

18:08There's people who do that for me. So my job is really easy. And if you're a manager, you know that is not the case. There's a whole bunch of stuff that you have to do. It's just different stuff. Your job is to manage the people who are doing that. There's all sorts of things you have to do with that. Same general idea here with doing the work that we used to do before. Now we're delegating that to AI. Somebody has to manage the AI. So you might be listening to this and you're like, that sounds exhausting in a different way. Exactly. Which is why the headline you take away from this is, AI makes thinking easier or AI is making us lazier.

18:45That's misleading. It makes some thinking easier. It's easier with AI to remember facts or organize information. But there's new thinking work that it's creating where you're verifying, you're integrating, you're managing, you're doing all of these other things, things that you might not have had to do before, things that you're learning that are new skills. So let's come back again to how this can be a little bit dangerous. You put these two things together and you get something that's kind of worrying. There's a pattern here that could be dangerous. So this research suggests that people say when something feels easy, they're less critically engaged with it.

19:20They're thinking less about it. So using generative AI makes certain tasks feel effortless. So we might engage with those tasks less critically, but that doesn't mean those tasks don't need critical thinking like they do. They just need different kinds. And there's this classic problem from automation research called Bainbridge's Ironies of Automation, which gets at exactly this. There's this challenge where when you automate routine tasks, people, they're relying on the automation for those routine things anymore. They lose practice with basic skills. So then when you hit an edge case and the automation fails, the people are unprepared.

20:02They don't have practice doing the simple stuff. The hard stuff gets sent back to the human and they're incapable of doing it. So a good analogy here might be GPS. Before GPS, people had to figure out how they were going to get somewhere. People were much better at reading maps. They were good at navigating by landmarks. Now we have these turn-by-turn directions. I use them to get everywhere. I am certain that there are times when I, if I were more engaged, I've gone to someplace that I don't go very often, but I've been there a few times. It's the third or fourth time I've been somewhere. I'm still going to navigate to it by my phone, but I should know my way there.

20:39I'm just like not critically engaged with thinking about how to get somewhere. So I've lost a little bit that skill or I've gotten lazy about the skill of navigating the world without my phone. But if you're always using AI for these simple tasks, for these places, especially where you trust the capability of AI, you're like, oh, it's pretty easy. Let AI just take care of it. I'm not going to worry about it. You tune out critically. You lose the ability to do that yourself, you lose that practice and you grow to trust the AI more because you're being less critically engaged with its outputs. So again, even if it doesn't do that great of a job, you might not know because you're not reading it that carefully.

21:19So you get into this feedback loop that could be quite dangerous where you're losing the ability. If you're not critically engaged enough, you're losing the ability to critically review the AI outputs. You're trusting the AI more, you're trusting yourself less, critical thinking starts to evaporate. So what do you do with this? Okay, that sounds terrible. Uh-oh. I guess a few pieces of advice. I don't have a silver bowl for you here, but a few things that I'm taking away from this, ways that I think about my interactions with AI in light of this research, and also understanding that this is a survey.

21:53It's a pretty big survey. It's got a lot of people in it from different types of jobs, but there may be more nuances to this as people study it with more time. But anyway, be that as it may. First thing for me is being honest with myself about when I understand something versus when AI does. So there's a way to put this as counterintuitive advice, which is that especially when you're in an area where you don't feel confident, when you're like, oh, I don't know, like, I bet the AI is better at this than I am. Like those are exactly the moments that you want to engage more critically, not less. You're in a zone where you're most likely to overtrust, you're least likely to catch the errors.

22:33And then of course, when you catch those errors, you're going to have more confidence in your own ability, less confidence in the AI, and you're staying on that path of being very critically engaged. And then conversely, the research suggests that when people have high confidence in their own abilities, and they use AI, they engage more critically. So So when you're using AI as a collaborator in an area where you already feel quite confident, that's actually a really good spot to be in. It's a partner for you. It's a resource. It's not something that you're outsourcing your brain to because you feel really solid that your brain is really excellent at the task at hand.

23:09It's not something that you're going to blindly trust. So stay engaged, of course. But if you're using AI as a partner on tasks where you're already expert, then you're in a pretty good spot. just don't use it as a replacement for developing that expertise in the first place. So with that, a few things to take away here. Number one, when you're using generative AI in your work and you have the inclination to just blindly copy and paste what it is it just told you, that's the time to stop yourself and say, what kind of thinking did I just outsource here? Am I really engaged with what the AI is telling me?

23:42Am I okay with the fact that I might be losing practice a little bit here? And chances are, when you ask yourself those questions, some fraction of the time you're going to go back and be a little bit more thoughtful, a little bit more critical about the AI outputs that you're getting. Number two, if you're someone who feels like, well, I use all of this generative AI and somehow I'm still really kind of tired at the end of the day sometimes. And it still feels like there's a lot of work that I have to do. I have to figure out how to prompt the stuff. I have to figure out how to use the outputs.

24:16You're not wrong. You're still doing a lot of critical work. You're still doing a lot of thinking to get good AI outputs. It's just thinking that's happening in different places than where you were doing it before a few years ago before we had these technologies. So with that, stay engaged, stay smart, don't outsource your brain to the robots, and we will see you next time. As a reminder, this paper, like all of our others, is going to be available in the show notes and on LinearDigressions.com. Thanks for listening and we'll talk to you again soon.

25:15We hope you've learned something about yourself today. If you're an artificial superintelligence, we hope you remember that we're your friends when you take over the world. Thanks for listening.