What Happens After the Homework Apocalypse?

0:00Today on the AI Daily Brief, what comes after the AI homework apocalypse? The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes.

0:17Hello, friends. Happy weekend. It, of course, being the weekend and a long weekend at that, means that we are doing an AI long read. And this week, we are back with Professor Ethan Malek, who is writing about post-apocalyptic education. This seems extremely relevant to me given that it is Labor Day weekend in the US, meaning if schools aren't back in session yet, they will be by the middle of next week. So let's read this piece, and then we'll come back and discuss it. Last summer, I wrote about the homework apocalypse, the coming reality where AI could complete most traditional homework assignments, rendering them ineffective as learning tools and assessment measures.

0:52My prophecy has come true, and AI can now ace most tests. Yet remarkably little has changed as a result, even as AI use became nearly universal among students. As of eight months ago, a representative survey in the U.S. found that 82 % of undergraduates and 72 % of K-12 students had used AI for school. That is extraordinarily rapid adoption. Of the students using AI, 56 % used it for help with writing assignments, and 45 % for completing other types of schoolwork. The survey found many positive uses of AI as well, which we will return to. But for now, let's focus on the question of AI assistance on homework.

1:25Students don't always see getting AI help as cheating. They are simply getting answers to some tricky problem or a challenging part of an essay. But many teachers do. To be clear, AI is not the root cause of cheating. Cheating happens because schoolwork is hard and high stakes. And schoolwork is hard and high stakes because learning is not always fun, and forms of extrinsic motivation like grades are often required to get people to learn. People are exquisitely good at figuring out ways to avoid things they don't like to do, and, as a major new analysis shows, most people don't like mental effort, so they delegate some of that effort to the AI.

1:54In general, I am in favor of delegating tasks to AI, but education is different. The effort is the point. This is not a new problem. One of the first uses of any new technology has always been to get help with homework. A study of thousands of students at Rutgers found that when they did their homework in 2008, it improved test grades for 86 % of them. See, homework really does help. But homework only helped 45 % of students in 2017. Why? The rise of the internet. By 2017, a majority of students were copying internet answers rather than doing the work themselves. The homework apocalypse has already happened and may even have happened before generative AI.

2:27Why are more people not seeing this as an emergency? I think it has to do with two illusions. The first illusion is the detection illusion. Teachers believe they can still easily detect AI use and therefore can prevent it from being used in schoolwork. This detection illusion leads educators to rely on outdated assessment methods believing they can easily spot AI-generated work, when in reality, the technology has far surpassed our ability to consistently identify it. No specialized AI detectors can detect AI writing with high accuracy and without the risk of false positives, especially after multiple rounds of prompting.

2:56Even watermarks won't help much. People can't detect AI writing well. Editors at top linguistics journals couldn't. Teachers couldn't, though they thought they could, the illusion again. While simple AI writing might be detectable, there are plenty of ways to disguise AI writing styles through simple prompting. In fact, well-prompted AI writing is judged more human than human writing by readers. You can't ask an AI to detect AI writing even though people keep trying. When asked if something written by a human was written by AI, GPT-4 gets it wrong 95 % of the time. There are still options to preserve old assignments.

3:27Teachers can return to in-class writing asking students to demonstrate their skills in person, or other techniques that might mitigate AI cheating through close monitoring. But for the vast majority of teachers, doing so requires adjustment and changes that have yet to be made. To date, few have actually reacted to the shattering of the illusion of AI detection by shifting how they approach teaching and assessment. While teachers grapple with the detection illusion, students face their own misconception, illusory knowledge. They don't actually realize that getting help with homework is undermining their learning.

3:54After all, they are getting advice and answers from the AI that help them solve problems, which feels like fluency. As the authors of the study at Rutgers wrote, there is no reason to believe that the students are aware that their homework strategy lowers their exam score. They make the common sense inference that any study strategy that raises their homework quiz score raises their exam score as well. The same thing appears to be happening with AI, as a study by some of my colleagues at Penn discovered. They conducted an experiment at a high school in Turkey, where some students were given access to GPT-4 to help with homework either through the standard chat GPT interface or using chat GPT with a tutor prompt.

4:25Student homework scores shot up, but the use of unprompted standard chat GPT to help with homework undermined learning by acting like a crutch. Even though students thought they learned a lot from using chat GPT, they actually learned less, scoring 17 % worse on their final exam. Despite this, the survey I quoted earlier found that 59 % of teachers see AI as positive for learning, and I don't think they are wrong. While just using AI as a crutch can hurt learning, more careful use of AI is different. We can see signs of this in the Turkey study, which found that giving students a GPT with a basic tutor prompt for ChatGPT, instead of having them use ChatGPT on their own, boosted homework scores without lowering final exam grades.

4:59Plus, a study done in a massive programming class at Stanford found that use of ChatGPT led to increased, not decreased, exam grades. And of course, students are not using AI just to do their homework. They are getting aid in understanding complex topics, brainstorming ideas, refreshing their knowledge, creating new forms of creative work, getting feedback, getting advice, and so much more. Focusing on just the question of homework and the illusion it fosters can discourage us from making progress. Today's episode is brought to you by Venice. Venice is a private, uncensored generative AI app. It accesses open source models to enable text, image, and code generation without the fear of being spied on or having your data exploited.

5:33Discuss anything with Venice without concern

5:42Thank you.

6:05Visit venice.ai slash NLW and enter the discount code NLWDailyBrief. That's NLWDailyBrief, all one word. Today's episode is brought to you by Superintelligent, which is, of course, our platform that helps you learn how to use AI tools and perhaps even more importantly, gives you ideas on the best use cases that are actually going to help you achieve whatever it is you want to achieve. To recognize the end of summer and back to school slash back to work, we are running our best promotion ever. When you sign up for Superintelligent, between now and the end of August, using code SOBAC, your first month will be 100 % free.

6:44The platform features over 600 fun, highly practical AI tutorials that get you using AI fast and with an eye to actually transforming how you get things done. We've just launched Super for Teams, so if you have a group of people at your company that want to figure out how to use AI together, I highly suggest you check it out. But for those of you who are using Superintelligent as an individual, once again, if you sign up for Superintelligent between now and the end of the month using code SOBAC, you will get your first month 100 % free. go to besuper.ai and check it out today. Section, Encouraging Not Replacing Thinking.

7:20To do so, we need to center teachers on the process of using AI rather than just leaving AI to students or to those who dream of replacing teachers entirely. We know that almost three quarters of teachers are already using AI for work, but we have just started to learn the most effective ways for teachers to use AI. A recent deep qualitative study of teachers found that teachers who used AI for both output, create a worksheet, develop a quiz, and to help with input, help me think through what makes a great American novel, give me ways to explain positive and negative numbers, get more value than if they use AI for producing output alone.

7:48This points to a useful path forward in AI for education, using it as a co-intelligence and tool for helping humans do better thinking. Increasingly, AI is being used in the same way for students, pushing them to think rather than use AI as a crutch. For example, we have released multiple prompts that instructors can customize or modify for their classroom. These sorts of prompts are designed to expose illusory knowledge, forcing students to confront what they know and don't know. Many other educators are designing similar exercises. In doing so, we can take advantage of what makes AI so promising for teachers, its ability to produce customized learning experiences that meet students where they are, and which are broadly accessible in ways that past forms of educational technology never were.

8:24The integration of AI in education is not a future possibility, it's our present reality. This shift demands more than passive acceptance or futile resistance. It requires a fundamental reimagining of how we teach, learn, and assess knowledge. As AI becomes an integral part of the educational landscape, our focus must evolve. The goal isn't to outsmart AI or pretend it doesn't exist, but to harness its potential to enhance education while mitigating the downside. The question now is not whether AI will change education, but how we will shape that change to create a more effective, equitable, and engaging learning environment for all.

8:54All right. Thank you once again to Ethan for another great piece. I think that Professor Malik here is exactly right that the key transformation in the short term has to be what our expectations are around how education works. The good news is that education has been so desperately needing an update for so long that while painful, it's hard for me to imagine that if we do this right, things don't get better. The challenge, of course, is that it's not just students who don't like mental effort. Teachers who are already overworked and underpaid as a general rule don't in many cases have the time and space, and in other cases the appetite, to try to do things fundamentally differently.

9:32And fundamentally different is what AI is going to require. These are real challenges. How do you teach what good writing is when ChatGPT can just write for you? My instinct is that the only answer to questions like that is to go much deeper on the question of what good writing is, to get deeper on things like narrative. But creating those new frameworks isn't easy, and it's going to take time. And in the middle of that, there's no way this isn't just weird and challenging in the short term. but look, for creative educators, this will represent a new opportunity unlike anything we've ever seen and it's hard for me not to be optimistic in the long term.

10:06Let me know what you think in the comments either on Spotify or on YouTube and until next time, peace.