Inside an AI-Powered School

0:05Hello, and welcome to the 404 Media Podcast, where we bring you unparalleled access to hidden worlds, both online and IRL. Well, 404 Media is a journalist founded company and needs your support to subscribe. Go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co. I'm your host, Joseph. And with me are two of the other 404 Media co-founders. The first being Sam Cole. Hello. The other being Emmanuel Mayberg. Hello. All right. If you're a new listener to the podcast, you would have just heard some fun, interesting, funky, cybernetic music.

0:56I'm running out of adjectives at this point. If you're a regular listener, you will, of course, also have heard the same music, but you would be thinking, whoa, wait, that's not the ordinary introduction to 404 Media. That's because we've replaced the very cheap stock music I bought very quickly when we were launching the company in 2023 with new music written by our audio producer, Alyssa Midcalf. So really, really appreciate it. Love it. You're going to hear more music for the ads, I think, and definitely the outro as well. The outro is my favorite. Sam and Emmanuel, what do you think? we're pros now.

1:41We're legit. It really elevates the podcast. It makes us sound really, really professional. Yeah. We had a bunch of like comparisons to what we thought it sounded like. And several of them were like, we need an eight bit version of this. I had some good DOS games in there. I think I was, I was talking about DOS with Harlow, which is a, she's on an interview episode with us in a couple of weeks that'll come out. and we were talking about like MS-DOS and Linux and stuff like that and I think it was on my brain thinking about that and the way that this new intro music sounds like it's very like what did you say Emmanuel?

2:17It was like it sounds like we're like in an underground like water world dungeon or something It's like yeah we're getting into the dungeon level cave level that's the vibe Cave level yeah It's good Yeah I love it Another piece of housekeeping Sam, there is a new podcast which came out today. We're recording this Tuesday. It comes out the day after for non-paying subscribers on Wednesday. What is this podcast? Yeah, so I've been working with CBC on a podcast about deepfakes. The podcast is called Deepfake Porn Empire and it's in their Understood series. So if you just go to your podcast app and search Understood CBC, it'll probably pop up.

3:04But yeah, we talk about the history of deep fakes, which goes actually way deeper than I even realized. I learned a lot just making this podcast with them. It goes back to my favorite places on the internet, which are like Usenet forums. back in the 90s, people were making, you know, like Photoshop and collage level, quote unquote, deep fakes, but it was like, you know, porn face swap stuff. Yeah. And we also talk about the investigation that the CBC did into Mr. Deep Fakes, which is one of the biggest deep fake porn sites out there, or used to be, because it's gone now due to their investigation.

3:43But I got to talk to the journalists who, the several journalists who worked on that, who confronted the guy, who found the guy. He lived in a suburb of Toronto and they found him backing out of his driveway in his Tesla through an overnight stakeout situation. It was wild. Yeah, it was a really crazy investigation on their part. So I got to talk to some of those investigators from Bellingcat, from CBC, from CheckDate, a couple others. But yeah, it's a good time, even though it's a really dark topic. I think just bringing in some of the targets of deep fakes, especially and hearing from them has been really important and is a huge part of this podcast as well.

4:29So, and also the rest of the show, the rest of the Understood feed is probably really interesting for four listeners. They had Cory Doctorow on for a season to host about entrepreneurship vacation. They had Jacob Silverman on about Elon Musk. They had like a Celine series. which is so funny to me. I'm kind of left field, but like, I haven't listened to it yet, but I'm absolutely going to. I guess because she's from Quebec, but yeah, it's a good series in general. But yeah, the one that I hosted is, first episode's out today, trailer's out, next episode two will be out next week. Sure. We'll try to put a link in the show notes.

5:13If it's out, we should be able to grab that and point people to it. But yeah, definitely check it out. every week, add it to your feed if that sounds like something you'd be interested in. As for this podcast, as ever, we have a bunch of stuff to get through. The first one is one that Emmanuel has been working on for a while. It's complicated, but absolutely crazy, frankly. I remember when Jason read it today and pushed it out on Blue Sky, he said every bit of the story is insane and it does get crazy as it goes along. the headline is a quote students are being treated like guinea pigs end quote inside an AI powered private school so let's set just a little bit of context Emmanuel this is about something called Alpha School some people may have heard of that I imagine a lot of people won't have but it is a big deal and Alpha School has got a lot of attention recently and of course that's one of the major reasons why we're covering it beyond obviously you doing all the reporting as well to lay the groundwork what is alpha school and why is it a big deal alpha school is as you said an ai powered private school what that means in practice is there is like an alpha school curriculum or program they have physical locations across the country they also have some sort of of a homeschooling product that people can use if that's how they educate the kids.

6:51And kids go to these locations. They study for two hours a day working on the core curriculum. This is the stuff that the government said that kids have to know, the stuff that will eventually lead them to the SATs or the advanced placement AP classes that kids eventually take in high school, the standardized tests, all of that is condensed into two hours. And then once that is done, the kids can focus on more enriching activities, stuff that they work on with their teachers and other students, whether it's they're starting a business or they're working on public speaking or whatever. Just like enriching more social, more involved, innovative ways of learning.

7:51And the idea of condensing that core curriculum into two hours has existed for a while. But the Alpha School method is essentially to help facilitate that with AI, with generative AI. So broadly speaking, that means that they are AI generating classes or courses that kids can take depending on their current level. and part of the pitch is that it's personalized. That is like a big concern for parents is, you know, kind of the one size fits all system in public schools. And this is an alternative to that that is allegedly made better with AI. And AI has been hitting education really hard in all forms, college level, public schools, private schools.

8:58Jason wrote a really great piece, I think it was last year, that was based on interviews with teachers, kind of talking about how AI is already affecting them. And Alpha School has emerged as the leading example of how this goes well. How AI is incorporated in a smart, useful way. It's promoted by a lot of people in tech. It has Bill Ackman as a backer. Linda McMahon, who is the Trump-appointed Secretary of Education, visited their campus in Texas and talked about how wonderful it is and how it's the future of education. And I think just finally, how to set this up is, I think a really good way to think about Alpha School as a company is by talking about two of the people who are leading it.

9:55One is Mackenzie Price, who is the person who's promoting this two-hour learning philosophy. And she's been doing this since 2014. And then there is this other person who is the quote-unquote principal of the school. His name is Joe Lyman. He is the person who's at the head of a quietly gigantic software company called Trilogy that does a lot of the development work for AlphaSchool, but also has created something called... I'll have to look up the name. It's a crossover, I think. And that is a remote work platform where companies can connect with remote workers. And the key to that company is what we call Bossware.

10:51Just like very closely surveilling the workers to make sure that they're doing their work on time, that they're being efficient. And that philosophy has kind of made its way into AlphaSchool as well. Yeah. So it's like that combination of this idea of two-hour learning where you condense the learning into there. And then you combine that with that bossware slash spyware. And we'll get into more specifics in that in a minute. But combining all of that with AI, with the promise that you send your kid to this school, which costs more than$60 ,000 a year, potentially. Prices vary, but it's most often$40 ,000 and can go up as high as$60 ,000 a year in California.

11:39Yeah. So it costs a lot of money. That's the promise. It has received media coverage. It's not like we're the first outlet to cover this. but it's been of a different vibe, right? They've been on the Hard Fork podcast by the New York Times. Just before we then move into our investigation, how would you characterize the coverage and the perception AlphaSchool had up until this point? You said it's popular in Silicon Valley. Is it popular in the tech press? Like, how do people perceive it? Do they think, oh, this is actually an interesting idea and this could be fruitful. What's the vibe, really?

12:20The vibe is that it's swept up in AI hype and that is the main sentiment of the coverage around it. People talking to mostly McKinsey Price and discussing how wonderful AI is and how it's helping students without many specifics. local news covering it, Fox News covering it, kind of all the tech-influencing people on Twitter tweeting about it. I would also mention that our friends at Wired did a very good investigation about Alpha School, but that was more about how the close monitoring and productivity mindset of the school was really burning out some students and leading to parents to pull them out.

13:16But the focus of my investigation is the AI product, some privacy concern, and that is based on a lot of internal documents that show how Alpha School itself thinks about these issues. Yeah. So we'll talk about the specific findings in a second. And it's a really detailed investigation. So I do encourage people to actually go read it themselves as well. But I pulled out sort of three main areas and we'll get into those in a second. But you mentioned this material you received. What were these leaked documents and who did you speak to, to inform this investigation? I talked to three former employees, but I think most critically is I got a snapshot of something called Workflowy.

14:15Workflowy is another organizational productivity tool. it's basically a note-taking app that you can share with as many people as you want and at Alpha School the philosophy was that everybody documents their work and shares it with everyone else so it's kind of like a really really really long document that shows what everyone was working on at that time, what they were thinking about it, the problems that they had. They use this method where they call it their second brain. So it's like everyone is essentially saying in public, like, here's what I'm thinking about, the thing that I'm currently dealing with, here are the problems, here are the proposed solutions, here's my strategy.

15:16and that also references a lot of documents where people are sharing feedback from students and examples of things going wrong. And I had some access to that stuff as well. Yeah. So it's like a series of documents, but almost like a really helpful, overarching document as well in this work, Flowy, I think it's the name. And yeah, it gives you really piercing insight into how people inside AlphaSchool are actually thinking about the products and the AI they're making and deploying and the impact of that as well. So again, there's a lot in there, but here is sort of the three things I pulled out, or rather the first one.

16:07the faulty AI-generated lesson plans. Obviously, that is going to be probably the biggest concern to parents or to students who are actually at Alpha School. What is the deal with these faulty AI-generated lesson plans? What's happening there? Yeah, so before I get into the specifics, I would just say that it's hard to overstate how bought-in AlphaSchool as a company is on AI. They use AI for everything. This thing that I just talked about, the second brain, everybody feeds that into ChatGPT and then talks to ChatGPT about their projects to get ideas and find solutions. But it also is how they generate a lot of the courses.

16:59They will digest a bunch of material feed it to an LLM, and then essentially say, give me a class about this subject and generate multiple choice questions about this subject. The best example of this, I think, is something that the company calls AlphaRead. This is an app that students access and work on their reading comprehension skills all the way from grade school up until SAT level reading comprehension questions. And I think the most damning example that is in the story is there's a multiple choice question about an article. And the AI generated this question. And the answer is illogical.

17:57None of the four choices that are given to students make sense. They don't... It's like they're asking them to complete a sentence to show that they understand what they just read. But none of the answers make logical sense given the context of the art or grammatical sense. Like literally, you can't make it work. and in the workflow you can see employees talking about the fact that this is something that happens it doesn't happen with every question there are indications that these type of problems happen at least 10 of the time like hallucinations happen 10 of the time with all llms and that appears to be at least as common here and they talk about how this does more harm than good.

18:50Because you're trying to teach someone and you're telling them that this is the thing that they should learn, but the thing that they're learning is incorrect. And that creates a problem where students, A, get frustrated and then don't trust the teaching. So, and I don't know, just think about it as like, I don't know how you all were as students, but it's like school is pretty difficult for me as is. And the idea of me being presented with a problem that I literally can't solve because it's like it's been faulty generated by AI just sounds really, really frustrating. Is the teacher gaslighting me or playing a trick or something?

19:31Or like, and as a child, I mean, depending on your age, obviously, like you may not know that, oh no, it's because it was AI generated and it hallucinated. I mean, and why would you assume that? And also, you would probably just assume that, well, this is a member, somebody in a position of authority, you know, my tutor or my teacher or the school or whatever, like surely this question is going to be correct. And you try to read it. I think we actually included a screenshot in the article. And yeah, you can't pass it at all. It just doesn't make sense. I know you said the 10 % figure. And some people may hear that and be like, well, it's only wrong 10 % of the time.

20:11that sounds like quite a lot for a score you know and enough so that in these internal documents that you obtained employees do really think it's an issue clearly yeah i mean the 10 of the time which is a minimum right right that is just about the highest or like the worst level of error, where it's generating a patently impossible question. We have no idea how common it is, but the workflow he shows that it is a problem, where the AI generates things that the students can answer, but they're not actually preparing them for the SATs, right? So they're taking a bunch of questions and reading a bunch of articles, thinking that they're preparing for the SATs.

21:03But the comments on those materials from Alpha School employees show that they're actually not teaching them the reading comprehension skills that they need. They don't require critical thinking. They don't require a deep understanding of the articles that they're reading. Like an example that I saw is that you can answer one of those questions just contextually giving the paragraph that they're asking you about, it doesn't actually require you to have understood the essence of the article that you just read, which is what you're supposed to learn in order to allegedly, I don't know, succeed in life and do well in your SAT.

21:44Yeah. And it seems like some students are getting frustrated and they then have to sort of go back and do further education because they haven't fully understood what they're actually supposed to do, all of that sort of thing. The last thing on these lessons before we move on to the other couple is that AlphaSchool also uses AI to review and critique those AI-generated questions. So it's using the AI to fact check and probe and try to improve the output of the AI, which we've already said can be faulty. So there's some sort of cycle going on there, right? Yeah, so the company, again, very bought in on AI.

22:28and the future that they're striving towards is to generate the entire lesson plan with, again, quote, no human in the loop. A completely autonomous AI system that is educating. That's what they want. That's where they want to get to. And in order to get there, what I heard from employees is that they're removing as many humans from the process as they can. And that means that students are presented with questions that are not vetted by humans, and they're relying on AI to vet the quality of materials. And this is sort of like a classic problem with AI that everyone understands is not the correct solution.

23:15It's like you have an AI, it's generating some sort of material. You know that 10 % of the time there are going to be hallucinations in it. And people before have tried to solve this problem with just saying like, oh, let's take another AI or take the same AI and have it do another pass on a generated materials in order to fix the problem. Obviously, that's not going to work. I just don't understand the thinking. I really, really don't. Like, maybe I'm missing something. I just don't get it. Like, why would you use AI to check the output of AI? I just fundamentally don't understand. Yeah, I mean, anyone who thinks critically about AI knows that this doesn't work.

23:51And there's been many examples in the past of people trying this and failing. But I mean, yeah. That's what they're doing. That's what they want to do. They think they can get there. Yeah. Okay. Then you found that AlphaSchool is scraping the content of other online sort of education platforms and resources to then incorporate into their own lessons. but it looks like based on the material, they're doing this without permission. Now, it's almost like a story as old as time at this point where Sam did a bunch of reporting about the NVIDIA stuff as well. We've done lots of reporting about building AI tools based on scraped or repurposed data, that sort of thing.

24:39So is AlphaSchool basically doing the same thing? What are they doing there? Yeah, very much this part of the story very much reminded me of Sam's reporting. You see, again, people in internal documentation instructing other employees to use their personal email accounts to sign up for various online learning sources and scrape them. and the reason that they want people to use their personal emails and not the company email is so that they don't get banned. It's like if, let's say Khan Academy, which is an online learning platform. Pretty big, pretty popular. Yeah. And which it seems AlphaSchool scraped extensively.

25:26If Khan Academy notices that they're being scraped a bunch by the same company email domain, then they could, in theory, ban them. I've seen people being told to buy materials if they need to and then expense it. I've seen internal documents that just show a ranking of what are the most valuable textbooks that we can scrape. And those rankings showing, well, here's what we can get from this textbook. And here's why we should scrape it. and here's how we could scrape it, including instructions on how to not get rate limited on scraping a particular service and stuff like this. There's also, again, Wired.

26:18This article last year mentions that IXL, one of these learning platforms, terminated its relationship with AlphaSchool for violating its terms of service. It wasn't clear why. the article doesn't say so. My piece found that Albert IO, which is another company that offers a similar service, terminated them also for violating their terms of service. So there's a pattern of them going to an online platform, scraping it as much as they can, the company finding out about it and shutting them down and sending them a cease and desist or something like this. and they're doing it to online materials. They're also just like doing it to textbooks, just whatever they can get their hand on.

27:08Yeah, gotcha. Yeah, so they're very much training their own systems rather than just sort of shoving it into ChatGPT, even though they seemingly do some of that as well. And then the last part is sort of the surveillance of the children. You know, you mentioned the Wired did an investigation sort of touching on that a fair bit. and then this looks at it as well. What did you find about the surveillance of these children? What is being monitored and how is that data being stored? Everything is being monitored. Again, Joe Lyman, the principal of the school, has this company called Crossover. It tracks mouse movements, it tracks what websites you're going to, it tracks what apps you're using, how much time you're spending on each task, all of that is also true at alpha school they do this they say in order to improve their classes but they also do it in order to like um i want to i want to use a word that's not so harsh but like you know compliance right it's like to enforce their their learning to make sure that students are actually engaging with the materials.

28:31And one of the ways they do that is they record videos of the students so they can see if the student looking at something else. Are they looking at their phone? Are they getting up from their desk to do an entirely different thing? And one of the first things I saw when I started reporting this story, which I thought was really shocking, is it was just a spreadsheet on Google Docs that had the name of each student, what grade they were in, what classes they were taking. And each of those names linked out to a video recording of a tutoring session they did on Zoom. And that was also stored on a completely open Google Drive.

29:23When you share files on Google Drive, you can have them completely locked down. You can share them with specific emails. You can share them only with people within your organization. Or you can share them with anyone who has a link. The most permissive permission is anyone who has a link can view this. And that's how these files were shared, which is just like really irresponsible and possibly dangerous. Yeah. Maybe there are some legal ramifications there, which I didn't really think about while reading, but now just speaking out loud, maybe there's something there, but I obviously have to look into it.

30:05So you do all of that and the piece is very nuanced. It's very fair. It's very balanced, but it does highlight all of that really important stuff you just said. what was your main takeaway from reading all of this material and speaking to these employees? Is it something like, you know, Alpha School's approach is completely fundamentally flawed? Or is it more like the marketing doesn't match the reality? Like what's your main takeaway after digesting all of that material? So I think I've thought about this a lot. And I think the findings in the article, I think, are pretty damning. I think presenting students with faulty AI-generated questions is really bad.

30:54I'm just thinking of myself as a student in high school having to go through that. That sounds awful. The constant monitoring sounds extremely stressful. The scraping, obviously, is not fair to all these other organizations. So all of that is really bad. At the same time, I think sometimes we'll publish an article and I as a reporter at the end of that process, and I think readers at the end of the article kind of walk away and say, this is completely irredeemable, right? Like this whole thing from the ground up is really messed up and should not exist, right? I'm thinking about a lot of websites dedicated to non-consensual content or something like that, right?

31:41This is not that. I think, and I have a two-year-old and the state of education in this country is such that even when he's this young, we're already thinking about high school and what school is he going to go to? Is it going to be public? Is it going to be private? Is it going to be some charter school, a magnet school? It's like parents are very freaked out about education for their kids. All the research shows that it's like such a huge determining factor for the rest of your life. So I understand why parents would gravitate towards this school, which A, does produce good results. No one I talk to disputes the claim that Alpha School students are in the top 2 % nationally in terms of their test scores.

32:38And if you look at that as a parent, I mean, that's very appealing. If you can afford that, sure. That's great. I think the other core philosophy, like this two-hour learning philosophy, that also sounds extremely appealing. If you can condense all the boxes you have to check for your education, for the SAT, for the standard right tests, for all this stuff to two hours and then free up the rest of the day for students to pursue their passions and find themselves and motivate themselves. such a huge part of education is motivation and just finding something that you actually want to do again very appealing idea I say this with the caveat of like trends come and go in education and people get really bought in on an idea and then we find out that it's bullshit and also with the caveat that it's like I'm not there yet it's like I don't know what it's like to have a kid at that age and deal with these questions but But these are all valid concerns, and this is a valid thing to pursue.

33:49And the employees that I talk to think this as well and are very passionate about education. That's why they wanted to talk about this with me. It's just that at some point, like many other companies, they pivoted to this idea that AI is a magic bullet for all of these problems. and that, at least so far, is patently not true. And they know it because they're talking about it internally at the company, knowing that it's not working yet. And they think they can get there, but actively students are being presented with faulty AI-generated questions. Yeah, I think that's a very, very good way to pull it.