Model Collapse: The Warning from AI Researchers

0:00AI researchers have recently come out with a report that they warn is going to cause model collapse for all AI models. And this is actually a really big issue that is currently happening to AI, perhaps one of the bigger ones. So today on the podcast, we're going to talk about what model collapse is, what the AI feedback loop is, how this is going to impact AI going into the future. Spoiler alert, OpenAI is going to be a massive benefactor of this entire debacle. So without further ado, let's dive into it. So the center of this issue is the fact that up until very recently, all of these AI models like ChatGPT and OpenAI have been trained off of human content.

0:41They've taken content like the entire, you know, library of Wikipedia, everything ever written there, everything ever written on Reddit, everything ever written on the internet up until fairly recently was all human generated. And that's what all of these AI models and LLMs have been trained on. And, you know, they've been able to take this data, they've been able to fine tune it and do a lot of different things with, you know, books and photographs and a lot of different content and train models off of it. But a fundamental question arises as all of this AI generated content is now starting to flood the Internet, which is potentially replacing human generated content as the primary data source for the next round of AI model training.

1:22So, of course, we have the big guys like Google and OpenAI who have built these massive models off of purely human content. But now, as a lot of these different AI startups are coming onto the scenes, they're looking to train their models as well. And some of them are still trying to get, you know, some of the same data sets like Wikipedia. But there has been news that a lot of different content on Wikipedia even is being generated by ChatGPT or different AI tools. And so the issue becomes, and it's starting now, but it's only going to proliferate into the future. become, I guess, a bigger issue is that more and more of the internet is going to be written by AI and it's going to, these AI models are going to be trained on AI.

2:00And there's an issue that arises when AI is trained on AI models, and that is called model collapse. A team of researchers from the UK and Canada took on this challenge and they recently published their findings in an open access Journal, ARXIV, and their insights, I think, might actually prove to be an obstacle for future generations of AI technology, because what they found was a little bit startling. They said, the use of model-generated content in training leads to irreversible defects in the resulting models. The study really particularly focused on probability distributions for text-to-text and image-to-image generative models.

2:41So the researchers concluded that models trained on data created by other models are prone to model collapse. So essentially, over time, the models lose sight of the original data, leading to distorted perceptions, and if there's an error, that error compounds into the future content that's generated. So to kind of show the gravity of the situation, Ross Anderson, who is a co-author on the paper and a professor of security engineering at Cambridge University. He said in a blog post recently, we're about to fill the internet with AI generated blah. And then he also went on to kind of emphasize that the overwhelming presence of AI created content could essentially grant an unfair advantage to companies that already have a really robust human data set, or control large scale human interfere interfaces.

3:32So I mean, it's kind of interesting, because like, obviously, chat GPT has an advantage for already having their data set collected and already having their product built. But there are other companies that are going to have advantages. And that's any company that can pretty much guarantee up to a certain point, their content was human created. So even a company like Quora, you know, Quora, I'm sure at this point is being spammed by tons of chat GPT responses and spam might not be a good word because perhaps some of them are really good responses, but they know for a fact that if they look back X amount of years, all the content on their platform would have been human generated, you know, before any of these tools were mainstream.

4:10And so they do have an advantage there, knowing that and being able to have access to that data where any new startups will not have necessarily that same control. So a sci fi author, his name is Ted Chiang. He's known for Story of Your Life, which is essentially just a novella that inspired the movie Arrival. He compared the degradation of AI models due to essentially repetitive copying to the increased artifacts seen in repeatedly copied JPEG images. So the phenomenon of model collapse, the researchers explained in their paper, comes when data generated by AI models contaminates the training set for subsequent models.

4:51So the model's preferences for more popular data over less popular or improbable data leads to progressive distortion and eventual loss of rare but crucial data attributes. So even with 10 % of original human author data in the mix, model collapse still transpires. And that is definitely it happens at a lower a slower pace. And you can kind of imagine if you wanted like an example of what this actually means and what's going on. If you just gave like an AI model, let's say an image one, like a picture and said, make a variation of this. So it makes a variation. And you said, okay, make a variation of that, make a variation of that, make a variation of that, make a variation of that.

5:30Eventually the, the, the resulting image would be quite different than the original image. And, um, there's actually some really interesting things that can happen, including like corruption of concepts. So like if you told it to generate an image of a green bear and it was like, you know, a green bear, but then it slightly became less green and more blue and like the image was slowly shifting in the future iterations, eventually the model could come to believe if it was training off of the data that it created that, you know, what might've started out as green eventually morphed to blue. And then when you ask it for something green, it could come up with something blue.

6:06So there's a lot of different issues. I mean, that's just an example of what could happen to images, but you can take the same example a lot farther where it comes to text, where it straight up just comes up with something that's completely wrong or false. But it has been, you know, sort of restated over and over again by the model in different ways. And if it's training off of its own data, or if it's reiterating off of that same data or training off of other AI content, right, like, let's say there was an issue with chat GPT, where it believed something was wrong. And 1000 articles were written about that thing in a future model came along and scraped the internet and got those 1000 models.

6:40Now that fact is like compounded into this future model, which is now going to continue to perpetuate that falsehood. So it's a it's a serious issue. And, you know, a handful of solutions have been proposed. Of course, there's the labeling all content ever generated by AI, which the EU is trying to pass bills that are going along that line. There have been bills in the US by Congress that are in that direction as well. it is definitely a big challenge though and Google is trying to tackle it and as well as a number of other companies but it's not necessarily easy to just go and label everything created by AI because essentially it's just kind of a cat and mouse game at this point where there's a lot of tools to obstruct the fact that a piece of content was written by AI so the second solution that's been proposed is just the you know option to keep a very pristine data set of exclusively human generated content and of course chat gpt and open ai will have something like that and so it's left to be said if there will be open source versions which would be absolutely massive if someone compiled an open source version of you know essentially the data set someone like open ai has now of course data is very expensive and so that's highly unlikely that that will happen unless there's some sort of leak or hack which i really wouldn't put it past um the the internet to be able to do so i think it's going to be really interesting and if that did happen then you could always continue to retrain or train all models based off of that human exclusive content.

8:06And perhaps you could introduce new content into it if you were sure it was human generated and just hope and assume that if there was something that slipped in, the vast majority of the original document or original data set would be able to offset it. So I think despite a lot of these kind of unsettling findings here, there is a silver lining for human content creators. So I think that this is something I've been saying for a while, but in a future that's really dominated by AI tools, I believe the value of human created content is going to rise a lot. Because I mean, if nothing else, it's going to be like an untainted source for training data for AI.

8:44But also, I think people really will appreciate if there is a way to distinguish being able to consume content that they can guarantee and know was created by an actual human. This wasn't an AI, there was thought and energy and time put into something. Honestly, that's one of the reasons why this podcast, I like talk the podcast. I don't really edit this a lot. If I say ums or buts or pause or think, I don't really edit that out. And this is something that I've actually decided to do very, very early on with this podcast is because I believe whether or not you like my ums now, there's going to be a craving for that sort of human input, that sort of human responses in content that you don't get from AI.

9:25So previous to this podcast, I've actually started five, six, seven different podcasts that were exclusively run by AI, meaning I had an AI voice. I got the scripts for different podcasts to be written by AI, then the AI voice, which I got a really good one. You couldn't actually tell it was an AI to read the scripts. And I had successful podcasts with thousands of listeners and, you know, hundreds of thousands of listens on them, all running off of AI. but the problem was, you know, when I started doing this podcast, I realized there really was a need to create something that was an actual human.

10:02And inevitably, I figured this crowd, if anyone would be able to pick out AI better than anyone else, and the value is so much higher. So I opted to host this podcast myself, in addition to the fact that it's, you know, really interesting to learn about all the different stories that are happening in AI. But in any case, this is a massive issue today. I think that this really highlights the importance of enhanced methods to ensure the integrity of, you know, generative AI models over time. And I think this is something that AI companies are going to grapple with well into the future. And perhaps the data sets of today will continue to be the most valuable.

10:39So when you see chat GPT that says after 2021, they can't, you know, they don't have any more data, perhaps the reason for that cutoff date is because the DaVinci model was coming out. And OpenAI didn't want anything that the DaVinci model created and put onto the internet to kind of influence GPT-4. And to be honest, there's so much content they already had that they can continue to train chat GPT and just improve it based off of the data set they have. So I'm not sure if, you know, including a ton more data or continuing to scrape the internet and adding it to their data set is actually going to make it more of an effective model.

11:17So I think this is going to be interesting. I think that while this is definitely going to pose a challenge to the evolution of the AI industry, I also think that it opens up a couple new avenues for different AI technology into the future. And so this is something that is going to definitely have disruption, but also create a lot of opportunities for new companies to come in and solve this problem.