TikTok Experimented With Users’ Algorithms. One Died By Suicide

0:00Some people treat ChatGPT like some kind of smart search engine, and some use it to get work done. ChatGPT Work is a new way of working in ChatGPT that can take action across your apps and files, stay with a project for hours if needed, and turn a goal into finished work. It's designed to help you move from a chaotic starting point to a reviewable first version. So all the source materials, briefs, and scattered information that you have to grind through to turn into something useful can just become something useful. Put ChatGPT to work on your most ambitious ideas and projects. Get started at ChatGPT.com by selecting Work Mode, available on Plus and Pro plans.

0:42Healthcare doesn't always work great. If you've ever waited on a refill or couldn't schedule an appointment, you get it. That's the kind of stuff Optum is changing. They're using data and technology to integrate patient care, pharmacy, and everything else. So healthcare is connected, not complicated. What's that look like? Cheaper prescriptions that are easier to get and care that looks at the whole person, how you need it. Optum is helping make healthcare work as one for everyone. Learn more at business.optum.com. As industries evolve faster than ever, companies need an environment that accelerates strategic growth.

1:17And Michigan delivers on that promise. From emerging startups to global enterprises, Michigan offers what executives value most, a resilient, innovative ecosystem, diverse communities that attract top talent, and a quality of life that supports work-life balance. With our unified Team Michigan approach, businesses scale faster and compete at the highest level. Michigan, pure opportunity. Seize your opportunity at michiganbusiness.org.

1:50Before we start, I want to let you know we talk about suicide and self-harm in this episode. Please take care while listening. Chase was a normal kid. He was an honors student. He was a football star. Loved hanging out with his friends. Chase Nazca was a 16-year-old boy who grew up on Long Island. He loved going to the gym. He loved weightlifting. He did his homework. He was just a normal kid, a kid who would connect with his friends on social media. Chase would go on TikTok and watch videos about weightlifting and basketball, motivational speeches, funny clips. That's the kind of stuff he was searching for.

2:33That's just not what the algorithm fed him. Chase also used the app to listen to songs about loneliness, hopelessness. And in the winter of 2022, his feed took a turn, serving him more and more content in that vein. This is video after video after video about depression, sadness, hopelessness, loneliness, self-harm, and suicide. Chase's TikTok was stuck in a dark loop. The videos he shared with his friends got sadder. And in February of 2022, Chase died by suicide. His parents filed a wrongful death lawsuit against TikTok the next year. Bloomberg investigative reporter Olivia Carville wrote a story about Chase for Businessweek in 2023.

3:23In response to questions from Businessweek at the time, a TikTok spokesperson said that it didn't comment on pending litigation, but that the company was committed to the safety of its users, especially teens, and that it would continue their, quote, significant investment in safeguarding our platform. For the past several years, across the U.S., thousands of suits have been filed against the world's biggest social media companies, alleging that the platforms have harmed the mental health of young people. These cases have been filed by school districts, families like the NASCA's, and by state attorneys general.

4:01And last week, on August 26th, one of the most consequential of these cases, against Meta, reached a historic settlement. Facebook has agreed to a landmark$17 billion settlement over teen social media addiction and settled claims filed by 47 states. Olivia says the Meta settlement could mark a sea change for the social media industry and bring some of the accountability people like Chase's parents and many other families have been pushing for. They've been telling me they're screaming into the wind. No one's listening. They've been calling for Congress to roll out laws. And we finally saw change through the justice system, which, you know, is remarkable.

4:45This litigation has been going on for half a decade. And August 26th was a huge moment for so many people. I'm Sarah Holder, and this is The Big Take from Bloomberg News. Today on the show, how did Chase Nazca's TikTok feed change so significantly? Bloomberg investigative reporter Olivia Carville on the consequences of a TikTok algorithmic experiment. And how Meta's landmark settlement could spark broader changes for how social media platforms approach safety for young people.

5:26TikTok is famous for its powerful algorithm. It learns what videos users like to see and shows them more of them. And a few years ago, that algorithm was under a microscope. So back in the summer of 2021, the Wall Street Journal and other media outlets started investigating TikTok's algorithm because it was so mysterious. And what these reporters started to uncover is that users who were interested in things like healthy eating or excessive exercise or fitness started to get pushed content that was more and more extreme. So healthy eating turned to eating disordered content or posts about breakups or sadness over a relationship breakdown turns to self-harm videos.

6:13Some users were getting caught in something called a filter bubble. essentially an echo chamber of content that could be as benign as cat videos or much more harmful. And so the company was under scrutiny for these filter bubbles in the summer of 21. By the end of that year, they announced they're going to start testing ways to break up repetitive streams of videos about harmful content like loneliness or extreme dieting. But this month, Olivia reported on a confidential internal document produced by TikTok back in 2023 that was turned over to lawyers as part of a legal case against the company and placed under a court-ordered seal.

6:56The document revealed that those changes to TikTok's algorithm did not go out to all users. As we know from what is contained in the confidential document, in early 2022, they made the call to roll out that safeguard to 90 % of U.S. users to ensure that they weren't going to be pushed into filter bubbles of harmful content. But they held back that control group. 15 million people did not get the safeguard. One of those people was Chase Naska. Chase Naska, the 16-year-old from Long Island. His account fell into that filter bubble of harmful content starting on January 25th, 2022. He was fed thousands of videos about depression, sadness, hopelessness, loneliness, self-harm and suicide right up until his own death.

7:48On February 18th of 2022, Chase Nasker stepped in front of an oncoming train and took his own life. What we know now, looking back, is that days before he died, he did view a TikTok video of a young man standing on train tracks with the caption, went for a quick little walk to clear my head. So he viewed content about the method of suicide that he ended up taking his own life by. When Olivia first reported on Chase in 2023, TikTok had not disclosed anything publicly about the filter bubble test. So in 2023, I asked the company, why was this 16-year-old boy being served suicide content by your algorithm?

8:34What the company disclosed publicly for that Businessweek article was, our hearts are with the family. This is a real tragedy. We do everything we can to protect young users on our platform. We have robust trust and safety detection teams. We block harmful content or anything about suicide and self-harm. The story is what they didn't say. What Olivia recently learned is that behind the scenes at TikTok, her request for comment sparked an internal review, which culminated in an internal document that the company titled Bloomberg Businessweek, Youth Safety and Filter Bubbles, Escalation Summary.

9:15Individuals from TikTok's algorithm team, their trust and safety team, do a deep dive of Chase's watch history in the weeks, hours and days leading up to his death. They review more than 7 ,500 videos that Chase had watched. And what the company determined is that through a back test experiment, what they call an AB experiment, which is relatively commonplace in the social media industry, what that internal review found is that this teenage boy's account was placed in an AB experiment. He was randomly selected in January of 2022. Within a month of Chase Naska being placed in that AB experiment, he was dead.

10:01Wow. And what did TikTok say about why they were running such an experiment in that confidential document? So all I can say is what we've reported on thus far, which is that the company was running these experiments to test user engagement. There are two really important quotes that came from the confidential document we reported on. One, the company said, TikTok's filter bubble prevention strategies did not take effect on this user by design. By design. By design. The second important quote in the document is TikTok essentially explaining why they ran that back test. Why was it important to conduct this experiment and hold back that control group from a safeguard?

10:49And they explained that the user impact decision was a delicate balance across safety and the ability to measure impact on DAU and core metrics. DAU is daily active users. So the company wanted to know how the engagement levels of its user base would be impacted between the 90 % of users who had the safer version of the algorithm and the 10 % that did not. 10 % of TikTok's user base was 15 million people. What has TikTok said in response to your latest reporting on this confidential document? So the latest statement from TikTok was very similar to what we got in 2023 when we wrote that story about Chase Naska, kind of echoed it almost word for word.

11:37What TikTok said in response was that their hearts go out to any family that experiences a tragedy like this. And they also talked about how they're doing everything they can to prioritize child safety on the platform. They talked about how they have robust detection systems to try and flag this kind of content and that they're working at safeguarding the platform, making it safer for kids. And don't forget that, you know, TikTok had more than 40 ,000 people working on its trust and safety team, trying to ensure that its platform is safe and trying to ensure that kids aren't seeing this kind of content.

12:14They don't want teenagers seeing nonstop videos about suicide. That doesn't help them in any way. And that's important to remember too. They've invested more than a billion dollars in trust and safety over the years, and they continue to do so. It's just that because the companies are so prioritized to focus on engagement, to focus on profit, to focus on being competitive in this hyper-competitive industry where they're all vying to get the next user, they all want to be the coolest product, so teenagers flock to them. If they have the teen market, you know, they're deemed to be cool, And that means that teens are going to use them and want to keep using them.

12:52Olivia, the intersection of social media, teens and mental health has been a major reporting focus of yours, as you said. Since you started covering the subject, have there been any significant changes to how these platforms approach keeping young people safe on their platforms? I've been reporting on the dangers of the digital world for children and for teenagers for years now. And over those years, we have seen the conversation shift. It has shifted from could social media harm the mental health of young people to this is a full-blown public health crisis. Our children are in need of help and we have to block, change, regulate these companies.

13:33Over the course of that time, the platforms have made changes to try and make their products safer, to add guardrails. But as we've seen in documents that have come out through litigation, a lot of the safeguards that they're adopting just aren't very effective. And oftentimes they will publicly tout like, hey, there's this cool new feature and look how it's going to protect teens from X, Y and Z. And then you ask the question of how many people are using it. And it's less than 1%. What we've seen from the industry is repeated claims of making the product safer when the reality is it hasn't actually been protecting young people.

14:13But that all changed on August 26th.

14:22After the break, what changed on August 26th? And how the outcome of a landmark case could change the social media platforms we use every day?

14:39Some people treat ChatGPT like some kind of smart search engine, and some use it to get work done. ChatGPT Work is a new way of working in ChatGPT that can take action across your apps and files, stay with a project for hours if needed, and turn a goal into finished work. It's designed to help you move from a chaotic starting point to a reviewable first version. So all the source materials, briefs, and scattered information that you have to grind through to turn into something useful can just become something useful. Put ChatGPT to work on your most ambitious ideas and projects. Get started at ChatGPT.com by selecting Work Mode, available on Plus and Pro plans.

15:21Let's talk about healthcare for a second. It doesn't always work the way people expect it to. If you've ever waited on a prescription refill or had a hard time getting the care you needed, you know the feeling. The system should just work better for everyone. That's exactly what the people at Optum are trying to do every day. They're a health care company linking patient care and pharmacy services and using data and technology to drive the whole system so care is connected, not complicated, for patients and providers. Things like making it easier to get care that looks at the whole person, from primary care doctors to mental health support and even in-home care, and then using technology to make sure they all work together.

15:59Technology designed to help doctors spend less time on busy work and more time with their patients. And those prescriptions? Optum is working to bring costs down, save patients money, and make it easier to get refills. Little by little, Optum is helping make healthcare work as one for everyone. Head to business.optum.com to see how.

16:47Whether you're sending money to family overseas, spending while on your holiday abroad, or paying bills across borders, Wise makes moving money simple, transparent, and straightforward. Wise offers 24-7 customer service and runs over 7 million daily checks to spot and stop fraud. And most transfers happen in under 20 seconds, which means your money arrives in less time than you've been listening to me. Join millions, saving billions. Be smart. Get Wise. Visit wise.com or download the Wise app today. T's and C's apply.

17:47pay up to$17.1 billion and made several changes to its platforms in the U.S., including safeguards aimed at protecting teen users. Things like blocking children from being able to access Instagram and Facebook from the hours of midnight to 6 a.m., putting a two-hour limit on teenage scrolling on Facebook and Instagram so kids can only scroll the platforms for two hours a day, increasing parental control safeguards. And what we saw from Meta on August 26th was a call out to the rest of the industry saying, join us in protecting teenagers, directly asking YouTube and TikTok to make those same product changes to ensure teenagers are safe because it's not going to work unless they all do it.

18:32And so part of the settlement is actually $5 billion is contingent on other platforms joining Meta and rolling out those same safeguards. Meta is paying$12 billion. It is paying an additional$5 billion only if those companies follow suit and introduce the same framework. So this is Meta's condition? That's right. That's right. They'll say, we'll give you an additional$5 billion if you can ensure that those other companies roll out the same framework. Because they're afraid of their market share. Right. If we lock out teens for six hours in the night, they're just going to shift to Snapchat, which doesn't help anybody.

19:07So let's make every social media platform play in the same rules, play the same game. So Olivia, I want to pause on this for a second because this is extremely significant in this space. Meta has to pay this huge financial penalty and it's rolling out these platform-wide changes that lawmakers and victims have been asking for for years. What do you think the effect of those kinds of changes is actually going to be? And do you think other social media platforms will follow Meta, will heed that call to action? $17 billion sounds like a massive number. And it is, don't get me wrong. But Meta is one of the wealthiest companies in the world.

19:51The last quarter, its revenue was$60 billion. So I think a lot of people have pointed out that really the financial penalty probably isn't enough to make the company do anything different. But really, the change lies in those product updates. That's the significant thing here. That's where the accountability is. And I do think we are going to see other platforms roll out additional safeguards to protect teenagers. And I think the conversation is shifting and the winds of change are coming. Meta is now rolling out changes that these families have been calling for for years. This August 26 settlement that Meta made comes as thousands of personal injury lawsuits have been filed against social media companies.

20:374 ,000 over the past four years. As more of these cases get litigated, what new information is coming out about how these platforms work? And do you think that this Meta settlement will have a bearing on how these other cases play out? We have seen over the past five years, thousands of lawsuits filed across the US, not only personal injury cases, and those are cases that are filed by individuals or families of kids who have tragically died, like Chase Nasker's parents. There are more than 1 ,300 school districts across America that have filed lawsuits against these companies alleging mental health harm and that have drained school resources.

21:17Dozens of state attorneys general have filed the same lawsuits against those companies. So this is a massive, consolidated litigation, nationwide litigation that has been compared to big tobacco, compared to the opioid litigation, some of the most significant mass-taught litigations in American history. We still have thousands of other lawsuits to go. The story is not over. Really, it's just begun. In response to your latest reporting on TikTok, on this confidential document, on the A-B testing that was done, senators sent a letter to TikTok leadership demanding answers to some of their questions.

21:57I'm wondering if you could just talk about the lawmakers' response to your reporting about the existence of this experiment and what they want from TikTok. Senators Marsha Blackburn and Richard Blumenthal have been advocates of a safer digital world for kids for years. They are co-authors of the Kids Online Safety Act, which passed in the Senate in 2024, but died in the House that year. And after the story came out, they wrote a letter to TikTok CEO Xiao Chu and TikTok CEO of its US spinoff, essentially asking the company direct questions about this experiment. How many other kids have been experimented on like this?

22:40How many other users were impacted? What other algorithmic experiments has TikTok conducted over the years? Asking specifically, who knew about this experiment? What are the names and titles of the employees? Did leadership know about this experiment? When did leadership become aware of this experiment? And they also asked for an unredacted document. They wanted to see the confidential document that this story was based on to understand more about what had happened. Those 13 questions were sent to the company shortly after the story was published and they gave TikTok a deadline of September 1 to respond.

23:18In that confidential document, TikTok also included its own recommendations to, quote, prevent future incidents. What were those recommendations and do you have any knowledge of whether or not the company has implemented any of those recommendations? So what we know from the information that was contained in the confidential document is that the company decided after that review was conducted and they realized what had happened to Chase Naska's account, they decided to reduce the size of the control group from 10 % of American users down to 1%. 1 % is still 1.5 million people, but it's a hell of a lot better than 15 million.

24:01They also decided to reduce the length of time those users would be exposed to filter bubbles. So they went from half a year to a year that users were being exposed in the control group to seven days. And that's significant. I want to end by talking about enforcement. Is there energy around broader federal intervention to change the way these kinds of platforms operate and the kinds of guardrails that exist to protect teens on these platforms? In order to get federal laws, we need legislators to work together. I mean, we're just in a deadlock. The Senate has their own version of a child safety bill called the Kids Online Safety Act that they are championing and that they want to get passed.

24:48The House has its own child safety bill called the Kids Internet Digital Safety Act, the Kids Act, that it wants to pass. And they're in disagreement over which bill should actually land on Trump's desk. So until we get some kind of agreement within lawmakers, we're never going to see the laws change. We're never going to see any kind of federal regulation. And I think that's why so many child safety advocates and so many families turn to the courts for accountability. And that is what we have recently seen. It's all still in flux right now. It was only just recently announced. But there is going to be an independent auditor who has to ensure that the platforms are rolling out the safeguards that they're claiming that they've rolled out.

25:27Who knows if we'll see TikTok, YouTube, Snapchat and other companies join Meta in making these changes.

25:38As for the wrongful death lawsuit that Chase Nazca's parents filed against TikTok, it was dismissed by a judge last year. after TikTok argued it was protected from liability under Section 230. An appeal of the case is pending. Personally, as a journalist, as a young mum, I hope that we make the internet a safer place for kids because it has just been the Wild West up until now. We have run a giant experiment, a psychological experiment, on an entire generation of children or a for-profit industry. And I don't use that word experiment lightly. That's how TikTok described what it did. It was an A-B experiment.

26:18We are literally experimenting on the minds of children without understanding the long-term impacts and consequences of how that is going to play out. Like, I think five years from now, if we look back and think, do we really let 11-, 12-, 13-year-olds sit in their rooms at night with a phone? You know, you give your kids this device and you give them access to everything, the entire world, but you give the world access to them as well.