AI Companies Are Opting You In By Default

0:00We all know home-cooked meals are so much better for you, but you don't always have time to pull it off. But with HelloFresh handling all the meal planning, shopping, and most of the prep, it's easier than ever to get dinner on the table quick and painless. Plans change and life gets busy. That's why HelloFresh lets you easily customize your delivery from week to week. Tailor everything to your schedule by adjusting your delivery date or just skip a week when you're not able to cook at home. Between being sick, travel, and generally just running around, I've saved time and energy by using HelloFresh's quick, easy, and delicious recipes.

0:36A recent Cavatappi beef ragu with parmesan and parsley was a hit with me and my family, and I was able to put together the home-cooked meal in just about 20 minutes. For free breakfast for life, go to hellofresh.com slash free404media. One free breakfast item per box while subscription is active. That's free breakfast for life just by going to hellofresh.com slash free 404media. HelloFresh, America's number one meal kit.

1:12Hello and welcome to the 404media podcast where we bring you unparalleled access to hidden worlds both online and iara. 404media is a journalist founded company and needs your support. to subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co. I'm your host, Joseph. And with me, it's a miracle, are all of the 404 Media co-founders, the first being Sam Cole. Hello. Emmanuel Mayberg. Present.

1:54and Jason Kepler. Dude, I can't believe it. When is the last time we were all on a pod together? It's been like a month at least. It's been at least a month, easily. Pre-anniversary. Yeah. And speaking of anniversary, we are going to record our one-year anniversary podcast special for paying subscribers only. We will get to that. It's just a lot of stuff came up, but we always appreciate your continued support, and we'll get to that soon. Breaking down what it's been like running a new independent tech investigation site for the past year, what we've learned, what's coming. I'm saying all this, but we haven't actually planned the podcast yet.

2:39Maybe it's going to be completely different, but I'm sure it'll be good. We'll also talk about why it's so late. We were snakebitten health-wise. Not literally, but nearly, yeah. I was bitten by a snake. Oh, there you go. um all right but for this episode we've got like a few stories about opting in opting out the lack of the opt-out i don't know a lot of opting in various different directions and the first being one written by jason massive e-learning platform udemy gave teachers a gen ai opt-out window it's already over. Very interesting words, opt-out window. Haven't really heard of that before.

3:23I guess, first of all, Jason, just what is Udemy for those who don't know? Udemy is this online class platform. Basically, they call themselves an e-learning platform, but essentially anyone can become an instructor there and upload classes, and then you pay for the classes more or less. And there's like a rev split. It's one of the biggest in the world. Like they have something like 70 ,000 instructors and they have like hundreds of thousands of classes. I think it's very popular for its like coding stuff, but there's also classes on like anything that you can possibly think of. There's a lot of like web design classes.

4:09There's a lot of like how to do digital art classes, how to use Photoshop, so on and so forth. and I don't know if it's the biggest in the world but it's definitely like among the biggest it's worth more than a billion dollars it has like hundreds of employees and it's a it's a big deal in this space yeah I've definitely seen it come up uh especially sort of in infosec and cyber security that sort of thing so there's this big platform all of these crucially human teachers and human instructors are making their own course content and teaching people online. Well, Udemy has this plan around generative AI.

4:49What were those plans exactly, or as much as we know about them anyway? Yeah, so back in July, they, like seemingly every other company, announced that they were going to have a generative AI program. It's kind of unclear. Udemy has been very transparent about this and yet very vague at the same time. I feel like a lot of companies have just launched generative AI stuff and have opted people into it. And there's not been a big lead up to it. Whereas Udemy had a webinar for instructors where they could learn about this. They also published an ethics statement, more or less. They published a frequently asked questions.

5:33They did publish a lot of stuff, but almost all of this happened on its own community platform. But basically, they told instructors that they are going to train an AI model on the instructor's classes. It is not clear what they're going to do with this AI model. They say that they're going to offer personalization to users, but they have said that they do not intend to offer AI-generated classes, as in Udemy does not intend to create new classes that are entirely generated by AI. But they also reserve the right to do that. Kind of meaningless then. We're not doing it right now. So, I mean, I think that like most of the instructors are very worried that they're going to train on their classes and then they're going to launch AI generated classes that compete with the human taught classes, more or less.

6:35Right. Yeah, that would be that's the big fear. Yeah, it would be very, very concerning. So, well, you said they made all these announcements, but it sounds like it was more, as you say, sort of on the community platform or Udemy's forum, something like that. Maybe not everybody is seeing those. Were people opted in by default? And then we'll get to the window part. But was it opt-in by default? It was opt-in by default, as will probably be the theme of this episode. But like many other platforms, they say, okay, we're putting you in this program. Here's how it works. They also crucially said, if you opt out of this program, you will not benefit from whatever AI things that they end up launching.

7:26And it's still unclear what this is going to be or how people are going to benefit. But they make it very clear that people... I read it as a veiled threat. I basically read it as if you do not opt in or if you opt out, people have a hard time finding your classes. That is more or less what they told people. They really heavily incentivize people to not opt out by saying, you're probably going to make less money from our platform if you opt out of this. Right. Which is obviously very, very concerning to people who, even if they're not making their entire living off it, which they might be, they're at least generating income from this.

8:07And the platform is just potentially undermining that or at least some sort of veiled threat, as you said. So people, I won't say opted in, I'll say they're enrolled in the program by default. Udemy then gives this opt-out window. Like, what exactly is that? I mean, it's exactly what it sounds like. So like on August 21st, they said, okay, the opt-out window is open. Meaning if you don't want to be part of this program, go into your settings and click opt-out. So that opened August 21st. And then on September 12th, that window closed. And basically, if you did not opt out by September 12th, which was three weeks ago, you cannot opt out until sometime next year.

9:01and even though Udemy has done a lot of posts about this like on its forum people that I spoke to said that they didn't get emails about this opt-out window that it was kind of like well if you were very active on the platform and were like logging in you probably would have seen something but a lot of people have created classes and then they leave it up there and people can take the classes at any point. So it's not like, it's not like, it's not something where you meet at 7pm every Tuesday. It's like, they're self-guided classes. And so people make the classes, they put them on this platform, and then they leave them there and they passively collect income from it.

9:40And if you're not like, part of the Udemy community, if you're not actively keeping up, it was very easy to miss out on this. And so basically, it's like, once that date passed, you had a grayed out option like it was totally gray you could not change the option you can't opt out and I also saw an email where someone was mad about this like an instructor was mad about it and said hey I want to opt out now and they said well you missed the opt-out window you cannot you cannot do it I have never heard of this and maybe that's because it is specifically about generative AI and AI models and that sort of thing.

10:24I mean, that's what we're talking about here. I'm just wondering if like, maybe this is a new thing we have to deal with. But I think it's absolutely wild as a window at all. But Udemy did give you some sort of actual explanation for why this is the case, albeit incredibly unusual. Like, what was the reason they gave for, we're only going to do opt-outs at certain periods. Yeah. I mean, the only thing that this reminds me of is like healthcare enrollment, like open enrollment, where you have a two-week period or three-week period where you can change healthcare plans. And if you miss that, you kind of like have to forever hold your peace till the next opt-out or open enrollment period.

11:06And that's kind of the model that this looks like. um so udemy says that basically they is too time consuming and expensive for them to be retraining their model constantly and i find this to be very interesting one it's like it's kind of absurd because they're taking these professors these instructors classes and training it on them and you should be able to decide at any point, like, hey, I don't want my stuff in there. But what they're basically saying is like, it is really hard for us to go in and delete this data. And it's easier for us to do this all at once, one time per year, than it is for us to, every time someone says, hey, stop training on my stuff, to go in and delete that.

11:58And I mean, they've taken some flack for this since I published the article. And I think that's right but at the same time this raises a lot of questions for me like about other companies because a lot of other companies say you can opt out of ai training but what does that mean like are they going in and are they deleting things that they have already trained on and if so how often are they doing that or they're just not doing it like but udemy is or something right Yeah, exactly. And in this case, Udemy says that they are going to go out and delete the data that it's already been trained on once per year.

12:41But I would imagine that a lot of other companies are saying, once we've scraped your data, you can opt out of future training where new content is being trained or scraped rather, but we are not going to go back and delete what we already have. And, you know, there's, there's like a couple, like, I'm not aware of any company that really promises to delete data. The closest thing that I can think of is Leon 5B that Sam reported on where there was a lot of child sexual abuse material in this massive large language model. And once Sam reported that, they took the model down and they like, quote unquote, cleaned it and they retrained the model and it took like nine months or something like that right right tim like what was the specifics there yeah it took them a long time just because it was you know it's not a simple thing to do to just say i'm gonna delete the the abusive material um but yeah you're right that's that's the only something i can think of and then there's like a couple um academic papers that i looked at and i think a manual has maybe written about a couple of these or at least looked at them before.

13:53But I found it very interesting that there's a few academic papers where researchers are trying to do large language model unlearning, where basically they train the model to not output stuff that is like undesirable, more or less. And this is usually talked about in the context of like racism, sexism, like, you know, terms of service violations, things like that. But unlearning is different than deletion and different than retraining altogether. As I understand it, unlearning is where you teach the language model to not output this stuff, but that doesn't mean that it's not still in there. And from what I can understand, unlearning is easier, less expensive and less time consuming to do than fully retraining a model from scratch.

14:50But I don't know for sure. Like, I think this is one of the big problems with generative AI, where it's pretty hard to get stuff out of there once it's already in. The difference is that with unlearning, which the reason people are researching this, I think, and the examples that they often show in research is for copyright reasons, right? So if you're using stable diffusion, and you wanted to not produce an image of Harry Potter or something, it would be great for whoever makes the model if you could cheaply unlearn Harry Potter from the model. But the reason it's easy is that you're not going back and changing the data that you put into the model.

15:30The model is already trained, and in the same way that we've reported many times that you can train a model to do specific things, you're essentially trying to train it to not do something. but from what I've seen and I looked at a bunch of these papers like none of the methods are perfect and people always find ways to pull that stuff out of it because it's in the data set and in order to really have it not be in there it has to be removed from the data set which as we know because of the Lion 5b example it's much harder especially with a large data set like Lion 5b to go back to it and remove specific things and then retrain the model like with Lion 5b it took them months before like they took the model down and then it took i don't know six months to put it back up um a much more expensive time-consuming process did they did they ever detail what that process actually involved because they were removing c-sand material right did was it ever said publicly like we went in and found all the c-sand one by one and removed it like was that ever disclosed or they did they wrote a really long detailed like company blog about it um it was tricky because if you just removed the offending stuff um you would end up with a roadmap to the offending stuff in the the data sets people already have so it's like you could just kind of match the difference right um so it's a much more complicated process than just like deleting and republishing.

17:08But yeah, it's, you know, it's like 5 ,000 words on their site. It's a complicated process involving a lot of math that I'm not even going to attempt to summarize. I think it's also tricky because they had to like, because it's an open source thing, they have to teach other people how to remove it because the data set has been copied, right? So very complicated, very, very difficult. Once you're scraped and you're in a large data set, very hard to remove you. Yeah. So I mean, this problem wouldn't necessarily exist with Udemy, which is training, presumably a closed source model that it controls.

17:47But that said, we don't know that much about how often OpenAI trains new models. Obviously, they have GPT-4 and GPT-4-0, etc. And those are big big steps forward, like in between these individual releases. But I don't really know, like how often OpenAI, for example, is updating these large language models. Like I don't know if it only does it when it has a big new release, or if it's making like incremental changes much more often and what those look like, and if things are being removed from it. And I think that that's like one of the big questions about large language models in general is how often are they updated?

18:39Like we don't, we don't really know. Um, or at least I don't, um, maybe there's like technical papers and things that do talk to this more often, but it's not something that is common knowledge. I don't think. So when it comes to this idea of an opt out window where you have the three weeks or whatever, or at some point in the year or whatever it is. Do you think we might see more of that just as more companies are developing generative AI in general? I think we need to see how this goes because after I wrote this article, a bunch of people raised the question about whether this is compliant with the right to be forgotten in the European Union, for example, where you are basically allowed to opt out of or ask a company to delete information about you at any point.

19:33And whether this is compliant with that law, I have no idea. And Udemy does operate internationally. And so I think it's something we could see more of, but people seem pretty upset about this. and understandably so in my opinion. So I guess we'll see kind of like if Udemy sticks down this road or if it changes its policy, says, you know, like, oh, we heard you, we'll train more often, something like that. But yeah, I really don't know because I simply don't know how, whether Udemy is just saying, hey, we only want to do this once a year because it's annoying for us or whether it is actually very expensive, actually very time-consuming.

20:23It's not super clear to me right now. Yeah, I mean, you bring up the right to be forgotten. And when I read the article, I was just like, how is this possibly legal to have an opt-out window? And I know that's me as a European with that perspective. But yeah, it really, really stood out to me. We'll leave that there. But when we come back, we're going to run through a couple of other stories also about opting in or opting out and sharing data without knowledge and all of that sort of thing from PayPal and LinkedIn as well. And that's not even all of the stories we did about this. We're just going to do those.

21:02We'll be right back after this.

21:14Lumen is the world's first handheld metabolic coach. It's a device that measures your metabolism through your breath. And on the app, it lets you know if you're burning fat or carbs and gives you tailored guidance to improve your nutrition, workouts, sleep, and even stress management. I've got a Lumen and first of all, it's really easy to set up and use. Every morning, I breathe into it to learn whether my body is mostly burning fats or carbs. With that data, Lumen gives me a personalized nutrition plan based on how my body is working that day. I also use it before a workout to learn if I need to fuel up and before or after exercise or eating to see how my body is working.

21:52Lumen has helped me understand my metabolism, showing me how my body turns the food I eat into fuel. and it's helping me work toward increasing my energy levels, feeling better during a workout, and even sleeping better. So if you want to take the next step in improving your health, go to lumen.me slash 404media to get 15 % off your Lumen. That's L-U-M-E-N dot M-E slash 404media to get 15 % off your purchase. Thanks, Lumen, for sponsoring this episode. When we launched 404 Media, I knew we wanted to sell merch. But I had no idea how to set up a store. Then I found Shopify and now you can buy those beautiful green hats that we made.

22:36Setting up and managing our store was so easy because we use Shopify. Shopify is the global commerce platform that helps you sell at every stage of your business. From the launch your online shop stage to the first real life store stage. All the way to the did we just hit a million orders stage? Shopify is there to help you grow. Whether you're delivering daily digests or serving sensational scoops, Shopify helps you sell everywhere. From their all-in-one e-commerce platform to their in-person POS system, wherever and whatever you're selling, Shopify's got you covered. Shopify helps you turn browsers into buyers with the internet's best converting checkout, up to 36 % better compared to the other leading commerce platforms.

23:20And you can sell more with less thanks to Shopify Magic, your AI-powered all-star. Shopify integrates with every other software we use, meaning I can seamlessly print shipping labels, track orders, and manage our entire 404 media store from just one platform. Shopify powers 10 % of all e-commerce in the US. And Shopify is the global force behind Allbirds, Rothy's, Brooklinen, and millions of other entrepreneurs of every size across 175 countries. Check out shopify.com slash media to grow your business no matter what stage you're in. That's shopify.com slash media. This show is sponsored by BetterHelp.

23:59I get jump scared easily, but Halloween lets us have fun with what scares us. So I check out haunted houses anyway. But what about those fears that don't involve zombies or ghosts? Therapy is a great tool for facing your fears and finding ways to overcome them. because sometimes the scariest thing is not facing our fears in the first place and holding ourselves back. I've been using BetterHelp for the last month to talk through anxiety I feel before public speaking engagements. This has helped me develop strategies that make me feel confident before doing something stressful. No matter what you're dealing with, therapy can empower you to be the best version of yourself.

24:36If you're thinking of starting therapy, give BetterHelp a try. It's online and really flexible. I've had a hectic few weeks and been able to get appointments whenever I've needed. Just fill out a brief questionnaire to get matched with a licensed therapist and switch therapists anytime for no additional charge. Overcome your fears with BetterHelp. Visit betterhelp.com slash 404media today to get 10 % off your first month. That's BetterHelp, H-E-L-P dot com slash 404media.

25:14All right, and we are back. Sam wrote this one. PayPal opted you into sharing data without your knowledge. So Sam, what was the change here exactly, and how did you find it? Yeah, so PayPal introduced this feature slash setting called personalized shopping, which I had never heard of until I saw someone flagging it on Twitter. And it's kind of deep inside of your settings on PayPal, where it says that it will let PayPal share products, offers and rewards you might like with participating stores. And you can toggle that feature on and off. So it's basically toggling the permission to share. You know, it says it's sharing products and offers, but really what they're sharing is your personal data.

26:12And they launched this without any kind of fanfare. I don't have any communications from PayPal about it. You know, people online were saying they hadn't heard of it before. So it's a little bit sneaky on their part, in my opinion. Yeah. And yeah, it's weirdly vague. um while also trying to be transparent with having the little setting and saying here's what we're doing it's not actually really telling what it's doing but basically sharing data about your purchases potentially with other retailers and as a company that sits in a very very privileged position as a basically a payment processor that's pretty concerning and pretty weird to be honest um we verified it was it was already opted in by default right like how did you do that did you check your account or i checked my account um i also made a new account and checked it um you know it's it was definitely already turned on lots of people were talking about and posting screenshots on twitter about how it was turned on for them so it's just it's very strange to me and i mean i guess it shouldn't be that strange considering the world that we're living in but and all the things that we just talked about.

27:25But it's strange to me to have it opted in for you, to have that feature at all, to have a little toggle and then say, we're giving you the option, but we're not going to tell you you have the option and we're going to opt you in automatically. Something else that was interesting that was happening with this was since we were just talking about Windows, up in Windows, there was a line when I first checked my settings that said, what did it say? It was like, we'll use the info collected about you after November 27th, 2024. And then I was like, well, that's a weird thing to say. We'll use it after November 27th.

28:06We're going to collect the data after the 27th. We're only going to use it after that point. It's like, what does that mean? So I emailed PayPal and I was like, hey, what does this mean? If you opt out after November 27th, what happens to your data? Obviously, they're not going to go in and remove it you know they have a clause in their policy that says we can't control what these other merchants do with that data um and they didn't reply and that was 24 more than 24 hours ago um but they did remove that line so you know it's like what was the purpose of that line exactly um i love it when companies do this oh i love it yeah it's so funny when you reach out for comment and they never get back to you but they change the exact thing you were asking about yeah they're like you know that they're forwarding this they've like created a bunch of tickets they're like get this line off the page um but and then i mean that that was weird and then the privacy statement link in that page in that settings page is a different url than the privacy setting or the privacy statement that you can get to from anywhere else on paypal and that url said um like the slug was like preview November 27th, 2024 or something.

29:20It's like, why? What is going on here? So they like publish their drafts or something? Yeah, I guess. It's so strange. But that link is still what you see if you're on that settings page. So anyway, mysteries abound. PayPal is still not replied. So I don't know. People were pissed off. It's something that no one likes being chosen for. you should at least give people the heads up or the option if you're going to give them the choice and a lot of people in the eu and other countries were saying that they don't have the setting at all so right as in uh it's not that they can't opt out it's just like this isn't even a thing no yeah exactly um and i mean just to briefly um bounce off something you said about i can't remember exactly but like is this actually weird or not that's um that's what i remember but like i what is going on with all of these companies we've covered like three or four doing really really weird automatically opt-in stuff and it's just like did everybody decide to do it in the past two weeks i don't know yeah people were really pissed off about linkedin which we can get into but yeah yeah no like for good reason people were mad about this do you want to kind to go into the LinkedIn story because that was that was a hot one yeah so this was actually a little while ago I think before we recorded the last episode it was almost like a a little fairway story but bringing in here because as I said all of these companies doing this but basically the long and short of it is that LinkedIn was using its users data for improving the site's generative AI, but we found it hadn't actually updated its terms of service yet, like its privacy policy, that sort of thing.

31:12This was again found by various users of the social network, and they were posting screenshots of the settings and where you could turn it off and all of that sort of thing. I just reported up with some context and got a statement from LinkedIn and that sort of thing. Before I get into that, a LinkedIn AI, I don't want anything to do with that just like gen gen ai built on like think fluencer lists and like b2b marketing strategy i think people talk about ai being like a danger to humanity i think that's the one we need to be like scared off the linkedin uh ai you don't know what's going to happen with that but the quote uh from linkedin on its site was well this is like the dialogue can linkedin and its affiliates use your personal data and content you created on LinkedIn to train generative AI models that create content, it asked.

32:09And then there was another line that said, use my data for training content creation AI models. And you could flick it off and on. Crucially, it was enabled by default, as I'm sure we've probably established by now. I, as I said, I reached out to LinkedIn to ask what's going on here? You know, why are people opted in by default? And why have you not updated your privacy policy or terms of service to reflect this at all? And they basically gave me the corporate equivalent of, I'm sorry, I'm sorry, I'm trying to delete it, which is they said, we're doing it shortly. And I don't know, maybe you should do that first before you start training on uh people's data i don't know um similar to the paypal stuff they say they're not doing it on users in europe generally either so there's obviously um something going on there but yeah i don't know what is going on with these companies but just a really serious a series of really really strange frankly some of them shitty decisions on automatically enrolling their users into various practices and i don't know i guess we're probably going to see more of this as well as more companies get into um generative ai and all of that as well one thing that uh i have not told y 'all or maybe i did long ago but back when we were at motherboard i uh like linkedin's kept adding all these AI features into LinkedIn where it would write your posts for you, etc.