Luxury Surveillance (With Chris Gilliard)

0:03Hello, and welcome to the 404 Media Podcast, where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist-founded company and needs your support. To subscribe, go to 404media.co. You'll get bonus content every week, and subscribers also get access to additional episodes where we respond to their best comments. And they also get early access to our interview series, which you are listening to right now. Gain access to that content at 404media.co. This week, I'm joined by Chris Gilliard, a surveillance justice expert and Just Tech fellow at the MacArthur Foundation.

0:39I've talked to Chris for so many stories over the years. He's one of the smartest researchers studying surveillance, privacy, and how they affect us and the people around us. His book, Luxury Surveillance, is coming out early next year. Today, we're talking about luxury surveillance or how big tech companies get us to opt ourselves and the people around us into surveillance. Here's my conversation with Chris.

1:10Chris, how are you doing? I'm doing great. Thanks for having me. Yeah, no, thank you so much for coming on. When we started this interview series all of like one week ago, I knew I wanted to have you on really early because I feel like our interests and careers and paths have intertwined a lot. I feel like often whenever I'm writing about some dystopian, horrible thing, You're one of the first people that I want to call. And not because I want to make you sad, but because I feel like you always have very good context for why a specific technology is working in a specific way or how surveillance is affecting people and spreading to the masses.

1:58And so one, thank you so much for being that person that I can call. I feel like you always have the right perspective and a very smart perspective. But two, I feel like this conversation will probably go all over the place for that reason, because we've talked about so much over the years. I guess I want to start with luxury surveillance because I believe this is what you're most known for in the public sphere. is spreading awareness of this idea of luxury surveillance, which is, well, I'll let you define it, but basically it's people who opt into surveillance because it's easy and makes their lives better, or so they think.

2:42And then that's distributed across the population in all sorts of nefarious ways. But how do you think about luxury surveillance? I know this is a project that you've been working on for quite some time. There's a book coming out, etc. But what is luxury surveillance for people who don't know? No, that's exactly it. I think that definition holds up. I started thinking about it mostly because most of the models that I saw for surveillance talked about, and I think this is important and it's true and it matters, but some of the models talked about surveillance that was deployed against people, like involuntarily, you know, often against kind of the most marginalized or most vulnerable populations.

3:31And that's true. But as I was writing about surveillance and talking to people and things like that, like a couple of things kept coming up. And one is that people would always, they would regurgitate some version of the nothing to hide argument, you know, or they would kind of privilege their own needs or desires in front of everything else. You know, so it would say if I talked about video doorbells, I'd say, well, I still need to get my packages and things like that. And so I started trying to think of kind of how to leverage that in terms of in terms that I thought would help people get a better grasp of sort of what some of the problems were with that.

4:13I mean, as you know, like I wish it moved the needle to just tell people that, oh, this is going to be used to deport your neighbors, you know. But unfortunately, sometimes that doesn't move the needle. And that people, I think if they understand the degree to which some of these systems come for everyone, sometimes kind of helps a little bit. And so the initial comparison I came up with was talking about Fitbits and Apple Watches as analogous to ankle monitors. and that's where that came from. Right. So I want to tell you an experience that I had recently and you can tell me if this is luxury surveillance.

4:56I feel like maybe it helps frame the conversation a little bit. Are you familiar with the Intuit Dome in Los Angeles? No, who plays there? So the Clippers play at the Intuit Dome and it's a new several billion dollar arena that opened last year. And a couple of friends and I wanted to go to a basketball game, watch the Clippers. So I bought tickets on StubHub. And to get the tickets, I had to download the Intuit Dome app. There was no way of transferring the tickets through Ticketmaster, which already have tons of problems with Ticketmaster. But such is life if you want to go to a concert or a sports game or whatever.

5:36So I had to download that. I had to give them tons of information just to get the tickets. and not only that everyone in my party also had to download the app so i couldn't get all four tickets uh you know i had to get one and then someone else my team on in my group had to get one and they have this uh this entry system that you you scan your face in the app and then you enter with facial recognition and that face is connected to your wallet and so and then the entire interior of the arena, they have tried to get rid of humans at every possible angle. So it's like they have stores in there, but they're just walkout stores where you just grab something and you leave.

6:26You have to scan into these areas with your face. And they claim that you can use your credit card without this, but those are always broken. And then of course, once you're in this area, You don't interact with any humans, basically at any level. And if you want to opt out of this system, they make you walk around the entire arena to go to one single gate where you can use the old barcode system. And so the first time I went, that's what I did because I was like, I don't want to do this. Second time, we were running late and it was a playoff game. And so I was just like, you know what? Like, fuck it.

7:07I'm going to do it. I'm going to do it. And I did it. I had a few beers, so I opted in. And I feel like that is my life in many ways, where it's like you can often try to resist these things, but it's often very inconvenient. They make things worse. But also opting into these systems in the first place, like download the app, give them all your information, scan your face, that was super inconvenient as well. And I don't have a question here, But to me, it's just like, I thought of you immediately. I thought I was like, this is luxury surveillance, or at least it's trying to give this like seamless experience where you give them your face, you just grab a beer, grab chicken tenders.

7:52You don't interact with any human beings. And like all the data goes like God knows where, like who knows what happens to it. And I don't know, I thought of you and I'm curious what you think of that experience. Yeah, I mean, it absolutely is. You know, I mean, it kind of engages another thing that I've talked about in the past, which is I call friction free racism. Right. That the the first of all, like who has the capacity to do this? Like who's got the phone, the credit card, like like all these ways that but also who fantasizes about this experience where you don't interact with anyone else?

8:30you know, I mean, I've worked in, I've worked in the, at stadiums and at the vendors and things like that. So I don't fantasize about the experience where I go get a beer and a popcorn and don't talk to the person behind the counter, you know, but I know that people do. And it is, I think a mark of privilege. But it also, you know, there's so many ways that it ties into so many other things, which is like going to a game is supposed to be a communal experience. The people who work there are part of that community. I mean, or they should be like, I think of them as that. And so to say like to pitch as a privilege or a bonus or benefit that you can kind of slide through the experience and all you have to do is pay for it with your face, you know, I think is, yeah, I think it's a prime example of that.

9:28And the other part is that you are incentivized to believe that it's to your benefit and that the only way to opt out, you know, I mean, we've seen it in the airport, for instance, is to undergo a series of sort of onerous steps in order to not do this thing, which is supposed to be good for you or, you know, be better for you. Yeah. Yeah. And I think that it's not a far jump from there, but I think a lot about the delivery robots that are driving around Los Angeles and college campuses and other places all over the country where, again, you've taken something that's already kind of abstracted, like you have traditional delivery, and then you have things like Uber REITs and Grubhub and the associated algorithmic management of those workers and the depersonalization of the whole situation where it's like, just drop it off.

10:29Don't talk to me. Don't look at me. Food arrives. And then you just literally get rid of the human or well, you put the human maybe in Nebraska or something with an Xbox controller to drive around the food and these little robots. But then also you attach cameras and microphones to the robots because these are like, this is the property of these delivery companies. And, you know, people don't like these robots, so they're attacking them all the time. And sort of the side effect of all of this is like, there are more cameras that can be subpoenaed by cops that often don't have to be subpoenaed by cops.

11:07They are sort of given footage willingly. And for this, it's like something where a rich, privileged person abstracts themselves from the entire food delivery experience, but then they're also opting in all of their neighbors to this type of surveillance. And I was wondering if you could talk a little bit about that because you talk a lot about that in your work where it's like you are opting other people into surveillance via your decisions. Yeah. I mean, frankly, if these decisions only hurt the rich people or the people who signed up for it, I kind of wouldn't care. But that's sort of the unspoken part, is that we're all sucked up into these systems when people engage in them, not only in the way that you mentioned, but it normalizes it.

12:09When these things are seen as luxury or aspirational or making your life easier, it normalizes it in a way I think that is really, yeah, unhelpful, like harmful, that when everyone has a video doorbell, when everyone is carrying around a microphone, when people are wearing glasses, you know, with cameras, you know, and we've seen it. I mean, we've seen it with ICE and CBP You know, we've seen it with, you know, states and municipalities requesting footage of protesters. You know, we've seen it with like Flock and automated license plate readers that these things that are promoted as some kind of benefit or, you know, safety for people wind up, you know, we've seen it in ed tech.

13:05That these systems, they wind up making people, lots of people less safe.

13:17I remember a past, a bad past, where whenever I needed glasses, I had to go to multiple places that all felt like stuffy doctor's offices. And I had to spend hundreds and hundreds of dollars on glasses I didn't really like and had to wait weeks and weeks to get them. Warby Parker changed all that. Warby Parker uses nothing but premium materials in each frame. They designed the frames in-house and their collection includes different silhouettes, colors, and fits to suit every face and every fashion style. I love that it's a one-stop shop. They offer everything you need for happier eyes. Eyeglasses, sunglasses, contact lenses, and eye exams.

13:56You can shop with them online and in stores. Warby Parker has over 300 retail locations across the US and Canada. And their glasses start at just$95 and include prescription lenses with anti-reflective scratch resistant coatings. Add a pair and save 15 % when you purchase two or more prescription pairs of glasses or sunglasses. This offer is available both online and in stores. Free shipping and free 30-day returns. I've been going to Warby Parker for years now, once I learned that they had figured out the glasses game. And I haven't really looked anywhere else since. And if you have a big head like mine, you'll love their virtual try-on system that lets you figure out which frames and what size you need.

14:39Gone are the days where you'll be disappointed with your glasses. Warby Parker has over 300 locations to help you find your next pair of glasses. You can also head over to warbyparker.com slash 404media right now to try on any pair virtually. That's warbyparker.com slash 404media. warbyparker.com slash 404media.

15:08Do you think that there's a way to design some of these systems in a way that they are safer? Or what is your perspective on... I mean, it's such a broad class of things. But what is your perspective on something like an Uber, a food delivery robot? And then I think we can probably get into Flock and Ring doorbells and things like that. But in Amazon, I want my stuff in one day. I'm going to be targeted with advertising that is going to hit me with really convenient things because they know so much about me and they know that I'm out of cereal or whatever. I guess, do you think that there is a way to design these...

15:53Design technology so that it is more respectful of users while also providing this convenience? or do we need to sacrifice some of that convenience for our own privacy and the privacy of others? Yeah, well, this is an interesting question for me, right? Like, I think some of these things shouldn't exist, you know? Like, it's a sort of you-don't-got-to-hand-it-to-them kind of question. I think that it's understated the degree to which some of these things are really corrosive and harmful to society. You know, the idea that I should be able to order non-essential things and a drone drops them off at my house an hour later.

16:35You know, things like that. I mean, there are use cases, you know, but mostly no. And so I think, so my kind of short answer is the only way to do that would be to divorce it from capitalism, you know? I mean, I think about the push towards smart glasses and there are some cases where people talk about them um you know for instance like translation and so would it be cool to have a thing that immediately like translate something that worked and wasn't wonky you know um but like so yeah like and i like i don't think those need to be glasses for instance like those could be headphones but i mean this is like a sort of like capitalism kind of ruins everything in that mostly companies don't, they're not incentivized in any way to produce these things at scale in a way that would protect people's privacy or wouldn't be invasive or wouldn't be corrosive.

17:47There are some decisions, I think, that make these things worse, you know, because like we, companies sort of try to maximize their intrusiveness. But a lot of these things, you know, like I don't really think should exist. I think that the idea that convenience kind of trumps all really has been. Yeah, I think it's a very harmful way to go about things. Well, it's this idea that everything should be frictionless. And if you don't submit to these systems, we are going to add a lot of friction, like an unnecessary amount of friction. Something you said made me think about my reporting with AI Slop, which is just like, we can talk about how AI has changed the calculator, supercharged a lot of these things that you're talking about and sort of the incentives of these AI companies.

18:45But I think a lot about how you'll talk to someone who's really excited about AI, generative AI, which is, I feel like, fewer and further between for me. But every once in a while, I'll run into someone who, a friend of a friend, who's like, yeah, I really love how I can storyboard with RunwayML or something. And it can help me write my scripts and things like that. And I think often of the best intentioned people using AI to make their little projects and things like that. I'm like, okay, that's cool. But the vast majority of what we see on the internet using these systems are highly corrosive, highly invasive, used to further inflame tensions.

19:32And they're just polluting the internet. And I think about that when you mentioned smart glasses and things like that as well, where it's like, yes, maybe they can be used for translation and there should be better accessibility tech. I also think about, I would love to use meta glasses to film myself cooking for myself. So I'd be like, oh, that's like a camera on my face for use in my own home. But that's not how people are using it. People are using it to go film viral TikToks where they're harassing people in public and things like that. And I guess it's just like, I personally feel I'm just like, we can't be trusted with this technology as a species, really.

20:14And the incentives of these big tech companies are not responsible use of these technologies. They're like, you know, use them as much, use them to replace human contact and use them to spy on your neighbors and use them to inflame tensions and keep yourself safe and that sort of thing. And I guess it does kind of go back to this, the incentives of capitalism, kind of, where it's always like maximalist, maximalist, maximalist. Do you agree with that or what? Yeah, I mean, and I hate it because it sounds reductive. But I think for a lot of these things, you have to ask why they exist. You know, why are companies pouring billions and in some cases trillions of dollars into these things?

21:05And also when, you know, sometimes even their own research says, like, this is the effect or this is how people are going to use it. Or now that people are using it, this is what it's doing. um even with those things like you know um when we think about like the chatbots um and how they've been linked to suicide particularly of young people and it's like like one of those should cause those the people at those companies to like you know go pull some plugs at the data centers, you know, like, but there's been far more than one, you know, that, um, and then, um, you know, companies will come out and say, well, we've addressed that.

21:52Or, you know, for instance, character AI, I think is now doing some kind of, um, or they, they are going to start putting in like, um, kind of age restrictions or something like that. Um, and, you know, like, it really seems insufficient, um, particularly given the harms, right? I mean, like, throughout, uh, even recent history, like, we've seen examples of when, um, children are harmed, when someone tried to do, you know, commit a terrorist act or anything like that, that there've been, um, pretty extreme reactions in terms of, like,

22:34reactions in terms of trying to like protect the public um but that seems like uh you know sort of like a bygone era now um but like because and like the thing is like when for something like sora you know i was just kind of rereading your art your piece on sora and the facebook slot and it's like why so you know like a really big important question is like why does sora exist you know like what is it for you know um you know a friend of our like mine and you know like actually how i met you was david columbia you know and when i think in 2022 um before he passed he wrote a piece called chat gpt should not exist um you know david was like always kind of like uh way out in front on stuff i always thought i mean so for people who don't know david columbia I was a researcher who wrote quite a lot about techno-libertarians and anarcho-capitalism and things like this.

23:38I think you could explain it better, but I remember him as always being far in front of the curve and things like this. Some of his early stuff on Bitcoin libertarianism, I feel like that was before I had even come close to connecting the dots there. If you want to talk a little bit about David and some of his work, I would really appreciate that because it's so sad that he passed and he did such great, great work. Yeah. So, you know, full disclosure, he's a really good friend of mine and a mentor. We wrote some pieces together. But he was really, I think one of the things he's known for is his work on Bitcoin and on cyber libertarianism.

24:20authoritarianism. Um, and, um, I, I'm not really going to do it like a great justice, but what I would, what I would point out is the connections he made between these technologies and, um, the far right and between some of these technologies and some of these technologists or, you know, venture capitalists and things like that. And this understanding or this belief that technology should sort of trump democracy, that innovation and the idea that software or algorithms or what have you, that companies and platforms and investors should have free reign in terms of putting technologies out into the world that are harmful because the most important thing is innovation.

25:14he called that cyber libertarianism you know like that um there's this still i think prevailing narrative i mean it's disappeared somewhat but that that silicon valley had long been sort of um liberal or left he does in his in his um book that was published posthumously this is a really deep dive into talking about why that really was never true and yeah i think that in terms of um the essay on on a chat gpt i think one of the the things he located really early on um which i you know i agreed with them but like we're seeing really come true is the way that it's like a very it's like um that chatbots um writ large are like an anti-democratic force.

26:02Um, and also the glee with which, um, the people, you know, kind of chat bot fetishists and gen AI fetishists, like the glee with which they think about putting artists and creatives, um, and professors, you know, um, out of work is tied to their, um, distaste, um, and animus towards democracy. And so, um, I think about him a lot whenever some of this stuff comes up because, um, when that piece came out, you know, he received a significant amount of pushback. Um, you know, um, I think it holds up really well. I mean, it was only three years ago, but most of it turns out to have been pretty accurate.

26:50Yeah. I mean, but a lot, a lot has changed and a lot hasn't since then, I guess. And this is not a surprise to anyone, but it's like, I have a lot of problems with AI and how it's used and how it's affecting people. And I feel like a lot of readers will email me and sort of say like, well, there's this and you don't talk as much about the environmental impacts as you should. That's like one line of criticism, which we don't need to get into now. I agree. I want to do more on that. And then the other is that we're not talking enough about the potential privacy and surveillance implications of chatbots.

27:30And to be honest, one of the reasons that I feel like I haven't done that much work on that is because my mind has not expanded large enough to understand the ways in which this data is so sensitive. There's the obvious things like sometimes it leaks its training data, sometimes you know there's been multiple instances of chats being leaked things like that but i honestly feel like the evilometer like has not been ratcheted to 10 yet in terms of like what is possible and we've started to see like a little bit of this with like chat gpt will call the cops on you if you start like showing suicidal ideation or something like this and it's like rather than like fix things they're just like oh we're just gonna like call the cops or rather than like to pull it out.

28:15There's that aspect, but I'm curious. I feel like you have definitely thought about this more than me. What are some of the privacy concerns, surveillance concerns of chatbots and our relationships with AI? I hate when people say our, but people are talking to these things and telling them a lot about themselves. And it does feel like a pretty big privacy risk, but I don't even fully understand the implications of it. Yeah. I mean, I think some of it, and I've been working and thinking a lot about this for the book, which is that the big push, and I don't even know how much this works, how much it's going to work, you know, things like that.

29:00But the big push towards agentic, you know, in quotation marks, AI, that the thing is now that these, that in terms of like making AI an operating system or giving it access to everything you do, like your calendar, your contacts, your emails, your doctor's notes, like all these things, that in order for these systems to work, not on a sort of like, not even on a sort of technical scale, but just in order for it to do the things companies claim that it will do, it will need kind of unprecedented access and kind of unsiloed access to like, um, your entire life. And, you know, because privacy is relational, not only yours, but everybody, you know, um, and, and it would need to link all of those things together, you know?

29:58So if I wake up in the morning and say, you know, like check my chat bot to see what my schedule is and ask it to schedule an appointment with my doctor and write a love note to my partner and tell my kid happy, all these things, right? That kind of access is unprecedented. And the other thing is, I think when the companies are saying, chatbots can be your coach, can be your partner. These things are not, you know, like, really, I don't know that the companies have like kind of like a, I don't know what their model for making money is. Like, I honestly don't, right? Except to get people to pay for this, you know.