The all-seeing AI webcam

0:00When will AI finally make work easier? How about today? Say hello to Gemini Enterprise from Google Cloud, a simple, easy-to-use platform letting any business tap the best of Google AI. Retailers are already using AI agents to help customers reschedule deliveries all on their own. Bankers are automating millions of customer requests so they can focus on more personal service. And nurses are getting automated reports, freeing them up for patient care. It's a new way to work. Learn more about Gemini Enterprise at cloud.google.com.

0:39Welcome to The Vergecast, the flagship podcast of streaming services that are just the living rooms of strangers through their webcams. I'm your friend David Pierce, and this is the second episode in our three-part series about AI. Like I mentioned last week, we're trying to find ways to make AI sort of real and in the world. We talk about AI in these big, huge, sprawling ways. we wanted to see if we could find three stories about how AI is actually being used by actual people in the actual world. And for this one, our second one, Will Poore has a story. Hi, Will. Hello. So if memory serves, at the beginning of this, we tasked you with not just like finding a story about an AI app, but like doing something different.

1:21Like let's see where AI is being used that isn't just to like write slightly better business emails. What'd you find? I went as far afield as I could go. I wanted to get away from the hype and skepticism cycle around all the AI tools that we're talking about, get away from all the product announcements that we're following. I wanted to make AI kind of unfamiliar to myself again. And a coworker, as I was chatting about this, reminded me of this thing that got shared around The Verge a ton a couple of years ago. It was a project that was called The Follower. And it used facial recognition to match social media photos to open webcam footage.

2:05So basically, this thing looked at unsecured webcams in Times Square. And then it looked at public Instagram photos taken at the same time in the same place. and used facial recognition to match up the people that appeared in both feeds. So you got this creepy second angle on social media posts that came from these unsecured webcams. Oh, man. That's one of those things that it's like you put all this information on the internet and you sort of exist in the world knowing that there are lots of cameras around, but I feel like nobody ever thinks about it. And we've talked about this a lot on this show, that having computers collect information about you is one thing, and it feels bad, but you don't really think about it.

2:47But then when it shows it back to you, it becomes, it feels different, right? Like it becomes visceral in a way that feels almost like horrifying, even though you're like, well, yeah, I took that picture and put it on the internet. Completely. And for it to be paired with, as you say, these webcam feeds that you know in the abstract are out there everywhere and following you around, but you just don't have evidence of you out there in them. And that's what this does. So if you scroll through this project, I saw this one photo of a guy in a leather jacket who's casually leaning against this famous bar in Dublin.

3:24It's a very standard sort of social media post. But then in the camera footage, you get to watch him try out different poses and angles. He A-B tests jacket or no jacket when he's taking this picture, you see how the social media post came to be. There's this other shot of these very two cool hip guys strolling through Times Square. It's really epic looking. But then in the webcam footage, you see them just walking through busy Times Square. They almost bump into someone who's wheeling luggage around. It is just like it totally changes your view on social media posts and how they came to be. That thing is like a shtick of some social media posts.

4:06Like I saw a TikTok the other day that was a person explaining how to sleep in an airport. And she like, you know, put her bags on and attached everything to her in such a way that it would be hard to get off of her, puts on her eye mask, and puts up her hood and goes to sleep. And then it was stitched by somebody who like walks over, waves in her face, and then just steals her phone. And it's like the like Like pulling back the curtain on how this stuff actually works is like part of how this all works on the internet. But the people doing it are still the ones in control. This is just like the world is telling on you in a way that it kind of like, like there are people who would make those behind the scenes video of them doing the 10 different tries, the blooper reel and whatever.

4:44But when it's somebody else doing it to you because they saw it happen. Yeah, it's like if that person stitching the TikTok was actually there for the recording of the first TikTok and messing with the person as they were doing it, which is just next level weird and creepy. Seriously. That's what I gravitated towards because it's all of this very publicly available data plus AI equals weird disconcerting stuff. And I just I figured whoever made that, whoever made the follower would have a lot of very different, very interesting thoughts on AI. So I called him up and talked to him about it.

5:22Just to get started, start with the hardest question first. Just tell me your name and what you do. My name is Dries Aporter, and I'm an artist working with technology. Dries is 33 years old. He lives in Belgium. And for more than a decade, he's been using technology in ways that are inspiring, silly, and kind of disturbing. His work is this unique mix of technical proficiency and outlandish ideas.

6:14like a really weird switch. I was studying media arts and I really loved it. It was something totally different. After that, I went to an advertising company and I worked for two years as a concept provider. That's someone that's come up with ideas for the agency. So that's the fuel mixture for this guy. Electronics plus art plus weird concepts. When I scrolled through Dries' portfolio, the first concepts that caught my eye weren't actually AI-powered. Lots of his ideas are these simple, clever conceits brought to life. They're little bottle universes that play by one or two really simple rules.

6:59Like, he built this interactive exhibit called Recharge. It's a phone charger that only works when your eyes are closed. In the gallery, you lie back in a lawn chair and plug your phone into an outlet. But there's a camera pointed at your head and a computer running face tracking software, so the exhibit only sends electricity to the outlet if your eyes are closed. He also made a website that you can take over and post anything you want, but only if you have more followers on X than the last person to do that. The latest person to post is a guy named Kev Adams, who just wrote, subscribe to Kev Adams.

7:36One other project that I still can't get out of my head is a little gadgety looking LED clock in a classy wooden frame that Dries actually makes and sells. So I have a project called Short Life. It's a clock. It's a clock that shows a percentage. And that's a personal percentage because it shows how much of your life that is already completed based on your life expectancy in your country. I have like different people that order it. Like sometimes I have fathers that bought it for the newborn or this year I sold a clock for someone that has more than 120%. So that person was 108 years old. Oh my gosh.

8:25And I will only sell 1 million. Just 1 million. It's a limited edition. Only 1 million. Order now before it's too late. If it's not clear from all these projects, Dries loves the elegance of a good, simple idea brought to life through lots of tinkering. The first idea is most of the time really complex. But then I try to make them really small. For me, it's important that I can explain the idea in one sentence. I also have this Notes app in my phone, and I note a lot of random ideas. Most of the time, they're really bad ideas. And what I do is I always go back to their list. And what I do is combining two bad ideas together that makes like a better idea.

9:19Do you have an example of two bad ideas that made one good idea? I think a good example here was an app that I developed named Die With Me. and die with me is an app which you can only use when you have less than five percent battery on your phone if you have less than five percent battery on your phone you can start chatting to other people having a battery below five percent how i came up with this idea i was in a city where i was not able to find my way back to my hotel because my phone died and for me that was really inspiring and i was like okay i'm like really i really really depended on uh on my phone i'm even not able to find find my way my way back and then i uh i had this idea like okay i will make an app that you can only use when you have a low battery and i noted that later uh that night in my phone and I think it was on my phone for like more than a year because I was not able to find like a good idea with it because I had like only this concept of okay let's build an app which you can only use when you have less than 5 % battery but I was like really okay it needs to be like a dating app or sort of a game but that was really not perfect for me it didn't click or something And then I had like a vague idea of like a chat app.

10:52And then just one day I was reading through all the ideas and then it clicked like, okay, Die With Me is like a chat app. That's the best concept. Of all the ideas that Dries mixes and matches in his work, there are two that just seem to keep coming up over and over again. AI and surveillance. He says he got started down this path after making a discovery on one of the sketchier corners of the internet. During my work at the advertising agency, I was really into exploring open cameras. When I talk about open cameras, I'm talking about cameras that are connected to the internet with a standard password or no password at all.

11:39How did you discover them and what was so exciting about that resource? I discovered them through a website called Showdown. It's sort of a search engine, but it doesn't index websites, but it indexes machines on the internet. So with this tool, you are able to find open cameras. You find cameras in living rooms and places where people, I think, don't want to have that publicly open. I checked out this website, Shoden, and it is exactly how Dries describes it. After just a couple of clicks, I found myself browsing webcams that definitely aren't supposed to be public. Some were innocuous, like close-up bird feeder cams.

12:29But I also watched people playing with dogs inside a doggy daycare business. There was another camera in a living room that I could control, like pan it around. I got really weirded out and closed all the tabs. Shodan is a site that you could do deeply sketchy things with. And this is a personal nightmare of mine. That I don't know enough about how my own gadgets work, or have enough of a handle on the tech around me generally, that I'm somehow broadcasting myself out into the world without realizing it. like I'm a sticky note on my webcam person. I'm superstitious about all of it. Okay, but wait, hold on.

13:08Well, hang on. Let's talk about how superstitious this actually is, right? Because there is a thing happening with all of these AI companies that there is this incredible race to get tons of data, a vast amount of that data that is just like a thing that someone made that is finished on the internet, right? Yeah, we kind of know what to do with an article or even like a YouTube video. But the idea of all of these webcams around just collecting data all the time, like if you think about live video as just a live stream of data, it feels very different to me. Yeah. Like you're saying, like you're finding these things where I think lots of people are setting up webcams and don't know, like we've all heard the horror stories over the years about people who are hacking into other people's like baby monitors and all kinds of stuff.

13:51And so part of me is I'm not really a tinfoil hat kind of person, But the idea of these are unbelievable fountains of data, plus there are more and more compelling reasons for people to want to get access to and store forever all of that data feels kind of terrifying, especially then if you can figure out how to ask kind of the right questions of all of that stuff. Like maybe that's the challenge there is we're going to end up with just 100 years of Times Square. And then the question is like, well, what do we do with any of this? But it's all there. You just have to know what to do with it, right?

14:24Yeah, that's, well, first of all, thank you for co-signing my paranoia. I mean, it's funny, I'm not a sticky note of the webcam kind of person. Like, I never really have been. My sense is, like, I'm not quite in the I-have-nothing-to-hide camp, but I'm in the, like, as long as I'm reasonably responsible, it's probably going to be fine camp. But, like, I've been to Times Square. There is probably video footage of you in Times Square. Yeah, absolutely. And that is where it's like, okay, it's no longer just like a thing that I'm in relative control of and should avoid making mistakes. It's just like being a person in the world, you become training data.

15:03Yeah. And I mean, that just fans. I think you're right that the vulnerability of my MacBook camera probably has not changed dramatically with the advent of all of these AI companies hoovering up data. But it just fans the flames so much more. And so like these are – When I look through all of Dries's projects, this is all what comes up for me. And so that's my reaction to it. And I'm so interested in his work because I think he has all of the same questions that we're asking. But instead of putting sticky notes on his webcams, he made a lot of really fascinating stuff around it. I was really inspired by this and it started a lot of my projects around surveillance.

15:50We need to take a quick break. But when we come back, Drees starts mashing together AI and sketchy webcam feeds and things get real weird fast.

16:20it easy to post your job for free, share it with your network, and get qualified candidates that you can manage all in one place. LinkedIn's new features can help you write job descriptions and then quickly get your job in front of the right people with deep candidate insights. Either post your job for free or pay to promote. Promoted jobs get three times more qualified applicants. At the end of the day, the most important thing to your small business is the quality of candidates. And with LinkedIn, you can feel confident that you're getting the best. Based on LinkedIn data, 72 % of SMBs using LinkedIn say that it helps them find high-quality candidates.

16:56Find out why more than 2.5 million small businesses use LinkedIn for hiring today. Find your next great hire on LinkedIn. Post your job for free at linkedin.com slash track. That's linkedin.com slash track to post your job for free. Terms and conditions apply.

17:17Welcome back. We're talking with artist and engineer Dries de Porter. At this point in the story, Dries has discovered a treasure trove of unsecured webcam footage from around the world. And being a tinkerer, he starts to wonder what tools he could build to do fun things with all this raw material. So I found an open camera and it was filming a lot of traffic. and I discovered that in the live image, I was seeing a person walking through red light and then it inspired me to like, oh, hey, I can automate this. Like it's really easy to use computer vision to check if the traffic light is green or red and then check if someone is crossing the road.

18:02And I made a lot of projects around like jaywalking. Did you know how to program? Did you know anything about computer vision systems when you had that idea? Yeah, I was exploring a lot, but I always explored with open source tools. So if I need a project and face recognition, I don't start writing it from zero. I use something that is out there that I'm able to use and I combine all those tools that are out there. So most of the time, I don't need to write everything from zero. Do you have ideas first and then you go and find the tools that will help you? Or do you learn about a new tool and have an idea?

18:49Which comes first? Yeah, most of the time, it's me exploring new technologies. I'm just playing around, even checking out documentation of software, documentation of a new API that is really interesting. Checking out the documentation of JetCPT, the API that sparks a lot of ideas for me. In this case, he taught himself how to use some computer vision algorithms from an open source library called OpenCV, which, by the way, is a dizzyingly huge and powerful set of tools. Chat GPT might get more attention these days, but open source libraries like OpenCV are putting things like facial recognition into a lot more hands, for better or worse.

19:40Anyway, Dries gave himself a crash course in Python and then used OpenCV to hack together this funny little machine. I selected an area where the traffic light is, and then it sort of counts up all the pixels and it searches for the main color in that area. If that main color is red, or if it's more red than green, then the software thinks it's like, okay, this is a green or red light. and then it checks in a specific zone if someone is crossing over with the person detection. But he still needed something to do with his auto jaywalk detector. He needed the second idea to combine with this one.

20:29So he added a few more steps and debuted jaywalking. It's an interactive installation where you can report jaywalkers from all over the world with the help of open cameras. that are pointed to crossroads. Visitors to the installation stand in a dark room in front of a bright screen playing a stream from an unsecured traffic camera. The screen also displays a big graphic showing whether the stoplight in view is red or green. So participants stand and they watch live as unsuspecting people cross the street. But all the while, there's one other thing in front of them. A big round button on a pedestal.

21:11The kind of button that looks so satisfying to hit. What happens if it identifies a jaywalker? What happens next? You get the question, would you like to report the jaywalker? And then the visitor of the interactive installation can report the jaywalker by pressing the emergency button. And then it will send an email to the closest police station with a screenshot of the jaywalker and that email address only exists for 10 minutes gotcha okay the system is automatically detecting jaywalkers and it's you're you're forcing a decision by the viewer of the the installation yes what they want to do with that have you like done any machine vision on the installations to know how many people press the button of all the viewers.

22:01In the beginning, it was not saving this data. Then later, I mean, like the last couple of years, I saved this data and it was a lot exhibited in Belgium and the Netherlands. And what I saw that is that more people in the Netherlands presses than in Belgium. AI plus surveillance turned out to be a pretty deep well for Dries. He made an exhibit that tracks celebrities using unsecured camera footage, another that constantly searches webcams for an anonymous man that his sister met once by chance. Both of those are physical installations that are built to run indefinitely in galleries. They're these little purpose-built boxes of electronics with screens that are forever showing live webcam footage.

22:48They're constantly on the lookout for whatever their algorithms have been trained to track. They look harmless and contained on their own, until you think about what they're actually doing. My favorite Dries project, though, is the entirely virtual Flemish Scrollers. So the Flemish Scrollers is a project that checks the live stream of my government. They have meetings and they live stream this on YouTube. And I made software that is constantly checking the live stream to find if a politician is on their phone. If they're for a certain amount of minutes on their phone, it will tweet them and tag them with the help of face recognition.

23:32And the tweets look like, hey, they're distracted. Please stay focused. Then you have like a short video of that person. The idea of using these creepy tools for accountability is just so surprising and hilarious to me. I want this in America. I want Congress fully surveilled by AI while they're debating tech policy. It's just too good. So that's my personal favorite. But the piece I wanted to talk the most about was The Follower, the one that matches social media posts with camera footage. Dries says that one day, he and his friends were hanging out on a terrace in a touristy area in Portugal, and they were watching people on the street take selfies.

24:15and they thought well we know where and when those photos are being taken can we go find them on social media they poked around for a while and came up empty but it gave Dries an idea and then actually I did the same but automated the whole thing so I found cameras again open cameras where people are taking a lot of photos and I saved videos for I think two or three weeks something like that. And then at the same time, I saved all the images from that location on Instagram. So all the photos that were tagged with the location around the camera was saved. And then the hardest part, of course, was finding the people in the videos at the right moment.

25:07The system did its best to match faces between the camera footage and the Instagram photos. Dries says he went through and weeded out false positives, and then he cut the best examples together into a video, and he posted it online. And what was the response to it? I mean, a lot of people were checking it out the day I launched it. People were really surprised. It was a hit, and I get why. It is this one-two punch of a reminder that anonymity is not what you think it is, especially if you're on social media, but increasingly if you're just alive in the world. The whole thing started a conversation that Dries wasn't totally prepared for.

25:49There was also like a lot of false information. Like there was a video saying like, okay, it's an open source tool that works perfectly and you just need to upload an Instagram photo and it will check all the cameras in the world. Yeah. That was really not the, I mean, that's not the tool I produced. Right. It's only working with like specific cameras. Although if part of your goal is to make people think about what's possible, the version in their head that's bigger and more powerful is also possible technically. So in some senses, do you feel like you're making people think about the bigger, scarier tool, even if the one that you made is smaller?

26:36Yeah, and I think what is also important to remember is that I'm only one person producing this. I have limited resources. My skill set is just a basic one. I mean, it's not a team that built this. It's just a bit of playing around, I guess. I mean, do you want people to think, well, all right, if this was just one person who built this, imagine what a whole team of people can do? Are those questions that you think about? Yeah, I think that was something that I saw a lot in the reaction. Like, okay, it's just only one person. Imagine what a government can do. Okay, wait, Will, I have just a really quick thought on this, which is there's this thing that Dries is doing where he's making all this sound really, like, theoretical and sort of, I just did a project.

27:30What if other people do this project? But, like, this stuff is happening. This is so, like, ingrained in the product of all of this stuff. I mean, we spent all this time talking about Clearview AI, which was just a search engine of people's faces. Like, that is what he's talking about here. Like, it's the same kind of thing and used for, I would say, largely terrifying purposes. We think about like, Nilay on the show always talks all the time about how his favorite use case for AR is to have it identify people when he's walking past them. So it can be like, oh, who is this person that I met before?

28:03Like, that's the same thing, right? It's a less nefarious version of exactly the same thing. Just reminding you of who a person is. These are all two sides of the same coin, but this is the kind of stuff that like isn't just happening on Dries' computer to make us wonder about who we are in the world. this is like this is the product in so many ways and he is right that i think he's at the like edge of it in certain ways but also is like dude is just building an app that other people are gonna want to use uh is he thinking about the the how this all feels to him like he is accidentally becoming like a pretty impressive tech developer somebody's gonna like want to download his apps on the App Store here at some point.

28:47How does he feel about all that? Yeah. So this is what is so fascinating and kind of frustrating to talk to him is he's doing all of this incredibly relevant stuff with a lot of parallels to things that we talk about daily. But when you talk to him, he likes to talk exclusively about how other people respond to this stuff or what other people read into it. And this is where I feel like I completely failed as an interviewer. Because after an hour of talking to this guy, I couldn't get him to say why he is so interested in surveillance or what he finds so powerful, scary, interesting about these tools, what he wants to say about AI.