Pokémon Go to The Military Industrial Complex

0:03Hello, and welcome to the 404 Media Podcast, where we bring you unparalleled access to hidden worlds, both online and IRL. 404 Media is a journalist-founded company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co. I'm your host, Joseph, and with me are two of the 404 Media co-founders, the first being Emmanuel Mayberg. Hey. And Jason Kebler. Hello, hello. Yes, Sam is not here this week. I presume she'll be here next week.

0:45But we have basically the Emanuel show this week, or at least in the free section. I mean, Jason works heavily on the second story as well. But the first section, Emanuel, you've published a couple of stories about Pokemon Go and data collection and AI. The first one is, Pokemon Go players have unwittingly trained AI to navigate the world. So the company behind Pokemon Go is Niantic. Is that correct? So what did Niantic announce exactly? And then we'll get into more of the specifics. Niantic announced something called a large geospatial model. This is a term that they coined. They explain in their blog post announcing this that that name is in direct reference to a large language model, which is the type of AI model that we talked about endlessly here over the past more than a year now.

1:48and what is the reference they're making there? A large language model, how does that work? You scrape tons and tons of text from the internet, usually without permission. And based on that data, you train an AI to kind of statistically assume what is the most likely next word in a sentence. And with enough data that sounds like coherent English that can answer you on a whole range of subjects. A large geospatial model is basically trying to do the same thing for the physical world. Joe, sorry to put you on the spot, but there's maybe a short paragraph in the story that you can read where they give an analogy where he talks about a church.

2:43Yeah, I've just found it. and I agree, I think this would be useful because I'm almost having trouble visualizing even what that would actually do, but I'll read out the section here. And it says, imagine yourself standing behind a church. Let us assume the closest local model has seen only the front entrance of that church, and thus it will not be able to tell you where you are. The model has never seen the back of that building, but on a global scale, we have a lot of churches, thousands of them, all captured by their respective local models at other places worldwide. No church is the same, but many share common characteristics, and LGM is a way to access that distributed knowledge.

3:26So basically, it's like predicting what an area is going to be? How would you characterize it? Yeah, exactly that. There is enough data about physical spaces in the model that the model would be able to predict how to navigate the space, right? It's like the thing that it made me think about is like when you're at a restaurant that you've never been to before and you need to go to the bathroom, right? It's like you don't know where the bathroom is, but you've been to enough restaurants to know that the bathroom is like not in the middle of the restaurant. It's somewhere off to the corner to the edge of the restaurant, right?

4:01And it's like you're kind of training an AI model to do the same thing. So it hasn't seen this specific church, but it has seen enough churches to know that this is where the entrance would be or something like this. It's funny, the restaurant example, because I always ask the staff. I also always ask, but you know where it's not going to be. You know what I mean? Yes, that's fair. That's fair. So for those who have lived under a rock for, what, five, six something years, let's back up a little bit. And can you just describe what Pokemon Go is specifically? It's funny because while researching these stories, I learned something that I thought was apocryphal but is apparently true, which is on one of the, on an April Fool's Day, Google put out this video where they kind of announced a fake integration of Google Street View and Google Maps with Pokemon.

4:58and it was just a joke about like oh you'll be able to see pokemon on and google maps um and it was such a viral video that the pokemon company and nintendo got in touch with google and were like hey can we make something like this and they did and that game is pokemon go it's a mixed reality or augmented reality game that really blew up in 2016 where you kind of go around the real world with your phone and like real locations in the real world have pokemon or gyms there that you can interact with and have kind of like a real world pokemon game where you're capturing them doing battle with other pokemons and stuff like this that's all based on like google maps data yeah i i mean it was obviously a phenomenon uh i imagine all of us played it i didn't play it all that much just a little bit but i have a very vivid memory of it was the first weekend it came out and it was a friday night in london and everybody in the street was playing it and you know like lads who were out at the pub were screaming in the road because they were trying to capture pokemon it was insane that was uh i i like played it in central park in new york and just being there.

6:25Actually, I don't think I was playing it, but I saw people... I could tell the groups of people that were playing it. You could tell by their behavior that they were just groups of people walking into the woods and stuff like that. I feel like there are also some stories soon after it launched about children wandering off into the woods and things like that. That was a bit of a panic for a while. Walking to the road and stuff, I assume. remember that and like putting themselves in danger or something so you're playing this game and the way you do that is you point your phone phone's camera somewhere in the real world like maybe there's a a pikachu or whatever outside a landmark and it's on the pavement and you have to film it and point your phone at it like that i mean what data is being collected there and i appreciate we probably didn't really know this at the time we were playing pokemon go it's sort of We kind of learned this a little bit later through reporting and through this announcement as well.

7:26But what is happening there? Is the phone collecting information about surroundings or something? So Niantic collects a ton of data from Pokemon Go, from Ingress, from a bunch of other games it has launched since the success of Pokemon Go. And that data is used and collected in a bunch of ways that I don't want to get into here. because Niantic did talk to me and they were very careful about saying what kind of data goes into this LGM product that they're working on. And this data is when one of those games asks you to take a picture or scan a real world location. So for example, they recently introduced this product, this feature in Pokemon Go called Pokemon Playgrounds.

8:24And that is where you can go to a physical location and pin your Pokemon to that location. You're like, on this bench in this park, I'm putting my Pikachu. And then somebody else can come there and that location is saved in the game so they can take out their phone and see it. And that data feeds into this product. They also had some other features that asked you to scan real locations and monuments and stuff like this. It is unclear to me whether that stuff feeds into the LGM or not. It's a little vague, but it's that kind of stuff that is informing the LGM. pictures of real places and things in the world that are also attached to location data.

9:22So Niantic knows where that thing is. Yeah. So I edited this piece and I was looking into this and it's not super clear which specific things Niantic is using for this specific product. But I will say that over the years, they have... I guess I just want to highlight that they incentivize players to do this. It's like you get little rewards if you do it. Right. It's like gamifying that type of collection. Yeah. And it's... I mean, okay. This is not a real example, but in my mind, I'm just like, please go scan the nearest nuclear silo. We put a Bulbasaur there so that we can get images of this.

10:05And to be clear, that's not a real example. But that's kind of the vibe where it's like, please go to your nearest monument or church and like hold your phone up to it and walk around it get it from all angles and like we will give you some pokeballs for for doing so yeah um it reminds me of another story from a couple years ago in the wall street journal byron tau wrote it and the headline was gig app gathering data for u.s military others prompts safety concerns and it was this app called premise where users were told, hey, go here and take some photographs and we will give you money or whatever.

10:46I'm obviously oversimplifying the piece, but basically a gig platform for going and doing OSINT for people. I'm not saying it's one and the same here. They're building an AI model here, but it is still the outsourcing of gathering data about certain landmarks. I don't know. I find that pretty interesting. Was this in the aftermath of Russia's invasion into Ukraine because I remember there were a couple stories immediately after Russia invaded Ukraine where it was there was like a panic over gig workers scanning specific things and then concern that it was being used for targeting in some way and it was like in the immediate days after the invasion and I don't remember like there was definitely some over over panic there like I don't know how it actually shook out but I do remember that there were definitely concerns that it was like, not Uber drivers, but you know, like gig workers, as you say, like taking pictures of plotting like different things that people were worried were being used for targeting.

11:51Yeah, definitely around that time and definitely those concerns as well. Byron found it was being, as well as that, this was being used by the US military, which was, that's like almost even crazier to me in a way. I was going to ask, what does Niantic say it's going to do with the data? But I feel like we touched on that and actually that'll be better for the second story. So before we get to that, in the headline, we put that Pokemon Go players have unwittingly trained AI. We got some pushback against that. I think Jason or Emmanuel elaborated in Behind the Blog, you know, sort of our behind the scenes paying subscribers article that we publish every Friday.

12:37but just briefly i mean a very simple question do we think that the english lads who were playing pokemon go on friday night when it launched and getting all rowdy do we think they were aware they were contributing to a you know a mass data collection model being used to power uh you know the generation of a new piece of ai technology can i give my version and then jason you can give the spicier version so i think there are two types of people who played the game in regards to this question and one category of people they think it's just a game they had no idea at all that any of this is happening that they're collecting data of any kind these are children that are playing this game.

13:30It's like, it's really ridiculous to assume that they would know that they're generating data for this company to use. And then there's this other crowd that maybe are people who read our website and are tech savvy and are concerned about data collection and privacy and security and all this stuff. And they rightfully assumed that the players are generating very valuable data and that Niantic will leverage it in some way. And I think that's a fair assumption, obviously. But even in that case, no one could have predicted, I think, that they were doing this to create something called an LGM, a concept that didn't exist back when the game first blew up.

14:20So some people may have had suspicions about Niantic using the data, but I don't think we could have assumed that this is what it would be used for. And surely they will continue to leverage the data in new ways as time goes on. Yeah, I mean, this was my behind the blog and it was probably one of the spicier ones that I've done. My post about it went relatively viral. So a lot of people complaining about that unwitting part were in my mentions. But Emmanuel pretty much nailed it. I think all I would add is that there have been studies that show that no one reads the terms of service for things.

15:06We know this. And it's not like you can negotiate terms of service. But studies have shown that 99.8 % of people do not read terms of service. And then of the people that read terms of service, more than 99 % of people do not understand what they mean. And there's that. There's the fact that Niantic was part of Google and then I believe it was sort of spun off. I don't know if there's any relation anymore. But it has always been something of a mapping company. And so there are people who said like, oh, well, that's like what Niantic does. But Ingress, their first game was released in like 2014, I believe.

15:48And then Pokemon Go came out in like 2016. and to Emmanuel's point, a lot of the people playing this are children. It's like these kids and their parents don't know the history of Niantic as a company or even what Niantic is. I don't think that a lot of people are researching the background and funding of the mobile game that they play. I play video games all the time and I couldn't tell you who developed half the games that I played for hundreds of hours, even if their name flashes every time I started. up just because it's like not something that i pay attention to and then the other thing is that you could assume well so i i just went through the the signup process recently on pokemon go before we published this and it asks for all these permissions it's not like it's saying hey we're building a a model an ai model based on this stuff it's like hey scan this thing and we'll give where we put a Bulbasaur and we'll give you some Pokeballs.

16:51And if you don't want to do that, you don't have to do it. But it incentivizes people to do this. And anyone who's actually playing the game and wants to be successful at it probably is doing these quests to some degree, unless you're highly informed. I think even if you're highly informed, what is happening with AI now is just new and I think unexpected. So when you use Facebook, when you use YouTube, you grant the company all these rights. And we all kind of know that because we know how these companies work. But I don't think that people who post their drone videography to YouTube could have assumed at any point that YouTube would, or some other company would then process all those videos to create an AI video generator.

17:45that would eventually put them out of a job because video is much cheaper to generate this way than pay them to make it. It's just... Yeah. It's the way that the data is being leveraged that is totally new and unexpected, I think. Right. And the very last point I was going to make is people said, Oh, what did you think their business model was? And it's like, Well, if you're even thinking about that, first of all, it's a free-to-play game, but it has in-app purchases and they make billions of dollars from those in-app purchases. That's a pretty common one. You also might assume, because as Emmanuel said, it's like you sort of know that you're giving them information.

18:24And you might think like, hey, there's a business model that I'm familiar with. And one of those business models is targeted advertising based on your location. And Pokemon Go has that also. And so that's not something that I like, Joseph. I know that's not something that you like. But that is like a type of thing that an informed person might think, okay they're using my location to target ads at me and you can make that calculation like i'm okay with that or i'm not okay with that but then what they've done is recontextualize that same information that they're using to deliver targeted advertising to you to then build like this big mapping platform that can be used for all sorts of other things which probably leads us into the second story yeah and i'll just add that even if you did assume it and you know you thought that oh my data is going to be used in some sort of way okay but now they admit it it's news they are coming out and saying they're building a model so i don't know that's new information so people can make informed decisions right but you're right it does go into the next story which is pokemon go data, quote, adding amplitude to war is obviously an issue, end quote, Niantic exec says.

19:40I kind of butchered that headline, but basically there was a conference run by Bellingcat, the OSINT organization. The person on stage was Brian McClendon, Niantic's senior vice president of engineering, and as you alluded to, Jason, formerly the co-creator of Google Earth, Street View, and Google Maps. Emmanuel, what did Brian say on stage that made you think, I should pull this out and do a second article on these comments? So I should say that the reason I blogged this in the first place was Garbage Day had a short item about Niantic's blog post announcing this. and it was referencing a tweet by an austinite researcher who was saying haha isn't it funny that all our pokemon go uh gaming is gonna fuel killer robots and that's a very provocative idea but i didn't say anything like that in the first story because there wasn't any information indicating that it would be used that way.

20:57And I asked Niantic and they didn't address the question. Well, what did you ask them? I asked them, will you have any restrictions on who you sell this data to? Specifically, will you sell it to militaries and governments? Will you have any limitations about it being used in some kind of lethal force? And they just did not address the question either way. And I didn't think it was fair to include because that would be pure speculation. And all the original blog posts said is that they wanted to use the LGM as a critical component for AR, which is augmented reality, and robotics, content creation, and autonomous systems.

21:39And that could be like anything, right? Jason has covered these food delivery robots in LA. So you can imagine it informing something like that. It would be very useful in that scenario. But then I found out about this Bellingcat conference where he was, and it's a fascinating talk, which I recommend everyone watch. And Bellingcat asked the obvious question, to which he responded that basically, if a military government uses it in any way that a regular consumer would use it, that's okay. but if they use it in a way that adds amplitude to war that's definitely an issue him saying that's definitely an issue is it's not a no by the way and i asked niantic as well does that mean you won't do it and they did not say that they said this is very early on we just announced this this is months or years away from being deployed and we're exploring all options which is definitively not a no it's a very long no yeah it's a very long Or it's not, but yeah.

22:50It's a very long, like, probably. Right, right. And, I mean, that's already incredibly interesting in those comments and sort of that dancing around the issue and him saying that, you know, adding sort of force to a war could be an issue for them. We can't really go beyond that at the moment because, as you say, it's being developed, but, like, what do we make of this? Is there a future in which this AI model could be used for military things it could be used for more innocuous things is it like basically a we wait and see you know and because i i think the one thing we do know is this data is probably quite valuable so um i think during his talk he shows this video about how um he just he shows this clip where they take all the scans of this courtyard with a fountain in the middle and And with the photos of that location, plus the geolocation, right?

23:55Just knowing where the phone is physically in the world, they were able to create a 3D model of that space, like a video game, right? So they take a bunch of 2D images, they take a bunch of location data, and they create what looks like a 3D level from a video game. Now, with a game as popular as Pokemon Go, if you could do that to major cities all over the world, and then one of those cities becomes a war zone, and you have what they call centimeter-level precision in mapping of those locations, then you don't have the problem that Jason's delivery robots deal with. Because you're navigating those spaces like it is a 3D level of a video game.

24:45You know what the space looks like. So if you have a spot robot mounted with a machine gun, it's much easier to navigate that space if you have that kind of data as opposed to doing things like we currently do, which is like you have a robot with LiDAR cameras, and they just go down the street and they try to figure out what is happening around them as it's happening and parse it out. And that's how you get them falling off the curb and you get self-driving accidents and stuff like that. You're not trying to parse out the environment as it's coming at you. It's like you literally know every inch of the environment.

25:23Or you can, the Boston Dynamics, whatever sort of dog robot can predict that in this sort of European style city or whatever, or this American style city, there are certain characteristics. and while I turn around this corner, there is a high probability that the curb is going to be this sort of height. The curb in London is going to be of a certain texture or a certain height or a certain angle or whatever, and it could maybe make predictions and movements based on that. So yeah, you can see the benefit of it. Yeah, and most, I should also say, and Niantic says this as well, most of this type of data that exists at the moment does come from dash cameras and self-driving cars.

26:05and the killer app aspect that Pokemon Go has is that it's a bunch of data where only pedestrians go. And different levels. If it's a dash cam, it's just one stationary sort of thing. And if you're moving a phone around, it's going to be all over the place. That's right. Yeah. Jason? I was just going to say that they're not my delivery robots, but I should start a delivery robot corporation, open source it. Yeah. I mean, why not? Well, actually, I'll tell you what, you can take that branding yourself. Maybe that can be a side project for you. All right. We'll take a break there. When we come back, we're going to talk about something completely different.

26:55We're going to talk about the rise of the AI pimping industry. That is a word or a series of words I didn't think I would ever say. We'll be right back after this.

27:31with over 200 of the world's best. That's why Wirecutter calls it, quote, an invaluable gift. I'm going to be doing a lot of cooking this holiday season, so I'm learning the art of home cooking from Alice Waters, French pastry fundamentals from Dominique Ancel, and how to talk about wine without being a snob with Emily Wines. My favorite thing about Masterclass is it works in whatever format is convenient for me. I can watch it on my computer, phone, or TV, and when I take my dog for a walk or drive, I can listen to the audio versions. There's no risk. Every new membership comes with a 30-day money-back guarantee.

28:09Give your loved ones a year of learning with Masterclass. Masterclass always has great offers during the holidays, sometimes up to as much as 50 % off. Head to masterclass.com slash 404pod for the current offer. That's up to 50 % off at masterclass.com slash 404pod. masterclass.com slash 404pod. It's the most wonderful time of the year for deals and also a good time to think about making some changes. Speaking of changes, how about changing your mobile provider to Mint Mobile with their 15 bucks a month deal with the purchase of a three-month plan? Unlike other life changes, this one doesn't take too long.

28:53The longest part of the whole process is the time you'll spend waiting to break up with your old provider. And Mint Mobile's website makes it very easy for you to port your device and your phone number over to Mint Mobile without changing your wireless experience. To get started, go to mintmobile.com slash 404media. There you'll see that right now, all three-month plans are only$15 a month, including the unlimited plan. All plans come with high-speed data and unlimited talk and text delivered on the nation's largest 5G network. You can use your own phone with any Mint Mobile plan and bring your own phone number, along with all your existing contacts.

29:29Find out how easy it is to switch to Mint Mobile and get three months of premium wireless service for$15 a month. To get this new customer offer and your new three-month premium wireless plan for just$15 a month, go to mintmobile.com slash 404media. That's mintmobile.com slash 404media. Cut your wireless bill to 15 bucks a month at mintmobile.com slash 404media. $45 upfront payment required, equivalent to$15 a month. New customers on first three-month plan only. Speed slower above 40 gigabytes on unlimited plan. Additional taxes, fees, and restrictions apply. See Mint Mobile for details.

30:12Hackers and cybercriminals have always held this kind of special fascination. Obviously, I can't tell you too much about what I do. It's a game. Who's the best hacker? And I was like, well, this is child's play. I'm Dina Temple-Ruston, and on the Click Here podcast, you'll meet them and the people trying to stop them. We're not afraid of the attack. We're afraid of the creativity and the intelligence of the human being behind it. Click here. Stories about the people making and breaking our digital world. AI machines. Satellite. Engine ignition. Click here. And lift up. Click here. Every Tuesday and Friday, wherever you get your podcasts.

31:02All right, and we are back. This is one that both Emmanuel and Jason wrote, inside the booming AI pimping industry. So how does this story start? And I think it might be good to give people a concrete example of what someone might see when they sort of scroll through social media and they come across one of these posts that you're talking about in the piece. I don't mean the educational ones and how people learned through AI pimping, but sort of what does the end product look like? Yeah, so this is actually a follow-up. So we did this story in partnership with Wired. It was the second story we published on Wired.

31:47And Emmanuel and I worked on it for several months, sort of like in the background for a while. and it grew out of reporting that I did and I believe published back in February about the rise of AI influencers on Instagram. And essentially, if you are scrolling through Instagram and let's say that you generally like a lot of Instagram influencers or models, I mean, these are usually almost always women, although there are sometimes men. We saw some AI-generated men as well, but it's mostly women. And it's mostly them in bikinis at the beach or by the pool or in front of a mirror, that sort of thing.

32:34And it's like modeling images more or less. And it's just like lifestyle content, for lack of a better term, like aspirational travel content. and you'll either see a grid post where it's just like a photo or you might see a reel. And that's a critical distinction that we can talk about. But basically, we saw both types of things. So yes, you have that lifestyle content, the sort of stuff you see on Instagram all of the time. Emmanuel, what does it look like? Is it photo? Is it video? and crucially, what is different about this? Because we're not writing about normal Instagram influencers. What is different about this content?

33:22Yeah, so first I would say it's very possible that you've seen this content and you don't realize it because it looks real. It looks convincingly real. You would not be able to tell that there's anything not right about it. But yeah, it's either still images, models, influencers, beautiful people that you can aspire to be, or reels, which are really being promoted by Instagram right now. It's trying to put them in front of you in many ways. And those look like real video. and then what's different is it that the face has been swapped like what differentiates this from a normal piece of instagram content it is what all this content that we highlight has in common is that it is originally content from a real human creator who posted it to their social media and someone else just took that video and used AI to make a deepfake video.