The Shifting World of OSINT (with Craig Silverman)

0:03Hello, and welcome to the 404 Media Podcast, where we bring you unparalleled access to hidden worlds, both online and IRL. Well, 404 Media is a journalist founded company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. And they get early access to our interview series too, just like this episode. Gain access to that content at 404media.co. This week, I'm speaking to Craig Silverman. He runs Indicator.media along with his co-founder. And I followed Craig's work for years.

0:46At this point, he started back at BuzzFeed News, which, you know, was really one of the best investigations, especially for technology news on the internet. He then moved to ProPublica before launching Indicator. And he's really good at OSINT stuff, but beyond sort of geolocating photos, which you usually might associate with OSINT. I follow him because he does a lot of stuff around advertising fraud or digital forensics. There's all of this data and information out there about domains and websites and advertising, all of this thing that is perfectly accessible to journalists or researchers. It's just knowing where to look, essentially.

1:31So in this conversation, we talk about how he got started in that, maybe some tips on OSINT newsletters for you to check out as well, and of course, what he's doing now as well. So I hope you enjoy this conversation.

1:50Craig, thank you so much for joining us on the show. I really, really appreciate it. Yeah, it's good to be here. Thanks. Absolutely. So listeners will have just heard some context about how I came across your work and how I've been following it for, I mean, years, obviously, at this point, I first came across it when you were, I believe, at BuzzFeed News, there was a lot of scams and then disinformation, of course, around, I think, 2016, I imagine, and sort of that election and the Russian stuff, all of that. Am I remembering that correctly? Is that what you were mostly focused on back then, sort of scams and disinfo?

2:27Yeah, yeah. So I originally joined BuzzFeed News, actually as the founding editor of BuzzFeed Canada. So the first year I was there, I was doing like Canadian news and managing a team of about nine other people. But then as we actually got into the spring of 2016, as New York bosses sometimes do, they're like, I don't know if we need all this Canada stuff. We like your team and you guys are good, but you're doing a lot about Canada. Can we do a little bit more about America and the world? And so everybody sort of got rejigged. And it's funny because I mean, when I first started talking to BuzzFeed, it was about doing this sort of debunking and internet viral hoaxes stuff that I was doing before I joined BuzzFeed.

3:11And then the spring of 2016, that's what I went back to. I went back on to like hoaxes and viral fakes and trying to figure out how to make debunkings of false stuff go viral, which no one has still figured out. The debunks always lose. yeah and it's funny because i don't mean to date this interview too much i try not to do that because this will come out later but today i was just reading um a very interesting philosophical blog post from an academic who said you know we have this epistemological problem where there are people just living in another reality that leads to jan 6 it leads to covid misinformation all of this sort of thing and he's like we've tried the epistemological solution which is fact-checking people calling people out and he believes and I think this is obvious to a lot of people he's just saying it's like a different way people link their epistemological beliefs and their knowledge very much to their social identities now I'm not gonna make a whole philosophical discussion but that's just front of mind for me about fact-checking and debunking when you were focused on that, did you feel debunking and fact checking changed people's minds?

4:24Was that the point? Do you think it did that? No. And I think actually that's the biggest problem that sort of fact checkers have is that people think that's their goal. And granted, like, I'm sure some of them, you know, aspire to that and would like to see that happen. But it's in a scenario where someone's mind is actually made up and relatively firm, it's very, very hard to dislodge that. And we're not even talking about fact checking, like to your point of the larger things, this is tied, it gets anything tied up into identity, beliefs, self-image, the way we present ourselves, the in groups that we're a part of.

5:03Those factors are tremendously powerful on the information we will accept or not. And so for me, fact checking, when it came to really, really big, deep questions that related to someone's identity, like that's not gonna do it. You're not gonna read one thing that's laid out in a sort of factual way and be like, oh my God, I've been so wrong. How could I have thought that, you know? And so like any fact checkers that went into doing that, who thought of that, they were quickly realized like, oh, that's never gonna happen. But, you know, where it was actually shown to be effective very early on, if we're talking like before 2015, probably like before 2014, studies that were done of the classic type of fact checking of fact checking claims from politicians found that they did a study where the academic senate a note out to a bunch of politicians who were running in a race and said some of them got a letter saying there will be fact checkers working during your race just letting you know and some of them didn't get it and what they found was that it appeared to have an effect where the politicians who had known that fact checkers might be watching and sort of checking their claims did seem to have it seemed to have an effect it seemed to make them maybe a little bit more cautious.

6:12And so what has destroyed the universe was kind of Trump. And then others like him basically being like, I'm not gonna I'm not gonna feel any shame. I'm not gonna change my what I say, I am actually going to double down and I'm going to flood the zone with shit. And there's nothing these fact checkers can do about it. And so when the personal shame element, or the sense of the norms disappeared in the political realm, that whole element of the political fact checking to actually get the politicians to change their behavior got smashed. And that was a very effective thing initially. But it can work where people don't have a deeply held opinion or view about something, they'll read a fact check, and they might be receptive to it.

6:50And so fortunately, that's still a somewhat decent amount of stuff. But it's not the things that are like deep, deep, deep, you know? Yeah. And well, I don't think you stopped that maybe like suddenly or anything like that. I mean, we'll get to it. But you just moved to other outlets. And of course, you launched your own one that we'll talk about as well. But was there a particular reason maybe you did fewer of those sort of debunking or fat checks? Or was it more, well, I'm a different employer, and they're interested in something else? Yeah, so, well, the big change really was, so before I joined BuzzFeed, I was running a research project that also had its own website called Emergent, where we would identify claims that were being covered by news websites.

7:33And we would look and watch over time as it was disproven or proved, what were the social sharing trends and also, you know, who was getting it right, who was getting it wrong, who was amplifying BS and who was doing good debunking. And so I was interested in like that model and trying to figure out ways of sort of tracking and making it work. And then when I got sort of put back on that beat at BuzzFeed in 2016, I did a bunch of those viral debunkings. There was a lot of stuff spreading, you know, there was still good money to be made, spreading hoaxes for profit on Facebook. And there is again now, strangely, which we can talk about.

8:07But at a certain point, we get to the end of 2016, and everything has blown up, right? Trump has won. And it's like, oh, you know, the Macedonian teens caused Trump to win. It was fake news, like all this stuff that I wrote about and didn't think people went crazy. And, and that's when actually, I realized like, we have to shift, I have to shift because all of a sudden, Facebook, now called Meta decided to pay fact checkers. And so it said, we're going to work with third party fact checkers, we're going to pay them to to tamp down the viral 100 % false stuff. And so I was like, Oh, well, I'm not, we're not going to take Facebook's money.

8:42But now they're paying people to fact check and do debunking. So I need to do things other than debunking. That was the big moment for me. They had almost were not taking the job thus far. That's that's exaggerating it. But like, Oh, they filled some sort of vacuum there, which was ironically of their own creation probably but they've done that so then you can move on to other stuff and i guess that would bring us to what i primarily follow your work for is that you're really on top of OSINT open source intelligence but i would also say even more like on the digital forensics side like when people hear OSINT they think of plane data or geolocating photos or something like that whereas a lot of your work is stuff on I don't know tracking pixels on websites who is information pdns all of this technical internet infrastructure stuff which I dabble in when I need to but you're definitely more on top of it um than me absolutely how did you get into sort of that digital infrastructure so not not beat but sort of a collection of techniques How did you get into that?

9:56You know, it's funny to hear you say that because I look at your work and what you do, and I often think of you as being more technical than me. So it's interesting to hear you see that as the reverse. So, you know, that comes up a lot. I think, oh, wow, I really admire this. And then, oh, no, it's just the imposter syndrome coming out, I guess. But yeah, how did you how did you get into it? You know, I mean, it's it's funny because so I promise I won't make it a really long story. But, you know, it started in 2004 for me. So in 2004, it's like blogging is kind of emerging and gawkers around and I'm following it.

10:30And I'm a freelance journalist just doing like anything I can to make money in Montreal, Canada. And I was like, well, I'd like to start a blog. I've always had kind of a little bit of an entrepreneurial streak. And so I started a blog and I ended up starting a blog about media. And it was about errors and corrections and accuracy. So like I would read literally six days a week, the online corrections pages of every English language outlet I had found and find the biggest, the most hilarious, worst corrections. And that was the initial content was just curating those. And it started doing really well.

11:01And I was like, well, I guess I should understand more about like fact checking and verification and accuracy. And so I started sort of doing research and writing about that. And before I knew it, within like, within a year, I was suddenly like kind of the expert on that because nobody was doing it, which is the beauty of niche stuff. And so it's just been a line from there of me figuring out what are the practices for verification in newsrooms right now that we can, you know, decide when to publish a fact or not. And then when, you know, looking at how the internet and particularly as social media emerged and smartphones merged, how that completely destroyed the old model of facts would come into a newsroom and they would have time to sort of deal with them and call around and decide what to do and then decide what to put out.

11:46And it completely changed because now mostly the information was already out there in some form, not most of it, but like a lot. And it was getting traction and it was getting more engagement and more views than the traditional journalism might ever be able to get. And so when you have potentially false stuff spreading, you know, stuff that's in the gray zone, that changed the whole dynamics of verification. And so I just became obsessed with, so how the hell do we do verification now? And in those early years, you know, from like 2010, 2012 on, it was like user generated content of people posting stuff.

12:21And like, how do you verify user generated content? How do you look at a photo and figure out if it's what the person claims it is? It almost seems quaint now. Yeah, yeah. Yeah. Like, oh, how do we verify this one single photo that maybe somebody posted maliciously or maybe it was just an accident or something? It's like, that's like nothing compared to now. Yeah. And if you think of like Hurricane Sandy, which I think was like 2012, that was a moment where people realized, oh man, you know, we have a natural disaster and there's like photos, people claiming there's a shark in the streets of New Jersey.

12:55The perfect example. Yeah. The street shark, which, you know, basically became, you know, its own meme. Um, there's like a guy, there was a guy with a pretty, you know, well-followed Twitter account in New York who I think, you know, put out a false claim. I know there was a claim about the New York stock exchange flooding, but I don't, I don't know if he was the one who said that, but, and so a bunch of publications like the Atlantic and other peoples started trying to do like real time debunking. And it's, it is quaint because if you look back on it and people were writing about it, like, Oh, there's this deluge on Twitter and on Facebook of this false information.

13:27and if you compare that to what we deal with in the span of a few hours today it's like it is quaint and it's like man i would love to go back to that you know it'd be so much easier like i could have a coffee figure this thing out you know rather than there just being a constant bombardment of ai slop basically um so you're doing that and you're doing the verification but then how does that move into i think the more technical stuff the who is data all of that sort of thing yeah It's just like trying to learn. And that's like the simple answer of it is there was this emerging discipline that I initially, I started sort of trying to document and write about the people that were doing it.

14:07So there was Storyful, a company that was very early in like gathering and verifying user-generated content and then licensing it to clients and verifying real-time feeds. There was Andy Carvin around the Arab Spring who was sort of doing real-time, some people call it like news DJing. He was like grabbing stuff on Twitter and like helping getting people to verify. And so I was sort of like studying that and trying to understand what they were doing. And then we tried to formalize it in 2014. We published with the European Journalism Center. I edited a free book called The Verification Handbook.

14:37And that was the first time anybody had sort of put this stuff down on paper. And we just basically recruited a bunch of people from Storyful and BBC and other places that were doing this and tried to lay down some standards. And so from there, then like because of that book, I was like, okay, this is like, this is my thing now. So I better, I better stay on top of it. And I just am always trying to find new skills and new things to learn. And now the nice thing is like, I learn new stuff. And so I can teach it. And I'm trying to create like a little bit of a flywheel on that. Yeah. I'll try to remember to come back to this question.

15:12But as you say, stuff changes very, very rapidly. And I'll try to remember to come back to that. But how do you stay on top of things because everything is changing so quickly like yeah um i guess i'll just give the example where years ago all of the osin journalists and the bellingcats of the world and people on twitter that sort of thing were big on facebook graph right which was a way that you could basically find these undocumented apis in facebook so you could map out someone's friend list or find the connections or whatever i mean i remember i went to a bellingcat training years and years ago.

15:49And you could just do that. Or you could figure it out yourself and you read blogs or whatever, blah, blah, blah, or you just probe it. And you get all these techniques that gets shut down. And then because Facebook, I think after Cambridge Analytica, right? They were kind of like, we need to close some of this shit down. Then there's a gap. How do you, as a journalist, stay on top of what's next? How do you go about that? Yeah. I mean, the Facebook graph example is the perfect one. And I think part of doing this work is you have to have lived through examples like that to remind yourself of the fundamental thing, which is that like tools are not actually the sustainable way to do this.

16:26It is your creativity and your thinking and your willingness to kind of poke and try a bunch of different stuff. Because Facebook graph was amazing. Like, as you said, you get somebody's Facebook account ID, you popped it in and you could literally see every like public comment they had ever made. It was awesome. It was awesome. Yes. Well, this, you know, this is the thing, like so many of these tools and approaches are double edged swords. It's like when I teach it, like I teach people today, what you can do with a phone number or an email address, and the eyes like when they realize what's possible.

17:01And so then usually the next exercise is like, yes, now you should be putting your own email address in and realizing what you have out there, because this can be used by, you know, people for journalistic purposes and for others. And so staying on top of it, I always felt like it's part of my job. And part of it is I just try to like read and sign up to as many newsletters and follow people in the area as as possible and just sort of see what they're seeing and that kind of thing. And then I also, you know, I try to sort of sometimes there's just stories that you find because you're interested in an area.

17:33And like, you know, I do a lot of ongoing monitoring of the things I care about. Like skeezy marketing is something I really am interested in. So I'm always like I follow and reading tons about that. And the one of the ways I stay on top of it is, is you got to figure out what the people who are like on the line or beyond the line are doing, because they're actually the most innovative. And they're the ones who know and figured out the new techniques to get this out of a social profile or to do that. And so one of the ways is just there's now this whole world of OSINT and people who work in this area.

18:04And then there's also like, what are the sort of bad guys or the people that are like in the gray doing? Because they actually have a lot to teach, even if you're just trying to figure out what their new nasty techniques are for you to potentially, you know, stay on top of. Yeah, I find today that OSINT is just, if you don't have the foundational understanding and the methodologies and all of that sort of thing, which I don't think I always do, to be perfectly honest but if you don't have that OSINT feels so much more fickle now like it feels delicate that you may find an interesting tool or a technique one day maybe I don't know there's some discord stuff going around at the moment where you can see the servers somebody's in potentially that sort of thing that's going to get shut down uh soon I'm pretty sure I've shut down OSINT techniques because I've reported on them because there is a public interest in, well, there's a privacy issue here.

19:02And, you know, we have to wear two hats at the same time. Like, could I use this for journalism? And on the other side, there's a public interest in covering this because this is fucking wild. Maybe it should be shut down. How do you think about that sort of dichotomy in your head when you're using these tools or these techniques? Yeah. I mean, I think inevitably because a lot of times I'm reporting on misuse or abuse of things. And inevitably, my first idea when I discover something new is, is I do think about sort of the abuse potential on that. And then, you know, while also maybe thinking like, oh, wow, I could do this thing I always kind of wanted to do or try to do.

19:38And so I feel like there's always those two minds that are sort of battling it out a little bit of like, man, I would really like to use this, but holy crap, maybe this shouldn't exist, you know? And it's funny because actually there's a very smaller example of this, but you guys reported on Archive.today, the archive service, getting subpoenaed. It didn't subpoena, but its host got subpoenaed by the FBI trying to identify who runs that service. And when I first read that, my first reaction was like, oh, crap, because this is one of the better archiving tools for social media. The Internet Archive isn't as good at archiving a Facebook account or other things.

20:17but archive.today does. And so my initial thought was, oh man, if that went away, that would suck because like there's a repository of archive things, but also I often use it so I can link to it. But the 404 story, when I read it, it reminded me like, oh yeah, that's right. Tons of people use this to archived paywall material and other stuff that isn't supposed to be accessible. And then I remembered like, yes, that's right. This is another use that this tool has too. Yeah. And we didn't put this in the piece because it's speculating between me and jason as in jason wrote the piece and i edited it and we're talking among amongst ourselves and when i see a subpoena like that honestly my first thought goes to that's probably a child abuse imagery investigation just like i've read so many indictments around c sam unfortunately because i used to cover that a lot because the fbi was doing very interesting stuff around that horrible horrible world when i see a raid on a academic's house or i see a subpoena against a web service and this i'm I'm pretty sure that's going to be CSAM.

21:15But again, that was just us speculating amongst ourselves. We couldn't put it in. But yeah, there was a trade-off in, do you report on that or not? But as you say, we do. And then, oh yeah, that was used by 4chan to horribly harass people for years and years and years. So the tools do that, yeah, basically. Yeah, yeah. And I mean, the CSAM point, I can totally imagine why your mind went there too, because it's also sort of like, well, why would the FBI care about this service? And, and I don't think that copyright infringement, because this is like the most active stuff around copyright infringement is like mass pirating of live streams and pay-per-view.

21:54Like that's where law enforcement actually will get involved because the NFL is screaming at them to do something. Whereas an archiving service where like you can read a wall street journal article, you know, that's behind the FBI. I feel like probably not. So I think you're right. Like there's something larger. And I don't think it's because, you know, the FBI or the Trump administration is like, we need to shut down Archive today. God damn it. Let's get up. You know, there's something else going on that sort of put them in the crosshairs. Yeah, I think so. And, you know, maybe I'm wrong and we'll see.

22:25But that's just sort of what we see at the moment.

22:46well i guess actually just before moving on to the next step of your career how do you think the osint industry has changed it is kind of in the question it's become an industry yeah right yeah yeah a hundred percent i guess i'm old enough that i do remember when it was like a very small universe and it definitely was an industry and there were like a few journalists and there were some academics and then there were some weird nerds who did this in their spare time, like the most famous being Elliot Higgins, who turned it into Bellingcat, which is incredible. And it was a small sort of known universe and had some intersection with people who did like threat hunting and information security, but it was sort of cleaved off a little bit.

23:23You know, the first thing I saw was the debunking and sort of anti-disinformation became its own little bigger, much bigger community, maybe not a full on industry, but suddenly money flushed into that at the end of 2016. And it was weird to watch that happen because again, it was a very small community. And the same thing has kind of happened with OSINT. You know, I feel like there's a new OSINT newsletter like every other week or two. There are companies now who sell OSINT tools, a wide range of those. And so, you know, I think that's one of the big trends is there was a spirit of this open source information, but also a lot of open source tools and approaches where it was shared and it was in the community and built for the community.

24:03And now there's, you know, there understandably there is a business opportunity there, but there are a lot of companies, you know, building paid products. And in some cases, they're building paid products on things that started as open source, you know, things around, like I mentioned, investigating emails. I mean, some of the open source stuff to kind of check where emails are registered on different platforms that started as a very open thing in the community. And now, you know, there are companies that are built on top of that. But the biggest trend, I think, over the last few years is just like the fake OSINT world, where I think, you know, there are some governments and then there are some bad actors who've looked at sort of OSINT people and OSINT accounts doing like geolocation and saw that these often established credibility on platforms as like a ground source of truth for people.

24:48And so we saw this emergence, particularly after the full scale invasion of Ukraine, of just these total bullshit accounts on X and tons and tons on Telegram calling themselves like, oh, sent this or oh, sent that. And they would, you know, it wasn't necessarily 100 percent false stuff they were putting out. But, man, like they were really playing with context and they were really leaving stuff out and they were they would share an image and they would make a gigantic leap of logic from it. And so that is kind of undermining it as it's being established more and more throughout militaries, throughout all these places.

25:21There's also this undercurrent of fake OSINT and bullshit OSINT that is sort of undermining what it means to actually do the work. Yeah. I mean, it became a massive grift basically on X where you would, as you say, call yourself OSINT insider. Sorry if that's actually the name of the news. It actually is a thing. Well, I literally just made that up. So it's actually the name of the newsletter of the guy you wrote about who used Steeler logs. Oh, there you go. That's a very, very funny coincidence. I guess we all, it comes to your, it comes to mind when you're in this industry and space enough, but somebody uses a name like that.

25:56Let's say not exactly that genuine, very funny coincidence, but you, but you'll do that and you'll do like a, the alarm emoji breaking or whatever. And then look at this photo, blah, blah, blah. And I don't know, these people, I'm not in their head, but it looks like they just looked at the people like bellingcat and yourself and others doing it and being like well that doesn't look too hard i just i tweet this photo and i get a shit ton of retweets and potentially you know some grifter money from musk on x.com and i can do that and it seems pretty straightforward um totally agree with that i i don't want to put you on the spot because maybe you don't and i wouldn't say this is an endorsement or anything but Is there a newsletter or two you think people should check out beyond your own website?