Mike Caulfield: Verified Methodology for Fighting Misinformation

0:00From October 7th to 9th in San Francisco, the Masters of Scale Summit returns. It's where the boldest minds in business, tech, and beyond gather to build the future. And we want you to join us. Hear from the CEO of the New York Times, scientists using cutting-edge technology to find cures, the leader of crypto powerhouse Coinbase, a retired four-star general, and many, many more. They're all on the same stage, and it's not just another conference. It is the gathering for remarkable people. Apply to attend at mastersofscale.com slash remarkable. That's mastersofscale.com slash remarkable.

1:02paths to transform your life and make a difference. I hope you'll read it. The other path is today's podcast, and we have with us a guest named Mike Caulfield. He is a research scientist from the University of Washington. He works at the Center for an Informed Public. Basically, Mike is renowned for his SIFT methodology, S-I-F-T. This is a crucial tool in the fight against online misinformation and it empowers educators and learners to critically assess online content. Let me explain the acronym SIFT. S is for STOP. I is for Investigate the Source. F is for Find Trusted Coverage. And T is Trace Back to the Original.

1:52In November 2023, my co-authored Verified How to Think Straight, get duped less, and make better decisions about what to believe online with another Remarkable People guest, Sam Weinberg. Mike's dedication to digital literacy has not only earned him the 2017 Merlot Award, but also recognition from top media sources such as the New York Times, NPR, and the Wall Street Journal. So let's welcome Mike Caulfield to Remarkable People. We're going to learn how to sift and figure out the truth of what we see and read and hear online. I'm Guy Kawasaki. This is Remarkable People. And here we go.

2:39Are you drinking liquid death? Uh, no. I mean, yeah, I guess. Dr. Pepper, is that liquid death? No, liquid death is a brand. I'm not talking about the carcinogens in Dr. Pepper. I mean, at some level, probably liquid death. All right. Let's get serious here because we have to save democracy. So first question, how do you verify news stories? So let me just set up a little frame about what we talk about when we're talking about verifying because I think sometimes people have a different conception of what you're doing. Generally, we're looking at a situation where someone has seen something on the internet that might be a news story, that might be an article, that might be a website, whatever it is.

3:23They've seen something on the internet, they've had a reaction to it. Normally that reaction is one of two things. Either this is absolutely evidence of everything I thought I believed, and I believe this is right, and so forth. Or it is, oh, this is nonsense, this is foolishness. And so the question becomes, is the thing what you think it is? And this is a question where we focus on the book instead of, is it true or false? We accept that you've come to something, you've already had a reaction. By the time you're checking, something's already happened. You already have an impression. The question isn't, what is this thing?

3:55The question is whether your impression was correct or whether your impression was wrong. And so when we talk about verifying news sources, what we're saying here is you see something on the web and you react to it maybe because it's called the Mississippi Ranger. And you're like, oh, well, this is a local paper in Mississippi. I hope there's not a paper called Mississippi Ranger. If there is, it's a made-up name. Like no libel. Wait, I got to go to the domain right now. I should have written down some fake names I could use. But you see something called the Mississippi Ranger and you're like, oh, this is just a local paper in Mississippi covering an issue of something that happened there.

4:30And so the question is, is it, right? If that's your impression of it, was your impression correct? And what we suggest on something like that is going to Wikipedia. So if it's a paper, if it's a paper of any size, and actually I worked on a project getting local newspapers on Wikipedia for a while and coordinating that. If it's a paper of any size, you'll have a Wikipedia page. You go there, and if there's a paper of this name, if there's not, it doesn't necessarily mean it's not a paper, but you might want to find something else. It might not be your best first stop. Alternatively, you might go, you might find that the Mississippi Ranger is one of a set of papers that's run by a political consultant who runs something we call a pink slime network.

5:13I don't know if you heard this term, but it's a network of a lot of things that look like they're news producing sites, but really they're auto-generated out of this stuff and usually for some sort of propaganda end. It could turn out that's actually being run by a political consultant. And so when you say verify a source, part of what we're saying is, okay, well, if I thought I was getting this from a local news source and that was behind my impression that, oh, this is really useful evidence for what I believe or don't believe, and it turns out, no, actually this is being run by a political consultant, or no, this is just a spam site, or no, no one's ever heard of this.

5:47then it's maybe not as useful to me. And the way you do that, the way you get that context is with a new source, when you're checking the source, is to start with Wikipedia. If you can't find something on Wikipedia, type in the name of the source into Google News, see if it comes up in a Google News, and maybe type something like funding, who funds this, location, basic sorts of things that would give you some context on that source. Okay, but let's, I was on the board of trustees of Wikipedia. And nobody believes in Wikipedia more than me. All right. But what if somebody says in response to you saying, check Wikipedia, oh, anybody can change anything on Wikipedia.

6:30Why would you use Wikipedia as your reference when anybody can say anything? Well, as you know, because A, that's not really true on Wikipedia. That's true of Wikipedia in 2006. I hope I'm not being unfair here, but I was on Wikipedia in 2006. It was true in 2006. You could get on there. You could say a lot of things. Those things would not be noticed for long periods of time. So the first thing is Wikipedia in 2023 is not Wikipedia of 2006 or 2008. There's just been a lot of effort on Wikipedia to build various bots, various things that look for things that don't have citation, vandalism, unsourced changes, new users coming in from unidentified IPs that are strangely editing a lot of pages with PR content, that sort of thing.

7:22So there's that issue there. It is true on some smaller pages. You can get away with this and that in Wikipedia for a little bit of time. That's not impossible. But in general, if it's a good Wikipedia page, you don't have to trust the Wikipedia page because you're going to come to the Wikipedia page. Anything that is contested, could potentially be contested, is going to have a link, a footnote to it, and you're going to be able to use those links to verify it. And the thing, too, is it doesn't have to be perfect. Wikipedia doesn't have to have an answer to every single one of these because this is the big thing.

7:57The web is abundant, right? If you came to the Mississippi Ranger and you want, this is the source I want to use. And it turns out you can't find any information on the Mississippi Ranger. It's not like you're out of luck. It's the internet. You can go and find a source that you can actually find information on. And so it doesn't have to be perfect because the question is not, is this specific source the perfect source for what I want to do? It's, is this source sufficient for what I want to do? Or should I move on and find something else? And you can move on and find something else where there's a better Wikipedia page.

8:29Do you think the day will ever come when you're asked this question and your answer is check chat GPT or Claude or Bard or Gemini or anything. You obviously said check Wikipedia. You did not say check LLM. Yeah, right now I wouldn't say check in LLM. There's a couple of reasons for that. They are improving, but a lot of them, the information is out of date. Some of them have gotten better with that. They tend to do really well with structure. They don't always do as well with sort of granular facts. So there's masters of style and structure. But the granular facts have been a persistent problem.

9:15And interestingly, there were some predictions that a lot of that would be ironed out by now. But there's a particular thing in an LLM that people may not realize, which is that those algorithms are set to have a little bit of flexibility in them, like a little bit of play. Otherwise, you'd always get exactly the prediction, same prediction for every set of words in front. They'd never get that sort of real generative quality. And that little bit of play that you have to put in there so that it can do some of these things is also the thing that is giving you what people call these hallucinations.

9:46It's going to be a little tougher to work that out than I think people realize because the same thing that's giving you some of the sort of appearance of creativity in the LLM, the thing that people associate with the generativity, is the same thing on the other side that's sometimes going too far and creating these hallucinations. I'm not saying that it'll never work out. The jury's out. For the moment, our recommendation in it is that people tend to think that LLMs are great tools for novices. We actually think they tend to be better tools for either experts or sometimes you find cases where a person is an expert in one field and has some moderate knowledge in another to do that.

10:35But because you're looking at the output and you have to evaluate it, novices can get overwhelmed with what they would need to check. So since we're on the topic of Wikipedia and LLMs, do you think that LLMs are an existential risk to Wikipedia? Because people will go to their favorite LLM and just ask a question where they may have gone to Wikipedia before. So I understand that Wikipedia would be one of the best sources for LLMs. But what happens if people don't go to Wikipedia anymore and they just go to LLMs? It's the same threat for search engines. Yeah, and it's even worse than that because the other worry is what if people that want to stack up Wikipedia credibility, Wikipedia clout, what if they start just using LLMs to write their parts of the Wikipedia page?

11:31And now you're introducing a lot of these errors potentially into Wikipedia via the LLMs. So even if you go to Wikipedia, you're getting LLMs. And I know that Wikipedia is working on some ways to do some detection and so forth and some policies about what you can and can't do. But yeah, I mean, it's an issue. Are LLMs an existential threat to Wikipedia? I think existential threats don't have to be successful. They just have to threaten your existence. You can have an existential threat that turns out not to result in the death of something. And I think in that way, yeah, I think so. I think there's a future where LLMs could do that.

12:14That would be really sad because, of course, a lot of the productive capabilities of LLMs comes from a lot of people putting in time and writing things like Wikipedia. It's a little bit of a, I forget, what's the opposite of a parasitic system, a symbiotic? You can either have a parasitic system or a symbiotic system, right? And there's one future in which LLMs are parasitic. They take all the stuff that people have worked on, provide a value, they suck out that value, they spit it back at the user, they erode the business model for these other things, and kind of just suck their host dry. And then there's a symbiotic future, which is a symbiote is like a parasite, but they live in a way that benefits their host organism.

12:56And that symbiotic future, I think, could be one where we figure out how to make these things work together, play to their individual strengths. And we teach people when you want to go to one and when you want to go to the other. But I think we need that symbiotic future. And I think part of that symbiotic future is people figuring out when it's best to consult something like ChatGPT and when it's best to consult Wikipedia. Okay, because LLM vis-a-vis search engines, I know I search on Google a whole lot less these days. You know, when I want to question like, how do I add an HP printer to my Macintosh network?

13:34Yeah. I used to go to Google for that and get 475 ,000 links. But now I go to perplexity, which is the world's stupidest name for an LLM. But I go to one of these things and it gives me the answer, not links, right? And this relates to something in our book, which is that what most people are looking for is a summary thing. Like your average informational need is a summary because the number of things in which you're not an expert far exceeds the number of things in which you are an expert. And the business model for summary is not great. It hasn't been great for a while. The business model is in making an argument.

14:14You take all your facts and maybe you do a little bit of summarization, but you make an argument. You say, this is the way things should be and you do that. Or the business model isn't selling people things. Like this will solve your problem, not I want to do a summary. There is this problem that AI addresses for some people, which is that people go to the internet and they quite rightly want a summary of something. And instead of getting a summary, they get a list of a lot of people making arguments for something instead of I just want to know what the thing is. first. Or a lot of people selling something saying, hey, that problem you have, here's your solution.

14:49And people get frustrated with it. And so part of what we have to do, I think, is, and this is outside the scope of the book, but we have to come up with a business model for summary to get people the answers they want. That is not, again, it's not a parasitic business model where the summary is coming from a lot of work that people did, but not necessarily giving back or supporting the people that did the work. We're starting to veer off into solving the problems of the world here, but that's okay. I'm up for it.

15:22In my simplistic world, if I did a search, how do I add a HP printer to my Macintosh network? Listen, just like on a search engine, if the right column has ads for toner cartridges and HP printers, I'm fine with that. I don't care. I want you to answer this question because I don't know how to answer it, which is, how do you tell if a large language model is making shit up and having hallucinations? How do I tell? Generally, I consult it for something that I have some idea about already. And that would be the sorts of things. And very often, I'm checking and understanding that I already have when I'm going there.

16:05Now, that said, you're talking about there's different sorts of knowledge and they're not the same. You're talking, for example, about procedural knowledge. You want to set something up. There's one nice thing about procedural knowledge, which is assuming you're not operating a nuclear power plant, you try the procedure if it works. And then if it works, great. That's confirmation. If it doesn't, then you find something else. And so for a lot of procedural knowledge, assuming you're not working with dangerous chemicals or something like that, yeah, I could see someone doing that. They want to know how to do this.

16:38And as a matter of fact, the classic example of that is LLMs are really good at writing code. And if you're trying to write a bunch of computer code to reorganize files on your drive by date and rename them or something like that, make a copy of that before you do it. The LLM can write it, you can run it, and you have some confirmation that the information you got back was good. The problem comes, and this is where our book fits into, and I'm glad you mentioned this because there was another interview I did, and someone was really obsessed with, why do we even need this stuff if I'm just looking up how to set up YouTube TV on my computer or something like that?

17:14And yeah, for that set of things, it's not really a book about that. But there's another set of things where you can't directly verify the knowledge that you're given. And that's different. So someone says, look, the federal deaths in this country are at all-time high. We need a federal intervention for this. They show you a chart. And maybe that's true. In this case, it probably is true. They are, right? But there's no way for you to directly go out into the world and verify whether that information was true or false. And for that sort of thing, I wouldn't trust an LLM unless you really know the subject.

17:46I would, in that case, try to find something that was directly written by a human, particularly a human that has reputational stakes. Someone who, if they don't take care with the truth, is likely to pay at least some reputational consequence, because that's how we build up trust, is we know, look, if this person gets it wrong, at the very least, it's an embarrassing next day at work, which is often enough to have people get things right. Therefore, I could make the argument that the fact that LLMs have hallucinations means that Wikipedia still has a place in this world. Oh, yeah, I would absolutely agree with that.

18:23And one of the things about Wikipedia and the way it's structured is editors have stakes. I don't think people understand this, but some people play Wikipedia like a video game in the sense that there's a dashboard there. And when their changes get reverted, it's painful. The flip side of that is sometimes you get these wars, which get very emotional about it. But people that work on Wikipedia have reputational stakes. They have a dashboard that shows how many times they created an article that stayed up, how many times they contributed an edit that stayed there, how many times it was reversed.

18:55And these things keep the majority of people in Wikipedia on track. You know, ChatGPT, like the company ChatGPT has stakes, but the actual thing producing the thing doesn't have any stakes. And that's a big difference. No, not at all. Yeah. Okay. So let's say that I go to a website and it's got this.org domain and I go to the about page and it talks about ending climate change and making America great again. That's not a good phrase. It looks like it's a legitimate.org,.edu, something. And so what tricks do people use to make a site look credible? And in your case, it's owned by a political consultant who's trying to foster, you know, anti-union voting or something.

19:46Kill the minimum wage or something like that. Yeah. Yeah. So here's the core of what most people do is we talk in our book about cheap signals and expensive signals. And an expensive signal is like your reputation. Like it takes a lifetime to build a reputation. You're very careful about your reputation. You have a history with people that you can maybe find online over years. If you're a reporter, you can look at the articles you wrote 20 years ago in the Washington Post and the articles you wrote yesterday at the Guardian. So there's reputation. That's an expensive signal. And then there's what we call cheap signals.

20:20And cheap signals are anything that gives the appearance of authority or expertise or being in a position to know that is relatively cheap to get. So a classic example of that is .org. The cost of getting a.org is like$12.95 on Yoda Namecheap and get a.org. But someone might look at that and they might say, oh, it's a nonprofit organization. So it's a cheap signal. Being a nonprofit organization and having a bunch of people that talk about your work over time and a bunch of different, that's very expensive. That takes a long time to cultivate. Buying a.org does not. In a similar way, having a good layout on these sites.

21:03There may have been a time where in the 1990s, having a good layout to the site, having a crisp look, at the very least, it meant that you had some money. You hired a web developer who could sling that code, get something up, cut it all up in Photoshop and lay it all out in HTML Dreamweaver or something. But it signals something, maybe not always a lot, but it signals, look, someone believes in these ideas enough to fund it. It signals something. Nowadays, it signals nothing. I think most people know this, but in the case they don't, you can get a website that looks as good as your average newspaper.

21:36Just go to WordPress, pick a template, start typing, and you'll get something. In many cases, it looks cleaner than your average newspaper because if you're faking a newspaper, you don't have to run dozens of ads. So that's a cheap signal, too. And so the people that want to fool you do is they look at all the things that people look at to get a sense of whether something has a good reputation. And then they look at the ones that they can get done in an hour or get done in two minutes. And they do that. And that's what they use to fool you. And so whenever you're looking at something, what we encourage people to do is think about, like, how hard would it be to fake that?

22:12And does that require getting in a time machine and building 10 years of relationships? tips, you know, or does that involve going to Namecheap and buying a domain name? And there's a vast difference between the two things. And what we found in our work is that people made no distinctions between those. As a matter of fact, people tended to overvalue the cheap stuff because it was more immediately apparent. They could see it looking at the page, whereas they tend to undervalue the expensive stuff because you had to go out and you had to say, hey, if this guy is an expert in this, there's probably at least a newspaper article or two that quotes them as an expert.

22:45So that sort of stuff took a little more effort, like just a little bit more effort, but it's so much better evidence than the stuff that's about the surface of the page or the domain name, or whether they have an email address you can mail, or whether there's an avatar picture of a real person who might be a real person, might be an AI person, might be some other person that doesn't know their picture's being used. And in this scenario, when you land at some organization's homepage, would you also go to Wikipedia and look up that organization? Yeah, absolutely. Absolutely. In fact, one of the things we found Wikipedia is best for is telling you what an organization is about.

23:25And that doesn't mean telling you whether the organization is true or false. It's like a nonsensical idea. Is an organization true or false? It doesn't even necessarily mean is an organization credible or not. It just means, is this the sort of source that I thought it was, that I thought I was getting my stuff from? So, for example, you mentioned some of these advocacy sites. You might go to an advocacy site, and one person might go to an advocacy site, stopminimumwage.com, and it says, we're a coalition of restaurant workers just looking to protect our lifestyle with tips, and this bill is going to be horrible for us.

23:57And that one person might go to that and be like, okay, I know they're not restaurant workers. I know this is run by a lobbyist firm. But I'm interested in seeing what arguments the lobbyist firm is advancing. If that's your jam, then great. Go wild. I want to see what a lobbyist organist thinks. I go to a lobbyist organization page to find out what the lobbyist organist thinks. Maybe they're making a good argument. Maybe it's something I should think about. But yeah, for most people, when they come to something, they think this is a research group or this is a community organization. This is a grassroots organization.

24:29I should say, again, I don't know. I'm just making names up here. So I hope that's not a URL that's in play. So the idea is you come to that page, you think it's one thing, you go to the Wikipedia page, and it says, hey, this organization was originally founded by a coalition of the nuclear energy industry and the coal industry. Okay, maybe they have something interesting to say. Maybe there's something – I'm not saying their facts are wrong, but it's also maybe not your best first stop for a summary of what our energy future should look like. You might want to go somewhere else. Okay. Okay. So now tell me, do you think that 100 Twitter employees sitting in Austin will have any impact on Twitter slash X?

25:17I guess it depends on what impact you're thinking here. Any impact is a low bar. Safeguard. Truth. Yeah, Twitter has placed its eggs in the community notes basket. And this is a way that users can add labels to things and rate them and so forth. Say it's inspired by Wikipedia. There's some elements of it that are reminiscent of that. There's some that are not. They've invested less in their trust and safety team. I just say approach these things with caution. On Twitter X, I've been advising people to the extent they stay on it to veer more towards their following tab at this point than therefore you because that algorithm to me seems like it's more and more tuned to just promote sensational content of a bunch of people that I've never seen before.

26:05Okay. But Mike, honestly, when you read the news that Elon Musk says we're going to get 100 people in Austin to address these issues on Twitter, did you or did you not start laughing? this is a yes or no did i start i i sighed let's say that i sighed yeah yeah i i i think if you want to do that at scale you need to fund it at a better level i think it's complex i do think it's complex i do think that even old twitter never quite had it right it's a hard thinking about how to do moderation, how to do labeling, how to do context, how to do all these various things. It's a lot more difficult, I think, than people recognize.