Who Decides What’s True on Wikipedia?

27:32Coleman Hughes:So this harkens back to my first question my opening paradox what kind of person spends all day litigating facts on Wikipedia for free, presumably, right? These people don't get paid, right? Do they? But they don't get paid by Wikipedia. No, they don't get paid by Wikipedia. That's true. And they're anonymous. So they don't actually get the kind of status that could translate into real-world benefits. They don't... Nobody knows that they're... They can sell their services if they want. It's something that is extremely widespread on Wikipedia. I've found myself, I think, well over 200, and that's extremely low-balled number, agencies.

28:19These are independently operating businesses that all they do is edit Wikipedia for money. So if you're paying the agency, the agency has to be paying an editor and or an admin. This happens all the time, all across the site. It happens in many different ways. It happens in what they call black hat editors, which are unpaid illicitly. They're breaking the terms of service. And then you have white hat, which declare the conflict of interest. But the New York Times has hired them, a white hat editor. NBC News has hired them. The government has hired them. So there is a lot of payment running through Wikipedia.

28:56For sure on the order of tens of millions of dollars. If it goes beyond that, I would not be surprised.

29:02Coleman Hughes:That's interesting. Okay, so let's talk about some of the specific instances of bias that you've written about, in particular related to Israel-Palestine and post-October 7th. So what we found in some of the research, I was contacted by a few sources on this. And the story is about this group called the Gang of 40. It's about three dozen editors who are decidedly, determinedly anti-Israel in virtually all their editing. They've edited a combined one million, made one million edits to around 10 ,000 articles having to do with Israel-Palestine. And that includes everything from like whitewashing the Hamas 1988 genocidal charter from articles.

29:51They'll just go through delete, delete, delete, delete, delete. They'll do the same thing with Palestinian terror attacks, so attacks by Hamas, in some cases Hezbollah, Palestinian Islamic Jihad. They would just mass delete. They get very subtle. These guys are very smart as well. A lot of them seem to have an academic bent. So they'll go through very deep historical articles and make subtle changes that change the framing. For example, there was an article called Hellenistic Judea, referencing the land. They changed to Hellenistic Palestine. You know, the article had been stable for years.

30:30Coleman Hughes:Hold on, hold on, hold on. So this is in that Greek Hellenistic period. Yeah. But wasn't it not called Palestine until the Romans started calling it Palestine? Wouldn't that have been later? That would be my understanding. I'm not an expert of the history of the Holy Land. I'm sure there was a good argument to be made that that's the case, that it probably would have been called historically Hellenistic Judea. But I'm sure they found an article in some journal that made the case about it. And maybe that journal was not exactly mainstream, but because it's considered an academic journal, they can use it to justify the claim, the change.

31:13And because there's three or four people who are guarding the page and specifically guarding, they're patrolling that particular title, they can just keep it locked. The same thing happened with the Zionism article where the Gang of 40, a few of the leaders of this Gang of 40 that I reported on, gained control of the lead section of the article on Zionism. So Zionism, this article would be the most important resource online for defining Zionism to the world. They grabbed control of the first few paragraphs, which is the most important part of the article. And they basically changed a part of a sentence that quite clearly makes Zionism equivalent to ethnic cleansing in the second sentence of the article, I believe, and then locked it.

31:57They put what they called a moratorium on it for one year that nobody could touch it. So they froze it in time. How they did that, it's not quite clear. They just grabbed this concept from the toolbox of a moratorium. They did it. These are the kinds of things that are happening on a moment-by-moment basis. It's happening right now. If I were to open up my computer and start looking around, I'd see them making changes to articles about Israel, about the Golan Heights, creating new articles with titles that make an ideological point. Like there was an article I just came across created days after October 7th.

32:32That is something like instances of when the IDF denied a claim of an attack that was eventually proven to be true. Like that's basically the title of the article. Is that ridiculous? That directly an ideological tool. They do dozens of things, dozens of pages like that that they create.

32:50Coleman Hughes:And of course, there would be no analogous article in reverse, like instances of attacks that Hamas has denied or Iran has denied and later turned out to be true. because it's too specific and it's too much clearly tilted at one side, right? I would imagine in this particular case that that equivalent article doesn't exist. There may be other cases where there have been others created, but for the most part, you see this major shift where articles that have been stable for literally years and years and years, after October 7th, everything starts to change. And this is actually something that, going back to our early conversation about individuals being attacked, I'm sure you know either of or know personally Naz Daly, Nazir Yassin, who is a…

33:59very supportive of Israel and said in an amazing tweet that he felt like he is now, he used to be Arab-Israeli, but now he's Israeli-Arab. He's putting the Israeli part first. He feels connected to his country. And the moment he did that, his page changed entirely. It was attacked. He became, instead of an entrepreneur and this massive creator, He became an influencer, same kind of like reshift as Andy know. And he was attacked specifically for advocating for normalization. That was the attack. That was considered a bad thing. Normalization, this is a point he just made with me in a conversation.

34:44Normalization used to be this sort of standard of what we wanted from the Middle East. We want normalization between countries, between Israel and the Arab countries. between Israel and the Palestinians, because that was a classically liberal idea. But Wikipedia, like the media, has lurched far to the left. And within the leftist framework, normalization is exactly what they don't want in the Middle East, as it has to do with Israel. They want global intifada, and they want the conflict to continue so that Israel loses. It's very zero-sum. And that's kind of the cultural environment that shifted after October 7th on Wikipedia.

35:23That became much more pronounced, and then that became operationalized.

35:45Coleman Hughes:and who are frequently overwhelmed by the constant deluge of movies, shows, books, celebrity news, and online drama constantly demanding your attention, this is the podcast for you. Every week, they'll be cutting through the noise to explain what actually matters in the world of pop culture, what everyone is watching, discussing, and obsessing over, or will be soon. As someone whose pop culture knowledge begins and ends with NBA highlights and chess, I'll be tuning in whenever I get the chance. So check out Second Thought, available now wherever you get your podcasts. If you've got spring fever, Lowe's has the cure.

36:24During Spring Fest, make your landscape stand out with three free bags of Miracle-Gro three-quarter cubic foot garden soil when you buy three. Plus, get up to 40 % off select major appliances to keep clothes, food, and dishes fresh all season long. Our best lineup is here at Lowe's. Valid through 422. While supplies last. Selection varies by location. See Lowe's.com for details. Soil offer excludes Alaska and Hawaii.

36:50Coleman Hughes:It seems like there's at least two problems. One is people editing and reframing and actually committing sort of lies of commission, right? And those are the easy ones to identify. The problem there is just if there's 100 people working to insert lies into Wikipedia articles or take out unflattering truths, do you have 100 people on the other side working in the opposite direction? Maybe not. But then there's the deeper problem of sort of curation, you could say. I know if you wanted to go to Wikipedia and find out about the transatlantic slave trade, there's going to be just a bevy of information there.

37:41Coleman Hughes:And there's going to be a main page, and there's going to be a lot of related pages related to subtopics of the transatlantic slave trade. If you want to learn about the trans-Saharan slave trade, there's just less there. There's less primary source books. There are fewer primary source documents in the first place for historians to look at. There's less interest in the topic in general, less knowledge of the topic from the Wikipedia anonymous admins. and so you know and god forbid you want to learn about slavery in Korea it's like there is a Wikipedia page it's just like pretty short and that doesn't really reflect Wikipedia's bias so much as the overall interest of the whole global academic community like it's actually tough to find a single full book on slavery in Korea and they had slavery for a thousand years and it was only abolished around 1900 so it's like a perfectly worthy topic of academic study, but it's, I kid you not, difficult to find a full cover-to-cover book that's just on that topic.

38:45Coleman Hughes:Trivially easy to do for American slavery, slavery in Brazil, slavery almost in any, slavery in Jamaica, you can find many volumes on it and so forth. And so in some way, Wikipedia is also suffering from the generic biases that academics around the world have been way more interested in discussing certain topics and way less interested in discussing others. And that sort of just reflects in general knowledge and in what Wikipedia pages even exist for us to fight over in the first place.

39:25Coleman Hughes:So there's another aspect. So you mentioned earlier that one of the big reasons this matters more and more is because because all of the LLMs are trained in part on Wikipedia. Obviously, they look at Wikipedia, they look at the sources cited in Wikipedia's claims. Obviously, they're trained on a lot of other material too. But at the margin, if Wikipedia gets influenced in one direction or another, LLMs get influenced in that same direction. And I'll give you an example recently. recently this guy Zed Jelani on Twitter he made a bunch of claims about a trip that I went to I went to Israel a few years ago I actually talked about it on the podcast at the time in my maybe my most viewed podcast of all time with with Dr.

40:17Coleman Hughes:Benny Morris the great historian of the Israel-Palestine conflict I had gone to Israel probably a a year prior on a government paid junket trip. And on that trip, there were supporters of Israel. There were also critics of Israel. And we went around the country. And obviously, this is one of those cases where the government would not pay for this trip if they didn't think it would be overall beneficial to how these journalists view Israel. So to that extent, it's a light propaganda trip. at the same time governments all around the world do this um and there is a difference between going to a quote-unquote propaganda trip in a country that has free speech and you can you know we can meet you can meet a supreme court justice record your conversation and ask them the hardest possible question about the occupied territories and nothing ever bad would happen to you as a journalist so you have total freedom.

41:15Coleman Hughes:You're not in North Korea being shown fake grocery stores. So there is, anyway, there's propaganda tours, and then there's propaganda tours. And so that was my sort of explanation of that. Anyway, this guy Zed Jelani, he had friends on the trip. And he just decided, woke up one day and decided to attack me on Twitter and say, when Coleman was in Israel, he refused to see Palestinians. And I was like, what? That never happened and doesn't sound like me. And so he basically got the details of this wrong, where there was a separate trip that other people organized a couple days prior to the trip we all went on, where they went and saw the occupied territories.

41:54Coleman Hughes:And it was planned at the last minute, so I wasn't able to come. Whereas the trip I had gone on and that we all were able to make had been planned for seven, eight months in advance. So most of us didn't go on that earlier trip because it was hastily planned. So that's what happened. And there were two separate trips. And he was portraying it as if it were one trip. And we had the opportunity to talk to a bunch of Palestinians. And I just said, I stayed in the hotel room with my arms folded, which is ridiculous. And everyone who went on the trip defended me on Twitter. But the thought that I really had is, I've been dealing with people lying about me on Twitter for years.

42:33Coleman Hughes:Generally, as I've gotten older, I respond less and less because I simply don't care. However, in the age of LLMs, I actually do respond very neutrally with the facts as they actually happened, way more so that the LLMs actually have the facts, rather than to persuade anyone in the public who's watching right now. Because I understand in two years, somebody's going to look this up on Grok. It's going to come up, hey, did Coleman refuse to see Palestinians on an Israel trip? And if Grok doesn't have my version of events corroborated by other people on the trip to look back on, then it's very possible they'll present it as if it's like an unsettled issue or like a legitimate accusation as opposed to what it is, which is total BS.

43:20Coleman Hughes:So increasingly, and I think Tyler Cowen said something like this, he writes for the LLMs, for posterity, because everything you write now is being fed into this superintelligence that's going to be sort of educating humanity for hundreds of years to come. Yeah, well, that's, you know, this is basically what we're talking about when it comes to data poisoning. Because people understand, like, you're sort of approaching this in good faith, and someone who wants to literally set the record straight. Right. There are lots of other people out there and lots of more than just people, groups, countries, whatever other agencies that are operating that seek to do the exact opposite.

44:09They don't want to set the record straight. They want to scramble the record. And they understand that when they do so, they can affect the same superintelligence you're trying to affect. but they can do it in a way that is highly, highly informed as to how these processes work, that is really well resourced, and that is very dogged and dedicated. So over the course of not just months, but years, you can, or sometimes it is months, but working editors who are working, you know, eight hours a day on something, clearly not doing this in good faith, clearly working for some unseen purpose. And they're affecting the same LLM downstream that you are.

44:52You're just trying to do it in a way that tells your side of the story. They're trying to do it in a way that manipulates information, contaminates it. We see the same thing. This is not just Wikipedia. The same kind of mechanism was on Reddit because Reddit is highly trusted by LLMs as well. The data is considered really good data. And because Reddit, like Wikipedia, are one of the few large-scale social media platforms that are text-intensive, but are not owned by a big tech company that's trying to create its own models. So they're free agents to sell their data to all the LLM companies, to the producers of these AI systems.

45:30And that sort of increases their impact. The difference, I think, between the two is that on Reddit, We still have this notion that these are users expressing views and opinions. On Wikipedia, we've been acculturated to the idea that what we see there is neutral and objective, almost as if it doesn't have an author. Because when you actually look at the interface, there is no author. It doesn't say this sentence, this sentence by user X and this sentence, this sentence by user Y or by Tom. It just presents it as authorless. And that gives a cultural impression of not just neutrality, but something that is really approaching ground truth.

46:12And this is something that Wikipedia itself also began to push in 2016, 2017, when we had the fake, quote unquote, fake news epidemic that Hillary Clinton declared in front of Congress in December 2016. And you're opening up this gap of disinformation because it's supposedly coming back to our Russiagate discussion. Disinformation was supposedly responsible for the first election. Supposedly, according to Clinton, the DNC, the Obama administration, outgoing. Clinton lost the election because there was social media disinformation, Cambridge Analytica. We remember that story, as well as all this other stuff coming from the Trump campaign.

46:55But in reality, what we saw there was, again, it was a case of information poisoning. And now what we're seeing is that stretching into the future, where these are now the systems that are shaping AI and how AI will unfold over the next 10 to 20 years. And they're being manipulated by small groups of people who understand how the systems work.

47:22Coleman Hughes:So if Jews supposedly control the world and control the media, why can't they figure out how to fight back and control the narrative on Wikipedia? That's a great question. I have no idea what the answer is. I really, I don't know. I mean, you would think that if a side was being attacked and dominated so badly on such an important platform, you would think that there would be some kind of mounted resistance. I really wonder what the answer is. I don't know what it is, but it is. My job as a journalist, I feel, is to document what I see there. And again, coming back to earlier in the conversation, of course, I'm a person.

48:07I'm a journalist. I have my own interests. I have my own values. I approach this in my way. And I would invite someone else to do the same thing I'm doing from their point of view as well, which hopefully there are people doing it. But I think a lot of the answer to the question, though, if I kind of step back for a second, comes down to the culture on Wikipedia. This came out of the Bay Area for most of the site's life. It was in the Bay Area. It was a culture, a very, very progressive culture. I think it was more classically liberal and libertarian in the beginning of the site. And then over the years, it became progressively progressive.

48:49And now it's, you know, so much of the politics that you can feel in these socially, culturally, politically resonant entries or articles are left-wing politics. And that's what's dominating. And that tends not to favor Israel. It tends not to favor the idea of the Jews as a nation, more as something that is closer to like a religious thread in the quilt, more of an American view of things. So it's probably the culture.

49:21Coleman Hughes:Yeah, I mean, I actually tweeted about this years ago because I noticed that every conservative commentator, including very mainstream conservatives like Ben Shapiro, was named as a conservative in the first sentence of their Wikipedia article, which I have no problem with. Ben Shapiro is conservative proudly. Nothing wrong with describing him that way. But I would notice that the equivalent liberal was never described using the word liberal or progressive. So like Ezra Klein, who is like definitely liberal, I think he would absolutely cop to that proudly, as Ben Shapiro would with conservative, would be described just as a writer and journalist, right?

50:06Coleman Hughes:Yes. And I went through like 20 examples of people that I kind of picked people I thought of symmetrically as equally liberal, equally conservative. And 201, every single conservative was named as a conservative and every single liberal wasn't. And I thought this is actually very deep, profound bias because it's a little bit similar to the point that critical race theorists used to make about American society. they would say like, okay, you think of the default American, who are you picturing? I don't think you're picturing a Hispanic woman, like you're picturing a white guy, right? But you don't have to call him a white guy, right?

50:46Coleman Hughes:It's like you, whatever you think of as normal influences a lot of how you think and see the world. What is the default? Oh, well, the default intellectual is liberal. The default writer, the default journal is a liberal. conservatives need to be named because they're sort of a special case of weird, different, they have an ideology, right? Whereas the liberal journalist is just a journalist. He has no ideology. Therefore, it doesn't need to be included that he's a liberal in the first sentence. And of course, that is, that when the ideology is invisible, it becomes more powerful because you don't see it as such.

51:27Coleman Hughes:Yeah, exactly. It's the lens. It's the lens. The reason it's not seen because it's not an object apart from the site. That ideology of essentially, broadly speaking, the left is what Wikipedia sees the world through. And you can get a sense of what it would look like to change that lens if you go to Grokipedia. Because if you take the same article on Wikipedia and Grokipedia, they're going to look really different. And even if you disagree with what's written on Grokipedia and agree with what is on Wikipedia, you're still going to see that there are two ways to present the issue. And Wikipedia is not the way.

52:06It's not neutral. And this is, again, like the founding mythos of Wikipedia has to do with neutrality. I'm not sure that that's what the founders really were seeking, but that's now what it's become. It's that this is about neutral information. And that's not true. That in itself is not a neutral statement. We know it's not. And David Rosado, who is a really great machine learning scientist out in New Zealand, he did this study. It's kind of what got me into reporting on this stuff. He analyzed language on Wikipedia. And he showed that if you look at language and entries having to do with politicians in the United States who are on the right, they were more likely to be associated with negative sentiment.

52:52And if you looked at ones on the left, they were more likely to be associated with positive sentiment. That held true for House representatives, members of Senate, presidents, Supreme Court justices, and journalists as well. So this is all there. And I think probably by now, a lot of people recognize it. I think what we don't recognize as much is, A, what we've been talking about, how much this is actually affecting the systems that are shaping our future. And then if we back up and go the other direction, you were kind of touching on this earlier, what you're seeing is that Wikipedia, you made this point about academia, right?

53:32There's not a lot of scholarship about trans-Saharan slavery, and there's not a lot of scholarship about Korean slavery. There's a ton about transatlantic and Atlantic slavery. However, and you're seeing like Wikipedia is expressing the bias of academia, which is true. But what we also have to understand is that the two things are very meshed together. Like academia and academic papers are part of the lifeblood of Wikipedia. It's part of the sourcing infrastructure. But for academia, Wikipedia is a window into the world. It's where all these papers, all the Marxist stuff, all the critical theory stuff gets put into this gigantic shop front where otherwise it wouldn't.

54:16It just wouldn't interface with the world and with the internet in that way as kind of neutral claims that it does currently.

54:29Coleman Hughes:So, I mean, we've been heaping a lot of criticisms on Wikipedia, I think rightly so. But to play the other side of this, isn't it still true that Wikipedia is quite often better than the average so-called expert that might have an explainer in a magazine or an article? And judged not against the ideal of objectivity, but against the actual alternatives, how bad is Wikipedia? I really think it depends on what. You know, so if you're talking about like you're just super interested in like some 1960s Porsche that was really special and there's an article about it and it's going to give you all this great information and photos and like somebody spent their time doing that, like that is actually amazing.

55:18And that's great that that's there. Or if I want to like looking at my library and there's an author, I'm like, you know, Anthony Trollope, I want to read more. I want to learn more about this guy's biography. I will end up on Wikipedia. There's no question. The problem is that what is happening on the site today was never part of the original vision of how this would unfold. It was a much more naive time back then. We're talking about 25 years ago. In internet history, that's the entirety of the public internet, basically, like 2001 till today. and the site technically hasn't updated much, but much more importantly, the structure, the infrastructure, the culture, the way that they monitor the site, the way that they administer the site has not changed.

56:06And instead Wikimedia Foundation, the NGO that owns the site has, again, this is that 2016, 2017 period, re-imagined themselves consciously as a social justice movement where they consider now their core mission to be the pursuit of knowledge equity. That's their terminology. And for them to be part of what they call global knowledge infrastructure. So they are pushing a certain kind of idea of knowledge, one that's rooted in concepts like equity. And they promoted DEI. They spent a ton of money on DEI, at least$10 million or$9 million. dollars. What I'm saying is that, yes, all that value is there.

56:50All that value of these, let's say, six million plus pages or probably more, where you can learn things, where you find things you never thought you would find. But that last percent is doing so much damage. And the reason is because the oversight at Wikipedia has failed. Oversight mechanisms at every level, at the level of the foundation, the NGO, and at the level of the arbitration committee, that's their Supreme Court, there's 10 of them. There's 10 of these arbitrators who are active right now. For 7 million articles, how could that possibly work? There's 600 active admins for 7 million articles, 7 million.

57:35I mean, it's hard to wrap our minds around the scale of that. How can 600 people patrol 7 million pieces of rich information on which there could be dozens of arguments flaring at any one moment? You can't. It's impossible. So we are at a place in our society where not just technology has changed, our very society has changed because of technology. And this one critical piece of our, quote unquote, knowledge infrastructure has not changed at all. And again, when we think about Grokipedia and what Elon has done, why it's so interesting, is because humans are writing the source material, but a machine is curating and condensing it for us in a way that's attempting to be determined to be neutral in as much as it can be.

58:25Coleman Hughes:So I found that Grok is really good at fact-checking tweets. community notes, like Grok combined with community notes has been fantastic for the pursuit of truth and the fight against bullshit. First, tell me if you agree with that. And then second, tell me about Grokipedia. Overall, is Grokipedia better than Wikipedia? And if so, why? In terms of the grok at fact checking, and I think this really core modality of community notes on X, which is, I think it is in a way, it's not a massive game changer, but it's an important step forward, I would say. Having that ability for people to be fact checked in public, not by four people sitting in an office somewhere, you don't know who they are, what they're doing, but by a group of people who are actively doing this and voting, on the outcomes and that kind of thing.

59:23Probably will lead to better outcomes. Could also be manipulated, of course. But I think in the case of X, you have here a company that needs to succeed. It has to make money. It has to win the trust of advertisers. It has to balance competing needs. And I think that generally requires much more engagement by management, by their product people. There's been a lag, of course. And X is by far not perfect. There is a lot going wrong on X every single day, but it's improving. That's the thing. It's actually improving, and you can see they're making steps to improve it. Comparing that to Wikipedia, where we're talking about the stasis that I described before, I think it's better.

1:00:08As Grokipedia, the articles that I've seen that are important to me, that I've done side-by-sides with, are way better on Grokipedia. Way better. I did a lot of COVID reporting, COVID origin, the lab leak stuff. I reported on that story for a year and a half, two years. I know enough to know that lab leak is not a conspiracy theory. I know that it has serious scientific substance behind it, leading experts around the world who are convinced that it is likely that no one's saying that it's 100 % slam dunk. But if you go to the Wikipedia article, it's going to say the word conspiracy theory dozens of times and throughout the article.

1:00:49It's not even that long of an article. Is that still true to this day? That's still true. Yeah, yeah. And it's in the lead section as well. It relies – one of those really important first instances where it's claiming to be a conspiracy theory, the citation – again, back to the academia stuff. The citation is from a paper written by two professors of literature. Because they're writing about some sort of literary analysis of conspiracy theories, and this one, by literary standards, this fits. But that's not a scientific theory. That's a literary theory. And then you go to Grokipedia, and it treats it as a serious topic that is not being polarized.

1:01:30It's not being stigmatized through this terminology. And you're like, you know, I use this metaphor often, but it's kind of like coming into a, from a dim, dim, dank room into an open space. You're like, oh, okay, this is what this could look like. And you see that again and again and again, how this is going to play out over time. We still don't know. It's unknown. I think the important thing though, is that Wikipedia was all about crowdsourcing the information through individual editors. I think where we're going is we can start to crowdsource information from different pedia platforms, Grokopedia now being at least an alternative, at least there's a diversity of viewpoint among the different major encyclopedia platforms, which we have never had in the history of the internet.

1:02:13We've only had one. We've had a knowledge monopoly for 25 years. And in year 26, it was appropriate, I think, to have a bit more diversity. And I hope we'll see more and more of that kind of thing. I don't think Grokopedia needs to be the last. It can be the second in a string of who knows how many major encyclopedias that all get drawn from by various people and various systems.

1:02:37Coleman Hughes:Okay, Ashley Rainsburg, thanks so much for coming on my show. Thank you, Coleman.