Is the Rotten Tomatoes Formula Fresh or Rotten?

0:00Hey, it's Danny Kelly, and it's officially fantasy football season, which means the Ringer fantasy football show is back with the latest news from around the NFL and everything you need to get ready for the fantasy football season. So join us at the Ringer Fantasy Football Show on Spotify or on our new YouTube channel.

0:40dinner, and drinks. Whether you're traveling for work or getting away for a minute, it's comfort that hits different. So yeah, Holiday Inn. It's a new day and a new stay. Book your next day at HolidayInn.com. This episode is brought to you by Accenture. When your advertising operations fall out of sync, campaigns slow down, insights get buried, and opportunities get missed. That's why Spotify and Accenture are working together to reinvent the rhythm of ad sales. Using automation, analytics, and smarter workflows to simplify campaign delivery and access better data across the business. The result?

1:15Less time spent on operations, more time connecting brands with the moments and fandoms that matter most. To learn more, check out Accenture.com slash Spotify. It is Wednesday, September 17th. A generation ago, if I asked you who is the most influential, critical voice in movies. Probably say Siskel and Ebert, A.O. Scott of the New York Times, maybe Ken Turan of the LA Times. They could really help movies they liked, and the dreaded thumbs way down would be a death penalty for a certain kind of movie. Today, no critic really matters. Not really. The internet and social media has unleashed a torrent of opinions on movies and TV and everything else, even podcasts.

1:57And in place of individual critics, the single most influential force in movie criticism is now Rotten Tomatoes, the aggregation site that deems good movies certified fresh via their critics score. But Rotten Tomatoes is kind of controversial in Hollywood. Filmmakers hate it because it's binary. It deems each review either positive or negative, not allowing much room for nuances like its rival Metacritic does when assigning a numbered score. And increasingly, there are a whole bunch of random voices included amongst those more established critics. Rotten Tomatoes, which most people don't know, is actually owned by NBC Universal with an investment from Warner Brothers.

2:37It's actually part of the new spinoff company called Versant that's happening. They say that they broadened out the pool of critics to address criticism that the site was too elitist, too white, too male, etc. But as I wrote in my Puck newsletter a couple of weeks ago, the result has been that the Tomatometer scores have gone up about 13 % over the past decade. Yes, great inflation on Rotten Tomatoes. So many quote-unquote critics are just film junket media or awards pundits trying to get access to stars or who knows who these people are. The end result, in my opinion, is that the most influential platform for advice on which movie to see has become almost unusable, at least for discerning people.

3:19That's my opinion, of course, but I wanted to have someone on the show from Rotten Tomatoes to defend the site and fight me and explain what their methodology actually is. Thankfully, Jacqueline Coley has agreed to do just that. So we had her on for a fun debate about Rotten Tomatoes, the state of movie criticism, and is it all going into the social media toilet? From The Ringer and Puck, I'm Matt Felony, and this is The Town.

3:48Okay, we are here with Jacqueline Coley, who is the awards editor and other stuff in her title at Rotten Tomatoes. Welcome, Jacqueline. I appreciate you coming on. I really appreciate you being here, Matt. First of all, I want to say thank you so much for giving us an opportunity because we talked about it before. There's a lot of misinformation about Rotten Tomatoes, so I'm really excited to tell folks about our product, tell folks about our brand. And fight me. And fight me about it. No! No! No, that's fine. No, no. This is going to be a friendly debate. I wrote a piece in my puck newsletter a couple weeks ago that was very critical of Rotten Tomatoes.

4:26I basically said that it has become unusable, that it has been flooded by awards, sycophants and random people. And the tomato meter does not reflect what people are actually thinking about the movies. and that the great inflation that's gone on over the past decade reflects the overall tenor of the critical community, which now wants to suck up to talent rather than inform the people who might want to go see these movies. And I was walking around Toronto. I was at the film festival and people who had read it said, you got to have someone from Rotten Tomatoes on and I appreciate you being that person.

5:08So let's get right into it. I think there's a lot of misinformation out there, or at least a lot of misunderstanding about how Rotten Tomatoes works, like how reviews are aggregated, who decides what is positive or negative. Is it a person? Is it AI? How the blurbs are decided, how someone becomes a tomato meter approved critic, a top critic, all of these things. So explain how Rotten Tomatoes works. Okay. Well, first let me say what Rotten Tomatoes is. So one of the things that I found to be a very big truth in your article, which is that we're the most trusted source for review aggregation. You are.

5:47I admit that. And you're in all the commercials. The studios use the certified fresh like they used to use the two thumbs up from Siskel and Ebert. Absolutely. Let's start with our critic community. So we have about 3 ,000 critics that together equal our entire critic community. This includes about 1 ,200 individually approved critics and about 1 ,800 outlets. And that group is what makes up one of our products, which is the tomato meter. The tomato meter is that number you see on our site that reflects the critical community and it gives a temperature gauge of what that community thinks about a film as far as recommendation.

6:23So first and foremost, that tomato meter is expressed in three different ways. If by chance a film gets more than 60 % of that critical community that recommend it, it gets a fresh distinction. That's our fresh badge. if the critical community above 75 % and also some other stipulations around top critics and some other stuff are met, then it will receive our highest designation on the tomato meter, which is certified fresh, which is the thing you love seeing on all the advertisements in our product. We're very proud of. If it's below 60%, it gets the rot. The splatter, the dreaded splatter. So that is the tomato meter.

6:57Now, how those folks are credentialed is we have a critic relations team. And this is comprised of the folks that once a year, they open up our application process, folks can apply as either outlets or individual critics. If an outlet is approved, every critic, while they're writing for that outlet, their reviews will appear on the tomato meter. For those folks that apply to be a part of the tomato meter as individuals, no matter where they write, those individual reviews will show up. And then we also have our top critics community. And we did actually a refresh on that community a few years ago in which we added about 100 or so critics.

7:33But the one thing I will tell you is that critical community is very similar to the critical community of your favorite platform. You mean Metacritic? Yes. They're not my favorite. I just think that they better gauge what the sentiment is than the binary system that Rotten Tomatoes uses, where there's some designation of positive or negative without any nuance. I mean, that's true. But the one thing I will tell you is I'll put our product to the Pepsi challenge up against any other product that is out there. And I think that we stand up because one of the things about this is when we express this tomato meter, we have it as the score.

8:08Sometimes we can have it as our critics' consensus have it expressed. But we also give folks the opportunity, if they want to, to dive in and read every single review. We link out to all those individual websites so they can read all the reviews. But I'm curious how these people are chosen. Is there a high court of Rotten Tomatoes where people are looking at their body of work and saying, OK, this person can be a tomato meter critic or a top critic? Or is it just you apply, there's a stamp and all of a sudden you get to do it? No, we have a very robust criteria. In fact, our criteria is up on our website.

8:46If you want to know every single bit of the criteria that these folks have to meet to be credentialed is a commitment to journalism. We read their reviews. we look at their history of reviewing not just one so like if somebody only reviewed horror movies that would be great but they wouldn't necessarily be someone we want to see folks that are reviewing a wide vast of films and also that the criticism is just that it is criticism of the work that there's not personal biases in there and no they're credentialed at a high level i would just tell you the folks that work at rotten tomatoes they read hundreds of thousands of reviews.

9:21Our curation team curates over nine... How large is that team? It's about eight folks that are part of our critic curation team. Okay. So eight people read thousands of reviews and decide who gets to be among the elite Tomatometer critics. No, I was saying that those folks are a part of our credentialing team. Okay. And that's what I was also going to say too, though, is that those folks are the folks that we also include in that process. And there is some kind of a designation that you are deemed worthy. And you recently extended that or expanded the criteria. Why did you do that? Was that because of the criticisms that it was too old white male?

10:06Or was it that you wanted to bring in different types of outlets so it wasn't as elitist? Why did you do that? what we did when we did our critical refresh in 2018, that refresh where we decided to credential individuals more so than outlets was actually us tweaking our products so that it can reflect the changing media landscape. Because what we were seeing, which is something that you also point out in the article, is that papers and outlets were sending less traditional critics to these places. We had more freelancers. And so we had to change our product to reflect that change. So the why is that?

10:43Well, and more and more outlets are just doing away with their critics. Chicago Tribune fired Michael Phillips. And they said is because they don't see a value in having a critic anymore. And partially because people are looking to Rotten Tomatoes. They're not looking to Michael Phillips. And Vandy Fair just got rid of theirs. Alan Steppenwald just got fired at Rolling Stone. Like the amount of critics is dwindling or serious critics is dwindling, I think because of the rise of social media and aggregation sites like yours. That's where I'm going to beg to disagree because you're talking about things that are corollary and you're trying to say that there is cause behind it.

11:19Really what has happened is the reason why those editors made that decision has very little to do, I think, with Rotten Tomatoes. It has to do with economic shifts within the market. Sure. Social media has killed all media and they are now feeling the pressure. I get that. I don't think that you have a particular issue with aggregation because there are tons of folks that do review aggregation, both in the audience space as well as in the critical space. I'll take, for example, Ty Burr, who worked at the Washington Post for a very long time. That position was eliminated and he goes on to Substack.

11:51Now he has his own website. That shift happened regardless of Rotten Tomatoes. Our job as a consumer product was because we know that this is a critical voice that we want to continue to have reflected in our audience, is to make sure that our product is available for that to be demonstrated because he still has an audience. He still has people that want to see that critical voice. And if they want to read his reviews, they still absolutely can. But if they're trying to make a decision and they want an efficient way to do it, whether it is that they want to engage in it on a shallower level, like just the score, the critics' consensus, or if they want to dive in deep and consume everything.

12:28There are lots of places now to find film criticism. I would say that the Ringer podcast, the big picture is a perfect example. They are critics that are talking about movies and are incredibly influential because they have a big audience, but they are not attached to the traditional critical mainstream. And now they are the mainstream. And how do you account for that on Rotten Tomatoes? Well, I mean, that's why we have our credentialing process. That's why we continue to look for new voices. That is one of the reasons why we are constantly making sure that folks that remain part of our critical community maintain that standard.

13:05We don't just say once you're in, you're in. There's an evaluation process to make sure folks are keeping up with that standard so that they can stay on the tomato meter. And my argument was that basically there's a lot of awards shilling going on. There's a lot of people with awards in the title of their publication or their sub stack. I mean, I use the example of Pete Hammond just because he's the one at Deadline that everybody knows. He's out there shilling during awards season for various movies. And then he also calls himself a critic for the purpose of Rotten Tomatoes. And you guys accept that and link to him.

13:40Well, first of all, I will just say I love Pete Hammond. He's one of my favorite people to talk movies with. And I think he is a credit to film criticism in the film conversation. Oh, you do. Okay. We can differ on that, but whatever. He's good for a blurb for the studios. So that's, he will always have value. I think your question should be with the editor-in-chief of Deadline. Because for us, we've credentialed Deadline and Pete is their critic. We are not in the business of telling people who they should put in, like that's not what we're about. Like that we are in the review and discovery business.

14:13We're in the business of helping people find, discover a film, let it be their next great watch, talk about it online and find something similar next time. So if an outlet like Deadline wants to designate an advertising salesperson as their critic, you guys will just accept it and be okay. That is not what I said. We have a very strict code of conduct. And if an outlet was found to be in violation of that, for example, that kind of thing that you just said, that would be a violation of our code of conduct because those people wouldn't be speaking on the films unbiasedly. I want to move on to you explaining the binary scale, because an 85 % fresh on Rotten Tomatoes does not mean that the average score was 85%.

14:58It means that 85 % of the critics on the tomato meter gave it what you guys determined is a thumbs up, right? So it generally rewards movies that are fine, that are okay. it does not necessarily reward the exceptional movies. It just kind of blends it out, in my opinion. A little gentle pushback on that. That is the metric that would work for you as a consumer. Not everybody wants to see certified fresh baddies. Some of them would rather see comedies that have a really high audience score and they're completely like, don't care. They ignore the tomato meter altogether. And that is why they use this.

15:39I think the best thing about Rotten Tomatoes is we kind of think of ourselves like Burger King. You can have it whichever way you want. If you're a person that values a curated group of critics from established outlets, we've got top critics. If you're a person that values seeing folks that have bought a ticket to a film so that you can find what they thought about the film before you buy a ticket, we have verified audience. If you want to see what the entire critical community of both new media critics, established critics, and international voices have to say about a film, that's our tomato meter.

16:10But why do you do that? Why is that the rationale for the tomato meter? Like, if you are having a real live person look at each review and determine whether it's positive or negative, why not just have them give it a score like Metacritic? Because we're not Metacritic. Okay. But you see how the criticisms will come there because it necessarily inflates the number, right? Because if you're looking at a movie, at a criticism of a Marvel movie, and it's like, eh, it's fine for a Marvel movie, whatever. That's positive. All of a sudden you look and Ant-Man or whatever is getting a 90 % certified fresh on Rotten Tomatoes, which is then splashed all over social media, which is then splashed on all the advertising campaigns, and it leads the consumer to think that this Marvel movie is something extraordinary when you then go to see it and it's like whatever.

17:02And people in the industry that reached out to me were like, we are doing a disservice to our customers. We are over-inflating the quality of these movies so they don't actually know when they're seeing a good movie. Let me start with first your assumption. The first part that you were stating about why we don't do others, this is Pepsi and Coke. it sounds to me like you want to change our product to be a different product than what we have. I get that. I totally get that. It's not what I want. It's what you guys have. And I'm and I have an opinion. Okay. But again, you were in the minority opinion because the market share the volume, the number of people who value Rotten Tomatoes is by far more in the positive standpoint.

17:44Now to bring it to your other point, again, I just think that inherently, there's a little bit of disrespect towards the audience. The audience is discerning. They know what they like and what they like may not be the same as say a person of a certain age from an established outlet. I mean, for example, we have moviegoers now that are a part of this younger generation where they don't care at all about those film critics. No, most people don't. They do, but the film critic they trust is somebody like Chris Stuckman. It's a YouTuber or a TikToker. And I don't think we should be in the business of telling those people that that is not what they should be listening to.

18:20I think we're in the business of making sure that our product can help them along that way if they choose to use us or not.

18:28At The Home Depot, get up to 15 % off all installed carpet projects for a limited time. Featuring brands like LifeProof, LifeProof with PetProof Technology, Home Decorators Collection, and Traffic Master. Take your pick of carpet built for real life and designed for real comfort. Plus, with installations starting as low as 49 cents per square foot and a free measure to get you started. We'll handle the hard parts for you. Offer valid July 16th, 2026 through August 2nd, 2026. Exclusion supply. For licenses, see homedepot.com slash license numbers. Class it up with Crocs. You know back to school is coming in fast.

19:04So why wait to find your new fave footwear? Step into a local Crocs store and step into your new look. Try it. Style it. Make it yours. Because the right pair doesn't just show up. It shows off. First day fits, handled. Walk out ready for whatever's next. Visit your nearest croc store today. I want to move on to the whole gaming issue and why some studios believe that they can kind of game Rotten Tomatoes. There was an investigation that Vulture and New York Magazine did a couple years ago where there are PR firms that believe that they could flag a review that they believed was inaccurate and get it changed and that they could kind of flood the zone with critics that were friendlier to their movies and get them past whatever firewall you guys have.

20:03So what does a studio do if they believe that there is an error or a review has been improperly categorized as negative? Okay, so I'm going to take the what happens when there's a question about a review. But first, I just want to correct a little bit of what you were stating. As far as what was written in the Vulture article in regards to Bunker 15, we were the victim of a crime. We were a victim of somebody violating our code of conduct as a way to justify their own selfish ends around a movie. And the underlying I would say about that article in particular is when we found out the malfeasance was happening, we conducted an investigation and we removed the malicious parties from our site for violating our code of conduct pretty immediately.

20:44More importantly, the underline of that whole affair that I found to be the most sort of like heartbreaking about it was the critics. To hear that critics today, because they are so desperate and because the landscape of being able to earn money and criticism is so dire that they were mortgaging and selling their integrity for less than what I made for my first review, that's the tragedy. That's the heartbreak. And it taught us that there's things that we weren't aware of. Okay. So Joe Blow, studio publicist, sees that a review that they think was pretty good was categorized as negative on Rotten Tomatoes.

21:21What does Joe Blow publicist do? So first of all, for the most part, I don't think to your quote, this publicist is involved in it. Let me tell you how reviews appear and then we can go in there where you see if they're involved. We have 60 % of our reviews that are submitted by self-submitters. That means the individual critic goes on the back end of our site, they load their review, they decide the blurb, and they give the recommendation of Fresh or Rotten for the film. Oh, they do? Okay. Yeah. 60 % of our reviews of that nearly 90 ,000 that we do every year, 60 % of those are done by the individual critics.

21:53We don't have anything to do with it. The other 40 % are curated by our curation team. That's where an individual person reads every single review. They put a fresh or rotten recommendation tag on. If after that point, there is a question, either from the self-submitter or from the curated review, and again, the studio has no idea which is which, then it is sent to a three-person curator panel, meaning three of the curators. I love it. Then three of those critics read the review and decide. If there's a plurality still amongst that three critic panel, then we get our head of curation involved and they weigh in on it.

22:32If there is still a question as to whether or not a review is fresh or rotten on occasion and very rare circumstances, we will reach out to the individual critic if they were not a self-submitter and ask them what their sentiments are. And if they were self-submitter, we will ask them if by chance it was done by mistake. That's it. Interesting. I love it. It's ultimately it's like Gladiator. Joaquin Phoenix is up there with his thumb up or down. Yeah. And so I just I'm like, where is this studio publicist? I think the only involvement that I see, and I will say this anecdotally, is occasionally I will see a publicist that says, hey, I think we're missing a review on our site.

23:09Can you make sure this gets added? And when that happens, we don't just add the review. The curation team sweeps the entire film to make sure that there weren't any other misses. Yeah, and ultimately it does come down to relationships that the studios have with individual critics, which I know you cannot control. But if the critics know that the studios are watching and if they have negative reviews on Rotten Tomatoes, that they're not going to get invited to the, you know, whatever screening and they're not with a publication that the studio sort of has to care about. That's where you can see sort of self inflation going on.

23:47I know that's not your fault. But that is an area where I know the studios believe they can gain the system is through the relationships with vulnerable critics. This is one of those situations that happens with Rotten Tomatoes. This feels like industry fan fiction. It's not fan fiction. I know those people and I know the publicists who say, oh, yeah, we've got our people. Lots of folks have big feelings and have a lot of, I would say, ideas around what exactly happens with Rotten Tomatoes. But I think oftentimes, because again, the average film receives over 300 reviews for a wide release. And the amount of gaming you would have to do to even make, it would take over 20 positive reviews that you kind of like guarantee just to move the score five points.

24:33But you know where it might be more beneficial is in the early reviews. And this is where during awards season, we see the gaming. Let's use the Springsteen movie as an example, where that movie got one screening at the Telluride Film Festival, where it was stacked with awards people. And these are people that are going to be covering and interviewing stars throughout the season. And they're at Telluride to get a first look at these movies. And the studio, Disney in this case, knows this. They had Bruce freaking Springsteen in the room watching the movie with these people. and lo and behold, after that movie debuts, it's got a 100 % on Rotten Tomatoes from 13 reviews from people that are largely awards people, some critics, but largely awards people.