SBJ Sports Media Podcast Bonus: Inside the NFL's View of Sports Media Measurement with Paul Ballew

0:07Hey everyone, Josh Carpenter here. We've got a bonus episode for you this week. Recently, Austin Karp sat down with NFL Senior Vice President of Data and Analytics Paul Ballew for a fascinating conversation on how data is shaping the league's future. The interview was so good, we wanted to bring you the full extended version. Here's Austin with the NFL's Data Chief. well this time last year right as big data was rolling out from nielsen the nfl was very vocal in saying that the league's audience was being underestimated here something that might be costing the league significantly in terms of viewers ad dollars future rights fees but the league did end up with its best regular season audience since 1989 with these new numbers now more measurement changes are coming from nielsen this season including co-viewing enhancements So to talk about how these changes are going to be impacting the Shield, we're bringing in NFL Chief Data and Analytics Officer Paul Ballou.

1:06Welcome into the Sports Media Podcast. Great to be here. All right, let's start right at the top. What's your view on the current state of media measurement? We're still on a journey. It's still a mixed bag. We certainly look at it, certainly dollars and cents, which matter. But we also look at it from an operational standpoint. Improving the accuracy of media measurement matters from that perspective as well. We want to optimize which games are on where. If we're missing something, we want to have those insights to adjust accordingly. It matters significantly for us. And when we look at where we're at, we've made some progress, as you mentioned.

1:40You think about things such as co-viewing last year's implementation of the out-of-home changes, broadening the sample. We've personally spent a lot of time with Nielsen improving quality as well. But on the other side, we still have very significant gaps in the measurement and the accuracy of the measurement. And we're about to go forward into this season with a lot of unknowns coming from Nielsen because besides co-viewing, they're implementing a series of changes into their measurement system that ourselves and the networks and others really struggle to get our arms around. So it creates a level of uncertainty and in some cases looks like a step backwards for Nielsen, which again just complicates the equation of everything we've been working on for the last couple of years.

2:20Well, Paul, you talk about accuracy. Why is this sort of accuracy, getting it right in media measurement, so crucial, particularly for live sports like the NFL, especially with you being the leader? Well, two reasons. Number one is it's essential to have accurate data for our financial model. A big part of our revenue comes from media rights, and we want to make sure that we have accurate information as our network partners and others want as well. But then secondly, as I mentioned just a moment ago, it's essential for us from an operations standpoint. We look at measurement not just at the top line, the 18.8 million viewers or the numbers on Thanksgiving.

2:57giving. We look at all the detail elements below it. Who's watching, demographic cuts, granular cuts, cuts in individual markets. And all of those elements are essential for us to ensure that we're broadening our reach. We're optimizing the ability for fans to consume our content. That's our goal. Our model is predicated on the fact that we have this great game and we want people to be able to see it and consume it and make sure we're doing it in a way that optimizes the fan and that equation and not having accurate measurement disrupts that as well as obviously it has ripple effects on the financial equation that the NFL and other sports leagues have with network partners.

3:34Paul, the NFL has really been outspoken on this topic. Why is it you feel that you really need to be out front on this? Because it's not something that you see leagues or properties usually getting into so publicly. And I'm talking about media measurement. Well, we think a number of factors are behind. First of all, we do have the ability to have our voice be listened to. We're an extremely important brand, very important element in the overall equation. Therefore, going on point, always bringing our network partners' perspectives into account, but going on point is a reflection of the fact that our voice will be listened to.

4:08So we never step away from the fact that we have a responsibility, we believe, and the fact that the shield affords us the opportunity. Secondly, the ongoing journey we've been on on media measurement has been so frustrating for so many people for so long that a few years ago we had a choice. We could have been on the sidelines and howling at windmills, or we could have jumped into this to try to systematically change the equation that was out there from a measurement perspective. And we chose to get into the tent and to invest time to see if we couldn't add into the journey everybody's on to try to improve measurement, including spending an inordinate amount of time with Nielsen.

4:48And the amount of time we spent with them is to help them throughout their processes, to provide our perspective, to even help them technically on a number of different fronts. So it's a combination of all of those elements. I don't want to make us sound like we're bucking for sainthood here. We're not going forward and saying we're trying to solve world hunger. It is essential to our business model, but it's the fact of who we are and what we can do to influence, both technically and our brand, that really prompted us to take a more active set of interests, voicing, however you want to describe it.

5:21We spent a lot of time on this topic. Well, I don't think anybody who has ever glanced at the list of the top 100, even top 200 most watched telecasts of the year on all of TV is going to argue with the NFL's importance. You guys just dominate those sorts of lists year in and year out. So NFL, clearly incredibly important. But, you know, drilling down a little further, we've talked a little bit about COVID. What other types of measurements are really important to the NFL here? So co-viewing has to be landed at some point in time, not just for the NFL, but other live sports and other large events as well.

5:51That is a deficiency in Nielsen's measurement. They would agree with that. They take an initial step this year, but it's only a partial step on co-viewing. It's only going to affect roughly a third of the audience that they'll be taking into account. They still have work to do over the next couple of years to adjust their co-viewing measurement, for instance, for out-of-home audiences. So there will be a significant push from us, a significant push from others to continue to factor in that. As the commissioner likes to say, who believes that only 2.4 million people in a viewing setting are watching the Super Bowl together?

6:23The numbers just don't pass logic tests and our own one-off surveys and so on have proven that there's an undercounting occurring. So co-viewing matters. The second part though for us with Nielsen and outside of Nielsen is the journey to move to transactional data instead of being panel only. And the world we live in with fragmented media, with exclusive streamers, with all of the other advents of technology, we need to systematically, with Nielsen and without Nielsen, determine how we're going to evolve the measurement from a data sourcing standpoint. And if you look at one of our biggest frustrations with Nielsen, it's been the inability for Nielsen to really advance beyond panel.

7:03Panel as a survey method is an archaic method for any of us in my field. We only rely on surveys when we can't rely on transactional or observational data at scale. So it's very important for us to crack that code, again, with Nielsen or without Nielsen. It is at that point where most other industries have, but in media measurement, we have it. Now, there's complexities and granularity and other things you have to deal with in terms of the nature of the currency, but it's time for us to do that. And then the third piece is to really continue to make sense of the fixes Nielsen is putting into the system here in two weeks.

7:39They're not aligned with ourselves. They're not aligned with the networks. They're not aligned with anybody who's trying to assess what those impacts are. But Nielsen's pressing forward very aggressively, and we're going to have to deal with the fallout associated with that, continue to make sense of it, and also try to help Nielsen make modifications post this rush to implement seven different fixes at one time. Is that also something you had to do with big data? Like when this first rolled out, it felt like there were things that you had to go back to them, you had to fix. And it sounds like Nielsen was receptive to that sort of thing, especially from their sports partners.

8:12Nielsen is on this curve of trying to figure out how they're going to be receptive and responsive versus their role of being the currency. and almost by definition of the currency, it causes a bit of gridlock in their operating processes because historically making change at Nielsen has been very, very difficult to do and takes a long period of time. And they're trying to figure that out and trying to sort it out. The challenge we have with them right now is, and what they're doing at the end of August, is they swung the pendulum almost all the way in the other direction. So three years ago to get them to implement out of home was very laborious and took time and validation and years to do.

8:51And now these fixes without validation, without impact data and alike, they're rushing to implement. And we know in sports, some of these changes will be detrimental and will add into less accurate viewership numbers, we believe. We're going to have to sort that out. So I'm empathetic that Nielsen's trying to figure out culturally this balancing act between I need to be responsive, but yet I'm the currency and therefore as the currency I can only move glacially. I get it. But in this case, we get a bit of a mess to clean up going into this season and a lot of work to do to make sense of it and explain to external audiences what it means and then continue to work with Nielsen while we're also evaluating alternatives outside of Nielsen.

9:31Well, it definitely feels like whack-a-mole for Nielsen in particular. I know they had issues with NASCAR and they were very public about issues they had with big data and so on and so forth. But I want to get back to the currency thing that you bring up. Where are we in terms of other currencies out there? You talked about VideoAmp at this time last year. I really haven't heard much about them or from them since that time period. Where else are we in the currency and other measurement sorts of companies out there? I would say we're in the experimental phase still. We're still learning. You have some networks that are trying to transact on other providers other than Nielsen.

10:04Early phases, though, I think for all of us, this experimental phase, though, is going to heat up and going to heat up very significantly because we're now at a point where the data is more readily available. We can harmonize the data, at least for streaming. And that part of the equation and as more activity goes behind that paywall, that creates an opportunity for us to bring data together and make sense. I would also say the frustrations with Nielsen over the last eight to nine months regarding where they're at, what they're doing, what they're not doing are at a pretty high level right now.

10:34And that becomes a forcing function for ourselves and others to look at alternatives that are out there. But it's going to take some time. There's a reason why Nielsen is the currency. They have this structure of 67 years of building this foundational set of capabilities. And Nielsen is struggling with modernizing. You want to find alternatives where it's appropriate to Nielsen, but it just takes some time. So some progress to really answer your question directly, but more work to be done. Paul, I love every time we chat, I always get smarter and learn more about the measurement side of things, which is so important to the media business, especially for the NFL.

11:10You guys being the beast out there in terms of all entertainment programming. But Paul Ballou, Chief Data and Analytics Officer at the NFL, really appreciate you coming on the SBJ Sports Media Podcast. Thank you. Always great to chat. Look forward to the upcoming season. Enjoy the games.