Dr. Joel Shapiro | Kellogg School | What an NFL Injury Analysis Can Teach Business About Resilience

0:00The CMO Confidential Podcast is a proud member of the I Hear Everything Podcast Network. Looking to launch or scale your podcast? I Hear Everything delivers podcast production, growth, and monetization solutions that transform your words into profit. Ready to give your brand a voice? Then visit IHearEverything.com. Welcome to CMO Confidential, the podcast that takes you inside the drama, decisions, and choices that go with being the head of marketing. Hosted by five-time CMO Mike Linton. Welcome marketers, advertisers, and those who love them to Chief Marketing Officer Confidential. CMO Confidential is a program that takes you inside the drama, the decisions, and the politics that go with being the head of marketing at any company in what is one of the most scrutinized jobs in the executive suite.

0:53I'm Mike Linton, the former Chief Marketing Officer of Best Buy, eBay, Farmers Insurance, and Ancestry.com, here today with my guest, Dr. Joel Shapiro. So today's topic, what an NFL injury analysis can teach business about resilience. Now, Joel is a professor at the Kellogg School of Business at Northwestern, where he has taught data and decision science for over a decade. He was also the chief analytics officer of a sales software company and has a degree in law. This is his second time on the show. Welcome back, Joel. Thanks for having me. All right. Before we dig into this case, let's remind our listeners about what you teach and why.

1:40Yeah, so I basically teach organizations how to be successful with data and AI. So it's sort of a combination of basic analytics to advanced data science, and now AI and large language models, all those tools. And across lots of different functions, lots of different industries, I teach them how do they actually leverage these things to get good stuff to happen. And, you know, I think with a keen focus on how to make smart investments in those kinds of people and tools and systems to get good business results. That's sort of my overall vibe. And why are you into this? Because this this is the kind of thing that's applicable in almost anything, politics, medicine.

2:17You know, why did you decide to go in this direction with this kind of teaching and and, you know, degree that you have? Yeah, well, I think, so I started out, my PhD is in policy analysis. And so I started out doing work in public policy and specifically around education, which I still feel very passionately about and I still care deeply about. Part of the problem for me was that I felt like I was doing a lot of work and writing a lot of reports that just sort of sat on a lot of shelves. And it was hard to feel like you were making a difference. And one of the real nice things about business is, you know, business moves, it moves fast.

2:54It moves fast, sorry. And I'm a pretty deliberate guy. And so working in business sort of keeps me moving fast. And I think that's a really nice sort of dynamic. And it makes sure that my work is good and timely and really has some real effect. And so when I leave your class, I am better armed to take data and AI and everything else and practically apply it to situations. Is that a good way to think about it? Yeah, I mean, you know, one of the things you're good at is finding the business problems that you can solve with data and AI, because that's always step one is what should we be working on?

3:32You know, let's not be data driven just to be data driven, but let's make sure that we are finding and articulating our important business problems. And then the data strategy and the analysis and the use of data flows from there. So I think that's a good way of putting it for sure. Excellent. All right, let's go with your recent study, which was just too interesting for me not to drill into it a little bit. The idea that predicting injury in pro sports has always been, as you call it, a failed use case that nobody can do it. And you decided, maybe I'm going to take a run at this. Tell us why you decided to look at pro sports and then how it relates to business.

4:14Yeah, sure. So look, the fact that when data first got hot, data science and prediction first got hot, you know, 10, 12, 15 years ago, something like that, people started talking about predicting injury in sport. Wouldn't that be cool if we could predict injury and all the things that a team could do better? And then for whatever reason, over the last handful of years, there's been, as you alluded to, this narrative that it's a failed use case. We just can't do it. And that's flat out wrong. Part of the reason that I know it's wrong is because I've built models and I've been advising a startup that builds models where you can predict injury.

4:49And I think sometimes where people get a little bit hung up is, you know, people will say, come on, you're telling me that you can point out exactly what body part and what game of the season a guy's going to get injured? No, that's much too extreme. But what's interesting is that we can find risk factors that just aren't otherwise obvious to it, right? The data tells us stories that our human mind simply can't see because the data is too complex. And sometimes the things that predict injury are pretty straightforward. Prior injury predicts injury. Age sometimes predicts injury, although there's sort of a survivorship bias when people get later in their careers, their pro sports careers.

5:31Maybe it means that they're less prone to injury, so they're less likely to get injury. But in general, the older you get, the more likely you are to be injured. And then weird things, too. It's like in the NFL, where you went to college can be a predictor of how likely you are to get injured in your first couple of years. And then you're like, is that because of the training program at the college they were in? Or maybe it's just that the college was selecting certain kinds of players, smaller, faster players, and they're more likely to get injured. Point is, there are a lot of things that kind of surprise us and that we can't see otherwise.

6:02And injury and risk factors are, in fact, predictable. And I know because we've predicted it. Yes, no one has been injured on CMO Confidential yet, though. So I feel pretty good. Not yet. Not yet. There's still time. So let's talk about the methodology on this and the NFL, why you picked the NFL and all the data. How did you go about tearing this apart? Yeah, so. You mean the model. Yeah, well, okay. So all good models rest on good data. And one of the best things for me about this project was that I was working with a company that had put a ton of labor into building a great data set. And so I myself didn't have to build it.

6:40And so what they did was they went, I'll explain why the NFL in just a second. But what they did was they went to a tremendous number of public sites and private sites and pay for sites and so forth and put together a spectacular data set over an 11 year period from about 2012 to 2023 of player level data that had injury data, all sorts of other information about the individual players and who they were playing for and the scope of injury and the nature of injury and number of games missed and so forth. Now, of course, I could take that. And as we'll talk about, I can relate it to team success because that's pretty easy data to get.

7:18The reason we did it on the NFL was because the NFL has very strict injury reporting protocols. And so the data is much cleaner than pretty much every other sport. And so good models rely on good data. And in this case, those strict injury reporting protocols were key. And it's really unique to the NFL. So this thing takes everything, including the college you played at, and I assume training regimens, all kinds of stuff, and you put it in your model, and it starts predicting things. And you create a metric here, I think, called cash wasted. Tell us about how the model takes all this out, and then what does it predict?

8:03And then we'll flip this to business, but the NFL part is just too interesting to leave. Yeah. So when we talk about predicting injury, you're exactly right. You take all this information that you can about a given player and its history and its training and its team. And are they playing on turf or a real grass or whatever else they're playing on? And you put all these kinds of things into a model and you can do a good job of predicting how many games they have left in the tank. Are they likely to get injured at a certain point in a season? where I got interested though was I thought to myself okay fine it looks like injury is reasonably predictable but how much does it actually matter to a team because we can tell ourselves a story that of course it should matter the more injuries the worse the team does and I felt like the very first thing that I had to do was establish does injury actually matter and if so how much can I ask one one question when you say it predicts injury so it would say like x percent of the team will get injured or on average you all have this like it predicts it like when when when the the model spits something out it says all right you know i'm the dallas cowboys or whatever here here's kind of my answer is it like what does it do and then then i want to go back to where you were going with with this answer yeah what it does is it says for any given person it tells us how many games they are likely to miss in the next season that's what it predicts So it's pretty specific.

9:27Yeah. And, you know, it's certainly not perfect, but what we see is that it's a whole lot better than, you know, sort of using intuition and gut. So it's a person specific, how many games are you likely to miss in the upcoming season? That's really the output that it spits out. So, and by the way, you can aggregate that over teams, of course, which is sort of a nice thing to be able to do. So I'm back to this question of do injuries actually matter? And I started talking to some executives across the NFL. And what I built was this metric called percent cash wasted. And I was trying to figure out a way to blend both injury and resource allocation.

10:06Because if a third string running back gets hurt, that's very different than your starting quarterback. And the best way that I could figure out how to build in resource allocation was simply to look at the amount of money expressed as a percentage of total payroll that a team spends on players while they're out due to injury. So a guy makes$10 million a year, misses 30 % of the season. That's$3 million cash wasted. Some people don't like the word wasted, but when I started to get pushback, I'm like, oh, I'm onto something because people don't like it, which is how I like to play. And so what I did was I said, does percent cash wasted matter relative to winning?

10:45And the answer is yes, it does. The more cash you waste, the worse you do. But what was interesting about it is that it was actually kind of a flatter relationship than a lot of people expected. In other words, if you get really injured, it doesn't impact winning as much as you might think. The average NFL team wastes about 10 % of their payroll. And in order to win another game, you'd have to get so much healthier that it almost feels impossible to, you know, sort of do anything meaningful to do it. So that whole notion of injury impacting winning is true, but all sorts of interesting implications because it's not as big an effect as a lot of people, including me, thought it was going to be.

11:33And when I look at the cash wasted thing, if there's a correlation to winning, by definition, does that say there's a correlation between the contracts and the pay that this is actually a good market value for how people get paid or not? Well, I don't know that I would take it that far. I feel comfortable simply saying that contract value tends to be a pretty good proxy for quality of player. Players get paid more if they're better and more valuable to a team. And so the fact that they end up, you know, they're being hurt hurts the team. I don't know that that does a lot to sort of, you know, sort of nail down market value.

12:19But I see where you're going with it. I don't feel super comfortable saying that. Okay. And so there's probably a lot of people that are interested in this type of research, right? So you have this conclusion, what do you do with it? And then what happens next when teams start taking this data and applying it to things that they do? Yeah. So one of the super interesting things that happened when I started sharing these results with with teams, and by the way, there's a little bit of nuance to this. If you waste cash on offense, that's more detrimental to your team than if you waste cash on defensive players.

12:55And so there's a little bit of, you know, sort of back and forth here with some nuance going on. But one of the executives in the NFL said to me, he goes, it's really interesting that there's not as big an impact as you thought there was going to be. He said, it's not as big as I thought either. I kind of feel like whenever we give a coach a break, we say, oh, they had a bad season, but they were injured. Maybe we give them too much of a break. like maybe we should hold them more responsible for a bad season even when they get injured and so different people are sort of looking at this and sort of taking away different from conclusions from it what i think it's really interesting is and where i tried to take this was can you be very injured and still be successful yeah right because that's sort of what i wanted to figure out like of the teams who are badly hurt can they still make the playoffs can they still do really well.

13:45And I got a little stuck there. And the reason was that disproportionately in my data, teams that were injured and did really well just happened to be the Kansas City Chiefs in a handful. And then you're like, so what's the secret? Having Pat Mahomes? Like, are you describing the Chiefs or are you describing a trend? It got a little bit funny. And so I couldn't really take a whole lot out of that. So I tried a slightly different tactic. What I said was this. Imagine that you are a team that in a given year, you're healthy and you do really well. You're healthy and you make the playoffs. What if the next year you're really injured?

14:22What happens to those teams? Does anyone stay good? Do they all do poorly? And this is where things got super interesting. teams that one year were healthy and good and the next year were injured about 25 percent of them stayed good yeah about 10 percent of them did a little bit worse and about 65 percent of them completely tanked and that became interesting to me because now i'm thinking to myself why do you have that separation yeah what's the variable in that and and you know i have looked at that a hundred different ways. And there's really only one big thing that I've come to the conclusion of.

15:04And that is if you have a highly paid quarterback and the quarterback gets badly injured, you are going to tank. No team has been able to withstand the loss of a highly paid quarterback and still have a good season. That's about the only thing in the data that I found. But that in and of itself is interesting because now I'm starting to think about things like leadership and CEOs and how do we plan for resilience more broadly? And that's sort of the connection that you asked about before, the connection to businesses. Injury for a team is unexpected. And trying to sort of fight through the unexpected bad stuff is incredibly important for any team, whether it's a business team or a sports team.

15:54And this notion of how do we actually plan to be resilient? Because I don't love the way that a lot of people talk about resilience. Resilience is a big buzzword right now. And when people talk about resilience, what they typically mean is that when things go bad, you fight your way through it. You've got the character and you got the moxie to fight your way through it. And I think that sort of undersells the concept of purposeful resilience. Like you want to build resilient organizations. So when things go bad, you can survive through them. And, you know, we can say that we build our teams for shocks, but what the real sort of secret is, what the ingredient is that allows us to build for shocks.

16:36I'm still trying to figure that out. And I wish I had that silver bullet, but that's where I am in the research right now is trying to figure out what exactly is it that makes teams resilient in sports and business alike. And do you have any like variables where you're thinking this feels like it's a good lead here? And, you know, and how companies recruit and staff and think about it? Yeah, so I have some things that I'm still sort of playing with. And I got to tell you that I'm very cautious about taking results from one context and extrapolating them to the other. There are some people who feel very free about sort of just making claims.

17:14And I'm always I always hold myself to a very high standard in this regard. But I will say something that I think is sort of fleshing out in the data is that in the NFL backups matter. Yeah. Having a good backup plan matters. The translation to business teams, I think, sort of speaks for itself. And teams that, you know, you can't nobody has the right resources to be able to have a highly paid quarterback, them go down and still have a good quarterback step up. But in all the other contexts, it seems to me that good, solid backups matter. And that is an important lesson in and of itself about resource deployment.

17:49I will tell you that I talked to a bunch of trainers in the NFL because you can imagine that if you could predict injury, one of the questions is, what are you going to do about it? And that's the same question in business. If you can predict that something bad is going to happen, what are you going to do about it? Right, the people are going to leave. You're going to get raided. A competitor is going to go. There's a lot of things that mess up with any annual plan almost immediately. And you see some companies always win, like some teams always manage to show resilience, and other companies don't.

18:20Yeah, and so in the NFL, it was interesting because I said to some trainers that I was talking to, imagine that I could tell you that a certain player was at higher risk of injury next year than you would have thought. What are you going to do differently? And every person to a T said, I don't know. Like if somebody is at higher risk of a soft tissue injury, we already do everything we know to keep players flexible and strong and so forth. And if you're talking about like a broken bone, well, I got nothing because accidents happen. And the only way you keep those things from happening is from keeping people off the playing field.

18:52You don't play them, which is kind of a ridiculous solution, of course. And so you can't really, according to these trainers, do a whole lot more to keep players healthy. What does that leave you with? you got to bring the right players on your team who are unlikely to get injured. And that's part of the roster construction process. When you think about spending your big dollars to bring the right players on, it's not just about productivity. It's not just about talent. It's about staying healthy and staying on the field because you can't be productive if you're not on the field. So flip this over to the business side, and let's talk about resilience in business because within this, there's a whole bunch of recruiting, longevity, you know, training is a different thing, but still training.

19:39Sal, there's lots of stuff. Flip this over and just, you know, opine on how our listeners should be thinking about this kind of data where they're sitting looking at their business orgs. Yeah. So I think it's an imperfect analogy to just think of roster construction as team construction, because the truth is, if you have a quarterback that plays half your games, they could still be valuable to you. if you have a product manager who shows up 50 % of the time, you're probably not going to keep probably not good. It's not, yeah, it's not, it's not a great analogy here, but so I think of resilience a little bit differently.

20:12I think about resilience as things like, you know, supply chain disruption. Have you built, do you have enough foresight, enough intelligence, predictive work, AI kinds of things to really be able to identify where you might see and when you might see some supply chain disruptions so that you can ensure that your processes are really set up to withstand those kinds of problems, right? It's not about the people like it is in football. It's about the business processes. Or, you know, I've done a lot of work with a lot of sales teams. And this is a little bit more of a staffing one, but sales teams who are really reliant on that one rainmaker, and that rainmaker leaves, and all of a sudden, they're kind of screwed, right?

20:56Their leadership questions, too. Apple has remained kind of a great company, even though Steve Jobs is no longer with them, right? And I don't know, I think about it in startup context, too. You got startups that are reliant on very easy, you know, free-flowing capital. What happens if it dries up? Are they resilient enough to still be able to move forward and capture new opportunities without that capital? So I like to think of resilience sort of writ large and the role of data and AI in helping us predict the bad stuff so that we can build these resilient teams in these resilient organizations.

21:32So if you don't mind, give our listeners a little primer on how they might think through from their seat, how they could apply what you just said about resilience is it's kind of like, it's forward thinking, it's looking at the pain points, it's thinking through things that can go wrong and the backup plans. Give our listeners kind of a little dose of what you might tell your class or your corporate clients, hey, here's how you should think about resilience from your chair. Yeah, so first I will do that, but let me just also point out that sometimes people think about resilience as like a back office, almost like a business continuity thing.

22:16And they're like, okay, that's fine. Resilience is fine. Belt and Suspectors. Nice, the engine blue wipes us out. We have to have a plan. Yeah, but it's just, you know, it's sort of a back office idea. And I think that's a mistake because I think resilience needs to be thought of really as a strategy unto itself. Like when things go bad, you wanna be able to stay aggressive. You wanna be able to seize opportunities. You wanna be able to grow when others can't. So look, the short of it is for any given business, you've identified your opportunities and you've identified some of the potential weaknesses and threats.

22:47What you wanna be able to do is you wanna have some sense of how likely some of those threats are to come to fruition. It's about having people who are really in tune with the marketplace. sometimes it's about having good data scientists, but at some point you've gotta be able to understand what risks are real, what the actual sort of magnitude of those risks are and how much harm they're going to do to us. I think we're really bad at thinking about the real potential harm when things go wrong and planning for it, because things don't always go wrong. So it becomes sort of a second nature to us.

23:22So I think it's really about having a good sense of truly what are the most important threats. Doing as good a job as you can is getting a sense of the likelihood of those things, whether that's with dedicated data teams or market experts, and then building in select places where if things go wrong, you're most likely to pay the biggest price. It's just a trade-off of the costs against the benefits. And then one of the things that's beneath the surface on all this is companies that actually are really truly resilient are going to perform in that top quartile a lot longer than other companies. You mentioned Apple.

24:00Are there any other companies or industries you would call out that say, good God, these people were super resilient or they weren't very resilient when you look out there at different things? Well, you know, once you hit a certain level of scale, it's easier to be resilient. You know, I think of, you know, a lot of tech companies, you know, Salesforce and those kinds of places that do a nice job. But part of that is scale. I guess resilience tends to be a little bit easier when you are selling goods and services that are more keep the lights on sorts of things. You're selling software that companies truly can't do without because they're just table stakes for simply existing.

24:43So some of those places almost have resilience a little bit more built in. So for those that are not keep the lights on kind of goods and services, there's a greater need to sort of look further into the future and make some better guesses and to plan for when things do go wrong, if they do go wrong. Yes. Okay. So you're looking at all this and, you know, I must ask when you look at the NFL, we're closing in on, you know, I guess we're what, 10 games or 11 games in 12 games in when this thing airs, any Superbowl picks or any interesting things leaping out to you when you look at your studies or i know you live in chicago so the bears you know snatching a lot of victories well i mean you know so i i found that i am about as bad as it gets at actually predicting winners in sports don't hold that against me when i start talking about resilience and predicting things going wrong i can predict injury but predicting winning i seem to be bad at But the Bears have had the weirdest seven and three start in the history of the NFL, as far as I'm concerned, playing some pretty mediocre football.

25:49So I'm going with simply it is their year. They're winning it all. I'll take all the heat from the people who think I'm crazy. But that's my predict. It's hard for me not to look at the data you are creating here on predictive modeling of percent cash wasted and think this isn't going to connect eventually to betting. Is that the kind of thing that's just inevitable, given big data and AI and the power of DraftKings and FanDuel? Or how do I think about all of that? Well, I think you're right. I mean, those two things are coming together very, very quickly. And the more data that we have, the more predictable things get.

26:33And therefore, the sort of the different dynamics in the betting systems and the bets that exist. You know, everybody has access to the same intelligence, of course. And bets tend to be more easily won or lost, I suppose, when there's asymmetry in that. So the fact that everybody has access to the same information means that maybe there won't be so much change. I just get nervous, you know, with a lot of the little bets. And we're seeing some things right now about, you know, some some collusion and people doing some cheating, you know, with pitches thrown and. Yeah. So, you know, that's sort of independent of the big data.

27:11Right. It's just that. But because you can bet in such granular, small scale kinds of things that it's easier and easier for athletes to sort of, you know, throw a game or throw a certain kind of pitch, I suppose, or whatever. it is. So that concerns me a little bit, but yes, you're right. The prevalence of better data, better intelligence, better prediction certainly changes the way that betting occurs. Kind of an inevitable march of big data on yet another front. Yeah, totally. All right. So this brings us to our last question. Practical advice for our audience we haven't discussed yet and or the funniest story you can share on the air.