Why Data-Driven Is a Trap

0:00Jason:I hate meetings. Hate them. Which means everyone around me does everything possible to avoid forcing me into a meeting, which I really appreciate. So when I do attend a meeting, it needs all my attention and my best note-taking skills. That's where Granola comes in. It makes sure that every meeting is more effective because you can remember exactly what it was that everyone agreed to, sparing you from the dreaded repeat meeting on the exact same topic. Granola is an AI-powered notepad built for the way that real people actually meet. Here's how it works. You take rough notes like you would in any meeting, and then in the background, Granola is securely transcribing that meeting.

0:40Jason:Then it turns everything into clean, structured, actually useful notes when the meeting ends. And the best part? Granola works through your device's audio, which means it integrates seamlessly into the video conferencing tools you already use. No setup. No awkward bots. It's just your normal meeting with superpowers. You get to actually listen instead of frantically typing every word and still walk away knowing exactly what was decided, who's doing what, and what comes next. If meetings are eating up your day, Granola is a no-brainer. You can try it totally free. Just head to granola.ai slash problem solvers.

1:17Jason:That is granola.ai slash problem solvers to get your time back. Try it for free at granola.ai slash problem solvers. When it comes to launching a business, we talk a lot about having the big idea, the vision. But you know, there's more to it than that. Like setting up your website, your phone number, managing public records, staying on top of compliance. It's the unglamorous stuff that can be a lot to manage. And that is where Northwest Registered Agent can help. You don't just form a business. You start with a complete foundation built for privacy, credibility, and growth. Northwest Registered Agent has been helping small business owners and entrepreneurs launch and grow businesses for nearly 30 years.

2:07Jason:They are the largest registered agent and LLC service in the U.S. with over 1 ,500 corporate guides who are real people who know your local laws and can help you and your business every step of the way. Don't pay hundreds or thousands of dollars for what you can get from Northwest for free. Visit northwestregisteredagent.com slash solversfree and start using free resources to build something amazing. Get more with Northwest Registered Agent at northwestregisteredagent.com slash solversfree. How do you know what to trust? Yeah, I think my answer would be you don't. You don't know what to trust.

2:50So this is all about the way you treat the data. If you say, I want the data to be this magic eight ball and I shake the data up, you know, and out comes yes, no, maybe try again. That is exactly where you would get stuck, right? Because you say, okay, the data is showing me the outlier. Now, what do I do with that?

3:07Jason:Running a business means solving problems. I tell you how the smartest entrepreneurs do it. Hi, I'm Jason Pfeiffer, editor-in-chief of Entrepreneur Magazine, and this is Problem Solvers.

3:22Jason:What makes a hit song? You know, like what makes a song that everyone wants to listen to that's on the radio all the time that gets billions of streams? And OK, hold on. Hold on. This is not a pop culture podcast. So not the kind of question that I usually answer or ask or whatever here. But stick with me because this has real world business implications. the answer to this question could be a mistake that you are making in your business. Because here's the thing, you know, record labels have forever wondered what makes for a hit song. And back in the 90s, a bunch of people had this idea, which was, well, why don't we use data to answer this question?

4:02Jason:Why don't we evaluate the data, the factors of every hit song and extract out that information and then craft the best song, the data-driven best song. And they did this. Do you remember this? I kind of vaguely remember this from decades ago. And the answer is that what it produced was completely boring. It was stuck inside of what made other songs great instead of having the freedom to be great itself. And the reason that I'm telling you this story is because this comes out of a book, this anecdote, comes out of a book called Data Inspired, Building an Organizational Culture of Inquiry for Lasting Transformation.

4:47Jason:And the author of this book, Sebastian Wernicke, is with me today because we are going to have a conversation about how to use data in the right way. The thing is that so many of us, when we're building our businesses and we know the data is available, why don't we use the data? We'll understand the customer will understand the market better. And then we use this data in a way that doesn't actually create the opportunity for innovation, but instead just traps us inside of what we already know, instead of giving us the freedom to figure out what people need that maybe they don't know well enough to have ever provided the data signals to begin with.

5:27Jason:Sebastian, welcome to Problem Solvers. Thanks for being here. Hey, Jason. Great to be here as well. Yeah. Okay. So, Sebastian, Did I articulate that correctly, that companies in their quest to use data properly end up trapping themselves inside of, let's say, knowable information without giving themselves a way to actually explore beyond it? Yeah, absolutely. I think for many companies and many leaders, data is a source for comfort, which I completely emphasize, right? So, I mean, you have to make all of these hard decisions. You are in a dynamic market. And you just want to be right. And so it's quite natural to say, let's turn to data to do that.

6:09And let's have as the ultimate goal to become a data-driven company. So data will help us know all the answers. It will help us make the best decisions possible. Data will help us get rid of the gut feelings of the politics. So everything that annoys us about our organizations today, once we just get the right data to the right people at the right time, magic will happen and all of that will go away. Now, this may almost sound like I'm against data. And of course, I'm not. I mean, data is really, really valuable. But the problem is, just like with that music example that you described, is if you're looking to data to give you all the answers and all you want to do is follow the data, you're running into that trap.

6:52You're not being unsuccessful, but you're becoming very good at staying the same. And of course, over time, that's not going to give you the big successes that you're aiming for.

7:04Jason:Yeah, it's funny because that thing you said there, being a data-driven company, that's what every company wants to be, right? I mean, everybody talks about being a data-driven company. And data, it is comfort, but it is also showing us that we're taking seriously the signals that we're getting, that we're understanding our consumer better, that we're understanding our market better. And so, Sebastian, how do we start to untangle this knot where we want the data, the data can be really useful, but we can also get trapped inside of it? Yeah, I think there's two different ways of looking at data as a tool.

7:39So one way to use data is for optimization, which essentially means I already know what I'm doing and now I want to do it better. So a very famous example comes from Google, where they about 15 years ago decided not to use a designer to choose the color of blue that they have on their website, because they wanted to optimize for clicks. But they basically said, let's try, I don't know, 40, 41, 50 shades of blue in the end to see which one of these blues will get us the most clicks. They run that experiment. So a very large A-B test, as you would call it. And they have the best color, right? This is the color that gets us the most clicks.

8:17And this is using data to optimize. And it's going to give you these 1%, 2%, 5 % incremental improvements, but it's not going to get you to do something differently. You will be staying within the processes, within the designs, within the products that you already have. And the other way to use data, but it's a way that data is not used as commonly, is if you're looking for innovation and transformation. And that requires you to have this completely different approach to data where you say, I'm not looking to data to make my life easier and get all of the complexity away, but rather I want to use data to see the full complexity around me, to see this rich picture of how customers are using my product, to see the full gruesome picture of our supply chain and all the things that go wrong, all the things that have to be rebalanced.

9:12and it's very important I think when you want to use data to be clear are we in this optimization mode which I would clearly call data driven right data is all the answers data is going to drive us forward or are you in this other mode which I like to call data inspired so are you using data to show you the next thing and in order to do that you then start looking at the data itself quite differently. So in a data-driven approach, you're looking for that one optimum direction. Whereas if you use it for that inspiration, what you start to do is you look at the outliers, you look at the anomalies.

9:51So you get very curious, for example, when you find a customer using your product in a way that is wrong, right? Where you say, this was never intended to be used that way. But instead of saying, ah, that's an outlier, we just want to know where to go. You say, okay, I will now get intensely curious about that. I want to understand what are they doing? Why are they doing that? And could that be the hint of the next evolution of where the product is supposed to go? Are they maybe using that feature that was just a throwaway feature for us, but it's showing us there might actually be a market we had never thought of?

10:25These are the ways that data can then drive innovation.

10:29Jason:That's really interesting. Do you have a real world example of someone you've worked with that did a version of that, looked for that outlier, and it turned out to be a great opportunity. Propel Fitness Water with Gatorade electrolytes, zero sugar, and vitamins. Propel hydrates better than water to help you get the most out of your workout and get back to your best self. What propels you? Propel with Gatorade electrolytes. Labor Day savings are happening now at the Home Depot with select appliances starting at$399. Plus, save up to an extra$1 ,000 and and get free delivery on appliance purchases of$998 or more.

11:07Jason:Get a Whirlpool laundry tower featuring industry-first UV clean technology designed to reduce bacteria in the wash without fading fabrics. Plus, with great prices at the Home Depot, you can save on select appliances designed to make laundry day easier. Shop Labor Day savings at the Home Depot today. Offer valid August 27th through September 16th. USLAC store online for details. Yeah, so I once had this wonderful project with a sports car company that was trying to figure out what are the configurations that we should build into our cars. So the way it usually works when you buy one of these premium cars is you go to the dealership and you say, this is how I want my car to be configured.

11:43And then they say, okay, yeah, we can build that for you. They charge you a lot of money and come back in six months, by the way, because we need to build that thing. And so they had the idea of saying, well, why don't we just use the data to predict the configurations of the car beforehand? So you go into the dealership and you say, I have this really special car in mind. And then the dealer says, well, it's standing right here. It's a little more expensive. That's how they make money with this. But you can take it away right now. And of course, that's a genius idea. It also worked very well.

12:15But the data showed us some super unintuitive configurations. For example, when we analyzed the data, it was really honing in on a weird type of stereo setup within the car. where the actual sales of that at the time were below 1%. So everybody was just convinced nobody wants that. This is an exotic thing. It was kind of over the top. And you would think, well, a premium customer, maybe they want something a bit more, I don't know, suitable for classical music than the over the top stereo. But the data was showing, no, no, build that into, I'm going to make up the numbers here for confidentiality, but 20 % of cars, build that in.

12:54put that and they did turns out everybody loved it and they wanted it right so the stereo that everybody thought oh it's this niche thing the data was actually telling them no no no no this is what delights the customers build more of that and we found various other examples where it was just very unintuitive configurations of a car that completely disrupted the assumptions that everybody had about their customers and what they liked and the way that they behaved because they just observed them the way that they were behaving in the real world, right? So the data that they were getting was telling them only few buy this type of stereo, only few buy this type of, let's say, you know, rim for the wheel.

13:37So that must be what they want. But if you look at the data in the right way, it actually turns out, no, no, no, you can build something much more exciting here. But that was all about listening to the outliers and the anomalies that we saw, because it was small percentages from the real data.

13:53Jason:So let's go a level deeper here, because what I'm thinking as you're telling me this story is, okay, that's really interesting, but how do I know which outlier to trust? The idea of the outlier is super interesting, but of course the outlier could also be just some weird person who has some weird interest that nobody else has. And I don't know if really anybody else is going to care about that or if that person is just thinking so differently that they're just seeing something that nobody else wants. How do you know what to trust? Yeah, I think my answer would be you don't. You don't know what to trust.

14:35At least you don't know from the data or there will be no facts coming in. So this is all about the way you treat the data. If you say, I want the data to be this magic eight ball and I shake the data up, you know, and outcomes, yes, no, maybe try again, then that is exactly where you would get stuck, right? Because you say, okay, the data is showing me the outlier. Now, what do I do with that? But rather if I say, well, data is more like a muse for an artist, right? So it gives me these impulses. Then I think that outlier is still quite helpful because you can then say, well, of course, now I have to look at it.

15:11Now I have to see are there customers like that? And I also have to trust my expertise. my intuition. In a purely data-driven approach, I think the images suggested that you want to get the human out of the decision as far as you can, right? It's data-driven that you want. And the assumption there is, of course, humans make stupid decisions, which they do, but that the sort of the pinnacle of decision-making would be something purely based on data. And that doesn't work in the real world, especially if you want to be innovative. So it's about finding that balance and tuning it where you say, okay, this is all the data I'm looking at.

15:50This is all the data I'm going to listen to. But then being aware that the decision-making and having that curiosity and also making the call of saying, this is crazy, or we're going to try that, that is still going to fall to you as a decision-maker. Data does not take that away. So I think the data-inspired approach is a much more balanced approach where you're really trying to find out, get as much evidence as you can, but you're also realizing we're not in a scientific lab. We can't make these decisions a hundred times and really see which one pans out, which one doesn't. It comes down to making a call.

16:28It comes down to then also accountability and being willing to say, okay, I'm willing to put my name behind that. And I will always be able to say I had evidence to do it, but I can't just point to the data and say, you know, the data told me so. I have to take responsibility as a leader and decision maker or an expert here.

16:47Jason:Sebastian, let's move on to another strategy that you have in the book, which I find really interesting. So you say, and I'm just going to read something that you write, and then I'd love for you to unpack it. You say, use data to support a bold guiding policy rather than letting the data dictate a safe incremental path. And this is really about performing a diagnosis of your actual challenge instead of just following whatever the numbers say. Talk to me more about this. Yeah. So I think there's a big temptation of saying data is our strategy or these days AI is our strategy. And it's not. It's just a tool.

17:25And so what the guiding policy is about is it's from a strategy framework that I find very useful from strategy guru Richard Rumold. Very highly recommend to read his books if somebody is interested in strategy. and he basically says strategy is problem solving. And it starts out with that diagnosis where you say, what's hard for the business? What is really the challenge that we face today? So for example, I recently worked with a medical device manufacturer and the CEO of that manufacturer was telling me, look, I have this problem. Everybody makes good devices. So these were implants. So everybody makes these really great implants.

18:06there is no way I can differentiate anymore on the mechanical quality, on the electronic quality, like everybody just makes very good ones. So I need another way to differentiate. That would be a good diagnosis, right? You're looking at the business and you're saying, oh God, what am I facing? There's no obvious solution. So what do we do now? And the guiding policy then tells you, how do I want to react to that? So what is my way forward? So you might say, well, you know, we might just want to buy a competitor. That could be a way forward to say, you know, we'll just, you know, become a very big medical devices company.

18:48Then it'll become cheaper. We'll make the profits. You could say, which was, of course, when they were talking to me, why can't we use data to make the product better or to build like an ecosystem, maybe for the patients or to understand more about, you know, the failure points, these kind of things. But your guiding policy still lives in the business realm. And the data can help you figure out the possible directions, right? So you could, for example, say, well, we've already had patients in this digital ecosystem. How did that work out? Did they engage? Did they not engage? So that will be very helpful evidence for you to decide your guiding policy.

19:28But whether you say, I'm going to go for, let's say, the mergers and acquisition route, or you're going to go for the ecosystem part, or you're going to say, no, we're just going to be this very data-heavy innovation company, that is your strategic decision. And the data is just something that flows into that. But it's your executive role. It's your strategy. And the sharper you make the guiding policy, the more things you say no to, the better, of course.

19:56Jason:You know, as you've been talking, I've just been trying to think of, have there been moments in which I created an echo chamber for myself as a result of data? And the thing that I came up with, I just want to tell you about this circumstance and then have you unpack what you're hearing and ways in which I could have done better, was that I had this idea. Okay, so I have this newsletter. It's called One Thing Better. And I used to just come up with a bunch of different A-B testing headlines and subject lines for the newsletter and send it out. And that was that. And I thought, is there some better way to be doing this?

20:35Jason:And I started to play around with an AI tool that surveys a kind of mock audience, right? So it's like, I want to survey a thousand entrepreneurs who subscribe to a newsletter called One Thing Better. here are three subject lines, which are they more likely to open? Then I decided one day, and I don't remember why, to just write some random stuff that I had never tested. And one or two times that the stuff that I wrote beat in the A-B test the stuff that had come out of the AI tool. And eventually I abandoned the AI tool and just started going back to being more experimental myself. And this taught me something very interesting about how I had found a system that provided me data and that felt very comforting and kind of worked.

21:31Jason:But once I broke out of it and I started to do the thing that the data couldn't anticipate and that the AI tool was never going to produce or really engage with on its own, I actually saw better results. I mean, I love that example, right? It's that exact beautiful illustration of, well, you were using data to optimize, optimize, optimize, but the data just didn't lift you out of that railway track that you were on. And then you found something that I think is very important. You sort of, you took the courage and experimented. And I think that is something that when we talk about data-driven doesn't come naturally because when we think about data-driven, I think it's a sort of an error reduction mechanism that we're actually looking for.

22:18We are not actively looking to make mistakes. No, we want to avoid mistakes. We want to avoid doing stuff wrong. And in order to generate the really interesting data, of course, we have to be willing to take that risk. And we have to be willing to be wrong so long as we are collecting data with it. So essentially, you designed a very targeted experiment, right? Where you said, okay, I'm going to try this out. that's outside of the data zone. I'm going to try that out. But then you collect the data and you learn from it. And I think that also applies to a company. So at some points, you just need to say, what are the handful of experiments that we want to afford?

22:58And the newsletter, of course, is a great example because, I mean, you can't afford to do it 20 times, right? Otherwise, your subscribers are going to say, well, something's weird here, right? And as a company, you can't do that either. but to find those two or three targeted experiments but also I like to call them honest experiments which means you're not doing that type of experiment where actually you have to be successful and you already kind of know it works but you actually go out a limb and you try something new and you then take everything that you learned from that back and say okay did we find something that worked or something that didn't and then we feed that back into the mechanism so you enrich your data and what you also describe i think that is something that's the the being scared you know and and being saying well i i i don't feel comfortable because i am kind of making decisions here where i i can't see far ahead i don't know what's what's going on that i think is just something that as leaders as creators as innovators as entrepreneurs you need to find ways to just feel as comfortable as you can with that discomfort because it's not going away.

24:13Or if you want to force it away with data, then you're back on the railway track. So it's just something I think you have to say, well, in that line, if you're in that profession, it comes with a job.