Bonus: Why You're Looking at the Wrong Data: Cohorts vs Averages with Lifetimely's Jason Prowd

0:00When anyone starts saying we know something, I'm like, I don't know if we know as much as we think we do. That kind of skepticism can help unearth some really interesting things. People will often say we have insights about our customer. And I always try and point out that it's not present continuous. We learned that in the past, it might have changed. You want your ideas to be challenged. Like we had a fashion brand here in Australia I was talking to and they were looking at their top sellers and they were like, OK, we know what sells well. And they also had assumptions about the value of customers over time and just kind of thought it didn't make much of a difference.

0:35But what they discovered was one of the items is kind of five or six on the list of volume led to a double, like twice the size LTV.

0:51Jason, I wanted to welcome you to the D2C podcast. Excited to talk about data. And I'm curious, you've worked with a lot of e-commerce teams. What's something that you think operators need to hear right now about how they're using data in their businesses? Yeah, great question, Eric. Awesome to be here. More focus, less metrics, I think, is where I'd start. So I think we're drowning. Most of us are drowning in data. That isn't the issue. You don't need more data. You need more insights. And I think businesses go through different seasons. And in different seasons, you need to focus on different areas.

1:26areas. And so I think that's a big thing for me at the moment. When I talk with customers, I ask them like, what are you focused on now? And you'll often get a list of 10 things. You can't do 10 things simultaneously. You need to drill down to what are the one or two things. And I think my take is people find that a bit frustrating because they kind of want to move all of the things. And that's okay. But my number one piece of advice would be have the hard conversation, focus on the number one thing you need to move now. That doesn't mean forget about all the other things. It just means for this week, this month, get focused on being focused.

2:00A hundred percent agree. It's funny in the D2C world, we've been collecting data on our subscribers since the first day we started the business. We had people kind of go through a survey. So we have all this data, but we've been in a bit of paralysis when it comes to how to make actionable choices based on this data. And I think that's a lot of brands, maybe stumbling point as well as they have all the data. And then what you're saying is insights are basically where you can kind of tell a story with data and take action, right? Yeah, for sure. I think there's this idea, I really like this phrase be like data and people talk a lot about being data led.

2:33And I always think that abdicates the responsibility of actually doing the work. You need to be like data, data informed. Yeah. So, so data captures something that's happening in the world. And then, yeah, you have to analyze it and tell yourself a story about, well, what does this mean I'm going to do differently? Right. The reason we're all looking at this stuff is you could, anyone can run a business in any way they want. Data helps you make choices about doing something differently. And I think that's the power, but also the danger. And I think the problem we have now is, you know, if we're having this conversation, we might've been not on a podcast, maybe on a radio show 50 years ago, our issue would have been not enough data.

3:08And now that we have the reverse issue. Yeah. Just drowning in it. So with all the different areas that people can focus on and look for insights. You know, AMP as a suite of tools covers a lot of different areas of the sort of e-commerce journey. But where do you recommend business owners start when it comes to driving these valuable insights? Yeah, for sure. I think it starts with what's your strategy. Although I would say by default, I think there's probably too much attention on the top half of the funnel and not enough on the second half of the funnel. So I think acquisition, totally valid, gets a lot of attention, I'd say by default, my starting position would be think about retention, think about subscriptions, think about average order values, all that kind of down the funnel repeat purchase is a spot where I'd say gets under focused when I talk to people.

3:58And of course, like some brands, if you're growing early, you might be focused more on acquisition. But I would say tilt towards thinking about how can you best optimize for the customers you've already you spent money acquiring. And it's only getting more expensive to acquire them. So you have to not have a leaky bucket and you have to be maximizing the value. Yeah, we all have leaky buckets. I think of it as like the challenge is you're carrying a bucket from one side of the room to the other. It's got lots of holes in it. And the goal is fill the holes and make the distance between one side of the room and the other shorter.

4:30And so I think you need to start thinking about lifetime value. And that's why a lot of people come to us, right? The reason when Lifetimely was created in the first place was because we heard from brands that they were struggling to understand, okay, we know how much it costs to acquire a customer for a order, but we know people order again. We want to understand what's the difference between one cohort and another, and are they actually as valuable as we think they are? And so digging into Lifetime Value kind of helps you get there. Yeah. And then what sort of, this is another challenge we have as a newsletter is we keep cohort data on people performing and how they've performed week over week.

5:06But again, we're not doing things. Again, because we don't have as many products to sell. We're literally just trying to keep people engaged for as long as possible. But when it comes to that cohort data, what are some of the ways that people are using cohort data to pull levers to make more incremental revenue? Yeah, for sure. I think there's some really fascinating ways from speaking to customers. I love speaking to customers and just hearing their stories about what they're doing differently. I think often cohort data in particular is used to do one of two things. So you either are validating your existing strategy.

5:36And I think actually that's often maybe the less interesting thing because you know kind of what you're looking for. I would say I like inverting things. And I see this from customers all the time where they're going and looking for cohorts they didn't expect or cohorts that are performing in ways they didn't expect. And so there was a wellness brand we're working with recently. They're female focused. They don't really have any attention spent at all on their male cohort. They have a small range of male products. And they went and used our data and they were looking at different cohorts and different lifetime values.

6:10And they were shocked to kind of find this male cohort had a lifetime value double that of their core audience. Small audience, yeah, but it was a small audience that acquired organically. They hadn't invested a lot in it and allowed them to riff on some really interesting strategies. They started adding male sample products to their female orders as a way to kind of intro to more potential customers. They tested that and then they thought, okay, we can see that's working now. So they could analyze that and see that that was tracking. And then they use that to justify more upper funnel investment, which I thought was really cool.

6:42So I kind of think about it both ways around like validate what you're doing. But also I think often there's gold you're missing. And especially like go digging and looking for things you didn't expect to see, customers you didn't expect to see on the roster or products that are selling in a way you didn't expect, for good or for bad. You don't know what you don't know. And I think that's the really challenging thing is you bring a lot of your presuppositions about maybe who your audience is or how they'll behave. And when you're able to really look at the data, you can find all this gold. But just cohorting, just as a term, you mentioned splitting your male customers from your female customers, but there's also dozens of different ways you can cohort the data, right?

7:23Whether it's like they came in during a sale period or they came in. Talk about what are all the dimensions that are the most valuable to even think about cohorts? Yeah, it's a really good question. I think there's so many ways you can cut it. A few that I'd focus on are thinking about when you've had seasonality in your business that might have driven, say, high one-time purchase rates, like campaigning around sale events, say like a Black Friday is a good example of that. Like that cohort might have, you've spent all the money, that might be an interesting cohort to look at for trying to drive repeat purchase.

7:53You could look at some behaviors of your better cohorts. So you probably have a sense of who are my VIP customers, who are the people we've been targeting, and you can take behavior patterns from them or what you're doing with them and apply them to other cohorts. I think that's interesting. I think looking at a combination of lifetime value and skew. So like what is the first order someone's making and how does that influence lifetime value over time? I think often, and I'm sure you've sort of seen this before, I think often what we do and what brands do is say, okay, to drive volume, I'll focus on my highest sellers.

8:30That makes sense. but I think the real trade you're trying to make is volume versus value and looking at that cohort analysis allows you to kind of say okay so I'm getting decent volume these are my top 10 sellers whatever they are most most businesses know that pretty well I think the danger can be jumping to focusing on the top one or two or maybe there's something at number five that's got a higher LTV and that can make a real difference like do that make that decision 10 times in a row and you create a lot of value. So this is a terrible metaphor. We can take it out if it's wildly inappropriate, but it makes me think of the propaganda about marijuana in the nineties, but how it was a gateway drug, right?

9:10It was the drug that got you into other drugs. Whereas if everyone started with like a heroic dose of mushrooms, that'd be less likely to advance, I think, right? Even though it's maybe a more impactful experience, their lifetime value with drugs would be lower because they didn't have that entry experience that maybe you get with marijuana. That makes sense. Yeah, yeah. Yeah, and again, I like that. I come from a country that can't even have a discussion about marijuana. Canada's a very different place. Yeah, that's right. I think what's interesting there is that you want your ideas to be challenged.

9:45And I see this from customers that I work with and people I enjoy chatting to is they're not looking at the data to validate a decision they already knew is brilliant. That's not helpful. What you want to do is look and say, I think what's often surprising is not like people know the general case. People know different customers are worth different amounts and different products sell in different volumes. Like that's not new to anyone. I think it's the nuance in the delta. So we had a fashion brand here in Australia I was talking to and they were looking at their top sellers and they had a view on their business, right?

10:19They know their business well and they were like, okay, we know what sells well. and they also had assumptions about the value of customers over time and just kind of thought it didn't make much of a difference but what they discovered was one of the items is kind of five or six on the list of volume led to a double like twice the size ltb right so like that was super interesting and they were spending about the same amount of money to acquire those customers so that allowed them to say two things one why are people doing this let's go talk to those customers so it can allow you to like inform ranging strategy and all that kind of good stuff as well so you can use these insights to inform what else you develop in the future but directly from the optimized growth angle it allowed them to expand like their cac caps you know because they're they're like okay great like all things being equal the cac to ltv ratio for this first order is half.

11:16We can invest more here strategically and grow the business more than we expected without having to find extra capital to do it. They've just reallocated effectively. And I think that's a really good cycle to get in that test, learn, reallocate. It doesn't require you to magic up new budget. It just requires you to allocate it differently. And so important headed into Q4 to have a track record, to know how much, we call it stalking the pond on the agency side here. We're on the acquisition side. So we're all about, you know, getting as many, you know, sales into that funnel so that you have as much remarketing to do, you know, through your own audiences.

11:53And I think that's something that brand owners struggle with is knowing how much to invest at in what phase of the funnel and what time of the year in order to see the maximized returns in Q4 specifically. Yeah, for sure. Yeah, I like that. So I'm just so AMP is the is the suite of tools that can kind of help marketers and advertisers, entrepreneurs. And then Lifetimely is what we're talking about specifically right now. Walk me through a little bit about what Lifetimely is and how entrepreneurs use it. Sure. So, you know, AMP helps commerce businesses. We work with kind of 40 ,000 brands, drive incremental revenue.

12:28And we have two ways we do that. One way is analytics, which is Lifetimely. And another way is through like on-site optimization, which is through back in stock and upsell which are both brilliant products as well but lifetimely is analytics so you have a question about your business you want to understand what's happening in your business right across the stack so lifetimely is used by people who work in performance marketing and crm and finance to kind of all have a common language to talk about how their business is performing and then ultimately just make better decisions i think like i think i'm in the business of partnering with brands, operators, entrepreneurs to help them thrive by making better decisions with their own business.

13:10And then how does it, because I think of a little bit like Google Analytics, where you've got to, you know, you kind of have to tease the data out yourself. And I'm curious, like when you hook this up to your store, does it surface these cohorts and these opportunities? Or do you kind of have to know what you're looking for? Yeah, we try and create smart defaults. So we know that every business is different and has kind of different types of questions they want to answer. But the way we've solved that is by having a good first use experience. So we've partnered with some of our best customers to create dashboards for different roles as an example.

13:45So we've spun up the particular metrics we've seen our best customers using and put them in one spot depending on your role. And we do the same thing with cohorts. We do the same thing across the board. So I kind of think of it as it's partly it's like 80 % baked, yeah? So it's sort of a good default position and then you can customize it, calibrate it for your circumstance, which we find works pretty well. I think, you know, our job isn't to tell people how to run their business. It's to help illuminate things they otherwise wouldn't have seen or would have missed. And so we find that works pretty well.

14:21But it's certainly something I'm thinking about a lot as well. Like I want the product to get better at doing that, being a better partner. So that's where I spend kind of all my time thinking about. And then paint me a picture of what that 80 % kind of looks and feels like to people who would be signing on to it. Yeah, for sure. So plug in your core data sources, Shopify, Google, Meta. It sucks in all that data, which is really cool. And then allows you to understand really a few things that are most popular. So the most popular things we see and the questions people want to answer is, what cohorts are performing how over time.

14:58So there's lifetime value and there's a whole lot of breakdown and calibration you can do there. Daily dashboards. So help me understand for my role, how my business is performing right now and focused on the right things. And then P &L. So daily live P &L to see how your business is tracking. We do a whole bunch of other things, got attribution that we've built over the last six months as well. Lots to do there, but I would say they're the top three things people love. We also have buyer journeys. So you can see what happens over time if someone buys X in the first order, do they then come back and buy Y?

15:31And that can help reveal things as well. So it's kind of those five things, but those three are kind of where most people get started and see a lot of value. And then how do you think about testing these things? Because it's like you can make all these changes, you can make them make multiple changes at once, and then you have no idea where you're at. So how do you think of people about going about actually testing these changes to make sure that they're statistically relevant? Yeah, I think that's a really, really important question. I kind of find people kind of bifurcate. People either don't do any experimentation and just like, oh, we're gonna, it's fine, which I don't think works.

16:04Like you should be experimenting. Or people are experimenting like way too much and not letting the experiments run. So I kind of think you need to get land somewhere in the middle, which is building the habit, the culture of testing and learning and picking a timeframe that works for your business. Like the problem is the pushback I get from people is they say either don't have time to test and my answer to that is, but how do you know where to invest and how to make trade-offs? So I don't think that works, but then you can kind of go too far. And so I kind of like having one or two things for most businesses over a two to four week period that you're testing, letting it run for two to four weeks and then reviewing.

16:46And if you keep that cycle going, you think about that over a year, if you learn 50 to a hundred things about your business, like that's awesome. You're going to be in a so much better spot than you were a year ago. I think often people get very, very excited and like want to learn everything in a week. The problem is you just can't collect enough unless you have like crazy high amounts of volume. You're not going to be able to learn stuff in super short periods of time. One of the phrases that we talked about kind of going into this talk was about how averages can be deceptive. And this is something that comes across on the podcast with almost every person we're talking about incremental revenue right now and figuring out what portion of your ad budget is actually driving the results and minimizing the portion that isn't.

17:31So talk to me in your world about how you see averages being deceptive. Sure. I think that headline numbers are super important to kind of track the performance of the portfolio and they're often things you might need to report if you have investors or board or boss like people want to hear like one number the problem is that you can't act on one number like if you're let's say your ltv to cac ratio or your dollar cac or whatever number you pick is up or down like what does that actually tell you like you want metrics that allow you to take action not just feel good or feel bad and it's a good test like if all that happens when you see a number go up as the team celebrates and all that happens when it goes down is the team feels sad like that's that's great not very not very actionable and so the the challenge i think for every business is every number is made up of a composite like everything in your business is a combination of a hundred different things happening so that you don't look at don't look at averages to try and make tactical decisions look at averages to track overall portfolio performance and so So we see people get a lot of value digging into, okay, so what actually drove that change?

18:43And often it's surprising, right? Like you won't know, like maybe a number, the problem often is a number can be up or down for good or bad reasons, depending on what you discover when you dig a bit deeper. And I think it's that digging a bit deeper and maybe going 20 % further than you do by default. I think most people kind of get the idea that I should go look at something beyond a headline number. It's just pushing it a bit further that you tend to get a lot of value out of. I think a lot of entrepreneurs, especially when they're starting, Shopify has made it so easy to almost paint by numbers in a way where it's like, okay, I need to set this up.

19:20I need to set this up. And it's like there's a lot of sort of out-of-the-box setups that get made. Whereas I think a lot of the big wins in e-commerce happen when you really focus on what is best for your brand, what's best for your customers, rather than doing something in a paint-by-numbers type way. What are some common assumptions or sort of set it and forget it flows that you think brands should maybe be questioning and looking into right now? Yeah, I think that's a really powerful point. I think it's awesome how Shopify have made it easier for everyone to get started, but that isn't the whole job done yet.

19:50And I think that idea of calibrating for your context is important. I think a good rule of thumb that I like is one thing I'll talk about something I use with when I talk about product development. People will often say we have insights about our customer. And I always try and point out that it's not present continuous. We learned that in the past. It might have changed. So I like the idea of just having a review on everything. Every assumption you're making in your business, have some cycle to review it. So if that's every quarter, go back and have a look at, well, wait a second. We said this was a good strategy.

20:23Is it actually working? So the first step is like have a process so that you don't get into this habit of thinking you know things that absolutely were probably true, but maybe they were true three, six months ago, 12 months ago. That's problem, kind of that's spot one. And then I think tactically, what you see is things working and people giving up on there being more value. I think a lot of attention in businesses goes to where things are going wrong. I'd start by focusing on some things that are working and saying, well, maybe they're working well, but couldn't they be 20, 30 % better? I think like retention strategies often look like that um there's not enough test and learn like a really good question I ask people often is it's great you have this subscription offer that's awesome and people will have a very positive response to that they'll be like it's working awesome in the business and I ask them how much have you played with it how much have you played with the upfront discount the cycle time like how have you thought about that and they're like it's working like it's fine and i'd say that's the kind of stuff where those assumptions can mean that you're leaving stuff on the table i think if things are going really wrong in a business people tend to focus on that super quickly because if you don't solve it you're out of business right so i think it's those things where something's working i don't think that justifies not testing out different tweaks and i think it's that nuance of tweaking that can be really really valuable, but there's a couple of spots I'd start, retention, subscription, like what those discount levels are.

21:58Yeah. I think we, in our pre-talk, we talked a little bit about the podcast I did with Jordan Narducci, the Nardog. Yeah. That was a great, great conversation. Yeah. Oh, he's been on our orbit for years back when he worked at Kellogg. And it's really exciting to see what he's doing now with subscription. Talk a little bit about some Some of the decisions that go into maximizing people's, because again, subscription is one of those things that people will just sort of set up out of the box and be like, okay, I offered the subscription. But talk to me a little bit. It's such a high impact area for businesses that are able to pull it off.

22:33Talk to me about some of the levers that people have to pull within the subscription world. For sure. And I think it's absolutely true. Like everyone's got into the idea that a subscriber is more valuable than someone who's not. But I think, again, it's like such a good example of push a bit further on the detail. So a really good example is a very, very common pattern is people offer an upfront discount and then offer it into perpetuity. So 10 % off, become a subscriber, 10 % off for lifetime. 20 % off, become a subscriber, 20 % off for lifetime. If you think about discounts as a way of investing in acquisition or investing your margin in growing the business, perpetual discounts are an exceptionally expensive way to invest that margin.

23:20And so I guess like my starting point would be my default would absolutely not be a perpetual discount. The problem with that is humans like don't value things off into the future as much as they value things now. And so I play with the shape of the discount. I see companies that do really well do that. And by shape, I mean, do you offer a bigger upfront discount, for example, to help get people on to a subscription in the first place. And people often, and I know Jordan picked up on this as well, like underbake the upfront discount and then overbake the ongoing discount. Like a starting position for me would be first order discount, probably what you're doing in your business, you could increase it.

24:00And that will probably be positive, obviously measure it. And then that ongoing discount you can reduce. And then you can also think about ways and ways to invest a discount. So for example, if you're running a program at the moment. It's a product I buy. I get a 20 % discount into perpetuity. I'm already a loyal customer. I question whether that's the best way to spend that money. They could probably offer me a 10 % discount and give me some kind of loyalty bonus or samples that I might really enjoy as a loyal customer to that brand, but also might drive new acquisition. I might share it with my friends or it might drive new additions to a subscription, particularly if you're like a a wellness product or something that works pretty well like that.

24:42We see a lot of brands. I've got a company in Australia we work with that sells protein, a really successful business. It's grown really well, but most customers buy one thing from them. And so their challenge is how do you expand a really loyal customer base to other items? And they're thinking about that with their subscription. I think that's really smart. I forget which brand it was. It was a cosmetic brand that came on and said that they discovered that people love their products so much they were using spatulas to make sure they got every little bit out of the product and so on their on someone's like third order or something like that they sent just like a little branded spatula or whatever for their product and they found that the users who were sent that gift had like an exponentially larger LTV just because of that like surprise and delight kind of gesture Yeah, yeah, yeah.

25:32And I think you can play with that, right? I think that's a really good point because let's play that idea out a bit. So let's say instead of offering, we'll go with the 20 % discount kind of option, you trial a model of 15%, but you're happy. You're already budgeted the 20%, so you're fine with that. And you take that 5 % difference and invest that in surprise and delight ideas just like sending that spoon or whatever happens to work in your context. The customer's not expecting it. Especially if you're selling a product that doesn't have a particularly high dollar value as an ongoing item. Like little surprise delights can be quite a cool way to do it.