How Incrementality Cuts Waste + What and Where to Test in Ecommerce

22:59Use the MTA to inform the way to do the holdout in terms of the right DMAs to exclude. Automate the spending, right? So like your media buyers don't have to say, okay, let's cut the spend at this time and then start it again. We just automate that. Automate the process of alerting you if things go wrong, right? So like Katie was saying, if something goes awry and one of our customers ran a test for four months, they were all inaccurate.

23:26Katy Mimari:How did they not catch it for four months? Well, because there's no monitoring. And that's why at Norpene, we built in automated monitoring. So we're going to be watching these tests, watching to see with a machine, not humans. And so, yeah, they were wrong because there was mistakes in the data science. and actually their in-house data science person caught it, but they ran four months of tests that were inaccurate, right? And they actually told us that we just went with the Norpene MTA data because it conflicted with the income tally to us. So that's why we built it, right? We saw a lot of human error, a lot of human work in setting up and running them.

24:05We're like, what if we eliminated that and used MTA data to make them smarter, right? So you'd have less opportunity costs. That was our philosophy.

24:17Sean Frank:Implementing AI with our trusted data is shooting in the dark and building that tradition from scratch, dealing with Tableau, dealing with data pipelines, dealing with all that. It doesn't have to be your headache. You can hire Ceres Analytics to handle that stuff for you. Ceres IQ gives your leadership team a single shared view of the truth. Contribution margin, channel performance, customer economics, all can be answered instantly with the power of Ceres Analytics. Do not fall behind. Ceres Analytics is experts. They've set it up for Ridge, TextCloud, and hundreds of other brands. Speak to them today.

24:49Sean Frank:Connect whatever AI tool you already use, and you can put that right on top of your personal data, privately, securely, and you can start talking to your data like your favorite chatbot. So we started running incremental tests like a year ago, and it helped us unlock new tools in new ways and new tools inside of the channel. So inside of Meta, everyone was just running conversion campaigns forever. We started moving up funnel. And so we have custom conversions on landing page view, on add to cart. And we put way more money into those because they just proved to be way more incremental. They proved to be way more incremental.

25:25Sean Frank:Oh, God. He's incremental. Incremental, Sean. So what that led to is like I'm spending, you know, 30 % more on Meta, but sales are up 60%, right? Like this month's sales are up like 80 % or whatever. And it's because I was able to add on YouTube because I finally had proof that I was working as a channel. And then I'm spending money on meta in new ways, either it's reach or video view campaigns or whatever else. And for so long, that was letting money on fire. But if you can layer on tools like incrementality, you can prove that it's working in new ways. With that being said, just because it's working for me doesn't mean it's going to work for you.

25:58Sean Frank:That's why you have to test it, right? If your takeaway from this is Sean says YouTube works, I should spend money on YouTube, that's a great way to destroy all the money in your bank account, right? So you have to actually start testing this.

26:10Katy Mimari:But were you running all those tests at the same time, Sean? Like, are you testing YouTube at the same time as Meta? Or are you breaking them up and each month doing a different one?

26:19Sean Frank:So we run, and this is where I disagree with Austin, we run more tests than anybody else. So, like, we ran like 25 tests last year. So the average test is lasting 10 days or two weeks or whatever, so shorter tests. But just there's always something being tested. Because what we think about is, yes, you're losing 5 % to 7 % of potential revenue by removing those geos. But the geos kind of rotate out, right? Like, you know, it's always different people that are getting removed from those tests. It's not the same 10 % of the country you're never going to hit.

26:52Katy Mimari:Well, but couldn't it be? Like, what if you've got the same zip code you're holding out on Meta, but then you're not holding out on YouTube and one's feeding the other? Like, isn't it kind of skewed by running them at the same time?

27:05Sean Frank:Oh, well, there's always one test going, but there's one test going back to back to back to back to back.

27:10Katy Mimari:Ah, okay. Not overlapping is what you're saying. 25 tests all at different times. I don't think I disagree with you, Sean. I think it's more depends on your mix and how much you're doing, right? Because the Ridge is, I mean, you guys are known for a lot of changing and tweaking and experimenting, right? But not because I know our other customers. some people have more of a steady mix and steady tactics right so they don't need to test as much so yeah it just depends right you guys are doing so many different things so i agree with you i think you should test a lot because because you're doing a lot of different things and tweaking all the time so it just really depends on the business right i think it's a culture of testing right we

27:51Sean Frank:talk about like the culture of incrementality um internally at ridge there's like a line of 10 people who want something to like they all want to test something and they have to get in line It's like, when do I get to run my test? And, you know, I think incrementality is so great because I think you can still do A, B tests on your website. Like there's this, it's going to skew it a little bit, but like, I think you can do all the A, B tests you want on your website when it comes to price or whatever else. I think those results end up being so, you know, so small to actually tweak the whole thing.

28:18Sean Frank:So like, not only are we running a ton of incrementality tests, we're running a ton of A, B tests. So there's like a hundred tests a year being run on my website. And that makes it just so much better. because we run it and then four months later, we run the same thing again, just to just double proof everything that we're thinking about.

28:34Katy Mimari:Have you ever had to go back, Sean, and like maybe in the first month in January, it said something was really incremental. You retested it three months later and it was not.

28:44Sean Frank:Yes, we have totally gotten burned. You know, we launched a bunch of women's products and women's was crushing it for us. We were doing, you know, like the first month we did it, we did 600 grand in like one pink wallet. And we're like, oh my God, people want pink wallets. The test we're showing it was incredibly incremental, right? So we bought a bunch of them. We leaned really, really hard in and then it failed. And it's because, yeah, it was like, it was totally incremental, but then we sold it to every person on earth who wanted a pink rich wallet. We could not replicate that success at all.

29:23Sean Frank:So, you know, we got stuck with a bunch of lavender carry-ons I still got. I'm like, we have to turn through these things. We bought 40 ,000 Lavender Ridge wallets because the first month we sold 4 ,000 and I'm like, dude, we're killing. We're making a ton of money off this. I should buy a bunch of them. Then we got 40 ,000 and I'm still trying to sell them today. I just think your team is unique. I mean, I know your team's unique. You guys just do an incredible job of innovating and trying and just testing so many things all the time. Not just like incremental tests, but just any kind of tests, like creative tests or influencer tests.

29:55I mean, you guys have been doing that for years and that's why you've scaled so, so well, you know?

30:00Katy Mimari:Well, what about for little loser companies like me, Austin? What am I supposed to do? Well, you guys are great. You guys do do great work, too. I think I think, you know, you know, you guys do great stuff, too. I think it's OK, Austin. You can just tell Sean how amazing he is and I'll just sit here. Sean and Connor, you know, they're just unique guys. We all that's why he's offering his podcast.

30:22Sean Frank:So it's taken the industry 10, 20, 30 years to bring incrementality to digital brands, right? Like, it's not new. It's been around forever. You know, we started hearing about it last year when Meta was talking about it nonstop, right? Maybe a little bit before, maybe two years ago. But why is it taking so long for companies like Bridge to embrace it? And Katie, why were you hesitant to embrace it?

30:41Katy Mimari:The reason we did it, I'd say, is because FOMO a little bit. It's like, everyone's doing it, so should I be doing it? And do I need to be doing it? I mean, the biggest reason, Sean, it took us a while to embrace it was because we were not omnichannel for so long. And because such a high level of percent of our spend went to one channel meta, which I knew was working, it didn't make sense for me to test a channel that I was not spending a whole lot in, right? Because the test would have taken longer. It just, I didn't, I wasn't, I don't know. I just didn't think it was a big thing. Like an example is a couple years ago when we first ran incrementality testing, we ran one on TikTok.

31:23Katy Mimari:And it actually told me that TikTok was incremental. And we were spending a couple thousand a day, so not that much. But my gut told me it was not. And we turned off TikTok ad spend completely and did not even see a penny in revenue dip. So I know that's not like what you're supposed to say, but I still, I'm skeptical of everything. And especially when you get on to Twitter or X and every other word of everybody is incrementality. Like it just becomes this like circle jerk of incrementality. However, we're leaning back into it now because I do think as our, as we scale, spin up and lean into other channels, it feels like a gut check.

32:04Katy Mimari:Like it feels like I'm, I know these things are working, but I don't want to make huge decisions on my gut anymore. right like I want to know if something actually is performing or not and honestly running that incrementality test last month and kind of seeing as it was progressing and knowing that what it was it was telling us very different things that our MTA was the end platform was we knew something was broken and it helped us flag it so that we could go try to find the smoking gun right Right. So so I kind of like Austin's idea of like, I don't know, maybe for, you know, us little guys on the short bus.

32:45Katy Mimari:I could see a space where I was doing incrementality testing for a couple of months, then maybe took a couple of months off, then tested again and kind of went through that cycle. But or did a different right, like do meta one month, do Google another month, lean into Pinterest, TikTok, you know, things like that. But I still don't think it's, I don't know, Austin, I feel like is it brands that are like 50 million and up are doing incrementality? Like, where do you see the sweet spot? Yeah. Curious to hear what Sean thinks. I mean, I think it's like once you start really, because there's some companies that maybe are lower in revenue, but the spend is really high.

33:24Right. And you're, you know, like, for instance, you're doing, I don't know, 20 million revenue, but you raised money and you decide or you have cash and you decide you're going to go do CTV. They're going to try TV or YouTube, like Sean's saying. They're going to lean into YouTube, right? And you decide you need to know whether it works. Would you agree with that, Sean? Like it's when you're really leaning in hard to something that's view-based. If you're scaling an e-commerce brand today,

33:48Sean Frank:ads alone aren't enough. After Sell focuses on the one moment that every brand already owns after checkout and turns the post-purchase moment into more profit. Monetize every order with post-purchase offers and thank you page experiences without disrupting checkout or hurting conversion. Enterprise-grade tech used by Gap, Ticketmaster, Macy's, and Target now driving results for brands like True Classic, Hexclad, Ridge, and Jones Road. I would know. This is the reason I ended up buying three pans from Hexclad instead of two. Aftersell has already generated over$1 billion in additional revenue for e-commerce brands, revenue that doesn't require more traffic or higher cap.

34:25Sean Frank:So check out Aftersell and tell them that the operator sent you. I think as soon as you start adding a big second or big third channel, or you have something like you really need to know if it's going to work or not before you invest a lot more, I think your mentality is a great tool. I want to highlight what Katie said, because I think it sums up the industry the best. Tools for tools sake. If you're marketers listening to this and people in your organization, they love new tools. They love shiny things. And it's kind of like if you ever see like a hobby guy in his garage, he'll have like $150 ,000 worth of tools and I'll never build anything.

35:01Sean Frank:It's like, that's what most marketers want. They just want every single possible tool on earth. But the counter is we need these tools, but we should use them begrudgingly. So I wish I never had to use any ad platform to make money. I wish people would just show up to my website and just give me free money. It doesn't work like that. So I have to use Meta. And then I use Meta to the full extent of its ability. And then I can't use it anymore so i have to go to youtube it's the next place right i don't want to i my life would be better if i could just go to meta then i have to go to youtube and now i'm at youtube okay i need mta to measure these things right okay now i'm going to snapchat and tiktok and tv okay damn now i need an incrementality so it's like these tools should be um they're important and but you should basically be forced into them but your marketing team's going to ask for them they're going to hear about how cool incrementality is going too many tools and i think austin aren't

35:53Katy Mimari:I mean, really, it's going to become even more important since like last. And we know metas change the way that they're right, like tracking clicks now. And it's becoming even harder to really give attribution to the like you said, it has to be within a 30 day window. Like if you've got somebody that shops for six months before they buy something, it's hard to even track that. And the way I think about incrementality is it's is that channel needed? Like, Sean, you know, I mean, really direct traffic is our second or third biggest traffic source. Like people just literally coming to Caden Lane.

36:27Katy Mimari:And so one of the things that's beneficial to us to test with like Meta is do they need to see the ad? Right? Like, were they going to come and buy it anyways? So it's not the channels all work. It's just what are you using them for? And where's your like, you know, where how high can you scale in your spend on it?

36:45Sean Frank:And how how big can revenue get? well spending the least amount of money possible because like that's really what we want to do even stuff like sms it's like are you sending too many sms's right it's like what we got to do hold outs we actually have to prove that stuff out so it's you know uh we're adding a tool to hopefully remove negative spend that's that's the beauty of incrementality and and and mta it's like if you can save one percent of your meta budget it's a million dollars a year so that's you said you

37:14Katy Mimari:You know, what's funny is I'm like sitting here thinking about this and like, I mean, I've been doing this 22 years, right? And the last 10 years specifically on e-commerce. But even five years ago, nobody really talked about testing. Like not even in plat, like not even store testing, Sean, like price testing or like button testing or colors or banners or images. Like why did testing become such a thing? Is it just because we're all dorks and marketing? Like why? Why now? Why is testing like such a cool thing? because it justifies us? Like, what do you think?

37:47Sean Frank:No, it's harder. It's like, it's like what I'm saying is like the,

37:52Katy Mimari:it's so funny. It's not harder to market online. And now we've got to basically, we can't just like see which way the wind's blowing. Yeah.

38:01Sean Frank:I said, we layer things on progressionally. Like it used to be, I knew people, I knew a brand. I'll talk about this brand. They're out of business now. They would, they're like, we're a holiday gift. So they would come in October. That's when the company would take nine months of the year off. Everyone would be laid off. They'd come back in October. They would crank their budgets, one ad, one landing page, one offer. They would make like$7 million. And then January 1st, turn all their ads off, come back the next year. That worked for four years. I'm like, there was no competitors. There was no one eating their lunch.

38:36Sean Frank:They didn't test anything. They didn't think about anything. And that was just the reality of like 2015 to 2019. And then as soon as Meta got a little bit hard, a bunch of people went out of business and now it's like okay the people who survived are methodical about finding the next dollar possible right and yeah and like imagine if you heard somebody only worked three months of the year made seven million dollars you're like yeah good luck but it's like i i'm gonna launch your product in two days that's i'm coming for you yeah yeah i think that's true also the i think sean i'm curious also and katie even seen it for a while, but the mix is diversified.

39:11People are trying more things. AppLovin wasn't around, right?

39:15Sean Frank:Yeah. Good point. Snapchat didn't really have an ad platform and TikTok didn't really exist. It was just Instagram ads. It was just Facebook ads.

39:25Katy Mimari:Austin, are we going to be able to do any kind of incrementality testing on chat GPTs? Yeah, that's all coming. We're talking to OpenAI about ads. And yeah, I think as the channels diversify and there's more stuff, we're going to need more MTA, more view through, which we didn't talk about that much today, but I'm really excited about that in our platform, the clicks and deterministic views. So viewing, seeing how ads convert on a view basis. I'm excited about you guys adding Pinterest. Like for us, that's a channel that, I mean, Sean, it's probably not a big channel for you guys because most men aren't on Pinterest, but for us, it's a big one.

40:04Katy Mimari:And there's a lot of conversion lag with Pinterest and it's hard to measure. And so that's where an incrementality test is extremely beneficial because we can really lean into it. Here's a fun stat for you. There's a company in the health and wellness space where even on clicks, so not even including view through, something around 23 % of their conversions take one year to convert. One year. Okay, think about that for a second. So they spent a dollar on Meta today. And including views, it's probably closer to 30, almost 30%, right? A third of that revenue will take a year to convert. So their first year in business, they just think they had the worst business in the whole world?

40:47Katy Mimari:They're just like, there's nothing.

40:49Sean Frank:Oh, I was going to say, I'll tell you about two personal experiences with that in the health and wellness space. I bought an Eight Sleep. It's$5 ,000. So like, I've heard about Eight Sleep for three years before making that purchase. I heard about it on podcast. I clicked on ads, the whole thing, because I have to convince my wife to be like, yeah, the bed's going to heat and cool. It's going to be awesome. And she's like, oh, another thing in my life. Right. And then I have a tonal behind me. It was also$5 ,000. So like these large ticket purchases in health and wellness, they will just take forever to actually convert.

41:19Sean Frank:Right.

41:20Katy Mimari:So everyone that's listening, keep advertising to Sean. In a couple of years, he's going to buy your really expensive toy. That's where MTA is important, right? That's where incrementality would maybe miss. Like even if you ran a three or four month test, you know, you're gonna miss, right? So that's where MTA, it's a good example where MTA and incrementality need to work together as a team. Right? Because if something takes a year to convert, you're just not gonna most likely see that in an incrementality test, right? Is that company you talked about still around? Which company? I don't know, the one you said that took a year.

41:53Everyone on this call would know who they are. Oh, okay. Very big company. Yeah, yeah.

41:57Sean Frank:The last thing we'll say about incrementality, You need a weapon to help remove the other tools, right? To help remove the other channels, to help remove complexity. Because a lot of our businesses is just a house of cards. You're stacking this thing on top of this thing. And somebody has a favorite way to look at data. So they want this dashboard. And somebody really likes this influencers. They want to sponsor them. And all of that stuff just gets piled onto your business. And you have to find a way to get to the heart of what's actually working. And incrementality is a great way to do that.

42:26Sean Frank:because you will get a readout if this actually drives revenue or not, right? And hopefully it lets you get back to the basics of your business, right? Like you want to be doing the smallest amount of things possible to do the most amount of money possible. And like, you know, you want the leanest team to do the most amount of revenue. Everything comes down to just being like, how flexible can you be as an organization? And incrementality is a great tool to do that. So I know it's counterproductive, adding more tools on top of your tools to audit your tools, but that's the world we're in right now.

42:54Katy Mimari:I'm just glad we're not playing a drinking game with how many times you've said incrementality in the last hour. He does.

43:02Sean Frank:Any AI talks? Should we say AI like 80 ,000 times?

43:04Katy Mimari:Oh God, that was the whole meta event. You missed it, Sean. The whole week last week was AI, every other word. If it wasn't every other word coming out of your mouth, what were you talking about?

43:15Sean Frank:But I am excited for the AI ad platforms. Like they are coming. Like ChatGPT ads are going to be awesome. I think that's open beta right now. Anybody can sign up for it. But everybody signed up, but they're all on a waiting list. Did you get in? I'm sure you're in. You know, I did get moved to the top of the waiting list. So, but dude, the crazy thing is like, you know, only like 4 ,000 people work at OpenAI. Like they are crazy. They gave me early access and I'm like, oh, cool. How long is early access going to last? And they're like, yeah, we're going to go public next week. It's like, oh, okay.

43:44Sean Frank:So I get four days. I got a four day head start.

43:46Katy Mimari:Oh man. Well, okay. So can I just tell my email if they're listening, they can go ahead and put me at the top because we're still sitting on the waiting list along with everybody else that I know. So that's good. Let us know. You test it out, Sean, and let us know how it goes. You can be my first incrementality test.

44:02Sean Frank:There you go. I'll burn the money. And then maybe we should just say, if things are, yeah, I said this a couple of times, but a lot of this industry is tribal knowledge. It's like group chats and Twitter posts, what's working, what's not working. And so bad information spreads just as fast as good information. Like I remember people being like, CPMs don't matter. Like if you have a high CPM, it's good. And I'm like, no, that's not true, man.

44:26Katy Mimari:No, it's so true. And just because somebody is posting about their results on their test that they ran for incrementality does not mean that's what yours is going to be for that same channel. Introducing full automation into incrementality testing is, I think, the future, right? So you're kind of like, back to Sean's point about having smaller teams being leaner. We're trying to bring incrementality testing at scale so that it's more accessible, maybe, to smaller companies at a lower price, like you were saying, Sean, because it's automated, right? You don't need all the human labor to execute.

45:04Katy Mimari:And the human labor to screw it up because I'm not, right? Like yours - That's the most important point, Katie. Yeah. every SaaS company says they are AI powered but very few can explain what it actually does for the revenue of my brand this is why PostScript's approach stood out to us they don't just build AI for demos or buzzwords they built it to drive real incremental revenue PostScript's AI called Shopper it shows up inside of SMS at moments with real buyer intent when shoppers are likely asking questions hesitating maybe even about to drop off Shopper can answer product questions instantly answer questions about fit, availability, recommendations, order issues, the kinds of stuff that people usually bounce for.

45:43This means more conversions, higher A, less lost demand. So you are driving more revenue and doing it more efficiently. Check out Shopper from PostScript. We use it at PILA, which is why I am telling you to check it out.

45:56Sean Frank:And it'd be so cool to just like inside of your Northbeam dashboard. It's like, hey, this channel's crushing it for you. Don't believe me? Test it. And you just click a button to test the whole thing click it and it runs it and then to katie's important point we've seen so many mistaken tests run by the vendors out there uh make sure it doesn't get screwed up and and find that because the opportunity cost is huge right um and then if you're going to test a lot like the ridge you know we'll alert you if if you know when they go wrong right and say hey you know i know you want to run this test you guys like to test a lot this one is statistically insignificant on day three.

46:31You should shut it off so you don't waste your money, right, automatically. So, yeah, I think bringing automation to the masses reduces cost, reduces labor, helps you run as a lean team. And yeah, it just made a lot of sense to us. And like Katie was saying, automatically injects into your MTA data the Calibrate, right? And Connor at the Ridge, he told me that was what he was really excited about.

46:57Katy Mimari:Yeah, because it's all right there. You're looking at it all together. Exactly.

47:00Sean Frank:And look, this is now the big three measurement. MTA, which is like you guys are the pioneer of. MMM, which is like the oldest one out there. And then you have Incrementality laid it on there. Is there another measurement vertical that we're going to hear about in three years or whatever that I don't know about now? Ooh, that's a great question. I think what we're excited about, we announced this two years ago with Meta somewhat quietly. They pushed it out where they announced that they were using our data and Google's data at the time, this is a couple of years ago, to train their AI ad serving algorithm, right?

47:31So basically, we called it Apex at Northbeam, which is really simply put, there's two ads, one with Katie, one with Austin, the one with Katie's doing way better than the one with Austin. To your point earlier, Sean, Meta thinks Austin and Katie are the same. We see that Katie's Northbeam data suggests Katie's way better. So we send that data to Meta and they optimize their AI ad serving algorithm against, we mentioned AI, thank God, we did it. And it optimizes against that, our data, right? So that's something we announced two years ago. Announcing here, we're rolling out a new version of that, we're really excited about in the coming months.

48:11So that's another, I don't know if it's measurement per se, but it is because we'll run A-B tests against it to see how does that data.

48:18Katy Mimari:We saw Austin, I don't know if you remember, but we actually saw huge improvements when we, yeah yeah apex was yeah i remember was a big deal for us and and i appreciate thanks and and the new

48:30Sean Frank:versions can be even more powerful you know the only thing i think northfield is missing that if you guys are doing the big three of measurement is just more testing tools on site and maybe like here's a big here's a big unlock it rich we sell a power bank on amazon okay it's doing fantastic um you know we thank you thank you we changed about two words in the title and sales went up 10 % just because you start ranking for more stuff, there's more visibility to it. That's all built in Amazon tooling. They have A-B tests you can just run there. But that was thousands or tens of thousands of dollars a day just because we changed three worlds in the title.

49:06Sean Frank:So bringing that sort of - What words in the title for anybody?

49:13Sean Frank:I'll send it to you. But yeah, it was just because there's more organic keywords to be had, like whatever. It just worked way better.

49:23Katy Mimari:And to your point, it was working, right? You just wanted to test it and then now it's working better.

49:29Sean Frank:Yeah. Always be testing. You know what I mean? But yeah, so you could bring the same sort of onsite testing, right? We're using a bunch of great onsite testing tools right now, but how do we get those things talking to MTA? How do we get those things talking to incrementality? All of that just like full stack sandwich to be considered. We are working on that. And I've had, yeah, we are working on that.

49:48Katy Mimari:Yeah, Austin, I want to be able to measure, okay, here, I want to drive traffic to different landing pages with different UXs and I want it to break it out. Like, I mean, I feel like this is a dash, it's probably stuff you'll already do. It's a dashboard. We already do it. We already do that now with good naming conventions. You can track landing page conversions with good naming. So we already do it. Well, what if you have bad naming conventions? Well, then those have to get fixed. But yeah, but we are, we're launching new tools in that arena, but you can already do it today. It just requires setup.

50:20Sean Frank:So if you need help, Katie, how about AI powered automatic naming convention? That's what I was going to say. Yes. Yes. Working on that, Sean. Are you in our meetings? Are you in our product meetings? Yeah. Yeah. We're docs in the North BMI roadmap. Yeah. Yeah. Well, you are first customer. So you, you know, you were there.

50:37Katy Mimari:Okay. Okay. So since you are the tester of all testers and I'm failing at it miserably, uh, for I'm gonna hire you as CMO. What is the first test you run for Caden? First three tests you run for Caden Lane, knowing that I already did a meta test. So what's my next one?

50:53Sean Frank:Well, one, I'd look at the meta test and figure out what we actually tested. Did we test Top of Funnel? Did we test, like, you know, creative style? Like, what was that actual test that we ended up looking at? Uh, I don't know. Austin, what did I test? I would say just, uh, overall channel lift. Overall channel lift. Yeah, so then I would, if I could, could do three tests so you're hiring me for more than one day you're hiring me for like you know six weeks or whatever uh and to have you come in for six weeks is about a million dollars so i have

51:22Katy Mimari:to oh my look this is not the question i asked you now i feel like you're dancing around because you don't know the answer let's get what go get connor he'll answer this faster for me that's true

51:32Sean Frank:yeah look i'm all i'm i'm the showman connor does all the work right um the first test i would test higher funnel ad strategies inside of meta.

51:43Katy Mimari:So you would keep digging into meta?

51:45Sean Frank:Yeah, because if we could test page views or add to cards, you could spend 30 % more, right? And what you want to look at is like, what's the incremental bonus? So it's like, okay, the MTA data is going to suck. It's going to be like 0.1. But do we get a 3X bonus? Do we get a 0.5X bonus? Like that helps you just spend more money on people who are colder audiences.

52:07Katy Mimari:So you're saying even if I did an overall list study as my first test, then break it out to maybe even top of funnel, bottom of funnel conversion. Totally. And the different, okay, so you would. And change the tactics and change the tactics. Yeah.

52:20Sean Frank:Yeah. So that'd be the first test. The second test would be, you guys have great content on YouTube. I have a kid now. And if you type in like how to burp a baby, the first video is Katie showing you how to burp a baby. And I think YouTube should be an amazing channel for you. I mean, my second test is how do we get YouTube to actually work? And I would shoot a bunch of different creative, you know, I would repurpose stuff you have. I would take the best performing shorts, we do some super edits and just be like, can we get this to be the same as conversion optimized meta with the bonus, right?

52:52Sean Frank:We'll get some sort of lift bonus. That's the second test. And then the third test, the exact same thing with TikTok because my wife's on TikTok and it's nothing but baby content. I'm telling you, like, it's just like a bunch of fat babies being burped. Like I'm like, our content would do so, so well here. So it anywhere where there's a young mom, female presence, which is going to be meta, going to be YouTube, going to be Instagram, are going to be TikTok. We should be spending six figures. So I'm going to try to unlock that for you. We'll kick it off. And then I'm going to try to, we'll do a YouTube one for it.

53:24Sean Frank:I'm going to make you spend, I'm going to make you put more of a catalog on Amazon and I'm going to test some words in the titles and we're going to get some good lift on there.

53:33Katy Mimari:Yeah, the three special magic words, we're going to do that.

53:37Sean Frank:And then I would make a whole line of baby products that are not just apparel. There's like a little spatula that you put butt cream on your kid. We have to have a Caden Lane version of that, way cuter.

53:49Katy Mimari:Okay, that is a first kid problem. I assure you with your second and third, the spatula you're going to be using to put butt cream on their butt is this.

53:57Sean Frank:I'm already there. I'm doing this. But the nanny, the nanny doesn't like using the finger.

54:02Katy Mimari:Everyone heard it here. Fancy Sean has to use a fancy spatula spoon because he can't use the butt cream on his finger.

54:09Sean Frank:I'm telling you, grandparents would buy the fancy spoons.

54:13Katy Mimari:Teddy's how he's three months old now.

54:16Sean Frank:Now he's five, five weeks. He's five weeks old.

54:18Katy Mimari:Okay, Austin, so you heard my brand new CMO give you the roadmap of what I'm going to be doing at Caden Lane, and you agreed about YouTube. So now, is that on? Like, what do we do? Can we, like, after this call, we get on and we set it up? Like, what do I do next? Yeah, I mean, I think Pinterest could be one that you want to test eventually too, right? Because, I mean, the testing roadmap is always predicated on your channels, right? Like, where you're investing. So, yeah, I think all your top of funnel channels, at some point you want to test. I'm sure Shaw would agree. But I don't want to have to do all this.

54:52Katy Mimari:I don't want to have to think about it. I don't want to set it up. Is Northbeam just going to come up? Where's my little AI buddy in Northbeam where I can just be like, hey, man, this is what I want you to do. And then is he just going to go, wait, she. It's going to be a she, because she's going to be very efficient. Can somebody just have a female AI bot? And just run the test. Can the NorthBeam AI bot be named Katie? Oh, you know, who knows? Who knows? It's, who knows? I don't know. Is that something you're thinking about? Like where I could just go in and the same way that I go into Sidekick and Shopify and I'm just like, tell me this and do that.

55:28Katy Mimari:Am I going to be able to do that on NorthBeam? Yeah. So to, you're going to be able to automate your flight of tests and it's going to automatically suggest what tests to run and how long. So maybe will it even tell me like, hey, Katie, I think you really need, you know, judging on your MTA for last month, I think that this month we need to test Pinterest and this is why. Exactly. That's right. And here's how, here's how much of a holdout. Great call up by Aaron there to, to ask this question. Um, yeah, not only when to run it, how long to run it, the holdout, uh, monitor it if something goes wrong.

56:01So really just automate it from a, a disease.

56:03Katy Mimari:So what, when is that going to come live? Cause I've got to give notice to Sean to like, you know, Yeah, I know. Because in the next month. Yeah. I mean, we could test YouTube right now. So we're ready with YouTube. So we'll start with YouTube, right? We'll go from there.

56:20Sean Frank:What's up, operators? Welcome to the RichPanel ad read. RichPanel has been a sponsor for over 12 months. I've been a paying customer for over 12 months. And guess what? I just renewed to pay again for another year. We have cut our SaaS bill in half and automation dropped our cost per ticket by 70%. Our CSAT has also improved from 88%, which is still really good, to 96%, best in class, all powered by RichPanel. I told them last year, hey, you guys need to do the same thing with returns. And now RichPanel has a returns portal. It's built to cut down your tickets and convert more refunds into exchanges.

56:51Sean Frank:They do the heavy lifting, data imports, self-service, retention flows, team training, all of it, and they'll be live in two weeks. If you want to save 30 % guaranteed on Helpdesk and now returns, book a demo. Also, the mid-flight statistical significance monitoring, I think is really important because, you know, we've got to the end of six-week tests and it's like, oh, yeah, I'm sorry, that test doesn't mean anything. So just being able to cut that off sooner, re-put those resources towards something else, unlock that holdout, yeah, I think that'd be fantastic. I'm with you. I think that's what's...

57:28That is what I'm really excited about, plus the fact that we could do better holdouts because we have your MTA data. We see the way the purchasing by channel is happening so we can pick the right length of the test, right? And we can pick the DMAs better because we see all the purchasing behavior by platform. So yeah, Sean, to your point, monitoring and then also better holdouts.

57:49Sean Frank:Austin, you see more data than anybody else. I'll tell you that Ridge is having an amazing 2026. Is that common? Is everybody crushing it? What's happening? Tactical and practical sales results right now across the industry. What are you seeing? oh wow great question what we're seeing i'm going to give you a really interesting stat that the highest performers uh fastest growing companies in 26 are outputting two times the creative volume of the lower performers right two times so that's just a really we did some you know data analysis and that's the step right on average right two times so it doesn't mean you want to create a bunch of ai slop right that's not the best way to go but the people that are intentional that really work on genuine, high quality, creative and iterate a lot at 2x the volume of the other people do better.

58:41So, and I think it's getting harder to market because of how crowded it's getting. Right. You see, yeah, just I think people getting more competitive in the ad space. Right. Just in general. I think, yeah, with AI tools, people are able to generate more creative faster. So it's kind of pumping the ecosystem full of more creative. So therefore, it's noisier. And so...

59:09Katy Mimari:Have you seen a surge of new brands coming on to the market? Yeah, I think Sean talked about it with a couple of expos, right? The idea that it's a good time to get in, right? With the tool, you can run as a leaner team. You can use these tools to move faster, right? So what does that do? that creates more competition. What does that mean? That means you've got to be better. That means you have to understand your data better. You have to create more authentic, creative.

59:38Sean Frank:Katie, and I think our product categories, you know, durables, I mean, baby's still very hot, but like we're still, you know, we're old heads compared to everybody else. I know a company doing pouches and they're, they're spending 5 million a month on meta and they have a team of four people. And it's like, what, what is going on? What kind of pouches? What do you mean? Like the Zen pouches. That's what you're talking about, yeah. It's like a, it's not nicotine, but it looks like nicotine. Like that whole thing, right? And it's just crazy that these very small teams are spending tons of money.

1:00:13Sean Frank:And because it's like, let's remove all OpEx, everything just goes into this marketing machine and we'll generate creative and we have a product, one skew, and sell the hell out of it. And then, you know... That's the dream, one skew. i'm over here inventing new things to try to sell and you have like a billion patterns a year you have to come out with i think that's why that's that's so important is that um nailing the creative piece obviously understanding your measurement your data but like what is your creative testing uh creative volume uh plan look like and is it um and it can't just be all ai generate creative i had a i did a hot take episode with cody at uh at jones row beauty and you know his point was like oh, authentic creative is going to be at a premium where it's like really, it cuts through the clutter because of all the flash in the pan, new stuff that's just blasting out.

1:01:04So I think doing 2X the creative of your competitors is important, obviously. That's what the data shows.

1:01:11Sean Frank:Yeah. You have to do all the stuff you were already doing, and then you have to add all the AI stuff on top of it. So it's just, you get way more coming out.

1:01:18Katy Mimari:But I still think quality over quantity though, Austin, right? I agree. That's when you said no AI slop. Like do not, anyone listening, this does not mean you build out your cloud code like ad generator and just have crap that you're pumping into metal. That's not the easy win, yeah.

1:01:34Sean Frank:I've seen people do it, though, make a ton of money. There's a company out there selling men's health supplements just ripping with full AI ads.

1:01:44Katy Mimari:My favorite thing to do now is to play the spot, the AI, with myself. and it is getting harder and harder. I also think that folks are really coming around to see TV, connected TV, right? I think that's a trend where I'm seeing people getting excited about investing in TV. It's hard, right? Because you have to have good creative back to the creative. But I think from behind the scenes, I can tell you there's some very large companies that of course I can't say under NDAs, but I think CTV is going to get really interesting. Austin, I turned mine on today. Oh, there you go. Yeah, literally today.

1:02:23Katy Mimari:Like, they literally slacked me today and was like, because we usually do a lot of linear. And I had tried CTV years ago, but it was a lot of just retargeting. And I think it's made a lot of steps, you know, in the right direction for segmenting. And yeah, we're kind of excited about it. I think CTV is the thing that people are sleeping on. But it's hard, though, because you have to have the resources to make creative. Anyway, make sense, Sean?

1:02:46Sean Frank:You know, Katie, we're in the exact same boat. We always bought linear because linear is just very, very cheap. It's like you can get dollar CPMs,$2 CPMs. Like, it's just nobody's bidding on the History Channel.

1:02:58Katy Mimari:No, and it's audience. Like, I mean, right? Like, we love grandmas. So for us, it was just a no-brainer. But I think if, yeah, if your audience is young males, like, I mean, you know, the Today Show is not their target audience.

1:03:14Sean Frank:When Connected TV came out in 2019 or whatever, it was Hulu running ads and it was so expensive. It was$60 CPMs, right? So I'm glad there's people like NBCUniversal pushing it out and even Netflix has dropped their CPMs. So now it's in a place where you can actually buy this stuff and be reasonable with it. Yeah, that's what I think. I think the people that figure out CTV and really nail it with the right messaging, the right creative, it's a good arbitrage, you know what I mean? So we have, more creative is better. glad to hear that's still that's still the mantra connected tv is working katie do you have anything that you want to say like oh i saw this and it's crushing or awesome and a third thing i mean don't

1:03:53Katy Mimari:take testing advice from me because i screwed up my first one so i would say the other thing is that um people don't nail um email sms messaging right i think it's like the kind of thing where people just kind of do the same thing that everyone else is doing. And I know the Ridge, like you guys do some neat things with your messaging where sometimes like, Hey, if this is a hard time of year for you, you know, you know, let us know. We'll opt out of sending you emails or SMS, right? Like you guys do that stuff. I think there's a lot of like opportunity and email SMS to be better. And I see, does that make sense?

1:04:33Katy Mimari:It's our top channels. Like literally we drive so much revenue through both. And we do a ton of segmentation, really. Like there's no more just you send an email to your active audience, you know, your 30-day clicks and opens. And I think there's a lot of optimization. We've had email and SMS revenue go up, whereas deliverability on email is harder, right? Like all these new rules about putting things into different tabs. And so there still is, yeah. And that's, you know, that's your customers, right? It's a great retention play.

1:05:07Call me nerd all you want, but one of my favorite things is when one of my existing software partners, in this case, NorthBeam, adds something that I historically would have had to pay separately for. Incrementality was broken and NorthBeam fixed it. So let's face it. Most incrementality tests are slow, they're manual. One mistake can invalidate an entire test and waste thousands of dollars. If you've done it, you know. Not anymore. So only NorthBeam incrementality automates your incrementality test design and monitoring, letting you focus on insights and not logistics. They're the only ones who build the test for you using your MTA data.

1:05:41I love this. NorthBeam continuously monitors your test in the background, making sure everything runs smoothly and maintains stat sig. Once your test is complete, NorthBeam then gives you actionable and precise results fed into your NorthBeam MTA dashboards. Finally, go to Northbeam.io and request a demo today.

1:06:27Sean Frank:has been totally ignored. There's been no innovations. There's nothing cool that's happened. And look, Clavio is the big game in town. I'm on Clavio. I tried to leave. It went horrible. So like, they're still the best solution. But can you believe how late they were to SMS? Can you believe they haven't launched? Like the cool new thing that they launched is like, we're going to help you copy or whatever. It's like, come on. There needs to be something really cool and innovative happening in email because it's been totally ignored. And as paid ads get harder, email becomes more important.

1:06:56Katy Mimari:I want Apple to start literally selling us notification space. Like I want to be able, like if a customer downloads a pregnancy app, like then I want it to somehow trigger a web banner on their phone. I was also just thinking about messaging too. You know what I mean, Katie? Like I was just thinking about like what you say to people, right? Like, because I sign up for a lot of customers' emails and stuff. And I just think like how you communicate with your audience, right? Like what you send them is something that I think people sleep on. So, and you guys asked for like a, I don't know, like an easy win or something we've seen.

1:07:31Katy Mimari:We've actually like drastically increased our traffic and conversion to our app. So, which for everybody listening, push notifications on apps are free. And if you get your best VIP customers to download your app, which is a better UX and they have special deals and they have special launches. and then you get to message them as much as you want because push notifications are free, that's been a huge win for us. And I'm talking like a very large portion of our revenue comes from our app now. Yeah, anyway, I think the Ridge, you know, a lot of folks do a good job. HexCloud, they do a good job with the messaging and the emails.

1:08:05Katy Mimari:And yeah, it's just - You don't have to compliment anybody that's not on the thing right now. Yeah, you only have to compliment me and Sean.

1:08:12Sean Frank:You know, Katie, and I just had a kid. I didn't realize how important apps are to keeping your kid alive. It's like the Huckleberry. It's like, oh, you got to feed him right now. It's like, okay, this is...

1:08:22Katy Mimari:We're going to have to have a whole parenting episode. You're using like the spoon to put the diaper cream on and you have an app keeping your kid alive. Like, I'm just, I'm worried.

1:08:31Sean Frank:Yeah, the app and the nanny. Yeah, those two things together. All right, cool. Well, I'll tell you guys what's working for Ridge right now. Obviously testing a bunch of different stuff. I think we have to get more of the tools we're using. And what that means is going to unexplored places inside of the app and bidding ecosystem. So if you're at a big enough scale to get out of ASC, try going a little bit up funnel. Don't have to run reach campaigns, don't have to run view campaigns, but get people to add to cart, make a custom event and then run to that. And you will get to be able to spend a little bit more money on meta.

1:09:03Sean Frank:Do a holdout test, figure out what your incrementality bonus is, that X multiplier, apply it to that. And now you have a new thing that's like, you're reaching new people you wouldn't have otherwise. You have to get people into that funnel. podcasts have been crushing it for us all year um we use a great agency uh and you know we're on all we're on the joe rogan's the comedy podcast like there's just there's more podcast space than ever before and they all have a ton of ads on them so like you can just you can just jump in there

1:09:30Katy Mimari:are all your pot wait are all your podcast ads uh the actual host reading or are you recording like pre-recording are they like host reading it it's basically 90 percent of hosts read but

1:09:41Sean Frank:But I think there's some that we have to do auto injection. But, you know, it's like, you know, name any comedian. We're on those podcasts. Also political shows. We're doing political shows on both sides of the aisle. So, yeah, look, man, trying to sell wallets any way we possibly can. Yeah. You know, you're doing it. Another place we're seeing a lot of good results is spending money on Twitter. Or X, right? Do you still call it Twitter or do you say X? Oh, yeah. I still wear Twitter, X, whatever you want to call it. They have a new head of product, Nikita, and he's like rebuilt the ad platform.

1:10:15Sean Frank:And the reason why it's so good is they'll give you a bunch of matching ad credit. So like if you spend$100 ,000, they'll give you$100 ,000 in ad credit. So your money just goes twice as far. And anyway, you can see really good results there. Now we have a very male focused product. And the app is probably more male focused than other people, but it's worth trying. What are you talking about?

1:10:34Katy Mimari:It's not a male focused app at all. I have a funny story for you. I was at dinner with Sean and actually Jeremy Kitsch, and we were in Beverly Hills having dinner, and I was looking on my Slack, and I saw the team, the Ridge team and the Norpeam team were talking about some weird numbers on X. Like, there's a bunch of conversions that happened in the last hour. And, you know, the teams work hard. It was like Friday or, you know, at like 8 p.m. or something or whatever. and uh and they're like where's this revenue coming from and it turned out sean had had posted on x and it sold a bunch of wallets so is that the one about the wall the the um the guy that you sent the suitcase to that was the is that the post you're talking about this this this was like

1:11:23Sean Frank:a year ago or something andrew tate retweeted me and uh we sold a bunch of wallets off of it but uh all right guys look this is a great episode i hope you guys listener of this show You're doing two, five, 10,$50 million a year. You have a couple tools with you, right? You're going out into the ocean and like you start off in the kiddie pool. You don't need anything, right? Then you're going to go a little bit deeper. Maybe bring your snorkel, go a little bit deeper. And that's where you start getting to MMMs. You start getting to incrementality. You have to test these channels and you have to, the biggest expense in your business should be how much you spend on marketing.

1:11:57Sean Frank:I have a 2X MER, half my revenue goes to marketing, 7 % of my revenue goes to people. So it's the biggest thing we spend money on by far. So if you can get a little bit better at it, you'll save a bunch of money. And these tools hopefully help you do that. So Austin, thank you for being here. Thank you for being a sponsor of the Operators Podcast. Katie, thank you for being the only operator who shows up to record these with me. So I appreciate it. And it's always a pleasure talking to you guys. We'll hit a freeze frame real quick.