Ep 601: 25% of Your List Drives 75% of Revenue. The Dangers of AI Email Segmentation

0:00Jordan Gordon:You might never notice that this thing is actually bleeding your program. If anybody out there is thinking about using a service that does AI customer segmentation, I'd like to discuss some of the stuff that you'll see. So many people are showing up and it's like their site's just got problems. A lot of people come in and they're just limping. Okay, let's just get you not limping.

0:31It's all killer, no filler. I'm Eric and I am here with the return of Jordan Gordon. Jordan Gordon-Levitt, I called you today. I don't know if you've ever got that nickname before, but I thought that was pretty good. And usually I have a clear idea of what we're talking about. And it's been a while actually since you've been on the podcast. And today I'm just going in blind because you say you have a good topic. Let's grip it and rip it.

0:52Jordan Gordon:First of all, so whenever I've Googled my name, Joseph Gordon-Levitt is what always comes up. And I've got to scroll through that to find any of myself on social medias. so uh what i wanted to discuss today was i'm not going to name any names but i want to just discuss the next kind of service that i'm just totally going to trash on and say nice right here we go hot takes trashing on a service nameless yeah so we we use on the world's best email retention podcast we you know often bring on the people who who run different services our contacts we I just recorded with someone from Rebuy. Don't know if it's live yet.

1:30Jordan Gordon:You know, and so because we're trying tons of tools, we just have people come and they've got all these different tools. We wrap our heads, we figure them out, you know, and then either one, I have them on the pod or two, I trash them. That's kind of what's going to happen eventually, right? And so if anybody out there is thinking about using a service that does AI customer segmentation, I'd like to discuss some of the stuff that you'll see, you know, with, with services in that space. And also just talk about, from a theoretical standpoint, what, what those kinds of services do. And from that theoretical standpoint, we can also, we can also discuss the, the, it's similar, the, the RFM tool that Klaviyo has, although we're not trashing that today.

2:18Jordan Gordon:But from a theoretical standpoint, what happens when you start doing this segmentation? What happens to your inbox placement? So those are the topics. Because you've articulated over the years a very clear sort of methodology as it comes to how you think about customer lifecycle, retention, all those things. So you're saying when you start turning it over to AI, you got to be aware of the bargain you're making. Yeah, that's right. The Faustian bargain. So your results, broadly speaking, are an intersection between inbox placement and message quality. So, you know, it doesn't matter how good the message is if it's not anyone's inbox.

2:53Jordan Gordon:And it doesn't matter how many inboxes you're in if it's a completely disgusting message. It's just you're not going to get very good results. I mean, technically, inbox placement is higher on the hierarchy because it must be in the inbox to get any kind of reaction. Even bad messages can get some kind of reaction. But everybody out there, don't take it to say that I think, you know, quant is better than qualitative. That's not what I'm saying here at all. So ultimately speaking, I've said this before, the more inboxing placement you get, the better. And broadly, you want to get as much inboxing placement as you can before it starts causing something you would define as a problem, which might be my audience is getting less engaged.

3:33Jordan Gordon:You know, I want a really engaged audience. I'm going to stop it there. Or, of course, I'm in spam, right? Those kinds of things that would cause you to, you know, to not want to beef up your inbox placement. And so from this pure theoretical standpoint, before we go into any specific tools, when you start segmenting, what you do is ultimately decrease inbox placement. Of course. Right? And so if we go back and look on some episodes that I had on the World's Best Email Retention Podcast, we did American Eagle versus Everlane. We did Urban Outfitters. And those guys were emailing me like 1.5 times a day.

4:14Jordan Gordon:Right? If you're on any large fashion brands, you're getting a lot of emails. So I would say they are not doing a lot of segmentation. So if you get a tool that does AI for customer segmentation, you know, the, the, what they say they're going to do is go through and find the most engaged addresses and message the most engaged addresses. So what that will mean in practice is you just message way fewer people. And I ran this analysis today, just before coming in here. And I looked at a client and I looked at people who'd been on the site in the last 60 days as a share of mailable list revenue for that client.

4:53Jordan Gordon:And 25 % of the addresses that that client was emailing were on the site in the last 60 days. And they were driving 75 % of the revenue, broadly speaking, right? So it's like obviously like an 80-20. You've got this small group of people that is driving 75 % of the revenue. Awesome. But also that small group, that 25 % was driving 50 % of the clicks. And so sometimes people shop in journeys, you're going fishing, you need to tease them in, they got a click, hey, the clicks triggers the browser band in and all this stuff happens. Those clicks are important. So you can't just walk away from 75 % of your audience.

5:41Jordan Gordon:And so these tools that purely, I mean, we do, our segmentation focuses on addresses that are likely to drive traffic. But if you go too hard down that path, what you do is you don't get enough inbox placement, which means you don't get enough opens. And if you don't get enough opens, people fall off your list and your list just starts to shrink. So these tools in in general, that really focus in on engaged addresses, they will shrink your list. They will kind of decrease a bunch of click traffic that will also decrease what's going into all of your other funnels. And it kind of just, it just really mirrors the whole thing.

6:19Now, are these tools black boxes where you can't, you can't sort of say, hey, like shift it back towards being more aggressive? Because I imagine brands can see that their like number of mails sent goes way down.

6:31Jordan Gordon:Yeah. And if you're watching what's going on, You'll notice that. But you remember when we went through this whole thing for these services that email people who've never subscribed to your service? We had this big toss for those kinds of services, not naming any names. And so, yeah, of course, if you're looking and you know what to look for. So after we dealt with enough brands blowing up and we knew what to look for, we could tell right away if those things were causing problems. Spoiler alert, almost always. And so, sure, we can look at that. But if you're if you are Joe Ecom and you're just trying to get by, right, you're just trying to increase units for you for this great thing.

7:11Jordan Gordon:And email is not the number one thing on your mind. You might never notice that this thing is actually bleeding your program. Choking you a little bit like a black belt jujitsu match that I know you love so much. That's right. Yeah, me and Hoist Gracie. So, okay, look, that was the theoretical framework. And I'll say I will loop in that theoretical framework. Also, there are tools out there that are not total garbage. But you also must think about that. Like with Klaviyo RFM, right? That does zero in on specific addresses. And you would want to use that in conjunction with your campaigns. You know what I mean?

7:51Jordan Gordon:And so you can use that to like, you can feed different flows and you can have much more complex rule sets. You can pass things around and then also use campaigns to make sure you get that blanket inbox placement to keep getting the opens and the clicks you need to kind of just to just keep your retention system firing. Now, here's the in practice. If anybody out there is thinking about using one of these tools that does AI for customer segmentation. I saw a few at ShopTalk, I'll tell you. Yeah, yeah, yeah, yeah. We had this like situation. It's always this way. Oh my God. I hope you don't mind the hot takes.

8:26Jordan Gordon:It's always this way. We're like, we go through and we're like, this is, this is garbage. Why, why did all these suppressed addresses get unsuppressed? So the system goes in and unsuppresses a bunch of suppressed addresses that the AI says are like high potential addresses. And we're like going through and eyeballing these things. And it's like, you know, clearly spam email addresses and just like, and go through and like this address has never, ever, ever engaged. There's been no measured engagement. I mean, essentially, like the machine is garbage in, garbage out. And it's just like spewing garbage back into the list.

9:08Jordan Gordon:And we're having these like total PTSD from the days when we were dealing with tools that message people who'd never subscribe to your brand. Right? Oh my God, all this garbage is being put back into the list. and um yeah we get on a call with one of the account teams and my number one brie who's brilliant was talking to the guys and they're trying to pin us up on it being like oh no no we're we're going for the high quality addresses and going through no no look at this this is garbage this is garbage this is garbage this is garbage and saying oh we don't always get it right well look you're never getting it right this whole thing looks really bad and then of course I've been in these situations with these really, really, really garbage tools where they're trying to pin it on you and they're trying to frame it so that it's your fault.

9:58Jordan Gordon:And it's like, oh no, you've got to use our system. What you're doing is wrong. And it's like, she just started laughing because it was so obvious. And someone's kind of laughing at their account team in the call. Following suppression tools or suppression list seems to be like a cardinal rule that these tools would have to build in, right? It's actually unsuppressing suppressed addresses. Yeah, that doesn't make any sense. Because the AI says it's a high potential address. Who knows why? Oh yeah, you made the mistake in suppressing them in the first place. AI knows better. That's right. We made the mistake.

10:28Jordan Gordon:Somebody who thoughtfully suppressed this list was wrong. And we're going to let this AI do it. So we spent some time thrashing on this anonymous tool. What are some other tools that you actually find a lot of use in in the Klaviyo environment? Yeah. Some totally unrelated tools or some related - Yeah. Well, first of all, is there anything else you want to say about turning the Faustian bargain brands make when they turn over their segmentation to smart AI because it can really send you some bad signals? My assumption is at some point, someone's going to crack this, and segmentation won't be as big of a part of what we do.

11:10You know, and segmentation is like, it's not the funnest part.

11:14Jordan Gordon:You know what I mean? The funnest part is making cool messages, for sure. I'll just say I don't think anyone's cracked this yet. Like, clearly, this is a popular product that I've seen people use. Clearly, people haven't cracked this yet. Or you haven't seen it, to be fair. Maybe there are brands, CDPs out there, essentially, that have cracked this. Maybe the ones at Shopify were telling the truth. But you've got to be buyer beware when you're letting AI take the reins. Yeah, for sure. And like, I mean, so look, tools in the space that we like, we're super into Sonar from Triple Whale. It's not the same.

11:50Jordan Gordon:It's not AI segmentation. What it's doing is it's increasing inbox placement by capturing more addresses. So if we leave the fact that this tool is garbage, if we leave that hot take away and go back to the theory, okay, well, the only reason you'd want to decrease inbox placement is if you are having trouble with your reputation. Otherwise, you want to increase your inbox placement. So what is something like Sonar or Black Crow does this as well, these tools that extend cookie length and grab, however they do it, they go in the session and they, hey, this is the fingerprinting devices, whatever it is they're doing, and they get more site abandoned emails sent, more browser abandoned.

12:36Jordan Gordon:More intent. They're able to ring more intent from the rag. Those are awesome because they increase inbox placement from addresses that have shown high intent. So your aggregate engagement goes up. While your total email sent goes up, you improve your reputation and you improve your revenue. That's what you'd want to do. You would not want to go and find a tool that's just going to crater how many addresses you send to, even if it worked. Even if it worked. I mean, I've never seen this stuff work. The thing that I always see work is find your safe send size, your safe audience, right? And give them one clean funnel.

13:14Jordan Gordon:Let's say you're like, oh, we're going to start micro-segmenting all these groups. Well, the more you segment groups, the more messages you need, right? Because like just, for instance, splitting up by male and female is an obvious example. If you segment by male and female, you need a message for males and a message for female. Or what's the purpose of the segmentation? So the best thing you can do is push everybody through one clean funnel and then one clean funnel with your automations. And then, yeah, then you can maybe get cute if someone does a category abandon or something like that after that's all set up.

13:46Do we use any tools for A-B testing on the CRO side?

13:50Jordan Gordon:Yeah, we use Shoplift on the CRO side for our testing. That's our official tool. I went to the suite party of my friend Drew at IntelliGym's at ShopTalk. It was cool. I feel like it's a pretty interesting tool. It has some pretty good use cases. You should check that one out. No, we do. We have a client that's on IntelliGems right now. They've just - Oh, nice. They've taken the trial and no complaints so far. I'll say that. We might have someone from IntelliGems on the world's best email retention podcast in two months. Who knows? Oh, nice. You should. And then I'm just curious from a personal level, how's it going?

14:24I know you've been, you know, you recently, I guess it's been, maybe it's been over a year now where you've been handling both CRO and retention in the Pilothouse ecosystem. How do you feel about having such a broad purview?

14:35Jordan Gordon:I'll just tell you the stuff that we're thinking about. So one, like super key findings since I've been involved, so many people are showing up and it's like their sites just got problems, right? And so at first we were like, hey, we're going to do all this super classy stuff and lots of split tests. And we do do that. But a lot of people come in and they're just limping. And so a lot of the work we've been doing is like, okay, let's just get you not limping and then we'll start getting cute. So that's just like a key finding. And so, you know, I came here from really big brands where the site was already working when I got there.

15:12Jordan Gordon:One of the service levels that we've been, that we're, I'm just like right now working on this with, with our executives. And it's, it's going to be, of course, I'm sure there'll be an episode for this on the World's Best Income Retention podcast where I'll sum it up for the audience to try at home. but there's like 15 you know really solid life cycle touch points i can't spout them off the top of my head but of course like your top landing pages your your your pop-up home above the fold home below the fold uh site one browse one cart one check out one uh welcome post-purchase uh category above the fold pdp cart checkout recommendation set or your discovery set Turns out you can pretty much.

15:54Jordan Gordon:Pretty much I can. Yeah. So look, we got 15 of these, right? And it's like, there's certain views that you take on these 15 and, you know, views would be, for instance, structural, like a funnel view. We talked about that before in the fall, but also like the persona view. Are you, are you speaking to the right, are you speaking to the right people in the right way in there? some technical discovery, et cetera. Half a dozen views. So what you should set up at home if you're in the audience trying to figure this out is you need to catalog these kind of, these top touch points and actually grab every single touch point.

16:32Jordan Gordon:This is like very arduous and it's just, but it's one of the things you got to do is catalog each of those touch points and then you've got to make a call on each view. On every single view, are we good or not? are we good or not on the structural view the path that people go through are we good on personas are we good on brand are we good on etc it's kind of like now that we are bringing together uh now that team is helping bring together all of these channels it's just been clear as we've been working with more and more brands that this is the one of the biggest things that's just laying on the floor waiting for someone to pick it up right is is going through those these key touch points and making sure that they match on all of your views.

17:15Big thing to pick up.

17:16Jordan Gordon:It's a big thing to pick up. I'll give you an example. I will give you an example. This is live. This is live from kind of what we're doing. Imagine a pet food brand, premium pet food, and the customer persona is a mom. The mom is buying it, and the mom is like someone who's looking to spend an extra$250 a month on pet food. and you go to the site and a home above the fold says free package of kibbles with each thing. It's like, whoa, that's not what we're talking, right? That's not what we're talking. That's not what's going to close the deal when mom is thinking, I want to get three extra years of life out of my pet.

18:00Jordan Gordon:That's what mom's thinking when she's spending 250 bucks extra on pet food. mom is not thinking about a free bag of kibble. So there's this clear life cycle touchpoint core touchpoint disconnect. And you just have to go through and catalog them all on a scheduled basis because, but especially when you've got a big team, you'll, it's funny. It's like these touch points, they just end up changing. Someone's like, Oh, Hey, you know what? We don't do that pack anymore. And so then someone goes in and remakes the page for that pack, but what they remake doesn't match some other kind of feature or benefit or whatever rule.

18:35So these touchpoints,

18:37Jordan Gordon:even the automated touchpoints, they walk away from you, right? This service level would be kind of like a little bit of a glue between landing page, website, and Klaviyo touchpoint. Landing page, website, pop-up, and email, and how all those touchpoints interact. I think we've been talking a lot on the podcast about Andromeda and essentially having your key customer avatars, ideas of who your customers are and knowing where in the life cycle they are. So being able to manage all that seems Herculean, but it's what brands have to do, right? To meet customers where they're at. And again, that's the key finding is like everyone, they show up either one, they're limping.

19:21Jordan Gordon:It's like, okay, let's help you limp. Or two, this is just not what's getting done. so like i i'll say to everyone out there in audience land it's like if everybody owns it nobody owns it and that's the thing about this that's the key thing is like we're talking about it's all post-click although you know that's because we're not that's because we do post-click but of course like you just said the advertisement's part of that too the ad is obviously part of the life cycle and so if you're saying oh the ads person's going to make sure that this matches and then the and then whoever's doing the landing page is going to make sure it matches and then whoever's in the email is going to make sure it matches.

19:52Jordan Gordon:And there's not someone whose role it is to just go through and make sure that whole thing matches like spoiler alert. It's not all going to match. Yeah. Well, let's leave it at there. Uh, remind everyone to go subscribe and download the world's best email and retention podcast to a burp. Thanks for the, the in-person appearance and buyer beware on AI segmentation tools. You definitely put, uh, put them on blast. Maybe, maybe we'll get some, maybe we'll get some who will hear this who will be like that's not at all what our tool does and they'll they'll come explain why why their tool if anybody wants to reach out to us and show us why their tool is awesome i i'd love to i'd love to hear it please contact me uh you know come to the come to a pilothouse.co fill out the loot form and i'll speak to you directly but risk getting anonymously flamed on the ddc podcast well make sure you can contact me and tell me your tool works and then have a tool that doesn't work there you go simple as that thanks jordan

20:55thanks for listening to today's episode if you're not getting the d2c newsletter you can subscribe for free at direct to consumer.co and if you want to learn more about pilot house's all killer no filler services take off to pilothouse.co i'm eric dick and this has been the d2c podcast we'll see you next time

21:20Thank you.