Bonus: The Hard Truth About Marketing Measurement with Constantine Yurevich of SegmentStream

29:17Constantine Yurevich:For geo holdouts, it's asymmetrical. It's always asymmetrical because the way how this confidence interval is calculated is different. It uses the same approach as in medicine. We use like placebo effect. We just reshuffle all these regions again and we just like run the geo holdout test again, and we see that there are some regions that were in test now in control, and we see that we've been continuing showing ads, but now we're again measuring incrementality, which is 7%. And you say, okay, even if in AA test we can measure 7 % incrementality, can we really trust 9 % number? Yeah, you cannot.

29:53Constantine Yurevich:So that's why we have, and so we cannot, in geoholdout tests, we cannot use this observation number. We should always look at the confidence interval and adjust this number and understand worst case scenario, best case scenario, neutral scenario, etc. And in many cases, what we see that like everywhere, everywhere on social media, I see that ROS is underreported. What I usually see is that ROS in ad platforms is usually overreported. So it's absolutely different story. And again, it depends who wants to push different narratives. I'm sure the platforms want to obviously, you know, take as much credit as they can.

30:36Where does SegmentStream fit in these three categories of marketing measurement?

30:42Constantine Yurevich:Oh, it's a good question. I would say, so we use everything. But the idea is that we don't put all our eggs in one basket. And this way we do not push like heavily any specific approach because our whole business doesn't depend on specific approach. Our whole business depends on expertise and very deep engagement with each client to find the best possible way. So that's why, for example, we have geo holdouts. And actually, I was a big believer in this methodology. Like for me, like, oh, it's so genius. You can just split regions and you can. And this is first like causal part when you understand that you make an action and you have a result.

31:24Constantine Yurevich:But then when I started deep in research with real data and with statistics and I understood that, oh, we invested like one year developing this module to the platform and now it has such a limited application where we can just answer some yes or no for very specific channels some clients do not even care about. But there are some companies whose whole model depends on this methodology. So far, I can tell you what I've observed so far in terms of what really works and what I would be doing for my business. If tomorrow I'm going to be running a D2C store, importing some products from China with high tariffs.

32:10Belts. I hear belts are really big. Belts are the new thing. So let's make it a belt business.

32:14Constantine Yurevich:Yeah. If I'm going to make a belt business and I understand that margins are very important for me, tariffs hit hard, I need to be profitable, I need to grow as much as possible. So first of all, I will always... So a lot of people say last click attribution doesn't work. Last click attribution is biased. You should never trust last click. And like lots of posts about last click attribution. My question is, why do we even talk about this? Why do we even talk about last click attribution? Why don't we talk about first click attribution? So why there are no posts like first click attribution is biased.

32:52Constantine Yurevich:You should not trust first click attribution. You should use MMM. So everywhere I hear last click attribution is like old school. People are losing money. You should switch to MMM and do holdouts. Why no one is writing about the same about first click attribution? Good question. It's funny. I like first, first touch. I remember all the platforms were always wanting when I, when I was in my marketing days, more than with the media here, they were always wanting me to, uh, trust impression tracking or, you know, when I couldn't see good results, they were always trying to say, but look at the impression tracking.

33:27You can see that this is working. So I think the platforms want first touch attribution to be a thing, but I think it's always thought that the last click is

33:36Constantine Yurevich:Yeah, okay, not first touch, first click. Let's forget about impression. Hopefully everyone knows that impression-based attribution is complete scum and it's heavily abused by ad platforms just to show impressions to whoever entered your website from different channels and then to credit the contribution, like claim the contribution. Let's put aside impressions. But why no one is posting first click attribution? Everyone is writing about last click attribution. It's super primitive. It existed for like 20 years. You just need to track one click and then like stitch it the same way, like you stitch last click to a conversion.

34:20Constantine Yurevich:Why there is nothing about first click attribution on the internet. Is it too hard because it requires all that stitching to the final click? It works exactly the same way when you, even easier. Once you have a click, you can ignore everything else that happens and just have a conversion. With last click, you need to re-achieve it to the last click. So it's much more complex technology, in a sense, than first click. Well, you tell me. I'm not sure. Because it works. And it is very simple. Who's going to pay you like 50K or 100K for MMM or geo holdouts or all these data signs, et cetera, if you can just use first click?

34:59Constantine Yurevich:If you just have an average D2C brand who is not very big, right now they invest 100k a month, they can just implement first click. And they will see actually a picture which is quite grounded in reality. It's a direct empirical observation and it's very interpretable. You can explain what you are seeing. You can see that actually there was... Of course, first click and true first click are different things. Because even now, if you're going to go to your analytics, you can see a lot of first click attributed to email just because of cross device, et cetera. But still, it's like, in a sense, it's one of the best attributions that you can find right now.

35:49Constantine Yurevich:Like, and what I would be doing immediately, I would do first click. What we do in segment stream, we understand that first click and true first click are very different things. So what we need to do is to build a very extensive identity graph under the hood. So that whenever you send a D2C newsletter to one device and another device, we have all the proper parameters everywhere, like user ID, email hash, and stitch it so that we understand that this is the same user across different devices. So there are many methodologies how this could be done. and it requires some work, not such a hard work as you need for traditional MMM, to implement proper infrastructure so that we can start collecting this data.

36:34Constantine Yurevich:And at some point, we see that without identity graph, and with identity graph, we see different numbers. Sometimes they differ two times for some channels. But again, it's a clear empirical evidence grounded in observation that now it was the same user with the same user ID, with the same email, with the same phone, with the same IP address, with the same specific parameters that we also add that you can send to a friend. And we know that actually you've sent this link to a friend and we can link you back to your initial click to the ad. But now it's like fully grounded in reality model that can be fully explained.

37:14Constantine Yurevich:And the explanation is quite good. Like this was the first channel that ever brought this user to the website. I like this explanation. Yes, with first click, maybe it's not so easy to explain like retargeting impact and lower funnel impact, but I don't care. Like what I care about is what really brings new, and this is what everyone is talking about, which channels really bring new customers to the website. New customers, exactly. And you can also stop tracking returning customers at all. This is what I would do. I would track purchases only from new customers. I don't care about returning customers.

37:53Constantine Yurevich:I don't want ad platforms to show ads. Again, there is definitely an incrementality in showing ads to existing customers. But this will happen anyways. Platforms will abuse this audience anyways. So I would say this will be a nice bonus. but I don't want to reward any platform for bringing back my existing customers that I can re-engage with in different other ways. Because they're all incentivized to get you to overspend on that bottom part of the funnel, which is I think what we talked about in our last interview, right? Where they're already going to convert anyways. The only, so it's essentially like first click or also multi-touch attribution.

38:32Constantine Yurevich:So we have both in our platform. We, for someone who is really focused on just driving new audience, they need to grow, they need to scale, first touch is perfect. For someone who really wants to make everything like super balanced, multi-touch might be a good idea. But again, some might ask, okay, but first click is nice, but there are some channels that we cannot measure based on clicks. For example, they view our TikToks, they view our YouTube videos, they viewed our podcast. There is no way. Like right now, after this podcast, many people are going to view it, but there is no way for them to click.

39:12Constantine Yurevich:Maybe there is some link to SegmentStream website they can click, but a very small portion is going to do this, while there's going to be a huge brand awareness. So what we found, the best way to measure it is a self-reported attribution. It's another evidence-based attribution. But the problem is that many brands use it in isolation from their main attribution. When I say isolation, they just, whenever, well, imagine like we're going to publish this podcast and in our HubSpot, whenever you submit a lead form, we ask you, how did you hear about us? And they will say, D2C podcast, D2C podcast, D2C podcast.

39:51Constantine Yurevich:So I'm going to say like LinkedIn, Facebook, et cetera. If we're going to use this attribution just standalone, it is also very biased because, first of all, not very small, but not everyone gives an answer. But also answers can be not equally distributed across different channels. Maybe D2C audience is very loyal and they recognize D2C podcast and your brand and they immediately can remember, oh, we saw it on D2C podcast because it brought a lot of emotions and it's high quality content, etc. But if they saw something on Instagram, they might not connect the dots immediately or Google search or something like very obvious that they don't even pay attention to.

40:34Constantine Yurevich:That's why like there is no need just to use this attribution. Instead, we combine this together. We use a methodology which is called reattribution. So whenever someone comes from direct or from organic or from brand search or from any other channel, which is considered direct, we connect their user journey with an answer they've given in a self-report attribution. And imagine someone came from direct and they answered, how did you hear about us? D2C podcast or YouTube. So we have channels which are called donors and contributors. So we never assign contributors to donors, but donors like brand, organic, direct always are reattributed if the answer is different.

41:26Constantine Yurevich:And we see that for many brands, like 90 % response rate, how did you hear about us? But also we bother only about the channels that we cannot track. So there is even no need to add all these options like criteria retargeting, X platform. You don't need all of this. You just need, okay, we cannot... Because I just confuse them. They don't have to know what they even came from. Yeah. You know what you call brand awareness channel. If for you it's YouTube, podcasts, influencers, and TikTok, just keep these options here. Maybe add online search and other. You don't care about all this micro stuff.

42:09Constantine Yurevich:You care only about these major channels where you need to get as many answers as possible if really someone remembers that they saw your ad on YouTube. And some people might say, oh, people do not usually remember. Sometimes they forget and they might give you a wrong answer. But again, if you are doing brand awareness, it's already like keyword is brand awareness. If someone cannot remember how aware they hear about your brand, probably this was no brand awareness. So when you have really good, for example, I remember, do you know the Monday.com platform and a few years ago, like it was crazy.

42:54Constantine Yurevich:They've had all these YouTube ads all the time appearing, appearing. And even if now you're going to tell me how did I hear about Monday.com, I will tell you YouTube ads. like three years later, I can tell YouTube ads because yes, I was annoyed by these ads. I didn't have, we didn't have YouTube premium by that time, but I still remember. So this is brand awareness. I still remember. So this combination of attribution, the best attribution that fits your business model and your goals, identity graph to be able to stitch everything like emails, phones, IP addresses, click ID propagation, et cetera, et cetera.

43:32Constantine Yurevich:and reattribution using self-report attribution already going to give you amazing results from the start. Great results if implemented correctly. After that, I would say don't bother about further enhancement of your attribution. Apply marginal analytics. And we talked about this on our previous podcast and maybe you can show the link to the previous podcast where they can learn in depth what is marginal because now once you have some model that you can trust because you can explain it, you clearly understand how it works. It is grounded in reality. It's like pure observation of the reality. You can now start applying math and some statistics because marginal analytics is already a third method where you have some causal inference and you have math.

44:27Constantine Yurevich:So you start shifting your budget, increasing budget, understanding elasticity of each campaign. But this elasticity is not measured based on some math. It's based on the attribution that is quite deterministic and you've chosen. So that's why there are so many talks about attribution is dead, etc. But in reality, like what I've posted recently, for the last 20 years, we haven't done any progress. This is a hard truth. This is a hard truth, especially for me working. And especially because usually I'm a very big believer in something like I have so much enthusiasm. And whenever I see some new technology, we immediately have a meeting with our product team, data science team.

45:13Constantine Yurevich:We invest time, research, implement. And then again, I see, oh, again, it's a, it doesn't work. It's a fairy tale, like huge confidence intervals. We cannot trust this data is just a marketing shit. And all the time, and for me, it was very, very hard to accept that the best thing we have right now is attribution. It was very hard to accept because like five years ago, I was like saying attribution is dead. We need something new. For me, it's not easy to accept this after investing 10 years in finding something better. but I can say like for the last 20 years with everything I know, like, do you know this Dunning-Kruger effect?

45:56Constantine Yurevich:When like, when you don't know much, you're very enthusiastic, you can sell, you're very confident, like you like show off. But more you know, like more you understand that you know nothing. And like, and this is something that is happening to me after I evaluated all these methodologies with real data, with real clients. and I understood that, yes, there is 5 % of improvement, the research that is being done, but 95 % is just huge budgets that are bumped into social media and informational field to push specific narratives. And the narrative essentially is spend more, spend more, spend more, spend more.

46:37Constantine Yurevich:You don't need tracking. You don't need measurements. Spend more because not everything can be measured. You can use some other methodologies, but you should spend more brand awareness, brand awareness, spend more. No need to measure. So this is an overall narrative, but then you just need to dress it up in very beautiful data-driven solutions, marketing research, different partners, partner vendors that are sponsored by ad platforms. And I was really surprised as there are so many vendors that take money from ad platforms, that subsidize their pilots to prove the efficiency of a specific channel.

47:17Constantine Yurevich:So for me, there's such a conflict of interest. Speaking of conspiracy theories, though, I saw your post about platforms are actively removing the first touch attribution. Why do you, like, speak, like, why, if it is, it's an effective way to see downstream results, why do you think they're removing it? First of all, because, like, let's start with, like, Google. Google is still, like, I would say, even if you're going to implement first touch, it won't give you such amazing results for YouTube, display, demand gen. Like, for upper funnel channels, it still won't give you such amazing results compared to MMM, compared to post view, etc.

48:03Constantine Yurevich:But for paid search, which is a primary driver of the revenue for Google, it might show that actually 70 % of your paid search is just cannibalizing your organic. It's just not incremental. People just click it because you have a Chrome browser, because they just don't want to type a full address of your website. because so it's reality like when when you apply first touch like raws of Google shrinks dramatically so why would you want to have this attribution also it's not very good for optimization because click might have happened like three weeks ago four weeks ago and now you have a conversion it's a huge delay in optimization so that's why like first touch is good to have like more strategic view when you can analyze like longer period of time but if you need to optimize right away usually we tell our clients to separate different channels like upper funnel mid funnel and lower funnel based on the rows they see based on first touch and reattribution and identity graph and then inside these three portfolios already optimized based on multi-touch attribution because it's much faster.

49:20Constantine Yurevich:Yes, we now understand the funnel level, but still you cannot wait three or four months to make a decision when your first click actually is activated. So that's why our goal is as experts, as consultants to find the best approach to work on tactical and strategical level. So I would say this is the main reason why first touch attribution was removed completely from most of the platform because it just shows you much worse. Your rows is not as amazing as the rows that you're going to see based on last click. And also in each platform, there is no deduplication. Whenever someone clicks on both Google Ads and Facebook Ads, every platform is going to claim the credit from this touch.

50:07Constantine Yurevich:But there are more. The question is why Google and Facebook launched this open source by easy and MMM frameworks. If those are so easy, just need to pull costs, etc., instead of launching them as a software as a service within their platforms. Why? Why they just haven't integrated inside a platform just to be able to measure the effect? Because they don't fully trust it, maybe? Because if something is not open source, you can use it at your own risk, etc. Now you have responsibility. And if you have responsibility, you show that actually this channel is incremental and people start spending more money and two years later they find out that it was complete scum, you're going to go to the court.

50:52Constantine Yurevich:That's why it's much, much better strategy to launch this open source than to have this like army of individuals brainwashed by different stories and sometimes kickbacks and these pilots that are subsidized by ad platforms. But now you have lots of individuals, lots of influencers, lots of small agencies, lots of vendors that even if someone going to be going to be responsible for something, like there is no responsibility for a big guy. The same thing why like geo holdout testing, geo lift library was abandoned by Facebook. Why? It was completely like no new commits on GitHub for the last like three or four years, because they understood like, it just doesn't make sense to run this geo.

51:41Constantine Yurevich:You can make so many mistakes. It's such a time consuming methodology. You can lose money. You cannot run your ads. It's so risky. But at the end of the day, after the concluding the experiment, you have some incremental with huge confidence interval. And almost every time your attribution results fall within this confidence interval. And they just abandoned this library. What's the most valuable thing that most advertisers are using SegmentStream to uncover right now? I would say when we start working with clients, we do not just plug into their website and just show some numbers. We analyze their whole system, how they track their users, how they track conversions, how they track their email subscriptions.

52:30Constantine Yurevich:Do they add proper tracking parameters to all their emails? Because emails are a great way to stitch customers between devices. So we help them to build the whole infrastructure and the whole architecture of proper tracking and identity graph building, and finally building the attribution they can trust. Then, of course, we always encourage them to add self-report attribution. and we also have a very robust methodology of self-report attribution. We never ask them, just add a dropdown on the checkout form. How did you hear about us with three options? Because these three options might be limiting the customer and sometimes you don't even know what findings you will see when they're going to answer and how you will compare this to what you're tracking.

53:11Constantine Yurevich:So for us, it's very important to compare what we track with what customers actually say. And that's why we always encourage to put like a normal text field with a free input. Like you can input anything you want. And then we process this using LLM and categorize into different categories. It gives us flexibility. First of all, it gives us a lot of finding. Okay, what customers that come from brand actually say? What customers who come from organic actually say? Where they came from? And then we can process it into proper categories. And then we can apply reattribution. but also we can learn. Sometimes we find some answers that we didn't know about, like different programs, some affiliate websites we didn't hear about.

53:57Constantine Yurevich:Word of mouth actually is an amazing way to understand what is the strength of the brand. And also it gives us flexibility to retrospectively change these categories if, for example, we miscategorize something. Because if you just put very strict select box, you just have three or five options, which are very strict. There is no other way. And also sometimes people are biased just to choose the first one. They just choose the first one and that's it. So first of all, building this like really good attribution that you can trust with all these technologies that we have, like identity graph, self-report attribution, reattribution, assigning donors and contributors, et cetera.

54:38Constantine Yurevich:The next step, of course, is marginal analytics because every D2C brand cares about actual margin. And people talk a lot about IROWs, like incremental ROWs, which does not exist. But the closest thing we can get to it is to apply marginal analytics. And we've tested even if you're going to take different attributions, like last click, first click, at some point at specific budget level, returns start diminishing. And it doesn't matter which attribution you use. You're going to have campaigns that scale linearly at a high budget, and you're going to have campaigns that scaled really well and had amazing rows with any attribution.

55:20Constantine Yurevich:But then returns started diminishing. Now you have zero incremental value, but you keep investing money. And this is an elasticity problem. And this is the calculation of diminishing returns for every single campaign. And of course, we work a lot with lead generation businesses, but probably this is not for the podcasts that have absolutely different set of problems. Like with e-commerce, everything is straightforward. Someone purchased, you have money. One thing that really helped us is to make a clear distinction between returning customers and existing customers. And we understand that if you're going to focus on existing customers, you might be leaving money on the table.

56:03Constantine Yurevich:While if you're going to focus on new customers, existing customers are going to see your ads anyways. So them. I know it's like, even if you're going to exclude everyone, everything possible in all ad platforms, you will see that existing customers still see your ads and there are still conversions from existing customers attributed to ad platform. You cannot just, you just can't prevent it. The same way some people say, if you use first click attribution, your retargeting will be not evaluated properly and you will not, because like if we have real first click, retargeting would always get zero value because retargeting cannot be the first channel.

56:40Constantine Yurevich:But just because our reality is so complex and unperfect, anyways, you're going to see a lot of customers who come first click from retargeting just because of the complexity of the world. So it's by design we use these flaws of the system just to be able to find the model that can utilize these flaws. And it's ultimately about finding those new customers. Yeah, but coming back, I would say still, even though we have all the sophisticated technologies, geo holdouts, marginal analytics, automatic integration, applying costs, we even have AI agents that completely manage your budget. So you just integrate attribution, reattribution, marginal analytics, and you can just click apply button.

57:27Constantine Yurevich:And you don't even need to go to add platforms, make bidding strategies, change budget, like everything going to be done by AI using API. But the biggest thing still that our customers find valuable is our expertise. So we try to cut through bullshit as much as possible. Yeah, like some people say that the easiest way to make money is to lie to people that love to be lied to. So they really want to hear the lie. And some people just tell them the lie. and everyone is happy and it works like that. But our idea right now is to work with people who want to hear hard truth. And even though it's hard in the very beginning, in the long run, they're going to win.

58:12Constantine Yurevich:It's also before we also try to sell truth to people who want to be lied to. It's a horrible business. Now we don't do this anymore. That's terrible. Yeah, yeah, yeah. So, well, if you're in our audience and you are ready to hear the hard truths, you've got to go check out Segment Stream. And just, I'll put your LinkedIn, you're always dropping your hot takes on LinkedIn. So you got to follow Konstantin on LinkedIn there. Anything else you'd like our listeners to do regarding Segment Stream? Yeah. So I would say you should perceive marketing measurement as going to a doctor and imagine. but you need to be serious imagine like you have a serious problem like cancer or something of course you wouldn't go to just one doctor and this doctor will tell you like you need to cut it out or you need to have different like opinion from different world views from different maybe you can even go to some shaman to check like you need to have because problem is so serious that when you have this this level of problem, which is usually like when you invest budget, it's core of the business, especially in D2C.

59:31Constantine Yurevich:You need to have different opinions from smart people. You can talk to them and make sure that you don't just follow the story that the best aligns with your beliefs. Otherwise, you're going to fall into confirmation bias trap. And this is something very important for founders to know about because still when you have head of digital who is working for salary, is afraid to be fired, is afraid to make mistakes, is afraid of hard truth, I can understand this. And that's what is heavily exploited probably by all these people who spread these fairy tales and myths. But when you're a founder, when you really see the money in the bank account, when you really invested your time, your life, and maybe your finances into a particular business, I think the truth is very important, even if it's not always very comfortable.

1:00:29Love it. Well, thank you for coming on the D2C podcast today. Get in touch with Konstantin if you want to know the hard truths. Awesome as always. Thanks, man.

1:00:37Constantine Yurevich:Thank you, Eric. It was a pleasure.

1:00:45Thanks so much for listening to today's episode. If you're not a subscriber to our newsletter, you can do that right now at direct-to-consumer, all one word, dot co. I'm Eric Dick, and this has been the D2C Podcast. We'll see you next time.