In short
Podcast Notes: Do This, NOT That: Marketing Tips with Jay Schwedelson
Episode Title
SPECIAL SERIES ==> Your Attribution Data Is Lying to You 🤥 <== | BATHROOM Break #95 COLLAB: The Marketing Millennials + Do This, Not That
Episode Overview In this episode, Jay Schwedelson collaborates with Daniel Murray from The Marketing Millennials to discuss marketing attribution, emphasizing the need to treat attribution data as a directional compass rather than an exact GPS. They explore practical strategies to validate data, such as holdout groups and geo-testing. The conversation also briefly veers into a light-hearted debate about the athleticism of Olympic curlers.
Key Concepts
- Attribution in Marketing
- Understanding Attribution: The episode posits that many marketers incorrectly rely on attribution data as a precise tool, often leading to misguided strategies.
- Last-Touch Attribution Critique:
- It is criticized for oversimplifying the customer journey and assigning undue credit to Google Search, which often represents the last point of contact before conversion.
- Modern consumer behavior shows that research typically occurs before a user reaches a site, thus inflating the importance of last-touch data.
- Practical Strategies to Measure Effectiveness
- Holdout Groups:
- Definition: A technique where a portion of the audience (e.g., 10%) is not exposed to marketing efforts to assess the real impact of a campaign.
- Benefits: Helps determine the true lift or effect of marketing initiatives by comparing the performance of marketed individuals against those in the holdout group.
- Geo-Testing:
- Strategy: Focus marketing efforts in specific geographic areas to measure effectiveness through targeted campaigns (e.g., billboards, radio ads).
- Importance: It can reveal localized effectiveness, helping marketers understand where their strategies are driving engagement.
- General Insights and Recommendations
- Red Flags in Attribution:
- If one channel claims 100% of the credit for conversions in a multi-channel marketing environment, it should prompt marketers to reassess their attribution approach.
- Simplified Data Gathering:
- Suggested tactic: Use a "How did you hear about us?" field in forms to gather insights directly from consumers, allowing for better allocation of marketing resources.
Notable Moments
- Humor in Marketing: The conversation takes a humorous turn regarding Olympic curling, with Jay and Daniel debating the relative athleticism of curlers compared to other Olympic sports.
- Personal anecdotes: Jay shares experiences related to altitude sickness, providing a relatable context to the discussion.
Conclusion This episode emphasizes the need for marketers to rethink their approach to attribution and explore more nuanced, effective methods of evaluating their campaigns. By implementing holdout groups and geo-testing, marketers can obtain clearer insights into their strategies and ultimately drive better results.
Additional Resources
- Jay Schwedelson's Social Media:
- [YouTube Channel](https://www.youtube.com/@schwedelson)
- [TikTok](https://www.tiktok.com/@schwedelson)
- [Instagram](https://www.instagram.com/jayschwedelson/)
- Daniel Murray's Social Media:
- [LinkedIn](https://www.linkedin.com/in/daniel-murray-marketing/)
- Further Reading:
- Pre-order Jay Schwedelson’s new book: [Stupider People Have Done It](https://www.amazon.com/Stupider-People-Have-Done-Marketing/dp/1637635206) (out April 21, 2026), with net proceeds supporting cancer research.
Tune In Join Jay and Daniel for more marketing insights in future episodes of the "Bathroom Break" series, where quick tips and strategies for marketers are shared in a fun and engaging format!
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Chapters
Tap a time to open that second in VOOn the Road and Altitude
0:45 to 2:44
Jay and Daniel discuss their experiences with altitude sickness while traveling.
“Jay's literally in like this puffer jacket and he has like a hot toddy in his hand with a beanie on.”
Understanding Attribution in Marketing
2:44 to 4:31
Discussion on the importance of understanding attribution in marketing strategies.
“But there are some tricks to actually getting attribution sort of right.”
Using Holdout Groups for Marketing Insights
4:31 to 6:22
Exploring the use of holdout groups to better understand marketing effectiveness.
“And I do believe with all the LLMs and all the AI tools, and they all will eventually start taking ads.”
Geo-based Attribution Testing
6:22 to 8:11
Daniel shares insights on geo-based attribution testing to measure marketing impact.
“Yeah, I think two things that I used to do is one is geo-based.”
Thoughts on Winter Olympics and Curling
8:11 to 10:37
A light-hearted discussion about the Winter Olympics and curling's unique position.
“I am because it's something I didn't grow up in cold.”
Transcript
Automatic transcript. May contain errors.0:07Welcome to a new special series called The Bathroom Break. That extra 10 minutes, you either have to listen to marketing tips or use the bathroom or both. But I don't recommend both, but that's your choice. This collab is going to be super fun. We have Daniel Murray from the Marketing Millennials and me, Jay Schwetelson, from the Do This Not That podcast and subjectline.com. Each episode in the series, we are going to go over quick tips about different marketing topics. And if you want to be in the bathroom, fine, just don't tell us about it. Thanks for checking it out. We are back with another bathroom break.
0:42And for those who can't see the camera right now, Jay's literally in like this puffer jacket and he has like a hot toddy in his hand with a beanie on. And he's in some cold place, but he won't tell me where he is because he's on the road doing some weird stuff. But like, he's really, it looks like he's going to get bundled up. I don't know what he's doing. You might be the stupidest person alive. First of all, I'm on the road in Colorado. and I'm not bundled up. I don't have a hot toddy, although I would have one if I knew what the hell it was. But I will tell you this, it's two hour time change from where, you know, South Florida, it is crushing me at like 7 PM.
1:20I'm so tired and they scare the hell out of me. I drink so much water though. Like you're going to, I've gotten altitude sickness before and Colorado is like high and I don't want to throw up. So I'm like drinking water and taking Advil. Like, do you get any of that when you go to high altitudes? Yeah, especially, I've been to, I went to Salt Lake City once and I went up to Park City and just a quick ascent, I'm like, I needed to get that tea. There's special tea that you're supposed to drink to help you with that. I've got some bad headaches in altitude. I feel like we're weak being on sea level.
1:58I'm very weak. It's funny, I was at a meeting in Utah and I didn't realize the altitude was like 5 ,000 or 6 ,000 feet in the middle of the meeting i had this crashing headache i went to the restroom i threw up in the middle of the meeting like i had and then we had to go back to the meeting i didn't know what was going on it was terrible it was so funny i was with my friend one time and we were walking in this place and that's like 10 years ago and he's he's walking and he's getting so tired and he goes what's the altitude here and i'm i'm like turn around and he's like it's he i'm like it's sea level dude he was just like sweating because he was just so tired there's an ocean behind him I was dying.
2:39That's amazing. All right. So let's get into what we're talking about today. So attribution is on a market on the planet that doesn't focus on attribution. But there are some tricks to actually getting attribution sort of right. But in general, attribution measurement is complete garbage. Daniel, are you on the garbage train? Yeah, I think I've always said this. and i've been a marketing ops person before i decided to like go full content and social but um i always said attribution should be seen as like a compass where you directionally can find some answers but don't use it as like a gps and a lot of people use it as i need to turn right turn left turn this way turn that way to get to my destination because that's what attribution is telling me.
3:29And I think that is the wrong way to use. Directionally, it could tell you these channels might be doing better than this channel, but I don't think people who use it just as the roadmap, it's the wrong thing to do. Well, especially last touch attribution, right? I mean, are you on the train that last touch attribution is the worst before we get into ideas on what you should be doing? Yeah. I mean, the modern day buying cycle is most people have done the research before they get to your site so um and and the problem is that i've seen it most companies have been at it is when they do that they automatically put more money in google search because that's usually like the last touch attribution because someone's branded searching you or um they're searching you online or they come in straight to your website but someone there had to be an action before they came to your website and people don't want to dig deeper into that and they just say, oh, here, executive team, look, we can put more money in Google because Google is doing so good for us.
4:31Yeah. And I do believe with all the LLMs and all the AI tools, and they all will eventually start taking ads. I don't care what Claude says, that there's going to be more last touch attribution assigned to these AI tools, even though it's the same stupidity as assigning it to Google search. And so there's one tactic that we do a lot with our clients that I don't think enough marketers do, which is holdout groups. So what a holdout group is, let's just use a round number and say you have 100 ,000 people in your database, right? And you have a whole campaign. You're pushing whatever it is you're pushing to get new demos of your new SaaS product.
5:05Or you could be a consumer product where you're trying to push some sort of new offer or whatever. And you could do this with 5 % if your file is big enough or 10%. And you take 10 % of your database and you hold it out. You don't allow it to get any of the emails. You don't allow it to be used for retargeting or remarketing. You do the best that you can to have those 10 % to not be exposed to any of the media that you are aggressively pushing out there. And then what you do is after the campaign concludes and you take all of your net new customers that you've got, and you can do it over a window of time.
5:40So it's okay. I'm going to take all the customers we got 30 days post the end of the campaign and all the customers we got during the campaign. And then you match it back to your complete database. Right. Um, and you say, okay, the people that we marketed to versus the holdout group, what was the actual lift that we got by doing the marketing or was there no lift? Did our marketing have absolutely no impact because the holdout group performed exactly the same and the delta between your holdout group and your, and the people you marketed to is your real performance. Now people, it's not an exact science.
6:15There's a lot of moving parts there, whatever, but it does give you a much better view on how effective your marketing is. I don't think enough people do that. Yeah, I think two things that I used to do is one is geo-based. So what I would do is pick a geo that and just concentrate, let's say, out of home there or radio there or connect to TV there and just see, like, get a baseline of what it is now and for the last year and see if there was a lift in that area with, like, doing that one tactic. and there isn't a lift you will know but usually you could tell there should be an incremental like branded search lift or some lift um that's one one thing i i try i've done a lot when i've done is a geo-based um attribution testing because that will really tell you if something is lifting up in that in that area it's more expensive though so that's like the the negative part of geo testing but it's helpful to see like does my tactics work in that area or not yeah and i think just in general if internally you're assigning a lot of value to one channel uh and you're doing multi-channel marketing or multi-platform marketing that should be your red flag like wait a minute we think that you know we're driving all this on instagram but really simultaneously we're running campaigns on Facebook, we're doing email, we're doing this all to the same people.
7:42It's silly to just assign complete value to wherever it is that ultimately the person's converting in that last moment. And a lot of people now are going to multi-attribution models. They have multi-touch attribution. They have media mix models. They have incrementality testing. they have like a bunch of things is like zero in on what and that's actually a better way to do it to have multiple multiple things going at one time because facebook's a wall of garden google's a wall of garden um they're gonna make it look better because that's how you spend more on those platforms so actually having multiple things going at a time and comparing drives but the easiest thing that i liked like the easiest thing to do is just add like where did you hear about us on your form that's like a simple thing to just get someone to like at least start getting directly and that's how a lot of people we used to do that and we used to get oh we heard in your youtube channel we heard it in this and then we we could now invest more into those channels because they're actually telling us where they're hearing us from so it's not exact science but it works sometimes I'm curious about something since I'm in cold weather right now.
9:00Are you an Olympics guy? Are you all in on the Winter Olympics? I am because it's something I didn't grow up in cold. So it's so interesting to me how good people are on these things. Because I know how hard it is to ice skate. And I know how hard it is to ski. And when I see these people going 90 miles an hour, I just think how I would die if I went one second off that. I just thought about these drops are insane. I read the funniest tweet the other day. It said the winter looks so crazy because there's people going around with knives on their feet and jumping off of hills. And then there's curling.
9:46I feel bad for the curling people because they always get so much hate or whatever. Here's my problem with curling and all of it. I watch the curling people. And then I say, I don't believe those are the best curlers in the world. Because it's not like Michael Jordan ever tried curling. These are just the people that tried curling. So I know they're very good. I'm sure they worked very hard. But they're not probably the best in the world at it. It's just they're the only ones who did it. Smart. I mean, that's the way to do it. Capitalize on a market that is not being able to do it. There you go.
10:23I feel like you would be a good curler. Thank you. Thank you very much. I appreciate it. It's the nicest thing you ever said. Well, with that, I hope everybody goes out there in curls. If that's what you say, I don't even know. What is that? What is that? Actually, I don't know. I don't know. I think it's curls. Let us know. Leave it in the leave us a comment. We'll see you at the next one. Daniel, come on, man. I got to get back to work. Get out of there. All right. While he's still in there, this is Jay. Check out my podcast, Do This, Not That, for marketers. Each week, we share really quick tips on stuff that can improve your marketing.
11:05And I hope you give it a try. Oh, here's Daniel. He's finally out. Back from my bathroom break. This is Daniel. Go follow the Mark and Millie podcast. But also tune into this series. It's once a week, the bathroom break. We talk about marketing tips that we just spew out. And it could be anything from email, subject line to any marketing tips in the world. We'll talk about it. Just give us a shout on LinkedIn and tell us what you want to hear. Peace out. Later.
From the publisher
Marketing attribution often feels like a guessing game where everyone just ends up giving credit to Google Search. Jay Schwedelson connects with Daniel Murray from The Marketing Millennials to break down why you should treat attribution data as a directional compass rather than a precise GPS. They discuss practical ways to validate your data, including the power of holdout groups and geo-testing, plus a random side conversation about whether Olympic curlers are actually the best athletes in the world.
Follow Daniel on LinkedIn and check out The Marketing Millennials podcast for sharp, no-fluff marketing insights. Subscribe to Ari Murray’s newsletter at gotomillions.co for sharp, actionable marketing insights.
Best Moments:
(01:50) Why attribution should be viewed as a compass instead of a turn-by-turn GPS
(02:22) The problem with giving Google Search all the credit for last-touch conversions
(03:10) How to set up a holdout group to measure the true lift of your campaigns
(04:09) Using geo-based testing to see if specific channels are actually driving growth
(04:45) Why it is a red flag if one channel claims 100% of the credit in a multi-touch world
(05:15) The surprisingly effective "How did you hear about us?" form field strategy
(06:40) A hot take on whether curling athletes are actually the best in the world
Check out Jay’s YOUTUBE Channel: https://www.youtube.com/@schwedelson
Check out Jay’s TIKTOK: https://www.tiktok.com/@schwedelson
Check Out Jay's INSTAGRAM: https://www.instagram.com/jayschwedelson/
ㅤ
Pre-order Jay Schwedelson’s new book, Stupider People Have Done It (out April 21, 2026). All net proceeds are donated to The V Foundation for Cancer Research—let’s kick cancer’s butt: https://www.amazon.com/Stupider-People-Have-Done-Marketing/dp/1637635206




