Your AI-Built Marketing Mix Model Is Fast. But How Do You Know It's Right?

0:04Madan Bharadwaj: Inside the Stack is a custom podcast series from AdExchanger, where we interview leaders in ad tech.

0:16Madan Bharadwaj: This episode of Inside the Stack is brought to you by M Squared. More than a dashboard, M-Squared is a hands-on measurement program, a dedicated analytics team, and a finance-grade reporting that turn numbers into confident decisions.

0:44Hello, everyone. Welcome to the latest episode of Inside the Stack. I'm your host, Andrew Byrd, and today I'm joined by Madan Badwadwaj, founder and CEO of M Squared, a marketing measurement company focused on helping marketers understand what's actually driving business results. M Squared combines approaches like marketing mix modeling and incrementality testing to help marketers move beyond attribution and make more confident investment decisions. And that's particularly relevant now as AI is making analysis faster and cheaper, but raising a bigger question. What is a good marketing decision actually worth?

1:22Thank you for joining us today. Madan, definitely excited to hop into this conversation. And just for the inside the stack and AdExchange audience to know, how would you kind of describe yourself and how you kind of fit into this world of measurement? Absolutely. Thank you so much for having me, Andrew. Fantastic being here. I appreciate the opportunity. My name is Madan. I'm kind of the attribution guy in the industry. For good or bad, I've been doing attribution for about 20 years now. It's my fourth company in the space. Before this, I founded a company called Measured. Before that, I was founding team at a company called Visual IQ.

1:59Before that, I was running measurement and data science at Dentsu and a lot of consultancies in between. So attribution, I know. Anything else? Your mind is just going to worry. So advanced measurement and attribution is what I'm all about. Thanks for having me. M Squared specifically is a unique company in this space. We have what we believe is the best tech platform for measurement, advanced measurement. We combine marketing, mix modeling, incrementally testing through a process called triangulation. We calculate something called multipliers and we generate essentially real time dashboards for attribution reporting.

2:40We call it causal attribution. But what's unique about what we do is that our services model is very unique. All my experiences before M Squared have been VC backed startups. So it's very challenging to offer meaningful expert services to my clients. At M-squared, what we do is we staff an attribution team for each one of my clients so that as opposed to my clients trying to understand my product and my technology, we do the opposite. My team understands their business and all their nuances, applies the M-squared approach technology platform to their business, and delivers bespoke attribution services for clients, which has been a big breakthrough in being able to drive actual business outcomes from all the insights.

3:26Yeah, absolutely. Absolutely. I mean, it sounds like you've had a long career touching a lot of different parts of advertising and marketing measurement that helped you build into the attribution guy that you are today. So definitely excited to talk about that. You know, you talk about being the attribution guy. Most marketers you talk to working at apps at ad tech kind of know what attribution is, but we're kind of having a conversation about like how attribution isn't fully solved. Why isn't that? Why is that the case? Do you dig into that a little bit more for me? Yeah, it's funny. You ask a question, right, Andrew?

3:58Now, attribution underneath what it is, is that we are actually measuring human behavior. Specifically, we are measuring how much we're able to change human behavior. In many ways, what we're doing is showing a piece of creative and asking for people's money. That's a pretty challenging thing to accomplish. And marketers, that's their daily jobs. That's what they show up every day morning at work to do. And attribution or marketing management is trying to measure how much of that happened and how much of our marketing efforts actually caused change in behavior. Human behavior is fairly complex.

4:37It's non-crivial. It's incredibly sophisticated. And so measuring human behavior, therefore, is also increasingly complex. And it's a moving target. It moves with human behavior and the platforms we communicate and how we communicate, the latest trends and the audiences to talk to. All of those things have an impact on measurability. And therefore, it's kind of an evolving challenge. I'm sure many people know that the Wanamaker code. Apparently, it's 1910 or something like that. Half advertising works, half doesn't. I just don't know which half. I mean, it's not half anymore. It's probably we understand a lot more than half today.

5:15But the challenge still remains after 100 plus years of having the code in the market. It still is a challenge and it will continue to be a challenge because we are it's a it's a moving, evolving business problem for us. And that's why we are all employed to continue to have jobs because of that. Yeah, absolutely. I mean, I think you bring up a good point. Human behavior is not something that you could just measure in a one size fits all way. Like, of course, you have creative, you have an asset that you want to put in front of their face. you want to reach them the right way and see if you can get a return on investment on that.

5:47So that definitely makes a lot of sense. But of course, as you said, measuring human behavior is a little complicated. So how would you say, how do you measure human behavior? What is the most contemporary way to attribute a marketing performance in that way, especially with how complex human beings can be? Right, right. So what's the contemporary techniques, there's something called causal attribution. We basically attach the word causal to attribution. We call it causal attribution. There are basically two techniques that are causal techniques in the industry. One is called incrementality testing.

6:19That's the gold standard for measuring causality. A second technique that estimates causality is what's called marketing mixed modeling. Those are the only two causal techniques in the industry. Now, how they come together, all marketers use both ways. Majority of the marketing leaders use both incrementality testing and marketing mixed modeling to some extent, but exactly how they get brought together for a particular brand is where all the magic is, right? This idea is called causal attribution. The contemporary, what is most contemporary in the industry today is something called marketing accounting framework, which is to kind of come up with a language that both marketing and finance can agree on, on how to describe outcomes.

7:04What are business outcomes? How do you describe them in a mathematical language allows you to measure the marketing performance and report the marketing performance in a way both marketing teams and finance teams and executives can all kind of be on. So now it becomes a currency. So now you can go back to your CEO and say, I took a million dollars from you last month, last month, last week. I put it on market. It broke$6 million in revenue out of which so much in COGS, so much in shipping costs, this and that. I broke$800 ,000 in actual contribution margin, right? Thinking of trying the unit economics of marketing in a way that everybody can understand is a big breakthrough.

7:45And that idea is what's called marketing accounting framework. And it's part of causal acquisition. And what many, many marketers have now come to realize is that because human behavior is so complex, there's no just single one truth. You cannot measure it using MMM or incrementality testing or any other kind of approach and and say and call it done and take multiple approaches and triangulate the ground truth and this idea of triangulation has really come to take root in the industry so all of these techniques have come together into what we call causal attribution and causal attribution is now the contemporary way to measure human behavior in a way we can drive investment decisions human behavior is too complex to perfectly understand but can we understand it enough to make marketing investment decisions and the the sounding answer is yes and there are a lot of case studies now in market many many brands are coming out and saying in their actual uh public statements that you would see mcdonald's and xperia come out and say that used incrementality and marketing performance measurement to uh drive gross margin growth or profitability growth and revenue growth and whatnot so this has become more more or less sort of mainstream uh i would say the last six eight months it's like a more or less mainstream that next year is going to be the year it really becomes kind of at scale adoption industry yeah yeah so you talk touching a couple things in that you talked on incrementality testing you talked about um mmm and you talked about a marketing framework that kind of all comes together to kind of build this attribution measurement cycle that you're talking about and to even get more complicated now we're adding ai to the situation you know this is something is sometimes people are adding AI on seasoning, some as seasoning, because it's like a good buzzword.

9:34So as we think about AI, it has made answers almost free, more ad analysis, more reports, more dashboards. So what is a good decision now? What does a good business decision look like based on all these moving parts coming together? A human still in the loop, but still adding AI on top of that? How does that make it more complicated? How does that make you look at what a good decision is in marketing as an attribution guy? It's a fantastic question, Andrew. Actually, what it unlocks is that the role of AI. It turns out that AI, if you think of AI as kind of this accelerator or automation layer, what's happened is that the surface area that we can cover to look for insights has just exponentially changed.

10:20Now, when I started in the industry, 15 years back or whatever, and we build a marketing model, building one model would take us, I don't know, call it six months. To just get off the ground, it would take us six months to kind of produce the very first round of insights. Now, maybe it takes a couple of days to get the first draft out, maybe a couple of weeks to land in a model that's credible. It's completely changed how fast we can get to insights. Now, what are credible insights? Which insights are credible? What is noise? That requires judgment, which is where the human beings come in human beings are kind of we call it human in the loop or expert in the loop where you can build a million models but you need to know which model is credible because a model is a projection of the business okay you can run an incrementality test but you need to know if you're reading noise or if you're reading signal how do you interpret an incrementality test because you're testing on real human beings the results are never clean if you get a very clean read you you probably got something wrong, right?

11:20So what happens is, therefore, the judgment becomes so much more important than it used to be. Because it would take us six months to get to a model, you might actually have to use your judgment, you know, three times a year, four times a year. Now we're getting to the point, you're making critical thinking, so business judgment calls, you know, three times a week, right? So humans in the loop have become so much more important in the execution than before because of AI, because we're accelerating time to value. That's the most important aspect of all of this. Now, what is, therefore, what should marketers do?

11:58What should leaders do, CMOs do, is that when you bring a go-to-market thesis to the table, if you're a CMO and you have a particular thesis for what you want to do with the brand, how you're trying to take it to market, what is your thesis for explosive growth for your brand? You believe that consumers are going to adopt your brand And because of certain value prop and how you communicate the value prop or some whatever it is that your execution is, you have to bring that thesis to the table. Now, when you take measurement and you wrap that measurement around the go-to-market, now what happens is that it can validate your thesis.

12:37It can say, oh, consumers do react to your messaging in this way. You can clearly see people showing up at your site and engaging your content, engaging all your PPP pages. So clearly your message is resonating, but your price is probably not resonating because you're not getting conversions. And maybe the price plus the messaging is really resonating. Maybe the place where you communicate, whether it may be the CTV, mid funnel, upper funnel, over funnel, whatever it is. So when you have a thesis for your go-to-market and you wrap measurement around it, that's when magic really happens. You can really figure out what part of your go-to-market is working, what part of your go-to-market is not working.

13:15Now we can fix it one at a time and really can unlock value in a very testable, repeatable, data driven fashion. That's where you find the compounding benefits of advanced measurement. Because AI accelerates all of this, this would take forever. In a prior era, 10, 15 years back, this would have been a 10 year project. Now there's an 18 month project to completely transform your go to market, which is very exciting. and the folks who are adopting it are going to see transformational outcomes for their businesses. And it's an exciting kind of time in this space. Yeah, yeah, absolutely. I mean, I think AI, one, is the speed at which it can work.

13:58So it's definitely up to the human who is like watching over it to make the final decision on that, especially based on your prior knowledge, what will look good in measurement and attribution. So that definitely makes a lot of sense. A little bit of a segue, you talk about finding alpha in marketing. Can you walk me through what that means, especially in relation to measurement and attribution, and especially what does alpha mean when you're a CMO, as you were? Yeah, it's a great question, man. So I come from the world of data science. I grew up as a data scientist. And for many years, I was thinking I was going to go into quantitative finance, like, you know, in trading and risk management and all that stuff.

14:40I was pointing towards a career there. So a lot of what I learned in school and after school was portfolio theory and managing portfolios, balancing portfolios, measuring it and whatnot. The finance guys, especially the quant guys, have a name called Alpha, a word called Alpha. alpha is about finding sort of arbitrage in market a particular stock is where it's blah but it's priced at blue and now you can buy it and sell it in a in a in a and use that arbitrage to drive business value revenue growth whatever it is right now in marketing we are now beginning to use so much data to understand you know how marketing works in consumers to drive drive engagement and transactions and growth, we are now beginning to use the same word with respect to marketing and data.

15:32What that means here is that it basically means actionable insights. Can you find an insight that's actually actionable that will drive growth for the business? And we call that alpha. In a really good example, I worked with a CMO called Jay Sung. He runs a beauty brand called Pacifica Beauty. And fantastic example, great CMO. He's one of those like story performance marketing CMOs been around forever. And he, all of these great CMOs have a thesis for their business. And you talk to them, they'll be like, hey, I believe this is what's happening in market, right? In reality, this is what's happening with consumers and how they're experiencing my brand and my marketing right now.

16:21Okay. Now you can't absolutely know it in perfection because you can't really get into everybody's heads, right? But there is a thesis for customer journey and how the brand is actually working in the real world. His thesis was, I've got about 200 SKUs. I've got shampoo, I've got conditioner, I've got soap, I've got this and that. I've got different beauty products. And I believe that each SKU is its own market. the shampoo skew is competing with other shampoo skews the conditioner skew, other conditioner skews the perfume is competing with other perfumes, every skew is an individual market on its own and it's competing on brand and price and utility and distribution and all of those different things so he's like, madame, if you build models for the whole brand Pacifica Beauty as a brand that's not useful for me it's useful in some context but most of my decisions are going to be at the skew level.

17:19So you need to build skew level M &Ms for me. Now, I've been in the industry for 20 years. We don't build skew level models. You build models at the brand level, maybe sub-brand level, maybe department level or business unit level. This idea of building skew level models is insane. Just building one model takes like weeks. Building hundreds of models, I mean, the scale of this was impossible, Even 24 months back would have been impossible. Of course, with AI, a lot of it is now possible. So we took a very specific approach to find the right structure for each SKU, build the right, you know, group the SKUs by modeling structure, build hundreds of these models and synthesize them into insights across all of them and bring it all together.

18:08Then we found that Jay was actually right. It turned out that media's incrementality, causal impact of media driving performance was meaningfully different by SKU. Some SKUs really moved the needle based on the marketing and the pricing and the brand. And some SKUs didn't matter at all. The brand was so strong, pricing didn't matter. No, media didn't matter and whatnot. It was just all distribution. Now, the funny thing is, he's an AI guy. Jay is an AI native guy. He's an open client. He's got an entire OpenCoin instance that runs his life. He has OpenCoin everything, all the way from, you know, what's my schedule today morning and what should I be doing in life.

18:56He has the entire set up on his machine. And he is fully AI native. And I've been working with him for many, many years. I've had him as a customer across three or four of his career stops. and he actually before calling he ran his own MMM he used the open claw MMMs to run an open source MMM called Robin he just had the open claw model pick up Robin run a model for him and he spent several millions of dollars 20 40 million dollars something like that in on media on Amazon well he came back and said hey cut all your budget nothing's working cut 30 million dollars in investment from amazon he was like i'm not gonna take such a radical decision based on some model some ai that doesn't fully understand the context and whatnot and i don't even know how to run these models well then he called me now you're the expert what's right about this model what's wrong about this model i walk him through the process of course i know the i don't know the answer to his problem but i know the process i said here is how to build the right model structure here is how you analyze if the model is credible or not here is how you validate you make good decisions from it.

20:08He was like, okay, now I actually have to pick you on as an expert into the execution. So then we did a project for him. We built these hundreds of models for him. And then we actually found out that the results were meaningfully different. Every SKU was actually its own market, like how Jay had hypothesized. And then we found out which SKUs deserved investments, which SKUs, you know, was just not useful to plow money into it, then gave him a completely different investment thesis. In the first round, we were able to find about 5-7 % top-line growth for them. And we also found a way for a test-learn-growth and further growth into different SKUs.

20:47So here's a great example of how AI is being deployed today to explore a much larger surface area for insights. Surface area around which you can look for insights is now significantly, exponentially larger. So our ability to find the right insights for the business to drive growth is meaningfully different and move fast is meaningfully different. So that's a transformation that we're all going through right now. It's pretty big time in the industry for this technology and this approach. Yeah, I mean, I think you bring up a good point. I think within marketing, you have all these niche models, these niche audience that you need to reach, and that really speaks to why the speed of AI is beneficial because it can help you measure a larger array of models and audience as at once.

21:38So that definitely makes a lot of sense. But I guess it's about that balance of making sure you still having that human judgment on top of it as you're measuring these things. I think you and Jay kind of found that balance working together. So that definitely makes a lot of sense. And to kind of, you know, coming to the end here, we've kind of talked about a lot of different things within measurement, attribution, you know, the specific nuances and working with that with incrementality testing and, you know, marketing frameworks and things like that. So as you're kind of looking towards the future and kind of future-proofing some of the products at M Squared or even just thinking about how the industry can think about measurement in the future, can you give me at least like three words to describe of what you see the future of measurement and how AI is affecting it?

22:21If you don't have three words in particular, maybe just like an overlock how you kind of see the future measurement and how it is affecting it i'll give you one word how about that okay and the one word uh i'll give you two words supercognition okay supercognition is an idea that everything that can be known will be known like we talked about this insight around hundreds of skews and whatnot surface area around which we can the large surface area for us to scan for insights. So any insight that's discoverable, it's waiting in your data will be discovered. I'm calling this idea called super cognition, which is you will know everything that is worthy of being known.

23:06And as a result of that, what will happen is that there's a word called efficient frontier in the world of quantitative finance. It's called efficient frontier, which is if you have a pot of money to spend, there is an optimal allocation of your investment to maximize the returns in your portfolio. That idea is called efficient frontier. Now, what will happen is all marketers will be able to manage their media investments, marketing investments, on the efficient frontier. So, first of all, we'll know everything that there is to be known. Because of that, we'll be able to manage all our investments on the efficient frontier.

23:45Now, what would be left is the unknown, which is about, well, how would consumers respond to a new message, a new kind of way to deliver our brand and whatnot? What would happen if it's poked in this way or just that way? How much will they move? How do I break the efficient frontier and find new insights about consumers and human behavior? That's what the frontier will be. And we'll live all, mostly living on that frontier. Most of the human work will be on figuring out what will happen if you do something that's not been done before. That will be the future. So we're going to live in an exciting time in the future.

24:19But it will take a bit. It won't happen overnight. There's a lot of work in just getting streamlined and data and processes. It's all broken. Change management is a pain in the behind for all organizations. It will take a bit to get there. But once we get there, the future is all about living in the unknown and discovering the unknown and operating in the unknown. So it's a fun time to be in space. Yes, yes, yes. I think that's a great place to end it. Thank you, Madan, for joining us for the Inside the Stack podcast. Definitely enjoyed this conversation. I think there's a lot of nuggets that the audience can take away.

24:52I mean, all marketers are measuring in some kind of way. So I think there's definitely a way they could pull from this and see how if they're testing AI with their measurement, or even if they're not and thinking about doing it, ways that they could pull from this conversation and do it that way. So definitely, again, Madan, and thank you again for the audience for coming. And I hope you enjoyed the episode and look forward to seeing you on the next Inside the Stack podcast. Thank you so much, Andrew. Thank you for having me. And look forward to more collaborations in the future. Yeah, absolutely.

25:24Madan Bharadwaj: This episode of Inside the Stack was brought to you by M Squared. More than a dashboard, M-squared is a hands-on measurement program, a dedicated analytics team, and a finance-grade reporting that turn numbers into confident decisions.