Brian Kotlyar Explains What’s Hype vs Reality in AI Marketing

0:00Hey, welcome to the CMO Whisper Show. I'm your host, Steve Olensky. Part marketing practitioner, part ad agency veteran, part journalist. I was a writer for Forbes for 10 years. I've had so many insightful conversations over the years with business leaders, to athletes, to celebrities, to, of course, CMOs. The only difference now is instead of sharing those insights through written form, I'm doing it this way. My guest this week is Brian Cotliar, the CMO at Hightouch, the leading composable CDP that helps companies activate their data warehouses to drive personalized marketing and business operations.

0:40With 20 plus years of experience at Intercom, Sprinkler, and New Relic, Brian knows how to cut through the MarTech noise, that makes one of us, and build teams that turn data into real growth. Brian, my newfound friend, welcome to the show. Thank you so much for having me, Steve. It's great to be here. It is my pleasure, my friend. I think we're going to have an incredible conversation, and I know my listeners are going to learn a lot. And I want to start with, I used to call this the elephant in the room, Brian, that two letters that everybody talks about. It's not the elephant. It's the whole damn zoo now.

1:12And that's AI. And in marketing and differentiating using it. So let me just jump in and get right to the, you're a perfect person to ask this question about the real difference between LLMs and reinforcement learning. And then how does it become some marketers actually use them? Sure. So as sort of this like tsunami of AI technology has arrived in the last few years, And we've all, I think, we've all deeply understood that like white collar labor, knowledge work, whatever you want to call it, is really never going to be the same. All kinds of things have started to be crammed under this moniker of AI.

1:49And the truth is that there's actually a lot of different disciplines and schools of technology that sit underneath this broad term. And so, you know, for a very long time, we've actually all quietly and maybe not so quietly been using AI in our day-to-day lives through the moniker of what is actually called reinforcement learning. And this has been going on for years, by the way. Basically, if you think about how Facebook works or Google works today, you know, when those platforms were first born, you know, you would be very precise. I want to market to young men in these regions with these interests or whatever.

2:22That's how it all worked. And then a few years ago, suddenly, it really changed. Suddenly, all that would happen is you would go to your Facebook interface or your Facebook salesperson and you'd say, here's my creative, here's my money, here's my outcome, figure it out. And initially, that was a little scary to do. But the reality was that it actually worked better than any amount of specific targeting because Facebook's ability to learn and harness all their data, given the breadth of platform they have, is just outstrips any amount of very specific by new targeting and media planning any of us could do.

2:49So that technology that underpins that ability just to go to Facebook and say, get me an outcome, the rest is your problem, is the underlying technology that's called reinforcement learning, which is a branch of AI, which we've all been using. If any of us have bought a Facebook ad, a TikTok ad, a Snapchat ad, or a Google ad in the last 8-10 years, we've been using reinforcement learning, just full stop. And that predates this LLM revolution that we're all currently living in now, where, you know, obviously most famously with like the chat interfaces of ChatGPT and Gemini and things like that.

3:19And so I think for us as marketers, what we're kind of learning to do is to think about sort of the full workflow of what the job of marketing is. And now what we're doing is we're figuring out what's the right tool for the job from this broad AI toolkit for each facet of work. Does that help those answer your questions? It does. It does. I want to keep going, though, because I think, and I don't want to speak for all marketers, but to elaborate, if there's more between the difference between LLMs and that reinforcement learning you're talking about. What can marketers, like, what should they be looking for?

3:51What are the main differences kind of thing? Oh, sure, sure. Yeah, forgive me. I don't know if I answered that very well. So, yeah, basically, the thing to think about is that LLMs, as we kind of famously know, are essentially like these amazing engines for guessing the next thing to do. And it allows them to take these creative leaps that mimic a human creative leap and are increasingly even better than human creatively across all these different domains. And so essentially, you can feed it an initial set of information, and it'll take that next step and the next step and the next step. And as the power of them gets better, they can go further and further along in this chain of logic and tasks they can do.

4:28Reinforcement learning is a little different because what reinforcement learning is about, it's about trying things, learning from what you tried, and then feeding that learning back into the next initiative or try you're going to take. So that's why, again, using the Facebook example, because it's all so familiar to all of us as marketers, it's why they have these quote-unquote learning periods where actually the algorithm is bad for a while. The reinforcement learning doesn't work initially because it needs to run those experiments and teach itself what it is you want and how to go get it. So you can think about it this way.

4:55An LLM is a great way. You give it sort of some context and a need, and it'll generate something for you. But now, is that something actually the right ad creative or the right coffee or the right email subject line to engage your audience, get the outcome you want? Well, the LLM has no idea. It knows it generated something that fits the specifications of what you wanted. But will it, quote unquote, work? Well, that's where reinforcement learning comes in. Reinforcement learning is essentially the way now to, in a very rigorous, high-scale way, test what's right. And so when you pair these two things together, you use LLMs to generate things, and then you use reinforcement learning to put them in front of real people, see if it's effective, and then tune it and tweak it and get better and better.

5:36Now you have this incredibly powerful system where you can generate vast amounts of creative or subject lines or ideas, and then actually bring them and put them in front of customers and tune them and improve and optimize and learn. And so you can create these really high-scale systems. Net new creative testing. Net new creative testing in a way that, frankly, human teams have just never been able to really achieve in the past. We've been very limited to simple things like, you know, art director, copywriter make a thing. Then growth marketer or media planner or whatever tests that thing. Then some scientist does some readout of the A-B test or something and brings it back to the media planner and they begin anew, you know?

6:19And that cycle is really slow and difficult to learn from. And frankly, not particularly scientific, but this new world is incredibly data-driven and scientific and at a scale and a rate that just boggles the mind. Not to belabor the point, but you bring up, be remiss if I didn't ask this question. So large language models and reinforcement learning, can they operate independently or do they need each other? For a marketer's sake, because there's like a level, I'll be honest, there's a level of computer science here that's probably not interesting. People in the context of marketing, yeah. As marketers, you don't need one to do the other.

6:52You can actually have... I can write creative and subject lines and work with my designers to generate things and then give them to a reinforcement learning engine like what Facebook does for ads or what HighTouch does. We have this product called AI Decisioning. And that's the category. But the idea is that you can use AI Decisioning to do this same style of reinforcement learning across all your email channels and SMS channels and post channels too. It brings that Facebook concept to all these other environments like your own channels, like your websites and stuff. And so you can do that using the creative you have today on the shelf, sitting in your digital asset management platform, sitting in your email service provider, sitting in your content management system without touching an LL.

7:30You could just use reinforcement learning technologies on ads and on own channels. Or you could be using LLMs to help your teams accelerate their creativity and accelerate what they produce. And you could flight them into the market without touching the reinforcement learning technology. You just do it the old way. Let me run this A-B test on my lifecycle program. Let me run this AV test on my ass. And that would be fine too. In fact, both are better than not doing it, you know, in a vacuum. But what's really magical is when you marry the two. Now you have the workflow improvements of LLMs helping you be more creative and develop things much more quickly.

8:03And you have the learning improvements, reinforcement learning, bringing this stuff to your customers in your emails, your SMS, et cetera. And the two together, so we had a customer tell us that, and I'm quoting, they learn more in six weeks than they'd learned in the prior 12 months by doing this. Really think about that, right? That's an 11, 10 and a half month improvement in velocity, just by turning on a piece of software. So literally saving years, years of effort. So, yeah, and thank you for elaborating on that. So I want to stay in the AI world for a few more minutes. This is the most like, let's just cut out the BS here.

8:42In your opinion, what's actually changing in the daily workflow of a marketer versus what's just hype? Sure. I mean, look, I would say that what I've observed is that if you think about sort of the workflow of marketing, ranging from like trying to understand what to do and figure out what to do initially, like almost like strategy, possibly, then this like period of making stuff in the middle, and then this period of like exposing customers to it and running experiments and engaging with them. What I see is that basically the bookends, the beginning and the end of that process are being really heavily disrupted.

9:15All right. Like seeking to understand what to do, seeking to understand what's going on. LLMs can really help you with that today as we speak. And then seeking to interpret that into briefs and strategies and things. LLMs are already really helping with that. They're starting to trickle into the middle in terms of helping with copywriting and editing and creative stuff. But it's still early days, like very basic things. LLMs have really disrupted generating subject lines. But are whole creative teams being displaced? No, they're being helped, but they're not being displaced. And then at the end part, the part where the materials actually meet the market, whether it's through an ad, as we talked about, on Facebook or Snapchat or whatever, or whether it's through an email or an SMS or a push experience like what I can help you with.

10:01Again, that's being really heavily disrupted. Those core workflows of let me plan my A-B test and send it out and study whether it's working. is getting completely basically deleted and replaced by these AI-fueled experimentation programs. And so increasingly what I suspect will happen is the middle will start to collapse too. You know, like the beginning has been changed, the end has been changed. The middle is hard because it is so creative and brand sensitive and brand centric. But the AIs are getting better at that too. And so you're starting to see it move more and more towards the middle.

10:32So what, if anything, is hype? Well, I think what's hype as we sit here is there's a lot of this notion of the entire workflow collapsing and being run by robots. And as we sit here, that's just that's factually incorrect. Like we talk to customers every single day. I run a marketing team, but I also interact with my peers and my customers every single day. And it is like it is a lie. If you say, oh, marketing has been completely transformed from nine months ago. It's not true. Just it's not true. But do you see these glimmers of how AI is weaving its way into each of the elements of the workflow?

11:10Yes. And can you kind of squint and see a world where instead of building these complicated journeys, like if this, then that, do this, then that. Instead, just like on Facebook, you hand them some money and a picture and you say, go. just like in not very long from now, will you just hand a robot some words and some pictures and a list of customers and just go through your email program and stuff? Yes, that's not very far away. There's this famous science fiction quote that the future is here. It's just not evenly distributed yet. And I believe that we're in that kind of a mode where you see in isolated instances, you see these very AI forward workflows present all over the place.

11:47But like these, like, I don't know how to describe them. And these apocalyptic or messianic, depending on how you think about it, statements about what's happening in the marketing are not accurate. Just go talk to anybody. It's like, you know, it's not true. So that is, thank you for that, because I think there's so much hyperbole and hype. It's ridiculous. However, in the real world, there are still some skills, right, that marketers need to have in the AI world. What do you think those skills are? So I think a lot of the skills, because the technology is adapting so quickly and changing so fast, a lot of the skills are actually soft and personality type behavioral things than they are like vocational training at this phase.

12:33Because the specific vocational thing you would do right now to harness some of these technologies is going to change in a month. And so I don't mean to be trite because folks may have heard this, but I do believe it's true. Like sort of being very curious and borderline naive to what's possible is probably the most important thing. Like cynicism about, oh, robots can't do that. Robots shouldn't do that is really dangerous because of the rate of improvement we're seeing. And so like coming to this as like a learner, no matter where you are in your career, I'm speaking as an individual, not as a leader, but beginning as an individual, like I think is the critical thing.

13:11I can say, oh, learn to prompt. I could say, oh, learn to use this tool or that tool. But it's not clear to me that that prompt skill that you develop is actually going to be the necessary one in a month to get what you want out of AI. I'm not sure. So in your turn, yeah, sure, do that because you're learning and you're experimenting and you need to be open. But is that like the critical thing that will future-proof your career? Nobody knows the answer to that question. You know, I think those future proofs your career is that is that willingness to lean in and learn and not get overly cynical about about these things.

13:43And then as a leader, I think it's a little different. It's about actually creating that culture across your team. You, of course, should be doing that. But the truth is, most leaders, we don't quote unquote work the way that like our teams work. You know, like when was the last time a CMO listener to this actually edited a script for a commercial? You know, like, so, but they probably want their teams to use AI to help them do that. They probably want their teams to use AI to help them instead of hiring some expensive production company, use AI to help develop that next ad that they're going to run or whatever.

14:16And so it's a lot about fostering the curiosity and creating the space for teams to kind of the old adage being like slow down to go faster. You might need to give your team permission to deliver something slower while they learn to use AI tools and explore AI tools for something versus demanding that they, you know, they hit their deadlines that they always have because this is a learning moment for all of us. Right, right, exactly. So how do you as a leader then the balance, right, between overwhelming them with AI but then reminding them, hey, don't lose the human touch? Well, I think the good news in a sense, right, the second is that the technologies still require the human touch.

14:54I've never seen... And I like almost as a habit, whenever I talk to any of my peers or marketer, I just ask them what they're doing with AI. And as they start to explain, what I'll do is actually I'll pause them and I'll say, can we pause before you answer? And can you screen share me? If we're doing this over Zoom, can I please see your screen? What actually did you type in? What actually did you click? And first of all, I think that's a good habit to your prior question about deflating hype. Because it lets me actually see what's going on and learn. So I think that's a good habit. Anytime somebody starts blithering on about AI, have them pause and say, oh, that's fascinating.

15:27But don't tell me about it at a high level. Show me what you're doing. So that's a good habit, I would say. But separate of that, what I would tell you is when they do show me their screen, 0 % of what these AIs are outputting is market-ready. Just a fact. All of it requires human curation and editing at this phase of the game. You can really get... It's remarkable how close you can get to market ready. But 0 % of what any of my peers have shown me is market ready. So the human touch, if you just set this on autopilot, you will not be happy with the outcome. You as a CMO will say, wow, my team is producing bad work.

16:03And the reason is that the AI left to its own devices will produce bad work. But the AI helping your team and then your team applying taste and judgment will produce good work much faster than it used to take to produce good work. It does. It does. So what are some ways brands are using AI in terms of personalization, for example? Sure. So if I talk to you about that end-to-end workflow, from the strategy to the making stuff to the actual execution, a lot of the personalization happens in the middle and at the end. It's like you need a lot of variations of content and creative in order to have options of what to show people that are appropriate to them, appropriate to their journey.

16:49And then you need a system that's able to actually deliver all those options appropriately to the right people in the right spots. And so just as an example, we have a customer, it's called Fundrise. It's like this alternative asset management platform. So if you want to invest as an individual in real estate or in obscure assets, they'll help you do that. And they are very AI forward. And so they produce these massive amounts of content variations. And they start always with what they call an investment letter. It's like this is their core point of view on the market. But that thing needs to be cut into countless variations for all the different kinds of assets they support and different types of investors that might be using their platform.

17:28And so they use AI a lot to help them take that initial point of view and refabricate it into all these different assets and variations, visual ones, recorded ones, videos, whatever. And that's a very AI-led process, though they do apply human judgment. Then they're like, okay, well, now we have something to show Steve versus Brian versus Sally versus Stevie. What actually should we show them? We have no idea. That's really hard. And so that's where you get to those end technologies, like the AI decision in what I'm talking about, where it's able to look at your data set and essentially propose these massive one-to-one experiments.

18:02Like, I want to show Brian this subject line and this asset. I want to show Steve this one and this one. and then start to run those experiments at like an unprecedented scale. And so what they've been able to do, I mean, this is in production right now. If you're a Fundrise customer, you're getting these communications, is they essentially market to every single one of their massive customer base completely uniquely. You and I would have completely different experiences of their marketing communications than anyone else. And it's that pairing of a lot of variety with this massive one-to-one experimentation engine that they operate using this AI decisioning concept.

18:38Right. Are there, in terms of measurement and metrics, we know the usual suspect KPIs. Are there other metrics when using and optimizing through AI? Well, I think that in the case of specifically that decisioning thing, it's actually kind of nice because the answer is no. You're still engaging customers through those same channels. It's still SMS. It's still email. It's still whatever. But the question is, is it better? So what typically customers will do is they'll have a holdout base because they're doing something. No company is just sitting on their hands, right? There's some existing email program or SMS program or web program or something.

19:10And so basically what they'll do is they'll do a holdout. They'll say, this is the old way. Let's leave it alone because we want to make sure this is superior. And they'll pick the metric on which they're going to judge if it's superior. Is it superior based on just engagement? Is it superior based on, you know, end lift in revenue or whatever? And then they'll turn on the AI and start to execute. And similar in a lot of ways to like, as I mentioned, kind of like how Facebook might work. There's going to be a learning period. For a period, it might be worse because it has to guess and try and learn.

19:39But what we see with customers is that given enough time, it's basically always better. Yeah. But it may need time. And it may depend on the metric you select. Because one of the things that's really challenging about marketing, as we all know, is that last click sort of wins everything. Classically, for the last 30 years of our lives, 40 years of our lives, we've been living with this world where just whatever the final point of conversion is sort of, quote unquote, wins all the credit. But we do a lot of stuff just because the four things that happened before that click didn't get credit doesn't mean they didn't help.

20:09You know, so you need to have a broad, in some cases, a broader, more open minded perspective of what, quote unquote, works and helps. But I don't think that's unique to AI. I think we've been grappling with that for a while. Yeah, I was just going to say that. So I think that a lot of the more sophisticated thinking we've all tried to embrace of late, I think still applies. It's just using slightly different tools. Yeah. You've talked about a shift that I'm interested in getting more of, about CRM first to data warehouse first marketing. For those not familiar, what is it? What does it look like in practice?

20:44Tell us about this shift and why you're seeing that. For a very long time, sort of the state of the art of gathering information about your customers in a way that's actually usable by a customer service team, by a sales team, by a marketing team, has been CRM. Just collect data in there that's associated with records of people and try to manage their relationship with them, CRM, using that technology. The problem that became apparent, I would say, really clearly about five years ago, it was sort of there before, but really apparent about five years ago, is that the kinds of information that we as marketers and businesses wanted to use to interact with our customers was more complicated than CRMs were really invented to manage.

21:27You know, we stopped being just interested in how old is Steve and where does he live? We started being interested in like, can I predict based on past purchase behavior, what Steve might want to buy next? And then use that information to engage with him differently wherever, you know? And at one point, that was a kind of a science fiction idea, maybe 15 years ago. But five or so years ago, it just became commonplace to want to do that. And businesses would turn to the CRM that they'd invested in to attempt to do that. And they would find that it was really not a sufficiently flexible, scalable piece of technology to help.

21:59And what they started to do is they turned to their IT teams and their data teams. And they were like, I want to do this thing and my existing technology can't help. What do I do? And so Gartner's actually did a good job of writing about this of late. You're starting to see these IT teams and data teams step into that breach. And the technology they're bringing to bear to help with that is the data warehouse. These cloud data warehouse environments where essentially you can dump as much information as you want. every little weird scrap of everything your customers have clicked on, looked at, engaged with, every prediction that you might want to do, every sort of statistical model you want to run.

22:33These warehouses are just these incredibly flexible environments for all that stuff in a way that CRM is not. And when you turn to that and make that your source of truth, suddenly these things that don't seem like they should be that hard actually are not that hard. Whereas in CRM, you might spend 10 months trying to wrangle your IT team to get you this one little data point so you could customize this one SMS you want to send. With data warehouse technologies, we talk to our customers and it takes them hours, days, maybe weeks. But again, you're thinking about collapsing a 10-month process to a two-week process.

23:04It's like night and day difference. And I lived this experience as a marketer where I transitioned from a CRM-centric world to a warehouse-centric world. And it was so eye-poppingly different in terms of its flexibility and speed and scale that I literally quit my job and came and joined High Touch because I was so excited about it. Wow. It's like, you know, sometimes you see something and you can't unsee it. Yeah. And that was one of those moments for me. I was like, wow, the whole way I've been doing my job for 15 years is wrong. I've got to go do something else. And that's what happened to me to get me to make a big career change.