How to Build AI Agents for Your Marketing Team with Kris Rudeegraap, Co-CEO at Sendoso, and Jessica Voss Head of Marketing at Kana

29:44Right. All of this is supposed to be part of this broader overall nervous system. And all of it should be informing the work that every person within your marketing org is doing. And that just has never been able to be the case. Right. Because relying on humans who are doing so much work and such a high volume of work, it was never actually possible. And so that's why what we do is we we aim for more of these outcome based applications. we're not focused on the human roles. We're focused on what is it you're trying to achieve? And what is it that, you know, all of the data and all of the technology can tell you and help to support you with.

30:23And so one of the things we're really focused on is this idea of decision loop or learning loops, where again, the outputs after it writes back, it comes back into the intelligence phase and that can inform every next thing that is happening. So I don't know. Chris, this is the part where we get to have a fun debate about kind of how Sando. So like, do you guys feel the lack of that in your environment? Or, you know, is this something that you would necessarily aspire to in the future? No, I mean, I agree. I think there's, you know, a need to think about, you know, the the intelligence, the data behind everything.

31:03And that's something that a lot of times gets overlooked or is everyone aspires to being more data driven. but then at times doesn't have the bandwidth to be more data driven. And that's where I think agents can come in and pick up. I think there's also the ability for marketers to focus on parts of the marketing business that they want to focus on and offload more of the tedious, repetitive tasks to agents as well. And I think that's something that is a newer concept to do where you want to focus on things that you love doing, whether it's a creativity piece, whether it's, you know, actually looking through the numbers and having the time in your day to then analyze and say, hey, we're going to shift strategy for this because of this.

31:50And I think a lot of marketers are always overworked. And so hopefully this frees up marketers to be more strategic. And that's how we think about it. Chris, if you were starting over, like what would be the first marketing agent hire you would have um hired like if i was like a seed stage four-person company or yeah or like just starting my ai journey as like a 300 person company with dozens um yeah i obviously it depends on the answer but um let's just go in like your journey because you were talking about your journey like what would you have like hired first based on your learnings yeah i mean i think i'll answer of that in the latter where it's like more of like, if you have a big team of marketers, what would you want to, uh, you know, what, what agent would you want to build first?

32:40I think it, because I think that's more realistic than being like a two person company with, you know, who would you, who would you hire first? So, I mean, my thesis there is that it's going to differ. There's going to be a really important, uh, uh, first step of like auditing who who's on your team, who's doing what there's oftentimes a lot of companies that have not hired for a specific role because it's, you know, lower on the to do list and maybe it's never going to get hired for, but now you can build for it and it's a critical piece. Even things like, you know, you might have a great digital team, but you're lacking on like really optimizing the digital ads for some specific channel.

33:16And that's something that you know you needed to get to. You don't have budget to hire for, but you know it's critical. I think something like that is like, okay, let's go get, let's go build for this agent right now. And let's, uh, you know, hire this kind of, uh, person through a building, um, that would have, uh, never been built. Um, so I think that's one way to think about it. Um, another way I'd think about it is like, where is there like leaky buckets in your, you know, marketing team right now? Is it like, Hey, you, you, you're super, uh, uh, you have tons of top of funnel. Um, but you're like mops workflows are falling apart because your systems are not working well.

33:56And so maybe it's like, hey, let's build a MOPS agent. Or maybe you're doing an infinite number of field events this year and your field marketing team's underwater. You know you're not going to hire another field marketer. Let's figure out if we can build a field marketing associate agent that can take on some of the busy work there. So I know it's not the perfect answer because it's not exact to what you asked for, Daniel. But I think that's where having the time to audit and spending time and figuring out what is unique for each company is a better answer than just like a generic hey this is the best agent no matter what yeah i think i mean obviously i think most marketing answers come down to like what's best for like goals outcome yeah size of that so i i like that you reframed it to hey like look where you're lacking look where the leaky buckets are look where this might be process fails.

34:53Look at your budget. See, can I hire? Can I not hire? Look, and I think this slide, I think Jessica pulled out this slide, but I think it's really good. I love this slide, yeah. Yeah, like what has surprised you in this journey versus the mistake? Chris, before you go into that, I just want to, Daniel, from our perspective at Kana, what we're seeing that a lot of the enterprises really are starting with, and it was actually similar to what Sendoza's journey was, was like, what are the functions of the SDR that can free up the SDR to have more like qualified customer conversations, but get rid of all the, there was so much technology that was brought in for outbound selling, right?

35:33And there's, it's just been incredible. And I think that's one of the first use cases that a lot of companies are looking at is like, how can I agentify my SDR outbound prospecting motion? But the two other, we did a research study that hopefully everyone's aware of called the agentic divide, where we pulled 50 CMOs, 50 CAIOs, or chief AI officers, and then 50 CDOs, or chief data or analytics officers. And we asked them, what's the low hanging fruit? Like, what's the primary use case you really feel like you need to agentify first? And what's weird is across all of these cohorts, there was not that much agreement on anything in the survey.

36:13But on this one, they were all completely aligned. They said campaign optimization. So, again, it's kind of like media mix modeling, making sure that you're putting the spend in the right media and channels and not, you know, feeling like you're burning your money on your campaigns. And then the other one that's really interesting is personalization at scale. So this is one of those things that, especially in B2B, I feel like people are really focused on how can we get more immediate custom and, yeah, personalized messaging done in campaigns at huge scale in our campaigns. And so those are two use cases that they all, all three of those cohorts agreed on.

36:54And I thought that was really interesting and worth sharing. well it does make sense in the case that like ai is really good at crunching numbers fast crunching like words fast and i just thinking back my journey as a marketing ops how much i would have loved the job more where i could have taken all like writing the process docs and just either like getting ai to do it or like voice to text and ai figuring it out for me and then also like the reports getting fed to me in two minutes instead of having to go get an analyst or having to pull the numbers myself and wait for the the pool and wait for the run and then that's a numbers get refreshed yeah it's like those things of like that took me hours and hours and then I had to analyze it and then I had to get an answer back and then that answer was old because we're now like three or four days of crunching numbers and stuff like that and it's just like this continuous loop where you're like feeling where ai does that for you now it's like super easy but i'm gonna let chris talk about this live because i think it's really interesting yeah yeah happy to talk through some of these surprises for us over the last couple years the first one was really like technically building agents while it's hard is not was not the hardest part for us it was actually kind of the cultural and change management of it all and so i think this is where having like a strong enablement team to come in and actually help with this change of how work gets done is critical.

38:27It might sound easy to just say, Hey, we built an agent. Everyone should use it because it's obvious agents are great. But if you don't get all the people on board, it's going to be an uphill challenge. That was one thing we thought we just like agent building was going to be the hard part and everyone would flock in and be like, I love agents. And that was a harder for us to manage. We also stumbled when we started building agents on some bad data. I think this is a challenge to a lot of marketers. So we actually took a step back and really cleaned up a lot of our data, really built out more of a data ontology too, because you can't say, hey, agent, go do this in Salesforce when you have like seven fields that look like they're the same field.

39:05So we had to build out almost this like new data dictionary, data ontology, so we could educate the agents on what's right. And we had to go back and talk to a lot of humans with tribal knowledge be like, why did we change this? Why does this field look like this name and this name and this? So there was a bit of like cleaning up of the data specifically so we could direct the agents better through some of our systems. We, prior to this new like agent center of excellence and AI engineering team, we gave everyone kind of clawed access and, you know, basically any tool they wanted. And there was a problem of AI sprawl where everyone was building agents on their own.

39:44Some had good agents, some were duplicate agents, some didn't have the right permission levels. So their agents weren't really that great yet. And so we tried to kind of take a decentralized approach first, which kind of backfired. And so then we brought it all together into more of this like agentic pod model and more centralized. And that was really helpful. Again, we just assumed everyone could build agents and that was not correct in the beginning. and then the planning for the agents was harder. I think this is something where, again, a lot of people on the call might be like, oh, I have an idea for an agent.

40:18Well, if lots of people have lots of ideas, but you have limited resources to build, how do you prioritize, plan, and think through what's the best next agents to build? Because there is kind of a capacity and time constraint. And so that was something that was a challenge that surprised us. I think, well, on the opposite side, where we look at the mistakes, one of the things that's easy to do is go ahead and audit all of your processes and say, hey, let's just automate all of these through agents. And I think you can take a step back to think about, was this the right process? Or was this a human-driven process?

40:55But if there was agents that could work 24-7, could we do this differently? And so it was taking a different mindset there. It's also easy to say, like, let's just go run agents, but without having human-in-the-loop workflow closed ownership, you know, and more transparency in what they're actually doing, you could run into these like rogue agent problems. And then I think the third mistake, similar to this, like, you know, the super agent that can do everything for marketing or trying to do all of marketing at once, you can run into this challenge of doing too much or spreading yourself too thin.

41:31So I think it's like finding pockets and saying, hey, this month we're just going to do this. And you know, it's going to be a smaller start, but eventually, you know, this could, this is like a multi-year process. I know everyone's like, let's build agents today and it needs to be done today. But honestly, if you start today and you build agents and then in a year or two from now, you're going to be ahead of the curve because this is a multi-year transformation that is happening in, in marketing and across all businesses. Yeah, I totally, I agree and sincerely appreciate, especially given how much you guys did internally, like these are, this is really sage advice coming from somebody who's been in the trenches with it.

42:12But, you know, starting with one use case or starting with one agent, you know, within your kind of organization is absolutely the right move, because you're going to learn so much about what your infrastructure looks like, as you point out, kind of like if there's data issues that you need to address, if there's governance issues you need to address, if there are security issues you need to address, if you need to understand, again, the change management and the culture of how your company is going to respond to it. Like, don't don't just decide you're going to do the whole banana and just, you know, work over the whole thing.

42:49It's really good to start with one use case and bring everybody along on the journey. I think one of the things that the leadership can do really effectively is to say, OK, like, let's all build together. Let's all learn together, because as we all know, this has not been done before. We're all on this frontier. We're all pioneers together and just being really transparent, but really focusing on the communication about here's what we're learning. Here's how we're pivoting. Here are the mistakes we've made. Here's what we're going to be doing instead. And here's what's working. And, you know, staying really close to the people on the ground who are going to have their hands on the keyboard, doing the human in the loop supervision of the agents as well.

43:24And you don't want to get into the scenario of like the duct tape marketing software and data people have in their orgs right now where like you buy all these tools, the data is everywhere. You're building broken processes everywhere and it's just like you have to strip it. it's better to start small, like rethink the process. Like Chris said, like actually like think through the process, how it would look like with an agent instead of just fitting it into your how it exists today. Because if you keep stacking all these agents on top of each other, you're going to then have to like find a way to pull it back somehow.

44:04And then you're going to be like this duct tape duct tape again. So like learn from the mistakes we made with software with agents. that's that's my like i think like we may i've seen it myself where i'm like i just built random automations because it worked like when i'm a one person marketing ops person in a small company but then when we scale scale to where we have to like bring in have multiple people learn the processes then you find everything that's broken and then you have to peel back buy a new tool rethink the process, rebuild everything. So you just don't want that to happen to you.

44:46It's going to happen to a certain extent, you know, like they jumped in early in the early adopters, you know, there is this agent sprawl. And it's, it's like, you don't want to, you don't want to tell people that they can't use it. You know, the shadow AI thing is real, but it's also a product of everyone jumping in and learning and training themselves, right? Which a lot of companies are relying on their employees to get in and kind of, you know, experiment with this stuff, because there isn't exactly a ton of regimented training that a lot of companies are bringing in surrounding AI. And part of that is because there is so much domain specific documentation or use case or knowledge, like what how marketing is using AI is profoundly different than how engineering is using AI is profoundly different than how, you know, the product management team might be using AI.

45:37And so it's not like you can just say, okay, let's all learn AI together. Here's your, you know, like e-learning course on how to use AI in your job. Everyone's having to kind of figure out how they apply their domain expertise to this. And that's why, you know, what Khan is doing is a little bit different than what some of the other larger AI companies that are saying will build with you. But they're surfacing kind of a technology platform and saying, OK, let's let's build together or like you can build your own agent on our platform. And our point of view is very firmly. It needs to be domain specific.

46:11We focus only on marketing and we just don't think that most marketers should have to build enterprise grade custom AI applications. Like it's just not I mean, I certainly didn't train in how to do that. That is not my area of expertise. It terrifies me. And so I stand really hard behind this. I just don't think most marketers think that they're in the right space to be doing this. Again, they're all learning as fast as they can. They're excited about the technology and what it can do. But we all recognize there's a difference between an individual agent that we build ourselves and Claude and what is going to actually stand up to the test of security, AI evaluation and judging, et cetera.

46:52And that's just simply not expertise most marketers have. and it's actually what you just said i guess the exact reason why marketers are gonna still be have jobs because like you need these domain expertise for this to work like it doesn't just work and it's not even like most marketers need domain expertise and then going deeper like company like knowledge yes like what like like company-based knowledge and then also industry and then and then like and then like industry expertise like a lot of people are in different industries that have different regulations and different ways of doing things and different audiences and like you need this stuff to be and that's why like the human and loop is like that's why i am not scared of ai because like all this domain expertise is needed with marcus but it will expose the people who weren't doing the greatest job that's the only negative or just you know yeah yeah or still you know kind of junior in their role and who haven't yet had the chance to you know get as steeped in it as other people have i think we're all we all share the concern for the people coming straight out of college right now trying to compete in the job market where they need to have this domain expertise but you can't have it until you've spent the time actually in the trenches learning it so yeah but i think the people coming out of college are the same the people like millennials coming out of college who just knew social media they're the ones who are going to be the ones build helping build these agents like i think that's going to be the like they're the ones who are actually down to learn this stuff like and don't have like like years of like i've done this always done it this way knowledge where you're you pushing back so i think we also have a ton of questions so i think we should answer yeah we have a ton of questions so i'm gonna uh i think we've answered some of them but if there's anything else we want to cover um let's see um what's this chris what's the stuff you're doing now that you couldn't do before?

49:12Great question. I mean, I think that building agents is something, I mean, building agents in general is something that like two years ago we couldn't do. So I think that that is the biggest thing in that I think we've evolved from like automations and workflows into building agents. So that's probably the number one big thing we could do. yeah I'd answer it like that I think agents in general are not something that has been around that long I also think this is a great question for you Jessica because I think Kana is like everything you're supposed to be thinking like you couldn't do before because that's like the point of the it's like rethinking your whole marketing org so yeah I mean I will just share like a personal like like a few weeks after coming on to the job at Kana in January we were on with a very large CPG company.

50:03And we were talking through what their use cases were and what their priorities were for the build. And while we were in the meeting, based on, again, like the, just the sheer amount of work that our engineering and product teams have done to develop these applications and these agents, in the span of 20 minutes, based on initial discovery and use case information, we were able to put together like a skeleton application for them that they could see before the end of the call. It's amazing. And that, I mean, I literally, my entire brain was lit on fire. I was like, is that real? Is that real? You know?

50:44So, I mean, just the sheer and, you know, Chris, you, you pointed out earlier, the, the, the coding or the building is not necessarily the hard part here. We can build custom applications for our clients in a matter of weeks, days or weeks, depending on whether or not we can get access to their data and their tech stack and customize it to them effectively. It's really more like the real barrier is access to their team and their time than it is like our ability to build. So that's crazy. But it's just, it's insane how quickly we can get these things done. What's funny is the cultural aspect of, you know, large enterprises, is how they evaluate and procure new technologies has not evolved at the same pace as like our ability to build it to their specifications.

51:35So that continues to kind of make me laugh a little bit because that timeline for evaluation and procurement has not sped up as much. I'm interested in this question for you, Chris. It's like, were there any roles you tested as an agent that kind of fell short for you or like didn't exceed or like didn't hit the expectation you think the agent would have done for that role? It's a great question. I'd say not yet. And for the most part, these agents, the way that we think about them are, it's a role that can do a lot, that have a lot of abilities and skills and tasks. And so we're, instead of boiling the ocean and saying like a field marketer needs to do these 30 things perfect day one, we're kind of stepping into it saying, hey, do this first three things and then we'll get to these other things later.

52:24but these are like the lowest hanging fruit of like what we want you to start driving outcomes on. And so because of that mindset, it makes it easier for us to not boil the ocean with an agent that might do some stuff bad. But it's also an iterative process again. So that's why the humans in the loop where, you know, an agent might start doing something and we're saying, hey, do it better like this. And we're tweaking it and making it optimized. So, so far, I'd say the answer to That is, you know, all the agents we built, we're continuing to iterate and evolve. And I think that's also one of the beauties of starting fast instead of like waiting years and because you have paralysis on like you're not ready.

53:07It's like I think start today because you can then learn tomorrow, the next day. And then in the six months, you're going to be so much farther and so much better off because it's all about iterations. and like you know there's just so much that you need to kind of tweak as you as you scale these agents yeah do you feel your do you feel like the ai has made your team faster overall or is it or is it just getting rid of the busy work and they just doing more creative stuff and more new stuff uh way faster uh way more new stuff doing things that were like 10th on their to-do list that they never would have got to that's being done without them really doing anything.

53:48Everything is up and to the right. So I, yeah, I can't say enough on how impactful building agents have been for us. Yeah. I think the other thing is it's not just more and it's not just better and it's not just faster, but to keep in mind, you know, again, I was talking about kind of the decision phase where it's a stop, start, continue. And it's really important that you focus on the stop portion of that, right? Like there are things that we now have better insight that we should not be doing anymore, or we should not be spending our money there, et cetera. And so those are when you're measuring the impact and the contributions of agentic marketing or agents in your marketing strategy, really focusing on what money costs, savings time meeting time have we saved as a result of stopping doing things that we'd done before because we thought they were working and in reality they really weren't the other thing i'll say too with our like building agent strategy from like an employee satisfaction or we have this enps is it's actually um you know created a better working environment for a lot of our employees who felt bandwidth constraint before and are now being able to accomplish more the other thing it did with centralizing and building these agents was it eliminated some of this like, hey, I'm behind as an individual trying to build agents.

55:10I don't know how to build them. I have to stay up late at night trying to figure this out on my own. Because I think a lot of companies for a while had this like, every employee needs to token max and do AI. And if you're not, you're going to get fired. And it's kind of this like usage theater, I call it, where it's like the more usage, the more mature you are as an employee. And so for us, when we consolidated this center of excellence and building out these agents, it allowed employees to take a deep breath and saying, cool, I now can contribute and co-build these agents, but I don't have to feel like I'm going to either A, lose sleep on not knowing how to do these or feel behind.

55:49And so I think that's been helpful. And so we have this two-tier model. We have an AI for all strategy where everyone can get licenses to LLMs and make themselves more AI literate. But then we have this agent strategy which is like helping the entire company, you know, drive results. And I think that's been a win-win. I think this is a good question about like, I know a lot of people are scared about like data and like, like data being shared that it's going to be like common knowledge with like the whole like competitors and every like different clients and whatever. yeah yeah like yeah and so what what how how do you like help with that like how where like how do you keep it to your data is not being leaked to and being trained on so um i think jessica you could probably answer this part um as well yeah i mean you know when we were so one of the things about the kana platform that we built is that's like a walled garden and so it is extremely secure None of your data, none of your queries, none of what's going on inside of the applications that we custom build for our clients is being submitted for the models to be trained on.

57:08So, you know, that's one way to do it, obviously. I mean, Chris's approach of the center of excellence really helps because it kind of helps to at least reduce the shadow AI, which is really kind of the issue where if your employees aren't trained on, you know, you cannot literally upload Excel spreadsheets full of PII into your personal cloud, you know, etc. That, you know, they are using that for training the models. That's the shadow AI is really the big issue there. But if the company is saying, here is the enterprise level licensing, which they have set up the actual contract, the agreement that it will not be used to train the actual models or whatever the specifications are and have redlined it themselves.

57:51I think that's generally the kind of the most important thing, but it has to happen on top of, again, a basic level of employee acumen and training where we say, hey, understand that if you don't have that specification in the contract and in the license, that anything you're uploading there is being used to train the larger models. Someone was curious about like what your web specialist agent does. Does it manage the website or what does it actually do? Yeah, I mean, so there's a few things off the top of my head. One, it's going to help look at like technical SEO for the website, where if there's something being added to the website, it's scanning for that.

58:38It's giving out SEO and AEO recommendations that the team Human in the Loop is deciding on. Should we create content around? There's a lot of reporting that we would never get to that gets pushed to us through like an agent that's looking at, you know, what pages have a lot of X and 10, where do we want to optimize? It's suggesting optimizations. So the way that I think about it is like if you were to go hire like a web strategist, what would be like the 10 things that you'd want to hire for that person to do? And then that's what this agent is doing. Some of them are live. some of those things that you might want for us are still in the works, but the things I just explained, those are all live and we have a backlog of another handful of things that we want to add to it.

59:26We also have this central strategy too, where for employees that are engaging with this web strategist agent, they can then suggest ideas through the Slack app through super orgs, and then that'll just queue up for future abilities this agent can do. So again, I I continue to think about this as an evolution where the agent day one is not going to be as good as it is day 100. And so you've got to just start now and then continue to add and make this agent better over time. And I think having like someone manage it, having someone having those performance reviews, having someone like being able to see the outcomes is like it's like how you would manage a human as well.

1:00:08You see on day one what they're doing, and then day 100, you hope they're doing a better job at their role because they've been getting all that knowledge of the company, and they're producing something. So I think that's – I know we're running – Yeah, can I just cover – I want to make sure – this is one of the key things that I really wanted to make sure to convey during this webinar, which, again, that agentic divide research report that we did. We asked everyone who should own an agentic marketing technology procurement evaluation and implementation process. And 40 % of the respondents across 225 respondents said the chief AI officer should be the owner of agentic marketing.

1:00:55But amongst the CMO cohort, a larger percentage said, oh, the CMO should still own it. And so, you know, when I read that, what I see is that this is an opportunity, but it's also a huge risk for marketing leadership and for, you know, the day to day marketers within the team to ensure you have a seat at the table and you are in the technology helping to evaluate it and to inform the technology team or the AI team, the innovation team on what it needs to be able to do and how it needs to work within your workflows. right or the way that you actually get your work done so I just want to make sure everyone on this call who's in the marketing function stay at the table if there's an AI technology decision that's being made for your organization related to marketing make sure that you have representatives in the room and in the process because agentic marketing is not going to work if it's purely a technology AI decision and it doesn't take into account again all of that domain and specialized expertise from the marketing organization yeah plus one i mean i've run into problems before ai where like then people don't understand what marketing needs and what we actually need and we're making technology choices of um like we can't buy a tool that will help us 10x our results because like it there's like an it problem for it and it's not like a security or anything like that And it's like, I can't access a field in Salesforce.

1:02:27So it's like something like easy as that. I want, I know Chris, you are willing to like talk to people after if they email you. And I just wanted to leave you with both like closing remarks. So people can, if they could, where they could find you, where they could find more to learn about this stuff. All that good stuff. Yeah, Chris, you want to go first? Yeah. Add me on LinkedIn if that's easy for you or email me. It's Chris, K-R-I-S at Sendoso.com. I, as you can tell, I'm passionate and love geeking out on this. So if you want to go deeper on any of these topics, always happy to throw one-on-one on the calendar.

1:03:04Yeah. And same here. Connect with me on LinkedIn, or you can email me at jessica at kana.ai. And we also put up some other resources if you're looking to kind of just dive into this and understand a little bit more about what agentic marketing eventually is going to look like in the future in space. we invite you to learn more from some of these resources we've got up well thank you everybody for joining and thank you both um you i've learned a lot from both you and um i'm sure everybody did as well and thank you again all right thanks everybody thanks so much for listening keep tuning in to hear more great insights from the coolest marketers from around the world.

1:03:49If you haven't already, make sure to subscribe and follow the Marketing Millennials podcast on Apple Podcasts, Spotify, YouTube, or wherever you get your podcasts. And if you like what you hear, I would greatly appreciate you giving us a five-star rating. It helps bring more marketers into our community.

1:04:18Thank you.