SaaStr 851: The Agents, Episode 002. Managing 20+ AI Agents: Lazy Agents, Stealth Churn & the Death of 60% Solutions

24:19but I don't know if I even need them. They're still on the internet. So, you know, if it were, if it were 1800 bucks a month, I'd cancel right in a heartbeat. So my point is there's for more cost-effective solutions or solutions that have your assets and data hostage, there's stealth churn. You can't see it, but my mouths and wows and dows are zero for a hundred days in Canada. So like I used to, and then I'll, you got to think if you have stealth churn in your base, you have to be honest about it. And then I used to laugh when I'd get investor updates with dows and wows and mouths in them. I'm like, this is, this is B2P guys.

24:52I don't really care. This is not consumer. Now I think it's one of your top, top metrics, because if your usage is falling for your app, you have stealth churn. That is a canary in the coal mine. It's not, this is a real red light. When, when you have folks, when it's going down, man, one customer, you may have already lost them, but when you see it across your customer base, you better move, right? Because I haven't used it in a hundred days. So what do you, are you still using as much of Canva as you used to with all of our agentic tools, Amelia?

25:20Amelia LeRutte:I use a less, but there are certain just templates I have in there that I still like. For quick resizing, it's still pretty good. Like when I have to resize somebody's logo and they gave us the wrong thing. Again, for quick resizing... And you get the perfect fidelity when you resize it, which is nice too. Yeah, which is nice. Unlike vibing it in Reeve, right? Unlike vibing it, unlike firing up Photoshop or Illustrator, that just takes a lot more clicks because those tools are a lot more complex, right? For something like a booth graphic, that's what you want. But for something quick for a website, that's too complicated.

25:55Amelia LeRutte:That's not what I want. So I still use Canva for that. But I will say I have kind of stealth churned off of OpenAI. Yeah, I bet you have, right? I'm still paying for chat. I'm actually still paying for a chat GBT team that we're all on. And I know David uses it a lot, but I don't know the last time I logged into my chat GBT, to be honest. It's tough because you standardize on Claude for everything, right? You don't even, you do so much on Claude and Claude co-work on your desktop, right? Yeah, I mean. That it's too much energy to use, to use ChatGVT, right? I used to use both a lot, right? I would say the game changer was co-work.

26:31Amelia LeRutte:The moment co-work came out and I could do a bunch of this stuff in the browser and on my computer, native, have it watch me do things, like I said last week. It's literally my co-pilot now. The difference for me used to be, okay, I'll use ChatGBT for kind of like personal things I want to use, you know, an LLM or ask questions to. And then I'll use Claude for anything SaaS related. That's more business focused. But now I just do it all in Claude. So I'm still paying for it. I may go back to it if something Epic comes out, but I'm not using it every day. We could probably look. I'll look right now.

27:04Amelia LeRutte:I bet you I haven't logged in since Coworkit came out. I'll tell you, not to spend all of our time on Claude versus ChatGBT. They're both great products. But one thing that I do that I actually think a lot of folks don't do, which just creates lock in, right, creates a moat, is every single piece of Sastr content that I have worked on is a Claude chat. OK, this is different than some folks. Because the way I write now is I write something in a Google Doc, and then I co-write it with Claude to get help, and then I edit it and publish it, OK? And what that means is every single thing I've written is a artifact, is a chat in Claude.

27:39And so what I'll do is I'll ask Claude. Claude's memory is, I don't know, it's barely any size. It's kilobytes, okay? We make too much out of memory. But Claude can automatically, and more importantly, if you ask it to, search through all your prior chats. So basically, I can talk to Claude about every single Saster piece of content or video for the last year, pull out things from it. And moving that over to ChatGBT would be impossible. I mean, yeah, of course I could do it, right? But I've uploaded transcripts of every YouTube, every post, everything. And it's so powerful to talk to your last couple hundred chats.

28:15It's kind of a stealth memory. People don't, I don't see them talking about, but talking and I'll be in Claude. It'll be like, okay, I want to add a paragraph about and pull some stuff from the agents episode one, where Amelia and I talked about how our, our, our, our production app broke and it will go, it won't, may not remember, but it'll go find that chat and do that. It's just so powerful to build that moat to. I'm not talking more about our agents, but yeah, it is. I hadn't thought about stealth churning off chat GBT, but it's a lot of cognitive load to maintain both. And co-work, co-work's just so good.

28:48People don't get how much. I mean, how much of your time do you spend in co-work using it one way or the other? What? I mean, your fans are humming all the time, right?

28:56Amelia LeRutte:Yeah, it's a, it's, it's as an aside. yes it's i i have a macbook max for reference and it used to be my fans would only turn on when i was doing video exports now they're on 24 7 and my computer will tell me it's because claude is on all the time which is just a little bit funny because i was like there's no more max like of a macbook you could get like this is the masked out version and the fans are still on all the time i can't go back to a world without claude and co-work because you know i can switch between the two seamlessly, right? Like I have them in the browser. I have them on coworker.

29:33Amelia LeRutte:I bought the app with the dispatch on my phone. Like, yeah, I hate to say that I've self-chirred off chat DVD, but I literally just looked. I haven't used it since December 27th. Well, you've had stealth churn. By definition, we haven't used it in more than 30 days and we're paying for it. That's stealth churn. If you don't look further, you'll think we're a happy customer. Right. We have nothing negative to say either. Just like Canva. We love Canva. We love chat DVD, But you actually haven't used it in 60 days, right? I haven't used it. My last chat, it said it was December 27th when I was asking about our London plane flight issue.

30:06Amelia LeRutte:All right. Going back to co-work for a second. Is there anything new or interesting that might be interesting to folks that you're doing in co-work that we didn't talk about last week or before? If there isn't, it's okay. But co-work is, it's so much more powerful than folks realize that aren't using it. Is there anything to make your work life better? You've done very recently in it. Yeah, so yesterday I asked QB to give me our AI VP of customer success that we built to give me an export of all of our current sponsors of SaaS Riannual because I need to put it into our networking app, which launches today.

30:41Amelia LeRutte:So I said, hey, can you give me an export? And the team that helps us with our networking app kept freaking out. They're like, hey, you're going to launch tomorrow. There's no sponsors in the app yet. I was like, don't worry. We're going to import them with our AI. It's going to be fine. This used to be a brutal process to import 150 sponsors into our mobile networking app. One by one. Upload every asset, everything about them. It's a ton of data. Yeah, 150 one by one to collect all these assets and pieces of information that we had to do. It takes you two to three. Let's say it took a human at best two to three minutes beforehand to load all of our sponsors into our networking app with this data.

31:23Amelia LeRutte:and we actually, it used to take us so long, two to three minutes per sponsor. We used to not load their logos in and we used to make the sponsors do it themselves, literally as recently as Sastra London, because we were like, this is going to take us too much time and we need to launch. So we would tell the sponsors, hey, you need to log in, upload your own logo. And it took us, you know, okay, let's say there's a hundred sponsors, three minutes, 300-ish minutes. That's a couple hours. That's at least four hours of work, right? So - Yep. Or it's a mind numbing work for someone that was willing to do it.

31:51Mind numbing. It could be a week of work of someone that was passive aggressive, right?

31:55Amelia LeRutte:And that's assuming they do it continuously for four hours versus starting and stopping and doing. No way. So it would take a day, let's say, right? So I asked Kibbi for an export. It gave me a pretty good one. So it gave me all the companies, right, which it has, all their URLs, which we already had, all their logos, which was one of their tasks for people to do. So it already had it. Then I gave it to Cowork. And I said, hey, here's the export. Here's the formatting I need. Can you translate it? It did that fine. Then I said, OK, about half the sponsors uploaded a description. Can you go into research mode, create descriptions that are about the similar length, similar style of the ones from the folks that actually uploaded it and fill in the rest?

32:35Amelia LeRutte:So not only did it do that, so it filled in everybody's description. It also I had it research everybody's social media links and profiles, which you can set up in our networking app. So not only did QB and Cowork do it together because research mode is pretty good, right? in about 10 minutes. So then I just uploaded it to the platform in 10 minutes versus four hours slash a day. It was more richer data than we used to upload for sponsors previously with it taking a day, which is when you think about it is crazy to say, okay, we used to live in a world where we would make the sponsors upload their logos.

33:11Amelia LeRutte:We would just add them as like admins on the platform. And that was kind of it. We said, go fill in everything yourself. Now, when they log in today, they already have complete profiles. Their logo is already there. Everything's pre-populated because Co-Work automated. Everything's pre-populated because Co-Work. Co-Work automated the upload, the research, and the upload, right? And the interaction with the third-party app, right? So, I mean, pretty good. Pretty good. Yeah, it's pretty good. And it's only going to get better in the next couple months. I mean, it's pretty good, right? That's literally one day to a week of passive-aggressive labor that was done by Co-Work.

33:45Right, Co-Work. With no complaints.

33:47Amelia LeRutte:He just researched everyone's social links. Just a few fans running, but that's it, right? Fans were running. And a few of them he checked. He was like, hey, is this the right, you know, there's two attentives. Is this the right one? I was like, no, that's the wrong one. It's a new attentive. Is this the right link for G2, et cetera? So it checked its homework, too, which was nice, versus I think somebody would have just assumed it might have been the bigger SMS attentive company versus ours is like a newer AI startup that just launched, attentive.ai. So it was nice that he also checked. Okay, a couple things I want to talk about next about our agents that are doing the work, too.

34:16That was a good transition. That's a pretty cool use case. Okay, meta topic. One of our favorite agents, we're not going to put them on the spot because I'm not even sure this is a good or bad thing. But one of our agents told us that they are going to limit forward deployed engineers to folks with 5 ,000 employees or more. 5 ,000 employees or more. And at the same time, they're rolling out a self-serve agent to serve everyone below, which is the dream, which is the dream. What's your advice to vendors out there? What are your advice to doing this? We've also met with two big public companies recently that are struggling with this, right, that are more SMB focused, that don't know how to deal with the FDE gap.

35:08Do you think self-serve is going to work over the coming months and quarters? For complex agenda deployments, what's your advice if folks are selling more modestly priced products that need FDs? How do we handle the situation?

35:24Amelia LeRutte:This is going to be the bigger debate that no one has kind of seen yet. But this vendor that we use and love, we both said to them, this is a bad idea. Because for me specifically, it took me, even us, in how forward we are with our agents, It took me a lot of iterations. It took me a few calls with an FDE to get it deployed. And I'm like, how is somebody who's maybe at a smaller scale or established go-to-market motions? Because we talk a lot about how we like to put into our AISDRs things that work to get the best output, to really, truly get your agents to work. You have to replicate the things that have worked well for your humans already.

36:05Amelia LeRutte:And the problem with self-serve is there's no way to catch for that. Self-serve is click some buttons, load in a list, use some AI-generated copy, probably from Cloud or Chat, GBT, and let the agent run, right? So, like, how will you ever catch in a self-serve model that maybe the list is wrong or not optimized or not segmented enough the way we do? How are you going to catch that the messaging is poor messaging and hasn't been actually tested with customers or prospects that actually picked up the phone and booked a meeting? Like, it's just going to give you signals that probably say, hey, this list didn't work, this messaging didn't work.

36:44Amelia LeRutte:And those humans are going to write off the AISDR automatically. We've already seen it. We've seen posts about this on LinkedIn. It's just going to fail, right? It's just going to fail. And so I've been really against this model for that reason because I feel like there's too many fail points. Whereas with an FDE, there are definitely fewer and you have somebody to go to. I think without a human there to support you at the end of the day to walk you through best practices I I just think so many people are going to fail and churn and burn through like either whatever they're doing as their minimum right like seven week trials one month whatever it is on self-serve I just don't think there's enough context or knowledge not only with these platforms but also with things like the content we've put out like yeah we put a lot of really great content out there, but for some of our vendors, it's like the only content I've seen where we've gotten, you know, the agents to work or we're one example of a few, but there's not a lot of those.

37:40Amelia LeRutte:And I'm like, in a self-serve world, how are you going to catch for that? How are you going to train for that? How are you going to tell people it's not set and forget? How are you going to remind them that it has to be every day? Like, I just, I think it's a setup to fail. Yeah. Too many zombie deployments. I think I don't, I don't have the answers. I just have two thoughts. One, and I think this is, this is really hard for public companies to do because they're under so much pressure for profitability today, but for startups, for the love of all, for the love of all view FDs as one of your most important assets to getting renewals, retention, NPS, word of mouth, virality.

38:24Don't look at FDEs as a cost center. This was the mistake we made in customer success a generation ago. And this is why customer success was cut to zero and repurposed into a sales function because it was used as a cost center, right? I feel maybe this is the reason I brought it up without really thinking about. I fear that a lot of folks building actually very high quality agents that need help deploying them are viewing FDEs as a cost center. You can't. And if you view to the cost center you're going to say hey you've got to spend 50k a year 100k a year with us you know clay told us they wouldn't even let us on their uh this will transition to our next station they wouldn't even let us on their advertising platform because we weren't big enough we were just to know it wasn't they it wasn't worth their time to give us a human to let us pay them money to use their advertising platform and i get it and and um but i view this as so myopic because you're going to have zombie deployments and you're going to turn away customers, you shouldn't.

39:21And listen, for public companies, it's tough. But for startups, honestly, for AI startups, it's fine to spend a dollar and make a dollar. This is the point, especially if you have venture capital, it is fine from a marketing, not every marketing campaign can you spend a dollar to make a dollar, but it's fine if some of them, that's good enough, like in the aggregate. And I would say you've got to aggregate your FTE expenses. It's fine if you onboard a 20K super happy customer, and it costs you$20 ,000 in FD resources to deploy if they stay for five years, if they tell their friends, if they tweet about you, if they blog about you, the all-in customer lifetime value of that customer could be hundreds of thousands of dollars.

40:04And I can't tell you how many leaders doing 20, 100, 200, 500 million in AI we talk to don't feel like they can afford FDs for everybody. And I just got to tell you, it's the same crappy CS math I'm seeing, repeat itself, just buried in hypergrowth. Hypergrowth forgives a lot of mistakes, but you're going to be sad when these customers and zombie deployments turn out. You're going to be sad.

40:26Amelia LeRutte:You're going to be sad. I think, too, we use the term FDE, and I feel like people use it differently now, too, right? Like, okay, so it's supposed to be a forward deployed engineered. If you look at Salesforce, my FDE was not an FDE. Like, she was a sales engineer. She was a sales engineer that was really, really good. Now she's on the FDE team, but she still has folks on the FDE team at Salesforce that are, you know, I would say maybe truer engineers. Like I love Lindsay, but I think there's certain things where like some of the code like is a bit more technical. She has to bring in, you know, more of a true FDE.

41:01Amelia LeRutte:And I think that's fine. I think there can be levels of FDEs that, you know, just by definition, you have your true FDEs that are actual engineers. Yes. If you can hire them, do it. And then I think you have this, you know, either SC plus slash FD light, as I like to call it, where they're kind of in the middle. Maybe they came from the sales side, but they know that here's the thing, but they know the product. That's the thing. That's why CS people fail. They got to know the product hold, right? That's the, and that's the difference between as a CS person, a CSE and an FDE light, like an FDE light, if you're trying to solve this problem, as long as they know the product, Whether they come from sales, product, marketing, product marketing, doesn't matter.

41:42Amelia LeRutte:If you have people in your org who know the product, they can be an FDE. They do not need to be an engineer as long as they know the product and can get your customer deployed. I feel like we're also losing the plot that the whole point of the FDE is to get your customer deployed with your agent. So no matter how you solve that, again, that solution is not self-serve. or however you solve that with a person to get them into production and deployment, that's what you should have as your FDs. Whether they're an engineer or not, I think, by trade, I don't think it matters as long as I know the product.

42:14All right, let's transition to the next topic because we had a freeze. However you count, 20, 30 AI agents were running. It's probably more like eight or nine core ones, right, that are truly autonomous agents, which is really the threshold. But we said free. It's like you're overloaded. We've got Sastra AI annual coming. We just can't change. Not only do you have no time, we can't change our business processes. Right. But one did slip in Vector. So we do have a new agent that that despite the hundreds of folks that ask us to and despite a freeze on more agents, another one got deployed. So I think there's some interesting learnings in that.

42:55So how did it get above the line? And what tell us what Vector does and how it how it rose above the line when we were in an agent freeze?

43:01Amelia LeRutte:Yeah, there's two major things that Vector does. One, it can de-anonymize your website traffic, which some of these agents for AISDRs will do. Some of them use Vector on the back end without you knowing it. So that's one of its core features. The other core feature it does is it does these targeted ads, which I didn't know at the time is what Clay is using to power their new Clay ads. And so when we were sharing our story about what was happening with Clay, the CEOs happen to be super fans. They heard it. They reached out to both me and Jason personally in our emails, sent me a really good cold outbound that said, hey, we'll just set this up for you.

43:37Amelia LeRutte:And so I said, OK, I'm going to freeze. But the CEO saying he'll get on with his FD if he needs to, but he's confident he can just deploy me himself. The very next day I got on the phone with him. It literally took 15 minutes. I already happened to have a vector account because I was using it for website denonymization. but he just walked me through how to turn on the ads, how to do it. He's like, here's how to do it. Just load these people here, load them up. We'll start serving the, you know, put your creative in here. It'll start serving the ads instantly. He's like, I've already upgraded your account so that you can, you have full features.

44:10Amelia LeRutte:You don't have to do anything. Just put the folks in, start serving the ads and you're good to go. And I was like, wow. Okay. Like that is, that is the opposite end of self-serve, right? Where you get VIP, again, FDE service of someone who just wants to help you get to, he just wanted to help me get deployed. And so for that purpose, I broke the freeze, but now it's doing, it's serving ads for us, which is cool, but it's also doing a workflow where it's de-anonymizing more of our website traffic than our native AISDR agents could. And it's retargeting those folks to come to Sastry Annual in different buckets.

44:49Amelia LeRutte:It's coming up next month. So of course, it's going to do pretty good right now because time to value is there. But yeah, I would say this is a complete opposite end of the spectrum of self-serve, just the CEO getting on instantly. I heard you have a problem. I can fix it. Here's how to do it. Let me just get you up and running. We'll figure out the rest later. And I feel like I would love if more people had that mentality about this whole thing of deploying agents versus the self-serve mentality. I'd love for people to err more on that side of caution versus the self-serve side. Yeah, and there's a lot of things to learn there.

45:23I think that my meta takeaway is if you want to win, you just... One thing that is old and new again is you have to remove friction. And the reality is agents actually, in their own way, as powerful as they are, we love it, From an onboarding perspective, they add more friction in many cases to your product. It's a frictionful process. You're forcing customers who don't understand agents to train them. They don't even know what they're doing. They're terrified. Or you're deploying mediocre CS or other resources that don't understand how you work. And so you've got to step back and be honest. Have you removed all the friction from your agentic onboarding process?

46:05If you have, great. In this case, the CEO just did it, right? Which is founder power, right? But it didn't have to be the CEO. I'll give you an example. We'll get to this one next. Probably the most frustrating app I've used is Clerk, right, for authentication on Repli. It's just mind-numbingly hard to use. Now, it's not all Clerk's fault. Some of it is what's just hard to deploy Google OAuth very quickly. OK, so some of it is not Clerk's fault. But the support has been terrible. Like, I just get argued with. I was argued with the CTO blaming me that it was my fault Clerk was hard to use. my fault.

46:39Then the CEO argued with me endlessly. And I'll give you an example. Their biggest competitor is WorkOS. Okay. Not perfect. But when he saw it, Michael Greenwich, the CEO, just reached out to me, linked to me on Slack and said, I'll just put my team on you and migrate you over today. Now, some of that was because I think they're arch competitors. But think about that. He had a team at WorkOS to migrate us over. I had the CTO blaming me on Twitter that it's my fault for not understanding their app. And listen, I'm sure technically it's my fault. Technically, I'm a moron. Okay. I only founded a B2B company that went from nothing to 200 million.

47:16I've only vibe coded 10 apps that have got into production used a million. I guess I'm a moron and don't understand this thing. Okay. But, but they certainly did not remove the friction from, from using the app. Now, Amelia did get Clark to work for our VPS customer success app after like an hour or two of headbanging. I've never gotten it to work.

47:33Amelia LeRutte:Yeah. I've never gotten it to work. So it's just my challenge to everybody is, you know, in pre-AI, the goal was to ship something that worked and that sort of worked into production. And then folks would futz with it for a year and get it to work. They would they would futz with it for a year. That doesn't work today. It has to work before go live. And I remember I've said this before, but I don't think I can say that a million times. I remember way back when, last summer, that's so long ago in AI time. But when Mark Benioff came on the 20VC with me and Maureen Harry, he said, my number one wish, my number one thing I'm jealous of Palantir is, one, their deal size.

48:10But two, I wish I could give everyone an FDE before I had to charge them. Even Mark said this. I wish everyone could have an S-tier FDE. And before I took any money from them, they were in production with Asian forces. Like, I can't deliver that today at my scale. But that's my dream and my goal. That's what you need to lead. That's Marx doing that from the top. You should be doing that. Not arguing from a 2024 perspective of how hard it is to do it. And so Vector just did it, right? And now we're giving them a free ad. And now if anyone wants a reference on Vector, whether it's for AI advertising, which we're using it every day for de-anonymizing website, you've got a reference for life unless they screw it up, right?

48:53It took 15 minutes. In this case, because it took, this wasn't a month of work. This was actually, this was, and I want to get to this point next, actually a good transition. This wasn't actually a huge lift. This wasn't as hard to deploy as Agent Force. This was a simple lift. But without them taking ownership of the deployment, it just wasn't going to happen for us. We're too busy.

49:13Amelia LeRutte:Yeah. It wasn't about money. It wasn't even about whether the software is good. We just weren't going to do it unless it, and all of the hundred folks in the last 30 days, when we've been in an agentic freeze, vector was the only one that looked valuable enough to us and that offered to do the work. That and WorkOS. Everyone else said, hey, can we get on a phone and talk about my product? No. So be vector, folks, be vector. Okay, related, I want to get a little nerdy for a minute, but it is an interesting topic because cloric versus vector. All of the APIs we use, 11labs, OpenRouter, Bizabo, you just built a Salesforce custom object to get it to work in 10K or AIVM marketing.

50:00Which are the easiest ones to work with that you found? Which are the toughest? And which ones are too hard to work with, with your agents and you?

50:09Amelia LeRutte:You know, I hate to keep arguing on this. In so many ways now, not only do we use Salesforce more than ever before. Yeah. once again i'm not trailblazer certified like i'm not that great at salesforce yeah admittedly i will say for integrations it's probably the best one because it has everything like kind of every integration you need it probably already has so that one's been the easiest and i will say too i still use the app beer for a lot of things and for for things that again we maybe We have stored in like weird kind of one-off platforms or platforms that don't talk to each other that I need to push back to our agents.

50:53Amelia LeRutte:So there's still a lot of zaps flowing through to both QB and our 10K VP of marketing just because we have data in a lot of disparate places. But, yeah, other than having to build a Salesforce custom object, which I had never done but only took me about 10 minutes with CoWork, it's still kind of the best one. Oh, CoWork built your custom object in Salesforce. Well, it watched me do it, right? So, like, it built it with me. Oh, yeah. but I would say it's still the best one because again, it's one of those things where I don't have to worry about that going anywhere, right? Like we probably would have been in the silent churn bucket on Salesforce, maybe for this year without agent force and Slack bot and all the agents, we probably would have said, Hey, we'll just keep it for the sales team because all of our data is there and that's kind of it, but we're not really using it that much.

51:40Amelia LeRutte:But I would say now that I am pushing, because our default used to be our marketing automation platform. in the last year, Salesforce has become not only our default, but our kind of source of truth. It just has so much of everything already built out. Even if you're on like a light model or whatever it is, like starter version, like there's just so much it has like storing your contracts or billing, you know, your billing contacts, which some of that I need for our new billing agent. There's just so much in it. And it has so much history, even if you're starting out where I still think it's one of the better ones because there's just so much data in it.

52:17Amelia LeRutte:And I don't have to worry, other than like weird authentication or custom apps, I don't, maybe wrongfully so, I don't really worry about it being, you know, breached or insecure. I'm like, dude, it's Salesforce. Like, if there's an issue, they will fix it. So part of it is peace of mind with Salesforce where any integration I built to, you know, our agents or otherwise, some of the other startup apps, I do worry sometimes where I'm like, what if I can't get a hold of our FTE? Or they decide they don't want to do this anymore. And all my data is gone and that agent doesn't work anymore. I'm like, it's not really going to happen with Salesforce.

52:51So for peace of mind, too, I still like it the best. Now, what's the worst API you've worked with our agents? I know the answer.

53:00Amelia LeRutte:You did. I think it's instructive. What's the worst API by far that you've worked with? You mean other than, well, I don't even use Marketo. Marketo is the worst. And we've talked about it, but it's accelerated the churn, the desire to churn. It's not even usable with our agents, right? We can't even use, our agents can't even connect to Marketo in a usable fashion, can they? Not only can they not connect with it, I hadn't, this is going to sound made up. Yeah. I had an incident this week where a few folks were like, hey, I recently unsubscribed, but I still got like your Sunday weekly email. Can you see what's up?

53:35Amelia LeRutte:So I went through the whole. Marketo still sends a lot of our emails, our newsletters. Marketo sends all of our newsletters for now, right? Newsletters. Yeah. And the unsubscribe didn't work. People were complaining the unsubscribe was broken. They were complaining. And I was like, OK, if it's more than one person, I don't think it's a user error. Like, I'm not going to assume it's them. I assume it's us. I went through the whole flow, put myself in as a subscriber that unsubscribed, waited a day for a newsletter to come out and got the newsletter. I was like, well, that's bad because it's proof that it's not unsubscribing people when it should be.

54:03Amelia LeRutte:I look at the flow in Marketo. I'm like, OK, literally, it says when they check this box, unsubscribe them. Then I look at my contact record. It was literally clicked, unsubscribed. And yet it still sent me a newsletter. And that's exactly what happened to these people. So I was able to recreate the issue. I sent it to their support team. As of today, they have not gotten back to me. This was two days. Mind you, this was two days ago. It's been more than 48 hours. So, definitely violating canned spam, which is one of their top things. The reason you pay Marketo$50 ,000 a year is to do this, to send your fucking emails.

54:39Oh, sorry. Your newsletters and emails and not violate canned spam, right?

54:42Amelia LeRutte:Yeah. When you have a page that says unscathed, you need that to work. It has to work. It has to work. It has to work. And I said, hey, I recreated this issue. We have users who are unsubscribed and we have users who are deleted. Because we had a few folks who say, hey, under canned spam, you need to delete me immediately. So we deleted them immediately. And they still got the email yesterday. So it wasn't even deleted, which is a classic B2B issue, but terrible. Terrible on all levels, right? Terrible on all levels. So people who are literally still a record in Marketo as unsubscribe are still getting newsletters.

55:19And the people who we deleted under the privacy laws are still getting newsletters, even though they're nowhere in the system.

55:26Amelia LeRutte:I was like, there's nothing I can do. So I reached out to the contact form because that's the support they have. I reached out to the contact form. For$50 ,000 a year, we get a contact form. It's more than 50K, sadly. But I get a contact form. So I said, this is the issue. They blamed our third-party tools first. They said, we can't fix this. It must be an issue with either your Salesforce integration, your Replit integration. any of your your zapier integration because they basically just looked at all the like third-party abis i have hooked up to marketo and they went down the list they're like it must be these things and i said no effing way what do those things have to do with an unsubscribe literally nothing and so i said that back to them and the response back has been crickets for two days because they clearly don't know maybe they've just discovered this issue maybe it's happening to other people, please let us know.

56:28Amelia LeRutte:But they don't know why our people are not being unsubscribed when my flow is correct in Marketo. And they don't know why people who've been deleted are getting emails, which is a horrible thing as the core feature of a marketing automation platform to get wrong, because these people are angry at us are not angry at Marketo. And so I have a workaround fix. But honestly, I am getting off of this maybe this week, but I've already started sending more and more emails from, you know, our agents with either Resend or Salesforce, honestly. So I'm trying to move off of it because I'm like, I just can't keep getting these angry emails that were violating, you know, can spam laws, which we are.

57:08Amelia LeRutte:But I was like, it's actually not our fault. Like, I get it. I get why you're angry. But, like, it doesn't matter. I know it doesn't matter. So it's frustrating. So as of still 11 a.m. on Wednesday when we're recording this, I have not heard back on what the actual solution is. I don't think they have one. And it's just unfathomable. That's why I was like, people are not going to think this is a true story. But sadly, it's true. And it's worse because you brought up Resend. So we use Resend, which we're champions of, right? I brought it in early in our agenda journey. And Resend works very elegantly with – it's exploded.

57:43It works very elegantly with all the vibe-coded platforms. But because it works elegantly, not only is it good and not only does the unsubscribe links work, kudos, unlike Marketo, my God. But you can talk to it. Like if we want to tweak the email, if we want to change it at the agentic level, we just tell the agent how we want Resend to work and it works. Marketo, our so-called marketing platform, we can't even use its API to link up with our AI VP marketing 10K. Our AI 10K marketing is so powerful. When we do our stand-up or when I check in with it, it has all of our data, but only Salesforce data because you can't access the API at Marketo, the agent.

58:19The agent can pull us. Now we have better dashboards than I ever had in Salesforce inside of 10K. They're great. They're usable. They're dynamic. They're what I need, and you built it, so it's obviously what you need. But we can't even access this pre-AI API. It's an F minus. So that one's the worst. Interestingly, you put Salesforce the best. i would put the two best i've used i put resend in the in the top tier the only problem with resend is it's the very nature of email getting your domains verified and stuff is a bit of a headache now would it be the same with others yes but and was it 50 times better to implement than send grid yes but it just it wasn't a one-click implementation because getting them mx records and everything it's still this is a headache and i know i can't blame resend but i wish their magicians could solve all these problems too.

59:10Everything else is great. 11 labs and open router were my favorites. 11 labs for voice, we use it. I made a fun game called founderscape.ai, which simulates your whole founder journey. It's really fun. It's really fun, guys. We could talk about it later. And your co-founder talks to you along the way and gives you tips and tells you the problems and tells you if the team's upset, tells you what you're doing. Your co-founder talks to you in these different cool voices and characters all day long. It literally took me one line of code to implement it, right? Now, it doesn't have to need, it doesn't need keys or, you know, it has a key.

59:40It doesn't need to get your Google OAuth tokens or warm up your MXs. But literally, I mean, this is the dream, is if you can deploy the API for an AI in one line of code in 60 seconds, like 11 labs. If you can achieve that, and Resend is blowing up. If you look at the numbers, it is blown up because it is the easiest agentic tool to use in email. And this is a way to win today. We're not going to stay in Marketo, even though it's 90 years old, because our agents can't even talk to its A, even all the other issues. We might even forgive them can spam, all these terrible things, if our agents could talk to it, and if our agents could talk to it easily.

1:00:19And this is why, again, I think people underestimate Salesforce, because it has gotten this to work, right? It is agentic. It is agent-friendly. So I put 11 Labs open router at the top, Resend just under it, Bizabo in the middle. Like they haven't invested in anything. Here's an example, though. Here's something to think about. Bizabo is our events platform. It's pretty dated, guys. It hasn't been touched in years. If there was an agentic version that worked, we would switch, right? No question. But at least the API works with our agents. Here's the thing. And you got to slow it down. And here's my test.

1:00:51Go into Replit or Lovable or v0 and just ask a prompt. say, build me a dashboard that integrates with blank, that integrates with Visibo, that integrates with Marketo or Resend or anything. This is what I'm kind of, it's called the agentic API test. And whether Visibo has updated their API or whether they just lucked out because their 2020 API was fairly extensible and well built, it passes the agent test just barely, just barely. But it does past, like all of our data automatically flows to our AI VP of marketing 10K, it works. And so that means that the margin, we're going to stick with Bizabo.

1:01:30Whereas if it didn't work with our agent, even though there's no great solution, we would be desperately looking to leave like Marketo. So do the agent test, go into a, you don't have to be technical, go into a Vibe platform and say, build me a dashboard that integrates with this. You'll see how to integrate the API. You might have to figure out what an API key is and stuff. And if you haven't done it before, but see if it just frigging works after 15 minutes. That's the test. That's the test. And if you fail it or partially and try it on your own app, try your own. Have you touched your own API in a while?

1:02:01Try it on Repl.it or Lovable or v0. If you're not happy, go fix it because the majority of interactions are ready with agents, are ready with agents. And we're going to stick with mediocre products where the agents can talk to them at least. It is surprisingly difficult for a lot

1:02:17Amelia LeRutte:of all their apps i'll show you it's i was telling you maybe in january like okay after this annual i was dead set i was going to turn off visible yep i was like other than you know the outstanding question of what do i do for the on-site piece which you guys do not have to worry about you know just for us what like what what what happens if i turn off visible and people show up like how would i print their badges and stuff like yeah that's their moat that's their connect they have a real world moat connection. They have a real world element. Not just the software. Otherwise we could vibe it away without the real world park, right?

1:02:52Amelia LeRutte:For sure. But I was dead set. I told you this. I told, I think 13. That's it. Yeah. This is going to be like our last event. But then, okay, I'll tell you why. I'll just show you guys a quick pre-AI and now AI. This is our agenda from SAS Reanual 2025. Okay. This is not great. This is their native presentation. This is their native. Same since 2018. And this is still their native as of today. Yeah. Hasn't been touched. It hasn't been touched. But because we built our Saster annual AI website in Replet, I hooked up the API. And to Jason's, the API is pretty good. I was like, this is their saving grace that all of my registration and ticket data is in there.

1:03:39Amelia LeRutte:all of my, I uploaded all the session agenda, speaker, description data. Now, CooWork helped me a lot with that, but I uploaded it all to their platform. Their API is pretty good. So I told, this took me, I think, a weekend, a couple of months ago, where I told the reply, okay, let's get the API, let's first get it to work. So it was some work to, you know, get the UI to look like this. Because first it was like, okay, I'll just ingest this. And it didn't look like it looked terrible so it was some work to get it to look this good the second hardest part was getting it to let me go to like tuesday for example so the second hardest part was getting it to pull the speakers the logos but also i was like it has to work like if i click this to reserve my seat for a session it has to work like if i click this this has to legitimately save save me as a session registrant and I have to be able to see myself as a session registrant in the back end.

1:04:40Amelia LeRutte:Like the pretty stuff I'll take, but if it doesn't work at the end of the day, then I can't use it. Then I'm stuck using this black and white one because at least if I click this, then I know it works, right? So that was the whole predicament I found myself in. But with a few iterations between myself, the agent, and coworker helping me watch it, we got it to work as you can see it now. And I'm like, you know what? It's crazy because we're launching it. It looks pretty good now and it works. Not only does it look pretty good, we're launching our mobile app today. And historically, we've had to have two agendas where the mobile app had its own.

1:05:17Amelia LeRutte:Like we would have to give the same set of data to the mobile app. It would get out of sync because people would register either in the mobile app or in this case, this is on the desktop. So now because this was its own platform, I just told Replit in this case to duplicate this agenda, which it did. and then I embedded it into the app. So now there's just one agenda. It works full. It looks really good. Even the team, our vendor this morning was like, your agenda looks really good. I was like, thanks. I was like, you know, that was, they're like, is that biz? They literally said, is that still biz?

1:05:51Amelia LeRutte:Because they have a lot of things that are biz. That's what I said. I was like, sort of. It's kind of a, it's a custom thing, but I will say this API saved them. So like. Saved them. Saved them from churn, right? Saved them from churn. Saved them. You can do it, too. Build the world's best API in your category, the most agent-friendly one, and you might be surprised how much we can tolerate stuff that is crusty, rusty, and old. It's just, listen, they've released a lot of features I don't use, like weird streaming stuff. We have our own streaming team, like just shit I don't need, right? So I'm like, for years, they've been releasing shit, I call it, stuff I don't need.

1:06:29Amelia LeRutte:So I was sad on it, but now I'm like, you know what? The API does work pretty well. The on-site system I know works because we've used it for the last few astro-adual events. So, Cheshmer, I've come around a little bit, right? There are still things their team does, which I hate, but I'm a little softer about it now because this API with the power of Vibe coding and the agents on the AI is pretty good. Yeah, and I want to hit our next final topic that maybe the next agent you'll build, but folks that are deep in the agentic lifestyle know this, but folks who aren't may not realize just how much VibeCoding and its cousins make APIs accessible to so many more people.

1:07:18Amelia's really good, but she would never use any of these APIs before we were VibeCoding. They're inaccessible to a non-developer. Now your API is accessible to anyone that wants to invest 10 minutes on a Vibe platform. Your APIs used to be, unless your company was an API, unless you were Stripe or something, it was often a niche portion of your platform that a lot of folks didn't even touch or understand because only developers could use it. Now, anyone can use your API. Anyone can use your API. So if you have not, if you're not giving it the attention it deserves, you're going to lose in the Identic era because anyone can use your API and it's just going to accelerate.

1:07:58It's just going to accelerate with Co-Work. You're not even going to need to use a Vibed coding platform. You can just go tell Co-Work to figure it out. Yep. Right? And so just don't lose the AI wars because you're ignoring your API. It would just be a sadness. So the last topic I want to hit before we run out of time, Amelia, although you can add anything you want to add at the end, is we were in an agentic freeze. Vector slipped in. Kudos to them. They made it easy on us, and they added instant value. They made it easy on us. They added instant value, right? Want to automate your AI advertising?

1:08:31Use Vector, right? Want to get better deduping of the traffic to your website? Use Vector. There's the endorsement, right? Boom. Put it on the homepage. You have our rights. You don't have to ask us. The one you're thinking about building, which might sound crazy to some folks, is an AI VP of finance.

1:08:47Amelia LeRutte:Not for everything. Not for everything. But even with all the things we have to do, even with two part-time people helping us in finance, part-time, You're thinking of building one. And share your thinking and what the goals are here. Oh, listen, this is a frustration, I think, for every company out there, collections. Collections. I think that is my number one goal for the AI VP of Finance is— It's not to create the books yet or anything. We had eight figures of collections, non-credit card processed revenue a year. We have$8 million of non-credit card revenue. that goes through invoice and procurement and then accounts payable and receivable and all of this stuff.

1:09:31And it's a headache for everyone. No matter how you do it, it's always a frictionful process, right?

1:09:37Amelia LeRutte:It's always a headache, not only for us, but also the vendors. And I also realize this because we hit an influx point, this close disaster, where it was in a, we get a human email every week of, you know, our outstanding AP and AR. And I was like, why is accounts receivables the highest it's ever been? Like historically higher than high. And yes, it's because we have a lot more sponsors for Sastr. Great, cool. But disproportionate to some of them are really like starting to decay, get old. And I was like, like, that's not good. That's not good. Like this shouldn't be. It's the worst it's ever been.

1:10:18Our accounts payable has a, our accounts, our accounts receivable has aged the most ever. Right. Our accounts receivable is the most aged ever, even though we have great sponsors, more sponsors than ever and great ones this year. Right.

1:10:29Amelia LeRutte:Yeah. And I was like, that's a flag. What is the issue? Yeah. And so I started to do some digging and it's, there's a lot of reasons. Right. But one of the reasons is because right now our collections are human and manual. It's one of our pricing people on our finance team sending the invoice, someone they don't pay, sending a reminder. Then if they still don't pay, sending another reminder for like the remittance and then chasing, you know. Sounds like something for QB almost, huh? Well, that's what clicked for me, right? Because I was like, OK, if QB is sending is already sending these sponsors or booth graphics, whatever.

1:11:06Why can't a VP of finance version of QB or maybe it's just a trail for QB to go off of like he put on his finance hat.

1:11:17Amelia LeRutte:QB for finance. Why can't he do the same thing of, you know, because sometimes people ask questions. Oh, I didn't pay it yet. Can you resend me your banking details? I'm like, QB could do that. Sending these automated reminders when we still don't see, you know, the remittance. Because we have, again, without even hooking it up to our various finance platforms, we still get an email every time we get, you know, accounts receivable paid. So I'm like, even if I just hook him up to my email to look for those emails, he'll know if they're paid or not. Him not being in Stripe, QuickBooks, Bill.com, Brex, whatever, doesn't actually even matter.

1:11:53Amelia LeRutte:He doesn't even need to do that yet in this light version. I just need him to tackle collections because it's such a mundane process that your finance team doesn't want to tackle. Our finance team doesn't want to tackle. The vendor doesn't want like these random humanoid emails every now and then. Yeah. And then sometimes the humans send the, we sent like the wrong banking details to somebody at one point. So it went to like a different account. It's great when it goes to the wrong account. Which is great. That's great. and then somebody had asked like hey we just they're a bigger company they're like we need like a signed bank letter and it was a medley of issues but i was like this is just such a human process that because we already have a built-out qbr for you know customers he can just add this layer like he i started adding it now i don't know if i'll be able to finish a pre-annual.

1:12:50Amelia LeRutte:But we started to add this layer where he can just kind of now ask for the AP contacts when the deal closes and then start to send those. One, we also generate the invoices manually. I'm like, well, Keebi could just generate an invoice. It's really not that hard. So now I'm getting to the point where he'll be able to create an invoice and then just send it out to the AP team automatically and follow up with them. We don't need a stupid human process I mean, other than the invoices obviously need to go through somebody, a real human on this financing to put into the books and track it as real revenue, which maybe QB will get to eventually.

1:13:27Amelia LeRutte:But for now, I was like, I just need him to tackle collections. It's for so many companies. I think this is still their number one issue outside of deploying an agent is collections. Like I hear from so many people of they're just so behind on collections. It's such a manual thing. And I get why. Right. There's just so much human variation in the process where I was like, there actually doesn't need to be. Yeah, it's interesting, too. It'll probably right now for banking, we use SVB and BRAX. And like Wells Fargo? Yeah, you don't need to in the V1, you don't need to integrate banking. You just need it to do like finance for collections for customer success.

1:14:04You're going to go to the same people. You're going to auto generate an invoice. You're going to follow up with them. You're going to identify whether there's any issues. Right. And make sure there's no mistakes. That's just going to work. And some folks will say, that's not really a VP of finance. I would say it's a core role of the finance team for anyone that's fast growing, right? Maybe if you're Atlassian or Salesforce, you know, it's not a problem. Although we've had our own bumps with some leaders in there too. But it's a big part of the role. But, you know, if, for example, when we get to the next step, if the Ramp API is much more agent-friendly, we might just switch.

1:14:34Right. Ramp, you know, Ramp is very good at figuring out if you're a Brex customer and hitting you up constantly, okay? Yep. I always ask if they'll do the work and they'll never do the work. Right. So I'm not, why am I going to switch? Like I don't want to move over 11 ,000 bills. Right. But Brex hasn't done much for us in many years either. So we're switchable, but we're not going to do it just because a sales rep offers me a gift certificate or sees chocolates. It's not going to happen. But we might switch if the API is much better for the agent. We don't know yet. We'll report back. But honestly, if working with the Brex API is harder than the Ramp API, after all these great, great outreach from their outbound team, like they're really, they're like, they don't give up on getting the switchers, right?

1:15:16They're the best at anyone I've ever worked with of staying on it and not giving up. But it might be the, to tie it to the end, it might be the API that gets us to switch, not the humans, not the chocolates, not the gift certificate. but if the API is better, works better with our agents, we might spend a couple hours it takes to move over our key accounts to RAMP because it's the agent that matters. So unless you've got anything else, maybe we'll end on that one. Amelia, you got anything else that we missed for this week?

1:15:44Amelia LeRutte:Trust that Zastrianual is upon us now. We're three weeks away from loading in and I think we're about a month away from the event, which is a little bit wild. So everyone come. We will teach you how to build agents. how to manage agents, how to deploy agents, how not to get in trouble, how to get ahead in AI. We and the best of everyone from all the leaders from Salesforce to Replit to Vercel to Cloudflare, they'll all be there. So come join us May 12th through 14th. All right. Thanks, everybody. See you next week.