SaaStr 850: The Agents, Episode 1: Who Maintains All This?

22:53Amelia LeRutte:Yeah, I'll tell you guys a story. And it's all, listen, it's all resolved now because this was a couple of weeks ago. So I'll say, you know, all's well that ends well. But in the moment, I was so mad. Like I was mad and frustrated at a point where I literally told the Clay team, like, this is the first time I've wanted to rage quit an agent, which has never happened with all the 20 plus. I mean, I've tried more than 20. We have 20 now on deployment, right? I was like, this is the first time I've legitimately wanted to rage for it because the answer that the AI agent gave me was so unacceptable, was so bad.

23:26Amelia LeRutte:And when I reached out to the humans at the first time, I couldn't get the help I wanted. I was like, that's a double handicap. What happened? So walk back for a minute. What did the agent? So Claywood announced cheaper, better pricing. For me, they just made the pricing more complex and confusing. But they claim it was cheaper. But they claim it's cheaper for many use cases, right? What did the agent tell you? So in other words, their own, you went on Clay. It has basically a support agent, right? It has an agent. And what happened? Sculptor. It's, to their point, they said this to me afterwards and made this point, it's still in beta.

24:02Amelia LeRutte:But I was using the Sculptor because I'm used to using agents and, you know, the built-in agents and all my tools. I'll give you guys the full story. So for the last couple weeks, we have these VIP summits we have for Sastra Annual. I was using it to do lookalikes. I've talked about this before on Workshop Wednesday. I think Clay is a really good place if you're trying to scale up your AI SDRs or anything like that, where you can make really solid lookalikes with Clay. It works really well for this. And we were seeing success with it. And so I was using it for that. Doing the same thing I had always done.

24:30Amelia LeRutte:It was kind of late at night on a Sunday. I was building a new list of, you know, VIPs to go out to for the summit. And suddenly Clay wanted to charge me about 5X to run this list than it did the week previous. Yes. And so I asked the agent. Hold on just for a second. Clay wanted to or its agent told you you had to pay five times? Its agent told me it was going to be five times the amount of credit. Clay, the software application didn't necessarily charge you five times as much. Its AI agent told you you had to pay five times as much, right? Correct. Yes. It told me I was going to have to pay five times as much.

25:05Amelia LeRutte:And so when I pushed the sculptor agent, I said, how can that be? I just did this on a very similar size list, a very similar arrangement. basically almost the same parameters. And it was, I don't know, let's say something like 2 ,500 credits, but now it's trying to charge me 11 ,000, which is a huge spot, a huge spot. I was like, it has to be wrong, right? It just has to be wrong because that doesn't make any sense. And so I asked a few questions and first I caught that the agent related to their price increase changed how it, how it billed, but it also defaulted to the most expensive action versus a cheaper action that would get me to the same outcome.

25:44Maybe I misunderstood what happened. I thought the agent wasn't properly trained on the new pricing of its own app. Both. Okay.

25:53Amelia LeRutte:It's both. Yeah, yeah, it was both. So like, as I was talking to it, I was like, okay, one, I caught that it was trying to use this higher priced, you know, model to get to the same goal as last week. And so I said, hey, use this other action instead. It's cheaper. And so that lowered it by about half. Then it went down like 5 ,000 credits to run the same thing. I was like, okay. We're halfway there. And that part of the issue I thought was problematic because most people won't do that. Most people are going to look at the$11 ,000 that the agent tells you and say, that's too expensive. And they would give up right there.

26:25The first issue was the agent told you to use the most expensive model, the most expensive product. So the upsell, the agent told you you had to use the most expensive version of the product that you didn't have to use. Correct. I didn't have to use it. So I called him out on that. The agent told you you had to, right? He recommended, yeah.

26:42Amelia LeRutte:Yeah, yeah. He was like, well, if you want to achieve that outcome, this is how you should do it. Are you ready to deploy and run the waterfall in Richmond? I was like, no, not for 11 ,000 credits. There's a better way to do this. Do you think that was intentional or you think that was poorly training the agent? You think it was intentional to monetize the base better or do you think it was bad training? I don't know, because it honestly happened the same time they announced this price increase. So I was like, I don't know. I don't know if somebody as a human at Clay unintentionally, like that was the consequence of them putting in this release for new pricing that the agent took it and ran with it.

27:16Amelia LeRutte:Right. Like I can totally see that as it wasn't malicious. It's just something that happened with their agent because they loaded in all the training for the new pricing. I was talking to the agent and the agent was like, well, let's use this super expensive, luxurious model versus like the normal, you know, Prius model you were using. Yeah, because I have a I know. And listen, we do use clay and it's a great product. But when they announce all this new pricing, I have a rule, a classic Sastor rule that I think is almost always true. When a company introduces more complicated pricing, even if they say it's a better deal, it's always a hidden price increase.

27:50I've never seen it be like, there are occasions where companies literally say, hey, listen, we've just decided to go cheaper. And it's usually when there's something brutally competitive. But usually when pricing gets more complicated, it's a way to disguise a price increase. Sometimes not everybody pays the higher price. But by adding complexity to pricing, you can essentially make segments of your users pay more. So it feels like that's what the agent was trying to get you to do. you try to get you to pay two to five times more than you had to at least yeah and it seemed like

28:23Amelia LeRutte:it right that was the and it led to the yeah it led to the second issue actually so once i caught it in this like trying to use a more expensive model trying to burn through all my at the time i was like i don't know i you know i have like three thousand credits a month you're gonna burn through my five months five months of credits yeah basically i was like no freaking way so then i got it down to about half once i caught it in the agent but then i could really only get it down and a half and I was like, you know what? I'm so tired. Heck it. Like, I'm just going to, even though this is more expensive than it was a week ago, I'm just going to push it through because I need to get this done.

28:54Amelia LeRutte:I was like, I just can't keep, you know, I would already been arguing with the agent for like half an hour at like 11 o 'clock on a Sunday. I was like, I just got to get it done. And so I asked the agent to, you know, push the enrichment live with the two X credits because I just had to get it done. And it said, okay, you're not going to have enough credits to finish. So you're going to have to upgrade because this is to, again, this is around the time they announced the price increase. So at that point, I was like, you've got to be kidding me. So I just hit the upgrade button. Again, I'm tired.

29:22Amelia LeRutte:It shouldn't be on the customer to figure out, tired or not, it shouldn't be on me to figure out that the new plan was going to be worse for this outcome. And I should have just stayed on my legacy plan at the time and just bought more credits, which in hindsight, yes, of course I would have done. But again, when the agent was suggesting to me, hey, if you just upgrade your plan, you'll have enough credits. I'm just going to click that button. That's what I did. I clicked the button from the agent. I clicked the one of the new plans or whatever. And then I finished the arrangement and I went about my day and I loaded up with my AISDR.

29:52Amelia LeRutte:So it was a double handicap. I get that. And then we can move on to the next topic. But I thought the second issue when you reached out to the Clay team was that, and I think this is something we all need to learn about agents, was they had actually not properly trained their beta agent on how their new pricing worked. Like they had not uploaded the documentation, the pricing. And we've seen this ourselves, if you don't keep, you know, digital Jason, the digital version of me, it broke for four months, the old version. It didn't upload anything for four months. I didn't know it silently failed.

30:22So if you don't upload, if you don't constantly train your agents, they will give your customers wrong answers. So I think the second part was Clay changed their pricing. They didn't train their agent with the new pricing. So that was one of the reasons they gave you bad answers, right?

30:35Amelia LeRutte:Yeah. Cause when I, when I flagged this to the Clay team, I was like, Hey, just so you know, like, obviously we use a lot of agents, obviously for families, it's a lot Varun was in London. Like, we're fans, but here's why I almost rage quit, right? And so I sent it to both, like, all the folks I know there and was like, you know, just this double handicap of it trying to charge me more, use a more pricier model, then you raising prices, then the agent telling me I had to upgrade to get something done, which wasn't necessarily even true, right? It just told me, it just made that recommendation.

31:03Amelia LeRutte:It's also telling me to use credits I didn't have to use. I was like, you may want to look into this because I don't know if other people would be as patient. I don't know if they'd say, hey, this is mission critical. I got to get it done. I'm going to use the credits and pay the extra cost. Like, I feel as a customer, like, I stuck with it. But I think other folks would be fairly frustrated. And so it's just a reminder for the folks listening, right? Like, I don't want you guys to ever, and we're sometimes in the situation, too, to Jason's point. Like, we're sometimes on the reverse side of this.

31:30Amelia LeRutte:We're with our agents. It is a lot of maintenance. It is a lot of training to keep it up to date. But because they hadn't trained their sculptor fully through every scenario, which I know is tedious, right? It's totally tedious. Like that's why. But that's the job. But that's why it was trying to get me down this upsell upgrade path when it didn't need to. Yeah, it was another example. Then let's move on to the next topic. I was using the AI agent on HubSpot's website on the homepage, right? Massive traffic, right? Yeah, massive. Trying to get a pricing quote for what HubSpot would cost at our scale.

32:06I just could not get an intelligent answer. Oh, yeah, I remember this. I don't know whether it was. Now, it might have been because they really want you to talk to a human. I think that was part of it. But it could have been better trained. And so all these stories are that some of the best folks, Clay here, to a smaller extent, HubSpot's others. Like, AI support is great. AI agents are great. But someone, you've got to read a segment of these chats every day. Like forever. You have to read these interactions with the agent because they're going to drift. You're going to forget to train them. They're not going to ingest documents properly.

32:39Or you're going to think they say things that they don't. Or your testing worked fine. And in reality, it didn't. So these are endless stories of agents not being as well-trained as they could be that are customer-facing.

32:52Amelia LeRutte:On the flip side, though, we had somebody this week reach out to our events team frustrated because they thought the answer they got from an AI was not good enough. And it's funny because I was like, sir, actually the answer you got was from a human being. You thought from not from... You made a mistake or it was from a human, right? It was too... They made a mistake when they bought their ticket on SASTR and then one of our humans on our side, minor mistake, super minor mistake. When they bought their ticket, they accidentally bought two. And so that is a human thing we have to resolve on our end.

33:31Amelia LeRutte:Like, I don't let the agents right now, any of our agents, touch our back end of ticketing and Stripe, right? Like, that's something we have to do, and it should be something we do. So this was a flow where he said it to Amelia AI. Amelia AI escalated to a human, as she should have, down this path. One of our humans, our events team, initiated a refund, as they should have, the way they should have. Because the human on the attendee side said, hey, you refunded me, but not with the ticket processing fees, like fully. It's this weird thing in Stripe. There's like a 3 % ticket processing fee. They didn't get the 3 % back.

34:02Amelia LeRutte:They didn't get the 3 % back because that's our standard is to usually when we do a refund. Now, this was a case where it was our fault because the website glitched and he got two tickets up the same. We should have just refunded that 3 % in the first case. The team didn't want to bother me and be like, hey, can I have an exception for 3 %? And so they just refunded him under standard. And this guy complained after the fact and said, thanks for my full refund now. But if this is how your AI support is, I'm really worried about the content. It's answered. And I was like, sir, I understand. I was like, yes, sir, I understand the frustration.

34:33Amelia LeRutte:But it was us. It was a person. It was multiple. It was people. It would be interesting if Digital Amelia could have identified the issue of the strike fees that a human might not have thought through. The agent might have done better here probably, right? For sure. Yeah, and that's the irony. It was a mediocre artist. Yes, so it is. Okay, the next topic I want to talk about, sometimes I'm slow, guys. Sometimes it takes me a little while to figure things out. I want to talk about, I want to share a story from this week, but a lot of folks read what we write, especially on LinkedIn, where people really like to be Yahoo's, and they're like, oh, Saster's results with its SDRs and GTM agents, those don't work at my company.

35:19Those are unique to Sastra. Sastra is such a strong brand or Sastra does events and tickets. It's not the same of me or Sastra is exaggerating or whatever. We're just, we're just trying to help. But I see this, I see this all the time. And what I basically tell people is like, we're just doing the same thing you should be doing. And you might get better results than us. You might get worse, but try it, right? It's working for us. But I didn't completely get why it works for us until I had a meeting this week, even after all these months, even after Agent Force, Monaco, Artisan, Qualified, Momentum, why does it work?

35:50But I was meeting this week with the CO and CRO of a great public company with thousands of sellers, thousands of sellers. And we were talking about agents and AI and, you know, some folks do get a one-on-one version of this show. And the first question the CO and the CRO had is that, what do you think we should do first? And they had some agentic products, but they said, what do you think you should do first in order? and I went through a series of what I thought they should do first. First, do something so that everyone can instantly get great quality questions answered on your website, going to what we've just talked about, right?

36:22Make sure they can instantly get an appointment set with sales, right? And I kept going through different things. And then I realized, I went through all the things. I said, you know what? You know why all of our agents work in go-to-market and sales? It's because there's no lead left behind. This is the flip side. We talk about how agents give you more coverage. We talk about how agents don't mind the B leads, but it's simpler than that. If all, if you deploy some, just some of what we do in AI agents, and then Amelia can share some details better than I can. Here's what you know. Anyone that has an issue, like with that 3 % stripe fee, gets to talk to an agent and then a human.

36:58Anyone that wants to know if they can get a discount, whether they get one or not, can find out in real time. Any, any sponsor that clearly doesn't have enough budget to be at Sastor can still find out if they can sponsor, but a human might be like, They raised$2 million a demo day. Like we don't want them, but the agent, the agent doesn't judge. Bootstrap, the agent doesn't judge. Any issue. And then, you know, and then for our use of agent force or others, you know, we were inspired by Mark Benioff to go back to our prior leads and prior ones that no one had touched. Right. And so clearly we have evidence that in many cases our agents do better than a mediocre human.

37:36I know it's a triggering way to say it, but it is true. But even if it's not, it may be simpler than that. It may be just if you touch every single lead, every prospect, every customer, the way they want to be interacted with in real time, you have to do better. That's all these agents have to do to add value to the organization is that there's no lead, no prospect, no customer left behind, whether it's in pre-sales, sales, customer success. We can talk more about QB or AI, but no one gets left behind. I think it's that simple. What do you think, Amelia? I know I'm slow, but this is my aha moment.

38:10And then with this public company, we just talked about all the holes in their funnel. And as well as they're doing, there's so many ways to not get touched.

38:18Amelia LeRutte:I agree wholeheartedly because I've said now, like, there are times even with the volume we're doing between all of our agents, our agents still sit idle, right? We do a lot of outbound. You know, all the normal, quote unquote, normal go-to-market motions. that again, I think sometimes people look at it and say, oh, that doesn't translate from staffer to me. I'm like, well, revenue is revenue and customer are customers. Like that's still the same across the board. And we still have a lot of the same go-to-market motions you guys are using. They're just sometimes employed a little bit differently or sometimes they have deadlines a little bit differently because we have events that are tied to it.

38:53Amelia LeRutte:But the behind-the-scenes motion is still very much the same. And even at our scale, I have not touched every single lead. I'll give you a good example just this week. We have been in a production sprint because there were a lot of deadlines leading up to today. And there were a few on Friday and there were a few on Monday, basically for SaaS renewal that's coming up in May. And so we had these big production deadlines. And so I was sprinting between myself, our designers, our production team, our customers, our sponsors to get some of their graphics. And we were sprinting to meet some of these deadlines that we had to meet because things take time to print.

39:26Amelia LeRutte:Sometimes you can't pay rush fees to get these things done. But what it led to was a major idolization across our agents because I just didn't have time for them. This is one of those extreme examples, too, where like no lead left behind. Yes. But also, if you have somebody who gets a little bit busy in a way a human can that agents, you know, are working 24-7 don't. You run into this issue. I've run into this issue over and over again. And that's why I'll keep talking about it. It's my agents will start to sit idle. And this last week, it's the most they've been idle since I've gotten them, you know, all up into production.

40:02Amelia LeRutte:Leads behind. We're leaving some leads behind. Even with our agents. 100%. But our agents are ready. They're ready. Yeah. But I was literally looking at it today. I was like, OK, I have a tiny bit more time today. And I've got to go get my agents out of idle mode. I've got to get them back into go mode because I have leads left behind for sure. 100%. Yeah, it's funny. It's a different topic and maybe one that is worth continuing in subsequent conversations. all our agents end up idle, right? It's an interesting topic, which is, you know, every human thinks they're so busy, even if they're really working 20 hours a week from Miami, but they think they're so busy.

40:35They don't have time. They don't want to follow up with the B leads or the C leads. Agents are so efficient. And we have so many there. All of them are idle 90 % of the time. They're sitting there waiting to do more. They're happy to do more work. They're sitting there idle. And it's, I haven't fully thought through where that will lead over the next 12 months. but you know in a sense we have an order of magnitude more capacity than we had pre-agents and then we had with humans right um how to fully exploit that is uh the best way to exploit it now is just make sure everyone that hits your website everyone that contacts me everyone in your database is followed up with in real time but it's amazing how much idle capacity we really have right

41:14Amelia LeRutte:yeah and i think that's a the website one is a new workflow we added which we could talk about more later, but we added a newer workflow where when folks hit our website, in particular for Sastra Annual, now they get like an automated follow-up. But it requires some human steps to get that data from the website to the agent. One, because some of our, again, because of how we're built, we have to have this step. You wouldn't have to have this step necessarily, but because we get so much traffic to Sastra.com and we get, not to, that wasn't a braggy thing. It's just, we get a lot of high volume traffic to Sastra.com.

41:46Amelia LeRutte:We get a lot of traffic to Sastra Annual and we have to filter it out. I can't, to Jason's point, could our agents hit up every single person that hit the website if we de-anonymize them? Yes. But do I want to? No. This is the human orchestration layer of yes, no lead left behind. If I think they're a lead, right? Like there's still some qualification that needs to happen there. And I don't have an agentic workflow yet, but it's one of those things right now where we have to do some cleanup of that data because it's just so much data and kind of push people into different buckets. And so could we have no lead left behind on the website?

42:20Amelia LeRutte:We could. But I think that there's also an air of we have to filter out some of our stuff because not everybody is a qualified lead, right? Yeah, I think it's interesting when you think about what agents can do over the next 12 months in marketing. I feel like marketing is pretty far behind sales, right? Sales is behind coding and support. But marketing, man, the reason we had to build our own AVP and marketing is we couldn't find anything that's any good, right? And I know folks are trying and kudos. And if you literally can do better than we're doing, email us. But it has to be better. We'll use your product if it's better than we're doing.

42:49But we don't need something that just is a GEO or SEO scraper or writes content. Like, we have a lot of content, guys. Like, a lot of folks reach out to us as, hey, we can write social media content with you with our AI market. Like, we don't. We got 99 problems. That's not one of them. But it's behind. But, you know, so we get maybe half a million uniques to our SaaS or web properties. I mean, people read content on LinkedIn. They read it on Twitter. They watch YouTube. but we get about half a million uniques to our websites. And those aren't leads, right? But traditionally, if we could use agents so that everyone truly has a customized experience to them, we can convert more of them to leads, right?

43:26Like we need an agentic lead magnet where everyone, just like now we have agents like Digital Amelia that you can talk to if you want on our websites, but we should go further where, and you don't even know you have a lead magnet because everything is like, who are you? Okay, Jason would love to have an ebook on how to do her agents. Amelia's pretty advanced. She's a chief AI officer. Maybe what she would like is to invite to our CMO summit or our AI summit or something. But if we had this hyper ability, not just to use some of the tools we use, which are really exciting, like all of our tools, whether you think it's ideal or not, they can de-anonymize your web traffic.

44:04They can identify better than ever these agents who's coming on your properties, But that's not as actionable in many ways, right, as you might, unless you're a spam farm, right? You don't want to just collect people's random email address and send them weird LinkedIn notes. I mean, people do that stuff, but, you know, I block them.

44:17Amelia LeRutte:People do that. But I think I am excited. But you could do something in the middle, which is you don't spam them, but you provide a truly contextualized value add during their experience, or you push them at least into the very top of your funnel, these 500 ,000 people where they otherwise might. That would be that, that once you, once, once that, that's a, maybe our current agents can't do that, but it's a meta idea of, you know, no, you know, no champion left behind, no visitor left behind, no anything that that's the next, you got me thinking. Okay. Next one seems small, but I think it's just interesting to note how are the big guys doing?

44:56And, you know, we've been relatively close to Salesforce because we were relatively early on Asian force on deploying it. We've done a lot of stuff together with Salesforce. It's been fun. And then I think maybe we even played a small role when Salesforce bought one of its partners Qualified to accelerate its agentic efforts. We thought it was a great idea because we used Qualified and we thought it has scale. So the deal closed. What did you see actually happen at the product level this week, Amelia?

45:26Amelia LeRutte:Yeah, I thought it was interesting. So if you go to Salesforce.com, just the homepage or any page. Without logging in, you'll now see a qualified agent versus it used to be a agent force, like support agent. Now they've swapped it over to qualified, which I think for a lot of reasons was the right call, right? I think it's part of why they bought it, right? Like in many ways, some of the features and functionality that qualified had just went a lot deeper than what agent force kind of natively had. Like, you could get there, but it would just take a whole lot of, like, extra steps. And so, again, I think they were so smart to buy it because it has all this and because you had to be a Salesforce customer to be on Qualified anyway.

46:10Amelia LeRutte:I think it was. And a lot of the team came from Salesforce, right? But I think it's interesting because when that module was on Qualified.com, it was a real person, Blake, that they modeled, you know, Piper after, who's their mascot. And now it's like a 3D avatar version. Yeah. Which, again, I think it's interesting. We've talked about it a little bit of like, do you have, you know, the rights to that person? If you use a person who's not the CEO or founder, there's a reason why, you know, Jason AI and Amelia AI are us. Like, we're not going anywhere. Like, we have all the rights to it. Like, we can still use our agents and our likeness.

46:44Amelia LeRutte:But I think it's interesting that probably for a CYA, if I had to guess, that Salesforce changed it from the actual human avatar that it was to a modeled, like, cartoon version of the same person. Yeah. Probably for a lot of different reasons there. And also just for speed, I think for Essence of Speed, like they already had the other video model. But I think to also signify that it's now Salesforce and, you know, they have a lot more customers and a lot more security layers that they have to go to. They changed it to a cartoon version of the same person. Again, I think it was smart to get Qualified on the website.

47:17Amelia LeRutte:The day the deal was pretty much done, they flipped the website. There's so many deep features in Qualified that now the Salesforce team can action on. The interesting takeaway, just we're trying, you know, Salesforce built AgentForce and they've got thousands of people working on it and we use it. And the most important thing for folks is it does work. It does work. But because it's so extensible and so broad, I mean, Salesforce has like nine different clouds and so many uses. It is work to configure it. And they bought Qualified, which is just a GTM tool. And classically was just to qualify.

47:52That's the name of the company, Inbound, right? So it's a very narrow set of, now it does some outbound and other things, but it's just a GTM agentic tool. So for their, especially for their CRM customers, which is still almost 20 % of the revenue, now they bought a tool they can roll out to the entire customer race. Does it do as much as AgentForce could do? No, but it's pretty cool that the day the deal closed, it's in production, it's on Salesforce's website. And now there is an option to buy a Salesforce native agentic product that is much simpler to deploy. And if your goal is GTM, right, you could have it up and running probably in a couple of weeks rather than maybe a longer period it would take to map out and figure out a full AgentForce implementation.

48:34Amelia LeRutte:Oh, for sure. Like I'll say like for myself, I had a much deeper learning curve with AgentForce than Qualified. Qualified UI feels more like that, right? So I think part of why they did the deal, too, is, OK, if you want to unlock this whole pool of people where Agent Force historically has been, you know, success is the number one use case of Agent Force. But now you want to go more go to market. I think that's why they both qualified and momentum together, right? Because now you've got RevOps for salespeople and you've got this whole GTM kind of suite that is easier for folks to use, better to use out of the box, I think, for GTM folks.

49:09Amelia LeRutte:Like I talked about last week, I had to build this like Salesforce custom object for one of our agents. Pre-AgentForce, we weren't using our Salesforce that much. But also I'm not like Salesforce trailblazer qualified. Like I was like, I'm not that good. Like I will confess to anyone listening, like I wasn't that good at Salesforce. I've gotten a lot better in the last nine months because of how we use AgentForce. but a lot of the times I am I like look for like um I look for either loopholes or other things I can integrate into our salesforce to make it easier because there's just so much data in there and there's so much you could do with it that's that's why now I use a lot of slack bot too because it can do some of it a little bit easier but again I think that's why it was smart of them to get qualified up and running instantly I think it's smart of them to expand in this kind of like go to market area where it's it's easier for folks maybe to understand you know qualified from Salesforce, that's just your go-to-market agent.

50:04Amelia LeRutte:Like, where it blends with Agent Force, who cares, right? I don't care. I told the team to just let people pick what they want. I was like, if you can do it in a way where people can pick either the qualified UI or the Agent Force traditional UI that it has, like, just let them hit a toggle and say, I'm going to use the call. Yeah, like, let the customer, you know, again, go to your customer, let them pick what's easiest for them of how they want to set up the agent so that they actually get to deployed agent in production. Yep. So, anyhow, I want to move on to our last and final topic that we will come back to every week.

50:36But I think the meta thing is, I think, listen, Qualified has, there's other folks like Qualified. There's a number of vendors that can do a great job with inbound and outbound, but we've used it, right? And so now there is an option that's natively in Salesforce, it's on their homepage, where you can deploy this much more simply than Asian for us. If you're deep on Salesforce, like, go for it. Get this thing up. It goes back, like, Whether it's the perfect vendor or not, it's native, it's built in. And in our theme of no prospect, no customer left behind, this is one thing you can do now, get it deployed.

51:08And this is, you know, just like a lot of CMOs have padded themselves on the back the last 12 to 18 months rolling out clay, right? And kudos to them. If you're in RevOps or SalesOps or Sales and you want to build it, roll out an agentic product in Salesforce, just do qualify. It's still qualified. Do they change it?

51:26Amelia LeRutte:Qualify from Salesforce. But this is, I think there are, and then maybe the competitors are better. I'm not picking sides at the moment, but this is a quick win. This is something to think about. Okay. The last thing I want to talk about is our team members. And maybe we'll do this each week at the end, QB and 10K. What's up with them? As a quick reminder, QB is kind of a play on QBR. It's our AI VP of Customer Success that Amelia built. 10K is our VP of Marketing. Her initial job was to get us to 10 ,000 attendees in the first 10 million of revenue this year. So we call them 10K. now they've all joined slack they're up early on slack we now have more daily check-ins on slack with agents than humans which is the future here today if you go on our slack like our slack just like our salesforce almost died before agents our slack almost died before agents then we added slack integration for agents now they're talking on slack almost too much but they're better check-ins than any human ever did right any check-ins okay so a couple things on qb and 10k first one because As you brought it up, you built and maybe just describe the challenges and the opportunities for our for.

52:34You added Salesforce integration to 10K for sure, maybe maybe two, right? Yeah. And it was interesting. So we did it in Replit and Replit has a native Salesforce integration. Right. But that alone didn't quite do what we needed to do. So you had to build a custom object in Salesforce, which if folks have been in the Salesforce ecosystem, you'll understand what custom objects do. But this was a learning curve and work for you to figure out how actually to get the Salesforce integration to work. I don't want to hear about it, but to give folks context for me, I don't log into Salesforce. But when I log into 10K, I see all of our pipeline and everything over history and time presented a way a normal human would want to see it.

53:18So Salesforce has now abstracted away from me. I see all of our Salesforce pipeline opportunities, everything done beautifully and visually, and I can drill into it right inside of our AIVP marketing, which for me is great. Okay. But you had to build it. So it was different than we thought, right, with the current integrations.

53:36Amelia LeRutte:Yeah. It's just, listen, the learning with any of the, I think, any of the Vibe coding platforms with Salesforce in particular. I get now why it happened, but it was frustrating for a few days because I just used the native. Because, again, why wouldn't I? Like, if you're Vibe coding, I'm going to go cap. One is you click on a Salesforce connector, you put in your credentials, you just log in once, right? And it gets a token. Problem is the token expired every 24 hours, right? Every 24 hours, which I didn't know. And the agent didn't really know it either. It took it a couple days of it failing.

54:06Probably because it's not a core feature like the Clay thing. It probably wasn't well trained on how the integration works because it's a corner. It's not a high volume integration.

54:13Amelia LeRutte:I was like, okay, there's got to be a way where the token doesn't expire every 24 hours. And it was just building a custom object app. Because in a custom object app, which I asked Claude how to do, super easy. Like Claude gave me really good instructions. And then I loaded up Co-Work so it could watch me do it. Right. I was like, just watch what I'm doing. Tell me if I go wrong. Co-Work was watching you manually build the Salesforce custom object. Yeah. It was watching you stream? In the browser. Yeah, yeah. It was watching you stream. That's pretty cool. Yeah, man. Did it catch errors while you were doing it?

54:41Amelia LeRutte:I didn't too. Well, it was pretty easy. Well, he gave me really good instructions, so nothing bad happens. So Claude gave you the instructions to build a custom order, then Claude Cowork watched you while you did it. Again, you're ahead of me. She was like, you're doing great. I wouldn't have thought to have Cowork watch my work while I'm doing it, but that's pretty cool. Yeah, you know what's funny? I haven't had Cowork watch Replet or anything. I should do that. I'll do that next time or next episode. So, yeah, I deployed this custom app, Objective Agent, and Claude and Cowork were walking me through, okay, here's how you can set it so that you don't have to, because I did get, I got stuck a couple times.

55:18Amelia LeRutte:I was like, you know, Claude, what you told me to do is not exactly where you told me, like it's using just kind of like, you know, older UI data. So I'd be like, is this the right button? Is this the right box to check? Is this the right field I want? And it was like, yes, yes, yes, that's fine. That's the one you want. And I was like, shouldn't I do this one? It was like, yes, you're right. Do that one. This is why I'm watching you. And so I did it. And then I found in building the custom object a way that we could have our token refresh every year versus every day. And now our Salesforce integration has not broken since I did the custom object.

55:49Cool. Well, it shows. So the good news is it worked and it's epic. It's cool, right?

55:53Amelia LeRutte:It's epic. Yeah. But it was harder than we thought. Like not every, there's some integrations, like my two favorites, Vibe Coding, are 11 Labs and Open Router. They're so easy. Like you can integrate these products in like 30 seconds. Instantly, yeah. And it's a credit to both vendors, to 11 Labs, but it's also a credit to the fact how they work. Like it doesn't have to be as complicated as Salesforce or Clark just by nature, but realize that not all of these are one click simple, even if they look like it, right? They are complicated. And this was probably, was it a couple hours of work or how long did it take you to really get the Salesforce integration working in 10K?

56:29Amelia LeRutte:Half an hour. Once we figured out the problem. I would have thrown my mic at the monitor because I don't want to build custom objects. But again, for you, it's half an hour. For someone that has never really used that side of Salesforce, it could be a bigger project though, right? Yeah, or you might not have access, right? Like I'm our Salesforce admin. And so I have all the right protocols and permissions to do that. Now, if you're building this, you're not one of your Salesforce admins or you have certain like company restrictions for, you know, because it needed API keys and auth credentials and it needed a lot of stuff to get those custom map options done.

57:04Amelia LeRutte:It may take you longer just because you're going to hit blockades, right? You may not have all the right permissions or somebody else may have to do it for you. And so just by nature, that may take you a bit longer. Yeah. Okay, last one that we had with QB and 10K. When you were vibe coding in the car and we were taking a Waymo to an event this week and you were saying how we actually have a bunch of Chinese sponsors this year, which is great for Saster AI annual. and they were complaining that QB, our AI customer success, which is great, but it was a little hard to interact with the app in English.

57:39So we asked Repli to localize it into Chinese. And so you've never done this before, right? So we did it in the car. I figured this was one that could be done on the phone. And it was partially true. What did you learn from the localization product to localize it into Chinese and Spanish? We had Repli do both of them, right?

57:54Amelia LeRutte:Yeah, I had them do both. You know, I'm going to pick all the, like, tablet. Obviously, English is native because I built it in English. and I was like okay I'm gonna I'm gonna pick because this is becoming a frustration point because we have last minute sponsors we have multiples of them are you know native Spanish speakers some of them are native Chinese Mandarin you know speakers and I was like okay I want to make this as seamless as possible for them to get up to speed I can't do that but QB could right so we were vibe coding the way mo and I said to Replit hey can you make a toggle so that you can translate everything I've put into QB so far into Spanish, Chinese.

58:32Amelia LeRutte:And then with the toggle, they can switch from English to the non-native languages. I don't know. And so it took it maybe 20 minutes because there's a lot of content in QB. And so it had to translate everything. I think it used OpenAI for all the translations. And then after we got out of the car, I took some screenshots and checked it with Claude to make sure nothing was inappropriate or weird with its translations. What did Claude say with the translations? It was pretty good. Pretty good. Yeah, yeah. So you took screenshots of what Replit built with OpenAI, and then you took screenshots to QA it into Claude to see if it thought that the translated app was good enough.

59:09Amelia LeRutte:Yeah. Actually, what I'm running now, too, in the background is I'm having co-workers now run through all of it. Just spot check, make sure nothing is weird, but now co-workers doing the full thing. So that's also an interesting learning. But, yeah, I think it took about 20 minutes in the car. I'm about to roll it out to those sponsors who need this. But my guess is that it will ease some friction because, you know, again, I'm using our native language. They're having to translate. They're also, you know, all of these folks are last minute sponsors. And so they're trying to get caught up really fast.

59:42Amelia LeRutte:And so this is something, again, we wouldn't have been able to accomplish with the tool we were using previously. Like there's no – like we would have had to get a person to translate it manually and then program that all into our – Well, it's worse because you have to build a harness or a system so that each chunk of text is dynamically translated to different languages and can be updated when it's changed. It's not even – it's not a – because if you really understand how localization – I mean I'm not an expert. I've done it. Like, you know, you do need a framework or harness because everything is constantly changing and being updated and it's dynamic.

1:00:13Amelia LeRutte:You're adding things. Yeah. Yeah. So it's not a one-and-done kind of deal. And so the agent Replit, I mean, it seemed to have done it all in the Waymo for the most part. Now, at first it was, as Asians are, at first it was a little lazy and only translated some lines and core text. Yeah, it only did the menus. Right? And so you had to keep saying, Moga, like it's the skulls. Keep going. It was hoping it was enough to just translate the core stuff and not the deeper text. You had to push it actually multiple times to go deeper, right? I did. But you know what? But when I think back about that now, I think it's OK because I see so many people flag the costs on Replit where, like, you know, maybe if you were cost conscious, you would only want to translate the menus and something like the task list.

1:00:57To save$20.

1:00:57Amelia LeRutte:Yeah, but still, right? So I was like, OK, to its point, maybe that's why I didn't do the function get ones. This is a complex project, right, relatively speaking. So, you know, if 25 percent of the cost gets you there, for most people, that might be a good place to stop, right? Yeah. Yeah. But the agent should have asked me, OK, I did these menus. Are you OK with me keeping, you know, to keep going? It would have been nice if you asked me. And we can wrap it. I was talking with another public company CEO this week about agents. And he asked me, have you guys used any of your agents for localization?

1:01:30And I'm like, I get the problem. You know, I lived it in my day. But no, it hasn't really come up. And then it came up in the car. I'm like, oh, we got to do it. Like, we got it. We got it. I think this is a pretty awesome use case for the agent. But let's see if it breaks. Let's see if it can literally do something that seems simple, but it's not to fully. I mean, Shopify, I think just last year, rolled out localization for a lot of its product. OK, Shopify is pretty. Shopify is what, 13 billion in revenue and they have global commerce and they just did this. And we did it in 20 minutes in the car, in a Waymo, on Replit.

1:02:00Man, that's an AI story. Maybe we should end with that one. Did we get everything to wrap on?

1:02:07Amelia LeRutte:I think it's a good one. I think my only other thing that happened this week with QB that I think is interesting, but I don't want to derail us. No, add it and then we'll wrap. Okay. Is there was this interesting moment, right? So I talked a little bit earlier. We have this big production deadline, not only for ourselves, but also for some of our customers who are sponsoring this after annual. They had a print deadline this week. A few people tried to, in a very human way, trick the agent. But the way where from the agents, I just think it's always an interesting reminder. You can't hide from the agents, right?

1:02:42Amelia LeRutte:And so a few people tried to, whether it was malicious or not, it doesn't matter, right? A few folks sent us their graphics and it was, I got a couple. Placeholders. They were placeholders, right? Placeholders. Either I got placeholders or I got renders or I got incomplete graphics. Deadline. So they uploaded whatever the hell they had on their desk. Yeah. Yeah. They were like, I'm going to send. They pretended they met the deadline, right? And they were like, okay, great. Like I sent this to you. We're all good, right? I was like, well, actually between me and QB, in a way that I couldn't previously because I didn't have enough scale or output.

1:03:17Amelia LeRutte:But now that I have the agent helping me, we did it in a way where not only did we and they got an automatic email. We got an automatic email and it was sent to all contacts because sometimes I would get like a random designer sending us, you know, the booth graphics. and it would say, here's what you uploaded. And I would, between me and Claude, we would check it pretty much instantly. And we would catch these things, right? We'd be like, okay, within the age of AI, you really can't hide. Like we can see it. You uploaded a placeholder or you upload it or a lot of times they would upload something that was incomplete.

1:03:53Amelia LeRutte:Things you might've been able to hide back in the day, but that you can't hide now in the age of AI because it just checks everything. I was like, it's between me and it. It's together. We're checking everything. Like, maybe I wouldn't have got this some of this a year ago, but we're checking everything now because I've got my agents running. And so I can see if you're if your graphics are incomplete or fake or real or not. And so it was just an interesting thing that people didn't fake the deadline, didn't fake the asset. And it can follow up in a kinder fashion than a human might be so that because they would they would pretend they did it right.

1:04:28And QB would objectively say, no, I appreciate it. But that's not actually what the graphic or assets that you're supposed to.

1:04:35Amelia LeRutte:Right. In a way that a human that we do get yelled at a lot by by folks. Right. For I don't want to go too deep. We can wrap it up. But but but a lot of folks, when they miss deadlines or when they don't have the assets or things they need, they they sort of try to blame us. And we're Zen about it now. But it's a human reaction. Right. Oh, that's not the deadline that David told me. Well, it's in the contract. It's in the, here's the 28 emails. David said I could, I could just sketch it on my iPad and that you, or David, maybe you said this and people would just make this crap up. But it doesn't work with, but QB's calm.

1:05:11Like QB doesn't get, get, get, get, get, get triggered. Right. And QB just says, thank you. I sent you 28 emails saying this is what we needed. It's late. Understood. Could you please do it by tomorrow?

1:05:22Amelia LeRutte:Yeah. And that's, and that's the thing. Like in a way where I was like, I don't know if I would feel comfortable pushing back to some of our customers like they are paying us money. But in a way that's super neutral, just not caring. QB and I would check the graphics. Yeah, better than a human. And if it was and if there was an issue, it would just tell that person. It would just tell everybody in the org, hey, because maybe, you know, maybe the CMO didn't know that their designer just uploaded a placeholder. That happens sometimes. Like, you know, it's fine. It happens. And so in a way that it was super neutral, it would say, hey, there's an issue with your graphics.

1:05:58Amelia LeRutte:Here's the issue. I'm going to email everybody that I have from in the org as a customer, you know, in this customer org. I'm going to let them all know. And I'm going to also say in that reminder, either some of them were like already past due. So it would say you're past due. You need to upload this now to avoid a late fee or not. And for some of them, when it was approaching the deadline, it would just tell them, hey, we need your fully revised graphics. This doesn't count as a submission. Again, in a very neutral AI way, thanks for trying. This wasn't it. This doesn't count. You might have been able to trick a human previously, but you can't hide from QB.

1:06:33Yeah, maybe we'll wrap on that. When I was talking with another public company CEO about AI, a great one, he said, well, you're QB, you're A of E, that sounds like support on steroids. I'm like, no, no, no. Let me explain. It proactively follows up on every possible deliverable there is and makes sure it happens. And so my comment, Tim, maybe we can wrap. I'll close and we'll wrap for this week. If you require any deployment for your product, for your company, for your application, if you have any checklists that you have to do, any training, do you have any onboarding for your company? If you have nothing, you don't need to listen to the rest of this.

1:07:07If you have any, your humans are probably not giving you 100 % coverage on this. They're probably being argued with. There's probably issues. It's probably somewhat your fault or someone's fault that's not getting done. I don't think there's a lot of great off-the-shelf tools for this. Build your own QB AI VP customer success. Amelia gave you the whole playbook. It's on saster.com. You can see our whole prompt and how we built it. Build your own one because if you do nothing else but completely automate the onboarding, right, of your customers with no drama, no complaints, no issues like we did, your life is going to be better, right?

1:07:37And so that's why we'll keep talking about QB and 10K. And we'll wrap. Hopefully folks find this useful. If we find that on YouTube, you all churned after 60 seconds. We may do this less often. If we find out people listen longer than average, We'll keep doing it. But we know this is the number one thing we're asked constantly, what's going on with your agents? So this is the agents. Amelia, thank you for doing this one. And we'll see you guys soon and continue the conversations.