SaaStr 828: The AI Revolution in B2B: Insights from SaaStr CEO Jason Lemkin and SaaStr Chief AI Officer Amelia Lerutte, and Qualified's CEO and Founder Kraig Swensrud

0:01Welcome to the official Sastr podcast where you can hear some of the best Sastr speakers. This is where the cloud meets. Up today on the Sastr Podcast. I recently had a CRO I've worked with for years. The deepest respect. Reach out to me in an almost panicked email. I need to learn AI, right? I'll do anything. I'll intern for you. I'll hang out with Amelia's office. I'll do whatever. And I hear this all the time. I need to learn AI. We hear it all. I need to learn AI, Craig. It's part of my job. It's the simplest advice tied to all this. Like, no, you need to do AI. don't learn it buying a subscription to Claude does not count as learning AI the way you're going to do it is pick an agent an agentic product pick the simplest possible one use case and I know it can be scary for some people deploy it yourself buy it set it up if you need a little bit of help connecting it to your CRM or your marketing okay that's okay you don't have to like do the connectors own the deployment oh you don't know what training is it's okay take your time do Do something simple.

1:01If you deploy an agent yourself, train it, QA it, test it, you will be ahead of 90 % of the world. Hands on. Yeah, yourself.

1:12Hey, everybody, it's Saster. Connect data, automate busy work, and empower teams like nobody's business with the one platform that grows with you every step of the way. Learn how Salesforce works for startups at salesforce.com slash SMB. That's salesforce.com slash SMB.

1:33Hey, everybody. It's Saster. FIN is the number one AI agent for resolving complex queries like refunds, transaction disputes, and technical troubleshooting, all with speed and reliability. See how FIN can deliver the highest resolution rates and highest quality customer experience at fin.ai.saster. That's fin.ai.saster. Hey, everybody. Get excited. Saster AI London is this December, December 1st and 2nd, and we're on track to completely sell out with the playbooks for AI and B2B. Join 2 ,000 plus BI and AI leaders for two days, two days of practical advice on scaling in the new year. We'll have speakers flying in from OpenAI, Wiz, Clay, Intercom, Fin, and all your favorite B2B companies, including yours truly with Harry Stebbins and more doing our live 20VC podcast.

2:24It'll be fun all in the heart of London. Don't miss out. Get your tickets and you can still go, go to podcast.sasterlondon.com. That's podcast.sasterlondon.com for a special discount just for you.

2:41All right. Welcome everybody. My name is Craig Swensford, the founder and CEO of Qualified, and I'm joined today in Qualified Studios by two very special guests. Of course, we have Jason Lemkin, who's originally the founder and CEO of EchoSign, sold to Adobe in 2011. And of course, everybody knows him today as the founder and CEO of Saster and also Amelia LaRute, who is the chief AI officer of Saster, which is totally awesome. Thank you so much for being with us today. Thanks so much. That's great. All right. So what does everybody want to know about in today's day and age? Agents. What's going on out there in the real world with agents?

3:20And Jason, something I read that you post on LinkedIn was people are wondering, there's a ton of agent platforms, agentic solutions, AI this, AI that that are on the market right now. Like, where do I actually start? And then I think somebody wrote, what's better, Replit or lovable? Which AI SDR platform is the best? What AI tool is the best for RevOps? And you had a killer answer to that. Can you share that with us? Yeah, it was learning because, you know, we started 2025 behind. We had a lot of AI speakers at Saster, but we were pretty AI lean here. We started with one general tool and now we've expanded it to 20 agents.

3:58And we use Qualified and 20 tools that are sort of maybe 12 are part of our core architecture, which Amelia can talk about. But it took me a while to know the answer to that question or to know the answer between Replit and Lovable. They're related. We could talk about it. But listen, today, the world's going to keep changing. This stuff is new, right? In fact, most of these products, I don't know about Qualified, but certainly Replit and Lovable didn't even work before like Claude 4, okay? It didn't even work before the start of 2025. The Replit, Lovable didn't even exist. Replit existed for a decade, but the AI just wasn't good enough.

4:30So everything's changed. And what I learned is pick a leading vendor, for sure, it does matter, but training's more important. Training is more important. There are very few agentic products, at least that have some of a complexity where you wanted to do something like a qualified, interact with customers, speak with authority, close deals, right? I mean, Amelia was telling me last week, qualified set up seven appointments for us, right? On its own. On average. Now, that sounds great. And it is great. Like, put that as a case study. But it wouldn't have happened if we put zero minutes of training into it, right?

5:02Right, 100%. So I would rather, listen, you know, we're on the qualified podcast. Buy qualified. We are super fans. But more important is just in any space. support, marketing, sales, coding, just pick a leader and go deep, train it, train it upfront for 30 days, commit the time and every day for the next 30 days, train it. And then every week. And it's sound, first of all, most people don't know how to do it. And the other thing is you think, I mean, you were at Salesforce. Think about how people bought enterprise software before AI, you would buy it from a sales rep. Then you might hire an Accenture or a Blue Wolf to deploy it over like a year and you would hope and pray it worked like the sales rep told you got it no one will tolerate that in ai this year no one will tolerate it time to value actually often has to be even before you get a contract signed right and so and training is this key piece to it you got to train it so that's i mean i you know we love qualified i'm deep on replet versus lovable but in the replet versus lovable they are different but most important is knowing the goddamn thing spending I've only 100 days into Replit, but I'm probably 200-hour expert now to make it great.

6:13And so that is just a sea change in how we use software. We're used to buying from a people person and then scaling it up over time and having humans do all the work, put the leads in and the context. Those days are behind us. But remember the old adage, which was like no one got fired for buying IBM. So in like the old days, people would wait to see like who's the emerging leader. So they would wait, wait, wait until there was like the IBM and then they would go. But you're just saying you don't have to pick the leader. Just pick a leader. Like pick somebody. Pick a leader and go. And go and invest the time in training it.

6:46And we're also at a weird time where in many categories there is no billion-dollar ARR company yet. You can say you're a leader. Maybe qualified as a leader, but as great as qualified is, it's not a billion ARR yet, I don't think. Not quite yet. Not yet. So we are seeing leaders emerge, but it's early. But at the same time, what is different from the old days, not only are leaders emerging, but we've never seen so many folks looking for new vendors. They're either looking for new vendors because they have AI budget. They're looking for vendors because they're under pressure from their boss to show AI innovation.

7:20And in many cases, it's the only incremental budget. No one is putting more budget into their old SaaS software. It's all going into AI. And so everyone's in market. It used to be, I don't know what you learned as a marketer at Salesforce, but traditionally you'd hoped a prospect would be in market every five to six years, right, for a big buy, right? Because it would take like a year to get it going, the second year to scale up, the third year to get frustrated, and the fourth year to start looking at a new solution ahead of fifth year, right? Now everything's in market again. This has never happened in – maybe when the web started, right?

7:56But everything's in market. It's in total flux. it means a lot of things for different players but if you're a buyer just get going it's the right peril right in 1999 2000 like everything was in flux yeah and nobody knew is people soft going to lead me to the future of hr you know is siebel going to lead me to the future of crm and it's kind of like that same thing right now like who's going to lead me to the future of in our case go to market but in all these other markets so even there you know as cloud got bigger You know, the only time I think in our lifetimes we saw intense pressure to go to the cloud was 2020.

8:31OK, and that was an exogenous situation where people I literally saw investments I made. We saw it in these signatures, but I saw it in companies I invested in, like talk desks for cloud contact center. Like you could wait 20 years to decide if you really wanted your contact center to go to the cloud. And then in March 2020, these businesses went 10x because I have no choice. Right. But that was the only time when people felt like they had to go to the cloud instantly. But now, yes, it is a small subset of tech and innovation, but there's so much money in I need to go AI today. Right. And you've got to be part of that if you want to win.

9:04For most companies, it's coming tops down from their board. It's like the board to the CEO to everybody else, move, move, move. And so, of course, that's where the incremental budget is coming from. You guys are a unique company because of who you are and what you do. But you said at the beginning of this year I had zero agents. Yeah. And now I've got 20 agents that are in production doing various things. And Amelia, you're the chief AI officer of the company. What have you got and kind of how have you done it and what have you learned? Yeah, we started with a platform for advice, right? So Sastr that Jason started was for giving founders advice, giving them, you know, how to hire my next VP of sales.

9:42What should their comp plan be? How should I hire the next CRO? When to hire a CRO? Don't hire, you know, a marketer who's only going to give you blue pens. Like now famously classic advice, right? For Sastr. And so we started the year with no agents. And we're like, okay, the number one problem we have is like outside of Sastr Annual or like Sastr London, all these events we do, people always want to ask Jason a question, right? Like they come up to him. Even here at Dreamforce, people come up to him and ask him questions. And you can't really scale that. And so we started with a platform that's called Delphi, and it's a clone of Jason.

10:14So it is all of your advice, your social media, everything on Sastr for the last 12, 13 years in the clone. And so. By the way, it's incredible for the founders and entrepreneurs that are out there. Try it. It is incredible. Yeah. Go to sastr.com in the bottom right-hand corner. Exactly. Yeah. Yeah. We were blown away at first how good it was. With training. With training. With training. Because it ingested all of Sastr, which is pretty. I mean, now I'm like, oh, a lot of tools do that. But back, like at the beginning of the year, I was like, that's impressive. Like, it's a lot of, it's like, what, 20 million words?

10:45It was 20 million words of content, 1 ,000 YouTubes, all the tweets, all the LinkedIn. It did break the ingestion engine. It did. But once it did, it was pretty good once we tweeted it. Yeah. I'm like, okay, that's pretty magical. So you started, you call that Jason AI? That's the Jason AI. Okay, so you started with the Jason AI. Yeah. And where did you go from there? So that was before our big event this year, which was in May. And so in the run-up to that, famously, we both were like, okay, this is pretty cool. And people are starting to use it more often. They like it for advice. Again, classic, like, but now it's like, how do I hire a VP of sales?

11:12They have the conversation with the AI and the AI can have follow-up conversations. So now we'll see people use it as like a copilot of, oh, hey, I'm thinking about hiring this person for my VP of sales. What do you think of their resume? Or I asked them this in review and they asked me for this comp. Should I push back on that? Like, what would real Jason do? And so that piece was cool and it's really good at that and it still is, to your point. But where I kind of saw some failings that we both saw, I was like, it's not as good at answering our actual like SASTR needs of, okay, we have this event coming up.

11:43So that's a specific use case of basically trying to clone Jason. But the interesting part, Damelia, for sure, it's a horizontal, you know, there's talked or Dreamforce say about horizontal versus vertical agents, right? Delphi is as horizontal as it gets. There's almost no workflow. There's maybe a couple little triggers. It can't do things that Qualified can, but it can do a little bit of everything. And so it was supposed to just be digital JSON. And what was interesting is, and this is an issue with a lot of agents, people would start using it for support. Okay. Hey, I got a booth at Sastra Annual.

12:12Can I have popcorn? Yeah. And we didn't expect that. Yeah. And this is an issue so many will have with AI agents. It will keep, if it's good, if it's bad, you'll have no ROI. If it's good, people will start using it. I'm sure you hear it all the time. People will use it for things you do not envision if they trust it. You got it. So instantly people wanted it for issues with attending our events and for sponsors. And we're like, oh, well, then it started to hallucinate. It was pretty good on my content. And so we had a magical moment, which is we uploaded our prospectus. And it only took a certain amount of context and training, but then it started to get good.

12:49So then it was pretty good at doing maybe 20 % of what qualified good. But here's the thing, and the people get this wrong, 20 % is a lot better than zero. Like if we never used qualified and only use this general agent, we wouldn't have the seven appointments because it can't set meetings. Yeah. But even 20 % is better than zero if it's good. Sure. Just the fact that instead of it being – like we used to use intercom and it would be a weak response time. Okay? So we switched to our agent and it could instantly tell you about 90 % right, your customer question. And so sometimes 20 % is enough.

13:21That's as far as we got. And then when we had time after annual, we're like, okay, let's do better. Yeah. Let's go from horizontal to vertical as a phase two. And I don't think that's a bad approach for some folks. Like get one thing working that's horizontal, gets confident, and then go, okay, now let's do SDR, BDR, marketing, dedicated this, dedicated that, right? So we went from horizontal to 12 verticalized agents in a sense, right? But we knew in that process it set us up for where we are now, like successfully with multiple agents. because I think otherwise if it had failed or if we expected too much out of it, I don't know, we probably would have stayed on the one or something or dumped it, right?

13:57So I think that's where we're like, okay, post-annual when we have more time, I started to look for vendors like a qualified, an artisan, a Delphi that we could bring on that would make sense for us, right? So Sasser has big scale, but internally we're very small. So it's like, okay. The number one thing that was driving me crazy was two things were driving me crazy. The first being that when you came to the website to become a sponsor or buy a ticket is very antiquated. Like, fill out this contact form, which I hated. I was like, not only is this the worst of filling out a contact form, it's very dated.

14:30And also half the time I round robin it to our sales team, they don't follow up. Right. At least half the time. At least half the time. So, like, I'm like, half the time, I know if I'm giving it to, like, one of the better reps, it's going to get followed up with instantly. But I give it to the other rep because it's their day on round robin. I'm like, they're either getting a response half the time or not at all. So have you blown all that up and you just do it all with agents now? Yes. And who manages the agents? Right now I do. This is why we changed my title to Chief AI Office. It's an issue.

14:58It's an issue, right? Just briefly, we can talk about that. The learning for folks that watch, I didn't know this at the time, stair-step it. I learned this for our agents and I learned it on Replit, okay? So the fact that we went from Delphi, a horizontal agent, which basically can self-ingest your content with no training. Like you do need to, but it's the least amount of training of the tools in our stack. Start easy, start broad, get a few, get a win under your belt. Even if you started with say qualified, I'm not a total qualified, start for the simplest use case. Okay. Because if you fail with your first agentic thing, not only will it, is it frustrating?

15:31You won't know. We won't know. Yes. You just won't know because we had the early win that, that was better than expected. Like our general digital Jason could do some sales. It could do some qualification and it would start to say, you know what, Craig, you're not big enough for SESTA annual this year, don't come. That was great for us. It would do it on its own. And so we gained confidence. And then I know we use different nomenclature from you, but going from that to SDR to BDR, so going from outbound to inbound, for us, different people are different. For us, outbound was easier than inbound.

16:01It was less risky to send a better email than a mediocre human than to trust an agent with a precious lead. You got it. Right? So if you want to go to the most complex part, at least somehow learn first, because it's just too risky. But if you can stair-step this stuff, you will build confidence by your second, third agent or deployment. And we didn't intend, but that's what we ended up doing. And that's how even us were successful. We stair-step. I think every company is probably a little bit different. But like the stair-step, we were talking about, you got to crawl, walk, run. Everybody like sees the prize of like, I'm going to be sprinting.

16:36And you're like, whoa, you know, where do I start? And you got to get the quick win. You got to build the confidence. internally. You got to kind of learn the tool. You got to invest the time. And like... You have to invest the time in the tool. But it doesn't take that long. You guys had zero agents at the beginning of this year. Yeah. And now you have 20. Yeah. So it's not like we're talking about crawl, walk, run in three years. We're talking about in months, right? Yeah. But I don't want to hear Amelia's thoughts. I would say there's different ways you could count. Because I think there's agents that are simplistic and there's rich agents and there's agents that take more of your time than others.

17:07For sure. But we think of us as maybe having 12 prime agents. I don't – without another human, this is about the most we can do. Okay. I don't – it was fun. Now, like, the bar is so high for us to add a 13th core tool. It better – we will. There's some stuff that is not automated, but it is, you know, the – because you wake up – Amelia wakes up in the morning. It used to be me when I trained her journal. I mean, it's an hour of work in the morning dealing with the outputs, and you're thinking about it all day. You said a couple of times, I can't remember what show it was on, but you said you put in an hour in a day and you'll reap the rewards of doing that.

17:45We'll never go back. Yeah, I'm never going back. No, for sure. Well, I do think stair-stepping was very important to both of your points there. But I think part of it, too, is when I spend that hour in the morning, it's an hour I used to spend with people. Like, okay, when we had human SDRs and human BDRs that, again, would maybe half the time follow up with an inbound lead or half the time, maybe a quarter of the time, send an outbound, right? Like, maybe even worse outbound. Like, this was an hour I would spend with those people either doing one-on-ones or going through accounts or, you know, going through accounts they should be following up with.

18:19Or, again, like, hey, here's a lead. Here's what they did with SASTR, like giving them the context of what people have done with SASTR. So instead of that hour going to that, now I just spend an hour with the agent. I'm like, okay. The agent, I wake up in the morning. I'm like, okay. The qualified agent has booked already, you know, whatever, a couple of meetings with David. I can see the conversation. I can give him any context. I can be like, Hey David, you know, just have a fight. Cause we use momentum too. I'll be like, okay, read the pre-read and momentum. So you can see what they did with us before.

18:44Make sure you just open it off. So it'll move the stage for you. And then like done. And then I'll check on, like, I literally go agent to agent. Like I'll go qualified. I'll go artisan. I'll go Delphi. Now I go agent force. Like I just go through all of our agents and I just check on them for that hour. But let me get your perspective on this. You said that same hour you used to spend with people, right? And inevitably, some of those people would move on from your organization or you'd find that maybe they weren't the best and you'd part ways in some way. But that hour you spend with the agent, it compounds over time.

19:18Have you found that? Yeah. It's so much more productive. I would say in the early days for any platform you get, that's a genetic, right? Most of your time is going to be more than an hour and it's going to be setting it up like all these at the forefront required a lot more setup time right like it just does because either you have to fix your data or it needs to talk to something else it needs to talk to something else or like us i was like you know i thought we had okay data quality in and i was like no it's terrible but it exposes that it exposes that right or like famously when we hooked up momentum which is like a bit of a recorder and a complimentary go-to-market tool to qualified, we had somebody leave our team that day because what it does is it will slack you a summary.

20:00It does a lot of stuff, but it'll slack you a summary after each sales call. And it'll also slack me anytime like a new opportunity is opened or meetings booked or like whatever. And this rap was like, I have no activity. So like it literally exposed that. Like, what are you going to do? You have to quit that day. You like, you can't hide. You're going to be found out instantly. So yeah, instead of my hour now being with those people, I spent an hour with all of our agents in the morning. Again, it used to be set up. Now I check on all of them. I see if there's any follow-ups we need to do. And then I do more things now, right?

20:32Like it used to be, okay, I'm gonna set up all the platforms to do their core things. I'm gonna set up inbound, outbound, support, ticket information, event support, like whatever we need. We have that all set up now with our agents. Then I was like, oh, now I can do email. Like I can de-anonymize some of our website traffic. I can send them a follow-up campaign if they didn't actually buy a ticket. Maybe it doesn't matter if they were on the website or not. Like now we have all these different use cases that it started to compound itself because I'm like, now I can just do a lot more in that one hour with like my five core agents than talking to two humans 30 minutes each.

21:06But it is more like this as a society, we're going to figure this out. It is because you can do more work. It is a higher cognitive load. You have to think more. Right. You don't spend time arguing with people. As I say with Amelia, the agents don't cry. Okay, they don't cry. and that is exhausting, right? But it's exhausting in a different way, right? Exactly. You know, if someone's going to cry, meeting with you, Craig, you just have to sit there for an hour, okay? But you don't have to think, you don't have to use all your brain cells for an hour, do you? With your agents, it's so many brain cells and that's, and so I just think that, you know, even if you think about coding, you know, everyone, I just wrote this up, everyone brags that 50 % of our code of qualified is written by a cursor or, but that doesn't mean your engineers only work three hours a day, does it?

21:50No, not at all. Yeah. And so I haven't seen a lot of engineers complain about it, but I do think for the best engineers, there's more cognitive load. Yes. Because you can do more. It's not as simple as doing more releases. That's more you have to think, okay, you know, engineering is very creative. And so the more of the road stuff that's done, it's fun, but it's exhausting. You know, you might have to think about how to make that agent work better eight hours a day instead of three hours a day. It's much better, but I don't know that everyone can make the jump, right? I don't know. So where do people start?

22:19You did a post that was about the layup roles. Yeah. And can you define, like, what are these layup roles if somebody is like, you have now almost a year's worth of experience. Yeah. And getting agents into production. You've seen what works. You've seen what doesn't work. What are these layup roles? Where should people start? I think the layup, maybe it's Captain Obvious, the layup are the things where there is a tool that exists that is trustworthy, where it's just not getting done in your org. Right. And there are a lot of reasons that might not get done in your org. It might be an area that's tough to hire in.