984: Building AI Agents Where 99.9% Accuracy Isn't Good Enough, with Raju Malhotra

0:00Jon Krohn:The headlines shout SaaSpocalypse, but I don't buy it. Neither does my guest today who argues the SaaS opportunity is now far bigger than ever before. Welcome to episode number 984 of the Super Data Science Podcast. I'm your host, Jon Krohn. Today's guest is Raju Malhotra, Chief Product and Technology Officer at Certinia, an Austin, Texas-based company whose professional services automation software is used by over 1 ,400 organizations around the world. We've partnered with Salesforce today to explain how their AgentForce 360 platform empowers innovators to turn their ideas into scalable software businesses.

0:37Jon Krohn:We'll look at how successful businesses like Certinia use this platform to bring AI agents securely and reliably to their customers, and how SaaS has evolved to include agentic capabilities delivering far more value than ever before. Ready for this? Let's go.

0:55Jon Krohn:Raju, welcome to the Super Data Science Podcast. It's a delight to have you here. How are you doing today? I'm doing very well, John. Nice. And where are you calling in from? I'm in the San Francisco Bay Area. The place to be for all things AI, for sure, Raju. So you're the Chief Product and Technology Officer at a company called Certinia. And so probably a lot of folks have already heard of your company. But for those who haven't, what problem does your platform solve for enterprises? Yeah. So John, think of Sertinia as a growth engine for professional services companies. So on one hand, we have customers like KPMG, IBM, PwC, companies that are pure play professional services organizations.

1:41And then we also have customers where services is part of some other organization or some other sort of business. So companies like technology companies like Siemens, Salesforce, Google, AWS, HP, Cisco, et cetera. And also some other verticals, accounting, financial services, et cetera, et cetera. But in all those sort of customer scenarios, what Certinia helps on is really help you sell, deliver, and manage your services business. And really think about this as connecting on one side to your CRM system, on the other hand, to the ERP system, and really helps you sell, deliver, and manage your business at scale.

2:26Jon Krohn:Perfect. And a big part of that is the topic of the past 12 months, AI agents, right? It is, yep. Nice. And are you able to like maybe make things concrete with a specific customer story so we can kind of understand, you know, you've given us at a high level what you're able to do with Sertinia and your AI agents. But yeah, like give us like a common user story. Yeah. So, you know, if we think about the AI agents, let me actually just back up a little bit. Sertinia has been in business for over 10 years and we've had a software as a service business working with 1 ,500 enterprise customers, 2 million users, and we'd really help our customers with a number of workflow automations, capabilities, really help them be more productive.

3:13And I would say for the last one or two years, we've been really helping our customers move from automation to autonomy, and that's really part of our AI story. And one example, just a few months ago, we hosted Siemens for one of our customer events where they shared their experience of using Sertinia staffing agent. And, you know, staffing is a big part of any professional services organization. And this, you know, organization within Siemens really helps on driving the services delivery. So staffing is a big part of what they do on a day to day basis. The Certinia staffing agent helps their team, the staffing management team, resource management team, really find the right resources that would be ideal for that role and then staff them.

4:09And as those changes happen, people sometimes get sick, they have to move around, they go on vacation. The staffing agent actually does that optimization in a semi-autonomous way, working with the human team. So think about this as, you know, the staffing agent really part of your team, but really taking something that might take hours to a matter of minutes and really deliver that type of quality experience. And in this case, the staffing is not just for one or two or just a few dozen, you know, services delivery people. You're talking about thousands or tens of thousands of, you know, assignments and they're changing.

4:50And it's really not humanly possible to do this without the help of automation. And now with the help of autonomy with the agent, it's actually much more easy. So Siemens has seen a huge example, huge benefit of the agents that we're talking about.

5:06Jon Krohn:Excellent. That's a great, easy to understand example. In terms of how the AI agent or the Certinia platform is involved, say in that example, If I am a staffing person at Siemens, I'm a recruiter at Siemens, and I want to use a Sertinia AI agent, how do I experience that? How do I interact with it? Yeah, so there are different ways of interacting with it. One is really from the Sertinia app, and you have this user experience that you can log in, you can actually have that connected with your CRM. So the nice thing is that Sertinia is native on the Salesforce platform, one of the largest ISVs on Salesforce platform.

5:47So you really have this intuitive environment that you can do your day-to-day tasks, and the agent really sits with you in that experience. The other experience that we see increasingly getting more and more popular and really loved by our users is it basically gets exposed within Slack. I mean, a lot of work now is getting done within Slack. So the staffing agent is actually sort of like your team member in Slack. And you can ask that staffing agent, hey, you know what? The project that we were working on, the proposal that we had from this prospect, what happened to that? Can we actually staff a new staffing plan?

6:25and what is going on with any pre-staffing type of situation. So really Slack is another way to interface that. But it's different ways of actually interacting with the power of what the agent brings.

6:37Jon Krohn:Nice, that makes a lot of sense. Yeah, so there's the app. You could go to certania.com, I guess is the URL, and be able to interact right there or in Slack. Makes a lot of sense to have it integrated right into the workflow. Make things, I guess it just feels like interacting with another person on the team. That's right. Yeah. Nice. Now, your business is built natively on Salesforce, and you've had several partnership announcements over the years. What does that mean to build on Salesforce? So you're right. Salesforce is a big partner for us. In fact, Sertinia is built on the Salesforce platform.

7:16We're one of the largest native ISVs on Salesforce platform. And that really allows us, on one hand, to provide a user experience, which is very easy, intuitive, seamless. So if you are using any other Salesforce products, the Sertini experience really blends in very natively in that sort of workflow. But what's more important, in fact, is that the data that we have access to as part of a shared org, as a native partner, is really the unified data that our customers have in their environments. And, of course, they have complete ownership and privacy of that. But now, using our capabilities, both our SaaS capabilities in the past and also agentic capabilities, we are able to access, tap into the power of the unified data that could be sitting in their same agent for sales environment.

8:14environment that might be the Sertinya created project data, but that's really data that is readily actionable and accessible instead of trying to ETL that in, trying to bring that in through some of the sources through integrations, it's actually really accessible. And that's really the power of how well the recommendations and the actions our AI can provide. And we are built on AgentForce natively as part of that platform.

8:45Jon Krohn:Nice. Yeah, we'll get to AgentForce 360 in a moment. That's going to be another big topic on this episode. But before we get there, Salesforce is a SaaS business. And we've been hearing a lot in the press in recent months with things like how powerful tools like ClaudeCode have become, Codex from OpenAI. there's been an impact on the stock market all over the world for SaaS businesses, where people are saying SaaS is dead. What's your response to this kind of SaaS-pocalypse sentiment? And does that give you any concerns about having Salesforce as the backbone of everything you're doing? So one thing is for sure that SaaS business and the SaaS itself is evolving.

9:33And I think that's how we see it. The traditional way SaaS has been done in a legacy way, in many ways that is dead. And now we are talking about agentic capabilities, but we don't see a world where you have agents that are sort of floating around enterprise untethered or having unfettered access to the data sources either. I think the world that we're moving towards is a combination of SaaS plus agentic. And whoever came up with the interesting words like SaaS, so clips that I can't even pronounce properly, should probably think about what is the new name for that SaaS plus agent. But really that, I think, is the power of it.

10:16And why that is actually powerful, because we do see the role that humans are playing in this workflow. It's not about just agents. It's not about just humans either. So it's a combination of how humans and agents work together. And in that sense, there are going to be workflows that are really optimized for humans. And there are some workflows that are optimized for agents. And I think where we are moving towards is that sort of optimization of human plus agents working together. So that's really the choice we want to offer to our customers, our users. And absolutely, we see the world where agentic transformation is really disrupting the world.

10:58In fact, professional services, if you think about it, is the most manual, most human business that can be. Our customers are delivering bespoke custom services at scale to their customers. And that's where AI comes in. that's where the disruption is actually the most. So in a way, we are helping our customers really, you know, transform their businesses using this approach of SaaS plus agents.

11:24Jon Krohn:And I think you're touching on something there that means that the total addressable market of this SaaS plus agents world is vastly larger than SaaS alone, because you're talking about all the things that people do as part of that addressable market. And so I don't, yeah, The SaaSpocalypse thing, I don't buy it at all. And you know, there's an interesting thing, John, which is you talked about the addressable market. If you think about professional services, sort of if you think not just the SaaS, but professional services, our belief is that there is actually a net new opportunity that gets created as a result of the agentic revolution that we are in.

12:05And why is that? But, you know, if you start from professional services, how they get delivered, they are very much centered on human delivery. So that means, you know, who's available? Are they in the right time zone? Are they in the right, you know, geography, et cetera? You add the agents and really think about this as a hybrid team of human agents. You unlock an opportunity on the on the demand side, which we did the research last year and we found it seven to eight times bigger than the opportunity that currently exists. because you unconstrained the fundamental constraint in the professional services, which is about skills.

12:41And now with the cloud, with OpenAI, with a bunch of other sort of options, that constraint actually goes away. So overall, that in fact increases the addressable market by seven to eight times, which is in fact a big tailwind for the professional services business.

12:58Jon Krohn:For sure. You're preaching to the choir here, Raju. All right. So before I went into the SaaSpocalypse thing, which is bogus, we were talking about Agent Force 360. So tell us about what is Agent Force 360. I know it's a Salesforce product, but tell us about it and how Certinia takes advantage of it. Yeah. So first of all, I personally consider Agent Force really closer to a platform. So we have a Salesforce platform that provides a set of APIs and tools and effectively platform as a service. AgentForce is the AI stack of the core platform, and it provides the agentic services in a trusted, privacy-connected data type of environment.

13:47So really, Agent 4.360, especially Agent 4.360 for partners, is all the ways that we need as an ISV, one of the largest sort of partners for Salesforce, to accelerate our transformation for ourselves and for our customers on the AI. You know, building an agent, by the way, is non-trivial, but building an agent that performs well in an enterprise use cases with compliance, with scale, with the type of permissions, with the type of capabilities that it should have is really, really non-trivial. And I think that's where we combine our domain knowledge, understanding of customers, and then the power of agent force on Salesforce platform and data cloud.

14:30I think it becomes a force multiplier for us to deliver the ISV agents, the package agents to hundreds and thousands of our customers much more easily than kind of DIYing ourselves or worse, our customers actually trying to DIY themselves. else.

14:49Jon Krohn:On this podcast, I'm always going on about how Claude Code is mind-blowing, but now Claude Cowork is making my jaw drop as well. For example, I recently wanted to quantify how healthy my sales pipeline is for my AI consulting business. I simply asked Claude to estimate my sales for the coming quarter, and it brought info from relevant Google Sheets and my Gmail to create a professional spreadsheet of clients with estimated revenue for each one. Whoa, this might have taken me a day. Instead, it was done flawlessly with Claude Cowork in minutes. Claude is the AI for minds that don't stop at good enough.

15:20Jon Krohn:It's the collaborator that actually understands your entire workflow and thinks with you. Whether you're debugging code at midnight or strategizing your next business move, Claude extends your thinking to tackle the problems that matter. Ah, and you'll appreciate that I can ask Cowork to show me data, such as my sales spreadsheet, and it provides an interactive chart right in the conversation. For problems worth solving, get started with Claude at Claude.ai slash superdata. That's claw.ai slash superdata. And check out Clawd Pro, which includes access to all of the features mentioned in today's episode.

15:49Jon Krohn:Clawd.ai slash superdata. Rajah, can we dive into this Agent Force 360 thing a little bit deeper? It recently opened up for anyone to build on top of. And so any of our listeners can go and learn about Agent Force 360 and see how it could be useful for them. How did it change what you can do at Sertinia? Yeah, so the Agent Force 360 really fundamentally changes how we are able to deliver the agentic capabilities to our customers. So let's actually unpack what's really needed for a good enterprise-grade agent to operate at scale. I think there are really three things that would be critical for that.

16:37First is the agent actually needs a lot of context, a lot of context that is grounded in your structured data, your unstructured data, your metadata, telemetry, et cetera. So a lot about how your business operates and that context is really a very good starting point for the agent. The second area is really about reasoning. and reasoning is how to use, make sense of what data assets, what actions, at what point. And the third one is, of course, the agent experience and how you interact with the agent through Slack, through app, etc. Now, the power of AgentForce, in fact, over the last, say, year, year and a half, as it has evolved, really the power of AgentForce 360 is in all those three areas.

17:26Context is really grounded in the Salesforce data, which is fantastic. what really think about reasoning is an area that it has actually has come up with a lot of new innovations. And the nice thing about this reasoning layer that Agent Force has developed is that it combines the best of probabilistic as well as now with the deterministic logic so that the agent is aware of guardrails in a much more tangible way. So just to give you an example, you know, as Sotinia, we provide financial research, financial modeling for our customers, and it's not okay to be 99.9 % right, right? When you are 99.9 % right, someone goes to jail.

18:11When you're talking about accounts payable, accounts receivable, the financial, you know, statements. And this is where the agent script as part of agent force really adds that determinism to the probabilistic, which is a core capability, usually a trend from an LLM perspective. So I think that's where the agent force, you know, just getting a little bit deeper, helps us deliver agents that are ready for the enterprise environment. They're actually, you know, compliant. They're scalable. They're built on the trust layer. I love that.

18:43Jon Krohn:And I was actually planning on asking you later in the episode about guardrails and what data these AI agents have access to, three-agent Force 360, but I think you covered all of it right there. Through having these guardrails in place, through having access to all the right data and telemetry like you mentioned there, how does that make an impact for your customers? What kinds of results are you seeing? Yeah, so we have seen a ton of great results with the customers who are experiencing the Sertini agents. And I would say the results have been really in two categories. And they usually start with productivity benefits.

19:24And then increasingly, I think they go to the creativity benefits, things that you just couldn't do before. In terms of productivity benefits, we have seen improvement in revenue, improvement in margin, improvement in utilization rate, improvement in just the availability of the team to be able to do some more creative tasks because the agents were able to help on the productivity. For example, in the staffing agent, if you have a staffing agent working with a staffing team, you can do some of the tasks about restaffing and other sort of changes to the team. You can really offload that work to the staffing agent so you can focus on more of a strategy work, more of a creative work.

20:08So the impact has been quite meaningful. Just to give you an idea, typically, if you have a 1 % improvement in the utilization rate for a professional services organization, that translates into 1.3 % to 1.5 % increase in the top line and about 1.5 % increase in the EBITDA, the profit margins for our customers. So if we can improve the utilization rate just by 1%, that has a tremendous impact. And we're talking about companies that are significantly large at an enterprise scale. Many of our customers, for example, are running over a billion dollars of their professional services business through Certinia.

21:00So just think about the impact of all these combination of agentic layer with the SaaS layer is having in terms of business.

21:09Jon Krohn:Sure. A few percent of a billion is not a bad number. Yeah, that's really cool. Great that you have those hard data for us on the kinds of impact that Cirtanya and these AI agents are making for your clients. All right. So moving beyond your company specifically, I've got some questions for you that are broader related to what you're seeing, what you're experiencing. As we've talked about in this episode, we've got incredible capabilities that are getting exponentially better all the time in terms of the length of the human task that can be replaced by LLMs. And what excites you about this current wave of AI innovation?

21:54So do you mean what excites me even more than the frequently novel moments that we keep living in, like Open Claw and Clark Opus Lease and Chad GPT moments? So there's so many exciting moments that I think we're just on a roller coaster that we're living through. But I think, you know, this may sound a little bit of an unglamorous sort of answer, but I genuinely feel excited about the fact that we are getting closer to the impact moment for AI, especially in the enterprise. And what I mean by this is, you know, we've seen firsthand our companies, our customers progress from experimentation to really thinking about ROI in production.

22:39And they're actually realizing the benefits of putting together a number of agents, applying that to their production environments, and starting to see, you know, how much meaningful impact it can have for their own business. And I know this is a disruption. There are a lot of changes happening. But I do think the tailwinds that actually help our customers be more competitive, be more successful over time and really think about their business transformation and their customers transformation in a meaningfully different way is an exciting prospect.

23:18Jon Krohn:Touche, for sure. And it is really exciting times. It's so wild. Hopefully lots of our listeners feel like they need a podcast like this to stay on top of everything that's going on. And speaking of which, a lot of our podcast listeners are hands-on AI practitioners themselves. What advice do you have for folks like that who are building AI-powered enterprise products themselves? Yeah, no, I think your podcast title, by the way, saves it all. I think it's about AI, it's about generative, ML, a lot of different types of AI, but really it's about data science. And I would say my number one sort of advice is that with the power of AI that we have with the LLM, with the reasoning, with the inference, with the improvements in the chips, data quality becomes far more important than ever.

24:10unifying, protecting, and making the data accessible to AI as well as to humans is really a key for differentiation, and really thinking about how that data quality really permeates through the organization. So I think that's always been a hard problem. Really now, I think it's even more critical. And it's not an easy answer, right? It's not a journey. I mean, it's not a 100 % perfect destination. We don't know that, oh, this is perfect, 100 % high quality data. We have arrived here. I think it's really the thought process of how we accelerate that journey to get to a better actionable data. And I think the other sort of advice, which may seem very kind of, you know, intuitive here is having that growth mindset, having that curiosity and learning, because things are changing so fast.

25:02I think that's where, you know, is really critical. And I know there are sort of studies which say we're basically, you know, there's a pilots and experimentation and there's so much going on. I think, of course, the focus should be on productivity and business impact and it is happening. But I would say in that world, experimentation is still very key. Learning individually the latest, you know, greatest things that are coming up, experimenting and having some type of first-hand experience and experimenting even as an organization, as part of your portfolio is going to be a key for differentiation.