#285 Raj Koneru: How Kore.ai is Building No Code Enterprise-Grade Agentic AI

0:00It's the DNA of our company, MyDNA. We're most comfortable building a platform that requires the security and scalability that an enterprise requires. And we know how to sell it to an enterprise. We know how to support an enterprise. The complexity, sophistication that an enterprise needs is not something that everybody can do. At the end of the day, the breadth and the depth of the platform that we have now comes from our experience from deploying this for 10 years. So we have about 1 ,000 people in the company and 650 of them are in R &D and product development. So as you go about building your AI agent, all the bells and whistles that we have seen across hundreds and thousands of implementations, we've built into the platform.

0:40My name is Raj Koneiru. I'm the founder and CEO of Core AI. So I started this company 11 years ago in 2014. And prior to that, I built a company called Kony, which was an enterprise mobile platform company. And prior to that, I built three different services companies, IT services companies, a couple of them that went public, and all three of them eventually got sold. And Kony got sold about six years ago. So that's how I got to Core AI and have been building Core since then, essentially. What's the premise of Core AI, and how is it related to Kony? well if you think about the progression of technology you know from mainframe screens to windows applications to then web applications um you know i built a company every time that was a transformation and i have this knack of uh anticipating that new technology before it hits and investing it ahead of time.

1:54So Kony, I started in 2007 when the iPhone didn't even exist. And the time was clear to me that users are going to use a smartphone to be able to access data and do transactions and consume information. While all of us were used to doing that on the web or through Windows applications. So I realized that enterprises would have to build out these mobile applications, and to do that, they need a platform, and a platform that enabled you to build once and deploy to multiple smartphone systems. That was the premise of Kony. We became the number one in the Gartner Magic Quadrant. We became the largest in the enterprise mobile platform space.

2:44So as users went from clicking a mouse, clicking on menus, and navigating through screens on the web, you know, mobile brought the touch interface, which, you know, initially with the BlackBerry type of interface, and then eventually with the Android and iPhone type interfaces. And that was a different way of interacting with data and information. and while I was doing that there was some articles I read about natural language processing and how the computer can understand natural language so I got interested in that and started imagining that users can instead of touching or navigating through screens or menus can just speak in natural language.

3:38And if the computer can understand what you meant, it can then collect the data that you need and present it to you. It was way ahead of its time. I mean, at that time, the modern-day machine learning models were just coming out, open source or otherwise. And as we started building out a platform for natural language processing and building what used to be called chatbots and voicebots, we realized we had to build the models ourselves for NLP. So we used some of the open source machine language models, but then we built out our own semantic model and a knowledge craft model and created this framework without writing code for people to be able to build out chatbots and then voicebots.

4:33The problem was that going back to 2015, enterprises and their IT teams really didn't understand language computing. You know, they were all schooled and experienced in UI computing, but not language computing. So we had to evangelize the space. There were very few players, us and Google and a few others. And then eventually, 2018 onwards, things started taking off, where specifically for a few use cases like automating customer service or automating employee service around IT workflows, HR workflows, makes sense. And we started growing pretty rapidly. And then, of course, you know, two and a half years ago, ChatGPT hit the market and LLM hit the market.

5:29it. So by then we became the number one leader in the Garta magic quadrant for two years in a row. We became the Forrester wave leader, the number one leader. And we had acquired like many hundreds of customers, large enterprise customers. And the platform, while you could create a chatbot and a voice bot without writing code and deploy that on, let's say, IVR or any digital channel like web and mobile and inside messaging apps like Slack and WhatsApp and whatever. And our customers were deploying that at very large scale, like a large bank, we process 300 million calls a year. A large health insurance provider, we process 300 million calls.

6:21At the time, a large telco, again, similar number, one of the largest e-commerce providers. We process like 400 million chats and calls. To do that, security and scalability was key. So over the last 10 years, we built this foundation where we could guarantee the security of the back and forth between the users and enterprise systems. and encrypt everything and make it only visible to the enterprise. And also from a scalability standpoint, nobody built a platform that was this scalable with these volumes. But when the LLMs came out, we had to change up the framework by which you can build and deploy.

7:10But the foundation of security and scalability still is the same. so by changing that up we've now made it an agentic ai platform where you can choose when you build what's called an ai agent now what used to be called chatbot and voicebot it's the same thing but with an ai agent you could choose the level of autonomous behavior versus prescriptive deterministic behavior so now we have this broad platform where you could build us build something completely deterministic like customers used to be before or completely autonomous with multi-agent orchestration which is a latest you know fad if you will with uh completely agentic or partially deterministic and past partially autonomous and we're the only one who can do that with that security and scalability and governance that has always been our bread and butter so that's where that's how the things have transformed.

8:13And the purpose of why we started as a company, which is natural language, input, computer understands it, better words than output, and then you continue the conversation, is now truer now with the LLMs being able to do planning and reasoning and understanding than before the LLMs. Yeah. Excuse me. So when you say agents, you're talking about conversational agents, not agents that will use tools or perform other tasks in the real world. Both. Both. Both. Okay. Yeah. So you could have a conversational agent where on one end you have a human, you know, initiating the conversation and the AI agent responding in context.

9:13And on the other hand, you could have an agent that gets triggered either by a human or by an event in a system and then goes through a series of steps using multiple LLMs and multiple prompts to process data and complete its action. So that kind of an agent is what we call asynchronous agentic workflows. So you could build either one on our platform. I see. And the platform is specifically for enterprise. Is that right? Is there a reason why you don't have a consumer grade product? Because, you know, agent building is now for everybody, not just enterprises. Yeah, I mean, it's the DNA of our company, my DNA, right?

10:06I mean, we're most comfortable building a platform that requires the security and scalability that an enterprise requires, basically. And we know how to sell it to an enterprise. We know how to support an enterprise. The complexity, sophistication that an enterprise needs is not something that everybody can do. Whereas a consumer-based, you know, something that does something like, you know, book you flight tickets or books you schedules and appointment, anybody can do because the level of sophistication and security and scalability is much different. much less of an issue in the consumer world.

10:45And, you know, at the end of the day, the platform can be used by consumer developers, basically. But our focus has been on the enterprise, which is where our comfort zone lives. Yeah. And so for enterprises that are, this is a sort of self-serve. If they go on and there's, with a login or whatever level of subscription you have, you can mix and match and build using conversation. In the background, is there coding going on or are you assembling pre-built components? Yeah, I mean, if the enterprise can self-serve, they have to learn our platform. So we have something called Core.ai Academy, which is like an online academy with about 250 courses, self-learning courses.

11:49Or they can call us and we can have somebody teach them. Many of them tend to use our partners, our SI partners. We have like 600 of them from the large ones like Capgemini and Deloitte and TCS and people like that to smaller local partners who have built out practices in our technology, basically. And there's no coding involved. It's zero code, basically. So you build your AI agent conceptually in your mind and then use the no-code tooling to be able to put it together. and then there's the whole testing framework to test your AI agent. There's an evaluation studio to evaluate what the AI agent is doing every step of the way and why the LLM provided the response that it did and what context was passed to the LLM.

12:41So there's the whole traceability, explainability, observability built in. So once you build it, then you can test it, you can evaluate it as you're using it and then be able to deploy it. basically at large scale on any of the clouds. We're cloud agnostic, so we run on all the clouds. Some customers even want to run us within their own data center, inside their own firewalls, and many customers do that as well. Yeah, but under the hood, I mean, if I go on and I'm using the conversational interface, say, you know, I want to build an agent that will process expense reports, right? Is there, you say that conversationally on the back end, what the user doesn't see, But is it like cloud code that's then coding up an agent?

13:46Or are there components that, you know, well, we have this rag component and we have, you know, whatever it is that stitches them together? Yeah, so it's more component-based. So there's an agentic RAD engine behind. There's a framework behind that would take your prompt and then string it together with your next prompt, basically. You can do conditional branching. There's tool calling. There's MCP clients that we built in. So you as a builder, you give the instructions through our no-code tooling. and that then constructs in the back end a workflow basically yeah which which then it gets executed it's not code generation per se basically and then you can feed in your documents or have have the agentic rack system extract content from different websites and then use a using a word embedding model basically it would do the ingestion basically into the vector database and then there is a predefined retrieval pipeline, which you can also customize, basically for your specific needs.

15:12So the combination of the workflows that you create with the agentic rack together makes up the AI agent. Yeah. You know, I've seen, I've been talking to a lot of people about agent building platforms and there are a lot of them now. How do you guys differentiate yourselves? I mean, I understand that you're in a little ahead of a lot of people, but yeah. Look, at the end of the day, the breadth and the depth of the platform that we have now comes from our experience from deploying this for 10 years. So we have about 1 ,000 people in the company and 650 of them are in R &D and product development.

15:56So as you go about building your AI agent, all the bells and whistles that we have seen across hundreds and thousands of implementations, we've built into the platform. Whereas others only started to build a platform, like an agent building platform, in the last year or two. So they're going to have to learn that experience over time, whereas we are the most experienced in this, where we've deployed at scale. Secondly, we have no code. we have a no-code platform thirdly, unlike some of the others they are not cloud agnostic they are the cloud providers themselves basically or they are a data provider like a snowflake or a data break and then they are tied to that data source we are data agnostic and we are model agnostic we support pretty much every model on the planet and we also support bring your own model, essentially we also have a model factory in a model studio, basically, where you can create your own enterprise data set and generate synthetic data from it to create a complete coverage golden data set, bring an open source model and fine tune it on our platform and host it on our platform as an API.

17:11So the breadth is all of that. The evaluation studio, the data studio, the model factory, the model hub, the no-code tooling, agentic rag, that's the breadth, basically. The depth is for the use cases. So we go to market with primarily three offerings. One is called AI for service, which is 360 degree customer service. The customer comes in, meets an AI agent who tries to automate that interaction. And then we integrate with every CCAS system out there. So the customer ends up with a human. The same AI agent starts helping the human agent in real time and listening to the customer in real time.

17:51And then once the interaction is over, we measure the interaction between the customer and the AI agent and the customer and the human agent to find opportunities for improvement and then feed that back into the AI agent and feed that back into the human agent. So that's AI for service. Then we have AI for work, which is employee experience. It's rooted in enterprise search where we have over 100 connectors which are optimized for a gentry grad. You can build your own connector without writing code. You can build five types of AI agents, like a prompt agent, a genetic workflow agent, a multi-agent app, a conversational AI agent, basically, to be able to create these AI agents by the employees themselves or by pro-developers for the employees.

18:43And finally, we have pre-built agents for IT, integrated to ServiceNow and BMC, HR integrated to Workday and SuccessFactors and Oracle recruiting integrated to the applicant tracking systems. That comes along with AI for Work. Similarly, for AI for Service, we have AI for Banking, AI for Retail, AI for Healthcare, which are pre-built agents so that you don't have to start from scratch. And lastly, we have AI for Process, which is what you alluded to earlier, which is the process automation component where it's not conversational but there's a process designer and a process monitoring and management tools that are on top of the same platform so it's the same platform for all three offerings with three different you know entry points for three different use cases and we're the only company that gives you that breadth of the use cases basically with the depth of the depth and breadth of the platform underneath So Gartner came out with the emerging Gen AI leaders quadrant like last week.

19:49And we were rated almost at the very top for Gen AI engineering, which is a platform, and for knowledge and productivity, also rated at the very top. Yeah, well, congratulations on that. But I just had the CEO of Boomi on. Do you know Boomi? Yeah, the integration company. Yeah. Do you guys use something like that? I mean, you mentioned MCP. Well, I mean, MCP is so new, but our customers who use Boomi, build that AI agents on our platform and then integrate it to services available and exposed by Boomi, basically, as they do with other competitors of Boomi as well. Right, right. And are you dominant in one vertical as opposed to another?

20:47No, financial services is about 35 % of our revenue. Healthcare is about 20%. telecom and technology is about 20 percent and then the rest is retail consumer travel and some b2b so we tend to focus more on large consumer facing verticals globally uh 70 of our business is in the us and the remaining 30 is in europe and asia so we have global global customer base across different verticals, but not as much in government or B2B sectors, more in B2C kind of sectors. Yeah. And, for example, in financial, what's the largest implementation? Is it sort of back office stuff? Yeah. No. So front office customer service is the number one use case for AI in an enterprise.

21:51So the ROI is easy to determine. You reduce the cost of service. That's what the ROI is all about, whether you reduce the number of human agents or you reduce the time a human agent takes to satisfy a customer, all of that, basically. So this is with the number four bank in the U.S. We process 300 million calls for them and chats. and that is our largest customer, but they've also deployed employee-facing use cases on the same platform for IT, for developers, for DevOps engineers, and they build all those on their own. They have over 100 AI agents, but the one AI agent that has the most volume is the customer-facing AI agent.

22:37Right. And that's for helping customers navigate how to do various things. Yeah. Check your balance, transfers, payments, apply for a loan, understand how you're spending your money. That's like 200 workflows, basically. And also asking questions. How do I apply for this loan? Am I qualified? Those kinds of things. Anything you would ask a bank, basically. Whether you have your bank account with them or have credit cards with them more investment accounts. Yeah. How do you fence off an agent from, I mean, I've built a couple of conversational agents for specific things, but if you ask them because their underlying models are, you know, the big uh agents foundational models yeah and and so if you build something specifically for travel uh but you ask them uh you know who was copernicus it'll answer that uh it you know and then you can just as you can with any of the foundational models go off on a tangent.

24:06How do you, or do your agents still have that capacity, but they're just fine-tuned for a specific use case and they're tied to a knowledge base for that specific use case? But if you talk to it about something unrelated, it'll answer. Yeah. You could do both. You could take an open source or a foundational model, fine-tune it with your enterprise data by creating a golden register. And then it's very limited in terms of its scope. Again, some of our customers have done that. But most customers don't do that anymore, basically, right? Because then they'll have to maintain the fine-tune model. worry about its accuracy, latency, all of that thing.

24:57So the way to do that is twofold. And the way we do it is using these two methods. One is guardrails. So we have a guardrail framework with an app platform. There's a bunch of pre-built guardrails around bias and gender and things like that, which you can use. But you can build your own guardrail, basically. So when the input comes from the user, you can apply one or more guardrails to see whether to let the input through to the LLN. And when the response comes back from the LLN, again, you can apply one or more guardrails to see if the response is within the parameters that you want it to be.

25:35Okay. And most of our customers use the guardrail framework and the guardrails that we provide or they built, you know, to avoid going out of band. The other one is context. So when you give the, when the prompt is generated by our platform, we also attach context, basically. And in the context and in the instruction to the LLM, we generate specific language to tell the instructor LLM that only stay within the scope of these topics. if the input comes unrelated to these topics just say that that's out of my scope essentially so by doing that you can't explicitly expect the developer to provide that you have to implicitly generate that and that's what our platform does so no LLM is 100 % going to follow every instruction but the quality of the prompt basically, and the quality of the positive and the negative outcomes that you want that get generated, you know, restricts the LLM to, you know, what you want it to generate.

26:51And then there's a third way of managing that, which is, okay, whatever the response the LLM gave, you run it through another LLM to determine whether it was staying within the topics that your AI agent is supposed to answer to. And if it doesn't, then it doesn't send the response to the user. It regenerates the response from the first LLM until it's sure that it's remaining within that topic. There's some latency and cost to that, but it ensures that you remain within scope. And that loop with a second LLM, is that a choice that the user makes? yeah you have guardrails that's the easiest one the second is the generated context basically and the third one is you know run it through a loop through another lm the third one is a more expensive option the first two are the easier options basically so most of the time you don't need to use a second llm call to be able to stay within context but that second llm call if you want that, is it, again, the user is dealing with the conversational interface or their dropdown menus or how do you make that choice?

28:17I mean, this is the choice that the developer makes to avoid the LLM answering out of bank. Yeah. The user interface, the input could be voice, could be text, they could upload images, upload videos. And if they did, we use a multimodal LLM to process that. Basically, the output, again, can be text or we provide within our SDK many UI components to take the text that was received and show it in a table, show it in a pie chart, however you want to display. basically or send a link to a video or an audio and be able to show that to the user so that is again something that's available to the developer to create that experience yeah i see uh but but yeah and what i was asking i guess when i said user i meant developer uh how does the developer tell CoreAI the platform, the platform is called...

29:26The agent platform. Or what's Gale? Oh, Gale was an early version of the agent platform. That was for the asynchronous workflows, but now we pull that into the agent platform as well. Okay. But the developer, when he's working on the agent platform, is his interaction all through conversational AI or are there buttons and drop downs and drag and drops and things like that? All of the above. I see. So some of it is conversational. Some of it is drag and drop to create a workflow. Some of it is, you know, making choices in your right-hand side panel, in the property panel as you're building it, basically.

30:13So it's all visual. or conversational it's never never i mean if you want to write code we provide abilities for you to write code to manipulate data or whatever but you don't need to yeah yeah um so you've been in this longer than a lot of the uh the people that are uh offering agent building platforms now uh A lot of people are talking about, you know, as the agents climb the complexity ladder, that you'll eventually have swarms of agents. I mean, you know, we're both paying attention to the same space. And you'll have managers who sort of manage their group of agents. But Steve Lucas from Boomi, who I just had on, he was talking about a headless organization.

31:20You know, someday that's what organizations will be. There will be one guy that handles all the customer-facing agents, another guy who handles all the business process agents. And there won't necessarily be a CEO managing all of the managers. I mean, where do you see this going? You know, that kind of a view is a very utopian view in my mind. Basically, the practical view of it is this is no different than any other application. You know, when you first got a mobile application from the enterprise, you know, you learned how to use it and you figured out, you know, what its value is and, you know, versus a web application versus something else.

32:19An AI agent, you know, there's a lot of hype and stuff about it, but it's an application. A human gives it input or it gets triggered, it does a job, and it comes back with a response. So either a human is a consumer or another set of application is a consumer, basically. The only difference here is that because of the planning and reasoning capabilities of the LLMs, basically, a lot of that and analysis and reasoning that we would do in our mind by looking at data, some of it it will do for you and give you the final result, basically. So for example, you wanted to write an email, write an NLL, you just give it the broad context of what it wanted to write and it does the hard work of actually writing it for you.

33:05But then you review it and say it's okay or not and then you go forward. Similarly, any application that you can build as an AI agent would do something similar. So my opinion is, look, people run businesses, basically, and people provide services and goods to other people. I mean, we are a human society. These applications help us to be more efficient, basically. So there's going to be swarms of these at the data price, much like there are swarms of applications today. Maybe more AI agents than we had UI applications before. The issue is not that. The issue is with these things is there's a new complexity with these things.

33:50And the complexity is not in building them or deploying them. It's then how do people use them? So there's a change management element. When we first introduced mobile applications, employees were not comfortable with it or when web applications came out. But over time, they got comfortable with it, right? Same thing with these. That's number one. Number two the governance and control of these things these things can discover each other each other and trigger each other they need to have identity they need to have discoverability they need to have an agent needs to have access to data so what access what data can they have access to and what data they cannot have access to because with mcp and tool calling you know they could just call any tool and go execute something.

34:38So restricting what they have access to is three. Which humans have access to these AI agencies for basically? And number five, most important, regardless of the AI agent being built on my framework and my platform or built on some other framework or platform, you know, while there may be communication through A2A and tool calling using MCT, basically, you still need observability and explainability across the set of AI agents, no matter who built them and how they were built. So there is a need for a higher level governance layer, what we call a control layer, that each enterprise needs to have.

35:19So that headless CEO is that governance and control layer, basically. But you're still going to have a human looking at that governance and control layer to ensure the army of AI agents behaving properly, basically. So it's the same thing. Like, you know, you have admins for UI applications and there's a central admin. You just need, in this case, a lot more governance because there's cost involved. There's LLN cost involved. There's compute resources, GPU resources that are being used, discoverability that's come into play, basically, all these things. And so our platform, while you can build great AI agents, you can deploy them at scale securely, which is our bread and butter.

36:03You know, we're implementing that control layer. So at the central layer, there's a control and governance layer, which is agnostic of which AI agents were built by whom and deployed where, using which models and connected to whichever data. And we uniquely can provide that value proposition to an enterprise because we're not tied to anything at the bottom layer. When I'm not tied to a model, to a cloud, to a data, where there is a preference to keep my legacy alive, we are the modern platform where everything can be used and consumed, but with the control that can be on top. Yeah. And so, you know, you mentioned or I mentioned swarms.

36:52And, you know, I'm having these conversations with people like you, with researchers who are, you know, looking at the world from the point of view, the leading edge of this stuff. but it really hasn't penetrated the enterprise at scale yet. I mean, people are still playing around. It's mostly customer service agents or conversational interfaces. how quickly do you see this progressing to where agents are really all over the place? I mean, think about it, Craig. Did you hear the word agentic less than 12 months ago? Before? I mean, did you hear the term MCD? I remember the first time I heard it.