How AI Agents Are Transforming Customer Support, with Decagon’s Jesse Zhang

0:05Hello and welcome to NoPriors. Today I'm talking to Jesse Zhang, co-founder of Decagon. Decagon is an early stage company building enterprise grade generative AI for customer support. Founded in August of 2023, their platform is already being used by large enterprises and fast-growing startups like Rippling, Notion, Duolingo, ClassPass, Eventbrite, Vanta, and more. Jesse, welcome to NoPriors. Of course. Thanks for having me, Ilok. Absolutely. Maybe we can start a little bit with sort of your background and what Decagon does. You know, you're a serial founder. You started another company before this in NianticBot.

0:39and um you know now you and ashwin have started decagon and you've been working on it for a while and have seen some really interesting adoption from companies like rippling notion eventbrite vanta sub stack and many others right so you you've really started to carve out a real um space for the company can you tell us a little bit more about what decagon does how it works what the focus is of the company of course yeah so quick background on me um grew up in boulder did a lot of math contests, stuff like that, grown up, studied CS at Harvard. As you mentioned, started a company right out of school.

1:13That company was eventually bought by Niantic, and then I left to start this company. Oshman and I, we met through mutual friends, officially met at this VC offsite. And when we got together, we were like, okay, biggest learning from first company is that can't really overthink things too much. We started by just kind of obviously being interested in AI agents. It's very exciting technology, arguably like the coolest thing from this generation. And we just talked to a bunch of customers like the ones you listed. We, I think over the years, have gotten a lot better at figuring out how to talk to folks and what questions to ask.

1:57And through that process, we kind of arrived at our current use case as maybe what we think is like the golden use case for these AI agents, which is customer interactions, customer service. The use case is very tailor-made for what LLMs are good at. And so we started building from there, right? And we still weren't thinking too much about division or anything yet. It was just like, all right, we had a lot of customers in front of us. How can we make it so that they're happy and they really like what we're building? And then that led to kind of where we're at now. I would say right now as a company, Tekagon, we ship these AI agents for folks to use on the customer service, customer experience side.

2:36The thing that's made us special so far is we have a huge sort of focus on transparency, I guess. So when people use us, especially these larger companies, it's very important for them that the AI agent is not a black box. that they feel like, okay, even though LMs are cool and there's a lot of things you can do with them that they can see how decisions are being made, like what data is being used, how do you come up with answers? And if I want to get feedback, I can, that sort of thing. So currently we're in production with a bunch of these, these large folks that have large support teams. Pretty much any company that has a large, sizable support operation is a good fit for us.

3:15That makes a lot of sense. It's interesting because I feel like one of the things that's been really striking over, say, the last year in the AI world is the CEO of Klarna posted on X or tweeted about the impact that AI has had on their customer support or service team. And Klarna's sort of like a buy now, pay later service out of Europe. And his tweet basically said in the first four weeks, they handled 2.3 million customer service chats. The customer satisfaction was on par with humans. It's 25 % reduction in repeat inquiries relative to people. it resolved customer errands or issues in two minutes versus 11 minutes for a human agent.

3:54And instantly they were live 24-7 in 23 markets in 35 languages because AI supports so many things. And so, you know, it had a huge impact on that company. And I think they sort of shifted 700 full-time agents to do other work, right? In terms of the impact of Klarna itself as an organization. What sort of impact have you been seeing with your customers as they adopt this sort of technology? and how do you think through the lens of what you're really bringing to these customers and the sort of satisfaction that their own end users have? It's an interesting way to think about it, which is all these people are shipping this use case, right?

4:30It's like there's a lot of evangelists out there, which is nice. The Clarion article is awesome. There's a lot of tailwinds for the industry. And I think one interesting thing we've seen is that the benefits that people get are all roughly in the same vein, but different people prioritize different themes. And so at this point, it's not really even that much of a hot take to say that like in a couple of years, these ages are gonna be super pervasive. People can use them for all these customer interactions. They're gonna be everywhere. And so to your point, like what is the benefit? So for our customers, it's always the same.

5:05It's what fraction of total work, in this case, like conversations can the AI agent do? So how much work is this saving us? And then two, how much happier are our customers, right? Like what's the customer satisfaction score, MPS score? Those two are often just the leaders by far. As I said before, like different people maybe value each one slightly differently. And then there's kind of other things like, okay, well, we want to make sure that there's accuracy, right? Like if we're in a regulated industry, you know, this has to be very accurate for us. So those are kind of where the benefits lie.

5:39It's like, we're saving a bunch of money. We're saving kind of time and resources. But also on the other side, we're kind of making the customers happier. And so that can lead to higher retention, more conversions. And it's kind of a lot more upside there. It's like you're giving every customer a personal concierge, basically, in their pocket that they can chat with any time, any language, 24-7. And that can be pretty transformational for a lot of businesses. Is there any example customer that you could talk about as a case study in terms of the impact this has had, how it's lifted their metrics, the success they've seen using Decagon?

6:15Of course, yeah. So we just did a big case study with a company called Built Rewards. Great use case for us. They have a very large user base growing very quickly. You're actually using it to either make points or make payments. A lot of my friends use the product. And then as a result, because you have a large customer base, like people have questions, people have things that they need help on. And so the number of support inquiries basically grows linearly with the number of users. And because of that, because they're growing so fast and basically exponentially, that means the number of support inquiries is also growing exponentially.

6:52So when they first started using us, that was like the main goal. It's like, holy crap, we're getting overwhelmed by all this volume. Can AI help here? And so the thing that ended up happening there was, yeah, within a basically a month of starting UseUs, they were able to stop scaling their team and the AI would take over a lot of the automation, just makes everything very smooth. And then now, basically, we're almost a year in at this point. They've been able to really restructure their customer support team. And again, we published a case study on this where they were able to quantify like, okay, what are the savings, right?

7:30And so, so far it's around 65 agents of just like headcounts. So very tangible difference. And then for us, it's also great because we're able to provide them the value there. It's like a very easy ROI, but the customer experience is also a lot snappier and they get a lot of social media posts about like, holy crap, like I just tried, you know, build reward support thing and it doesn't feel like any sort of AI or chatbot system we've ever used before. So that makes us happy. Could you tell me a little bit more about what you've built from a technology and infrastructure perspective? So I guess there's the core models that anybody can access, right?

8:08The GPT-4Os of the world of GPT-4, the Cloud Sonnets, et cetera. And then there's all the stuff you've built on top of it to actually make this work well for your specific use case and for customer support agents. Could you tell us a bit more about what you all have had to build over time? Of course. Like you said, everyone has the same access to the same models. We see ourselves very much as a software company. And we're obviously doing a lot of work around AI and using AI models a lot. But I would argue that most applications nowadays are real software companies and AI models are kind of tools that everyone can use.

8:43And so most of the sort of alpha or most of the special stuff that you build is on top of models. It's either the orchestration layer or the software around it. For us, there's been a big focus on both. The orchestration layer is kind of how you can use all these different models together. You probably have evals set up that measure how good each model is at certain things. You put them together and the whole goal of putting them together is to mold it around the business logic of the customer. That's part one. The other thing you build is just very classic software. You have this AI agent there.

9:18It's all the things I was saying before. Transparency is a big piece. You really don't want this to feel like a black box. That's just their answering questions. And so how can you build all the tooling to see, okay, what's the data that the agent's using? What steps is it taking? Can I analyze all these conversations that are coming in? If you have a million conversations, it's like, okay, no one's reading all those. So how can you make it so that the AI, the LLM, can read every single conversation, tell you how stuff's going, find gaps in their knowledge, give you a breakdown of like, okay, here are the big categories you should care about.

9:52There's like a bit of a trend here that's been interesting. So that's all the software around it that we're building. And that's typically how it's structured. And the orchestration layer, I think it's gonna be different for every agent, right? Like our agent versus like a coding agent, that orchestration is gonna look pretty different. But at the end of the day, you're kind of just building a sort of a structure on top of the LMs. Yeah. It seems like we're very early in the days of true agentic stuff. And that includes the ability to sequence chains of events that include certain forms of reasoning.

10:28Obviously, there's things like O1 and other things that have been coming out to start to try and address this, but we seem quite early in the scaling curves. What do you think are the main pieces of technology that are missing to really take you or your sort of vision to the next level in terms of how these agentic systems should work? Yeah, so one thing we were talking about the other day is there's actually different types of intelligence with the AI models. And a lot of the recent developments with O1 or Sonnet and stuff like that has been around, I guess, like quantitative reasoning intelligence.

11:03So they've gotten better at coding. They've gotten better at math. and for us actually those things help but they're actually not the biggest difference maker so in our use case that type of intelligence that matters the most we would probably describe it as instruction following so you just have a bunch of instructions like can you follow it to a TV and I'm sure there's other kind of types as well but for us we're excited to see developments on the other areas too and people everyone's saying like oh there's a plateau happening with the core models and the intelligence. I think when most people say intelligence like that, they're probably talking about the reasoning capabilities.

11:44For us, and the agentic flows that we use, the instruction following is a huge piece because you have to just think about a customer service, SOP, or a workflow or something like that. You just have to be very accurate about it. And so I know there's research going on about this in the major labs. And I think that's one thing we're looking forward to next year. One other area that it seems like really touches on customer success and customer support and sort of user experience is also voice-based support. And I think one of the things that's a little bit under-discussed in the AI world, because we keep talking about large language models and understanding of text, and obviously that stuff is crucial to everything else, but I feel like we almost under-discuss text-to-speech engines and the ability to understand spoken word and then respond with audio, right?

12:36And so there's companies like Cartesia, 11 Labs, OpenAI, Google, et cetera, who are starting to provide some of these services and APIs. How much of an impact does that happen what you're doing? Or is that a separate type of product? Or, you know, how do you think about the voice component of these things? Great question. A huge impact. So we have customers now trying our voice agents. And if you think about our space, right, like you have the overall problem is the same, which is you have a bunch of customers. They have questions or issues or things you need to talk about. And the channel really doesn't matter for them.

13:10It's like some people prefer voice. Some people prefer chat. Some people prefer email. Some people prefer SMS or something like that. And so our job is to handle all of those. And obviously, you start with text because that's the most it's like the easier one. and it's easy to evaluate for the customer as well. I think just now you're getting to the point where you have big companies that are very interested in voice and actually they've seen the results of a text-based agent and they're like, okay, well, yeah, we should be able to generate voices and do the same thing for phone calls. None of this would be possible without the models that you just listed, right?

13:48And those companies, so LM Labs, OpenAI is doing some cool stuff, Cartesia. And I think there's also been huge strides this year with those models around how realistic the voices sound. Also, latency matters a lot in our use case, because if you're making a phone call, you expect things to feel very snappy. So yeah, big topic for us. And as these companies get better, I mean, we're working with them pretty closely right now on how you can actually build these things well at scale. But as they get better, that's also going to be huge for us to keep delivering these voice agents. Makes sense. Yeah, my sense is one of the issues is latency in terms of it takes enough time to take an audio stream or somebody's talking, translate it into text, feed that into a language model, and then output it as voice again, that it feels there's a lot of pauses or people have to kind of wait.

14:50And there's different things that people have been trying to do in the background, like streaming the potential solutions back out and then being able to try and shorten that latency timeline. Do you feel latency is still an issue or is it just solved by integrating voice directly into the models in a deeper way for some of these services? Or when do you think latency becomes a solved problem for these sorts of application areas? Latency is a big deal here, of course, with voice models. So nowadays you have the voice-to-voice models that we're playing around with, OpenAI is doing a lot of work here.

15:22I think there's obviously a lot of trade-offs there. Voice-to-voice latency is great. Sometimes, though, with these production use cases, you do need the extra computation cycles. So, you know, fetch data, do multiple model calls, or there might be other reasons that you can't do voice-to-voice. And so, okay, that's one option that you would consider. The other one is the one you described where you're kind of going through your transcribing or doing speech to text and then doing all the computation within text and then generating the voice at the end. That always causes a little bit of extra latency, of course.

15:57And so as you mentioned, a lot of folks have figured out fairly clever ways to get around that. You can start generating stuff first. In our use case, you can always do something like, hey, give me a sec. I'm looking up your data. So these are all things we're playing around with. I think for each customer that we work with, there's different trade-offs. And so we're really trying to base what we build on the things that we're hearing from them and the sort of priorities that they have. That's cool. One thing that I think is kind of interesting is the number of companies in the AI world today that have been founded by people with Math Olympiad or IOI or other sort of backgrounds, right?

16:37And I think you were sort of involved with Math Olympiad stuff in high school. I think Decagon has actually hosted some math Olympiad events for the team, which isn't like your typical happy hour. But there's other teams and companies. I mean, before that, there was ramp and things like that. But I think the BrainTrust team and the Pika team and Cognition, which launched Devon, and then you all kind of have that common thread. Where do you think that comes from? Like, why do you think this community is now so active in AI? That's a good question. I mean, yeah, we're actually all around the same age as well.

17:08So we've known each other since like, you know, middle school, high school. Well, one, it's a great community. For us, we have a lot of people on the team with math contests, coding contest backgrounds. I think it's more so that this community was always there. Math contests has been around for a while and a lot of super smart kids that go through that. It's also a great way for folks to kind of get to know each other and get connected and build friendships. And I think the main thing is that now in the last few years, maybe the last five, six years, because startups have been a lot more mainstream, a lot of folks in this demographic have gravitated towards startups as opposed to traditionally it'd be either academia or quant trading and things like that.

17:57So they're just a big influx of these super smart, super talented people that come into the startup world. And because there's this community aspect, you know, folks can see what other people are doing and like, you know, what sort of works and types of companies that people are building that, I wish I didn't say they're all the same, but I think a lot of folks with these backgrounds are now kind of working on startups. And that's why there's a lot of, you know, I guess, progress in the companies that folks have been building. And are there ways that you all have been sort of supporting each other through the startup journey?

18:33Because I feel like every generation, there's sort of a clique of people who built some of the more interesting companies who all kind of interact. They provide advice. Maybe they angel invest in each other. Like there's kind of a thriving community. And every five to seven years, it kind of shifts who it is. And I feel like, you know, the IOI sort of math Olympiad community or coding competition communities are kind of very engaged right now. Is there any formal version of that or are you all just kind of informally helping each other? Yeah. I mean, I angel invest in a lot of the companies you just listed.

19:01A lot of their founders are angel investors in our company. It's very informal, obviously. It's just casual friends helping each other. I think the main thing is that with company building, there's just a lot of service area. As you know, it's just like, how do you hire people? How do you do sales? How do you build this thing? How do you structure comp? There's infinite things. So yeah, having the other data points is obviously super helpful. So I hang out with them quite often, play games, play card games. It's a Chinese version of Bridge that I play with a lot of these folks quite often. And it's just, it's fun where you just kind of hang out.

19:42Everyone's kind of in this relative same stage of life. And so, yeah, like you said, there is definitely a lot of camaraderie and help that goes around. Is coming from this background from the sort of math Olympia community impacted at all how you think about hiring or your hiring practices at Dekagon? A little. I mean, if someone else has the same background and has gone through the same contests or programs, obviously that is pretty good signal since I have a good idea of what those people have done. My co-founder, Ashwin, also a similar background. He didn't grow up in the U.S., but in India he did a lot of these contests as well.

20:18And so, yeah, I think there's some correlation with people who just as kids just did a lot of this stuff. And then, you know, now we're all adults and, you know, there's some sort of, you know, signal there when you're talking about hiring. But for the most part, like, it's, there's so many talented people here, whether or not you, you know, did math contests or not. NSF, you know, at Declan, at other companies that I think our hiring process has been more or less the same. It is like a nice sort of trigger for events, I guess. So when you host these events, people come out, you can get a nice community of folks that are interested in the same things.

20:55And we're going to be hosting probably more. And not all of them are going to be contest-based, obviously, like puzzles and things like that, where you just get a lot of fun engineers and people bringing their friends. And that's pretty important to us. And then I guess for AI writ large, what are you most excited about in the coming years? or if you were to extrapolate out 12 to 24 months, what are you anticipating most gamely or what are you waiting for? So obviously the model's getting better is awesome. The model's getting better across different modalities, also awesome. We talked about voice.

21:28There's also other parts, other modalities that are also tangentially interesting to us, right? So you talk about a lot of our companies are, a lot of our customers have software products. And so it'd be awesome if you're asking questions to AI agents and it has like context of your entire screen and all the interactions you've done and stuff like that, that would be great. And you can even go a step further and have it actually help you navigate stuff. So there's just so much you can do there where you talk about the other modalities or even just more advanced model capabilities. We've seen the computer use demo from Anthropic.

22:02Probably, in my opinion, not production ready yet, but as that gets better, there's a lot of cool things you can do there. So on the model side, that's one thing we're excited about. But on the core model side, I think one thesis we have is as the years go by, again, AI agents, I think at this point, undeniable that there's going to be a reasonable explosion of them where there's used on a bunch of different use cases. I think some use cases will take longer than others, but the value that they're providing is pretty undeniable. So there's definitely going to be a lot of AI agents out in the world, in our use case, customer service, in other use cases.

22:41But one thesis we have is that the nature of the work of human agents and people like us is also going to change pretty drastically. And one of the things that are going to change is that there's going to be a lot more people that are supervising and editing agents. And so that's something we think about. We're excited for a lot of the sort of innovations there because right now, a big part, like I said before, Or we care about letting the human agents for our customers and their leadership team to go in and make changes and monitor the agents and just have a lot of visibility and control. And what does that look like?

23:19If you compare it to a human, if you're monitoring a human, you can give them feedback in real time. You can be like, oh, no, no, don't do this. You did this thing wrong. Please do this next time. When you're doing that with an AI agent, there's a lot of different possibilities because they have some things that are different than humans. They're infinitely scalable. You can hard code things sometimes. So that's the other area probably going to next year that we're looking forward to. That's really cool. And do you view that as a main area of differentiation for you relative to some of the other folks in the market providing customer success and support tooling?

23:57Yeah, right now, that's probably the biggest thing. The interesting thing about our space is that, and I think this will probably be true for a lot of AI agent spaces, is that the results are very quantifiable. You're basically taking the agent and you're benchmarking against, okay, how good would a human be? And how much money is this saving me? How much better quality is the customer experience? And so because of that, when people evaluate us in our space, it's pretty like quantitative evaluation. You're like, OK, cool. This kind of works. Like, let me just put you out into production for 1 % of the volume and build up from there and maybe do that with another option.

24:34Or, you know, a lot of the old school companies like Salesforce are going to be this is a very exciting space for them, too. So they're going to have alternatives. And then you just benchmark everyone, right? Like how good are the stats? How good are the metrics? How good of a job is everyone doing? and I think so far we've been performing very well and the main reason for that is this sort of transparency piece, you know, giving people observability, explainability, control over the AI and there's still a long way to go in that field, right? Like there's still so much more you could do and that's been our specialty so far.