#280 Aytekin Tank: How to Fully Automate Your Customer Service with AI Agents

0:00JotForm is two decades old. We have 30 million users. We are hosting 50 million forms for people. So we are now this large company and we have this support team of 200 employees. And we also have like a large support load. Like we receive over 2 ,000 tickets every day. So we decided to use our own product. And when we actually started using our own product, the resolution rate with AI was around 30%. The vision is having every organization, every small business have their own AI agents. so that when you want to talk to that business in the middle of the night, like 3 a.m., you can talk to that business.

0:38You can get information, buy stuff, and set up appointments, whatever you need to do. All the tools that you could make available to your AI agent could be done by the AI agent. Our sponsor for this episode is Extreme Networks, the company radically improving customer experiences with AI-powered automation for networking. Xtreme is driving the convergence of AI networking and security to transform the way businesses connect and protect their networks to deliver faster performance, stronger security, and a seamless user experience. Visit extremenetworks.com to learn more. That's extremenetworks, all run together, dot com to learn more.

1:32So I started JotForm two decades ago. And the reason I started JotForm was before JotForm, I was working for a media company and we had like over 100 websites. And I was a developer and I was constantly creating these online forms for our editors. And I was creating all these different kinds of forms, like contact forms, surveys, questionnaires, payment forms. And then I decided that when I started my company, forms could be the stuff I could do because I couldn't find a product that could actually take away from me. I had to do it manually. And then I started JotForm. I created this easy-to-use form builder two decades ago.

2:23And we have been a bootstrap company since then. Still, we are bootstrapped, no outside investors. And we have been steadily growing over the last two decades. And how we got into AI, I can talk a little bit about that. A couple of years ago, one of the things we do at JotForm is we have these technical hack weeks. During these hack weeks, our product teams, they try to come up with different ideas, different product ideas, prototypes, or different technologies. and some of our teams like like two years ago some of our teams actually built this this idea that like you can actually fill out forms by voice right and just because like you know the you can use AI to understand and ask questions and and then get the answers and just even during like this hack week they were able to actually implement prototypes so we saw that this was possible so we started working on this product uh but what happened was um while we were building this product like you know uh filling out forms with ai uh with ai agents that uh we discovered that like most of our beta users were actually using it for different purpose they were actually using for customer service they were mostly like they were putting it on their website as like chat but uh because they discovered that um you know in addition to like form uh answering features we all say that these like knowledge base features like rag and just you know uploading all this knowledge or giving your website url so that you know it can learn from it and they discovered that it's just like it can really give good answers provide like customer service answer their questions and like just you know one of our beta users they were doing this like huge um you know uh event like a festival and like they were getting all kinds of questions and all those questions the answers for those questions were actually available on the website but people were actually asking to chat but because people don't read and they you know they're busy they're they don't want to read stuff they just want to get answers quickly and our agent was able to answer questions like you know what's the price of this concert like where can i park my car you know where can i put my tanks things like that like just all kinds of questions and it was answering them really well and and because this was a festival in a short amount of time they they were receiving a huge number of questions and like in a normal settings like even though this is like a you know large festivals like they don't have a lot of employees So if you think about it, it can scale, the AI can scale, you don't have to wait on the line for a long time.

5:23So they were able to get answers. So once we actually looked at what's happening, we pivoted our product and we started calling it a customer service product. And we are calling it Jot from AI agents. We released it two months ago. And, you know, people are using and really getting, you know, their questions answered. And actually, the biggest user in our platform is JotForm. Like, so JotForm is like now two decades old. We have 30 million users. Like we are hosting like, you know, 50 million forms for people. So we are now this large company and we have this support team of 200 employees. And so we also have a large support load.

6:14We receive over 2 ,000 tickets every day, support tickets every day. So we decided to use our own product. And when we actually started using our own product, the resolution rate with AI was around 30%. So we are actually comparing the answers of our human team with the AI team. And we started with 30%. And basically, we actually got like 20 of our support employees started actually reviewing all these AI answers. And then they started kind of like flagging them result or not result. So we decided if it's resolved by comparing to how a support employee would answer that question. And because of that, we are pretty sure that we are really measuring it correctly.

7:08And that in the early days, essentially we launched this two months ago as well, during the launch of JotForm AI agents. And during this two months, we started with 30%. And now we actually reached 70 % resolution rates. and the the ai agent can actually answer those questions and the way we were able to do that was um basically like we had to rewrite our you know rag embedding system or you know the database because we saw that it wasn't handling it right uh like we moved to a product called gina for our rag embedding and uh we basically our product teams actually you know pick different topics like someone is trying to change their email address.

7:52Someone is trying to create a form by asking to AI, all different kinds of questions. So basically they implemented these solutions and then we connected them with API to the AI agents. And basically what makes this powerful is AI, what makes AI agents and like regular chat LLMs, chatbots is what makes them different is AI agents can actually do things as well. They don't just answer questions. They can do things. And in Jotform AI, for example, there are lots of tools that the AI agents can use. They can actually do appointments. They can connect to your Google Calendar, Calendly to be able to take appointments by voice or on a chatbot.

8:42So they can make calls to an API to be able to answer questions. In the case of changing an email address, they are actually making an API call to authenticate the user to be able to make sure that this is the real user that the AI agent is talking to. And also taking their new email address. Maybe if there's a verification email sent to the old email address, then they have to enter some numbers as well. But this is all done conversationally. And the AI agent can actually use these tools. so uh because because of all these tools um we were able to go from 30 to 70 percent i would say half of the improvements came from improving our react system improving our knowledge base like we had to write tons of new user guides because basically the our ai agent is using our knowledge uh our user guides and we discovered that our user guide was missing like so much and And then once we added all these missing pieces, the AI agent was able to answer much better.

9:47But half of that came from that knowledge. But half of some actually came from implementing all those tools the AI agent uses to be able to resolve the problems of the user. And so that's really good. Getting to 70%, it wasn't easy, but it was possible. So we envision that, you know, we are going to be able to go to 80 % as well. Like we can see that we will be, but after 80%, it's just going to get so much harder and harder. But it's okay because it's always like, so one of the things that the AI agents can do is they can actually like, you know, transfer the chat to a human agent. and if we can handle 80 % and if you have to transfer 20%, you know, we are fine with that.

10:36That's still better. And another thing we discovered was like, so we implemented this system separate from our support ticket system. And our support ticket system was, we were getting like over 2000 tickets every day. And that actually went down to like, went down like 25 % to 1500 a day. and it's even going further down now. But it didn't completely replace that. So people, what we discovered was some people, because we are also showing our average response time on our support system that says like, our average response time was one and a half hours when we started this. But because now the AI agents are helping, our actual average response time went down to 30 minutes now, like just much faster.

11:26But even though it's 30 minutes, like many people, they don't want to wait 30 minutes for an answer. They either want to get like an instant answer or they don't want to just bother with support at all. So that's another thing. Like we are actually supporting more people now. On average, like we are supporting every day. In the past, we were responding to like 2 ,000 queries, 2 ,000 support tickets. But now we are having like 3 ,500, you know, the support, you know, threats. And, you know, about 2 ,000 is actually coming from the AI. And then 1 ,500 is on our human support. So we are actually, you know, providing more support to people.

12:12But, you know, we are able to provide a better support thanks to the AI agents. Yeah. And it's just so much fun to be able to use our own product. Yeah. Let me go back to the original AI application was in using voice to fill out forms. Yeah. That's actually how we started this project two years ago. You know, we thought that, hey, you know, forms are boring. Maybe you want to be able to fill out a form on the phone or over voice on a website. or maybe just want to talk to a chatbot as opposed to just filling out this boring form. And that's how we started. And then when we released that, only 10 % of the beta users were actually using it for that.

13:01And 90 % of our beta users were actually using it for a completely different purpose. We never thought that that would be the main use case for this product. But how does that work? I mean, does the AI ask, like, what's your first name? What's your last name? Does it track the form and ask questions? Or are you looking at the form as the user and telling it first name Craig, last name Smith, and it finds the box? I mean, how much instruction do you need to give the AI? This is the landing page, and I can create an agent here. And I will create this voice agent. And then I want to create, let's say, I want to create an appointment form, form appointments agent for a personal trainer personal trainer

14:17and it's gonna create the agent in the backhand basically it's doing some research to be able to create a good one so that you don't have to like play with it a lot to be able to have something useful

14:40So now we can see that I have this Alex appointment scheduler. So this is the AI agent builder. This is kind of a place where I can actually play with settings like, you know, the voices. JotForm AI agents are powerful tools that handle... JotForm AI... Yep. Okay, good. So I can pick a voice here. JotForm AI agents are powerful tools that handle... JotForm AI agents are powerful tools that handle tasks like answering questions. You know, I can play with it. But so I can actually train my knowledge base at information here. And it actually also created a form, probably created a form for me as well.

15:22So here we are. We can actually see the form here and I can take a look at the form. I'll have to share this tab so it actually created this form because I said personal training appointment form appointment agent it created this form on JotForms form builder this is JotForms form builder we are on now and it created this form and I can take a look at I could change things on this but I feel like this is pretty good so I will keep this form so now I'm going back to the agent agent builder. And just, you know, I'm ready to test it actually. Yep. We could just test it here or I could just open it by itself here.

16:10Okay. Let's try it.

16:16Hello, I am Alex, your AI agent for scheduling personal training sessions. How may I assist you today? Hi, Alex. I would like to set up an appointment.

16:29That sounds like a great plan. I'm here with you and ready to help get everything set up smoothly. To proceed with scheduling your appointment, may I have your full name please? Aytekin Tank.

16:43Thanks for sharing your name Aytekin. We're making good progress here. Just a bit more to go. To continue, could you please share your email address by spelling it out letter by letter to ensure accuracy? A-Y-T-E-K-I-N at J-O-T-F-O-R-M dot C-O-M.

17:07Got it. Thanks for sharing your email so quickly, I tick in. We're almost there. Just a little more to finalize your appointment. To make sure I have it right, could you please spell out your email address letter by letter? For example, is it a E-T-E-K-I-N at J0TF0RM.com? It's A-Y-T-E-K-I-N at JotForm.com.

17:38Thanks for confirming that so clearly, ITEK-IN. We're getting everything lined up nicely for you.

17:47To ensure I have it right, could you please spell out your email address for me letter by letter? A-Y-T-E-K-I-N at J-O-T-F-O-R-M dot com. Dotform.com. Is that correct? Yes.

18:10Got it, Itekin. Thanks for sharing that detail so clearly. Everything's shaping up well for your appointment. To proceed, could you please share your phone number? It's 415-555-5555.

18:31Thanks for sharing your phone number so smoothly, Itakin. We're getting everything lined up nicely for you. To confirm, is your phone number 415-55555? Yes.

18:47Perfect. Thanks for confirming that for us, Itakin. We're almost ready to move forward with your appointment. To proceed, could you please let me know a convenient date and time for your appointment? Tomorrow, 9 a.m. Got it. Tomorrow at 9 a.m. sounds like a great time. We're getting everything set up just right for you. To confirm, the appointment is set for April 23rd at 9 o 'clock in the morning, Eastern Daylight Time. Do you confirm this appointment? Correct.

19:19Your options are, yes, I confirm. Yes, I confirm.

19:38And yeah, so on. I think it might cut. Yeah, sometimes it gets stuck.

19:50But let me also show you something else. Did it fill in the form then? Take a look. Let's take a look. Yes, it did. Let's take a look at the conversation here. So we can see here it actually got my name correctly, which was actually hard like we had to kind of uh train my name on it uh yeah just because it's nothing uh american name but uh you know it got my name uh my email address uh phone number and the appointment correctly and um let's see and the conversation is actually here right yeah and uh so one of the things that like i was talking about the tools so basically what i could do is like if i'm a personal trainer and if i'm using uh like google calendar to keep track of my um you know my appointments i could actually connect to this and I'm ready.

21:00I'm ready to use this product right away. And just, you know, I could use, you know, one of the options, you know, like I could get a phone number, buy a phone number. Like it's just$10 a month. I could, if I have a website, I could put a chat bot on my website. You know, I could use text, you know, just get a texting number or like a whatsapp connect to my whatsapp account um and um be able to you know take a purpose is this uh what models are you building this on yeah i mean um so that's that's the great thing about uh building building products on top of ai because um like usually of like uh and i've been building a product for like you know two decades um and there's there's this there's this uh thing called software entropy and it's just like you know software actually gets worse over time like just stuff stops working like you know you get stuff uh broken like just uh out of blue for some reason like it shouldn't happen right but uh with ai it's actually like the uh the ground uh below you is actually getting better um so like we have been you know working on this for uh two years now and in addition to all the like uh stuff we have done um like one of the biggest uh improvements actually came from the um the lms the you know uh chat bus we are using um um so right now uh we just switched from like 4.0 to um 4.0 to 4.1 and even that uh made a big difference um even our support agent had like a like a couple of point improvements um just by moving to a new agent and so we are basically using two two different um APIs.

23:09One of them is OpenAI 4.1 and the other one is Gemini 2.0. And we are also moving to 2.5 on Gemini as well. Like we are just testing that out and we see that that's also going to improve things. So we also give people options to use like, you know, which model to use, but we actually prefer to use multiple models because this actually gives us an ability like we have dashboards like we can actually compare the performance of different models to each other so that like you know we will usually run like 50 of the conversations like we have about 10 000 uh users right now but our biggest user is ourselves like uh job forms own support and uh so you know we are actually comparing the performance of different models that's why we always run like two of them together and if if we see and sometimes it's just our prompts like we have to write different prompts for different models like you cannot just write prompts for chp like open ai 4.1 and then use that for gemini like we gemini we have to change things uh sometimes like we have to improve our prompts like sometimes the default is not with the llm it's actually our fault but uh because we are always uh like comparing them to each other like we can discover that one model is actually performing much better.

24:37And then if you see something like that, we will actually move most of the conversations, like 90 % of conversations to some model. But right now, the performance of Gemini and the OpenAI, CHPT, is very similar. They are very good compared to the older versions. So we are keeping it 50%. Yeah. Yeah. Another question. the knowledge base that you're attached to. Can you set up your knowledge base on the platform, or is that something you need to contract JotForm to do in a bespoke way? I mean, if I wanted to build a chatbot for my podcast and load all my podcast episodes into it, is that something I could do on JotForm or would I have to contact you guys and have you build a RAG system?

25:45It's all SaaS service. You just go there. You add your own knowledge base. You can upload your files. You can link to your website. You can link to your YouTube channel to be able to get the transcripts and use that as a knowledge base from your podcast. So basically, it's just it's all SaaS service. It's all free in, you know, up to up to 100 conversations per month. It's all free. And then if you get over 100 conversations per month, it's still like, you know, very affordable prices. But it's also SaaS service. So you can build your knowledge base. base. And one of the things we did was actually, we actually made it so that you can actually talk to your own AI agent to be able to train it.

26:42So if you think about it, let's say you hired a support employee, would you just dump all the documents on the support employee and say, okay, learn this and go talk to the customers? Or would you actually talk to that person, like train that person, talk about your culture, how you talk to customers, what's important for you, things like that. You would probably talk to this person, right? The same way you can actually teach your agent. So we have a section called teacher agent. You can have conversations with your agents. You can train your agent. In the training section, I was talking about the knowledge base.

27:18You can upload files here. You can enter knowledge. You can link to your website, upload files. On uploading to the knowledge base, is there a limit as to how much you can upload, how many pages?

27:37There's actually a limit. It's available in our pricing. I don't remember exactly, but we give a lot of space for our users. And what happens is that if you add like small amounts of knowledge base, you actually don't need to use the rack, right? So I believe it's about, you know, I can't remember the exact number, so I don't want to give the wrong number. But it's until some point, like, you know, your knowledge is actually basically given with the prompt. But if you actually load lots of data, lots of knowledge to your knowledge base, then what's going to happen is that we are going to automatically actually switch your data to the RAC, the vector database to be able to query the questions correctly.

28:32Because the context window for the LLMs, like the AI, is limited. So we don't want to give everything about... With Jotron support, we have hundreds and hundreds of user guides. And you don't want to look up everything when you're asking a question about some specific questions. So, yeah. And it's all happening. You just enter your data and then we take care of all those things for you. Yeah, great. And yeah, this is the teacher agent section. Basically, good evening. It's wonderful to connect with you. Imagine if you're sharing a cozy cup of tea as we chat. Feel free to tell me what's going on in your mind.

29:19So I could teach my agent, can you change your name to John? So its name is Alex.

29:36Okay. So basically it says that it can change its name. And right now it's Alex. So updated name is John. Yes, proceed. My name has been changed to John. And so as we can see, it's actually here. We can see that the name of the agent is now different. If you go here, it's now John. And so I can, you know, I can, you know, train it as well. Like, you know, I don't work on Tuesdays because it's taking appointment. I don't work on Tuesdays. Right. let's say if it will be able to take care of that scheduling purposes hopefully um it's not gonna do that oh obviously i will have to test it to see if it really does that change but um we could also check our knowledge base to see if added this to our knowledge base somewhere pin a maybe i don't see it here but maybe i will just test it here um uh let's see