#196 Avi Kedmi: How SysAid Uses AI to Redefine IT Service Management

0:00We measure every piece of answer that is the top 10 that we provide the AI. we give it a distance from like how accurate it is, right? So sometimes you as an admin can see an answer that the machine gave and then below it, it can see five different service records or tickets and two knowledge-based articles that were used in order to provide, to construct that answer. And also what is the power of each one of them? So we fully put the transparency around it. And being as an admin, you are able to understand the accuracy. By popular demand, NetSuite has extended its one-of-a-kind flexible financing program for a few more weeks.

0:45Head to netsuite.com slash IonAI. That's IonAI, E-Y-E-O-N-A-I, all run together. Again, head to netsuite.com slash IonAI. Netsuite.com slash IonAI. for its one-of-a-kind flexible financing program. Hi, I'm Craig Smith, and this is Eye on AI. Today, I speak with Avi Kedmi, the CEO of SysAid. Avi shares insights on the evolution of AI and IT service management, discussing how SysAid leverages AI to transform service management through predictive modeling and generative AI. We explore SysAid's unique approach to automating IT services, reducing ticket volumes, and enhancing customer satisfaction.

1:42Avi also delves into the future of AI in IT and the potential for AI-driven automation. I hope you find the conversation as useful as I did. So, Avi, the way I start is by having you introduce yourself, give your background, how you got to SysAid. And for our listeners who might not be familiar with SysAid, explain the essence of what the company does. 20 plus years in what you call the high-tech industry. and through that I would say one of the few things that I've done that's very relevant to I think the topics we're going to talk today is for about seven years I had a startup that I've opened and eventually was acquired by a live person and that startup main technology that was built is a predictive modeling that was allowing us in 2005, 2008, to provide value to customers that were trying to predict within behaviors of users online towards what they're going to do.

3:04So a bit about my background quickly on that front. And in 2012, I joined LivePerson and run the product teams in what we call the revolution from live chat to messaging, which we all message all day today. But in 2012, it wasn't understood and known for all like it is today. And a lot of the AI related was basically, I would call it a lot of the foundation around NLU and applications around it and what it can do to conversational assistant into users. So, and about a year plus ago, I joined CSAID and CSAID is a ITSM ESM vendor. What is ITSM? ITSIM is basically IT service management. It's not something that there's a saying, not every CEO knows what's their ITSM tool, but every CEO will sign eventually a check for an ITSM tool.

4:14Any company in the world that has more than 30, 40 employees, doesn't matter if they are gold mining or printing T-shirts or a high-tech company. Eventually, they have a system that is a help desk and a few more tools around it that support all of the employees when they connect with IT. So this is what we do. We're a longstanding vendor in this space. It's a highly, highly competitive space. And I joined CSAID. One of the primary reasons was I sensed that early 2000s, the revolution brought in by ServiceNow to the world of service management was around cloud, movement from on-prem to cloud. And now we are in the second revolution, which is the AI and specifically the generative AI.

5:17And I felt like the opportunity here with CSA that had thousands of customers across 142 countries in 42 languages is an opportunity to be a leader in this AI revolution. I'll say a bit more just for a second. and what is CSAID sweet spot? We play mostly on the mid-market, and our value compared to the others is mostly around ease of use, no code, and a lot of things that you can do just by drag and drop and working with the interface compared to others that provide a lot more hand-holding to make things happen. So it's value to money and ease of use as a focus, and last year also a lot around AI.

6:09Yeah. So, IT service management, that's providing systems for employees to interact with the IT department. Is that right? Yeah. And it usually starts by the IT department and then it grows to the HR department and to the finance department. As companies want to be service oriented, they want to make sure everything is tracked and has a ticket associated to it versus just a jungle of emails going around. Yeah. And so you have a product called SysAid Copilot that you position as a comprehensive AI suite for IT service management. Can you talk about the specific AI technologies powering SysAid Copilot?

6:59Yeah. I'll tell a story before that. But so I will say my history with AI AI started with what I explained a bit before, the machine learning. And I still remember it was somewhere around 2006 or 2007, and this was the first time that the machine we've built around predictive modeling started to work after fine tuning and moving it from MATLAB to Java. And that moment where you see the power of AI and the inside of it is usually something that happens for people sometimes once in a lifetime because you understand there's no limit to the machine. You give it thousands of parameters and their values, right?

7:46Linear or nonlinear. You provide outputs over success or not. And then a full model is built and could be refreshed every hour or so to predict a new user with their parameters what would be the likelihood that their outcome will come. And I remember that moment. And the first time I was introduced to CSAID, I felt similar. This captured me because great AI is built with a great set of data and also a great set of actions that you can do with the data. If not, it stays high level just as a question and answers. So I'll say you talked a bit about, you asked about our product, the co-pilot product.

8:37Why is that unique? It's unique because if we talk about the perfect match for generative AI, it is a world of service management because the service management world holds all of the tickets, information, conversations between employees and IT service people. It includes also an IT person at the end summarizing the resolution. There's literally within the ITIL, the certification of how you do service management in IT. There's an area called solution resolution. You write, how did you resolve it? There's also a full world in the system of knowledge base that throughout the years companies build knowledge base around what is allowed to do with your computers or not, etc.

9:28And it includes also everything around workflows and automations and all actions around it. Let's say someone wants a new laptop and who needs to approve and where it's connected to budget, etc. And the last piece, it's all of these ITSM systems also have an install in all laptops across the org. It means it's a fully contained environment for AI with all the data needed to understand the issue and to fix the issue and all of that while connecting to the end user with no human touch. So that's what we've built. What we've built is basically, I'll call it, it's a version one of our platform that takes all of this data in real time, also keeps updating it And whether you are an agent that now is trying to resolve an issue or whether you are an employee that now 6 a.m., you're a new employee and you want to ask a question and IT only opens up at 8.

10:29Or you just want a fast answer. You just talk to this machine and it provides answers and actions for you. So that's what we've built, basically. And the central knowledge base is in the cloud, or how does the cloud play into that? Yeah. So the knowledge base in the world of ITSM, it means the articles that are written by the IT team to help people in some sort of self-service portal to read and find how to resolve their issues or ask for requests. And that exists in the cloud? Yes. And we connect through it because we hold all that knowledge base within our system. Now, sometimes there are clients where some of that knowledge base exists in SharePoint or in other environments, and our system allows you to connect and get access.

11:27And then we pull all of these five or 5 ,000 or 20 ,000 files whether PowerPoint or Word docs or Excel, and we bring them into our platform ready for you when you have questions to get an answer and an action to it. Yeah. I mean, you started this journey long before generative AI. and even the predictive work that you were doing started before the validation of deep learning. So you've seen this transition in this category that people refer to as AI. A co-pilot is using large language models. Does it have a conversational interface for users? Yeah. So if we talk about a vision that is conversational, right?

12:29So the vision is around being able as an employee or as an agent that helps the IT agent to have conversational, not only textual, but also voice, right? And large language models today provides a really unbelievable experience around it. I recall, call it even three or four years ago, in my previous place that I worked, how hard was it to provide an AI that was smart in NLU and being able to have a conversation that is powerful. It had to have a lot of training sets. It had to understand much better the context. and still at the end, it provided an okay-ish experience for users. And there was always this joke, if you take more than three, four minutes, you can break it.

13:29It's easy to find a way to break it, right? And I think the large language model just nailed it completely, right? It's very hard to break them, again, in the context of where we are, not in the context of asking really weird questions, not relevant, but in the context of where we are, it's just very hard. And if you equip them with a large set of data, ready for them in the right way, and again, in actions, they do a very good job. I would say in most cases, they don't miss. So I think that the power of what we're seeing now is applications on top of large language models that are providing, in my experience through many years, an extreme value to businesses.

14:19businesses. Think about it today, if someone using our system at 7 a.m. in the office and the heater is not working, they open a ticket, what's going on? It's freezing here, right? Someone answers them, hey, people are on it, a few hours is going to be fixed. Second person coming 10 minutes after asking the same question, the AI will know to give an answer back and to converse on it and to explain that there's people on it and to understand that there's, by the way, it's going to take a few hours, but that's an estimation. That's not a guarantee. There's a lot today that you could just see about the future coming today.

15:03And conversationally, if we talk about the future, which can be done largely today by large language models, is imagining an agent waking up in the morning and his work is launching small AIs that are just running and doing his job while he strategize, right? Think about them waking up in the morning and going to their interface and clicking a button and saying, you know, ticket 663. I don't understand what's going on with their Microsoft Teams. Reach out to the user, be nice to them. They just came back from a holiday. I know that. And find a way to restart or reinstall, see if it doesn't solve, go back to the Windows machine.

15:42Help me to find a solution and come back to me when you're done with it. And you just ship these small things that are taking actions and they take the time and they connect with users. So I think this large language models are a true revolution where there is a potential on an AI contained environment. Yeah. And are you building on top of a private model that you access through API? Or have you taken up an open source model that you fine tuned? What's happening in the background? Yep. We did a lot of research as we started. And what we've seen, and it's so far, is that the models out there, the LLM that are out there, they are really powerful and they're improving almost on a monthly or quarterly basis.

16:49I know sometimes I'll be provocative a bit. I know sometimes there's some companies that keep saying that they're investing, they're going to build their own models. And I'm not sure it's wrong. I'm just saying that their chance today to bit OpenAI or some other models, it just does not exist. And these models include today a large set of data that is powerful. And also the speed of how they give you back the answer is powerful. So we are fine-tuning where we need to our world for IT. Our prompts are very relevant to what an agent would expect or what an end user would expect. But overall, we have a process here at the company where every month or two, we are checking not only what's out there, but also truly checking the data with existing models like Gemini and others that are new lately.

17:48For now, I would say OpenAI is beating everyone. I don't know if it will be forever like that, but it's clearly something that we're still heavily more on OpenAI. The results and the performance are, for us, solid. Yeah. And then for the knowledge base, can you walk me through onboarding a new customer? And I would imagine that they have to do a certain amount of data prep to get all of the IT manuals and that sort of thing into a format that the LLM can use. Do you help? I mean, first of all, I guess, do you help on that data prep? Yep. So I would say we have a process to make sure that the clients knows what they need to upload and what's the minimum that they need to share with us if there are new clients, a new client.

18:49But I would say the power of what we have today has two elements to it. The first is not like the past. Today, even if it didn't upload anything, there's a lot of knowledge outside with LLMs that from day one you can have. Right? So, and I'll tell you, there's a lot of companies that there is a repeating theme of like Microsoft Teams, connectivity to their earphones and issues around it. Right? So not sure you need to upload a document for it. What you have outside can solve it for you even before that. But I would say when we built this platform, and if I can give it as an advice to others in different fields related to Gen.AI, people tend quickly to just build up something that calls OpenAI or other models and get great answer and be okay with it.

19:44And what I've learned is that the flexibility people expect related to AI is always greater than you would imagine. Because of the risk and because of the power of what AI can do to their business for good and bad. So what we've built is we built a platform that one, allows you to connect any data sources you have today, like SharePoint or other knowledge-based area, where we can fetch the information on our own and take that hustle from you. We also, within this, allow you to add any kind of website or internal information that's out via HTMLs or public websites that all that documentation, all that information is digested within our system.

20:36And, of course, we explain to you what's the best way to summarize tickets in order for the AI to work well with them. So everything around what we call the data pool is a key part where we're allowing you to decide what do you want as a source of information and what do you don't want to be part of it. And it's mostly drag and drop. And like I said, C-Said is a place in the mid-market and the mid-market is all about time to value and ease of use versus a lot of handholding and many partners that needs to write code for you. On the other side, what do we do with that? We take these documents and we've built a process that smartly understand for each type of these documents what to do with it.

21:31So some documents can be policies that are 50 pages, right? And we cut them to pieces and we understand the titles and we summarize each area and we hold them in vector databases, right? Some of them could be Excel's where you need to understand what is this Excel, what's the meaning, and how you store it differently, whether it's through columns or through rows, etc. And also everything around the ticket data, right, that has a lot of sets of information. We're treating it not just like take all the data and summarize. We're treating it of where do we take the data that has the resolution? Where do we take the data that summarize everything around it?

22:10Where do you see that there was code associated with it that you now, we need to separate it in a way that we're going to offer different things around it. So each one of these pieces in our world, there's pre-processing that is happening to support our way to eventually serve you the right answer at the right time. Yeah. Yeah, that's fascinating. And all of that is automated. So the user has an interface and drags and drops documents into the interface, drags and drops Excel sheets, drags and drops URLs. And then in the background, SysAid, your systems process that information depending on the format?

23:06Or is there a human who's sort of managing that? How much of that is automated, I should say? I would say 99 % is automated. just because, again, we play at, you know, we sell to schools where the IT manager is also the one supporting the people, is also the one that is buying the computer. There's like three, four people that do it all in many aspects. And with that, we have to make it automated. So yeah, it's fully automated. And like I said, the product itself, even before AI, controlled a lot of information. So as someone buys the system, the first moment they get the first ticket, that's a new data coming in.

23:57And the second ticket is a new data. So for example, like I'll show an example of how AI is powerful in the world of, let's say, an existing client that already have 5 ,000 tickets and knowledge base and workflows, et cetera. And now when a ticket comes in, we can automatically categorize it to the right category. And how do we do it? We go and look at all of the tickets in your history and we ask, with the help of the LLM for these tickets, how to connect the information to the decision of the user to put it in a category like hardware or software or a mouse, right? And we find through that the understanding of what was driving the decision to call this or to specify a specific category for it.

24:49And we use it when a new ticket is created. So what you see today is this allows now people to no longer click create a ticket and select category, select subcategory, give it a title, give it a summary, understand its urgency. These things take a lot of time. And people, they'll just look at you and me. They're natural. They will just say, I would like a new mouse. And everything in the back does it for them. So if that makes sense. I want to elaborate a bit about the world that we've built, but also some sort of, again, this podcast is also what kind of tips and tricks for people that are building future apps on Gen.AI.

25:34When I was younger, I thought, wow, the AI is doing such a great job. Why is human in the middle? Right? How can it be? And I still remember this person that was a key part of building this algorithm. He said, it's a monster that's doing amazing, but sometimes it can miss something very small that's in front of them. And that lesson taught me that any AI machine you put out there has to come also with the ability to fine tune it. With the ability to say to it, what do you want to make sure that's not going to be okay? Or to fine tune it because you want to say to the machine, you're doing a great job, but can you have shorter answers?

26:29can you make sure they're not too technical can you make sure they're not too technical related to technical aspects can you make sure that you give less links and more given the description or you may say can you make sure you don't talk about our competitors you can just talk to it as a machine and that's a big part of what we've built as part of the platform is the ability to really monitor and fine tune and see everything and And it's amazing. You can see today, the generative AI basically doesn't take an answer and we just tell it, write it better. You provide to it the top 10 answers you believe are the right one.

27:11And you say, look at it and construct an answer. And the power of that construction is where we see the power of the large language models coming to play. Yeah. And that ability to tweak the model or add to its prime prompt or system prompt, do you give that ability to the user or that's internal to SysAid? I mean, each user, are they able to ask the model to give shorter answers, for example, or less technical answers? Or is that done at the SysAid level? So it is on the user level. Any kind of an admin that manages the CSA account within their environment has the power to decide it. It's not something we control, they can decide.

28:18But again, the way we've built it for them is that if five years ago you would say you do regex, right? If someone is asking about security, then don't do that. Then you had to do typos. Security with a typo. Security with that. Secure. And now you can just say to him, anything around security topics. So we've built it in a way that these admins, they speak freely with the machine and they tell her what is a set of rules they believe they want to make sure that the system is not allowing to happen. Yeah. Yeah. How do you ensure the accuracy and relevance of the responses? I know you mentioned vector databases, but in the background, is there anything else that you're doing to ensure, to guard against hallucinations and that sort of thing?

29:23Yep. It's a good question. So I would say a couple of things. The first is we measure every piece of answer that is the top 10 that we provide the AI. We give it a distance from like how accurate it is. Right? So sometimes you as an admin can see an answer that the machine gave, And then below it, it can see five different service records or tickets and two knowledge-based articles that were used in order to provide, to construct that answer. And also, what is the power of each one of them? So we fully put the transparency around it. And as an admin, you are able to understand the accuracy. Now, here come some of the tricks that we're doing, tricks in a good way.

30:19There's a difference. It's all about the experience at the end. Right. OpenAI not only are providing today good answers, but also if you look at their interfaces, they provide an experience that you feel comfortable. Right. And it's not, you can't say the same on all of these LLMs. Right. So we put a lot of focus on the experience. So if you are connected to a chatbot, I expect from you to always get an answer. Sometimes the answer could be, I don't know how to help you. But let's say you have another channel, which is email. Let's say someone opens an email and just write in the subject, hey, I don't know what's going on.

30:57My computer is low. It sucks, right? If you look at that, in this case, the AI will get the score. And if it's not forming internal information, it will not take the risk to give you an answer that is not sure about. So the way we play with accuracy is also cross channels. Some channels we're more relaxed on the accuracy because it's a conversation. We can be more relaxed around it. And some channels, when people get it in front of them in an email, as an example, as the AI take action, we only with higher scores, we would give you an answer. So we count the scores between zero to a hundred and anything that is used as an answer from outside information is automatically counted as lower score, right?

31:49Because we trade off always the inner information of the company. And you said the admin has the ability to sort of train through reinforcement learning, I would guess. is that they can look into the system at any particular time and check answers and maybe choose between different answers or is that something that someone is doing on an ongoing basis or is it sort of spot checking? I think it's more ongoing basis. They need to put a process to themselves once a week to take a look and see that they feel okay. But in the past, systems like that had a very long rule-based, regex-based, right, that at some point it wasn't manageable.

32:59the approach we took is to fine-tune the machine by talking to the machine. To fine-tune the machine by talking to it. Just like some sort of in a relationship, I would say to someone that I care about, I would say, I would like next time for you not to talk about this topic or for you to make sure you bring more facts when you put a perspective, right? So there's no way for you to have a rule base where you see the 10 rules you decided in terms of the fine-tuned guardrails. We have it like that, but it's more you talk to it. You can look at the answer and just write below it, general note to the machine, I'd like you to do ABC next time.

33:44And it will automatically use it as a prompt related to these topics and allow you now to keep seeing the results that you're expecting. That talking to the machine is fascinating. Some things that you weren't able to even dream about them, right? Right. Yeah. No, it really is remarkable. And you said you play in the mid-market. Why is that? Why not go after Fortune 500, Fortune 100? The service management space, according to Gartner, and again, if I'm not mistaking, is now about$10 to$12 billion a year, addressable market growing to$17 billion plus in the coming years. It's a massive market. And I think that everybody needs to find their spot where they're the strongest.

34:41Like I said, it's a world where a lot of competitors play in it. And I think the future will bring CSA to go up also. But for now, we believe that this is a place where we can shine above all. We call it, we're trying to call it AI to all. Because at the end, a lot of these AI that is provided to companies today is served with complexity. Yeah. Right? Served with high maintenance, right? I used to remember going to companies about a year ago when we started this, and I said, this has got to be amazing. I want to talk to you about it and say, is it going to be like RPM that it comes with the three full-time developers forever?

35:28And that plus the cost of the software is less than how much it costs to have agents.