#295 Fergal Reid: Why Your Bots Fail and How Agents Fix Your Customer Support

19 Oct 2025 · 44 min

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Eye On A.I. Podcast Episode #295 Summary

Episode Title

295 Fergal Reid: Why Your Bots Fail and How Agents Fix Your Customer Support

Episode Description

In this episode, host Craig S. Smith engages with Fergal Reid, Chief AI Officer at Intercom, to discuss the failures of traditional chatbots and the effectiveness of AI agents in enhancing customer support. The discussion emphasizes the transition from scripted bots to advanced AI agents capable of solving real customer problems across various channels.

Key Takeaways

  • Chatbot Failures: Many chatbots fail to provide meaningful customer support due to their limitations and reliance on scripted responses.
  • AI Agents: Advanced AI agents, like Intercom's Finn, can effectively handle complex customer interactions and provide resolutions in real-time.
  • Configuration and Customization: Finn is designed to be highly configurable, allowing businesses to adapt it to their specific customer service needs without needing deep technical knowledge.

Key Concepts Discussed

  1. Transition from Traditional Chatbots to AI Agents
  2. Challenges with Traditional Bots: Bots often fail to deliver satisfactory customer service due to limited functionality and inability to engage in meaningful interactions.
  3. Role of AI Agents: AI agents can handle more complex queries and perform tasks such as issuing refunds, thereby reducing reliance on human customer support.
  1. Intercom’s AI Agent - Finn
  2. Product Overview: Finn is a sophisticated AI agent that can operate across multiple channels, including text and voice.
  3. Capabilities: Finn is not limited to answering questions; it can execute tasks like processing refunds and returns using secure procedures.
  4. Market Performance: Intercom reports over $50 million in annual recurring revenue with thousands of businesses utilizing Finn.
  1. Metrics for Success
  2. Key Performance Indicators:
  3. Resolution Rate: The percentage of interactions successfully resolved by Finn is currently around 65%, showing significant improvement over time.
  4. Customer Satisfaction (CSAT): Monitoring customer satisfaction to ensure that the service meets user expectations.
  1. Customization and Configuration
  2. Highly Configurable: Finn allows businesses to customize its behavior and tone without needing to delve into the underlying machine learning complexities.
  3. Self-Service Model: Users can set up Finn with minimal intervention, though enterprise customers may require additional support.
  1. Market Dynamics and Competition
  2. Growing Market: The demand for AI-driven customer support solutions is expanding rapidly, driven by the need for efficient and reliable service.
  3. Adoption Barriers: Companies often face hurdles in adopting new technology due to existing contracts with service providers and regulatory concerns.
  4. Competitive Landscape: Intercom positions itself as a leader by focusing on quality, reliability, and a transparent pricing model.
  1. Future Aspirations
  2. Expansion Beyond Customer Service: Intercom is exploring the potential to apply its AI capabilities to other business processes beyond customer service.
  3. Continuous Improvement: The company emphasizes ongoing development to increase Finn's capabilities and reliability.

Conclusion

This episode sheds light on the evolving landscape of AI in customer support, illustrating how advanced AI agents like Finn are revolutionizing the way businesses interact with their customers. The emphasis on customization, performance metrics, and the ability to handle complex tasks positions Intercom as a significant player in the AI-driven customer service market.

For More Information

  • Intercom Website: [fin.ai](https://fin.ai)
  • Follow Craig Smith on X: [@craigss](https://x.com/craigss)
  • Follow Eye on A.I. on X: [@EyeOn_AI](https://x.com/EyeOn_AI)

Sponsorship

This episode is sponsored by AGNTCY, a project focused on developing a collaborative layer for AI agents. For more information, visit [agntcy.org](https://agntcy.org).

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:00With a lot of AI, while we've built this very deep product, we've made it customizable, and then customers will start to stretch it because it can just do a lot, it can deliver a lot of value. We don't really want our customers, you know, writing individual prompts and trying to get down into the machine learning because, you know, we're able to go and test and optimize fin across thousands of businesses and that just helps us get better quality than you can get by targeting anyone. Build the future of multi-agent software with agency. That's A-G-N-T-C-Y. Now an open source Linux Foundation project, Agency is building the Internet of Agents, a collaborative layer where AI agents can discover, connect, and work across any framework.

0:52All the pieces engineers need to deploy multi-agent systems now belong to everyone who builds on Agency, including robust identity and access management that ensures every agent is authenticated and trusted before interacting. Agency also provides open, standardized tools for agent discovery, seamless protocols for agent-to-agent communication, and modular components for scalable workflows. Collaborate with developers from Cisco, Dell Technologies, Google Cloud, Oracle, Red Hat, and more than 75 other supporting companies to build next generation AI infrastructure together. Agency is dropping code, specs and services, no strings attached.

1:51Visit agency.org to contribute. That's A-G-N-T-C-Y dot O-R-G. So, yeah, so my name is Spurgle Reid. I'm Chief AI Officer here at Intercom. I've been in Intercom about eight years. I have a technical background, a PhD in sort of machine learning applied network analysis, degrees in computer science and maths, and been kind of working in AI and machine learning for a long time now. And I joined Intercom about eight years ago to kind of start building really the machine learning function in Intercom. So, yeah, that's me. Okay, and we're talking about agents today and specifically Intercom's agent.

2:33Is it an agent platform or is it an agent? Yeah, there's sort of a blurry line between those two things. We would describe Fin as more of an agent. However, it's a highly configurable agent. You can really adapt it to do a lot of things for your business. We're still a little skeptical about very pure agent platforms. and they tend to not deliver the quality that's needed today to accomplish any one task end to end so we've kind of been a bit more opinionated than that we've built an agent product and which is fin but we've done a lot of work to make it very highly configurable and customizable but we don't really want our customers you know writing individual prompts and trying to get down into machine learning because you know we're able to go and test and optimize fin across thousands of businesses and that just helps us get better quality than you get by targeting anyone so yeah and intercom what was intercom doing before it launched finn intercom is um traditionally a sas company it's been around you know 12 13 years something like that and really launched the business messenger way back when it's one of these messengers that appeared in the little corner of a website you could talk to people intercom was one of the first if not the first uh to build a product like that which then kind of went everywhere and intercom kind of evolved from that into a customer support platform so you know people at the other end of that messenger customer support agents answering customer support questions at scale giving really good customer service indicom's mission traditionally was you know make internet business personal really really kind of give great product experiences on the internet like you might get if you had previously you know you went into a coffee shop and you had that highly personal service so that's kind of what we did traditionally and we're sort of an early adopter of ai we had a previous generation product before finn resolution bot we put a lot of work into to answer customer questions.

4:39And we're really well positioned when the current wave of AI came to sort of take advantage of it. And you're squarely in the customer service space. Is that right? That's where we are today. I think our future aspirations are broader than that. But today, definitely the product we have, Finn, is targeted at doing customer service and does a really great job of it. Now, customers do stretch that. Finn can do a lot. It's a very powerful product. It do see customers stretching it where they're sort of you know they buy fin primarily for customer service but then they start doing things like you know upselling or customer success so you know it's definitely an area in which our product is is being stretched and being pulled broader by customers we have some customers who use fin today and they use it entirely in a pre-sales way so you know i think with a lot of ai while we've built this very deep product target of that customer service we have made it customizable and then customers will start to stretch it because it can just do a lot it can deliver a lot of value for them so yeah that's roughly where and and the agentic uh capabilities what what can finn do and is it is the interface through chat or or uh voice i mean it's text or voice or uh what yes it's it's it's a pretty full feature product at this point We have over$50 million of annual recurring revenue, and it's used by thousands of businesses.

6:12We've got more customers than any of the other sophisticated customer service agents. And so the product, we've really built out the product. We've been doing this for over two years at this point. And so we have a very full offering. We built just FinFord the Messenger initially. Then it would run across things like WhatsApp and all the different channels. but then we built fin for email we had to build like a different architecture there to really deliver very high quality performance so we're kind of doing that 18 months ago over the last year we've built a cutting edge version of fin that uses voice uses like the latest sequence to sequence voice models that's now running at scale in production there are loads of customers now loads of end users who are sort of like talking to fin over a voice interface and it's magical it feels amazing to you know be hear a phone conversation where someone is is talking to fin and it's just kind of answering their questions and and so yeah the product is getting getting pretty broad and it's getting a lot of capability yeah and as i said the the agentic capabilities uh i mean fin can you give us a use case where where it's uh it's not simply uh question and answer it's it's uh act of doing something yeah i mean it it does a lot of things and and you know obviously i would say like i wouldn't say just question and answering answering questions is a huge part of customer service and you know we've invested a lot in building the best kind of question answering piece and there's a big difference between aging capabilities in that space but yeah finn can do lots more than that So we have customers who do things like, you know, if they have an issue, a refund, they have it handle returns.

7:59And, you know, we've built a very rich and deep product to kind of enable it to do that, to kind of, you can hook it up to your external systems, you can hook it up to your APIs, you can sort of give it the parameters around that. we've built a very rich you know uh almost staff style you know product to enable you to specify and to tread together the complexity of the procedures that your business needs in order to do something like actually issue a refund every time we talk to a customer they're like oh i know my refund policy it's really simple and like two weeks later they actually finally get them to write down the refund policy on paper and it's this big sprawling thing with all these edge cases because we've had to build a functionality to do that and to enable people to use llms in those like real production use cases so yeah so it's it's pretty and and on on uh refunds because that's a pretty complicated use case Finn actually then pushes the button to release money or wire money or yeah it absolutely does we absolutely have customers who have deployed this in like consequential areas in areas where if it gets it wrong their business will lose money, their end user will be extremely disappointed.

9:31We absolutely have customers who are using it in those ways. And, you know, it's a lot of work to do that. Like, it's a whole different ballgame when you start trying to do things like that. You have to convince the security team that, you know, what if a customer tries to jailbreak this? Or what if a customer tries to refund or return a different customer's product? You have a whole different level of complexity that you have to deal with there. You have to have like an API you can call. You have to like set up the API to work with Finn. And we've done a lot of work to try and make that easy.

10:04And we have like, I don't know, at least five years, maybe a lot longer of investment in this space. You know, we were building low code API integrations for customer service bots back before the current wave of LLMs. And what we've done now is we have integrated the next generation AI LLM technology with all that sort of like battle tested, secure and easy to set up interfaces of before. And of course, we support MCP. When we saw MCP coming, we were like, wow, this is amazing. This is a potential huge unlock. And so I think we're in the first wave of people to like really support MCP. yeah yeah and that's uh model context protocol that allows uh models to talk to each other or agents to talk to models is that right the um uh in in the customer do you focus on a particular vertical or is this any customer service in any industry No, we don't focus on a particular vertical.

11:11And we have lots of verticals that the product's very successful in. It's very successful in financial services. It's very successful in software as a service. And areas where people want to give really great customer service, where they're willing to pay for really great customer service, where it's worth it, where they want to use a very high quality product and fin does really well on but we don't constrain ourselves to a particular vertical and we kind of think it's probably a mistake to do that and you know the reason why is that building a product that's horizontal like this like there's so much commonality across the different use cases the ai layer is it you know you can invest deeply in the ai there we've done things we've built custom models we've done a great deal of testing and optimization And so we really have this product that's like unified.

12:03And then we have thousands of customers running it in production and we're always improving it based on how well it does for those customers. And so like our product, our standardized product really has outperformed vertical specific competitors who have a much smaller volume. We really think that there are just really deep scaling and experimental laws in AI. You really want a standardized product envelope, a standardized product core that can be optimized and tested en masse across a very large number of customers, but then deeply customized for the individual business. And that's really what we've built.

12:43So we're not in a world where we're telling customers to go and prompt the agent. Instead, we've built a core that's standardized and that we have continually made better over time. but then with a customization layer, it can adapt it to a specific business. Yeah, and most of the customer service solutions that I've talked to work in tandem with a call center or a BPO. Does Finn do that as well? Actually, I mean, you can deploy it, and we have customers that deploy it in every configuration. You know, it's a very mature product on a very mature platform. We've been in this space for a long time.

13:30But I would say Fin is more likely to kind of compete with BPOs. Like we typically see, you know, the typical pattern is a customer of ours will deploy it and they will often end a contract they had with a business process outsourcer that used to do maybe their sort of tier one or their frontline support. you know even a year ago and the product was much less mature a year ago even a year ago we were talking to customers who were like yes i've deployed fin and i have ended my relationship with my bpo i still have my my technical support team i still have my own internal support team but the bpo to which i was outsourcing sort of the simple queries um it's just gone and finn entirely does that and you know more reliably and more cost effectively and that was even a year year and a half ago and fin's really really pushed up the quality level since then and a lot of people's internal tier one support as well is now doing something else and so what happens when a customer hits a wall that fin can't resolve is it uh routed to to your internal customer support yeah exactly so you know i i should say to to the customer's internal customer support yeah To the customers, internal customers' word, absolutely.

14:49Yeah. Again, we've been at this a long time. And so probably like eight years ago, we were building Resolution Bot with the previous generation tech. And while like the core engine is completely different and the modern AI is next generation and works much better, we were solving all those problems around escalation and how do you gracefully hand off to a human? And so we've great solutions for that. They're configurable. and we have like almost a decade worth of technology investment in there but again we completely rebuilt we built fin completely from scratch we were out the door on gpt4 launch day with our new gpt4 powered product and and so we you know it's it's an entirely new architecture i think we're on generation four internally of it now but uh but it's connected to all those old battle tested systems that we've had for a very long time and so escalation yeah we've had great escalation handover to humans for a long time very recently we shipped a custom ai model trained ourselves that's great at detecting exactly when to escalate exactly when to to hard escalate when not escalation when to offer an escalation to the end user so we've even built like custom ai technology in there to get particularly good at that that's a problem that we would be carried and i'm curious is is there some metric about uh what percentage of calls end up being escalated uh and is it is lower with fin than with other solutions or yeah yeah so so we have um our core metric that we care the most about we call resolution rate And it's essentially the percentage of times when Finn is involved in a conversation that successfully resolves the conversation versus when, you know, you have to escalate to a human.

16:43And that for us is about 65, 66 % at the moment. And that's sort of our North Star metric. We care deeply about it. It was about 35 % when we launched Finn. And we have this like team, my team, it's about 50 people in it. We care deeply about constantly moving that metric up over time. We're constantly doing A-B tests. We're constantly taking any new change to the AI of Fin, running it at scale in production, hundreds of thousands of users in each version, and then checking to see, does it increase the resolution rate? And over time, we've managed to, through that scientific process, get that resolution rate up and up and up and up over time.

17:21So it's now around 65%, still working on getting it up more and more. And we're incredibly proud of that. think that's best in class and anytime a customer kind of runs a head-to-head between us between one of our competitors a customer always tells us our resolution rates higher they choose us and occasionally we even um test specific competitors and we've always sort of won those tests and so yeah we're really proud of that and and like everybody says this and we just always tell our customers please just do an a b test do a trial and see how you get on and i i think you I think we're gathering momentum at the moment in the market as legitimately being the one with the highest resolution.

18:04And when you say it's highly configurable, what do you mean by that? And when you onboard a new customer in an industry that maybe you're not as prevalent in, what kind of data do they have to provide to optimize the solution? Yeah, so I guess, you know, like all these questions, the nuanced questions. So the first thing I will say is, you know, Intercom traditionally, we believe a lot in like self-serve. We really like to build products rather than services. You know, with a product, you can maintain it yourself. You can change it yourself. As we make it better, it gets better for all our customers.

18:49A lot of other people are attacking this via a services approach. they're you know putting engineers in and custom engineering it and then the product doesn't get better over time which isn't great so uh you know look we really kind of built this in a productized way and we believe in productizing it for everybody and then increasing the quality over time for everybody and so you know initially on day one of finn you could set it live and we had customers that set it live with no intervention no onboarding no information from intercom at all They would just go, they would configure Fin in our easy to use product, and it would be live and it would do a certain resolution rate for them.

19:28And, you know, over time, we've had loads of customers that have done that. However, we have learned as we have gone up market and as the product has matured, that there's an element of business transformation to this that our customers need. So now we have sort of built the muscle really over the last sort of six months, the last nine months. We've really built the muscle that we have to have teams of people. So we have a R &D services org. We've also built a professional services org whose job it is to help customers get successful. So if you're an enterprise customer, you're an upper mid-market customer, you need a lot of help.

20:05You maybe need to negotiate and talk to your security team through these deeper procedures. maybe you have a voice deployment you need to get that voice deployment live it's on very big scale you need that help you need that services team so we've kind of we've been forced to build that muscle probably about six or nine months ago we didn't have a great muscle nine nine twelve months ago we didn't have a great muscle for that some of our competitors were starting to do it some of our competitors were starting to make headway at us because they were going and kind of by hand in an unsustainable way, probably.

20:40They were getting their core teams to build a product for each customer. We thought that was a bad idea, but we needed to compete with it. So we built this kind of services org. We really upped their game there for those large enterprise customers. So yeah, so today we have customers that turn along. They get like 50, 60 % resolution rate out of the box. It's transformational for them. We also have MotionNav for the big, really big enterprises, where it's like, no, we can go deep, got a services team and do all that transformation with them. You know, the business process change that is hard and complicated and it needs to be gone through.

21:19Yeah, and when you go deep, as you say, are you fine-tuning the LLM behind the agent or is it putting rules in place around the agent? So we do not fine tune the LLMs on a per customer basis. We're suspicious of people who say they do that because you need a very large volume of data before that makes sense. Maybe the biggest customers in the world. that might make sense. But even then, we're much more in favor of fine tuning LLMs that will work for customer service generally, because then you have this huge amount of data from many, many different customers. And in our experience, that tends to outperform training for any one customer.

22:18So that's really the way we go. So when we talk about configuring deeply for an enterprise customer, we've built this product-based configuration layer on top of BIN above the level of the llms so you know essentially it turns into prompts for the llms but managed prompts you know we don't believe that customers really benefit from having the ability to rawly prompt a large complicated system like bin which has maybe 10 or 15 different prompts underneath the hood instead we have learned the control that they need for their business and we've given them easy to use windows to get that control so to change finn's behavior to change its tone of voice make it compatible with their brand but without them kind of messing around accidentally in the core llms so it it's quite configurable it's quite customizable you can change your order it does things you kind of can change the policy a lot sort of the mechanism you know a lot of ai products they accidentally mix up customer control over like the policy with customer control over the mechanism and then you end up with all these failed deployments or mediocre performance we don't do that we kind of have a standardized mechanistic layer and then like configuration and policy layer that we have done the hard work of productizing that we expose and that way customers they they get the improved performance of that kind of core engine getting better over time they get that while maintaining the configuration layer they need sometimes there's a trade-off there but you know the hard work of building a product is to kind of give customers the configuration options that they need and i think we've done that we've been doing that for about a year and a half of finn's life cycle i think we've we've really done that they can give a guidance without accidentally breaking you know uh you know this is a crowded space as i'm sure you're you're aware more than me uh how do you is the market so large that it doesn't matter uh there there's there's always another potential customer uh if you if you you know lose one but or is it um i mean what strikes me about this space is and i've talked to a lot of companies in the space is uh i i still uh i don't i don't think yet in my personal life i've run into one of these uh generative ai agentic customer service solutions when i'm dealing with companies I'm still going through the phone menus.

25:19I'm still getting the canned answers from chatbots. So I guess the question is, it appears to me that it's kind of an endless market at the moment. How do you deal with competition? And why is it taking such a long time to penetrate? Is it just companies have bought a solution and they're going to amortize it out before they switch? Yeah, I think an interesting question with a couple of different pieces to it. So firstly, I think the market here is huge. Like the total addressable market here is like absolutely massive, right? So many people doing these kind of rote customer service tasks, it's going to go away.

26:10And like all human customer service won't go away, but all the road stuff will. And even a lot of the stuff that's not road, a lot of the stuff where it's like asking a question that can be answered from any sort of knowledge or documented knowledge we think is going to go away. And so we think there's a massive market here. In terms of competition, we legitimately believe that we have the best agent and we have a lot of data to back that up. And, you know, we're, I think we're starting to win in the market. I don't know anyone else that has, I need to pull the exact stats. We definitely have over 5 ,000 customers, 50 million, over 50 million in ARR.

26:51We're growing at about like, you know, we're on trajectory for a hundred million and two quarters. We're growing about 4X year on year. Like, so, you know, there's an adoption curve, but we're growing very, very well. And so, you know, feeling pretty good, there's always an adoption curve, right? If a business wants to buy something, it's not a consumer product. There's always an adoption curve. There's always, you know, stakeholder management, the existing solution. There's all these concerns, you know, is it good enough? Can I adopt it? And we've been going through that, you know, for almost two years at this point.

27:28And it takes time. But we're seeing that very rapid growth. and you know really the market is turning and so we're feeling pretty good and then in terms of like competition yeah we have competition that's growing really fast as well and again huge market really expanding so good luck to them and but we are setting out to win and um and we think we we're really quite sure we've better technology we've better products we've a very big investment here and so um yeah that's that that's all compete and see who can give the best product and answer the most end user customer support questions that's how we bill we build a dollar per resolution we resolve the question we get a dollar if the end user is not happy and they talk to the human we get no dollar so that's sort of like putting our money where our mouth is in terms of our belief in having the best product and making it work and there's a lot of vaporware out there there's a lot of big claims out there and our retention numbers are really good and then let's see how the the market plays out.

28:30But yeah, we want to build the best, really good product for our end users, and we're going to make a good go at it. Yeah. Do you think on the adoption curve, we're still just ramping up? I mean, I know you're not in sales, but how long does it take a company to make a decision to adopt Fin, for example? So, I mean, this market is very structured, right? You have the top tech companies. We have some great customers. Tropic is an amazing customer of ours, and obviously really AI forward company. Other companies like Amplitude, Synthesia, like really, really good kind of tech companies. They make decisions quickly.

29:21They run, they're very sophisticated, they're very technical. They can run tests. They can really assess the quality of a product. On the other hand, we have customers who are, you know, more like a utility or a financial services company. And then they insurance customers like, you know, they will have more diligence to do. They care a lot about like, is the regulator OK with this? Is this going to get me into trouble? Can you really be sure this won't hallucinate in production in a way that will cause me trouble? and so you know there's an adoption curve and different people in different industries are in different curves so there's no one answer i can talk about the the growth of the product overall and i think the growth of the space overall to you know the whole space is growing um and so uh so yeah i think it's maturing all the time i did it is fast it's a very fast growing space for b2b but b2b takes time you know these are not consumer yeah and and do you think that adoption will accelerate uh as as companies realize they're going to get left behind if they don't upgrade their customer service uh because their competitors at their level i mean again the the the most honest and clear and data-driven way for me to answer that is to say like fin has had double digit uh month on month growth for at least 18 months right and you know certainly maybe maybe maybe one month just a little bit seasonality effects but on average trading three months double digit month on month growth and for at least 18 months and so like that is an acceleration adoption curve that is the kind of curve you see when a market is maturing and when the product has got like good product market fit and so um yeah i mean i i think i think that's the best evidence i have of an accelerating market adoption and you know there's also the things with this viral dynamics people people do encounter it out there in the world and then they look and think wow this is great i want this for my business and so you know I think we're sort of seeing that.

31:43But we're still early here. This is a huge, huge, huge market, and it takes time to penetrate through it. And, you know, like voice, we have built, our voice product is a next-generation voice product. It's built with sequence-to-sequence models. We have competitors, even very new competitors who kind of build the voice with, like, text-to-speech and speech-to-text. It's kind of janky. We've gone, like, for the newest models. It's really, really great. you know we couldn't do that we couldn't have built that product a year ago so like voices is like the paint is still wet on voice but it's really great and it's next generation and so um yeah i think you're going to see continuing adoption like ai for voice there's no comparison with the best models now versus where they were like even 18 months ago you know So, yeah, very exciting.

32:33Do you think that because the market, there are so many players in the market, I mean, obviously there'll be a period of consolidation at some point. How far do you think we are from that consolidation where the winners will emerge and they'll either buy up the smaller competitors or the smaller competitors will fold? It is so hard to know. I think we can only wildly speculate on that. I think huge markets with an absolute ton of value being delivered. And so like that would kind of tell you that like there'll be funding and opportunity to continue grow without a creative consolidation for a long time.

33:18On the other hand, you know, our product FIN returns, excuse me, our product FIN returns a healthy margin to us. I believe there are lots of businesses in the wider AI space where their margins are more questionable. And so, you know, people are just optimizing for growth at all costs. And it is possible, therefore, that there will be a shakeout or a period of consolidation if something happens in the wider macro environment or in the environment of AI to kind of drive a cooling. Overall, I would probably welcome a bit of a cooling. I think we have a great product. It's not just hype. It's got positive margin.

33:58So it would suit us strategically if there was a little bit of a cooling. I don't know if we're going to get that. I do think that the DeepSeek moment was interesting. I think the Chinese AI models are very, very powerful and compelling. and so maybe that will lead to a cooling um i just don't know it's a big space and that's a real macro question yeah well that's interesting because what what kicked off this space really was uh the uh was generative ai and you know ultimately the transformer algorithm um I mean, you're a technical guy, so looking, what do you think has to happen for the market to go through another surge?

34:54What breakthrough are you watching? What bottleneck are you guys working on? Yeah. I'd say a number of things there. Personally, I believe that we are in overhang at the moment here, where from an industry point of view, even if the AI models completely froze at their current perspective, at their current capability, we would have at least a decade of huge growth industry to deploy all these things to many different valuable applications. So I really believe that. But I do think that, you know, the core AI capabilities continuing to improve and they are continuing to improve is an accelerant to the whole space overall top to bottom.

35:42And, you know, really interested in things like reliability improving. We're like constantly working in our procedures products, you know, the one that like takes the actions and calls APIs. We're constantly working to get the best reliability we can out of LLMs. And, you know, we're working and we're investing in trying to improve that reliability at all these different layers. And also we really had some improvements recently in, you know, sort of the small models hypothesis, which is this sort of idea that you can take smaller large language models and you can train them to be specifically good at a given task you're interested in, really specialize them at that task.

36:25You kind of take a model that knows how to speak the language, knows how to retrieve a lot of information, maybe knows how to do some reasoning, and then you fine tune it in a sophisticated way to be really great at a specific task. So we have a sub part of BIN where we summarize the end user's question. It's really important to do that because sometimes, you know, before you go and go searching and you go trying to retrieve content, you really want to kind of canonicalize or summarize your question. We built a custom model that does that by kind of like fine tuning one of the kind of the QAN models.

37:00And that's performance is really excellent. And that's replaced a call to a much bigger LLM with like better performance overall. So I think you're going to see more of that. I think that the whole tech stack top to bottom is like radically changing. There's innovation at all levels on it, whether it's things like replace an LLM with like a smaller custom model, build a custom model from scratch. we've deployed a whole lot of custom models we built from scratch and finn as well and then also at the kind of the frontier llm layer they're getting like better and better all the time so uh yeah innovation at all yeah on you you mentioned insurance and banking those are you know highly regulated industries where uh their data is valuable and proprietary you guys are excuse me you guys are a SAS product uh operating operating from the cloud uh what do you do when someone needs an on-premise solution we don't currently provide an on-premise solution um and you know that's a long trade-off in SAS obviously you know we describe ourselves and as an AI company now I think it's going to be a long trade-off in AI as well and you know and it's just it is a velocity trade-off if you go on premise it's very difficult to keep your product development lifecycle iteration fast enough and it's just there's so much value to be unlocked in the world for customers that are willing to trust the cloud that that's where we're focusing for the future and you know the customers we have in these regulated industries they they've crossed that bridge they've crossed the bridge to the cloud a long time ago and not not every bank has but many have and so we think there's a ton of market there and we're not going to kind of slow down and take the velocity hit of going on yeah that's interesting oh yeah i just asked because you talked about smaller models whether you were looking at that.

39:11We could. We could deploy. We have enough of a stack now that will run entirely on models that we could deploy, that we could deploy to somebody's cloud if we wanted to or on-premise if we want to, but it's just strategically not the direction. And on the agent capabilities, and you alluded to this at the beginning about how customers are stretching uh how they're using the product uh are you looking beyond customer service uh if you have this this powerful solution that presumably could run other business processes i think that is a very interesting area that i think we're not quite ready to talk about just yet but it is absolutely an area of interest to us how could it not be we have this product it's excellent to customer service and then we have customers stretching it using it for pre-sales use cases getting a ton of value out of using for success use cases and so it's a very interesting and insightful question strategically uh but i think we're not ready to talk to you okay well this is uh fascinating and uh if people want to give finn a try it's fin.ai or what what is the yeah absolutely if you go to fin.ai and again it's a very self-servable product we have put a lot of energy into that and if it's an enterprise customer we definitely say get in touch reach out to sales if it's like an smb we'd say give it a try yourself uh you can probably unlock a ton of value we do we do the work of making sure it's self-servable and productized yeah and and actually i do have another question is this uh targeted for and priced for you know fortune 1000 or something or is does it uh is it affordable for a small and medium-sized company?

41:17It is absolutely affordable for a small and medium-sized company. We really pioneered outcome-based pricing here, where we charge a dollar per resolution, and we're quite proud of that. And that means that if you have a thousand support queries a month, and it's taking you several humans' worth of time to do them, you can afford to pay that one dollar. Also, if you have a huge support organization and you're outsourcing to a BPO, it should also work for you. So we're pretty happy that that price really scales with the value that it's going to deliver for the customer. And, you know, we have history around that.

41:55Intercom kind of got its pricing wrong in the past, in a previous generation. We've really learned from that. We really believe very strongly in fair, transparent, outcome-based pricing. And it seems to be working pretty well for us. Are we like squeezing all the value out that we could? Possibly not, but that's not even the right way to think about this. This is a growing market and growth is what we care about. Build the future of multi-agent software with Agency. That's A-G-N-T-C-Y. Now an open source Linux foundation project, Agency is building the internet of agents, a collaborative layer where AI agents can discover, connect, and work across any framework.

Read the full transcript

42:40All the pieces engineers need to deploy multi-agent systems now belong to everyone who builds on agency, including robust identity and access management that ensures every agent is authenticated and trusted before interacting. Agency also provides open, standardized tools for agent discovery, seamless protocols for agent-to-agent communication, and modular components for scalable workflows. collaborate with developers from cisco dell technologies google cloud oracle red hat and more than 75 other supporting companies to build next generation ai infrastructure together agency is dropping code specs and services no strings attached visit agency.org to contribute that's agntcy.org

From the publisher

This episode is sponsored by AGNTCY. Unlock agents at scale with an open Internet of Agents. 

Visit https://agntcy.org/ and add your support.


Why do so many chatbots fail in the real world, and how can AI agents actually fix customer support?

In this episode of Eye on AI, host Craig Smith explores how teams move beyond scripted bots to production-grade AI agents that resolve real issues across chat, email, and voice. We look at what makes agents reliable at scale, how to configure them safely, and how to manage them like digital workers alongside your human team.

Learn how leading companies approach agent onboarding and governance, which pitfalls to avoid, and which metrics matter most for success, including resolution rate, CSAT, and cost per resolution. You will also hear how to enable actions like refunds and returns through secure procedures, design human handoff that customers appreciate, and build an omnichannel rollout plan that scales responsibly.


​​Stay Updated:
Craig Smith on X:https://x.com/craigss
Eye on A.I. on X: https://x.com/EyeOn_AI

More from Eye On A.I.

All 266 episodes
#295 Fergal Reid: Why Your Bots Fail and How Agents Fix Your Customer SupportEye On A.I. · 44 min
Listen in VO