#138 Dan O'Connell: How AI is Disrupting Business Communications

0:00The piece when you mentioned the dataset of what you train with, which is unique for us as being the communication providers, we power all these conversations. And our founder five years ago, you know, had the foresight to say, look, if I can go again, power communications on any device and transcribe it. Those transcriptions, the training data that we can then go and leverage and build to power this large language model. A lot of our features are designing for recruit teams, for sales organizations and customer success and service teams. Those costs that we absorb ultimately are sold or passed to the buyer, right?

0:35There's very few AI features that we do that are just free features. Hi, I'm Craig Smith, and this is Eye on AI. We're excited to have Dan O 'Connell, Chief AI Strategy Officer at Dialpad, joining us today to discuss how artificial intelligence is transforming business communications. Dialpad offers a cloud-based communications platform aimed at understanding conversations and driving automation. And Dan provides an inside look at how they're using AI, like natural language processing, speech recognition, and large language models, to deliver features such as real-time transcription, call summaries, sentiment analysis, and virtual agents.

1:26I hope you enjoy the episode as much as I did. Before we begin, I want to give a shout out to our sponsor, UAI. They're a fascinating company that does a number of things, but I'm most interested in their Mind Studio, which is a platform for building AIs on top of large language models that you can deploy for free or for profit. Think back to the days when smartphone apps were first getting started, and if you were able to have been among the first couple of thousand people to build a smartphone app, you would have learned a lot and may have earned a lot of money. So give MindStudio a try. Visit uai.ai.ai slash MindStudio to build your own AI today.

2:24Yeah, I'm Dan O 'Connell. I'm the Chief AI Strategy Officer here at Dialpad. I've been here for five years. They acquired my startup five years ago, which was a real-time speech recognition startup called FockeyeQ. Our largest investor was Salesforce Ventures. And it was our belief at that time to, if we could go capture conversations and transcribe them and analyze them, then it opens up a plethora of opportunities around driving automation insights and assistance and dial pads. It's a cloud communication and collaboration software. We call it, we focus on customer intelligence, which is how do we go power communications on any device anywhere in the world?

3:04and then how do we help businesses understand how to help me within those conversations. And then my role today is I oversee our strategy, our partnerships, and I write back on product needs. Okay. How does AI play into this? And, you know, I'll use the bit earlier where you describe what Dialpad does, but how are you using AI? Yeah. And what's unique for Dialpad is, so we were a spin out from Google for a little bit more history. Our founding team, myself included, came from Google. Our founders had built what was called Google Voice at that time. Google didn't want to commercialize that product.

3:50This is roughly 12 years ago. they left Google Ventures wrote them the first check which minor is the founder of the Android operating system for any place it's not on the iPhone sits on our board and that really sparked the genesis for this communications company called Alpat and then over the past five years as I said it'd be key regularly apparent to say look our focus is not just about powering communications on any device when when I approached Craig around and Craig is our CEO around partnership it was if you can go power communication that's one thing but every business wants to understand what's happening within those conversations whether it's to better run the business and make better decisions by understanding insights or whether it's about driving automation or providing assistance like suggesting answers to questions and so when they acquired our business ai runs entirely through our stack we own our entire ai stack so we do all of our own speech recognition, which allows us to do things like real-time captioning or transcription.

4:49We do all of our own NLP work, which allows us to do things. And NLP would be natural processing for those that may not be aware. Allows you to do things with text for a computer. So I can start to understand sentiment or I can infer customer satisfaction. We do our own semantic search that powers our own recommendation engines. It allows us to build features like suggesting answers to questions. And then just a week ago, we announced our own large language model. which again, I think people get really excited about in terms of what is the next generation chatbot look like, or what are these next generation capabilities within communication software?

5:24So we sit in this unique position. It's a long ramble. If we own our whole stack, we're really proud about it. And it's really about powering communications on any device anywhere. And then it's about how businesses understand what happening to do. It's I thought. Yeah. Okay. Okay, so we're on Dialpad now. Are there features here? I see there's AI notes on the menu bar. How do some of these things work? Yeah, so our focus on our meeting platform here, right? So we have AI notes at the bottom. You can click that and see a real-time transcription happening. What we do at the end of a conversation is use our own large language model to empower instant summaries.

6:16And you'll see some other things that can pop up in AI notes at the end of the calls. When we generate summaries, we automatically capture the questions that are being asked and identify the topic. And then again, what you'll get at the end of this day, and I'll share it with you, it's an instant summarization conversation, which is, it's great to have this long transcript, but what you suddenly realize right it's match it as magic gets transcriptions and what you quickly realize is nobody wants to review a transcript right um people want to get the short gist of a conversation um and then if you think of you know this meetings platform is just one one piece of what we have but if you think about um us as a contact center provider for businesses uh so powering their service and support teams suddenly we can go not only transcribe those conversations we can identify the topics that are happening and action items and again generate those summaries and then instantly write that information to the cr route but the nature of these features and i can share some demos and features on things is the unique part for us is we can do all of this in real time so that opens up these really unique opportunities to do things like suggested answers to questions or route a conversation where it's being set to you name yeah um you mentioned And sentiment analysis, how would someone use sentiment analysis on this platform?

7:35Yeah, we thought about, you know, the interesting thing about sentiment analysis is our approach has always been thinking about it in three ways. There is sentiment that happens at the start of a conversation. And let's focus on sentiment around a support conversation. So somebody calling into what contacts are, because I think that's most relevant. That's what we see as most interesting. And, you know, you and I can get the sentiment of this conversation. I don't think it's going to be terribly informative for us. But you can imagine you care about customer satisfaction and sentiment. So we think about sentiment in real time, meaning if you do, Craig called into our contact center running our software, you would see a streaming sentiment of that conversation.

8:19We look in 30 seconds windows. So we know that Craig is getting more agitated or less agitated. hopefully it is you call in you're pretty agitated right uh you may ask for an escalation to a manager and hopefully that region is talking you off the ledge and resolving your issues and so that sentiment is is becoming more admissible uh and then leads to a call resolution so one is you see sentiment in these 30 second windows uh always mentioned once the call ends there is then the notion that you need to think about sentiment across um the entirety of that conversation right and that ultimately evolves into customer satisfaction which is you called in did we resolve the issue and what happens for many businesses is all ends and you are you typically will receive an sms or an email survey that says hey craig you know what would you rate your experience the problem with doing those sms surveys is that very few people reply to them and the people that ended up replying tend to be in two camps one it's the agent did a really exceptional job and somebody must have seen your praises, which doesn't happen a lot.

9:24Or it's, I'm really fed up and my issue wasn't solved. And I want to tell you how bad and frustrated I am. So it tends to be a little bit polarizing. The data tends to be polarizing. The nice part of thinking about settlement for the end of a conversation is suddenly you can measure settlement in real time. It gives you an indication of customer satisfaction for every single conversation. So that's the second way that we do this. And again, it's kind of like this magical unlocked for these businesses because it's, hey, you don't need to go buy new software. You don't need to send out surveys. And by the way, we'll give you far more data for every single interaction.

10:00So that's the second way. And then the third way is you want to start stitching together sentiment across multiple points of contact in the same business or multiple calls from the same person. And that gives you some notion of account help, which is, it's not that Craig has called in once. It is Craig has called in three times in the past week with increasing frustration, right? Or decreasing customer satisfaction. And then that gives you the notion for these businesses to start thinking about proactive service to say, I don't need to wait for Craig to call me the fourth time. So I'm pretty sure every Craig calls me the fourth time, he's going to tell me to make the contract and he wants a refund, but how bad we are, and it's going to go hop on Trust Pilot or G2 Crowd and tell the world that.

10:45This allows you to start and get into a place where you can help businesses be far more proactive and ultimately play offense. So those are just three examples of how we think about sentiment. I think one time, a lot of people will think of sentiment as this easy thing and suddenly it kind of brainstorms into these others. The last piece of my ramble, I would say, is we focus on the context of conversation. Anytime we get into sentiment analysis, a lot of times people ask us about intonation and tone as opposed to contact. And we really focus on the context of the conversation and what our data scientists would tell you is when people call into a contact center, they're very specific around their needs.

11:27Most people don't call into a contact center and have sarcasm, play around. They're very clear about what they're looking for. And so we don't spend time looking too much at intonation and tone. and we think you can get pretty much ready. You can build very accurate and directional models from just focusing on the context of where it's being set. Yeah. And the large language model, so instead of leveraging one of the commercial models through an API, you've built your own. I'm curious about why you decided to do that Or are you, when you say you have your own model, are you fine-tuning an existing model that you access through an API?

12:19Yeah, and there's a couple different angles to that. So one is, as I share, one of our key philosophies about our business is we want to own our own. time. And I think what's important around that and being a builder of startups is if you own your own stack, it allows you to control your own destiny in terms of innovation. And if you leverage other third-party APIs, it's my belief, and there's no right or wrong way to build software. This is just my idea. This is why I agree. If you outsource or leverage off-the-shelf pieces or APIs, you don't control that innovation. At times, the API might not do things that you want to be able to do it and you're dependent on somebody else to then go and make changes and and change their roadmap what we found when in these initial foundational models um so let's just you know we we played around with chat gpt for example for well open ai chat is one is we got unpredictable outputs so and i think many people can experience this when you go to chat gpt sometimes you ask it to do something you may get a millisecond response sometimes you get a second response sometimes it might take 10 seconds sometimes it might tell you they were timed out on capacity of demand we can't process this right now try again if you think about building up productivity you know building a feature that should be commercially available you can't have that experience you need to be able to make sure that the user experience is really tuned and exceptional and so having unpredictability in the response is just one the issues um number two is unpredictably um um you know they had offered fine tuning as of uh i think yesterday they made announcements but if you think back that they've been working on this for four months there's no fine-tuning um right uh on those pieces um the gpus that are needed to go and run these with really large foundational models they're expensive at scale um and also if you think about um you know specific needs in the contact center uh if you think of a large a foundational model is kind of the library of congresses it's got all of this knowledge you don't actually need all of that knowledge um and so it's our belief of like you can build a smaller domain specific large language model that needs less or less demanding computing resources allows you to go scale it allows you to have predictable output allows you to reduce hallucinations and then is more cost effective allows us to drive our own innovation roadmap so it's our belief of that's our philosophy is all of those benefits.

14:48But I will say there are places within our pack, in our tech stack that these, that the large models make sense and world leverage. And Google is our trusted partner on that side. So Google Vertex is a foundational model that will have some specific use cases within our stack, but a vast majority powered by Rope. Yeah. And the model that you build, how, how large is it? I'm curious about this because I've been talking to people about the cost of training large language models, not only of putting the infrastructure in place, but the power generation or the power consumption needed to train. And then, of course, collecting the data.

15:34So, yeah. Yeah. So two things. So we don't, we don't comment on, on parameter size. Cause I think one thing when you get into size is there is this belief that, that larger is better. And I think what's true for many use cases of get it software is you want something that, that, and this is almost scoffed out for people, which is faster, cheaper, better for customers. And so the fortunate part for us is look, we do not have a model that is going to be the same number of parameters. Like our intention is not to build a model to compete with a large foundation. It is, we are building a model that is designed to be smaller and is designed to be domain specific and use case specific, right?

16:13And so that is going to provide an enhanced experience that's going to be more predictable in terms of the output. And again, when you talked about trading and building up more cost effective for us to go and manage, right? And scale. The piece when you mentioned the data set of what you trained with, which unique for us as being the communication providers, we power all of these conversations. And our founder five years ago had the foresight to say, look, if I can go, again, power communications on any device and transcribe it, there's transitions, the training data that we can then go and leverage and build this large language, could power this large language model.

16:54So we focus on, one, opted-in user data, So we're clear around users opt in to share their data. We will take that data. We will anonymize it. We will strip it of any personal identifiable information. We're SOC 2, HIPAA compliant, you name it on the compliance. We have very clear on and off switches within our software that allows the user to turn it on and off. And then you get the data retention. It's your data. You control it. You should be able to blow it up and eat. and and so all of that said you then have access to this data that you can go one leverage to go and build this large language model and you have the team of experts that sits in-house that you know has experience in and nlp and speech recognition and machine learning you have those capabilities so again that's part of the reason we all wanted to do this in-house is we have that expertise in data sets but yeah uh can you comment on uh on the cost cost of training uh your model is it as expensive as everyone claims uh extensive and i'm not trying to dodge terms i don't i will probably have a job see me suddenly will probably yell at me if i tell you the kind of you know the specific cost uh for us look like we are uh not a public business uh right we're privately funded uh we have to be mindful of our costs and you see those you see me smile, laugh, I'm like, I think these are reasonable costs that, again, it comes down to, I think the costs are manageable.

18:27We think it's the most important thing for us. When we talk about we have five pillars as a business, AI is number one on that pillar. So for us to be worried about the costs, to say that, hey, this is our most important thing and we're worried about the costs a bit. And we have to manage costs, as I said, as a private business. We have to focus on that. And then a two is we have the data set and then the in-house team of experts to be able to go and do that. The other thing with these investments is we're focused. A lot of our features are designing for recruit teams, for sales organizations, and customer success and service teams those costs that we absorb ultimately are our sole they'll pass to the to to the buyer right there's very new ai features that we do that are just free features we do do you know and i apologize for these long rambles um tradscription for example is a free feature we don't charge for that the reason we don't charge for it is again we do all of this in-house We do it for basically one-tenth of one petty in terms of doing the transcription and the NLP cost.

19:39We are far more cost-effective and far more accurate in terms of the output than if we were to use a third-party HAPR. Yeah. So how does... Dialpad was in existence before generative AI or the transformer algorithm came along. So how did that affect your product? Yeah. Yeah, go ahead. Yeah, we've been an AI-first business for the past five years. I know a lot of businesses will say that they are AI-first or AI-powered, but we truly, since our acquisition of showing up and we had built one of the first real-time speech recognition engines, we have been an AI-first business. I will tell you, it's this really interesting moment in time.

20:28you know i i grew up in in the heart of silicon valley and unfortunately two passions growing up they were sports and in computers so uh i look at this moment when chat tpt showed up and we literally the next day had pretty much an all-hands kind of executive staff meeting where it was hey the game has changed and um we had obviously you know our nlp team was and you know i say initially concerned because i think there were those reactions like hey do you need an nlpt can this large language model suddenly be able to do everything you've invested in overnight and what does that mean for the competitor dynamics for businesses that have never had ai before can they suddenly go and show up and do everything that you've done overnight so we literally had the first time we had played around with it was the the maybe the the oh crap moment uh where like can we better align and understand this and invest in it and start making decisions?

21:28And I think that's the nice part of being in a small startup that, again, cares about innovation is you can have those types of conversations very quickly and figure out what are the opportunities because we're not the cruise liner. We're still a small speedboat. And so we can be pretty nimble. If we need to go change direction, disrupt the roadmap, we can go and do that. Yeah. Yeah. And to that point about being smaller and nimbler, how do you, is that me or I'm getting an alert? How do you compete with a company like Zoom, which is adding a lot of similar features themselves? So, yeah, I think for us, like, there's a couple of different ways that we would compete with them.

22:14You know, we offer a single piece of software to get power communications, understand it, as I said, whether that's in the contact or sales side, whether it's for recruiting teams or just your back office telephony. For us, it's been heavily investing in AI and being able to tell that story, again, having features that are going to give drive insights, power assistance, or provide automation. and I think speed matters at the end of the day. And I think that's something that can get glossed over at times by different people in markets, the way they look at it. But the reason that the largest businesses in the world constantly over time get disrupted by smaller businesses is that those smaller businesses push innovation.

22:56They carve out time for innovation. They don't have to manage their business to a public market and some of the pressures that show up there. And they understand that they're not going to, They're not the king of the hill. The leader in the market has everything to lose. And so when I look at Zoom, I think we can out-innovate them in terms of the pace of innovation. We've demonstrated that. I think Zoom has had an incredible run-up in terms of the opportunity through COVID. And I look at them personally as, you know, you're at the top of the hill and you got a lot to lose and we got everything to get.

23:30And so that means that we are not a public business. We get to run our business differently because of that. We get to take risks because of that. And our executive team is aligned on that. And AI is the key innovation. It's going to be the place where the battleground is kind of won. Yeah. I'm trying to figure out where this noise is coming from. Is that from Dialpad? I'm getting a... You don't hear it on yours. I don't hear it. Yeah, I don't hear it. Yeah. Yeah. And where do you see this going? Not only Dialpad and your market, but the spread of generative AI and large language models? Yeah.

24:20For these, I think the things that get me excited is obviously I think it's readily apparent that these digital agents or next generation chatbots that are actually capable of it. I think the chatbots that we've had in the past have been good at password resetting probably. But I think it's readily apparent that we're going to have these digital assistants that can guide us, whether that's guiding us through music theory, whether it's guiding us through how to play piano, maybe it's guiding us through the workouts that I should be doing and designing a workout for me. Maybe it's having, again, the digital agent that's writing my first drop of the blog post.

25:00I use large language models a lot for generating ideas and first drops of that. So I think that's very evident for that. For our business in the communication collaboration space, for me, what I think that the large language models open up is the ability to completely disrupt the CRM market. And then I constantly think about, you know, How does a business like Salesforce get disrupted? If you and I were to build a CRM business today, we would build it the exact same way. The CRM business today has always been reactionary. You and I have a conversation. I go into a database and I write my notes in there and I'll play it.

25:38But if we were going to build that, having the technologies that are available today, you would probably give away communication. I would want you to work on, you know, power that whether it's a SMS or team messaging or voice or video, whatever it might be. I want to understand those conversations. So I need transcription. I put a large language model on top of it to go do the automation and categorization of that data and then write it into the database that I own. And I would completely say you can't write into the data. No person can write into the database, right? We're going to use the machines to write it.

26:11So it's always structured. You know, that's the perfect vision that I think we get to. And I think that business has not been built yet by anybody. And I think it's one of probably the biggest, best opportunities out there in all of software. Okay.

26:30I'm running out of questions here. Are there aspects of Dialpad or of the integration of generative AI into this industry that I haven't touched on that we can talk about? Yeah, I think one of the ones that becomes most apparent to people is the next generation chatbot. And I think people get very excited about that because they can see the type of natural experience, conversational flow, is largely which bottles are capable of. But really, I think what captured people's imagination with chat GPT just sounded like you were talking to a friend. What I think is really surprising and interesting, which doesn't get as much attention, is the ability for these large language models to categorize data.

27:22And I think when you think about all of the time and wasted energy that every business in the world, they sit on a bunch of data and then they have different teams try to go through to categorize it to get it structured. And I think there's just this massive opportunity for the large language models to do that categorization and get unstructured data into structured ways. And then to also then tell you things about it. I've seen some really incredible demos where somebody can load up their data into a large language models and asks it. And so I think we're going to see the whole way we think about data analytics and data warehousing and the visualization of that is going to get completely upended by large language models.

28:06And I get really excited about that. And I don't think it's had as much fans there as the chatbots of the future of the visual assistants that are showing up. And I actually think that other piece is more viable and interesting and probably valuable at least enterprise type in terms of enterprise. Yeah, and are you doing that? Are there projects at Dialpad that you're working on in data categorization? I mean, presumably in this conversation, a large language model could go through and pull out names. Yeah, we take it a step further. So it's not just about capturing things like the action items that become rare, but it's also about outcomes and purpose.

28:57So again, if you think of what we do for our summarization and our contact seminars that we generate, and again, I'll share some videos for you. We automatically summarize that conversation in the four or five sentences. We immediately identify the purpose. So what was the reason that this person is calling in, right? Might be a refund request, might be technical support, you know, for example. We identify the outcome. So was it resolved? Did it need a follow-up? And then you have any of the action items. And again, all of that instantly happens through the power of a large language model. and that's just the tip of the iceberg with these that we're working on.

29:35That's it for this episode. I want to thank Dan for his time. If you want a transcript of this conversation, you can find one, as always, on our website at IonAI. That's E-Y-E hyphen O-N dot A-I. That's it for this episode. I also want to thank our sponsor, Mind Studio by UAI, which is giving creators the opportunity to build and deploy generative AI apps for profit. UAI has an emerging AI marketplace, and MindStudio is the best way to build apps with generative AI. Anyone can do it. MindStudio uses conversational language to program incredibly powerful AI tools. No coding knowledge is needed to start your AI business today.

30:32Check them out at uai.ai. And start building your AI app today.