#292 Peeyush Ranjan: How Nurix Is Redefining Voice AI for the Enterprise

0:00The number one thing which we really invested in Nurex is actually in making sure that the latency is really low So let's say you're talking to a model which is going to perform certain tasks for you. There is in chat and all, it's very clear. You can have more latency, even in voice when it comes over, there's expectation built. But when you call into a business and you're talking to a person, there's no expectation of latency. It's like almost human-like behavior requires as I finish, you talk. And not only that, there is a little bit of parallel conversation going as well. For example, if I'm speaking for a little bit longer and you're saying, and all those things, like, you know, those are markers of like, yeah, you're nodding along and you're following me.

0:45Those are also part of the conversation. And if you just keep quiet for too long, then also I feel like, oh, are you even listening? Are you there? My name is Piyush. I'm a technologist at heart, you know, and I've been working on a variety of different problems. which had like an intersection of technical depth and worldwide impact thanks to a long-time association with a company called Google where I joined like 19 years ago. I left them just end of March. And over the period of time, I have worked on a variety of problems from search to workspace and fintech. And most recently, the past three years, I was working on AI as part of Google Assistant and the Gemini app.

1:35And as of March, you know, I left Google to join with our ex-colleague and a friend for the last 10 years, Mukesh Panshal, in creating, in partnering in what is called Meraki Labs, which is about incubating and building, you know, deep AI companies we all see that it's changing the world. And Nurex is first of those companies which I have been involved with. I'm on the board and obviously giving a lot of technical input and architectural input on how things can work and how they can scale. Nurex itself, yeah, go ahead. No, go ahead. So Nurex is one of the companies incubated Yes, yes. And, you know, I'm just starting.

2:35I mean, this is the first in a really important space, which is, you know, of conversational agents. So if you ask me, like, about Neurix itself, I think the toughest problem that I feel like Neurix is handling, which is required, is actually having a world-class voice AI solution. You know, most of the conversational agents, there are many different ways conversations are had, right? You can have text and email-based or chat-based and voice-based, you know, and of course, in future, like video and other things. But currently, voice is the cutting edge of that, which has the hardest set of problems within it.

3:19That is one big space which Neurix is going deeper into in a variety of ways through new innovations. and then there is, of course, the other area is how do you make this agent work really well for enterprise use cases? So there's a voice set of problems and then there's an enterprise set of problems. And the goal which Neurix has is to, through this, be the smartest, you know, operator for the brand. Like, you know, when you are talking to Neurix for, you know, when deployed inside a company, it is essentially a representative of the brand that you are talking to them, whether it is from starting from sales all the way to support.

4:04And also, at the same time, it's not just that you are talking and getting information. It's the smartest agent which you can talk to because it integrates deeply into the enterprise and is able to help you through all your workflows. So those are the two big facets of it. yeah uh i've been talking to a number of people about voice and about uh translation also uh i just talked to a company called deep l i don't know if you know them uh but they do uh they do translation uh and then i i earlier i had on the podcast a company from uh i always get their name mixed up because there's so many similar names.

4:52I think it's Speechmatic. Does that sound? Yeah. They do very natural sounding. Their emphasis is on very natural sounding speech. So one of my questions is, you know, the underlying tech exists. It came out with with the advent of transformers open ai does i talk to open ai's chat gpt when i'm in the car and yeah it's remarkable uh even the standard voice how natural it sounds uh so how do you how does a new startup fit into that crowded market what what what differentiates i guess That's a great question. I think that there are a couple of things which have to be kept in mind. So let me first talk about the voice part of it.

5:56The voice part of it, the number one thing which we really invest in Nurex is actually in making sure that the latency is really low. So, you know, when you are talking to, let's say you're talking to a model which is going to perform certain tasks for you, there is, in chat and all, it's very clear you can have more latency. Even in voice, when it comes over, there's an expectation built that, well, I asked something, it is going to take a while for it to get to the answer. But when you call into a business and you're talking to a person, there's no expectation of latency. it's like almost human-like behavior requires as i finish you talk and not only that there is a little bit of uh you know parallel conversation going as well uh for example if i'm uh speaking for a little bit longer and you're saying and all those things like you know those are markers of like yeah you're nodding along and you're following me uh those are also part of the conversation and if you just keep quiet for too long then also i feel like oh are you even listening are you there So there are these kind of psychological aspects of it, which are above and beyond the actual likeliness to a human speech.

7:14This is actually about conversation. So we invest a lot of time in making sure that the latency does not drop. And one of the reasons why it is important is when you're calling into an agent, well, two things. One is when you're calling into an agent, you might ask the agent to go do something. right therefore your your pipeline of activities can have latency introduced in many places but we don't want your voice to be uh your voice interaction to have any kind of latency so we have invested a lot of uh engineering in there to actually be able to respond to you super fast irrespective of what is happening behind that is one the second thing is what i was talking about that human interaction and dialogue between two people is actually a little more nuanced than the over and out Roger, kind of like walkie talkie style.

8:05So here we have our own proprietary dialogue manager, which also is trained to understand that, okay, this conversation which is going on, is the person stopping and thinking about because you suddenly ask them, hey, what are your last four or what is your driving license number? And the person is quiet, it's probably because they're fishing for their driving license, as opposed to, it's your turn to talk again. Or if If the mm-hmm is happening, then that's what that means. So that is the second investment in there. And the third thing, which is beyond a single person-to-person conversation, is imagine a call center, right?

8:42Now, there are like 500 people calling simultaneously. It naturally, from system design, right, engineering-wise, it can have impact on latency of each one of them, depending on how you architect it. So we have to think about that architecture as well, that when the load goes up, it does not mean every person feels that their agent suddenly becomes slower. So all of these are aspects. This is one of the things which I feel we are feeling is the more you get deeper into the domain of how the models are being applied, the more these nuances start to show up and you have to go solve them because the models are very, very capable and they'll only become more capable later.

9:23But you have to understand these nuances and you have to solve them properly. so that you can deliver a world-class experience on the other side. So the point you made about these capabilities, which are there, yes, they are there, and we leverage them wherever they are. And then we solve these problems so that the person calling into the business feels like the other person is really as human-like as possible. Yeah. And one of the things you're talking about is endpoint detection, right? Knowing when the other side has completed a sentence or something. Yes. And what are the underlying models?

10:02Or do you, are you, is Nurex sort of model agnostic because, you know, there are new models coming out all the time? Or are you working off of open source models and fine-tuning open source models? Yeah, I'm curious about the architecture. Yeah, so there are parts which are our own proprietary models, like this turn-taking piece is our own proprietary model. You know, we need to really make sure that we understand the conversation which is happening, which are many times business-specific conversations. And also we understand, you know, when is it that – because we also operate in a variety of noisy environments and all that.

10:49So we have to make sure that we understand when really there's an interruption versus there is not. So that is proprietary model. But then when you start deploying that into a particular customer, then there are specific models depending on what is the task which they want to apply it for. We are able to integrate with OpenAI. open ai we are able to integrate the azure implementation of that we are able to integrate with gemini over gcp through vertex uh all of these are available and they are all integrations depending on the deployment which we are going after uh there are other places where we also have uh custom models like uh the speech to text i i think i mentioned that like you know the the real world noise detection and saying okay what exactly is this person saying and understanding uh the speaker is stalking uh into the phone as opposed to somebody else or there is two other people in the background talking, all of that is actually done using a proprietary model because we want that experience to be perfect.

11:52So that and the dialogue back and forth is done by our own proprietary model. And when deploying into the business, then we can choose what is the best model for that particular use case. Yeah, and this product, it's not a consumer product. It's going into the tech stack for various other applications as the voice engine. Yes, it is actually an enterprise product. It is still an end-user product in the sense that the end-user interacts directly. They'll dial into or they will call into or they'll message a Neurix agent. But that Neurix agent is deployed for a customer. like if it's a retail customer, will deploy it and the retail business will deploy for, and the end customer of that retail will call in and say, where is my order?

12:47And that call will be fielded by Newrix agent. So that solution is built by us and it is available, but they feel like they're calling into the business. And then it is Newrix agent, which then picks it up, talks to them. And based on how it has been configured and deployed, it will actually go look up in the order management system or something. and then be able to give them the answer back or, you know, act on that order depending on what the operating procedure is. Yeah. And again, you know, am I wrong that this is an increasingly crowded space? I mean, just from the number of people that contact me or the various conversations I've had, it seems like there are a lot of people out there trying to tackle this problem.

13:34And is the market large enough that it'll support many, many solutions? Or are companies going to start... Well, one of the other questions is these guys at D-Bell are doing translation between language pairs. and they're not doing actually they're not doing voice to voice they're doing text to text and voice to text but presumably you need models fine-tuned for each language language. Is Nurex focused solely on English or are they looking at other languages? Nurex is focused on multiple languages. We are currently available in 20 plus languages. And, you know, we have customers in US and in India and in both of these countries, multiple languages are needed.

14:45You know, especially like in US, you at least have like English and Spanish and in India, you know, almost every state has its own language. So, you know, it's multilingual. I will say one thing. It's not a language translation experience. It is actually a conversational agent experience. So you can talk to this Newrix agent in any language and primary value you get out of it is the fact that it can actually act on the business's behalf as your customer support or your customer sales agent while you are talking to it. And you can talk through like a chat or a voice or whatever. Now, like you said, it's a very, you know, happening space.

15:28A lot of people are entering. Generally, the two things I will say that, yes, it kind of shows that the market is ready to accept solutions. And we are seeing the same thing. Like you said, we have over a quarter million calls which were done just this year, and we have 5x quarter over quarter growth. So seeing this kind of growth, it's not surprising that a lot of people are coming because they're like, market is ready. Now the question is, just like we have seen it many times in the past, that when a market starts to become ready, a lot of people are there. But right now, I would not say that the market is saturated because there's so many people who actually have call centers and voice-based representation or the representatives, both outbound and inbound, which still are not AI agents.

16:27And that whole thing can be much better. The cost for the business will be lesser. The error, because these are very repetitive at times, tedious jobs, the error rates can go down. There's churn in terms of users. So your productivity can be up because you don't have to train as much. At the same time, simple things like I'm sure, Craig, you have experience as you call a number like, all right, you are on the hold music, right? There is not going to be hold music anymore, right? So those kind of things. And I think that the world will look different if we just, you know, and I'm sure that you will agree with me.

17:03In AI era that we are, world looks different, you know, six months from now, right? So there will be a different world. But right now, people are entering it. It's validating the market. And it's not done yet. I actually feel very confident that we are building a solution which actually is at the edge of this. like this voice problem you were talking about, like making human-like voice possible out of the system is non-privial. There is a lot more psychology involved in it. We spend a lot of time making sure, squeezing out every bit of latency out of it. So you feel like you're talking to a real person.

17:38There's a lot more dialogue management involved. This is why our proprietary models are there. So a lot of these, the science is going on and it's obviously, hopefully, you know, the fact that we are investing so much in the science will help us, you know, see traction. And I think we are starting to see that and hope this quarter-hour growth continues, right? So, yeah. Yeah. And then how do you think the market will then eventually coalesce around a few big players? or do you think that companies will end up specializing in, like there'll be the dominant player for India or the dominant player for China, the dominant player for South America?

18:28Or do you think one company will cover all that and become dominant globally? um i don't know how it eventually will be but i can tell you the uh current heterogeneity that exists and that heterogeneity like somebody who covers that really well can become dominant or it can become the way the market gets segmented right so the biggest heterogeneity is that, you know, and this we have learned from our scaling, that you really have to go in and, you know, be like, get into the business and deploy, right? It's not that you're from outside, you throw something over the wall, and the business says, all right, let me go click, click, click, and be done.

19:20Because if you make mistake, it actually affects the operations of the business. It affects the profitability and things like that, customer retention. So you have to go deep into it and deploy. And there's a lot of learning which is happening in that place. Like I will give you an example, just to give you examples of how current technology cannot just be thrown over the wall. It tells me that you cannot today have a business which goes, all right, I got it, go everybody and take it over. Here is an example. You must know about RAG, right? Retrieval Augmented Generation. Now people take documents and they kind of stick that inside that.

19:54It's a very standard technique. However, if you're going inside an insurance business and you have put all the policies in there, it's not as trivial to tell a customer that yes, your current claim is supported or not supported. Because if you make a mistake in your rag, if you did not pull the right chunk out, you might end up denying something which should have been covered or you might end up covering something which should have been excluded in the fine print, right? So we invest a lot of time in understanding the structure of the document based on the domain that is, so that we can chunk it properly and we can retrieve it properly.

20:30You know, we even spend time in like rewriting the query to like, you know, when the person comes and says, well, is this called my policy, then we know how exactly do we rewrite. Now, this is all domain specific work. Now, the reason I give you this example is if you take all over the world, this work has, this is not easily generalizable. And this is what we are seeing with everyone, not just in Newrix, like everybody else, is that they have to go in almost Palantir style, like they have to go into their customers, you know, building and kind of work with them. And that means the answer to your question that will there be one which will win all, I think, till technology gets to that place.

21:11And maybe, I mean, Newrix is building a platform called new play, right? We will have a platform which will make it super easy for you to pick and choose. And when those kind of platforms start emerging, we are starting to offer one, and those become very capable, that is when we will be able to answer this question. It's hard to tell right now, but right now, to keep the quality high, it's a lot of work. We have been able to scale the deployment, and that is why we are pretty excited about it. and and i'm just curious your sales operation how do you figure out who might need the solution and and then uh yeah i mean because there there are all these different sectors and all these different players.

22:03I mean, as you said, customer service, customer service alone is a very fragmented market. So yeah, I mean, what's the strategy? Do you go after one vertical? Do you sort of have a massive sales team that's calling everybody who could possibly need voice? Yeah, well, Craig, we are still a startup, right? So we are like, maybe, I think, six months or a year. My involvement has been less than six months. But the thing is that we are still a startup. So the two main things, which I will say, we would not turn down a customer who comes in and says, we want your stuff, right? But at the same time, when reaching out, we got to go where momentum is and where we are actually like, we have referenceable customers.

22:59And we are seeing a lot of traction in specifically, like, you know, in e-commerce market and in financial services market. The example I gave you insurance was a financial services and e-commerce is the example I gave you the order return thing. We are seeing a lot of, you know, traction in there. And, you know, but I'm thinking that given the growth pattern we are in, we also not like, you know, that doesn't mean that all our revenue is coming from just those to? We are a startup. Why would we say no to more customers coming in, right? Scale as fast as you can. So that is the answer to your focus and sales question.

23:37But we are primarily seeing traction in two big markets. And I think we are not in a state where we can have salespeople everywhere. Our founders are actually knocking on the door, showing up in the conferences, talking to people, We are founder of sales right now. So that is how we are. Yeah. And the dominant player currently seems to be 11 Labs.

24:06How do you view them as a competitor? And how do you distinguish Nurex from 11 Labs? So Eleven Labs is a great technology provider which we can use. The basic idea is that they do a great job of generating human-like voice for any kind of parameters, right? But if you notice, we start from like fielding a phone call and having a conversation. Now, in having a conversation, we have our own, you know, speech to text models with all the noise filtering and the dialogue management. But if we have to generate a response, we can use 11 labs. It depends. Like we use Google's TTS as well. We can use 11 labs.

25:01We can use others. They're all technology choices to us. The overall approach is where is it that the problems have not been solved, we want to go solve them. And wherever they have been solved, we want to go utilize them. So we can focus on the layers where things have been solved. So that is how we see it. We don't see it as like 11 labs as a competitor. It is actually our supplier and a partner, if you will. I see. Yeah. Yeah. Yeah. This is fascinating how these new industries are developing and stratifying. Absolutely. Yeah. Yeah. So you mentioned that someday we won't have to listen to hold music.

25:41It still drives me crazy. Yesterday I was calling somebody and it's like, you know, listen closely as our menu options have changed. And, you know, why is there a menu?

25:59So what's holding enterprises back? I was wondering, is it because these guys have bought a system and, you know, they want to wait until they've amortized it down to nothing? Or is it that companies are just risk averse and they want to see how the market evolves? I mean, what will it take for those, what are they called, IVR, I can't remember. Yeah, for all of that to go away, where you call up and you speak to somebody and you can't really tell whether it's a human or not, and they handle the call fluidly. Do you think that's a year away or 10 years away? Oh, I definitely don't think it is 10 years away.

26:59Let me look at the amortization thing first, and then I can talk about what might hold somebody back. I don't think it's amortization is going to be the biggest issue because the savings, you know, not only the cost savings, but also reduction in errors and mistakes, increased opportunity of better cross-selling and upsells. selling when somebody calls in. And above all, all of this, because it is run by Nurex's AI, we can provide analytics saying, look, these are the kind of things which you could do with the calls. Today, call analytics is actually very, very hard, but we can do a much better job because everything is running through our digital system.

27:44So all of this will have enough business value that you will actually justify the new investment. So that is not the primary blocker in my view. But the primary blocker, I think, would be the businesses building confidence in these non-deterministic systems. In the IVR, it's like, you know what I mean, right? It's a very, very clear IVR tree you're going through. And then there's like, okay, I can do a set of things and then a human comes in. But here, if you really wanted to remove all that tree, you would call and immediately it would be like, hey, Craig, this is Bob from Target, right? If you're calling Target.

28:26And then it's like, okay, what do you want? Now you have a non-deterministic system. What are the expected outcomes? What are the operating procedures? What are the guidelines and guardrails? And making sure that the system runs within that, they have to build that confidence. And this is one of the spaces, by the way, we invest a lot in. Like I talked about the RAG. We have a lot of the simpler parts here are that, hey, you have a set of back-end services. Can we integrate with that? That is system-level problem. We can integrate with that. That's easy. You know, MCP is available. They're like out-of-box integrations.

29:00LLMs can do tool use. All that is there. But making sure that the LLM actually does not deny a claim when there was actually a possibility or does not say, oh, yeah, go fill out this form without calling out that, no, you had an exclusion or a series of such things or does not misunderstand an acronym which is specific to a particular domain. These kind of things actually are problems which are being solved in the industry. We are investing a lot in that as well. And I think that that, because it is new and being solved, there is a level of comfort which has to be built. but we are seeing a lot of traction in people trying it out right so the thing is that the business once they try out and they see the difference oh yeah this is great my errors have gone down my business has gone up i'm able to transparently see what exactly is happening in this part of my call center now naturally all right let's do more of those so i think that that adoption will happen and fairly so because this is their business they don't want to risk that Yeah, yeah.

30:03And so you guys build not only the voice generator, you build the, well, yeah, describe again. I mean, you're not simply voice generation, you build a higher level solution. So describe sort of end to end what that solution is. Yeah, absolutely. So now imagine that, imagine you are a retailer, you know, and you say that, hey, I would like to try out, you know, your service for, you know, people who call in to find out where their order is or want to cancel their order or like, you know, status of the order, right? Very simple three things. And you have a backend. You have order management system, which has, let's say, HTTP interface or some other sort of internal interface.

30:58so what we do is first is that we and we say okay what kind of endpoints will you have will you allow people to you know call you in but they want to check in through a chat bot on your website or do you want to like allow them to check on whatsapp or sms or whatever those are the ways they can talk to you now the when they call the calling to the voice is the most technically savvy part of course here because of the human thing we talked about when you call in are uh the phone lines obviously end up with us and then we immediately figure out what you are saying using our own custom speech-to-text model.