Douglas Adams’ Babelfish concept just got much closer - Interview with Sharath Keshava Narayana of Sanas

0:00So welcome to the next edition of the PunkCX podcast. On the podcast today, I have Sharath Keshava Narayan, who is the CEO and co-founder of Sanas. He has over 20 years of enterprise software sales experience, as well as AI expertise. He's a serial entrepreneur and prolific investor. He co-founded Observe AI and invests through Carrier Venture Partners. Sharath is committed to driving technological innovations and fostering inclusive communication globally. He's an accomplished entrepreneur and investor, and he's been the co-founder and CEO of Sanas since January 2022. And he's played within Sanas, he's played a key role in developing the world's first real-time accent translation algorithm, no less.

0:47So, Sharath, welcome to the podcast. How are you doing? And also, is there anything I missed out in my intro that you'd like to add? Adrian, firstly, thank you so much for being gracious with your time. I am excited that I'm worthy of a Punk CX podcast. Thank you so much for inviting me. I think you covered it really, really well. I would always say as an entrepreneur, and if you've done it for long, you just say you're plain dumb lucky because there are a lot of people who are as good as you are, but sometimes you get to win and somebody else does not win as well. I've just been very grateful for some wonderful people who've been a part of my journey and folks like you who've been supportive and put us on the map.

1:30So thank you so much. You're very welcome, sir. Now, we mentioned it's that you're in that sort of speech to speech or speech AI technology space. And you've been working in that space for a while now, specifically in historically focusing on noise cancellation and accent modulation within Sanus. But however, you're now turning your attention to real-time speech to speech language translation, which I think offers exciting possibilities for both service and experience. And it's like proper sci-fi stuff, but we'll get into that kind of like in a minute. But before we dive in, I wanted to talk about sort of current translation methods because there's a whole bunch of stuff kind of like around.

2:12But I wanted to get your view on the state of play and where you think some of the current translation methods can fall short. Yeah, let me take one step back. I think I'll get to your question very quickly, but I think speech has been one of the most exciting problems that technology took a long time to solve. I think talking about translation for, I think, three decades now, multiple people have attempted building translation systems. And I'll cover a few reasons as to why I think it fell short so far. but I think for me personally I've been building speech systems for almost nine years now first with Observe then with Sanus and and interestingly there are essentially three ways you break down speech one is what we all knew breaking down speech to text and doing analysis on it and the other one which became more popular with the advent of agentic systems which is the TTS modality of breaking down speech but there was always this in between which I feel is more true true and authentic and any speech researcher will resonate with it of how do you break speech to speech directly right and and that was something that we uncovered discovered happy accident you can call it a few years ago and and that's the overall stack we're trying to build where hey let's not have an intermediate form while we break down speech let's break speech into what is more natural called as phonetic sounds because adrian is made up of four different phonetic sounds and The beauty about phonetic sounds is there is no language barrier for phonetic sounds because phonetic sound is a spoken emotion or a spoken sound, if I have to say it in layman terms.

4:00So it was the invention we did four years ago, right? And everything that we've built within the stack, and I'll come to language translation now as to why that is exciting and why we think we can solve that differently, is because of this invention of essentially breaking down speech and outputting speech without an intermediate form factor. How people traditionally broken down speech. And when you now come to specifics to language translation, if you think about how traditionally most people did it, one was they obviously broke down speech to text and then they analyzed text. And especially in language translation, the complexity goes up.

4:40If I speak French and you speak English, first, my French has to be broken down to French text and has to be translated to English text and then finally English speech. So there are four different friction points. Every friction point adds to latency. And that's how speech to like doing language translation always took a lot of time. And used to refer it as near real time. But as a speech researcher, for me, hey, is it either real time or is it not real time? There's nothing called near real time. Right. So, yeah, I think those were some of the challenges today. One was they were doing it from speech to text and text to speech.

5:23That modality never worked. The latency was sometimes as high as 10 to 15 seconds, especially for languages like Japanese, where you need to understand more context before you actually start converting them or translating them. It was always much harder. And the other thing that I saw was because the output was synthetic, all the natural emotions were lost. Right. Right. And I'm like, hey, why do you bring translation as a technology in when I don't sound like me? I never understood that whole concept of making you sound like somebody else. I genuinely think any communication system will thrive if you don't sound as natural as how you are.

6:06Right. So I think that was that emotional nuance was, I think, was a major missing piece on most translation technologies today. And then even like big tech companies like Apple came up with the AirPods. It was a big release a few months ago. But and I was one of the first few people who went and bought it because I'm excited about it. But even there, it's about you speak something, you see the output on a text and then you communicate or show it to that other individual. My challenge with most translation systems today was there was not a synchronous form factor. It was never natural. And the output, they tried showcasing it in multiple different modalities, which is never continuous and never simultaneous.

6:54And that's the reason why, though the industry and in startup world, we refer to this whole thing called TAM, the target addressable market for something like translation is so massive. has always been untapped and fell short because nobody could build something which was natural, which was actually real time and something that had a very natural flavor to it, was never built because I always genuinely believe emotion should never be an afterthought, be the heart of every conversation. And I should always have the ability to choose how I sound, right and not like somebody else so yeah those were the i would say the reasons why traditional translation technologies never worked and with our invention of actually breaking down speech to speech without any other modalities in between we genuinely think we have a realistic shot of actually making this science fiction a reality well and so i mean that's fascinating because i understand the whole idea as you say kind of like their speech and then you kind of add a you have to recognize the language that it's in and then you're going to translate it into that kind of text and then then then there's the the again the translation if you go in speech to speech end to end as it were then you've got to then translate it back and then then have it go through a voice and a creation a modulator that goes up and then you have all these spaces and then that That creates all the latency or the pause, as it were, the cog grinding in the background, something you don't kind of hear.

8:31So where did the phonetics thing insight kind of come from? Is that based on sort of academic sort of research? Are those kind of patterns? Or as you say, tell me about this happenstance thing that you kind of came across. No, I think it was obviously a lot of academic research. I think Sean, my co-founder, again, a genius kid, I think when he was studying in Stanford, he wrote a white paper of, hey, if you have to truly translate speech in real time, speech to text is not the modality or the form factor you should use. You should find something else. And he found out a way if you can actually break speech into phonetic sounds, you can actually do speech in real time.

9:11Right. And that was the happy accident. Me meeting him was a happy accident and I got excited about this. And then we started thinking about, okay, what could be applications for it? And the first thing we thought of was accent because he had a close friend who went through some kind of bias and discrimination. So we applied it for accent. But the holy grail for all of us was that, hey, Sean, you're Chinese. You're from China. I'm from India. You speak Mandarin. I speak Hindi. if we both speak in our native tongue which is where we are most comfortable in understand each other very clearly that will be the holy grail right but i think building accent first was important because we before you take that leap to change a different language let's kind of understand of breaking down when two people are still speaking the same language because there are so many things in communication right you have background noise you have background voice, you have various dialects, you have people who are tired, people who actually have disfluencies like stuttered.

10:12Let's solve all those nuances first for one language. And once we have solved it for one language, now let's take it to the next leap of actually doing it with a different language. And so the real-time sort of speech-to-speech translation, I mean, are you doing everything straight off the bat, kind of all languages spoken by all peoples, or are you focusing on a few kind of initially and if so kind of why why those ones i i think uh the initial requirement came from one of our customers we work with one of the largest healthcare companies in north america uh and they were talking about five languages which are most important for them because they want to save lives and they're like hey these are the five most common calls we get with requests of wanting to speak in french people want to speak in spanish people wanting to speak in Tagalog, people wanted to speak in Japanese and German.

11:07And then it was Hindi, I think. So they give us a list of 25 languages, which is a part of their queue. And then we went and validated because when you build something which is truly innovative, yes, you always want to make money as founders and as investors, you are responsible to kind of make money for them. But my first principle thinking is, hey, let's build something that people truly want that can make a fundamental difference to people's lives. Once there is adoption, money will come. So that's what we went and we validated across multiple healthcare companies. Then we went to non-healthcare companies with much larger Fortune 100 companies and they all gave their wish list.

11:51Somebody said, Russian is very important for us. Somebody said, Cantonese is very important for us. Somebody said, Mandarin is important for us. And somebody said Korean is very important for us. So we took the whole list and we, for now, distilled it down to which are the top five and which are the top 25. Right. So that is what we launched with our app. We said, hey, let's build this in public because if you're building this for the world, let the whole world give us feedback. Right. So launched it on iOS and we launched it on Android on 15th October. And it's been a very humbling experience.

12:23It's been six weeks now. We have users from 76 countries now who have downloaded the app. And it's funny, I see a lot of calls which are shorter, which I think are more business-to-business calls. And then I think I see four-hour conversations, five-hour conversations. That's when I see that they're either talking to their family in a different location or they're talking to their girlfriend. So, because I don't know, when will you talk to somebody for four to five hours? So it's probably for that. But it's beautiful, right? I think now we're building it in public. We're getting a lot of feedback.

13:00A lot of them has been very positive. I think the most common feedback we've gotten that, hey, I still sound like me when I speak French. I still sound like me when I speak Japanese. So we have five languages which we call a score, which we think our accuracies are very high, like 95 plus percent, like Spanish, French, German, Mandarin. And I think Italian are the first five we've launched. And then we have Portuguese and a bunch of other languages which are part of the top 25. We call it beta for now. But over the next 12 months, move all the beta languages to general available languages like Spanish.

13:38and i think we're the best at english spanish pairs and english french pairs i think that's where we have the most demand right now but then japanese and korean and portuguese and german are like fast follows i would say yeah yeah that's that's the overall and so let me check my understanding does that mean that within the app somebody can either say make a call and then switch on the kind of the translation and they say i'm speaking one language to another person speaking another language and then it will do the simultaneous kind of translation on a phone call or you could be sat side by side and you could have it run there and you could run i speak this way it translates and come and speaks out in a different well speaks like me but in a different kind of language to that person sat next to you i mean is that kind of roughly kind of how it works Yes.

14:29So I think we've launched two modes right now. One is we've tried to mimic how you would do a video call and a FaceTime today or a WhatsApp today, which is a very form factor, right? You will see your face, you'll see the other person's face as a thumbnail. It's exactly that form factor. So that's very relatable. So you do a video call, you can switch off the video if you want, or if you want to see the other person, you can do that as well. but what we've added to that is sometimes it's also helpful because we want to get feedback from the users if we are doing it the right way we've added the translation layer also on it so while you choose your input language you choose the output language so if you're talking to somebody who's french you choose your input as english and output as french and you speak in english naturally like speak and then the person on the other side will hear it in french but you'll also see the translation on the screen so that you can validate it or the other person can validate that, hey, you're saying the right things.

15:27And then there is a solo mode that we did. It was inspired because we all travel and I was supposed to travel to Japan in August. And I found that if I knew Japanese or if I knew how to communicate in Japanese, my life would have been easier. I just came back from Paris last week and I use my app all along. It's called a solo mode where you're standing next to somebody and you just switch on the app and you speak in English. And the person who can see your screen and hear in French and that person will respond back and you hear it back in English. So it's called a solo mode. And that's mostly done for traveling.

16:06And we thought end of the year, the whole world would travel. So they might find a use case for an app like this. that's how we've launched it for now. It works. The goal is to get it under a second. For now, it takes between one to three seconds. Languages are faster. Some languages are slightly slower. We are also in the mercy of the internet, unfortunately. And we've built a version where it should be very easy for you to use. It could be a very crowded place. It could be a very noisy space. You still have to do it very, very accurately. So we have focused a lot on accuracy, on it being 95 % plus accurate, at least for our core languages.

16:45And we've not optimized so much on latency yet, but at least the feedback we're getting from people that, hey, it's very easy, that one to three second window, it just has a cold start problem. But once you start talking, it's so simultaneous that you don't hear any lag. So we've seen very less instances of crosstalk or overtalk that generally happens when two people are communicating like me and you are. we are seeing a lot less feedback on that okay but my goal is to get it to sub one second and we're working on it we're very close i would say also i mean it's uh you know as you know i focus on the um on the kind of service and experience space and i'm sort of like i'm thinking about that and i'm thinking i'm really excited by the the possibilities that this could offer in that particular sort of space in terms of how brands serve their customers better in many kind of ways, particularly kind of like brands that are working cross-border, is it where?

17:44I mean, I know you've done almost like a general release, like a public release, but it feels like there's kind of like some really big application areas in that sort of service and experience sort of space, right? Yeah, I think so. We're launching the B2B version for enterprises in Jan. There are two ways we are doing it. One, we are initially launching it on our cloud just for people to test it in an enterprise environment. But ultimately, we want to containerize this algorithm and give it to enterprises because we work with a lot of Fortune 100 companies and they are not very comfortable having their data and customers.

18:25communications on a third-party cloud. So that's our ultimate goal. So we think we'll probably be 90 days away by April. We can potentially containerize this and give it to enterprises. And the applications, like you rightly said, I think we've had so many interesting use cases come along. Obviously, CX is the first because what brands are now seeing is they have a bunch of subject matter experts. But whenever there is a customer on the other side who's not comfortable in speaking the same language as the expert. They have to bring in somebody else on the call and the whole experience gets disrupted.

19:00Now, this expert with one click of a button can speak in any of the 25 languages, mostly covers a large part of enterprises' requirements. That's beautiful, right? So you can speak in French, Japanese, Portuguese, all with one click of a button. That is very exciting for brands of all sides. But I think the biggest interest we have gotten is one is healthcare because for them, it's a very complex process. And think about insurance in the US, right? It's probably the most complicated subject to understand. And people take years to become experts at it. And it's about saving lives, right? I think one of the first applications we are testing out is with 911.

19:44Today, it took seven minutes for them to get a translator who could speak the same language as the patient and somebody would die in seven minutes. Sure. So I'm personally very motivated on making that work. We're seeing, we're going through testing right now. We've gotten great feedback on that. And a lot of large healthcare companies are saying that, hey, we want to install this. Like you have a point of sale system. We have a point of voice system. We want to put it on every bed next to a patient. If a patient is more comfortable speaking in Spanish or Japanese or Korean to a nurse, to a doctor, right?

20:18We want everybody to get access to the best healthcare. So that's another massive rollout of applications I think will happen. CX is obviously very natural today. Most brands have come to us and told us that whenever we bring another person on the call, we've seen our CSAT dips by sometimes as high as 50%. percent. And the time taken to solve the problem is 2x or 3x more time. And generally, it's never solved on that call and people call back again. So the first call resolutions will go up if we have the same expert can switch a button and speak a different language. And then I'm personally excited about education.

20:53I think my big thing of launching Sanus was to bring the whole world closer together and people being able to communicate very freely imparts a lot of knowledge to each other right i genuinely think that the future we are building uh is where nobody needs to choose to speak a language which they are not comfortable with right to communicate with one another right so yeah i i think one of the big use cases now is uh there are a lot of teachers in India, there's a massive education company in India that have over 10 ,000 teachers. And a lot of them were, especially for languages like physics and math, for subjects like physics and math, they're more comfortable if they can communicate in Hindi or in their native tongue.

21:41And they have users in Japan and Korea, in Europe and also in the US. Now imagine unlocking massive opportunities to people, to those 10 ,000 teachers. so I'm most excited about education as a use case I mean so it's funny because I mean I've been sort of exploring this sort of space for a few years now I remember talking to somebody back in 2021 about this topic and they reckoned that they were we were 10 to 15 years away from as I described at the time there's like this Babelfish application like you know what we any sci-fi buffs will recognize from this idea that well you've got this little fish called babafish you pop it in your ear and you speak to anybody you know anywhere and that you know speak to anybody speaking any other language from anywhere in the universe and that was featured in the hijackers guide to the galaxy and it sounds like this is becoming much much closer so that's really sort of exciting in terms of breaking down those barriers I think I keep telling my product team on it and my tech teams on it.

22:47Babelfish used to be science fiction. Now it's a product roadmap. That's the evolution we've gone through. So I'm very excited about the possibility of this. I think it's already real, right? I think we have users from 76 countries who are using the product today. And this will be released to all the enterprises early next year. This is as real as real can get. I think there's a little bit of latency side that we have to improve that we will over the next one year. I think it's real. And that will truly unlock global communication, in my opinion. And the more applications I keep thinking about, it becomes so more meaningful.

23:28I think there has been an explosion in content creation the last 24 months with TikTok and Instagram. I think think about every podcaster everywhere in the world. If you can give them a tool like Xanus and they can speak in Russian, Cantonese, Mandarin, Japanese, which they're most natural in creating content. But now they communicate to the whole universe in a language that they understand. It will open up, especially most of the developing markets in LATAM and East Asia. That market will explode with the product like Xanus because it generally gives everybody a voice, which is the biggest inspiration for Sean and I in building Sanus.

24:07I mean, I can imagine it. It's sort of like you do a podcast like this and then there's maybe a little kind of button somewhere in the interface and go switch to X or switch to Y. And then you end up listening to it in your own language, which is fascinating. And one thing that is unique is even today, some of the technologies are available where, especially for a podcast thing, which is not async, which is not real time, where you can hear it at any time you want in a different language, that always exists. But what did not exist was keeping your natural tonality flow through, right? You should use it.

24:44And if you choose to make this podcast available for your German audience, for example, it will sound very much like Adrian with all the idiosyncrasies of how you speak and how you present and how I speak and how I present. It will follow through, right? Which is why I keep coming back to our invention when you break down speech to phonetic sounds, phonetic sound is no language. So it'll make sure that all your emotions flow through exactly the way it's intended to flow through, right? And that's what we are excited about. And everybody who's, when you download the Sanus app on your iStore or your Android, you have to go through a one minute setting up process where you read a paragraph in your tone and we have selectively chosen specific texts for you to read.

25:36And the intention is to bring out the right emotion of how you speak in that phrase sentences that we ask you to read. And it captures it so beautifully that everybody who's used it, that's the first feedback they leave in a review, that it sounds just like me. I didn't know how good I sound in Portuguese. I tell you what, that's probably something we should talk about kind of offline is like, how do we do that kind of like almost in a, I don't know, in the interface is almost to show people what they are to the possible kind of like might be, but that's probably for another conversation. But no, that's just, I mean, just as really exciting, I can see all sorts of different applications and sort of, as you say, breaking down those kind of barriers, you know, between languages and, you know, and just making communication kind of much, much kind of like smoother.

26:25So that's kind of, that's really, really exciting. but um i don't know if they i mean that's all i really wanted to understand because i just wanted to have you on the show because that that was the thing it's a bit like oh shah has got this babel fish sort of thing and it's very real kind of like very near kind of like to to being as perfect as douglas adams described it in the hitchhiker's guide to the galaxy and so i mean it's it's brilliant to hear the where it's where it's gone to and i'll definitely go and kind of check out the app and i'll encourage everybody else to kind of do the same sort of thing and look out for the general release for B2B applications in the new year.

27:01But Sharath, is there anything else that you wanted to add that I've missed out before we can start wrapping up? Because there's quite a lot there. I know it's a lot to take in, but I think one of the things that I would message to most of the enterprises that we work with, clarity and authenticity does not have to be trade-offs, right? With Sarnes, we want them to be the new standard where we don't compromise on anything. You solve exactly the way you want to be sounding, right? All your true emotions actually flow through. And what we see around us is, if you look at the VC ecosystem, and that's usually you read between the tea leaves, most speech companies are getting massive rounds of funding, right?

27:49and we all now know that voice will be the new keyboard right number of i'm sure you must have felt the same the number of people leaving voice notes today versus people never used to leave voice notes even 12 months ago right it's almost like nobody wants to type anymore people just want to say people want to communicate people want to express themselves and i think with the tool like sanus with everything that we are building right from the bottom of the stack to eliminate background noise, background voice, to harmonize dialects, now with being able to translate languages in real time, this will be the new voice standard.

28:29And that's exciting for me and everybody in my company building this. And yeah, when you make mythology or fiction a reality, it just feels very special. And we are very proud and happy to have given, have gotten the opportunity to invent something new. Nice. that can truly change the world. And it's always fun to build something which is applicable to 8 billion people. Because always, you talk about your time being, hey, this is applicable to half a million people, a million people. You never say, hey, what we've built is applicable to the entire planet. And that's what we feel that we've done, especially with language.

29:10So we are extremely stoked about it. And yeah, I would love for all of you to, I think you have a lot of people who watch this podcast. So I hope everybody uses it, gives us feedback. We'll go through the same learning curve that we've gone through with noise and accent. It's usually six to 12 months. We'd have numerous humbling experiences, a lot of feedback, but we chose to build it in public. We're not afraid to take feedback because it'll only make us better. I truly think everybody should be using Sonos. Perfect. Well, I'll definitely kind of check it out and encourage other people to do so as well.

29:47But so Sanis, before we go, I just want to ask you some quickfire questions to kind of boil some stuff down. So the first one I want to ask you was to complete that sentence, or this sentence rather. And the sentence is, if you want to improve your customer experience, Sharath says, do this, dot, dot, dot. Complete that sentence. Over to you, Sharath. What's your best advice? if you want to improve your customer service use sanus because i think sanus is becoming the verb these days and sanus means clarity sanus means authenticity smart sanus means say no to bias say no to discrimination and sanus says sanus means communicate very clearly right that's the message i'll give it to everybody that hey if you want to if you really care about your customers you should be using sinus uh and we also have a free version if nothing else i know everybody's going through budget constraints use our noise cancellation technology it's free for the whole world and that's the least you can offer to your customers awesome thank you now so for my second question and obviously it's a punk one because it's a punk cx podcast what company or brand do you think takes more punk approach to customer experience and why?

31:08I hope they become our customers also. So that's why I'll probably say it out loud. I think when I think about customer experience, I've always thought about American Express. I think they take the punk approach to CX where customer is the king and they treat them like one. Like every time I've got on a call with Amex, I've always felt that I'm special. I hope they use SANA someday and they should, but they've generally done exceptionally well. I think they are my true North star when it comes to customer experience and everybody should look at them that way. Awesome. And the final question that I've been asking people over recent months is, tell me a good news story because we seem to be kind of laden with lots of, you know, doom scrolling, kind of fueled by kind of like bad news, you know, on a daily basis.