$6.6B AI CEO: How to Make Your First $10,000 with AI | ElevenLabs CEO & Co-Founder, Mati Staniszewski

0:01This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new.

0:06Marina Mogilko:It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up required, compatibility and availability varies 18+. We paid about$5 million to the entire community. Meet Mari, CEO and co-founder of Eleven Labs, a company that has grown into a$6.6 billion leader in the voice AI space, shaping how we talk, work, and even earn money. They've created an entire voice marketplace.

0:44Marina Mogilko:Now anyone can clone their voice and earn passive income. Can you name some opportunities that you see that can make people a dozen amount of money so they can make a living like 10k a month? Something that's immediate. Business and you just want to make good money. I would try to take those voice agents and go to, let's say, local doctor's office. 11 Labs built the world's most realistic voice deck. The question is, can they control what happens next? Most of those companies just don't know this is possible. You don't have to be the coder, you just need to... If my voice is authorized to use my credit card to buy anything and then somebody just uses the resemblance of it...

1:19I think it's going to happen, but...

1:22Marina Mogilko:Hey guys, welcome to Silicon Valley Girl. We have one of the guests today whose product I've been using for a while now, so I'm going to ask a little technical questions as well. But please welcome Mati from Eleven Labs. Thank you so much. Thank you so much, Marina. Great to see you again, and thanks for having me. Yeah, thank you. I feel like you're one of the pioneers of this AI industry, because when I ask people what apps they're using, when I'm talking about apps that I'm using, I always mention Eleven Labs because it's been a lifesaver. I wanted to start with a question about the role of voice in AI.

1:53Marina Mogilko:So what it feels to me is that 2023, you know, we started adopting ChatGPT. It was all text. And then these voice capabilities became more and more powerful. It understands what I'm saying now. It understands my accent. If I mispronounce something, it still gets me. Do you feel like we're moving into the era where voice is our main tool to interact with AI? I mean, 100%. I do think that voice will be one of the key interfaces to the technology around us. And that shift is happening. like you said, it's like a few years back, you wouldn't even dream of this being possible. And now I think it's becoming a reality where it allows you to transfer so much information more than the text.

2:33You can get the emotionality, the inflection pattern, the imperfections reflected in the voice, which of course makes it easier for the, if it's an input, for the technology to understand a lot more about the setup or what you are trying to achieve. And then if you hear it back as well, I think it's a lot better and more pleasurable experience as well.

2:54Marina Mogilko:How do you see voice transforming businesses? Do you have any cases where people are using voice to generate leads or convert leads? There's definitely a few different areas, whether it's on the more classic customer support use cases where you, instead of having a old IVR system or no system, you can now deploy a voice agent that will take the calls instead. and will both delight the customers on the other side because it understands you, it's quick, it's good, but then also just performs better. And then outside of customer support, we are seeing that across the entire lifecycle of the user journey.

3:32In some places where it adds an experience that wasn't possible before, in a simple case is inside of the product or even outside of the product, and you might have seen back in the day there was those widgets for chat. Now you could have a voice agent that helps you navigate through the product experience. So it becomes your partner, programmer, product person that helps you navigate through that lifecycle. And you also mentioned, so of course some of the big pieces is in inbounding and outbounding. We actually use it ourselves in 11 Labs too, where of course we do have a standard flow. We have people that will answer their reply and take a phone call too.

4:14But if you want to go quicker, you can speak straight directly with our agents to understand our product offering, understand our pricing, understand what you can do with the product, which helps you accelerate through the pipeline, depending, and sometimes self-disqualify if you are not the right fit for our product offering. And sometimes helps you accelerate, okay, this is exactly the set of use cases I can do. This is how I can deploy. And then routes it to other people.

4:38Marina Mogilko:So it doesn't actually convert? In some cases, it does. In some cases, as a quick step back, we have a few different tiers. We have like a business tier and an enterprise tier. So it does convert immediately sometimes to the business tier program. Because it's a preset. Because it's preset, it's self-serve. On the enterprise side, we all still run KYC checks, so it doesn't do that immediately. But on the business one, it does. And then we've seen some of those voice agents also from a lot of the technology and platform we built help in a completely different non-commercial aspects too. Quick follow-up question about the sales process.

5:16Marina Mogilko:Have you measured the conversion percentage into sales with the AI voice salesperson? We did, but given it was, and I don't remember the number off the top of my head, but given there was an alternative before, it would have been just waiting. So it was just a net new amount of leads. And we received so much inbound of using a lot of the products, which we are lucky to have, that it helped us just convert so many more leads that we would have otherwise taken weeks, months, or maybe never gotten into. How can I set this up for my company? The easiest one would be to register on our platform. So that part of offering, and we have two key offerings, is our agentic platform offering.

5:56You jump into the platform and we help you abstract two elements. The first one is all the research or experience complexity. So we help you connect the speech, the LLM elements, the text-to-speech elements. So the agent speaks in a smooth and a quick way. So it's a very low latency, a reliable part on that side. And then there's a second part where you will need to spend a little bit more time on bringing your business logic in place. So example could be what's the knowledge base of how your business operates or what are the questions you want to be asked? What are the materials you want to surface?

6:31So you would bring that into the platform. Then we have a set of workflows that you can set up. Effectively, imagine like if this happens, this happens, or if this happens, I want this function to trigger. This could be if someone is calling me and I want to schedule an appointment, we have a predefined workflow for you to be able to do it. So it can look into your calendar. Okay, I'm selling a course, basically.

6:54Marina Mogilko:Like what Language Trip does, we sell courses. So basically, we want to be able to sell courses to people. Can I do it in different languages using my voice? You can. Wow. So you could, and so it's selling the courses and the people would call in, buy the course and off to the go and maybe they onboard with the agent later on to help. How do they buy over the phone? Do you send them a link, ask for their email or they just, yeah? Depends, but the simplest would be what you suggest, which is we do have an omni-channel solution where you effectively get a link as part of that and you can leave additional details or you have a follow-up on the email of like a checkout subscription for the course.

7:32so both of those would be possible or you could depending on how that website is set up you could effectively embed the agent on your website so it helps you redirect to the subscription page it guides you through it and they check out themselves live with the agent that helps them fill in the form but like you said one of the great things on the function side is that you can you can you can switch languages you can hand over like that's fascinating for my business So, I mean, you've been pioneering a lot of that language learning work. And I think this would be amazing because both it would switch the language and it would switch it with your own voice.

8:08If that was your own voice, it continues speaking in that same manner. And then, of course, the last piece is all the integration. So we support integrations.

8:17Marina Mogilko:I didn't know you had it. Congratulations. Thank you. It's one of the big, so maybe that's a good cue for me as well. Because when we started the company, we, of course, started from pioneering of the research and the speech side. So text-to-speech, voices, and then we expand it to speech-to-text, orchestration models, now music. But as we think about the research, it's always how we can push the audio frontier forward. I love how you found this new opportunity and now it's a bigger chunk of your business, as far as I understand. How much would it cost for a business like mine, small business, to have AI answer the calls and sell?

8:50I think the, and of course it depends on the volume, but I think what hopefully will happen is that both you will see more people coming through. And if we set it up in the right way, maybe this will mean even opening up the channel, which over time hopefully means even more calls. But I think to start, it would be in order of hundreds of dollars per month.

9:07Marina Mogilko:It's also IP calling, right? Is it integrated in that? Yes. So we integrate with Twilio or telephony systems. So whatever we're using. So you can bring any phone number that you already have and it works. I don't know who, currently, do you already accept any of the calls coming through the telephone too, or it's all on the website? We mostly try to navigate them to WhatsApp because a lot of people who are calling, they don't speak English, so they don't feel comfortable. But if we advertise that it's, you know, Marina's voice, AI, nobody's judging your accent. Because I feel like when people even talk to me, if they're not a native speaker, the first thing they do, they're like, I'm sorry, my English is not as good as you.

9:45Marina Mogilko:And I'm like, it doesn't matter. But I feel like even using English to make a phone call is such a huge barrier for non-native speakers. And I feel like if you understand that you're talking to AI, it just makes it so much easier. That's true. It doesn't judge. You can do mistakes, which is maybe, you know, like there's a completely other aspect where you, of course, been helping people learn languages for a long time. But maybe there's even an aspect where they could practice speaking their language with you, which would be like a you know kind of a slightly different of course deployment but completely possible where you can give them tips improve and effectively create a marina's duolingo that people yeah have dynamic experience with which is another kind of incredible area that's growing in that tech space yeah let's let's talk about that part so we talked about deploying 11 labs to work as a sales agent let's talk about like i have this number here where you paid two million in royalties to people who kind of share their voices with 11 Labs.

10:47Marina Mogilko:Can you talk about that? How can people start making money by sharing their voice with 11 Labs? So it's one of the efforts we launched in the early days where we effectively created a voice marketplace, voice ecosystem, where every person can create their own voice, go for authentication flow. You need to record roughly 30 minutes or more of you speaking. then you have a perfect replica of your own voice that speaks in the language you recorded plus all the language we support so you have usually 30 or so different variations now with the new model we are releasing will be 70 so so you have the voice that that's now available for your own use and then if you decide you can share it to our marketplace and if you share it to your to our marketplace a specific period of time specific conditions of what you are sharing it for then other people can use it across 11labs ecosystem and when your voice is being used you get paid back as a result this way we have now almost 10 000 voices that people shared and created what is incredible is it spans so many different languages accents um different styles so like now if you are logging to the to the to the platform you just have this incredible platform voices and we pay uh pay voice uh pay voice time back so it was i think two million dollars at beginning of the years that we paid back and now um i think last time i checked was a few months ago we paid back five million dollars to the entire community how much does average an average voice creator make it depends uh of course you know like so it's like the the like probably in total approaching close to 10 million dollars and we have close to 10 10 000 voices um so that would be like you know if you if you if you take the average uh but i think it it's um especially given a lot of the voices got are kind of new and it takes a little bit of time before they take attention you also to actually make it successful ideally you try to engage some of the community around that they can see the voice whether it's the discord or reddit some of the other forums it definitely helps break through that initial and if not over time we also try to surface new voices and and get them out in the audiences so it really depends i think it'll be a lot of people in like a few hundred dollars per month category.

13:02And that's probably what you could expect if you do a little bit of that effort and what you could earn. However, I think it's true that it's, if your voice sounds very similar to other voices, it's very much larger.

13:16Marina Mogilko:Yeah, it's interesting how many voices, like in general, do you... And how many can you distinguish? Yeah. But if you have a unique voice, if you have a new accent, Yeah, if you're a creator, right? Exactly, then it can be incredible. Our first voice, one of our first voices that got shared and it was a Spanish voice that had a very deep way of speaking, the prosody. And that voice became one of the most popular, not in Spanish, but in English-speaking countries and became like our top 10 voice where it was just such a unique and different experience. Interesting. Let's talk about the nuances of cloning your voice.

13:53Marina Mogilko:Because for example, so what happens sometimes in my team, we clone my voice using all the different mics that I have. But sometimes we insert it and it's still slightly different from the video because the way we use it is that, you know, we recorded something here. I recorded some brand deal or whatever, and then I start traveling and they're like, could you rerecord this phrase? So we just take a piece from the video, redo it with a phrase that the brand asked for. But then we insert in the video and it's slightly different. Like the it sounds in a different way. Are there any ways to fix it?

14:25Marina Mogilko:Yes, of course, we ask it to remake it again, but it's still not exactly what we recorded. No, it's of course a tricky problem where when you create a voice, you most likely take the voice throughout the entire video and then you create that voice. And then it is effectively the average of how you spoke around that video. but in a given scene, you will have maybe changed the intonation pattern a little bit or the emotional pattern slightly off that average. The ideal way would be to affect for us to do more of the conditioning on what you do pre and post in the video. So we take that more of as an input and we try to morph it in in a slightly better way.

15:08And then there's a second thing. Sometimes even though I know you'll try to clean up the voice and then add the background sounds, background effects, they might be by just the process be mixed in and then it doesn't smooth entirely. So from our side, what we hope to do over time is that as you insert those videos, we can precondition it after three seconds and after and it will sound better. So that's something we are working.

15:32Marina Mogilko:Okay, so you have that feature. So I upload the video. So we are working on that. Not yet. It's not applied. But it's going to be the big piece. We need that feature. We definitely need to bring it there. I think in the short term, what you mentioned is what we see as the most common pattern, which is redoing and regenerating. But the other thing you could try is try to, instead of taking longer audio samples across the video, just take even a few seconds, which I know sounds like maybe it will be a worse result. But if you just take a few seconds from that fragment and create that lower quality version, it actually could sound pretty good.

16:06Marina Mogilko:Okay, thank you. So where do you see all of this going with people recreating their voices? Will everybody have a clone in two or three years? Because we could have thought about 11 Labs when I heard about it like two or three years ago, right? I couldn't think about a salesperson using my voice. Now we have it. What do you think is going to happen in two years? What is this new use case that this all is going to unlock? Interesting question. Of course, we are seeing entirely new ways of interacting with voices. So I do think, yes, you will have your digital AI voice. And I think even a step further, you will have your own digital voice agent that does things for you, that you want to make sure it's authenticated, people know you operate.

16:49So, you know, like we spoke about the example of how people can call in, you can configure a voice agent. But I think the other side will be also true. I will have my voice agent. Calling the bank?

16:59Marina Mogilko:Because they use voice authentication, right? It's going to change. I think that's not the best mechanism for the future. Anymore. Not anymore. But like say you want to book a restaurant or follow up about appointment. and a healthcare, and you want to make sure that they know your most recent details or that it's confirmed, I think you will want an authenticated version of VoiceAgent. I'm saying they're authenticated because, like you say, most of the verification that they don't will fail, and you want to know that it's a permissioned voice. So you will need to start embedding watermarks and metadata around that.

17:34But I think to kind of go back to your question of where it all evolves, I think there will be like an interesting pattern where, and I think it will happen on both sides as a user, but also as a business, you will be able to serve so many different voices to your customers or you as a customer can decide what voice speaks to you. So to speak for specific examples, we are working with a company in Korea, Korea and Japan, a multinational company there, which has a very different age groups calling in. Instead of older patients and then much younger set of people. and they want to serve, depending on the data, the number that they are calling in, serve different voice to that group, both in terms of how it speaks, how it sounds, but also the style and how it speaks.

18:25Of course, it's a generalization, but roughly they wanted that if an older person is calling in, the voice speaks much slower, much calmer, less emotionality. It's a younger person, much quicker, a lot of higher amplitude of emotions. And I think this same pattern will start happening across everything where if you are calling in a specific region, you might have an accent of that region. If you are calling a restaurant that's maybe representing a specific cuisine, you get a voice of that cuisine speaking with you. And maybe there are like variations of all those different types, which can work.

19:01And then separately, as a person calling into any of those services, you could pre-select that too. So if you are calling a bank and you enjoy speaking always with the voice of this specific style, then you can select it and that voice will be the voice of your preference. We've seen this happen in a company and also in Asia where they created effectively a travel agent or like a Google Maps competitive product where you can select a voice that narrates your direction. And one of the voices they selected became like viral and everybody wants to use it now in the travel directions because it just made for such a better experience.

19:44So if I extrapolate in the future, I think there will be a lot more both personalization, but also selection that you can choose into. I think 100 % true. You will have your own authenticated voice that you can use for your voice agent, for your content.

19:59Marina Mogilko:That has all the information. That has all the information that you can use. That's very interesting. I like that part. Like having my voice call and be authorized to use my data. How do you talk about impersonation with voice? If my voice is authorized to use my credit card to buy anything and then somebody just uses the resemblance of it, will there be any metadata that could be detected by other systems? And what would it look like? First of all, I think it's going to happen. I think the assumption we should be going with is that where you will have good actors, good technology trying to avoid it, but then there will be also more permissive technology and bad actors trying to abuse it with any technology shift.

20:44And already now, there is a lot of open source technology, other commercial technology, which doesn't have the same safeguards that could clone your voice and create a mimicking and that sounds like you. So I think any system that we think about devising in the future kind of needs to assume that you can create a clone of a voice and make it a perfect replica. Now, of course, if you're like us, I think about 11labs, we can and we do add safeguards as you create a voice, so you cannot do that. Or if you do, we detect it and moderate and can flag it internally if we are not sure. So whether it's being able to trace everything back to the account or moderate what text was used when it was trying to do a scam.

21:31But to core off your question, like as we think about the future, the ideal system, and it would require cooperation from a number of parties, would have three different layers. And then the first layer is instead of trying to check for AI, you actually check for human. That's easy for me to say. Of course, there's like, how do you check for humanness? But a simpler step, original step could be that you, on the devices that you use, so on my telephone or on my laptop, I am encoding that this is my phone, my laptop. When I'm calling from it, it's being decoded on the other side. They know that this is the device I use, so most likely this is me.

22:07That's the first layer. Second layer is actually what we spoke about earlier, and that's possible, you watermark authenticated AI. So if I'm using a specific tooling, the tools that can add this watermark are known, and I watermark that within the content. It's not super straightforward, especially in audio, because if you add a watermark in content, it can affect the quality of the content itself, but it's roughly good. And that's the second layer. So you check for authenticated AI. And then the third layer is by default is AI, and you assume it's AI. So if it didn't pass the first or second layer and you see content that hasn't been authenticated or proved for being a human, it's AI by default and you don't trust it.

22:53And then you can add more mechanisms on top of that fair layer where you like try to explicitly check or add additional signal of like, ah, this is real. But that would be a mindset shift where today if you look for content, you're like, oh, maybe this is AI. It should be the opposite where it's like, oh no, this is definitely AI. Is it maybe human or is it maybe AI that was created with creator's permission? And then you have those cases in between that will be interesting as you of course create the content And you mentioned that sometimes if you need to re-record, you might create an AI voice, of course, with your permission.