ElevenLabs' Mati Staniszewski: How Voice Becomes the Interface for Everything

0:02So I love line charts and bar graphs as much as the next guy, probably more. The story of 11 Labs is also interesting from a human perspective, but as you started a company with a childhood friend. So maybe take us back to 2022 or earlier and just tell the human side of the 11 Labs story to start.

0:22Mati Staniszewski:I have the most luck in the story of 11 Labs because, well, it started in 2022. It feels like it started 17 years ago when I met my co-founder, Piotr. All the names in Polish are complicated, luckily, for us. But we met in high school, became best friends, took all the same classes together. And then through the years, did everything together. So we traveled together, studied together, worked together. And time is on our side. We are still best friends. It's working out. And part of what started 11 Labs is inspiration from where we are both from. We are both from Poland. suburbs of Warsaw. And there's a very peculiar thing in Poland.

1:01Mati Staniszewski:If you watch any foreign movie in Polish language, all the voices, whether that's a male voice or a female voice, get narrated with one single character. So as you can imagine, pretty terrible experience. You have literally one voice narrating everything. It usually also, on purpose, is kept in monotone. So you are meant to interpret your own emotions for that content. And while we grew up with this, this is still happening today for majority of content. And that kind of opened our eyes into one of the clear things across the domain, across audio domain, across the future, will be this ability for everybody to speak any language with the same emotion, the same intonation.

1:41And we started diving deeper into that problem and realized the problem of audio exists in so many other domains too.

1:47Mati Staniszewski:whether that's narrating the content around us, whether that's the books not being available in audio form, whether that's the news articles that we could read, whether that's that language barrier, or in the future, as we heard in the previous conversations, the future where human noise, the robots are around us, the voice will be the primary interface to a lot of the technology and something we would love to fix and solve. Excellent. And Eleven Labs builds frontier models for audio. I think there's a paradigm now where to build a frontier model, you have to start with hundreds of billions or billions of dollars and then figure out the rest later.

2:23Eleven Labs did not take that path. May I talk a little bit about your approach towards building this company, why this hasn't been replicated? Is that even possible in 2026, et cetera?

2:36Mati Staniszewski:I think that continues that great lack in timing because we started in 2022. to for those of you working the domain at the time. That was a year of crypto and metaverse. Nobody was still working on the AI side. Even further, people were starting to work, of course, on the text models, on the visual models. But audio as a domain was still considered a big niche. There's so few researchers in the space working on that work. So for us, that was a good part of picking that domain where, A, we were excited about where that future is called. We felt that people around just didn't realize the value of that domain.

3:14Mati Staniszewski:But three, the requirements of what you needed to solve were very different. The audio models were smaller. So you don't need as much compute as you need for some of the other sister domains. The data needs are big. But while there's a lot of audio data, we knew that the thing to actually get that audio working, you will need to figure out how to transcribe a lot of that data and annotate all the data, which we knew we can do. And then ultimately, it all boiled down to architectural side of can we solve that part in a good way. And here, my co-founder is one of the smartest people I know and a great researcher and has been able to assemble some of the best people in audio to help us.

3:52Mati Staniszewski:And we took a slightly untraditional approach at the time. We started in London. We had a lot of people between London and Warsaw and started a company in a completely remote way. So we wanted to hire the best researchers wherever they were. where we were going through the classic GitHub scraping and trying to reach people based on their work instead of based on their presence. And based on that work, we would reach out to those people. We would always share our samples and try to get them to join the team. And that's how we assembled the first set of people who we think are some of the best researchers in that audio domain.

4:27Mati Staniszewski:And through the years, they still help us crank a lot of those models into production. Then we launched the product. I think the slightly different approach we took was monetizing very quickly. So trying to get some of the revenue stream back so we can fund a lot of the work in the models. We tried to stay healthy on the margins so we can continue investing with the assumption that it's better for us to figure out that stream and be able to be independent in that development. But then as the ambitions grew, we knew that we needed to train models. So we, of course, brought a lot of money externally as well.

5:04Mati Staniszewski:And I think like projecting to today, one thing that's clear for us is there's still so many of those niches that people don't tackle that you can start with and then step by step start opening them up. I think a lot of customers see 11 Labs through their narrow needs, right? Maybe take a zoomed out view. Like what is the suite of models that 11 Labs works on? How do you prioritize them? How do you organize R &D, et cetera? So we started with the first text-to-speech model, so the model that could finally understand the context of what's being written. And based on that context understanding, get the right emotion, the right intonation from text.

5:45Mati Staniszewski:So if it was a happy sentence, you get that happiness out. If it's a dialogue, it can pronounce the dialogue out. And then continuously started adding that. So it started with the problem of breaking down language barriers. is the things you need to solve dubbing is transcription, so understanding, then the translation, and then text to speech. So we first solve text to speech. Then we knew we needed to add the other component, which is speech to text and being able to transcribe content in a great way. Then how we combine those models together. So that's kind of what the first three models in the first couple of years.

6:20Mati Staniszewski:And then, of course, the other thing started happening across the space, which is that a lot of the reasoning models started becoming quick enough and smart enough at the same time where you could imagine those interactive experiences being possible. And that's where we started launching more of the real time streaming models across audio, and then combining those into conversational experiences. So we added effectively all the stack, all the turn taking and orchestration to create a voice engine for a voice agent. And then on the other side, as we realized that the emotionality is something we can solve, we added some of the hardest modality in audio, which is music and being able to produce music.

6:58Mati Staniszewski:So today we span entirety of the research of audio, whether it's text-to-speech, speech-to-text, combining those models together in both localization with dubbing, with orchestration, with voice engine, and then being able to do that across music as well. And all those things and all that interesting development work, Was there any oh wow moment in terms of what these products are capable of that you can remember? You know, there's so many and it's kind of the bar changes for all of us. The first moment for us was, well, first moment for us, they always use my voice as a testing voice because it has this weird accent.

7:36Mati Staniszewski:And the first time was like when we could replicate my voice based on a good sample. That was like a first wow moment to myself. And you always go for this moment like this is not how my voice sounds like. and then you listen to yourself side by side and it's like definitely how it sounds like. Unfortunately. Then the second moment was where we first got it to laugh and people were like, okay, this is actually the thing that makes the whole experience more human. The laughter, the pauses, the ums, the ums, the imperfections. So we started getting those out and that was the moment for us because we made it to the top of Hacker News with the first AI that can laugh model, which was a very proud moment for us.

8:18Mati Staniszewski:And then, of course, through the years, kind of that extended where you might remember in 2023, 2024, there was a Javier Malay speech that went viral where he could speak other languages. It was translated into English. And it was the first time where he could still hear his voice out there. So that's the kind of continuous wow moment that was something that's completely impossible. And then we saw that happen time and time again with Narendra Modi, with President Zelensky, all the way to recently one of the, I feel like, pinnacles of the voice performance, Matthew McConaughey giving his newsletter and his iconic lines in Spanish and Portuguese, where for the first time his family who speaks that language could hear him speak those languages too.

9:06Mati Staniszewski:But for more recent pieces, the two ones that we are excited about bringing to production, I think the first one is finally figuring out the emotional intelligence in that interactive experience. So in the voice agent experience where it doesn't only get the right intonation emotion, but can understand the other side. So if somebody is stressed, it gets and delivers that soothing, reassuring emotion. If someone is excited, maybe it matches that. If someone speaks slowly, it makes sure to slow down. And that emotional intelligence is something that we are finally seeing internally a path to solving, which will be just a continuous step change to what's possible.

9:48Mati Staniszewski:And then the second one, which will apply there but also apply into general audio spaces, audio general intelligence where you can combine audio models together in one stream. So you could theoretically have a model that narrates, then pauses, and let's say starts singing with that same continuous voice. and that's something that's extremely hard to combine today and something that would be possible, I think, very soon. In voice, you mentioned voice agents, and it seems like everybody is, at least on the customer side, everyone's buying a voice agent. And I think intuitively you think customer support, the old phone tree replacement.

10:26What's actually going on in the world of voice agents and what do you think are the most interesting, overlooked opportunities, spots where startup founders should focus?

10:35Mati Staniszewski:Yeah, of course, the customer support is probably the one that everybody heard and knows about very well. I think the second thing and the second thread we are seeing is increasing shift to revenue-generating opportunities where voice agents can act in sales, whether it's inbound or outbound of sales. It doesn't replace the entire experience but takes and amplifies part of that experience. Maybe a good example is Deliveroo, where Deliveroo will have voice agents that contact the restaurants to capture their opening times. And based on their opening times, they can update the riders and drivers, and of course the people ordering on when to get to that work, all the way through to the inbound sales, where increasingly people, that's a good example of Deutsche Telekom, will be contacting to inquire about the service, inquire to buy a product.

11:24Mati Staniszewski:And instead of going through the dropdown, instead of going through the form, you can speak with the voice agent to leave that information. We do it ourselves, too, so we have a good metrics of an understanding of what's happening there. One, of course, so much simpler and quicker to go through instead of going through that form. But the second thing that started happening in that inbound sales flow is we had a lot more information that people started leaving because they would speak about the use case they're coming with, but then where it's not working, where it's working, some of the other use cases that they are evaluating, which we can combine and then just deliver such a much better experience afterwards.

11:58Mati Staniszewski:On the overlooked side, I think my favorite example is the citizen support education and healthcare will completely change. on the citizen support, like all of us would benefit from just generally better government access, whether that's understanding how to fill in the taxes that I think many of you went through earlier this month, all the way through to just learning what is the policy for travel abroad and how that might affect the space. We recently seen that work deployed in government of Ukraine, who we think is one of the most advanced governments on that front. We traveled to Ukraine working with their team.

12:39Mati Staniszewski:And what they are trying to solve is they have a government app, which every citizen can access and get information about what's happening. But given the war, given the front line and lack of that access, they wanted to figure out a new channel for people to be able to call in and get that information. So they created a voice agent effectively, where you can call in and get the information about what's happening on the front line. And you can get education help and some of the lectures delivered to your kids all the way through to proactive engagement about staying safe and staying out there. And maybe last example on education front, and that's probably my favorite one as I think about that changing.

13:19Mati Staniszewski:It's just how incredible would it be to have someone that is an incredible teacher available 24-7 where you can ask him questions, whether it's Karpathy, all the way through to Richard Feynman. and you can learn physics with them on the headphones while you are teaching that subject or learning that subject. And that's something that we are seeing pockets of. Like a great example is Masterclass, where Masterclass, of course, collaborates with incredible teachers to deliver static lectures. But recently, they launched an interactive version of that. So I don't know if that will be a good reference for this audience, but we recently worked with them on bringing Gordon Ramsay that can teach you cooking.

14:01Mati Staniszewski:So while you were in the kitchen, he can shout at you effectively to get better. Or maybe a better one, there's a Chris Voss where you can, of course, learn negotiation, but you can learn by negotiating with Chris live on the phone to get better, which I thought was a phenomenal subject. Having negotiated against Madi a number of times around financing rounds, I understand now. I think it helps you to say this, but I think the opposite is true. I have some more questions. I want to save time for the audience as well. Maybe one, as Constantine mentioned, more than$100 million of net new ARR in Q1.

14:36Obviously, the business is going very well. And you're sort of pioneering the startup founder, building a foundation model, applications. Any counterintuitive lessons about building a company in this era that for the founders in the audience, they might want to take home with them?

14:52Mati Staniszewski:So we are, just for reference, we are just over 400 people, over 400 million in revenue, but still keep the teams extremely small. So it's like rough, arbitrary, a little bit. It's less than 10 people. It's for each of the research product. Even the go-to-market ops talent teams are all smaller than that size. Most of people will have 10 direct reports or so. So it keeps it relatively flat and allows us to move a little bit quicker. One thing that we've done, which is in this model, and very surprisingly, this is a very similar model that we've seen actually with the government of Ukraine. Each of the teams, even the teams that aren't technical teams, will have engineers within them.

15:33Mati Staniszewski:So our people team, our go-to-market team, our legal team will have an engineer in that team that helps to build, of course, automation, upscale, uplevel the rest of the people. And recently, that really helped, because as I'm sure many of you are going through, So everybody will be vibe coding and coding a lot of the help, even if they are not technical. So now that kind of shifted the responsibility, but shifted the requirement of how good the review needs to be for a lot of that work. There's security, infrastructure implications. You will want to make sure that the output is right. And I think on the engineering side, you can put that expectation.

16:09Mati Staniszewski:On the non-engineering side, the ability to do that is relatively hard. So that technical resource in those teams helped us a lot to figure this out. And in general, there's just so many incredible work you can do by having that, whether that's the scraping on the hiring and recruiting front and analyzing what worked in the past to improve in the future, whether that's upskilling the legal team on how to use those tools and then figuring out ways of, we recently introduced this scoring system. For those on the go-to-market on sales side, you frequently will end up in this negotiation with your sales team of, can I give indemnity provisions?

16:46Mati Staniszewski:What's the liability cap? Can I give the set of clauses? And then you kind of need to draw the line of how many things you give. And I ended up being in so many of those conversations that we gave already a lot or we didn't. So now we introduced the scoring system that you can give per size of the customer. You can just give a few of those points out and in. We just made it so much easier. And of course, that's fully automated now with how we work across that team. So that was one of the unintuitive of small teams, bringing technical talent in the non-technical teams, keeping relatively flat. We also have no titles, which allows us to bring people and really optimize for impact that they are having.

17:26Mati Staniszewski:And then you can grow as quickly as you want. The tenure will not define this. And many more. So we'll see. It's a four-year-old company, so we'll see if that helps. JOHN MUELLER - Any questions? Oh, no. OK, Sonia. Are you seeing people deploy voice agents to actually negotiate on their behalf? And then when you, are you starting to see agents actually negotiate with agents? Sorry, I do three-part questions. When that world happens, do you think the agents are actually talking to each other the way that humans talk to communicate and negotiate? Or do you think it's beep, boop, beep, boop? Do you think it's, you know, it's all done instantaneously?

18:08Like, how's that world going to look like?

18:10Mati Staniszewski:So one, early inklings of that, we haven't seen any truly successful on the negotiation front. It was like more, you know, kind of order taking, what's the price, can we capture that, and then kind of goes back to the team, so not real negotiation. But there's a few startups that we see, especially on any, like, organizational shifts of, can I organize this event, calling, calling out of places, getting the price, and then calling again with, like, our budget. So that is happening, and I think this will shift. I think emotional intelligence, this is the big part that will start being important in a lot of that work, where it's not only the content that matters, but how you deliver, when you pause, that work.

18:47Mati Staniszewski:And then maybe the extreme version of that, which agents are not, like most of the people wouldn't do it, and they are not good at that. Today you will see a lot of interruptibility built in, where a human can interrupt the agent, but with negotiation you also want the opposite, where an agent will interrupt the human. It's kind of the extreme version of that. On the second part, on the agent-to-agent part, some of you might have seen this. We did a hackathon over a year and a half ago, and that was exactly the case, where agent was speaking with another agent. They detected that they are both agents, and they swapped over to a different language, and also get more efficient transmitter of information than just the classic spoken word.

19:32Mati Staniszewski:And I think this will happen 100%. Like the big question, will it be really voice? Will it be other transmission of information? And depends truly on what the infrastructure is built for. And I think this will define that experience. Yes.

19:52Mati Staniszewski:It's you at the catch box. Hey. Curious how you're thinking about the need for voice in the future where agents do more and more of the work. So basically, what are the kind of use cases maybe we're human conversation. I think it's more of a follow up to the last question. Like first, you all of us will have so many different devices around us. And step from that, you will have robots around us. So of course, voice will be such an important interface to instruct and be able to interact with those devices. In many ways, I feel like we see a lot of developments of intelligence. But then the real bottleneck of the future will be how we communicate with that intelligence.

20:33Mati Staniszewski:And I have voice and visual part will be a big unlock to be able to actually get the most of that intelligence value in those settings, which isn't yet possible. But on the flip side, the value of the human-to-human interaction will only increase. So whether that's the events like this one, whether that's events with your favorite artist, will increase in value with that ability of having voice all around you. But the trust will be such a big part and something we optimize for like in between the agent and human of you know in the future where all of you will all of us will have a voice agent for example to call and book a restaurant or give information to a health care appointment.

21:18Mati Staniszewski:and all of that will require such a high degree of trust that this is you and authenticated you. So there'll be like a level of encoding and decoding for real, then encoding, decoding for watermarked opted in human. And then by default, everything else will be fake, which is kind of the opposite of how it is today. You'll detect for AI, but it will detect for real authenticated AI in the future and assume it's fake.

21:50Andrei spoke earlier about jagged intelligence. Do you see similar odd places in audio where models are good and are bad that you might not expect? And what are they?

22:03Mati Staniszewski:There's still so much on the bad side. I think that we spoke a little bit about where we see the voice agents working. So this combination of the models together and support settings works really well, works reliably. early sales starts working, but the moment you start swapping to a true emotional interaction, not yet working. It doesn't get the emotion that well. It's slightly too slow. So that is still, I think, a big step change that should work. Same will apply in a very different domain on the music side. I think in the music side, you can get good production music. You cannot get top charts music, even with artist input.