AI Audio Updates: Spotify, Particle, ElevenLabs

0:00Welcome to the podcast. I'm your host, Jaden Schaefer. Today on the show, I want to talk about some updates in AI audio. We have a couple new features from a bunch of big companies rolling out from Spotify. They're rolling out new AI-powered prompted playlists where you essentially create a prompt and it builds a Spotify playlist for you. There's some interesting pros and cons to this. Particle is an AI news app that now is able to listen to podcasts for you. So like, you know, instead of just having to give you summaries on news from top news organizations based off of articles, it can actually go listen to podcasts for you, pull out interesting clips from those podcasts.

0:35You don't have to listen to the entire thing. Personally, as an avid podcast listener, this is something I'm excited about. And finally, Eleven Labs has written an entire report on the state of AI audio in publishing and news. So let's get into all the AI audio news. But before we do, I have a major announcement that I'm super excited to make. And that is that my own platform, AIbox.ai, that lets you access over 50 different of the top AI models all in one place. so you don't have to have subscriptions to all of them. We've just completed an entire redesign of the whole platform from the ground up and an entire overhaul.

1:07So if you've tried it in the past, I'd love for you to check it out. Today, you get access in a playground to all of the top AI models. And we have a vibe tool builder where you can describe any workflow or tool you'd like to create. And we link together all of the different AI models, input prompts for you and build tools to help automate a lot of the stuff you do. We've simplified a ton. I personally have spent weeks and hours painstakingly doing all of the UI to make this thing look amazing. That's just kind of what I obsess about. And so if you enjoy it, let me know. And if you want any more features or things added to it, let me know.

1:37But we just put a massive upgrade. So if you want to check it out, it is AI box.ai. And we also rolled out new pricing tiers. So you can actually get it was 20 bucks only in the past. If you want to test it out and do some smaller scale stuff, we have an 899 tier. And we also have annual plans. So you get 20 % discount if you get an annual plan as well. So go check it out, AIbox.ai. All right, let's talk about the state of AI and audio. I want to kick this off talking about Spotify's new feature. They have these AI-powered, they call them prompted playlists, and they're rolling out to premium subscribers.

2:09It's not for everyone, and it's specifically in the UK, Ireland, Australia, and Sweden. They did some earlier tests in New Zealand, and they've launched this in the US and Canada. So they're rolling this out more broadly. If you're in the US and Canada, you should have access to this already. It's pretty straightforward. Basically, instead of going and just searching for different playlists, which is what I will typically do, it will use AI to create them from scratch. Now, there's a bunch of pros and cons for this. And I mean, I'll automatically admit my bias in this, I've actually spent a decent amount of money invested into creating playlists and promoting them to get users to follow these playlists.

2:43And then if you have music that you put into that playlist, it automatically gets listened to. So this is a good way to get music listened to. And you know, obviously, you got to make good stuff that follows inside of your genre that your that your followers will like, but doing these AI playlists essentially kind of kills off that entire industry or that entire way of making content. Now, if I'm being honest, I think that this is probably where the future goes in a way. And that's because oftentimes, you know, as myself as someone that is a playlist creator, they call them playlist curators, I sometimes get really busy, and I don't update my playlist for a month.

3:16And maybe there's a bunch of great songs that would have been perfect that the Spotify algorithm could have found that I didn't find. And so I think that that's an interesting concept. Now, will these playlists automatically be updated that you prompt? Probably not. So you also have to automatically prompt them. So the pros and cons here are that the user, while they will get a fresh playlist when they prompt it and create something cool that maybe a pure playlist curator, if they weren't on top of it, I wouldn't have added. You also would have to continuously kind of reprompt or regenerate that playlist to keep it fresh if you wanted new stuff in there.

3:49So possibly that's actually just making more work for the user. So pros and cons there. Besides that, I think there's a bunch of cool things you can do in the prompting itself. You can add things like the vibe, different scenarios, aesthetics, memory area, era, genre. You can say something like songs that feel like the final score of a specific movie. And you can also ask for mostly new music or you can ask for mostly music from your library. So So there's a whole bunch of really cool things there. I'll give you one example that I saw in a Spotify demo. They said, find one artist I haven't listened to yet, but would probably love, and an artist I've only heard one or two songs from and introduce me to them, build a playlist of songs that'll give me an overview of their catalog so it feels like I'm getting to know them, put the songs you think I'll like most in the top spots.

4:37So you've typed in something pretty long, elaborate and specific like this, and it generates it for you. Now, I think this is awesome for getting custom music exactly how you want. The only downside is it's super cool to get this, like you get to customize your algorithm basically and actually get the correct direction. But also something as a user kind of kills me to be like, man, why couldn't they just like make like a, I don't know, like some sort of button that I push and it automatically does that for me. Like if there's something people are frequently putting these huge long prompts for, I don't want to have to be a quote unquote prompt engineer or someone that has like these giant saved prompts in the notes app on my phone that I got to copy and paste into Spotify every time I want to get like this new fancy playlist.

5:20So that's just maybe, yeah, that's just me maybe complaining, but I really just wish the algorithm would automatically do most of the stuff for me because I don't want more work. I don't have to copy and paste every time I want a fresh Spotify playlist, but you know, it's, it's probably way less work than going and manually finding songs and finding playlists perhaps. So there's, you know, pros and cons. This is still in beta. I think a lot of users say that they're hitting caps after about 20 to 30 prompts. So you get 20, 30 prompts in and then it's not going to let you do anything. I think Spotify is managing costs.

5:51And they're also trying to learn what kind of prompts are creating like the highest satisfaction, right? I think if people are doing 20 to 30, they're probably not getting what they want, since I probably would need like four different playlists, and I could be happy for life. But maybe that's just me. All right, let's talk about what Particle is doing. So they are an AI news app. They're built by the former, some former Twitter engineers. They're now treating podcasts kind of like a first class news source, not a separate medium that you go to. And I think this is, this is interesting, right?

6:19They have a new feature, which is called podcast clips, and it basically finds interesting moments inside of podcasts. And it puts those clips alongside related news stories in their newsfeed. And I think basically what they're saying is that you shouldn't have to listen to an entire hour long episode just to find 45 seconds that actually matter. I actually, I think this is really cool. As someone that creates obviously a lot of podcast content, I host a lot of podcasts, I listen to a lot of podcasts. I think this is awesome. There's certain podcasts I listen to every week, for example, and they're like one and two hours long.

6:49And sometimes I just don't have the time to catch up. But it would be sweet to just be able to go and say like, hey, give me like the five top things happened that you know, the top moments, the things that were said. And some of those podcasts really do break news or give you insights that you can't get anywhere else because there are people working on, you know, businesses or inside of organizations. So like there's really good juicy data in there, valuable stuff. And if I got a, you know, an hour long drive to work and an hour long drive back, I'll get through that no problem. But sometimes I don't have to go to work or sometimes I don't have to do that long drive and I don't, or I don't have a long flight and it'd be really nice to be able to just get the highlights.

7:24So do think that that's very, very cool. They are wrapping this inside of a product strategy that is pretty familiar and that's just basically a freemium plus. subscription. So if you get Particle Plus, it's$2.99 a month or about$30 a year. And then you get all of those kind of summary controls, you get multiple voices for an audio feed, you can listen to the news, there's a bunch of other cool premium features. But they do have some, you know, some elements of that that are paid. Next up, let's talk about what's going on over at 11 labs, one of my favorite AI audio companies, although they are so expensive that it kills me sometimes, but I have spent thousands of dollars.

8:01I might be getting close to$10 ,000 of 11 labs credits I've spent over my life. So obviously I love this company, but I have a love hate relationship because it doesn't feel good to have spent$10 ,000 on a company in AI. But the places that 11 labs, I think are really crushing it lately. And also like full disclosure, this is somewhere I spend a lot of money on as well is in things like audio dubbing. So recently I've had AI Chat, AI Hustle, AI Applied, three of my top podcasts, all translated with dubbing via 11Labs into a whole bunch of different languages. So people could listen to this podcast in French, Spanish, German, Mandarin, like a whole bunch of different languages, but it's still my voice talking.

8:42So I think that that's a really cool feature that 11Labs enables. And they have to do a bunch of creative things, especially if there's two different voices. You have to, you know, first take the voices and clone the voices, then you have to determine, you know, which voice is which. So you have to identify who the two speakers are and kind of do those two voice clones separate. You got to do the translation, of course, which isn't too crazy. But then something that 11 Lines does that I'm impressed by is in order to get the lip sync working, meaning if you're going to use like an audio that gets translated for a movie, right, let's say the movie clip is 30 seconds long, you don't want the audio file to come out after it gets translated into Spanish as like 45 seconds.

9:23And now because every word and every translation and phrase could be shorter or longer, it is pretty tricky to do this. But there are a bunch of like stretching tools and techniques that you can use. And so this is kind of what 11 Labs does. So essentially, if you you can use these audio files as dubs for videos, and you see kind of the same technology with like Hey Jen and people like that. But I've been really impressed with how 11 Labs has done it. And specifically, you know, I use this for podcasting. A lot of other publishers and news organizations are also doing this. And this is a really big moneymaker for Eleven Labs.

9:59So I think the upside to this is obvious. You get a lot more engagement and retention if you're translating this into other languages. It's a lot more accessible for people that might, you know, not speak English super well or whatever the primary language is super well. And then like monetization, you can obviously make more money by growing the user base or the listener base and specifically in different countries because a Spanish company is not going to advertise on my podcast. But if I do a Spanish podcast, I could get that advertiser. And so by being able to go into different languages, you have a lot more monetization opportunities, which I think are really interesting.

10:37So overall, I think there's a lot going on. I think Spotify obviously is now making audio discovery a lot more like chatting with a smart assistant. Particle is now making podcast audio more like scanning headlines. And then publishers are now turning into audio by default. And then they're also going to be doing multilingual audio on demand. So on all of these news organizations that used to just be articles are now becoming audio because they have this AI ability. And I think this is going to be a trend we see more and more into the future, which I'm pretty excited about. I think AI is showing up everywhere all at once.

11:10It doesn't just kind of make things more convenient. I think it really kind of changes how we consume media, whether that's podcasts, articles, or music. So I'm really excited about a lot of the advancements we've made in all three of those areas. I think the next phase is pretty obvious. You can kind of see where this is going if you look at the roadmap, and that's language. I think conversational audio, interruptible voice experiences, and then also kind of interactive news where the story is not just a page you read. It's kind of like a conversation that you're having with the news. I think these are some of these use cases, how they expand.

11:42A concept I've been thinking a lot about lately is that it's not just about who has the best AI model per se, and that's who's going to win in this. I think it's who can come up with the best use cases. And I think there's so many interesting plays that can be made right now, especially with all of the advancements we're seeing in AI and audio. Thank you so much, everyone, for tuning into the podcast today. If you enjoyed this episode, make sure to leave a rating and review wherever you get your podcasts. It helps the show out a ton. Honestly, it's like the number one way if you ever get anything out of this show, you could say thank you, drop a comment or leave some stars over on Spotify.

12:15It helps a show a ton to get found by new people, which is big. But also, I really appreciate them. I read all the reviews helps me know what you guys like to hear and what's helpful for you all. So if you want to drop a review, I would be super, super grateful. Thanks so much. Make sure to check out AIbox.ai to get access to all of the different models in one place and check out my new redesign. I'd love to hear what you guys think of it. It has been a labor of love with blood, sweat and tears, but I think it looks amazing. So we'd love to hear what you guys think. All right. Catch you guys all in the next episode.