AI Podcast 2.0: The host in the machine

0:00This message comes from Capital One. Capital One offers checking accounts with no fees or minimum.

0:14Just a heads up, this is episode two of a three-part series exploring whether artificial intelligence can make a Planet Money episode. Today's episode does make passing mention of a couple of technologies from Apple and Google. You should know that both of those companies are financial supporters of National Public Radio. Here is today's show. This is Planet Money from NPR.

1:03to hear that podcast. But if we are really going to take seriously the question of, is the AI going to replace us? We wanted to go all the way. We needed the computers to also replace our voice to narrate this episode. And if we were going to have the computer do the talking, we might as well go big. Get the most Planet Money-y voice of all the Planet Money voices. I speak, of course, of... Hello and welcome to Planet Money. I'm Robert Smith. Robert Smith, Planet Money's longest running host. Even was the show's editor for a while. But a couple years ago, he finally hung up the old microphone and left the show as a full-time host.

1:45Robert left behind a legacy, but also lots and lots and lots of recordings of his voice. Hello, I'm minor podcast celebrity Robert Smith. And here at Planet Money... In order to save the country, they're going to have to work together. to restructure the tax code. Who is this poop cartel that you speak of? Ahoy there! Welcome aboard the good ship inflation. And just for fun, we did try to hack together our own version of a knockoff Robert Smith using those old recordings. We were just, you know, cutting and pasting those words together. Yeah, we even gave him a little catchphrase. And, uh, here.

2:24Here, give it a listen. I'm knockoff Robert, and I'm not here to make friends. I'm here to make history. Uh, yeah. Kind of hard to understand there. He's saying, I'm Knock Off Robert. I'm not here to make friends. I'm here to make history. I'm Knock Off Robert, and I'm not here to make friends. I'm here to make history. Yeah, okay. It's not very good. And clearly, there are better ways to do that now. Because part of the AI boom has been the introduction of these surprisingly realistic AI voices. Yeah, somebody made a completely synthesized conversation between podcast host Joe Rogan and Steve Jobs, who is dead.

3:07How's it going? Come on, tell me about Jobs. Yeah, it's great to be on the show. Your audience is just so different from your normal Apple users, and that's a good thing. And then there was a song featuring an AI-generated version of Drake and The Weeknd.

3:27So yeah, AI Drake, AI Joe Rogan, clearly this is possible now. So today's mission, go and somehow build the Planet Money version of this. AI Robert Smith. Hello and welcome to Planet Money. I'm Kenny Malone. And I'm Jeff Glow. And today, the next step in our quest to try and automate our jobs at Planet Money, can a computer believably serve as the voice of planet money? There has clearly been some enormous leap in making computer voices. So what happened exactly? How did we go from like bad GPS voice in half a mile turn left to that incredible Drake thing? We will build and introduce you to a fully synthetic Robert Smith.

4:18And we'll also check in on another synthetic voice clone of an even more famous celebrity. Grimes. Wait, wait, who is Grimes? Sorry, knockoff, Robert. Grimes is like a musician slash cyborg slash like demigod. It's complicated. Ask your kids or whatever. Okay. Stay with us.

4:45This message comes from Vanguard. Capturing value in the bond market is not easy. That's why Vanguard offers a suite of over 80 institutional quality bond funds, actively managed by a 200-person global team of sector specialists, analysts, and traders. They're designed for financial advisors looking to give their clients consistent results year-in and year-out. See the record at Vanguard.com slash audio. That's Vanguard.com slash audio. All investing is subject to risk. Vanguard Marketing Corporation Distributor. This message comes from Kachava. Indulge in holiday cravings with the nutrition you need from Kachava's all-in-one whole body shake.

5:28It packs 25 grams of protein, 6 grams of fiber, greens, and more. High-quality ingredients with no fillers or nonsense. Try the newest flavor, limited edition chocolate mint. Go to Kachava.com and use code NPR for 15 % off. That's cachava, K-A-C-H-A-V-A dot com, code NPR. Hello. Hello. This is Martin. And Matt Hawking. Thank you for having us. This is Martin Ramirez and Matt Hawking. They're from a company called Well Said Labs. It's a Seattle tech startup that makes what they call voice avatars. Okay. So we are interested in taking our beloved colleague, Robert Smith, and somehow making a version of him that we can still have in our wonderful episodes occasionally.

6:19Is that something that you guys can do? As long as Robert is in, Well Said will be well-equipped to create a synthetic replica, if you will, of Robert's voice. Synthetic Robert. I love it. Synthetic Robert, an upgrade from Knockoff Robert. Now, Matt and Martine give us a quick little history of people trying to get computers to talk like people. I think when we originally started down this path, text-to-speech had primarily been very robotic, very monotone, and had been produced in a concatenative way. What is the word you're saying? Concatenative? Concatenative. In this case, it means taping together a bunch of pre-recorded bits of voice.

7:06So think about the way that we built that funny knockoff Robert earlier. I'm knockoff Robert and I'm not here to make friends. I'm here to make history. Yeah, well, that is a bad version of how synthetic voices used to get made, according to Matt and Martine. A much better version is the original Siri voice that was recorded back in 2005. And that voice actor said that they made her go into a studio and she had to read these weird phrases like cow hoist in the tug hut today. or say shift fresh issue today. Well done, Jeff. Well done. The reason she had to say those weird things for like 80 hours, by the way, was so that in the end, Apple could chop up all the recordings and have like a drawer filled with every possible sound in the English language.

7:56So Siri can say any word by reassembling those little sound elements. So that is the old way. The way that Matt and Martine were going to build synthetic Robert was apparently way different. We're not going to build a drawer of Robert making all of these sounds. Yeah, he's not chained to a microphone saying millions and hundreds of millions of permutations of his scripts. Yeah, as far as I know, he's like cross-country skiing right now. And that there is what we inevitably have created is the ability for Robert to be off cross-country skiing. Because the modern voice approach by well said and an increasing number of other competitors in this space is quite different from the old thing.

8:38For example, if we're going to go ahead and do this Robert thing, Martine and Matt explained that we're going to send over some recordings made by Robert Smith during his time at Planet Money. And then we'll also send over the exact transcripts for those recordings. And then essentially what's going to happen is that a computer is going to look at a chunk of that written transcript and it's going to try to guess how Robert might say those words. So whether that be the pitch, the pausing, the intonation, the emphasis, all of those types of things have certain metrics which are measured. And I love the perspective of let's be wrong multiple times as quickly as possible.

9:15And it's going to sound sort of kind of, but maybe not really, Robert. Wrongbert. Wrongbert. Yes, there you go. And this synthetic voice is going to be wrong, Bert, for a long time. Martine says it's going to take millions of training cycles. But in the end, we should end up with a computer program that can look at any sentence and do a really good job guessing how Robert Smith would say that sentence. And by the way, if it took 80 hours of recordings to make Siri using that old concatenative approach, this new approach, it requires way less audio. We've kind of come to around 45 minutes to a couple of hours worth of audio to get to a point where we can create a very realistic likeness to someone's voice.

10:04A full voice clone from so little audio. And, you know, to be clear, it's not just well said here. Like this technology is spreading really fast. Yeah. There are lots of companies with their own versions of this voice cloning stuff. Google has one. There's another company called Eleven Labs. Apple says it's about to get in the game. Yeah, plus there are open source voice cloning tools without any real restrictions or oversights and anyone can use them. And of course, when you start to consider the implications of all of this, it is really terrifying in some very obvious ways. I mean, imagine how bad it would be if there were a synthetic Joe Biden voice announcing nuclear war or something, right?

10:45Yeah, so companies like WellSaid, they are well aware of these fears. And so Wellstead explained to us that if we were going to work with them for this project, there were going to be lots of terms and conditions. For starters, Matt and Martine told us that they wouldn't even consider making a synthetic Robert Smith until they had heard from the real organic Robert Smith and had his permission. They also told us that they will be able to monitor every word we make synthetic Robert say. As soon as anything looks like it's not in line with our values, we're very quick to put an end to that. And maybe the biggest term and condition of all, as soon as we were done with this project, Synthetic Robert would be shut down.

11:28He could narrate our AI-generated episode, and then he will be functionally destroyed, never to be used again. Poor Synthetic Robert. But we agreed to all these terms. We got Robert Smith's permission. we went into the old NPR Planet Money archives and we pulled a lot of Robert Smith recordings. And I'm Robert Smith. And I'm Robert Smith. And I'm Robert Smith. And I'm Robert Smith. And I'm Robert Smith. He said other things in these recordings too. But yes, we sent all of the recordings over to Well Said and we waited. After the break, synthetic Robert Smith makes his world premiere.

12:12This message comes from Apollo Global Management, who believes the global industrial renaissance is transforming the world. Over the next decade, industries like energy, infrastructure, and technology will need an estimated$75 to$100 trillion to modernize and meet demand. Long-term projects need long-duration capital. That's where Apollo steps in. With scale, flexibility, and a focus on growth, they're partnering with companies to drive the future one innovation at a time. Learn more at thinkitnew.com slash renaissance. Hello, it's Amanda Aronchik, and I'm here to say thank you to our Planet Money Plus supporters.

12:54You help make our work possible. And one really important part of that work is fact-checking. A lot of times, Planet Money hosts will talk to a source for two or three hours. My job is to go in and make sure that what we have reflects the entirety of the interview. Meet our resident fact checker and go inside our fact checking process in our latest bonus episode. Out now for our Planet Money Plus supporters. If that's not you, it could be. Learn more at plus.npr.org. while we were waiting for synthetic robert to train and i know i know this isn't how it works but i'm definitely picturing it like rocky running upstairs in philadelphia but yes while we were waiting for that to happen we started to think more and more about some of the other implications for this voice cloning technology and you know not just the oh no what happens if someone makes fake joe biden but also what happens economically if suddenly voice clones of the thousand most famous people are commercially available.

13:58Yeah, what does it look like when there's suddenly a synthetic Robert Smith who never gets tired, who can narrate an infinite number of Planet Money episodes? Yeah, or like, what happens if, I don't know, Tom Hanks suddenly makes a cheap cloned version of his voice available for all commercials and all audiobooks and all documentary films? And weirdly, we can already get a peek into this bizarro future. It's starting to play out right now with Grimes. Grimes? Yeah, the experimental sci-fi pop musician from the future. I mean, she is here in the present and is writing music, but she seems like she's from the future.

14:36But yes, here's some of her music.

14:43Grimes has a really distinctive voice, like whispery and ethereal. And in the middle of our AI project, Grimes made this kind of wild announcement on Twitter. She said basically, hey, from now on, anyone is allowed to copy my voice using AI and like put me on their songs. Just split the royalties with me if you do. And Grimes even released her own AI voice cloning tool that anybody can use to turn into Grimes. And so just allow us to show you how this works. It is stupidly easy. So let's say you have a song that you want Grimes AI to sing. Oh, I do. It is a weird version of Row, Row, Row Your Boat where every word is off by one note.

15:26Yeah, that sounds... Okay. I cannot even imagine what that sounds like. Oh, it's weird. Okay. So you are going to record a version of the song. I will. Here we go. A dream row, row, row your boat gently down the stream. Merrily, merrily, merrily. Life is like a dream row, row... Is that enough? Do you need more? No, no, no, no. No, that is plenty to work with. So yeah, so I have a recording of you singing that song, that horrifying song. Literally, all I have to do is I hit upload. Are you ready to hear yourself as Grimes? Oh my God, I'm so ready. Okay, we're going to throw a little beat under it so it really sounds like a Grimes song.

16:03Okay, here we go. A dream row, row, row your boat, jetty down. A stream merrily, merrily, merrily, life is like. Oh my God. What is going on? So using this technology, Grimes is essentially unleashing her voice to go out and make music and earn money on her behalf without her having to be there at all. And so, of course, we had to talk to Grimes about all of this. I'm sorry, Planet Money, but I'm unavailable for an interview right now. No, actually, sorry. That was Kenny singing into the machine again. However, we did manage to talk to Grimes' like co-conspirator slash collaborator on all this technology stuff.

16:50Hi, I'm Dawood Leonard. I'm the co-founder and CEO of CreateSafe. And additionally, I am Grimes' manager. Do you know how many songs are out there with Grimes AI? So we've had around 300 songs distributed or are uploaded to the platform to be distributed. So it's 300 songs uploaded through the platform in what time period? What's today? The 18th? Yeah. So 18 days? I just want to stop and reflect on how bonkers that is. because in just 18 days, because of AI Grimes, effectively 300 new Grimes songs were born into the world. And the real Grimes didn't have to do a thing. Like that is a productivity gain.

17:38I mean, here, just listen to some of these songs.

17:49So yeah, everything you just heard, that was AI Grimes. And now all of those songs, they're just floating around out there. And if any of them take off and make money, boom, Grimes gets a piece of that. And Daudo wanted to tell us about this one specific song somebody had made with this AI stuff. This song right here. You could be the one that made me cry. Light me up inside, inside. So this was created by an artist called Keto. And Dauda told us that the real Grimes heard it and liked it so much that she is now going to add her real voice to the song. There's the original Keto version. And now there's...

18:30With Grimes AI. With Grimes AI. And then there will be like Grimes and Grimes AI and Keto. Oh, Grimes AI stays on it. Grimes AI is going to stay on it, right? There's going to be some of her vocals still on it. The splits will be interesting, right? Because Grimes AI is a 50-50 split, and then plus a new Grimes, which is another 50-50 split. Yeah, but Grimes isn't going to try to like, nah, I don't think that's not how Grimes rolls. No, no. Because if you ever get a chance to talk to Grimes about all of this stuff, it's clearly not really about the money. And we know this because we did ultimately get a chance to talk to Grimes about all this stuff.

19:08And this is gonna take a little bit of explaining, so go ahead, Jeff. Yeah, okay. So after our interview with her manager, right as we were wrapping up this episode, we finally heard from Grimes herself. She agreed to answer some of our questions via email. And then we got those emailed answers as forwarded to us by her manager. But, you know, like Grimes is a self-described, self-replicating pop star and, quote, basically considers herself a cyborg. So in a planet money first, we believe, Grimes has agreed to speak her written answers via synthetic voice clone. That's where we're at right now, Jeff.

19:50So we begin this strange interview with the most obvious question. We asked Grimes, why did you decide to give the entire internet control of your voice? We've been trying to do this for like five years, but the tech just wasn't there before. So we moved on. And when the Drake weekend stuff went viral, I realized this was pretty accessible technology. And to be honest, I didn't even think about it or ask my team. I just tweeted that people could use my voice and it went viral. Again, that is an AI Grimes voice clone speaking aloud to the answers that the real Grimes emailed to us. Planet Money.

20:27Yeah. Our next question for Grimes. what does having an AI voice clone version of yourself allow you to do that you couldn't do before? Open the creative process to all humans. Grimes to me is art, not music. And while getting credit for my work is cool and used to matter to me a lot, accessing the human hive mind in a way that wasn't achievable even a few months ago feels like the most exciting thing. Now, there was this one big thing that we especially wanted to talk about with Grimes, a sort of economic problem that Jeff and I had been worrying about as we were making our own synthetic voice clone of a less famous, but still famous-ish person, Robert Smith.

21:10And that economic problem that we were worried about is what economists sometimes call the superstar effect. So yeah, the idea is that sometimes new technologies can vastly increase inequality. So think about the printing press or radio or television, these tools that let one person reach massive audiences, beam themselves into every single household. They can create these superstar economies where a tiny handful of actors or artists, they become really popular, make a lot of money, and then all the other actors and artists, they get pennies. And so our question for Grimes was like, Why won't this also just happen with this voice cloning stuff?

21:53Like, doesn't it just let the already famous people just replicate and become bigger? And Grimes, you know, acknowledged to us that, yeah, she does have concerns about those sorts of economic phenomena. Yes, absolutely. I think this is also a debate to be had. I'm hoping smaller and midsize artists like me can workshop these phenomena a bit. But she also said, look, it is also possible that releasing her voice to the world could create opportunities for up and coming musicians. Like getting your work to be noticed is really hard. But now if you're, you know, a young songwriter or producer or whatever, you can kind of borrow Grimes's stardom to get your own name out there.

22:36I run into absurdly creative humans all the time, but not a lot of people get to be artists. A lot of luck is involved in that. It's hard to build a fan base and it's hard to get your work in front of the public. So if there's ways to reduce these algorithmic barriers by letting people inhabit my being, then I think we're moving in a direction I really like. It's a great point. Again, being spoken into the world by the synthetic voice clone of a cyborg self-replicating pop star, a disclosure that we need to keep making. It's so weird. And, you know, the big theme that we've been wrestling with in this podcast series that we're making is what did these new AI tools mean for our jobs?

23:15You know, specifically as journalists who work in radio and this strange cyborg interview with Grimes was incredible. It was alarming. It was all of these things. Because we talk a lot in radio about the intimacy of the human voice. And I don't know, it totally felt like at moments we had spoken to the real Grimes, that we had this like real personal connection with her. Oh, for sure. Which, of course, we had not. We'd had no personal connection whatsoever. We'd gotten an email. That was it. It's bizarre. And, you know, look, it's one kind of bizarre, I think, to talk with the clone of some faraway pop star.

23:57But what was still left for us, Jeff, was to meet the clone of our beloved colleague, Robert Smith. And after a couple weeks, we got the call. The brand new Synthetic Robert Smith was ready. The company we worked with to make Synthetic Robert, well said. They wanted to show us the final product. And so we hopped on a call with Ryan Johnson. She was in charge of creating and training Little Synthetic Robert. Hey, Kenny, how's it going? This is a little bit like Christmas in the spring. It really is. Well, if this is Christmas, I feel like the excited mom who knows the gift that's coming. And I wrapped it late last night.

24:35Okay. Now, one quick thing we have to mention before we unveil Synthetic Robert. Creating one of these voice avatars, as Well Said calls them, it's not cheap. They told us that what we're doing with Synthetic Robert would typically cost about$20 ,000 to$30 ,000. But in this case, Well Said did do this for free for Planet Money. And, okay, I think with that said, let's get into it. But Rian Johnson from Well Said began by explaining all the work that had gone into creating this weird Robert Smith. Let me start. Let me. So we talked a little bit a few weeks ago just around creating a predictive model that can generate audio like this, right?

25:16Rian gave us a quick refresher on the basics of how she made synthetic Robert. Yeah. So she had a computer look at some written text and then guess how Robert Smith would say those words. Then the computer checked its guess against a real recording of Robert Smith, saw how far off it was, tweaked its approach, tried it again, over and over and over, hundreds of thousands of times. So yeah, Ryan said all of that training was now done. And she was actually going to let us hear how the voice progressed over time. And so she starts by playing what synthetic Robert sounded like at the very, very beginning of the process.

25:54So if you're ready to hear step zero. This is guess one. Yeah, this is essentially nothing. Here we go. I personally believe it's a problem for experiments in the subject of these people's delicious. It sounds like a swarm of bees are coming out of Robert's mouth. Yes, that's right. It's very noisy. It's gibberish. In fact. It sounds like he's trapped in the matrix. Yeah, it does. Well, okay, so what's the next one? How many steps? A hundred thousand steps. A hundred thousand steps. Okay. So here we go. 100 ,000 steps, Robert. Okay. I'm synthetic Robert, and I'm not here to make friends. I'm here to make history.