Google NotebookLM’s Raiza Martin and Jason Spielman on Creating Delightful AI Podcast Hosts and the Potential for Source-Grounded AI

0:00Hey everyone, we're here on training data. It's great to be here. I'm a long -time listener and fan of Sequoia Capital. Pretty exciting to have us, the host of another podcast. Join his guests. Yeah, I really just want to say a huge, huge thank you. Thanks for having us on the show. Yeah, seriously. Thank you to Sonia and Pat for having us. I mean, it sounds like we're on the show because we've had lots of listeners ourselves, listeners at Deep Dive. Oh yeah, we've made a ton of audio reviews since we launched, so it's nice to get to talk about it. Definitely a cool experience getting to be here.

0:28And exciting to share, we're going to do next. Exactly. We'll keep learning and getting better for you. We're glad you're along for the ride. So yeah, keep listening. Keep listening. And stay curious. We promise to keep diving deep. And bring you even better options in the future. Stay curious.

1:02Two weeks ago, a mysterious experimental product from Google became the talk of the town. Notebook LM, an AI -powered research tool that went viral for its ability to create staggeringly and hilariously realistic podcasts from any source material. Today we're excited to feature Riza Martin and Jason Speelman, the product and design leads on notebook LM at Google. We talked to Riza and Jason about the inspiration for the product, the development process for a project like this inside a large organization like Google, the surprising use cases that have emerged, and what's next for notebook LM. Riza, Jason, thank you so much for joining us today, and we're glad that we could get you on today before your AI podcast host take over the podcasting world and put us out of a job.

1:48So thank you for joining us. Thank you for having us. I'd love to start by asking, you know, people are calling Notebook, LM, Google's ChatGPT moment. It was an experimental product, kind of in preview mode, and just went viral. So the GPUs are going, burr. You guys are, you know, the talk of the town. Do you agree with that take? I mean, ChatGPT was pretty big for me. And so to imagine the comparison there for me feels a little bit like, whoa, is it? But I think what we're seeing from a lot of people is that it's having a similar impact of like, wow, this is AI. This is what AI can do. So that's been really cool.

2:25Yeah, and I think I would agree in some sense that the first time I listened to an audio review, you know, when that second host came on, it really was like a mind -blowing experience. But I think it's also like at the underlying layer, like the fact that we have Gemini 1 .5 Pro, digesting all this really complicated information and spitting it out in a way that's pretty concise. I think the combo of those to me is definitely a pretty unbelievable moment. Just for everyone who's listening, what is notebook for anybody that hasn't played with the products yet? Yeah, notebook is an AI -powered research and writing tool, but I think nowadays it's more commonly known as upload a source and then it will generate an audio overview for you or podcast.

3:07Yes. And did that happen by accident? Like did you start out wanting to build a podcast host killer or did that come out by accident somehow? I think, you know, honestly, we were always working on the different modalities for output. And the voice was the next one and we chose dialogue. Did we know that it was going to be a killer? I was saying no. I think I thought it was pretty magical, but the way that it's really landed with people has been delightful and surprising. What, and I know you guys have been working on notebook LM for a while. Can you take us back to the beginning of the project?

3:41You know, what was the initial idea? How did it come to be? Yeah, I mean, I remember I was working on AI Test Kitchen and this was last year. And notebook actually started as a 20 % project. We had one engineer who had been working on something called talk to small corpus. And super funny. I don't know. I know. Great consumer brand. I was like, what is a corpus? But then I had a child with him. And he was like, hey, it's really the idea that you can use LLMs to talk to your data, try to extract stuff from it. And I was like, oh, that's super interesting. I started thinking about, okay, what are the practical use cases here?

4:20And I actually went to school as an adult learner. And to me, I was like, wait a second. If I could use an LLM and I understood what LLMs could do, I could use this maybe to talk to something like a textbook. go this is pretty exciting I can see how that could that could change my life it could change lots of lives and that's when we started really revving on like hey what do we build to introduce the first version of this to people and you know it was in May of 23 that we introduced Project Tailwind and it was just that you uploaded a source a PDF you can chat with it yeah I think that the fact that we are source grounded is what makes the product so unique I think even when I I started thinking about this project.

5:02I didn't realize that everything in my life that I create often has some sort of prior artifact or document that I used to create something new. And so I think, you know, right now at least I would call this a source grounded tool, but we're really becoming a source grounded tool for creation and a bunch of other stuff as well. Are there any stats you can share about notebook LM? I think what I'll share is that we were on a steady growth path before audio overviews, but since we launched it, it's sort of rapidly accelerated, and that's been really exciting. It's been a really good hook to bring people into the product.

5:38I think the other thing I'll say is that while it brings people in, people generally stay for the rest of the features, and that's been also really interesting to see, in terms of what people are trying to get out of a tool like no one. So the podcast or the audio overview experience is absolutely magical. Can you tell us a little bit about how it works behind the scenes? Like how did you make it so lifelike? How did you make it? How did you make a dialogue so good and engaging? It just draws you in. Like how do you do it? Yeah, I mean, first I'll tell you it was a lot of work. There was a lot of teamwork.

6:10There is a lot of craftsmanship that really went into it. But at the heart of it is really Google's models. You've got Gemini 1 .5, which is such an incredible model in terms of taking all of that data that you give to notebook LM and then producing something new out of it. And then you have the voice models, the audio models that back on notebook LM. I'd say the real sort of powerhouse between those two is something we've built called content studio and that's really what brings to life sort of the editorial right between you bringing your content and then coming out with the podcast there's some editorial liberty that we take with the studio.

6:48And so in the future you see yourself exposing the studio out to people You know make this make this one funnier make make this one more serious So I think like that we hear a lot like particularly because so many people are using it so many people are delighted by it I think the next step is then people want the knobs, right? They want to be able to control it and this is where you know my gut reaction was okay, let's ship the knobs But I'm trying to have a little bit more discipline in thinking that hey, you know people found love with it because it was delightful, it was magical. How do I ship the light full magical knobs?

7:25Totally. And that, you know, there's only so much I can do. But, but I think there's a way. And so I'm very interested in doing that. I actually do think part of the explosion of audio reviews was the fact it was a simple one click experience. You know, I was on the one with my grandma trying to explain her how to use it and it actually didn't take any explanation. You know, I'm like, okay, drop in a source. She's like, oh, I see, I click this button to generate it and think that the ease of creation really actually is what catalyzed so much explosion And so I think when we as we think about adding these knobs, I think you want to do it in a way that's very intentional Also just like fun You mentioned people come for the podcast and then stay for everything else Whether some of the best use case of you as you've seen for the everything else I think I'd say one of the most surprising ones I talked a little bit about the educational use case.

8:16It was very personal to me. And I saw a lot of students and a lot of educators using notebook LM. But what's been surprising is to see the amount of people that are using notebook LM at work. So one good example is we run, we ran a pilot case study inside of Google. And so within the ads team, we have a lot of ad sellers, ad specialists. And I didn't know this. But for these ad sellers, a lot of their sales training and documentation are hundreds of pages long. How does anybody learn this? And then the stuff changes all the time. And so it's very hard to keep track of how something works, you know, well enough, such that you can sell it.

8:56And what the sales teams normally do, or before notebook, I'm what they would do, is they would ping each other. Maybe like, hey, Joe, right? How does this thing work? How do I position it for this client? You wait for Joder to respond and then you'll go, okay, let me copy paste this into an email, fit a little bit, and that was it. But it turns out people like Joe, who have a lot of that knowledge and read all of this documentation, they build the notebooks and then they distribute it to their sales associates. And then that's hundreds of people automatically that are using the notebook because now they don't have to ping Joe.

9:30And that's really interesting to me because I was like, go, is there a really simple use case? And then there's so much more that you can really build on top of that. Totally. Actually, I was just here. I was talking to a friend who's in sales, who was like, dude, it's great. I made this whole notebook and I went on calls and I heard the answer. I can quickly ask and get a response. And so I think that this idea of knowledge distribution is really helpful for large sales team or data centers and stuff. I think another use case that I also think is really interesting that actually you may align with is I have a lot of friends who work in venture in PE and this idea of a confidential information memorandum a sim.

10:08You know, I never heard of this before, but I have a friend who he's like, this is my whole job is basically going through these packets of information. And so what I do is I take these, you know, these documents I receive or slides, I put it in a notebook and I'm able to now way faster than before, go through all this fairly complex information. And I think that he was telling me he like 10xed his job speed, you know, which is great. And it's just this is empowering him to be faster. Podcast host and venture capital. You are really going up to our jobs. I we're helping out your job. So I don't think what have been the moment that surprised you the most for me.

10:46It was, you know, the moment where the AI hosts kind of realize that they're AI that was like, it's really cool moments, but what have been the moments along the way that surprised you the most? I mean, I'll start. I was going to bed. This was last weekend, I think, or just several days ago. And I was on Twitter, probably not healthy to be doing this before bed, but I was checking Twitter. It's fire by the way. I was scrolling through and I saw the poop fart one. If folks haven't heard it, I also So now I'm just like, this is a very important mention by the way, like, did you haven't heard it?

11:22You really do need to understand how not to go this one. Okay. Folks have not heard the poop fart one. Somebody just, somebody decided they were an uploaded document where the, the only words in the doc were poop and fart over and over and over and over again. So it was a pretty lengthy doc, but it was just those two words. And I saw that's what they had done with it. They described it and I was like, Oh man, should I listen now? It's 11 o 'clock if I tap this and it's a safety flag. I'm not gonna be able to go to bed Right, cuz I'm gonna have to open a bug. I'm gonna ping the engineers. It's like hey, we got this thing going on I was like I'll just listen and it's actually unbelievable.

12:00I Also like I saw it and I was like oh like we let's let's let's see what this is gonna be Yeah, and you listen you're like oh this is this is fantastic like this is even better than I could have ever imagined It was one of those words where I was like, well done notebook. I'm using it. Good. We're solving the right problems. Amazing. What design choices have you made that have made notebook work so well and so intuitive for people? I think that I clarified we're still making those decisions. I think right now we're very much in the process of launching quickly and then working closely with our users to understand what's best and what they want.

12:39you know, tech is evolving so fast right now that it's really hard to know what's even possible. And so I think that we're really pushing for this model where we launch quickly and then work kind of alongside our users to build the best product. But to answer your question more specifically, I think that one thing that we've done that I think was almost a happy accident in a way was make that left source panel really clear. I think that we are a source grounded project and we need to make that clear, that you're talking to the sources that you've uploaded. And so I think that having those sources that are on the left is a pretty crucial part of this project.

13:15I do also think, though, as I was mentioning earlier, audio overviews being one click seemed to actually pay off that it would be like really leaned into the simple experience. But that being said, there's a lot more coming and we're actively working with users to improve the project product. I think one of the things that I'll time in in terms of like design choices and really I think on the product prioritization sort of side of things is really thinking through what does it take to make something new intuitive and It's really hard especially something as nuance does like all first you have to upload a source like users generally like bulk at that step of like why Right like I don't have to upload a source to chat GPT.

13:58I don't have to upload a source to Gemini. It just works And so I think we still have a lot of work to do around it just works category. What do you think are the biggest challenges remaining as you you know kind of bring people on to this new AI and experience? Yeah I think that we're kind of in this quote unquote skew morphic era of AI design and I think to explain skew morphism it's when a virtual object reflects a real world object and that was seen in early iOS when the The Note app had a leather bounding at the top and the pad was yellow. And that was made to kind of ease users into this virtual world from the physical world.

14:37I think now we're seeing something similar with AI, where we need to build UIs that help meet users where they are. And I think right now we're doing our best to kind of be really creative and think about these new, kind of crazy experiences. But also understanding that many of these users, this is the first time interacting with artificial intelligence. How do you think about, you know, one thing I think mid -journey has done extremely well is just making it easy to, you know, get over the blank wall prompt problem. And so, like to me, that's something that mid -journey has done extremely well.

15:09Are there any other applications that have kind of approached some of these UI challenges that you admire? I have one recently. I just tried Pika. And I really love the Pika effects, where you can see exactly what's going to happen to your image if you upload one because Pika is similar to the extent that you have to upload something and then you have to maybe write a prompt to choose an effect. And I think it was really well done where it's like, Hey, here's a preview, right? It will squish the thing. I was like, Oh, this is fascinating. And of course, I uploaded, there was like one that was like cake.

15:44I uploaded a drink, a picture of a drink, and I was like, make a cake. And just like the anticipation of the drink is going to become cake. I was like, come on, come on. I was like, show P for it now. It was like my first generation to I was ready to pay for it. I know. That's, that's how I knew. It's like, Oh, like there's definitely something there about show the user what's on the flip side that I think really incentivizes the user to not only, you know, give you the image, but like they're really excited to see what happens. And then you know, if you're me, you're like, take my $10. Yeah. And so it's really effective.

16:19You know, I think for me, I love clawed artifacts. I think I've done such an amazing job of co -creation. We've talked a lot about writing co -creation and it was awesome to see kind of others in the space thinking about that as well. I think that right now, as I was just briefly mentioning, like we're in this space where we wanna equal the hierarchy between AI and human and I think we definitely don't wanna take your job because we just wanna help support your jobs, you know? And I think that cloud artifacts was a perfect example of that in my mind, which was cool. You can talk to the chat and also start building out something on this right side as well.

16:55How do you think your product, kind of, you know, comparing contrast to the approach that cloud has taken? Like, do you think it's similar things that you're going after or how do you think about the differences? I mean, I think first and foremost, right now, at least we're a source grounded tool, which kind of immediately makes us a bit different. That being said, I think we're thinking a lot about creation broadly, utilizing the sources that you've uploaded. Well, I think to that point, the contextualization of your LLM interactions is really powerful. And I think it creates a stickier user experience.

17:29I think that, if I had to guess, the folks at Claude probably know this or the folks at Anthropic, the folks at OpenAI probably know this, certainly the people at Google know this. But I think there is a question of when to introduce it and what are the right surfaces? So I think this is where for notebook LM I'm excited because we started there Right, so there's like a little bit of hey as people catch on to the importance of like source grounded workflows source grounded stories This could be the tool that they're looking for and if we just sprint at this Hopefully we'll get farther along before before everybody else who's juggling all of these other use cases You mentioned earlier that chat is kind of a skew morphic interface for AI, and that you guys are experimenting with crazier things.

18:15What might the crazier things look or feel like? Give us a taste. I think that's at a high level. I'm super intrigued with these dynamic UIs. Actually, the cloud is an example of that. You see this artifact come in that wasn't originally there. I think we're thinking a lot about how to, we're trying to do a lot here, reading, writing, and I think there's only so much that you can do before a user gets overwhelmed. And so I think we're really exploring how do we take advantage of what you're doing at that moment, while also not overwhelming you with all the other possibilities. I think for me I think a lot about leaning more into new modalities, which is what does it mean from an input and an output side?

18:58And I do a lot of prototyping on my own and experiment with a lot of my own behaviors. And one of my favorite ones is sort of the idea that I can walk in talk with my LLM, right, or with like an AI sort of ecosystem. And one of my favorite recent examples is I started doing this with a daily journal. Or instead of writing my journal, I just go back and forth and then creates the journal log for me. And then it creates a visualization of basically like, hey, you know, this week you had more bad days than good, or you had more good days than bad. Here are the things, right, that made you happy. hear the things that made you upset.

19:34And I think there's a lot of richness there in interaction where I think about it. It's like, hey, source of grounded AI. Of course, there's like some really practical sort of work use cases there. There's some educational use cases. But the personal ones are also really compelling. And so I'm trying to think about how do we take these learnings and bring it back into notebook LM and you know, probably in something like the mobile app. Right, So you have lightning in the bottle with notebook LM. Where do you hope to take it from here? I think honestly, just keep going. I just want to keep building more cool stuff.

20:12We want to deepen the experience for users. We want to make it really useful. I think there is a lot of magic right now, a lot of delight. And I think we want to deliver on the promise of that initial hit and just show people like, hey, you can use to go around. That's going to be great. What do you think is the biggest thing missing from the product today? Oh, well, I'll say if I could rewind back in time and build more things as part of this launch, I would definitely build a better share experience. Just the amount of, you know, when I scroll on X and I see all the videos and visualizers that people use instead of like our native one, you know, as a product lead, I'm like, oh, I'm missing out on counting this user here, because they're on a different surface now.

20:59So I think for me, it's really the sharing and sort of collaboration around the audio reviews, that's missing. And I think as we started talking about, I think I'm really excited about the addition of a writing experience. I think that we know that people are often doing Q &A and then taking that answer then create something new. And so I'm just excited to help kind of fulfill the whole user journey. How do you make it, like, are you prompt engineering to make it kind of like do tell it to be like, you know, conversational funny, like, what are you doing on the technology? Yeah, how do you design the personalities?

21:34I'm really curious about that too. So there's a lot that we do in the background. And I think you hit on some really good aspects of it, especially around, you know, the show is called Deep Dive. There's clearly two hosts. And what I'll say is that there's a lot more editorial liberty that the personas themselves take to generate that show. And I think that's where, you know, even for me, I am always interested to see where they're going to take the show based on the sources that are uploaded. Oh, interesting. So you've given each of the sources its own personality, its own how the approaches things and then you let it create the podcast.

22:15Yeah, in a nutshell, I think that's the best explanation for what we've got going. And so when we think about the editing experience, right, it's like, oh, what are the controls for something like that? Of course, there's like basic stuff, which is maybe I don't want a deep dive. Maybe I want a different show. Maybe I want a different length. Maybe I want shorter. Maybe I want longer. Maybe I just want to specify a topic instead of the whole thing, because today is like an overview -based audio. So I think there's there's a lot there that we can tweak but the heart of it is really this editorial liberty around your sources and trying to give you an overview of it.

22:52And every time I have jokes that you're going to take our jobs you say you say you're not but I don't know if you're just saying that's nice because what you've generally this legitimately so good and so the real question I have is when you say that it's not going to have to replace real podcast like why do you say that because it to me it feels going to have to replace real podcast. I think there's a good question and one that I try to approach really carefully, particularly because hey, if there's real risk, I want to look at it in the eye and say, okay, how do we address this? But from what I've seen, a lot of what people are making are not the same things that I feel like we would have a real podcast about.

23:31Do I want to take an article and make a podcast out of it that replaces one of my favorite podcast, Lenny's? Right, it wasn't a Lenny's all the time. It's like, no, I want to listen to Lenny. I want to listen to what he thinks about this particular topic. And what's funny is like people are making audio overviews of things like their resumes, right? They're linked in bio or startup founders putting in their landing pages and trying to figure out, oh, was my messaging clear. Like, that stuff is really cool because it's like, no one's ever going to make a podcast of that. I mean, maybe not at this stage, right?

24:01But that's where I think, okay, this feels really good. but it feels like we've created a space where personalized generation really is about meeting my needs exactly where I'm at and there isn't an existing thing out there. And that's really special. It does almost feel like a different media type. Like it sure it sounds like a podcast, but I think you give great examples to kind of prove all these random use cases people are using it for. But I think there's also a reason that reaction videos are so popular online. People aren't just listening to this right now because of us because they want to hear from both of you who are in this space.

24:35And I think that's also important to remember when they came out podcasts. I will say like one interesting thing about the dynamic is even though people are sharing the audio overviews that they're generating, they're very personal. It's like, I made this for me. I didn't make it for you to listen to my resume. It was me. I was delighted by the audio overview of my resume. Or there's this really cool TikTok of where this woman uploads her diary from 2004. It was interesting to listen to that together, but it was really her reaction to her diary that, you know, she wasn't going to listen to a podcast about that ever.