Andrew Mason: Craft and Control in AI Content Creation with Descript

0:28Hey, everyone, and welcome to Generative Now. This summer, Descript has rolled out a new suite of AI-powered features bundled into a tool they call Underlord. Andrew and I talked about Descript's unique approach to audio and video editing and the future of media creation in a world with AI. So check it out. Hey, Andrew. Hi, Mike. Thanks for doing this. It's my pleasure. So nice to see you and get to spend this time together. So one cool thing about this conversation that you know, but for the audience's sake, is we're talking on Descript Rooms. And this is like your solution for capturing podcast videos and interviews.

1:08Is that right? Yes. It's like our third or fourth attempt at a solution. But I think like the final solution that is the correct one. So we just launched it. But yeah, it'll be rolling out over the next several weeks. yeah it's awesome i one of the the first thing i noticed about it which i really like is that the interface is white i find that a lot of the other um products for this they're all dark and that actually darkens your face right like you're doing a podcast you want it you want to be like lit up and you want to have like good lighting but then you launch these tools and it just like darkens everything so it was the first thing i noticed about rooms yeah i think it does have dark mode for those people that would rather be dark.

1:52Um, but Descript is a, is a lighter UI. So, um, yeah. One of the interesting things I thought about this when, when I heard we were using this and I heard about the feature is I feel like you and I've been talking about Descript for years. Obviously, you know, I'm a huge fan of the product, um, especially given my, my love and experience with podcasts. Um, but I feel, I feel like Descript has always walked this fine line of being really, really good for podcasts, but also really good for other things. And, and rooms kind of, to me is, is Descript saying like, we are a podcast tool. Is that, have you, have you thought about that or have you thought about like that statement with this tool?

2:34We think of Descript as a, as a horizontal creation tool. Um, Um, people will often try to pigeonhole us in terms of whether we're for podcasting or whether we're from video. We obviously started out with audio podcasting and we just divide up the tool space a little bit differently. We think there's, you're either making narrative media that's driven by speech or you're making like music and there should be one tool for making narrative media. and it shouldn't be specific to audio or video or unscripted podcasts versus a scripted like creator style video or explainer or something and then we think of remote recording what we're doing with rooms as kind of table stakes for a modern creation tool right it's like it just it just stands to reason that if you're trying to build something that is collaborative, that you are going to include as part of that the ability to record multiple people at the same time.

3:45Podcasting is one of our biggest use cases. It's one of our strengths. A lot of people use Descript primarily for podcasting. But our hope is to build something that can span the full spectrum of audio and video creation. Trying to subdivide it into categories can be tricky and not something that users should have to think about too much, we don't think. So we just try to make a tool that works for everything. That all makes a ton of sense. Obviously, podcasting is how I heard about Descript when I was building at Anchor and Spotify. But I know because I hear from people all the time that they're using it for a bunch of different thing is you mentioned podcast is still one of the biggest use cases like what are the other forms of narrative media have you seen emerge as the biggest use cases just broadly it's podcasts video content and clips where clips is taking longer things and then pulling highlights and cutting them down into stuff that you're distributing as promotional content on on social media um video content is inclusive of a bunch of businesses making videos for um content marketing or support um that sort of thing and then obviously like creators making it for their youtube channel or or even shorts speaking of shorts the clips thing has gotten crazy like i i feel like there's this thing going on in the world right now where the clips is kind of more important than the long form, at least in terms of public consumer-facing content.

5:23Have you all started to think about it in that way at all? It's a hugely popular use of Descript. And what surprised me is the degree to which people will use Descript and other tools like it to load in a video and just simply add a burden captions layer or something like that. Like we're going through an interesting time where I think there are people using tools for extremely thin benefits that will get consolidated one way or the other. But it's like the sole job to be done of a subscription that they have right now might be like adding a captions layer onto a video that they edit entirely somewhere else.

6:06You mean the tools get consolidated that they're paying for, they're subscribing for? or for sure the types of content get consolidated into like one you know suite i think the tools get consolidated i assume that that there are like clips will stick around for a while but the idea that somebody is going to pay twenty dollars a month for something that only lets them add a caption layer to a video i don't think anybody thinks that's that's how it's going to be a couple from now yeah i mean and also it feels like um the apples of the world the googles of the world like they're sprinkling more and more you know exactly ai editing into their you know uh tools that are that are baked into the os so yeah i think that's actually a good point i could see a lot of like the table stakes stuff just getting like sucked into apple and google workflows it's something that we've seen since we started descript seven years ago i'm sure you saw this a lot with anchor is that you'll see these little kind of explosions of growth and activity around a particular job to be done that's important.

7:18But there's a question of whether it in itself is a sustainable, it's actually like a standalone business, or it's just going to be something that's a feature that might eventually end up getting commoditized. And I think the tricky job of the entrepreneur is to see through which ones are worth to go after versus the things that you should just stay on course. And that's been just turned up 10x by the arrival of Gen.AI and this explosion of little point solutions for different types of things. given that clips are becoming like a form of first-class content like how much do you think about the platforms and how the platforms will need to evolve or should evolve to support this like clips sort of feel like a bit of a hack to me like you want like the long-form content is the is the is the primary content vehicle and then we're sort of like hacking tiktok and reels with these short things to drive consumption back to the long-form thing like at what point should these should things just kind of merge on the platform side to support both and it just it's just like one unit of content you know i'm saying absolutely yeah there's a there's a world where the concept of the clip is just a um a view setting or something like that or like like really like clips feel like the kind of thing that should be personalized um or could very well be personalized to the individual user based on what their interests are when you're trying to build a video creation tool how do you carve out something how do you carve out like a category of video that you're going after where the value feels sustainable and unlikely to be subsumed by what platforms are doing so we look at things like like batch creation of a thousand different videos for ap testing ads or something like that is the kind of thing that um is unlikely to be a sustainable competitive advantage for us and what we try to focus on is the type of video where craft and control tends to matter most creator videos and most videos on a business's youtube page today there's a degree of of craft and choice and that goes into that it's really hard to imagine how ai is going to fully subsume that stuff anytime soon there's definitely categories of video where gen ai can drive the creation of those videos especially like more emergent categories videos that maybe previously would only make sense as a dock or a slide deck for example um but a lot of the kinds of videos that are created today require a level of human touch and and that's where we think we're really strong that definitely resonates coincidentally this morning i watched this i don't know if you saw this video that casey and istat put out about this like fire hydrant like fish pond in brooklyn you see this no um it's amazing you should watch it the human touch like is all i actually as i was watching it i'm like wow like nobody could make this other than casey and i think in the world that you're talking about it almost feels like premium content almost gets more premium like the best stuff is considered really really good and then there's probably some line where everything below it can just be commoditized by AI, right?

11:06I'm finding it really hard to predict what the world is going to look like around this stuff in a year or two. Five years ago, most people would blanch at the idea that they would one day be listening to audio books that were completely generated by AI speech because it's really, really hard to imagine what it's going to be like once it crosses out of the uncanny valley and it actually feels good enough. And now we're basically living in that world where it's so good that you might not even know most of the time that you're listening to something that's synthetic. AI avatars, I think, are on the verge of entering a similar category.

11:50But then they're different because one could make a case that AI speech is utilitarian. There's a practical need to deliver content through the ears, but AI avatars actually isn't delivering information. It's just window dressing on the video, right? The information is coming from the speech. And so the choice to use synthetic human feels like a deliberate trick to force a sense of connection and authenticity. Once it gets good enough, will people feel the same way about it that they do with AI speech? I don't know. It's really hard to predict that kind of thing. maybe this is what you're getting at but does it reach a point in which it's just so obvious that everything is synthetic so we don't need to feel the forced fake human connection and everything is just synthetic and like there's need to be a fake avatar right just like just play the voice i know it's a robot it's like i get it you don't have to try to connect with me you know yeah it's wild uh it's it's really interesting uh to see how much has been done with synthetic voices synthetic video, synthetic images, and be talking to you because I think Descript, Descript's one of the first companies, maybe the first company that I remember stepping foot into this world.

13:22You know, you guys acquired a company, Lyrebird, back in the day that I was super familiar with because I wanted to partner with them at Anchor. And when I saw that you had acquired them and what you were hoping to do with it, it, it seemed like the future. It seemed like science fiction in a good way, in a cool way. And as you're saying, like now we are, we are fully, fully there. Every company's doing this. And it's happening so fast too. So AI for Descript has come a super long way. You started with Liarbird, which I, how, what, what year was that? Was it like five, six, maybe even eight years ago?

13:57It was about five years ago. Yeah. Or 20, 2020. And now, you know, you have, you have Underlord, you have all these different AI features. What features, I guess, have made the biggest difference in either the creation of the content or the output of the content for Descript and its users? We raised our Series C from OpenAI. And when we announced the Series C was maybe a month or two before ChatGPT was released in gpt4 when we were working on the round sam altman gave us a preview of it and he framed it as i'm going to give you access to this thing and um maybe half a dozen people have seen this and all of them have completely changed their roadmap as a result of it that's so this is chat chat gpt yeah gpt4 in a chat gpt like experience so our first version of it was we had like a slack a slack bot that was basically chat gpt sure enough like we played with it and it was one of those things where you just never forget it you know like the moon landing or 9 11 or something and uh i don't know which analogy is going to prove to be more appropriate we'll see uh we like everybody else kind of rushed to think about what do we do with this what do we do to take advantage of the technology that this alien that's just landed on earth is going to is going to provide for us it's there's so much that we could do and i think what we've learned with some distance is that the most important impactful AI innovation for us to be focused on is the same old boring dumb one that kicked off Descript, which is fulfilling the promise of the text-based video editor, audio video editor.

16:02That is AI, right? I mean, we don't think of it that way as much anymore, but it was progress in automatic transcription and the phonetic alignment between text and audio that enabled the creation of a text-based audio video editor. And our ambition from the beginning has been to, if you could reinvent the interaction paradigm for video so that it feels like something, like the gold standard of creation tools, which is the word processor right there's like no cognitive distance between an idea and its expression if it's just so fast and intuitive if you could pull that off for video then you really unlock the medium and getting that right is super super hard you have to figure out like if you think about it text is a um one-dimensional tool like you're either going forward or or back with with words and video and timelines the timeline editor that we're pulling from is two-dimensional you have both time and then you have you have like a stack of of layers so you have to figure out how to kind of port everything over working with one less visual dimension and representing that in this one-dimensional space and make it feel completely natural like it couldn't be any other way.

17:33And so a lot of that is just like good old-fashioned product work and design work. Descript is fundamentally a design idea that's propped up by technology that we're trying to prove. And all of the generated-based AI stuff is super transformative and important, and we have to do that as well. It's really easy to allow it to distract us from just fulfilling that core thing, which is what I think is the real unlock for Descript if we can pull that off. Would you mind giving some examples of maybe like since ChatGPT, what are some of the aspects of that core vision of making audio and video text editable that have been changed by sort of the iterative innovations in AI since ChatGPT.

18:27I get speech-to-text, right? And now let's edit the text and let's convert it back into video. But how does each subsequent version of GPT impact this? How does some of the other tools like, I don't know, Sora or Genera Video, how does it get back to that core mission with each of these new innovations? One of the most popular features we had before which seems like trite these days uh but before uh generate chat gpt came along was called was filler word removal so you would record something and then you'd click a button and it would remove all your ums and uhs and likes and so on um and we had you know plans to take that further and further oh you could if somebody does a script or scripted recording you could remove what appear to be retakes.

19:19If somebody does an unscripted recording, you could remove kind of rambling diversions or stuff like that. And when ChatGPT came along and these LLMs, it was just like instant AI buffet for stuff like that, for kinds of auto editing of the script that it could do really well. So it was just like an accelerant on the stuff that we knew that we wanted to do. some of the other stuff that we're we're working on now is like doing similar types of automatic editing but on the visual layer so storyboarding a video dividing it up into different shots and then filling those shots with with content a lot of tools do this already but nothing's very good and i think that speaks to it being still a relatively hard problem to do well some of that is the fact that most of the models are doing it are just looking at the script.

20:14They're not looking at the pixels on the screen. So as this stuff gets multimodal, the quality will get better. But there's also like, if you're just going through, you know, a library of stock video to have kind of cheesy B-roll over your camera recordings or your audio recordings, it just feels kind of cheap. And that's what a lot of them seem like. So I think like, And we talk about that internally as window dressing. That's the term that we use. If you think about a lot of video, this video is a good example of it. Most of the information is actually being transmitted by audio and the visuals are just there to keep people engaged and interested.

20:58Occasionally, it'll show a chart or something, and that's actual information. But it's mostly window dressing. And I think figuring out how to create strong window dressing is still unsolved. Nobody's done a really good job at that yet. The multimodal stuff is super interesting. I'm glad you mentioned it. I was thinking about that a little bit earlier in the conversation that I have to imagine everyone at the team right now when they see you know gbt4o being demoed or you know more and more companies are talking about their multi multi-modal models i have to imagine that just triggers like flurry of ideas for your team when you can start to talk to descript or descript can start to take in other forms of interaction from the human and vice versa um am i right about that are you all really excited about these models i think they just overall improve quality if you're just using text and you're not listening and you're not looking at the pixels then you can only tell half the story like if you're listening and you can hear um edit quality you can hear tonality you can like things like you can hear sound quality there's a different set of um like you've just really increased the scope of what kind of judgment this uh ai assistant can have and the ways in which it can be useful what about even like the interaction layer i mean you mentioned earlier there's nothing more efficient than a word a word doc um like are we ever going to talk to descript like it's you know an edit an editor an assistant sitting right next to me we're working on this thing together in real time like how much of the ui gets abstracted away over time as the ai improves when gen ai arrived there was this um belief that basically the future of ui is going to be a text field or speech and it's it's that simple i don't think it's that simple If you're a writer working with an editor, the tool is good enough that you're not dictating what you want them to do.

23:12You're going in and you're putting notes in there yourself or you're making the edits yourself with the suggest mode. It's sometimes hard to capture in speech the kinds of control and craft that people want to do. So I think that's an important part of the equation, but maybe what makes us different is how focused we are on just making the, like reinventing the tool and getting that foundation of a really strong tool figured out. Maybe let me invert the question then. At what point does AI start to drive the UI? So instead of an assistant going off and doing a thing for me, how much now does AI help make the interface and the experience of Descript dynamic based on what I want it or need it to do?

24:04um maybe let me take a step back a little bit it feels like we're we're we're moving towards a world where software interfaces these things like literally can be dynamic and generated on demand based on the need like do you think about that world at all for dscript especially for workflows that are so visual and dynamic and based on all these different modalities yeah have you seen any examples of of that no not in production yet but i've definitely seen a bunch of examples of like here draw you know draw this interface on the pane and like the ai will turn it into a product in real time right and then i think about you know tools like devin or you know magic uh another one of these automatic coding things like it just seems like we're moving towards a world where like software is going to be dynamic right and i just imagine that opens up possibilities for a product like descript i think there's like a literal way to interpret what you're suggesting, which I don't think is what you're suggesting, which is that, you know, you'll have like a, a panel with, with inputs that shift depending on, on what's happening.

25:09Something like that, like maybe like it, it, it seems tricky to do, to, to do well, because I think ultimately like people really value, like knowing where things are and the, the predictability of tools. but I do think there are some other existing design patterns that one can use just like suggestions and things like that that probably get it what you're what you're talking about yeah like you could it could be literally be processing the audio and video in real time and making proactive suggestions just as it's as it's hearing or watching the content that's pretty interesting actually to talk about Underlord like I it seems very all-encompassing and it's I think it's fairly new is that right yeah Underlord is our take on an AI assistant because nobody wants a AI overlord but having an Underlord to do your bidding we could still control it seems like it would be a nice thing so Underlord is a side panel in Descript that kind of houses all of the different types of things that you can delegate again we think of it as like a kind of eager, well-meaning intern.

26:26And those are the types of things that it can do. It can do a lot of those like script editing things I was referring to earlier, like removing multiple takes or editing something for clarity. If you ask it to, it can improve sound quality. It can add chapters or generate show notes or a blog post based on your video. like a bunch of things like that and the list is growing monthly of the kinds of capabilities it has we can translate that's just like our version of siri or something that's really cool um which of those features do you feel like customers have been more willing to embrace um versus maybe the ones that they're not ready to sort of trust and and hand the keys over to an ai for the ones that are really popular are some of the most boring ones they're things like create a blog post or create chapters or stuff like that like the easy stuff that you could do a week after gpt launched the stuff around auto editing um some people love like and some people have more more mixed results um with with whether it's working and some of that comes down to like when you're editing people's recordings are the, do the edits sound good?

27:42And do they, do they look good if there's a, if there's a camera component to it? So I think that's also an area where multimodal is really going to, um, is going to be helpful. That makes sense. Do you find in general people, not even with specific features, but do you find in general users are, are willing to, to start trusting these things or is there like an overall hesitancy, um, with leveraging AI for, something that is so creative and so personal. So I guess overall, where do you think the trust level is of consumers with AI inside of Descript? When we're building new features and we're building new AI features and thinking about how they make it into the product, if this makes sense, we're thinking about building new knobs and dials for human-shaped hands.

28:31and then we build like apis to let alien shaped hands control those knobs and dials that's cool the idea there is that like the ai is just doing things that the human could do if they wanted to and they can undo them they can go in and fine-tune any decision that the ai has made it's not kind of like a one-shot thing it's all just going into this this tool um same knobs so same knobs for human and alien just two different sets yeah the alien is just controlling the the the human knobs they're not accessing features that the human can't reach or doing things that the human can't do and i think that's different um than a lot of like the point what you see in a lot of the point solutions that have that have come along for video creation where it's more like just you know add a script and we'll give you this fully made um slop video and it doesn't feel like it it's good enough and then there's no recourse from there because you can't really get in and and tune it into something that's good enough where the experience of doing that is so clearly an afterthought um so so that's how we've tried to launch these these features in a way that maintains trust where people can try them and if they like them they keep using them if not they just use the human um version of stuff that's really really smart um and i and i have to imagine like would actually very much help bridge the gap for people that need to like work up the trust to to use these things i wanted to talk about maybe detour if you don't mind uh obviously detour was the project you all worked on before leaning into Descript.

30:22It was audio, you know, audio walking tour product for cities. Really, really cool product. I wondered about you, like, do you ever think about that product and think about the tools and technologies we have now through AI and think about what you maybe would have done differently or what is now possible that maybe wasn't then? I don't think about it a lot, but it's hard not to notice that it feels like there's an opportunity to take another swing at that space in the face of Gen.AI. And maybe someone's doing this already. I'd kind of be shocked if they weren't. It's a very different product. So the idea of Detour was that we were going to make these prestige audio tours that were high production value, narrated by the people that knew the city best.

31:15And this was part of the failure of Detour is just we could not figure out how to scale content creation. It took six months to make a tour and would cost anywhere from$20 ,000 to$30 ,000 just to make a single tour. and then you know when you're giving apple 30 of of everything and just like all the mechanics it's it really had like the worst imaginable mechanics as a as as a business even though it was a special product if you actually went through all the trouble that it took to to try it out um but but the fundamental idea is like audio tours are sterile and dry and boring and they could be so much more cinematic and immersive if you take advantage of this augmented reality mobile phone experience and really invested in getting the storytelling right.

32:04The version that would be interesting today is something that's completely generative, right? And it's completely responsive to where you're going. And I think it wouldn't be as emotionally profound as what we were creating, but it would definitely tick the box that a lot of people are looking for, which is satisfying curiosity and getting information and some kind of connection to a place. So it's a product that feels like it's worth existing, building, and probably way less niche than Descript because it's so much more scalable. Everything you said makes sense to me. I do wonder if it actually could be emotional and cinematic, just given how good some of these models are getting.

32:43I'm sure you saw 11 Labs is, you know, licensing all, you know, these personalities from these estates, you know, imagine you're walking around a certain city and like some famous person associated with that city is like, oh, let me tell you about this building that I used to live in. Or I don't know that just it could get pretty wild pretty quickly. And then the other thing I think about with detour is like, I can't remember when you started that, but AirPods were definitely not as ubiquitous. no they weren't yeah that would have been great and there was cool things you can do with the air yeah airpods were this incredible moment where um just taking away that tether taking away that cable turned headphones i think for a lot of people into something that you would bring with you when you needed them like when you knew you needed them to all of a sudden they're just part of your daily carry you always have your airpods in your pocket along with your your wallet in your phone and that definitely just changed the game for something like detour where people can be a little bit more spontaneous sure you've also thought of like where some of these devices could end up going you've probably seen the concepts of the airpods with the camera on them or you know the orion glasses which just you know just got demoed from uh meta the other day like the the opportunities uh seem seem pretty interesting for for a product like this i'm curious ask you the same question but about groupon um you know groupon a lot of technology didn't exist that exists today when you were building groupon and i wonder if ai could have improved that product in some way groupon i would have um fought you kicking and screaming screaming if you made this assertion when i was running groupon but it wasn't really a technology company it was a glorified blog where we would have a daily post of um of some cool thing to do it became more of a technology company over over over time where we were doing personalization and a bunch of different and and eventually be trying to become like an operating system for local commerce um in a way that i don't know maybe square has been the most successful at but hasn't fully materialized um how would you use ai for i have some ideas okay what do you got so well the first like maybe most obvious one is you just mentioned personalization obviously you could leverage machine learning to make the discovery of stuff really really hyper personalized in a way you probably couldn't back then but the the bigger idea that comes to mind for me uh did you see some product just launched called arcade where you could literally like describe something via gen ai and then it'll actually make that physical thing it's a jewelry thing i do wonder if you could use gen ai to almost like do massive multivariate testing around what people would buy and once you reach that threshold like however many people you needed to convert a group on into an actual uh sale then you go and you produce that event or product or so basically like you're making up concepts to be sold and if a hundred people say they're interested then you go and produce them interesting yeah would be hard to pull off i think the tricky thing on personalization and that we ran into like the the thing that broke groupon was um the the first version of groupon was like very simple one deal per day on something to do in your city we didn't have multiple deals we would only have we would only have one um and i would say that was always like the version that was most cherished by customers in those early days because it was all about discovery and um and putting something in front of you that is not what a personalization algorithm would would suggest and what made it so cool is like oh my god there's a like sensory deprivation tank.

36:50I will, I never would have thought of doing something like that. You know, now that I see it for 50 % off, if I'm ever going to have this experience in life, this sure seems like the right time to make that decision. Cause I don't know when this is going to come around again. I don't know when I'm going to have this opportunity again. So it created that like urgency around, around trying something that's outside of your comfort zone. um and then what undermined it was the clones and having 40 or 50 different group on like deal a days in a city so now all of a sudden you effectively have a liquid marketplace with a lot of different offerings and on you know that if that sensory deprivation tank goes away, it'll be available on some other site some other day, or in any given day, you can get a mani-pedi on one of these sites.

37:48I do think there's a version of Groupon that could be a really cool business that is still focused on discovery and one thing per day. If you really kept VCs out of it almost, it probably has a limit on how big it can get to make that concept work, but it could be a great lifestyle business. And I don't know. And that's the version where I'm like, I don't know if I would use it, like how helpful AI would be with that. Outside of Descript, what's your favorite AI product or feature that you've been using or loving or at least inspired by? So there's a note-taking tool that I have been using called Granola.

38:31It's similar to what you've seen with one of those other note-taking assistants, but it works a little different. where it's just recording your computer audio and in a little basically document editor, it's taking notes and taking an outline. And it does a couple of things that I think are just incredibly clever. One is that you're taking your notes in real time about the call in Granola, just in a blank sheet. And then when the call is done, it goes through and it augments those notes with details. and the outline that it comes up with is is like an outline of important bullet points and then you can drill into it and get verbatims from the actual transcript it's like it's a really clever idea and i i also just love using products where you're like god whoever made this is just good at their job like they're just a great like this is a person that loves products and knows how to make good products.

39:33So I just like have a lot of respect for the product chops of the people that made that. I couldn't agree more. I love it as well. Full disclosure, I'm an investor in the company, so I appreciate you naming it. Andrew, this has been a blast as always. Love talking to you. Thanks so much for doing it. Thanks, Mike. Thank you all so much for listening to Generative Now. If you liked what you heard, please rate and review the podcast. It really does help us. And if you want to learn more, follow Lightspeed at Lightspeed VP on YouTube, Twitter or LinkedIn. Generative Now is produced by Lightspeed in partnership with Pod People.

40:09I am Michael Magnano, and we will be back next week with another conversation.