The Impact of AI on Podcasting (Harvard Data Science Review Cross-Post)

0:00Hi, everyone.

0:01Katie Malone:For this week's episode, we're going to do something a little bit different. I had the pleasure of appearing on the Harvard Data Science Review podcast recently, along with Jon Krohn, who you may know from the Super Data Science podcast, and the HTSR Editor-in-Chief, Xiao-Li Meng. We had a great conversation about how AI is impacting podcasting itself, a little bit meta, but a really fun chat. And I thought it might be something that would be fun to share with this audience as well. So without any further ado, let me bring you the episode from HDSR about AI and podcasting. Enjoy.

0:41Jon Krohn:Hello and welcome to the Harvard Data Science Review podcast. I'm Liberty Vittert-Capito, the feature editor of the Harvard Data Science Review. And this month, the HDSR editor-in-chief, Shali Meng, sat down with two podcast hosts to talk everything AI and podcasting.

1:01Jon Krohn:Katie Malone is the host of Linear Digressions, a podcast about data science, machine learning, and AI. She's a physicist by background and has worked as a data scientist in startups, high growth tech, and enterprises, as well as teaching, speaking, and writing about AI. And John Crone is co-founder and CEO of the AI software company Y-Carrot, author of the number one bestselling book, Deep Learning Illustrated, and host of Super Data Science, the data science industry's most listened to podcast. So what can podcasting teach us about AI and what can AI teach us about the future of podcasting? Let's get into it.

1:44Katie and John, so nice to see both of you, even just virtually. And welcome to this episode of of HDSR's podcast. And today I want to ask each of you that what is your show trying to do? Who's the fool? And what do you hope someone takes away from listening? Maybe we start with Katie.

2:05Katie Malone:Well, thank you. It's a pleasure to be here. Appreciate the opportunity. I host a podcast called Linear Digressions. When we first started this in 2015, I would describe it as being about data science and machine learning. Like everything else, AI has eaten that world lately. So now that's more of the focus and that's kind of how I describe it to folks. And it has a little bit of an interesting origin story. I was teaching a course, an online course about machine learning at Udacity. And in the course of, no pun intended, putting that together, we had a lot of interesting content that just we didn't have space for in the regular course.

2:41Katie Malone:And at the time I was really interested in podcasts and I thought, well, I wish there was a podcast where I could learn some stuff about data science, you know, not be too high level or too in the weeds. And it didn't really exist. And we had kind of this trove of content. So a friend and I kind of pulled some microphones into a closet and that's how it got started. Did it for five or six years and got, frankly, a little bit burned out on the process. But we came back about six months ago and I've been doing it weekly since then. So it's been a really interesting exercise to contrast what it was like getting started a while back to now.

3:20Katie Malone:So what I'm doing now is still back to those original roots, you know, really aimed at talking to folks who are interested in learning more about AI these days, interested in having a little bit of technical or scientific rigor to that conversation, learning something new, but not getting so in the weeds that you feel like you're reading an academic paper every week. And it's been a really fun experience. So Linear Digressions, like I said, come check it out. Do I remember correctly, I was on your podcast? You were, yes. Yes. And I was also on John's. Thank you, John, you invited me. I think, John, you probably have at least one of the most popular data science podcasts.

4:00So what do you want the listeners to take away listening to your podcast?

4:05Jon Krohn:Yeah, I mean, exactly the same kinds of things as Katie was saying. We want people to learn something. We want them to hopefully smile a few times as they're listening to the show. We try to be a little bit lighthearted about what can be heavy technical concepts. And, you know, it should be entertaining a little bit, at least, while also educating you a lot and keeping you up to date on all the latest things in data science, machine learning and AI. So we already mentioned now every conversation seems you cannot avoid AI and for mostly the right reasons. So I want to start the part about how AI affects the podcast self from the more kind of mechanical aspect, like the use of AI in production.

4:50From research to guest preparation, to editing, transcripts, promotion, translations, whatever it is, how much you're using AI now, and how much you'd like to use AI if you haven't. And maybe this time start with John.

5:06Jon Krohn:Yeah, we use AI as much as we can without losing a feeling of authenticity. You know, I want it to be my voice in the program. Obviously, I'm conducting interviews or I'm doing deep dives into topics where it's important to me that those parts, that my voice is really me. But we have AI everywhere else. I mean, so we have an operations manager, Sonia, who is incredible at stitching together lots of different AI workflows and automating episode production as much as possible. So, for example, the show notes, the episode summary, that's something that these days LLMs are so good at. And something that I actually feel really bad about is that we for many years, we had a brilliant professional writer with a PhD who did such a good job of creating these.

6:05Jon Krohn:But, you know, she wasn't somebody from our niche. And so this year, we got to a point where Claude Fable 5 could create episode summaries as well as, you know, this professional human writer, but also had the in-depth knowledge of our niche that meant that there were, you know, fewer mistakes based on an understanding of what we're doing. So we obviously want to be augmenting as opposed to replacing people as much as possible. But every once in a while, AI comes in and is doing something actually better than we can even find a human to do. But one of the more interesting aspects of where we've included AI is in making animated shorts.

6:48Jon Krohn:We're on YouTube as well as on audio-only podcasting platforms. So we hire out to a contracting firm that uses AI and this contractor takes our hour-long interview and converts it into half a dozen one-minute long shorts, but they're animated. He uses AI for so many things, to figure out what clips to use as well as to generate the video animations. And they are stunning. They're my favorite part of the show and I have nothing to do with creating them. What you can do text-to-video these days is so compelling. Wow. Kitty.

7:24Katie Malone:My setup is kind of similar, I think. So I found that AI is a great sidecar for me in the production process. I mentioned a little bit that I got burned out at one point and came back. And a big part of the reason why I got burned out was I really enjoyed creating the episodes, recording. I could put up with editing. Producing just took something out of me. Like writing those descriptions, even though it might not take that long, I just dreaded it every week. There's a lot of little tasks like that in making a podcast. And it's just me. I do this as a hobby. And so it takes care of a lot of that stuff for me, which is one of the things that makes it sustainable.

8:03Katie Malone:One of the things that I have experimented with using AI for and I've really had to find the right niche, John, I'm curious if this is similar for you, is around the research and preparation process. I find that, you know, having a good prep process where you're either learning about the topic that you're going to discuss or you're doing some background research on someone that you're going to interview, which it makes it makes for good content. You can't skip that. But doing that on my own the first time around, there were a lot of blind alleys. I would wander down. You spend a long time reading something and it wasn't the best resource on a certain topic or trying to find the CliffsNotes, the summary, the high level stuff to drill into when you're preparing.

8:46Katie Malone:So I find that having AI as a research partner and a little bit of a scout is super helpful. When I let it go too much farther than that and into anything that feels like scripting or even sometimes suggesting questions or lines of discussion is where it starts to get real jagged real quick. And I've ended up, I think, in a similar place where for reasons of authenticity, quality, and honestly, my own personal satisfaction from the project, I tend to stay hands on. I don't have a lot of AI involved in actually creating the content or even necessarily like setting the editorial direction. Your answer just reminds me a lot more questions that I'm trying to answer as the editor for a journal, right?

9:34Because we face these issues when you have, for example, when you talk about research. And I obviously use AI as well, not only just for research for doing podcasts in the journal, but also doing my own research. There's really two big questions. And I face this question really now daily, routinely because of editing the journals. One is, how do you ensure that's trustworthy, especially in the scholarly research side? You know, trustworthy, this is obviously very important. And the second part is how much you need to disclose, right? People say, make it transparent. You can push hard and say you disclose everything.

10:11But in reality, just like these days, you don't disclose that I'm using calculator to do my arithmetic, right? You know, there got to be some line somewhere. So I want to actually hear from you two, whatever you're doing in terms of podcasting, in terms of research, like in that aspect. One is the authenticity, which you already mentioned. How do we tell? And the other, the transparency in your, like when you have these podcast materials, how much you tell? So this part is actually from AI or this part is from humans, if you do that at all.

10:42Katie Malone:Yeah, I had an easy answer for that second one around transparency, which was I just had a whole episode about how, because it's a podcast about AI, right? Sure. And we had a series that I did about 10, 11 episodes that were just unpacking agents kind of piece by piece. And as the icing on the cake at the end, such as it was, I walked through the agent that I use for the production process. I walked through some of the AI tools that I use for my research. It was a little bit of a gimmick, but I in the episode, I recorded myself, did the speech to text. I put it in the transcript with the agents themselves.

11:21Katie Malone:And then I had them like reply back. What do you do every time? You know, what's easy? What's hard? What goes well? and then used a speech generation software to have it be like an interview. But there was an AI on the other side. It was kind of fun. So I err on the side of extreme transparency on that because it's like good content. But I will say one of the other things that we do as part of the little ecosystem that I have here is a substack newsletter that's mostly summary and notes. And there's a little bit of original content that's not covered in the main episode. But that's all AI produced.

11:55Katie Malone:and is through Substack, which has recently started an integration with Pangram. So there's a little button now that you can hit and it says, whatever, 70 % AI generated or 90 % AI generated. I always edit those newsletters, but it's been something that I've been thinking about a little bit more recently when all of the creation process that leads up to that point, everything that happens before the newsletter is like me and my kind of human curation and presentation, AI takes that transcript, produces a newsletter, and then it says this was like 90 % AI generated. And I'm like, well, I don't know if that's really fair.

12:30Katie Malone:But it's an interesting question because I think there's different kinds of contributions that AI can make in some of these cases, and they might have different implications for how we think about the human authorship. And I don't know that we have all the machinery yet to tease some of that stuff apart. John, what's your take on this?

12:49Jon Krohn:As usual, I agree with everything that Katie says. And going back to the authenticity bit, I want to reiterate how important it is that things like Katie mentioned editorial direction. That's something that I think, you know, as the host of a show, as the owner of a show, people are expecting our opinions, our guidance on where things are going. And they want it to be our unique take and be based on our unique experience. I think part of why people listen to a podcast is because there's personality in it as well. It's different than consuming something as an academic paper, for example, or consuming a blog.

13:26Jon Krohn:Having that authenticity in the editorial process is important. And it's also fun. It's not something that I would want AI to replace me on. Although something that has been occurring to me recently is thinking, well, I wonder if it could make suggestions or ideas. I haven't done that yet, but it has occurred to me that I'm like, I bet I could probably have some good brainstorming sessions on, you know, given recent topics we've been covering, what are some ideas for what we could cover next? And use that in kind of a back and forth dialogue. And so this really gets into, you know, Katie's point there about this 90 % AI generated for the newsletter.

14:03Jon Krohn:It's something I worry about that now any model that Anthropic releases is going to have these invisible watermarks based on the text where people can say, okay, probabilistically this was AI generated and that's kind of Claude approved. But what happens in circumstances where you come up with the ideas behind something, you are deeply involved in the research, you draft bullets of how you want the content to turn out, and just the prose is composed by AI. Then it's like 100 % AI generated, but actually I did almost all the work. Right, right. No, it's a really interesting question. But there's also the other side, right?

14:46There's the side of accountability, right? Whether something's done by AI or something done by human, if there's legal liabilities, how do you do the attribution there? So that actually leads to my next question, which we already started talking about, is not just about the use of AI in the production space, but rather how the AI is changing the content.

15:07Jon Krohn:Yeah. I mean, I think, you know, if I want to be doing a deep dive into a particular topic, or I want to be preparing for a guest, it would be insane to not be using AI to be pulling resources and summarizing information for you. We have an interesting process at the podcast in terms of research where we have another contractor who does research, and he's an amazing data scientist in his own right. He has huge amounts of agents working for him, helping him with the research, you know, curating things, there's still a human aspect to what he provides to me in the end. It was pages of information on a guest that I'm interviewing with suggested topic areas, suggested questions.

15:43Jon Krohn:And so I think that there's this interesting dynamic evolving where we are more and more integrated with AI operations. And whether it's Sonia who's doing our operations, whether it's Serge who's doing research, whether it's Anthony who's doing our shorts, all of those people are using AI in the flow of what they're doing. It's making their output better. And they're able to then provide me as the person who's getting the guests on the show, asking them questions with much better context, much better ideas than I otherwise would have. And I personally would not want to go back to an era where I didn't have all of this magical power at my or our team's fingertips.

16:25Thank you. Katie?

16:27Katie Malone:Yeah, I think on my side, I don't think that the advent of AI has moved us that far on linear digressions in terms of the content that we cover. We've always tended to be a little bit more on the technical side. My nerd brain likes to talk about how these algorithms work a little bit more than what's happening right now in the AI hype cycle. There's plenty of other good people who are commentating on AI and different aspects of it in society. And I've been doing this long enough that I know where my sweet spot is. And it's not like algorithm of the week. It's about kind of thinking about more evergreen topics.

17:06Katie Malone:So that hasn't changed much. And I think that's also reflecting something that's true for me, John. I bet this is true for you, too. And Jowli, to a certain extent, you can only do this if you love it. And at least for me, that's the angle that I love. And it always has been. And it's continuing to be. There's just a lot of good stuff out there to add. I think what has changed with AI relative to data science and machine learning is it is a social topic of extraordinarily high salience. And I think that is a challenge, of course. Like it's a social challenge just for like all of us as humans. um it's a a bit of a challenge sometimes for the the podcast and the content to you know find the the lane through all of that that's i think true to the audience and the interest that they have and not trying to get sucked into any monster of the week type topic but at the same time i think there there are things that are technical topics that have real implications for how we live our For example, we did an explainer episode very recently.

18:16Katie Malone:We were just talking about this a second ago about how AI generated text is watermarked or detected or whatever. I think it's super interesting from a technical perspective. And my little nerd brain was just like lighting up on like on all of the synapses when I heard about that. But like there's going to be people that that, you know, has a false positive for somehow or it misrepresents their contribution to something. And, you know, somebody is probably going to have like a really bad day or a bad month or a bad year because of that. I don't know who. And I like I hate to say that, but I think the question of liability related to AI actions is similarly fraught.

18:54Katie Malone:So anyway, I think it's an area that's very technically interesting, but what's, of course, making it a little more complex and important, I think, is all of the implications that those technical pieces, when they actually meet the real world, they have an impact on all of us as members of society. One thing we find that's very useful is to formulate things in the podcast version, having two people talk to each other. And this, as you know, LM Notebook does extremely well, right? And they're so real. Do you guys see, like, really the use of AI in the podcast format, particularly in the education space, any new directions or any things you would imagine that could be very useful, effective for education, but we haven't fully taken advantage of?

19:44Katie Malone:So I learn a lot from podcasts and my feed reflects that, I think, if you were to dig in there. So I can definitely see that. And I think that there's a lot of possibility. And Notebook LM is probably the best example of this, of not just that I can learn whatever somebody had the interest and inclination and expertise to make a podcast on, but that I can say, like, here's the thing that I want to learn about. and the podcast is the medium and AI can kind of fill in the blanks. I mean, I'll also say that some kinds of technical topics do require a certain amount of hands-on and struggling with the content.

20:23Katie Malone:I'm a physics PhD, so I'm like, ah, no pain, no gain. So to some extent, I don't think it's a substitute for, you know, the kinds of learning that you do through interaction where you're, you know, you're in the conversation and you're talking back and forth, of course. And I I don't think you were suggesting otherwise. But yeah, the idea that, you know, that kind of auditory, explanatory, you know, dialing the level up and down kind of format is now something that can also be customized and fit to an individual and their interest in what they're trying to learn. Like, I think that's an incredible opportunity.

20:59Katie Malone:I think that's super cool. John, you have any thoughts on?

21:01Jon Krohn:So the first episode of every January, a brilliant data science and AI entrepreneur named Sadie St. Lawrence comes on my show, and we talk about what happened in the past year and what we think is going to happen in the coming year. And whatever year Notebook LM came out, both of us had as our pick for the most jaw-dropping AI moment of the past year was Notebook LM. It is astounding how well it can create a real sounding conversation. So I have a fellowship at a company called Lightning AI. And it's this sprawling business. And so recently, the CEO of the business, Will Falcone, he started using Notebook LM to create podcasts about what we do at Lightning and his vision for the business.

21:47Jon Krohn:And then he has the whole company listen to these Notebook LM generated podcasts. One thing we have thought about, we haven't really done it, but it's not hard to do. For journals like HDSR, we can be creating a podcast for each article. We did some testing, which is great. It's just a few minutes of a great summary of that. So I have one more question before we do the magical one question. And so far, we've been talking about the use of AI for podcasts as a platform. I want to just have one question on how the AI is affecting organization and the individuals. And I want to start with John, because you started from saying like how the AI has changing.

22:28So to work for your podcast and how does that affect you as an organization? Like, do you have to hire fewer people? I assume the revenue is better. You know, what is the effect on organization and what would you like to see there?