Gary Hustwit: Revolutionizing Filmmaking with Generative AI

0:05Hey, everyone. Welcome to season two of Generative now. This is the podcast where we talk to the builders who are creating the world's most exciting AI products and companies. I am Michael McNano. I'm a partner at Lightspeed. And today, we've got Gary Hustwood on the podcast. Gary's a filmmaker known for documentaries like Helvetica and Rams. But his most recent film, Eno, does something pretty different. Eno utilizes a first-of-its-kind generative software system to make sure that no two viewings of the film are the same. Gary and I talked about the future of AI and film, innovating outside of linear narratives, and where the film format goes from here.

0:42So check out my conversation with Gary Hustwood. Gary, good to see you. Good to see you too. Thanks for having me. Yeah, thanks for doing this. Very, very excited to talk to you because I think you are a person that is doing probably one of the most interesting things that I've heard of in terms of applying AI to a mainstream form of creativity, in this case film. And I'm very excited to get into that with you. But it'd be great maybe before that to dig a little bit into your background. I was very fortunate enough to see one of your films a while ago, Helvetica, which I thought was incredible and super inspiring.

1:20I'm a designer, so definitely meant a lot to me. But why don't you tell us a little bit about sort of how you got to where you are right now, because it's a pretty incredible journey. It's a long story, Michael. The short version and the long version, I think both start with when I was a kid and one of my friends got the first Macintosh, like the week it came out. But it was when I first kind of discovered graphic design and fonts. I didn't really know what fonts were. And I'd never seen the word Helvetica before until I pulled it down on the first Mac dropdown. And that, I think, really kind of changed the way I thought.

2:04I just really wanted to watch a movie about graphic design and typography. And in 2005, when I first had the idea for Helvetica, the movie, there was nothing. And so at that point, I think a lot of my process is just kind of imagining things and obsessing over them so much that I have to just like, okay, I'll figure out how to do it and just do it myself. And I think it was just at a time when fonts and just graphic design became more of a thing that we all did. You know, like my mom, you know, like had a favorite font and, you know, it was about a personal kind of expression. It was like, oh, I'm a Times New Roman person or I'm, you know, I like this or I like that.

2:51You kind of ended up expressing yourself through the font that you chose. And it was really kind of the first time that that happened on a, I think on a mass level. So yeah, those were probably all factors. There's another reason I think Helvetica was successful. And I think it also relates to what I did with Eno, the Brian Eno documentary we just premiered. There's a almost kind of a where's Waldo kind of thing in Helvetica, where we'd go to different cities and we'd talk to a designer, but then we'd kind of go out in the streets and kind of look for Helvetica in the, in the wild. Every shot, the audience was kind of like, oh, where is it?

3:29Oh, there it is. And where's it? Oh, here it is. Like our brains are very much, um, uh, attuned to, um, to puzzles and defining patterns. And, and I think that that movie sort of engaged that, that part of our brain. And I think that, you know, what we did with the, with the Eno film is the, is the same way. The film is different every time you watch it. It's a different mix of scenes, different music. And in a way, the audience makes the connections between the different scenes and makes their own kind of version of the story versus having the story sort of like imposed on them. And maybe that's a good segue to Eno, which obviously takes a very unique approach to format.

4:12that. Tell us about this film. Obviously, like we said at the top, something very, very innovative that I haven't really heard of anyone else doing. So would love to hear about the film and what it was like to work with Brian Eno. Yeah. Well, I mean, Brian has been a kind of, you know, creative hero of mine for a long time. I did a film in 2018 about designer Dieter Rons, the German designer who was the design director of Braun and huge impact on Apple and the way Apple's devices look and work. And I was just kind of like out of the blue. It was like, wow, Brian Eno would be amazing to do the soundtrack in this movie.

4:53Just kind of cold emailed his manager. And Brian was also a fan of Dieter Rahn's. He agreed to do the score. It was amazing. We had a great, great time working together. But around that time in 2018, when that film came out, you know, I just was kind of thinking, like, why do films have to be the same every time we watch them? Because, like, I would go out and show my films. I've been making films now for 20 years. I've gone to hundreds and hundreds of screenings. Like, I can't watch my films because I've seen them too much. So like I start them and I go to a bar and I come back in 80 minutes and I do the Q &A or something.

5:36It's literally impossible to watch the same thing hundreds of times. It's like torture. So my background, again, is kind of in music. And even if you're a band with a hit song and you're playing it every night, it's different every night. You can change it. You can extend the section. You can have a guest come up. You can change the lyrics if you want to. and film doesn't have that. Like I wanted film to be more performative basically. So I was kind of dreaming of ways to do that and reached out in 2019 to my friend Brendan Dawes who's incredible British digital artist and is somebody I had known to the design world and said, hey, could we do this?

6:22Could we make a film that was assembled in software that was different every time it screened. And that's when we started kind of working on this idea. So right around that time, I thought, well, I just worked with Brian Eno. Brian's kind of famous for using generative technology in his music all the way back to the 90s, the first like very primitive generative music-making programs that came out in 96 and 97. He's always been a pioneer of using technology to kind of enable creativity and think about new ways. So I thought, who better to kind of, you know, use a subject for, you know, like a generative film than Brian.

7:06So I had asked him before when we were working on Rams, I was like, you know, about like, what about making a documentary about your whole career? And he's like, I hate bio documentaries. I don't want to think about my past. I'm only thinking about the future. you know I don't want to do it and and then when Brendan and I kind of didn't really demo of the generative film idea and showed it to Brian you know he was just blown away and he was like I'm in and you know let's do this and in the past few years we've just done very intense filming things with Brian he had an incredible archive of 500 hours of visual material everything from like videos that he almost like home videos you know video art things he made when he was living in new york in the 80s or studio sessions or lectures in tokyo in the 90s or whatever and we you know digitized and restored all that stuff which took a couple years as well and then we're also working on the software platform that that we used to to make it so So, yeah, this idea of like linear static media is just a really it's a constraint that goes back to the beginning of film, like 130 years ago when it was a physical medium.

8:31And that film had to run through cameras and projectors at 24 frames a second and you had to be able to duplicate them. So, of course, you had a master and you would make copies of it. Of course, it had to be all the same. But generations of audiences have just kind of accepted that constraint. Like, oh, yes, film's going to start here. It's the way it works. It's always going to be. Yeah, it's always going to be. It's like we were so dazzled by the moving image that we didn't really question, like, oh, well, does it have to be the same every time? Right. And obviously, when when film went kind of digital in the 90s and in 2000s, that constraint is gone.

9:12But all we've done is just do the same thing. And that's it. There has been no innovation on the formal side. And I think there's just so many possibilities that, you know, film was just kind of, you know, an opening to this conversation. Like, OK, here's an idea. We premiered the film at Sundance a few weeks ago and people were blown away. It's pretty it's pretty simple idea, I think. You know, just make a movie that's different every time. And but that took a lot of both programming work, but it also took a lot of kind of storytelling and filmmaking work. But there's so many kind of ideas, I think, in that movie, both Brian's ideas, which are amazing on one level, but just the idea of like, can we think of film as this changing, evolving storytelling structure and not just as this static linear piece of media?

10:13Yeah. Now, is it is it deterministic and that like it'll always tell the same basic story? You just might see slightly different footage to get there or is the story actually impacted by the generative feature? Yes and yes. Why the film works is that's one of the reasons it's all it's all it's all Brian. We have kind of the same start of the movie and the same kind of ending scene in the movie. And then kind of what happens in between, you know, I think we established very early on that you could be anywhere the next scene. You could be in 1973 with Roxy Music. You could be in 2023. You could be anywhere in between scene to scene.

10:55and again the audience sort of makes the connections. What the system does is sort of intelligently like arrange that each time, creates transitions between these different scenes that have been selected for that version of the film and then also kind of creates pure generative scenes using our raw material and those are created dynamically that are unique to each cut. It tells its own story, basically. So these scenes, I mean, it basically writes them. Yeah, but we've also edited them in a way that they can appear or not appear or when they appear, depending on where they appear, they still kind of do a job.

11:40They're giving you some sort of information or sort of story beats that, again, kind of fulfill what we as film watchers think is like a you know an engaging narrative arc even though the arc changes every time it's interesting like just seeing the you know we showed there were six screenings at sundance so all six unique versions like every time we show the film it's a completely different film you'll never see the same version twice um but uh some of them are are they have a different vibe some of them are like really energetic and lots of music and some of them are a little bit more introspective and a little bit about brian's interior life or more a deep dive into his ideas and and then you know some of them are a mix of those things so i think it's it's fun and again that that that idea of um we're always looking for patterns and trying to kind of connect the dots to a story like you get to do that in so many different ways with with this film and that idea that it's it's unique it's a unique experience like every time we show the film it is only for the people in this room you're the only ones in the world who are going to experience this with us so it turns it into more of an event than just like i'm going to watch the same thing everyone else has seen right so in in addition to the footage that you're feeding the system are there other parameters that you feed the engine to kind of maintain the integrity of the narrative arc or is it really just, hey, we're going to have the same beginning, the same ending, and the AI is going to figure out everything else?

13:21No, there's definitely like, you can look at each kind of scene as a type, whether it's about the creative process, it's just a music, pure music thing. It's a big idea about, you know, Brian talks about AI and generative tech in the movie, along with a lot of other things? Or is it just a kind of a personal history thing? You can kind of establish a rhythm of those scenes, of those types of scenes and how those come out. You aren't going to have four big ideas in a row. You want to kind of have a, again, establish a rhythm. So that's part of it. Obviously, there's a ton of metadata on, you know, every piece of media, whether it's an edited scene or it's just raw material in the system.

14:17And, you know, the kind of rules and ways we programmed it are, again, with our knowledge as filmmakers about, you know, what kind of what works. And again, if for whatever reason, it works. I mean, there were early outputs that did not work, but through, you know, trial and error and training the system and also adjusting the way that we edit the scenes and even the type of material that we put in. So you put all the stuff together and really doesn't matter what order or what things you get or don't get. I think it's all interesting. And then the software also has some presence. I mean, we thought that, I don't know, it's just like a magic trick or something.

15:08Like if you don't know that something is happening that's different, you know, you just wouldn't even notice it. And I think that we definitely kind of let the code kind of come through in the film when there are decisions being made. You know, you'll kind of see the system scrubbing through, yeah, scrubbing through file names, looking at deciding, making that decision. I mean, really, it's made that decision in a thousandth of a second, you know, at the beginning of the process. But we still let it kind of visualize that or scrubbing through frames of video and like these bursts of footage that kind of come through in different areas.

15:46So there is a visual. It's part of the visual language of the movie a little bit is this is the code. Tell us about the software. Like how was that made? What went into it? Were you a part of that process or was that your partner that you mentioned? Yeah, Brendan is really the mind behind a lot of that. I'm the guy who's like, yeah, go, that's cool. Could we make it do this maybe? And then Brendan's like, yeah, he's from Liverpool, so he talks like this. He's like, we could do that. And he goes off and then we kind of like work, you know, keep playing around with the thing. It's like Brendan is just a genius at code, but he's also just like a, I mean, he's an artist.

16:30He's not just a core coder. He's like, his whole thing is like data, you know, using data to make visualizations or physical or digital, you know, art. And also that it's just, you need to have a kind of poetry and code. um so that's how we approached this i mean he's just a film lover as i am um and and also just a fan of of of eno so you know we approached it with um you know trying to kind of make something that was true to what brian was about and like a structure that was like organic to the way brian worked. That was really what drove us on the coding. Also, it was like, you know, Brian has these oblique strategies cards, which are these like creative prompts.

17:27Like if you're stuck and I don't know, like you could pull a card and it's like, you know, try it in reverse or whatever. But we did use some of those ideas both in the programming because, well, you'll see when you You see the film at certain points like like Laurie Anderson or David Byrne might just kind of appear on on screen and pull one of these cards. And then what depending on what card comes up that then skews the film and the rest of the film in a different direction. Oh, wow. So there were a lot of kind of things like, again, trying to build Brian's ideas into into it, too. But yeah, a lot of it was just kind of Brendan and I bouncing ideas back and forth and working with the material and then working with our editors, Marlee McDonald and Maya Tippett, on kind of editing the work and formatting the work so that kind of the generative engine could kind of work with it in the best way.

18:29so i i think i read that since the premiere you've you've added even more footage yeah how does that work does that expand the length or is does it just change it expands the it changes the the um the the possible variations basically it just adds more um we're just continue adding more to the to the pool right so yeah we've added dozens of of scenes and and more and more raw footage to it since Sundance. So every generation of the film will be different as we keep modifying the software, but also since we keep adding more material to it. And you mentioned you have editors, and obviously you were guiding your partner throughout the software development, but does it work such that once the software is written, it's kind of baked, and at that point, the role of the human editor changes, or does there have to be this sort of this tight partnership between the software and the human editor?

19:29Yeah, I think for this at this stage and in kind of, you know, experimenting with these kind of structures, there has to be like a back and forth because we'll we'll we'll say, oh, kind of maybe each scene does need a little bit of a bumper on on the end, a little bit of like a landing pad. You don't want to start something with someone talking right away because, you know, you're thinking about how the puzzle pieces are going to fit together and then kind of watching outputs, watching outputs, watching outputs. This isn't working. What can we do? How can we change that? Is it a programming thing or is it an editing thing?

20:08I think that But, you know, as we've seen, there are some incredible, you know, tools coming out with AI generated video and all these other things. But you still need to be able to tell a story. Like I can make a technical tool, you know. I mean, the team that made Sora, it's incredible. It's like amazing, like amazing step. But they're kind of really even kind of not sure how the artists are going to use it. They're going to kind of put it out there and then people will figure out what the tool is for. So you still need that. Whatever the tool is, you still need that, you know, the ideas and the storytelling and to be able to kind of make something that really connects and has a soul.

20:56So as cool as the I think that the tech is, it is also really about the human part of it, too. There's got to be that balance. Yeah, for sure. Definitely want to get into Sora, so I'm glad you brought it up. But before we do that, we talked a little bit about streaming earlier and sort of streaming as being the kind of delivery mechanism for films like Helvetica and basically every film that we watch today. It strikes me that this film can't be delivered that way. And so maybe two-part question. And number one, how do you watch this film technically? Like, how is it actually being projected or displayed?

21:40And then what will need to change for a film like this to be consumed by the masses at home, you know, on my smart TV or on my, you know, my Apple TV or something? Yeah. Well, for the live events, which I'm super excited about, for those, we're creating the film live on stage in real time. Like we're making it for that audience, which is going to be really fun, which is actually a lot easier to do than have the system render out a perfect file with 5.1 audio and no glitches. That's actually harder. We can do that. We can just render out a video and we make a DCP. It's a digital synonym package.

22:26It's how theaters show films. We can do that or we can do it live, which is actually much more fun too, because it makes it a little bit more of an event or a performance than just like another film screening. So the live screenings, that's one thing. Streaming it, it is like, I think the highest use is everybody who streams the film gets a unique version that only they get. Because we can, there's like billions of possible versions of the film that come out of our platform. That would be, I think, the dream. The current streaming, the big streamers are nowhere near that capability right now. You know, something I was talking to, I'm not going to name the streamers, but I was like, well, we can just give you 365 versions of the film, And every night at midnight, you just push a new file and film's different every day.

23:30No, we can't do that. That's impossible. Even that? Interesting. Even that. I think that the best they even were thinking was like they could maybe do a different version every month. But even that with hundreds of different territories, that was going to be a huge lift. So that's the level of, I think, technical, you know, innovation we're at, which I think is going to be, you know, they're going to be completely disrupted by what's coming. And I think they don't even maybe realize it or are just starting to realize it with things like Sora. But like a native AI streaming platform is something that will happen where all the content is dynamically created, where viewer preferences actually get baked into the creation of the films or the series.

24:24I think there's just so many huge, you know, possibilities that you could do with with the technology. And really, they're they're they're using it for the for the other side of it just to kind of like recommend you more of their sort of linear, you know, true crime series or whatever. Yeah, I definitely agree with that. I mean, if you think of what we were talking about earlier, when we talked about the business model of Netflix early on, and then compared to now, there is a cost to the content, right? There's a cost that they incur when they, I guess it used to be when they would stream a piece of content, but now it's when they acquire or produce a piece of content.

25:03And I think per your point, there's probably a world in which that cost goes away and the creation of each additional piece of content is completely marginal when it's generated on the fly. But the flip side of that, you know, almost sounds a little a little scary. Right. And it sort of contrasts with what we talked about earlier, how the human is the person that sort of or, you know, is the thing that's driving the story and the creativity in this world. we talk about, it almost sounds like the human would be removed from that and the platform would just be leveraging AI as much as possible to optimize the model and the cost.

25:39I mean, is that a potential sort of bad outcome here? Or, you know, I'm thinking about the platforms and their incentives, you know? Oh, yeah. And their incentives are definitely about, you know, know, cost reduction. I think that there's a world where, you know, I think one of the possibilities with the Edo platform and, you know, which is using real video, but it's the way that it's presented, the formal experimentation and that the kind of that can be, I think, a way that, even with Sora, even with AI-generated video, traditional filmmaking is not going to go away. There are still going to be filmmakers making things.

26:27You can't make 20 Days in Mariupol or a documentary like that in AI. There's still going to be lots of film being made. But the idea of this sort of hybrid using kind of traditionally shot film, but using it in ways that are, you know, informed by AI and other tech, I think is sort of a middle ground to pure AI generated things. Even if you have pure AI generated content, I think it's still going to have to have some sort of human story interaction to make it compelling. And also, I think it depends on what kind of content we're talking about. We're talking about feature films, we're talking about Marvel movies, Are we talking about kids' animation?

27:18You know, are we talking about kind of more like industrial kind of films like, you know, Synthesia or, you know, that sort of stuff? I think those things are obviously completely changing and will be a huge shift in kind of the people and equipment you need to make that stuff. um but um yeah i think it's it's it's all fascinating and and and like i'm i'm excited about the possibilities i'm i'm always thinking about like well what do the new tools what are the capabilities that that they um that they make possible it's not for me it's not about oh i can do this cheaper and quicker now i i'm thinking about okay well what can i dream of now that that with this new tool that i couldn't do before like what what does it unlock creatively for me as a filmmaker.

28:11So I think that you either kind of embrace the tools and look at them in that way or you don't. You mentioned Sora earlier. Just to recap, OpenAI last week announced a new video model called Sora where they enable anyone with access to this tool, which is not available yet publicly, I should mention, generate any video via text-to-video prompt. I think the limit they've talked about is up to 60 seconds, but have to imagine that's going to go up and have to imagine it's also going to be possible for people to string these things together. What do you think the nearest term impact of that on filmmaking?

28:48Do you think we will see more filmmakers as a result or do you expect traditional filmmakers to start leveraging this to augment their creative workflow? You just mentioned when something new comes out, your question is, hey, how do I use this to do something I haven't done before? Is that what we're going to see for more traditional filmmakers? I think so. I think for some filmmakers that are open to that experimentation, definitely. I think there are going to be some that are purists. Are we going to see a ton of AI in the final Quentin Tarantino film? Probably not. But yeah, any technology that makes the medium more accessible and lowers the bar to entry we're going to see a ton of new voices are we gonna see just a flood of of this stuff to the point that any online video that we watched we just have to assume is synthetic um maybe because i think that like nobody expected you know sora level quality this quickly and it's really their first pass i mean in in a year or two years this is just going to be just indistinguishable, I feel.

30:05So when that happens, you know, how does that impact just anything that you watch online? People are, you know, it's beyond like deep fakes and, you know, people are all sort of anxious about, oh, it's an election year. We're going to see all kinds of like, you know, fake video content with tools like this. I think there's a bigger question about authenticity here and a bigger question about

30:38trust, basically. So it's going to be interesting. I mean, I'm excited to try the tools. I'm excited to, again, innovate with kind of some of the formal structural possibilities using traditional video, too. I still think enabling creativity, even though a lot of people would say, well, it's really taking away from creativity and you know it's it's this shortcut that now you're not kind of learning all these other like parts of the the craft and and all this stuff but um i think that's ends up being true with with any technological advancement that we've seen over the past 100 years yeah it feels like you know the conversation uh among sort of the entertainment industry film hollywood you know thinking a lot about the writer's strike from a few months ago It has been a tone, I think, of fearfulness about the future.

31:28You sound a bit more, I would say, optimistic or pragmatic, I would say. Is that a fair characterization? You're a little more optimistic about how people will balance this? I hope so. So I'm optimistic just because I think that the possibilities to in ways to tell stories is going to exponentially increase from not just things like Sora, but just like, again, more kind of thinking around how this can change cinematic storytelling. So let's talk about Animorph. That's probably a great segue. So basically the techniques that you leveraged for Eno, it sounds like you and your partner are now turning this into a service or a product.

32:19Tell us a little bit about that. Well, I mean, it partly, again, because the experience at Sundance with, I think, the rest of the industry being kind of unprepared for what we put out. I think that we want to innovate in ways to stream films like this and ways to kind of integrate kind of AI into streaming, into story structure, into live events. I mean, there are so many different things that we could do right now, even with this kind of very kind of basic platform that we created for Eno. I mean, give me like 2001 and let me dig around in Kubrick's boxes and get all the alternate takes and all the music and all the dialogue and everything.