Generative AI in Video and the Future of Storytelling (with Runway CEO Cristobal Valenzuela)

25:58But now you have. We want to talk all about beyond just, you know, movie making and all the other, you know, use cases and applications here for runway But even just staying in that like the past decade we've talked about this so much unacquired has converged all of commercial movie making into like the most monolithic non -iterative, expensive, barrier, and keeping, you know, unoriginal 25 and a 25 top -rosing films are reused IP every year. Right. Marvel was one of the best acquisitions of all time and one of the most anti -democratizing forces in Hollywood over the last 20 years. And so you're like coming up right the right time to liberate this.

26:40I think the best movies are yet to be made and the best stories are yet to be told. Like we could consider like the golden era of cinema happening in one particular decade, many years ago. I actually think that we're yet to see like the true like golden era of cinema. The moment more people are able to create what used to be only the realm of small studios or small agencies or small teams are now, it's going to be now festival to anyone. We just released it because this happened like four days ago. A new version of our video generation model that has created some sort of insane wave of creativity and filmmaking.

27:18People are using it to create all sorts of interesting fascinating short films. and I was just chatting with a producer of a major production house and I was showing him a clip. Someone made you can search for it, it's called Commuters. I think I tweeted about it. Just robots in the subway car in New York. Incredibly well crafted, very great like cinematography, well shot, 30 seconds, it's short. If you look at that and you think about how long it took and how expensive you think it was, you might guess around like a couple of weeks and a couple thousand dollars. like all of it was just made with one person really on a couple of hours just using runway and a few other tools and The results are just like astonishing.

27:58I mean, it's just so good. And so that's for me where like We're if you consider that we're stealing the Early early stages of this this technology like again think about the camera. We're in the 1910s of the camera It's a black and white camera. I can't like works, but like it's a lot of work to be done We're gonna get to like a labor resolution and quality that will enable people to this real real real wild creative stuff your six or so years in at this point has the thesis changed at all and where you'll need to play in the value chain Like I could imagine thinking at first. Oh, we want to create Models and we want researchers to use it and we want application developers to build on top of it But now you have a full -blown application and you've had to build a lot of like real user experience for novices or You know people that aren't that well versed in filmmaking.

28:45How is that evolved? There's a lot of, I would say, fundamental pieces of the technology had to be built. Again, we were discussing the origins of PyTorch and TensorFlow, which are the frameworks that nowadays every model is using. And those are just like a couple of years old. And so if you want to deploy a video generation model to the wild and two millions of users, you need to have some proper infrastructure in place. And so from the very beginning, at Run, we actually, we started building those underlying systems. And eventually I would say that you get to a point where for someone who's shooting a film or who's telling a story Models don't really matter like no one cares really No one cares beyond like their researchers themselves or the engineers themselves Or at some point like you hear our technologies and like you care about it You will like go deeper into it But if you're a storyteller like you care about tools that are expressive and controllable And that's really thing you care because that's the thing you want to use It's like Shopify, like if you don't really care how Shopify works, you just like, let me take credit cards, you know.

29:47And every piece of like major destructive technology has gone through similar stages where like the internet at the beginning everyone wanted to chat about rudders and the internet highway speed or whatever you want to call it. And you have all these terms to refer to understanding this technology. Really now, there is like no one curse, you just open the website and like it works. And if it doesn't work, you complain about it. It's somewhat similar, but like our goal is really not to upset around the technology, because when you obsess around the technology, you don't find real problems to solve.

30:18Our problems, and we state this as the company's vision, we're a research driven company, we've built some of the perhaps most important models, or we build really important models in the space. At the same time, our goal really is to move storytelling forward. We're a storytelling company. We're a company devoted to creativity. And the way to deliver that is like, well, we have to build it, we have to build everything from scratch. So we'll go back to the baseline lowest level possible to make it happen. But it's always good to obsess around people and not acknowledging. It feels like a very sort of Nvidia approach, right?

30:52Like Nvidia is, you know, my sense from the long series we did on the last year is they don't do what they do just to have the coolest technology. They do what they do. So first so that people can make the coolest video games possible. And now so that this can happen, right? Exactly. I think the best companies or companies that are obsessed around customers and users and use cases rather than technology. I think a common misconception is since we're also excited about the technology is that you have you started obsessing around that in itself and people models and data sets, all these things dominate conversations nowadays.

31:29But few people are asking themselves like for whom and what, right? And I think we've always started the conversation from the other side, which is like, yeah, filmmakers. How do you make their process and their lives easier? So let's work backwards towards that. Switching over to the business side of things, there's clearly lots of different use cases. And so how do you do sort of like pricing and packaging and go to market and customer segmentation when the tool is so versatile? It's hard. It's hard because this is a new field. And it's an evergreen field that's changing radically. And so the one thing I would say we've learned over time by building runway and building some sort of like learning and heuristics around that is that Over -optimizing for the wrong thing at the wrong time can be very costly And so making sure innovation is not the core.

32:15How you thinking about the company and the research on the product needs to be like front and center And so monetization and value capture can really depend on the type of model and the type of output that you can make And I think we're really early on that journey video and capacity and resolution will continue to improve. Efficiency is running these models will also like go down. Right now we have anywhere. Enter the efficiency stages of the technology where things are going to get cheaper, faster, leaner. I think eventually you will get to a very similar like traditional SaaS model which we already kind of like have.

32:50But all more optimized. I think there's a lot of optimization that has to be done to make these models really really effective to be used. Hopefully in a real -time basis very soon. And do you find that the way that you build user experience and controls and the application workspace and support and all these things for enterprise level customers? I don't know who your enterprise customers are. Are they like Hollywood filmmakers? Like are there different versions of the product and the experience that you need to craft for different audiences? Torey, again, I'll go back to the camera. The camera can be flexible enough to be used by a consumer, but you also have red cameras, which are like professional filmmaking cameras.

33:24as I have all these controls and systems and settings. It's interesting because those controls and ways of manipulating and having the flexibility that you want in a creative tool, specifically for genocidal models, hasn't been invented. This technology and these things that we can do were in there a couple of years ago and so you really have to think about what we call primitives or metaphors to interact with the technology. If you think about panning and zooming, those are all concepts that we've come to understand so we can make sense of how to control a camera. So you need to make the same metaphors, or similar metaphors, or distinct metaphors to try to control these models that don't work in the similar fashion, the similar way.

34:04And so a lot of the times, a lot of the research that we do is on that, that kind of like aspects of understanding, I actually agree that one of the hardest things to do these days on, I would say, the field of AI is actually product building. It's not building the models, it's not finding the models. of course that's challenging and it requires a lot of things but ultimately I'm going to go back to the people problem on the goal of what you're trying to do is trying to find the best interface and the best product to solve a need using a model. That's where the real tonages start to appear. We spoke about this a few episodes ago in our original exploration of Generative AI here on ACQ2 with Jake Saper and Emergence of the models and the technical abilities is critical.

34:51but you need the UIs and the workflows. In many ways, that's the much more scarce and harder thing to develop. It is. It's unknown territory. We haven't entered the full spectrum of what's possible with these models, and so there's so much to be explored around how to use these models. And how to, again, I'll go back to control. The control is just key. You need to have control in a creative tool. and so coming up with those metaphors is totally new. It's a whole new field of research and exploration that we haven't delve into before. If people are noodling on this and they're like, I want to get into this field, what do you think it takes on a founding team for a startup to build a successful paradigm -shifting AI company?

35:36Well, that's a big question. I can give you, guess a sense of what we think is required to work at runway and how we like decide on who we hire. And I think a few things that we tend to look a lot for and I think our key for working in the space is one humbleness, really not getting attached to your ideas and really willing being able to like learn and question everything. It's a feel that's moving fast and you have a lot of preconceptions around how things should work because they've always worked like that. You're gonna get DC very soon. Again, think about painters understanding the camera as a new paintbrush.

36:18And the thing is that it's not a new paintbrush. It's a new thing. And it requires you to think about it very differently. There's new challenges with a new ways of using it, new art forms that are emerge with a new artist, the idea for photographer, a filmmaker, might not have even like, conceivable for someone who's painting in the 1800s. It's similarly to now, if you come with a lot of preconceptions around how creative tools should work and how creatives are actually working and not really questioning us to why, a good mindset there is to have like a first principles like view of the world, just go and ask yourself why a lot.

36:52Like are people really so deep into analysts or video editing systems because they like them a lot or is it because that's the only thing they know how to use, right? I don't know if they're specific, but traits that I think we tend to look a lot for when hiring people that could also, I guess, be extrapolated to if someone wants to build in the space and again take this with a grain of salt is I would say having a first principle is kind of mindset. I would say the first thing the second is just humbleness and being able to learn a lot and Not getting attached to ideas because the space is moving really fast and I guess the last thing is just focus on people focus on customers and focus on the goals of things they want to achieve.

37:31When you and your co -founders were first starting, did you have like a perfect skill set where, hey, one of us understands the plight of the video creator, and another one of us is a pioneering researcher in foundational models or were there things that were like totally missing that you had to figure out along the way? No, totally missing. I mean, the thing that's interesting is that now you can always tell the story backwards and find all the ways of connecting the dots. There's this very inferential book that we used to give everyone at Runway, which I think has really helped us shape our understanding of how to build teams and products and research.

38:07It's a book called Why Greatness Cannot Be Plan by Canon Stanley. And it basically outlines that the best way to build great things is to start laying stepping stones. And every time you lay a stepping stone towards something, you can look around and new doors will open, new doors will be closed. take one of those, take as many as you can, clip experimenting, move to the next one. And so really, I guess to your point, like the AVL, even like foundational models, 10 years ago, wasn't even in the realm of what people thought. I mean, it wasn't a term. And so a lot of the things that we thought, you might need today, I see this idea of like, prime engineers, and prime engineers being like a job people want to hire for, those things didn't exist.

38:47Just like four months ago. And so it's not that you need those skills to assemble someone or to assemble a team or a company is more of like just have a mindset of understanding that those things are going to be possible someday. It's funny, you're saying that I was like that book and that name is ringing so many bells and I just looked up. I think this is one of Patrick's biggest episodes last year. The author Kenneth was on and best like the best last year and I remember listening to it and just being like, wow, if that is a completely orthogonal I'm not going to all the way to think about building companies, growth mindset, a new way to think about it.

39:22I just love it. With that in mind, of like, you've got to be humble, you've got to learn a lot because everything is changing daily here in this space. How are you thinking about business model right now? What is it right now? You mentioned you essentially SaaS business model, SaaS pricing. As I think back about this, every time there's been a major revolution in technology in the video filmmaking space. There's been a business model revolution, right? Like, if I think back to you know, Kodak, like the business model was, yeah, sell cameras, but you make a lot of money selling and developing film.

39:58Like that was the primary business model. And then you think about digital photography and you're like, oh, well, you know, Apple, I made all the money there. He has sell a device that includes this technology. What do you think it looks like here? I think it's through a little towel. it's final shape or form or be able to categorize it so definitely into something. I think interesting insights are customization matters a lot because again control matters a lot and so fine tuning is going to be really relevant for large customers and enterprises. At the same time I think distribution opens new possibilities for consumption and so business models based not on the creation but on the consumption side of things, I'm going to go back to thinking about film and video as a game or closely more related to the space of a video game, then you have a much more options and business models that can be built around that as well.

40:55That makes sense because in a scenario like that, the cost actually accrues at time of consumption, not at time of creation. Exactly. The creation components might be different because you might charge people differently or a value might occur different, but also the consumption might be different as well. The compute is happening at the creation time. And compute the musicians will go better over time. So right now it's going to be constrained, but over time it won't, I don't think it won't be that much. Which actually if you step back and think about it, if that is the way this space evolves, I think that's going to be a successively much better business model than apples.

41:33Because if you think about like apple monetize it, I'm using apple writ large. you put red camera in there too. Anybody selling a device makers, they monetize a fairly large amount up front. But then all of the consumption of photos and videos taken on Apple devices, that's a trillion plus dollar economy across social media, everything. Apple doesn't monetize any variable rate with that. Yeah, I mean, yeah, okay, sure. But like the monetization happens on InsticRHM, on Snapchat, on TikTok, etc. And software makers have figured it out. Adobe makes a bunch of money every month by people consuming their software.

42:17Right, but if you could monetize on a variable rate basis with consumption, that's just a way, way, way bigger opportunity. Yeah, and I think that again, we're not there yet technologically, but I think we will. And I think it's interesting to explore those. I think the most of the other thing that always has changed create their own markets and create their own like business models I think this is the case like we're new business opportunities and new business models will be born out of it I think we're already seeing initial like behaviors that will like make that the case Yeah, and obviously it's too early to tell but like I'm sure Tim Cook would in a heartbeat trade apples current business model for half a cent on every Instagram view, you know, or every TikTok view out there.

43:00Maybe. I'm not sure Tim Cook would trade Apple's business model for mod. Fair enough, but the world may go in that direction. Yeah. Chris, I'm curious. This is going to be a little bit of a finance -y question. And let's take it away from runway and talk about companies in general that make foundational models and productize them to sell them to customers. So for a company like that, comparing it against like a SaaS company, Do you think 10 years from now the income statement between 2018 SaaS company versus a 10 years from now AI company? Do they look the same or are the margins actually different because even if inference costs go to zero They're still large trading costs required on an ongoing basis Yeah, anything comparing research companies and research labs with traditional SaaS It's perhaps the most for comparison and again, first of all, I think every company is different So every company can operate differently can offer different like Strategies to try to capture or create new markets or compete and so not all companies and not all research labs might have Similar or even the same strategy I would say so I think first of all like everything in life it really depends But overall I would say that Perhaps a better comparison to think about like research labs and companies building foundational models more like a bio company where there's an intense capital that needs to be like put up front to do their research to get to where you need to go.

44:27And then there's a lot of like commercialization on top of it and product and can be built on top of it. And more importantly, I know how of how to do it the next time and the next time and the next time and also build in the infrastructure to do it multiple times and scale the margins and the ways of I would say thinking about the investment here and the long -term value kind of like captures comes more from like upfront investment that you're able to have to do to train a model like Gen 2 and Gen 1 and then starting to like commercialize that after as well. Certainly right now the amount of money being raised by AI companies is very large and my assumption has been that's largely because of training costs that it's just the compute to build these types of companies is just much more expensive at least right now in history.

45:14There really shouldn't be anything else about the company that's much more expensive, right? To go to markets the same, the talent, maybe a little bit more expensive, but not... No, the talent is definitely more expensive. I mean, there's... There are only a few people in the world that are able to do the baseline. I mean, a few. There's definitely a lot, but it's not a crowded market. And so... Right, you're not hiring iOS engineers here. Exactly. And so research really matters, and that talent is expensive. That's one thing. I wouldn't say that's the main thing, but you definitely consider that.

45:45You won't see that in other SaaS businesses where you can assemble an amazing business with great engineering full -stackered folks. Research is different and it takes a type of talent that's coming more and more, but it's still rare. So that's an expensive part of that. But for sure, Compute models a lot. And I would say long -term commitments on Compute also matter a lot. So it's a scarce resource. And so you need to make sure that you get your hands into those resources. If you want to do the kind of work that you want to do, there's an element like on our Nike episode we talked about, you know, how really Nike and, you know, Adidas and the other scale players.

46:22They lock up the world's footwear manufacturing capacity for years at a time and nobody else can produce at that scale. And Vidya and Apple with TSMC same thing. Yeah, and there's the same element happening here with Compute. There is. And so being able to just compete there, it's a requirement. If not, nothing else, my matter. That's a capital that you need to just get there. Are the hyper -scalers sort of reserving that capacity for people with deep pockets? Or if you're a startup, can you sign a big long contract, even though you haven't raised the money yet? I'm not sure. I think every cloud provider might be trying to do something different, so I can really like speak to everyone.

46:58I think these days getting compute is hard, it's really hard. And just going into AWS and asking or getting like, or other cloud providers and getting one GPU, it might be hard because there's a lot of demand. And there's a lot of different demands coming from different companies who are trying to get capacity up to speed to train and also run models in inference. Hopefully that double gets off. Yeah. I mean, I guess that kind of brings us to your capital structure and your most recent fundraise. For folks who don't know and didn't see that recently, you just raised $141 million led by Google, but also with Nvidia in the round in your VCs and plenty of others.

47:37There's a very strategic element to that, I would imagine. There is. It's an honor to be able to work with some of the best companies in the world. I think that's, first of all, one of the main takeaways of being able to partner with companies like Nvidia, Google and Salesforce is to make sure that again, we understand that this is not just about models, it's about getting into the hands and building great products that solve actual problems in the real world. So who better to partner with some of the best companies in the world to actually do that? Is the way that the financing market is playing out right now, tell me if this is directionally correct or not.

48:13Pure financial investors are just at a disadvantage because they offer commodity capital, whereas you can go raise a lot of capital from people who can provide access to the scarce resources in AI right now, I namely compute where you could get your dollars from one or the other, but if you go with a corporate investor who actually has this sort of access, it's trajectory changing for the company. It is, and I think it's the investor or less, maybe, so also being radically redrawn and reinvented. I think that Friedman has been leading a great example there, building their own cluster of GPUs and offering that to their companies.

48:48I think that's something rare to see, and perhaps and imaginable just a couple of years ago, but it tells you a lot that value comes not just from capital, specifically with like the last couple of years where interest rates were zero, perhaps capital was actually like just free and cheap. There's more value that's required to build real companies, and if you can provide that by providing or giving infrastructure or doing more than just capital, of course for companies that will be a lot of companies. So funny, all the VCs thought that AI, you know, by becoming further jobs, making investment decisions.

49:19No, it turns out the AI disruption in VC is whether you have a GPU cluster or not. It turns out the platform teams we want at all long were actually just GPUs. I love it. I love it. Totally fascinating. Well, as we start drifting toward a close here, one question I do have for you is for people whose interest is peaked by this, and the answer can be technical answers or it can be more abstract answers. What are the canonical pieces of reading that people should go do if they want to set aside a weekend or a week or an hour and just try to get deeper on a high level understanding of where we're at today?

49:57On just the overall feel of the AI or in particular? Yeah, favorite pieces you've read about it. My favorite pieces, it's a hard one. I remember like Carpathia wrote a blog posting 2015, Colé thing, the unreasonable effectiveness of of recording your networks, which is something I think not in use these days, but I think it opened my eyes into like, why would we possible? So that's a great like more historical piece that I often go to. And then for the visual domain that I can perhaps more relate to since we're building a lot of visual tools these days at Runway, there's this piece by an artist called Kyle McDonald that speaks a lot about using early computer vision models and early, early, early nearly models for video making and image making.

50:39I go back and read that a few times because it always brings me a lot of interesting ideas and concepts around Where things were just a couple of years ago and again coming back to like state of rate of change these days And then besides that you know these days there's so many things going on that's hard to keep up to I think Just Twitter is a great source of like material these days and but not really like getting attached to anything again because perhaps something you read that I'll recommend last week might become obsolete next week, so it's hard to define. We spoke a bit about your history and being at NYU Tish.

51:15How did you find yourself at this intersection? Have you always been fascinated by both engineering and filmmaking and visual arts? Or did you start in one or the other? What's your journey been here? My journey, and I think the journey of my co -founders as well, has always been I mean, very inspired by a combination of multiple things. I have a background in economics and I work as a business consultant for some time. I did art and I exhibit in major places. I've doubled as a software engineer and freelance for some time. I think we're more particularly interested in just, we're very curious people.

51:52And we've understand that the best way to have is just to be able to learn anything. and when you learn that you can learn anything, that's a superpower. And same with my co -founders, they're like engineers, researchers, turn to artists and artists turn into engineers. And that I think gives you some perspective of how to build things that would say break them old or like the systems that we might have established around what the AI world is and what the research world is and what the engineering world is. And you really start understanding that those are just arbitrary like silos and worlds that you can break apart if you know how to speak the languages of those.

52:27I feel like M .I .U .s always had that as at least the TIS program, it's always had that as kind of part of the ethos, right? Were you part of the ITP? What's that? Yeah, you're familiar with the T .P. Yeah. Is that what you were a part of? I was. Yes. It came to study at NYU at ITP and ITP is, it's a rare program, an intersection of like R -dontanology. What does it stand for? It's one of those names that was given, like, I ago and I think perfectly encapsulates that moment of time and technology at that moment in time. It stands for interactive telecommunications program. That's right. That's right.

53:03I love it. It's so old school. Nowadays you do more than telecommunications, which perhaps was the thing people were thinking about in the 40s were when Red Burns founded the program. The best way of thinking about ITP and I think the ethos also of runway that I think we've with God Inspiration from ITPs and art program for engineers and engineering school for artists. It's a frontier of the recently possible. So you can come and do things that are rare and weird and unique. And so thinking about computer vision and AI in 2015 and thinking about it in the realm of art was rare and weird. Now it's all over the place, but I think it was the fact that we were able and willing to go into that that got us where we are right now.

53:49Super cool, the Comple Circle. I remember right after I graduated from college and lived in New York, I of course read Fred Wilson's blog every day at USB. And that's how I first learned about ITP was him talking about it. Dennis Crowley was here. Yeah, Dennis Crowley from his founder, first work. I think the initial ideas came out of from a class that I think is still running called The Games, where you create live like real games in the city. And I think Dennis created like a real -sized pacamining New York. I remember reading about that. In fact, I remember hearing from a friend who was interested in that, back when I was in high school, a year before 4Square came out, that someone from NYU had done this crazy real -life pacman.

54:31I remember the story now, Dennis. It was originally called Dodgeball. And Google bought it. And then he was at Google for a few years, and then, you know, it, like many things at that time during Google, it went nowhere, and then he left and restarted it as 4Square. Exactly. It's a great place. If you want to build and explore technology, ITP is a great place. So cool. Well, Chris, I know there is one other part of runway that we haven't talked about yet, which is runway studios. And that's fun because it's just very cool art that people can go and check out. Tell us a little bit about what runway studios is and how people can view it.

55:06Runway studios is, I would say, the entertainment division within runway. And so we have runway research that pioneers, the research models and the things that we need to make sure we can keep doing to push the boundaries of the field. And Renway Studios is the creative partner of filmmakers and musicians and artists that want to take these models and push them to the next level. And so we've helped produce short films and music videos. We have an active call for grants that people can apply to get funding to make content and make videos and make short films and make even future films with Renway.

55:38The best way to think about it is think about it as a, it's Pixar, it's a new type of a department or company within Runway that it's really pushing the boundaries of storytelling from the creative side of things, not just from the technological, technical side of things. And Pixar's an amazing analogy because obviously the films were to showcase Render Man and the Pixar computer. Like Pixar was a hardware company. I'm a huge fan of, of course, of everything, Pixar -wise. But I think that the key lesson for me there is like, when you are able to merge art and science, great things happen. Yeah.

56:11Yeah, love it. Well, working people reach out to you or runway if they're interested in being customer play around the tools or working at runway or working with you guys in any way. Yeah, I mean, we're hiring across the spectrum. So if you're interested in working with us, just go to runamel .com slash careers. You can also find me on Twitter. In the office, we have a New York and Tribeca. We spend most of the time here at the TV space here. And then run research and studios. Just search run research and run your studios and you probably find the right, the right links for that. Awesome. Well Chris, thanks so much.

56:43Cool. Thank you, Rayce. Thanks Chris.