Runway’s Cris Valenzuela: Building The Hollywood-Friendly Video AI That OpenAI Couldn’t

0:00It's a dream. It's like you have this thing that basically conjures whatever you want. And the first thing people struggle to do is like, it's not to use the tool. It's like, what do you want to make?

0:11Cris Valenzuela:You know, what is in your head you want to make? Welcome to the Upstarts podcast, our weekly show where we talk to emerging startup founders about their upstart moment. Upstarts are challengers who punch above their weight and take on the status quo to improve the world, all while building a big business too. I'm your host, Alex Conrad, founder and editor of Upstarts Media. I'm excited to be joined by the co-founder and CEO of Runway, Chris Valenzuela. Thanks for joining us, Chris. Yeah, of course. Thank you for having me. This podcast is brought to you by Mercury, banking redesigned from the ground up.

0:42Cris Valenzuela:You've historically made these pretty technical video models using AI. You're also based here in New York. We're at the New York Stock Exchange this morning. And you have the distinction of being the only guest that I think in the run-up to the show I've ever run into on the street. because we both live in Brooklyn. Yeah, we're neighbors. Why are you building this company in New York? We started, the three founders met in NYU and we started the kind of initial beginnings of the company while we were at school in New York. And so it was very natural for us. It wasn't typical back then, to be honest.

1:11It was just like eight years, nine years ago. But I think we just, I love New York. I love the city itself, the people, the culture. It just feels very special. And it's becoming very special for the company as well to be able to be based here in New York.

1:27Cris Valenzuela:So for people who are not familiar, what exactly is Runway and how do you describe it to people today? Runway is an apply research company that builds video and world models. And so what a world model is, is kind of an extra tier of AI research and AI capabilities. These are basically models that can understand and take actions within the world, maybe in a similar way that we do. And then the way you get there is by training large video models. And so we've been working on this for quite some time, and now our models are getting deployed in entertainment, in movie making, in robotics, in many other industries, in gaming.

2:04And so Runway builds these world models, pre-trains them, develops them internally, and then we build products on top of them. And so those are kind of the best way to think about it. It's like an apply research company that builds the products, builds the infrastructure, and of course, builds the models to make these world models work out in the world.

2:22Cris Valenzuela:A lot of people are talking about world models in the tech world. For people who are not deeply in that online discourse, why is the world model new and so exciting in this moment? A lot of the paradigm of AI that we've gotten into until now is based on this idea of training large language models. If you've probably heard about LLMs, right? LLMs are just large AI models that are trained on language alone. And so they're trying to get a sense of reality by describing reality. The thing about language is that language is not reality. Language is an obstruction of reality that we humans have created to try to approximate to what things are happening, right?

2:57So a glass of water is not really a glass of water. it's like we made up the symbols and the representation of that. And we train models on that language alone. What we do with world models is slightly different. We train them on video data. And if you show video data and models enough kind of information in pixels, there's enough information about what water is and how it should behave in containers, in other spaces, that the model starts to understand slowly. Not by describing it, but by simulating it. So that's a good way of thinking about it. Language models describe reality.

3:30Cris Valenzuela:world models simulate reality. Where do you get the data, the video data to train off of? It depends. We need to partner with like partners with industry folks. We have people, we buy data, we collect data. Depending on the task at hand, you might need to search from different points. It's a very technical challenge to do. What made you guys confident that you could do it better or have a different approach than maybe some of these big famous AI labs that our audience is familiar with? We started the company in 2018. And I think back then there wasn't a lot of intuition into how to build the models and the kind of models that we're building these days.

4:07And I think over time you build some sort of experience and intuition into what are the best ideas and approaches you can take to scale deep learning models for them to work at this kind of pace and level. And I would say that most of the best research labs these days are not, have been working on this for at least a couple of years. And I think that separates the kind of, I would say, the best from like the new, the kind of like the ones that are trying to build the best models. And I think for us, that's been the lesson. We've been working on this for quite some time. And we have a natural sort of research ideas and approaches and experiences that allow us to differentiate more than others.

4:44Cris Valenzuela:Now, people are probably aware that OpenAI had this consumer video app that they launched a great fanfare and then they recently sunset it. It was actually a miss that OpenAI, Sunset Zora, when we saw it first coming up, it was the first ever real competition we felt we had. And now, two years after they announced that they shut it down, a few things are true. One is it's very hard to build the kind of products and models that video requires and the users demand. And specifically, consumers are just very hard as well. for us it's always been around well we're building towards a world where we think this might become the foundational like base on which you make every piece of content out there from from ads to media to films to things you share with your family with friends or just things you make for you and so the level of craft and the level of kind of control they need to have in the tools needs to be critical when people think of runway people use runway one of the things that keeps coming up as a core differentiation is that it's not just one prompt and you make a thing.

5:48It's you're in control. This is a tool and you wield it in the way that you want. And the outputs are a reflection of your process. And I think the way we think about it might be different from how others might have thought about it, because you can see how different the building of products and models is. And I think, at least from us, from our professional enterprise-prosumer side, which is a lot of what we're focusing on, that really reflects really well. And people come back to us because of those kind of things.

6:16Cris Valenzuela:So the initial idea comes out of this kind of weird lab at NYU that is a mix of like art projects and technology. Can you talk a little bit about sort of what you were doing there and how deliberate versus accidental, you know, developing runway was there? Yeah, that's a good question. So, yeah, we've the three founders met at ITP. So ITP, the best way to think about it is it's art school for engineers or engineering school for artists, which is another way of saying that it's a way to explore the reasonably possible, the things that are adjacently like or new in terms of technology and like possibilities from a technical standpoint.

6:54At the time when we started working on first ABS of runway, the deep learning revolution that we're in was beginning to take shape and form. It was 2015, 2016, 2017. And a lot of what we're trying to do at the time was really understand what was possible. Because we need to remember that in 2018, the state of the art, and I think we suffer from collective amnesia because we forget this, the state of the art AI models, the I think that what people were freaking out about was a model that could distinguish a cat from a dog. That was revolutionary at the time. And to be honest, it was because you never reliably could do it with any other process or technique.

7:35But suddenly now you can go from identifying objects to identifying different kinds of breeds of dogs, which also at the time was fascinating. So the applications of AI and deep learning back then were very boring, if that's a good way of saying it. I don't know, people were thinking about classification or like financial systems or security or organizing data. And for a minute, I thought there was like maybe something else that was more interesting coming from a mix of both engineering and kind of art background where you can see this technology basically as a tool to express or to create or to imagine things.

8:14And so really what we tried to do really early on, once we start training early models, both on kind of computer vision and then on like generative side, was to just build things, prototypes. A couple of things we built back then was we built this film kind of recommendation system or think about it more as a search engine that you can search for visually for shots. So I remember I went to like New York, to a street in New York. I recorded a bunch of people walking and then I put it to our system. and then the system returned me like seven films in which like the same type of shot was like in a scene.

8:49And so I can remember Children of Man, it was like the same shot, but like way nicer done. And so you got all those like kind of opportunities of, well, you couldn't do that before. Now you have the system that can do it. What does it mean? I don't know. We're just building and experimenting and one experimentation led to the other, led to the other, led to the other. And then eventually we're like, oh, we should take this more seriously and start building a product around it and a company around it.

9:12Cris Valenzuela:Did you have a good sense of who your ideal customer or power user would be then? And has it played out the way you maybe initially anticipated? No, I think to be honest, when we started, it was too early. I think we were building something that wasn't a comparison, there wasn't a reference point. So it was hard to wrap your hand around as to why this will ever exist. And again, this is why we sometimes have collective short-term inertia because now there are a lot of things that are obvious. But like a couple of years ago, a lot of things weren't these obvious. And so the use cases of other models, mostly because in a way the models weren't working as well.

9:49So you were like, why would I ever use this? And I think a lot of what I had to do was like prove that you can get better models by improving both data and algorithms and training and everything else. And once you cross that threshold, you start kind of realizing more of the use cases more early on. But I would say that early on, if you're doing kind of research and discovering new things, it's really important to have an open mind. and be kind of less obsessed with what you know and more of what you can discover. And that has allowed us to understand how the models can be used for things that I could never imagine for, which is exciting.

10:21And from a business perspective, what I like thinking about the most is instead of like a company winning a market or like being good in solving specific like need, it's more about creating a net new market. You created something that didn't have a name and you help people do things they never thought they could. And that becomes the new standard. That's how I like to think about what we're doing.

10:43Cris Valenzuela:What was your sense of how artists and sort of the film industry would react to a tool like this? Was this something that you felt they should be supporting and excited about or feel challenged by? I know you've said in the past that you think maybe the next great filmmaker doesn't come from L.A. and Hollywood anyway. So what was kind of your feeling about that relationship? So, you know, back then, I don't think there was a sentiment or a point of view in terms of how you should feel about it. I think it was, at the most, it was just very exciting. Like you were able to do things you couldn't do before.

11:15I remember we trained this model that you could write. This is like before Transformers, anything else. But it was a website. It was a web editor. And I trained it on like a couple of authors that I liked. So it was Bolaño and like Jane Austen and a few other folks. And you could write something and you could get a completion of a sentence kind of as if that person was being your editor or was like helping you. And it was kind of cool. It was it was it was using like LSDMs and it worked in the browser in real time. So it's like a cursor for coding from like 10 years ago. And when I share it with people and with writers, I think people just got excited.

11:52It's like, oh, this is this is very cool. And I could think about all the things I could use it for. And of course, it wasn't as good as the stuff we have right now. But I think the excitement when you show people projects like that and ideas like that is it begins to open their minds into what else can they do. And I think that was at least my perspective on the whole field was like this is a whole new medium. It's a whole new way of thinking about our relationship with technology. And if we really need to think about them from a creative standpoint, then what we need to really think about this is how do we get creative minds to be part of the process?

12:27because one thing that becomes clear, maybe kind of touching back on what we just spoke, is that if you design the systems in isolated ways with one particular point of view of the world, then you're not going to get like the kind of things that you can do in runway and kind of models and stuff because you need to have more of a creative, artistic point of view and actually create tools for them. And so I think we've always thought about it that way. And the excitement that we had back then, I think we still have it today.

12:51Cris Valenzuela:I guess I'm trying to picture you create these visually impressive videos that maybe are in the manner of a movie or something, and researchers are playing with this, how do you start making money? Who ends up being willing to actually pay for what otherwise is a really cool thing to post on social media or something? There's always this utilitarian point of view about models where people are like, well, why would I ever use this to make a movie if I'm not a filmmaker? I'm not trying to sell something. I don't need ads, right? And I think an underappreciated aspect of a lot of this technology is the reward it gives you personally to be able to make stuff like this.

13:30And this is something I would say technology has allowed us to do over time. We just sometimes forget about it. I compare it to like going to the gym, right? So if you go to the gym and you exercise, a lot of times you're going there not to become an athlete or like to monetize your performance in some way and get sponsors because you're going to be the best athlete at particular things, right? You just go there because you feel good. It's good for your body. you feel overall excited to do it hopefully more and more right i like to think of sometimes these models and the things that we're building as kind of a gym for the mind where you are exercising your creative muscles and once you exercise that once you practice that enough times you like it and you're willing to spend money on it because it reward there's a reward mechanism that you're activating your brain that you never had a chance of activating before you're able to make things.

14:21You're able to creatively concept about things you never thought you could. We have users who are engineers, we have users who work in finance, who you've never might have thought of like a runway user, but they are creative. They still have creative ideas, have creative minds. They're thinking about things. They just don't have the technical experience or the kind of resources to execute some stuff. And when you provide them with the things, then if it helps them be creative and they're going to be able to and willing to pay for it. So there's definitely that component of it that it's like an answer to, well, how to monetize this?

14:56I will build things that people like to use a lot and people make hundreds of videos per day. We have people making thousands, even like a week. And of course, on a more utilitarian aspect, well, of course, like this is incredibly useful for studios and agencies and content and gaming studios and everyone who's selling or who's in the business of pixel making. If you're making pixels for a living, this is a completely new way of making pixels with way cheaper and way more controllable methods.

15:26Cris Valenzuela:And I guess I'm thinking with you guys, was it sort of like, hey, you can take this beyond that fun side project and actually use this in commercial ways or ways that, you know, advance your business. Like I'm trying to think what was the aha moment that this is, you know, valuable and sustainable for people. There are a few moments of different products we've had over time where you start seeing how valuable things are. And so, again, on the more individual consumer side, you can just see the stats of people coming back and generating over and over and over again. You're like, why are you generating so much stuff?

15:59And you start understanding, you interview people, and you realize it's just like they like it. They're obsessed with it. And then for companies, we have times where, I don't know, I remember like the late night show with Sibin Convert started to use Runway. and having seen them it was an interesting like a kind of concept and like realization because the person who found out about it just saw like a video about runway and then started to use it he didn't tell his colleagues he was using it but he was like 10xing his like work and so everyone thought he was just like doing some insane amount of like work overnight he was using runway that was his weapon and then at some point he's like okay i'm gonna tell you what i'm using i'm using this thing called runway and and then they became one of our first kind of enterprise like how would they be using it um so we had an early version of runway that allowed you to kind of automate a lot of the editing process of making things um and that's kind of the mantra of runway it's like if you want to make something it should be fast whatever you want to make an early model was a way of just editing things and removing things from scenes that would have otherwise taken you you know hours or maybe days they were able to do in a couple minutes and they were using then those kind of films or scenes or videos into like the show um into like the the goldbird show is like runs live but you have like segments that are recorded so those things were using runway that was like i don't know like four years ago and have many times in like our lifetime where that happens over and over again where like there's a film that was like recently like um screening on theaters that had a couple scenes done on runway and like when you know about what they managed to do and how they did it.

17:35It's kind of insane. It's great. You're starting to see the beginnings of like that creative revolution. Yeah.

17:41Cris Valenzuela:Have you seen the social reaction evolve? Like, for example, do you think if these customers, whether it's a TV show or a movie, if they told their audiences, hey, we used AI to help make this scene or these two scenes, do you think the audiences would see that as cheapening the process or they wouldn't like it? Or you think people are kind of proud to say when they're using it? And has that changed at all? I really want to get and I think we're approaching that time, to be honest, where like you don't have to say how you make things. Right. Like and if you think about how you consume anything, you never care about what cameras were used, what editing software was used to make a movie.

18:16What was the last time? What was the last movie you saw?

18:19Cris Valenzuela:I liked Project Hail Mary for the story, not necessarily how it came together. Right. And people might be like, I want to understand the behind the scenes or whatever. But like 99 percent of the world goes and watches a movie because of the story. right makes you feel something and think if you think of ai as a technology within the evolution of storytelling um dating back to like filmmaking even before that we're reaching a point where like i can tell a really good story i don't have to tell you how i made it who cares how it was made did it work as a story or not you're going to judge it not based on the technical merits of it and the kind of execution of it you're going to judge it on like did it make you move and feel something in some way.

18:57And so that's how I feel about it. And I think we're now at a point where that's beginning to happen, where people are really engaging with really good content that can be generated, mixed or created fully that really doesn't matter. But now you're seeing stuff and you're just engaging with it and you like it. And that's, I think, where we're going to go very soon. And to be to be fair, this conversation, we had had it many times before in the history of like storytelling and creative where like we tend to emphasize for technical reasons the way things are made until eventually you realize like no one cares just move on you

19:31Cris Valenzuela:know as a year-old startup we experienced a lot of firsts at upstarts media so when it was time to process our first international wire following a london event i braced myself this is going to be a painful lesson instead mercury made the process easy issuing an invoice sharing routing numbers and processing the wire with just a couple clicks. No phone calls, no paperwork, no learning curve. We're still figuring plenty of stuff out. When it comes to our banking, Mercury's already got it all figured out for us. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC-insured bank.

20:07Cris Valenzuela:Banking services provided through Choice Financial Group and Column NA members FDIC. We've had CEOs on the show talk about the technology being the hardest part or just finding the first customers or raising funding from VCs who didn't believe in the product until it was obvious. When you think back on the journey so far, what has been, you know, the biggest challenge or on the show, we call it an upstart moment where maybe you felt like you were punching above your weight the most or, you know, facing a very high stakes challenge. There are a lot. I mean, I think building a startup is about like overcoming those moments over and over again and they never end.

20:41You need to start getting used to it. maybe maybe one that like comes to mind and maybe feels like counterintuitive just given where we are and like how everyone feels and knows that this is what models are able to do and how the economy is changing and basically ai is in everyone's mind a lot of the hardest things we had to do back then was to convince people investors and like talent that ai had legs and that it could have it works and again it sounds counterintuitive because like now of course it works. It works pretty well. But back then, it wasn't as obvious. I have a presentation I've shown to our team.

21:18I think we do a version of it almost every onsite where I show screenshots of emails I've gotten over time. And some of them are from some of the best investors you can imagine, like the top, top 1 % of the top 1%. And it's funny because the emails are like, you know, Chris, we're not going to invest in your Series A or your seed. And the reason is that we don't think Generalitative AI is yet like a market or like there's no opportunity yet, like define in quote. And I have this perfect email. Generalitative AI is not like a thing, you know? When you're trying to build something that hasn't been proven, that's counterintuitive and you get an email like that, you're like, oh, maybe either maybe I'm wrong.

21:59Maybe they're right. Or maybe they're wrong and I'm right. At that time, you're not sure. And there's always doubt because you get 50 rejections and you're like, okay, there's something odd here. But I think I've realized that in some cases, a really interesting thing about building something as special, I think, as Runway is, is you have to be able and willing to be non-consensus driven. I think there was consensus at the time that something didn't work until they work. And now, of course, they work. And now it's consensus that this is the thing everyone should be investing. But at the time, that was a very hard time for us, for the company, because we had to convince people that this is worth spending time on.