Beyond Uncanny Valley: Breaking Down Sora

0:01Yeah, honestly I was very, very surprised. I mean, I know it will last often talk about how quickly the field is moving, how hard it is to keep track of all the things that are happening. And I was not expecting a model so good coming out so soon. We've generally converged on it was going to be a win, not an is. I thought it was maybe six months out a year out. And so I was shocked when I saw those videos, the quality of the videos, the length in the ability to generate 60 second videos, it was really amazed. This is obviously the worst that this technology will ever be almost definitely, right, where at the earliest stages of progress here.

0:40I will always tell that that is one of the secret weapons of diffusion models and why they're so active and practical. If you were to ask many people at the beginning of 2024, when we get high fidelity, believable AI -generated video, most would have said that we were years away. But on February 15th, OpenAI surprised the world with examples from their new model. Sora, bringing those predictions down from years to weeks. And of course, the emergence of this model, and its impressive modeling of physics and videos of up to 60 seconds, have spurred much speculation around not only how this was accomplished, but also so soon.

1:21And although OpenAI has stated that the model uses a transformer -based diffusion model, the results have been so good that some have even questioned whether explicit 3D modeling or a game engine was involved. So naturally, we decided to bring in an expert. Sitting down with A6CZ General Partner Anshane Mita is professor of computer science at Stanford, Stefano Armando, whose group pioneered the earliest diffusion models and their applications in generative AI. Of course, these approaches laid the foundation of the very diffusion models deployed in Sora, not to mention other household names like Chachi, T and Mijor.

1:57And perhaps most importantly, Stefano has been working on generative AI for more than a decade, long before many of us had even an inkling of what was to come. So throughout this conversation, Stefano breaks down why video has historically been much harder than its text image counterparts, how a model like Sora might work, and what all of this could mean for the road ahead. And of course, if you want to stay updated on all things AI, make sure to go check out a16z .com, flash AI. Enjoy. As a reminder, the content here is for informational purposes only. Should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16z fund.

2:43Please note that A16z and its affiliates may also maintain investments in the company's discussed in this podcast. For more details, including a link to our investments, please see a16c .com slash disclosures.

3:00This is like a conversation I've wanted to have with you for a while. We've been talking about flavors of this conversation for a long time, but I think given how quickly things are heating up in this space, it sounded like a good time for us to check in and see how many of the assumptions we've been talking about about the future of diffusion models and video models are tracking, you're the world's expert in this area of research. And so I think it just be great to start with her lab sort of involvement in the origins. Yeah, I'm excited to be here. Hi, everyone. My name is Stefano and I'm a professor of computer science at Stanford.

3:31I work in AI and I've been actually working in generative AI for more than 10 years as we before these things were cool. I teach a class at Stanford on deep generative models, it's something I started back in, again, 2018. And I think it was the first thing in the world on this topic and yeah, I encourage you to check out the website. It's called CS236. There's lots of materials if you want to dig deeper into how these methods work. And I've been doing research in generative models for a long time. As you mentioned, with my former student, Young Song, who is now an open AI, who did some of the early work on the future models or score -based models as we used to call them back then.

4:10Back in 2019, the time generative models of images, video, audio, this kind of continuous data modalities were really dominated by GANs, generative adversarial networks. And we were really the first to show that it was actually possible to beat GANs at their own game using this new class of generative models called future models where we essentially very generic content, which generate images by starting from pure noise and progressively denoising it using a neural network, until we turn it into a beautiful sample. We developed a lot of the theory behind this model, how to train them, how to use core matching, a lot of the initial architectures and some of those choices are still around today.

4:49I think that work really kick started a lot of the exciting things we're seeing today around the fusion models, stable diffusion, so -or -out, of course. And in addition to that early work on the foundation of the Fusion models, we've worked on a number of other aspects of the Fusion models like the DIM, pretty widely used like Fish and Sam, the procedure that allows you to generate images very quickly, without too much loss in quality with my student, and Chen Ling Meng, who is the city of pickle labs. We developed as the edit. It is one of the first methods to do controllable generation, to generate images, business sketches, and things like that.

5:27And so yeah, I'm excited to be here today and discuss what's coming next. So given all of that work, all of the experience you've had in really the ground truth of diffusion models and their limitations, what was your reaction to seeing a model like Sora come out last week? Yeah, honestly, I was very, very surprised. I mean, I know that we'll ask often talk about I quickly just feel this moving, how hard it is to keep track of all the things that are happening. And I was not expecting a model so good coming out so soon. I don't think there is anything fundamentally impossible and a new, it was coming.

6:00It was just a matter of time and just more research, more investments, more people working on these things. But I was not expecting something that good happening so soon. I thought it was maybe six months out a year out. And so I was shocked when I saw those videos, the quality of the videos, the length in the ability to generate 60 second videos, it was really amazed. Yeah, I do think every time we've talked in the past, we've generally converged on, it was going to be a win, not an if, that we'd see video generation get so good. And so it's reassuring to hear you were as surprised as I was when it came out.

6:36But before we get into the details on what maybe some of the breakthroughs were there on this time frame. Maybe we can just spend a few minutes talking about video diffusion sort of one -on -one for folks who may not be as familiar with these models. Why has video diffusion been so much more complex than texture image generation? And what's historically been the main blocker in making it work? Yeah, that's a great question. At a very high level, you can think of a video as just a collection of images. And so really the first challenge you have to deal with is that you're generating multiple images at the same time.

7:07And so the compute cost that you need to process and images is at least 10 times larger than what you would pay if you just want to process one of them at a time. And basically, these means a lot more compute, a lot more memory, and just much more expensive to train all your large scale model on video data. The other challenge is just a data challenge. I think a lot of the success we've seen in the diffusion models for images was partially due to the availability of publicly available data sets like Lyon, large scale image and tapion data sets. We're scraped from the internet and they were made available and people could use them to train large scale models.

7:47I think we don't quite have that for video. I mean there is a lot of video data but the quality is kind of like a mixed bag and we don't have a good way to we can filter or screen them or there's not a go -to data set that is available and everybody's using to train these models. So I'm guessing some of the innovations that went into the SORA model are actually on just selecting good quality data to train the models on. Captions are also hard to get for video. I mean, the video data is out there, but getting good labels, good descriptions of what's happening in the videos is challenging. And you need that if you want to have good control over the kind of content that you generate with these models.

8:24And then there is also a challenge of video content. It's just more complex. There is more going on if you think about a sequence of images as opposed to just one that are complex relationships between the frames, there is physics, there is object permanence, and the principle, I think a high capacity model with enough compute, enough data, I'm potentially learning these things, but it was always an empirical question, how much data are you gonna need, how much compute are you gonna need, when is that gonna happen? Is the model really gonna discover all these high level concepts and statistics of the data essentially, and it was surprising to see that it's doing so well.

9:00You just laid out very clearly what the general obstacles have been, both on architecture, on data set, on representations of the world to be a video as a format. Since the release came out last week, there's been a lot of speculation around how this model can achieve such impressive results. Some folks were even speculating that there might be a game engine or 3D model, sort of explicit 3D modeling or a geometry involved in the inference pipeline. But in their article, opening up describes the approach by saying that they train text conditional diffusion model jointly on videos and images with different durations and resolutions and then apply to transformer architecture on spacetime patches of video and image latent codes.

9:40And so could you just break down print -leam in terms for folks who might not be as familiar with scaling laws and what's going on here? Sure, yeah, I can try. I mean, there is certainly some secret sauce here and I can try to read behind the lines of what they said. They release the idea out training on videos and images, it's not new. It seems like one technical difference that they are hinting at is the use of a transformer architecture for the backbone, for the denoiser, for the score model. People often use the convolutional architecture back from the days where kind of a young initially started using units as a score model, which was actually a key innovation that really enabled a lot of the success on images and people kind of still ported those kind of architectures over to video data as well because they make Make sense, we expect that there is a lot of convolutional structure and convolutional architecture might be a good idea.

10:30It seems like they moved on to a purely transformer based architecture and probably following the work by Signing Shae, and then why you did some of the initial work in this space and developing good transformer architectures and play VIT based architectures for diffusion models. It's possible that that gives you better scaling with respect to computer data and just happens to work better. They are also referring to latent codes, so it seems like it's unlikely that they're working directly in pixel space. Working on latent representations was one of the key things, the key innovations behind stable diffusion latent diffusion, my idea of first compressing the data into a latent representation that is a little bit smaller or a little bit more compact.

11:15It's expected there is a lot of redundancy in think about the different frames in a video, and so it might be possible to compress it almost losslessly into a lower dimensional representation. And if you can do that, then you can kind of train on this lower dimensional representation, and you can get much better trade -offs in terms of the compute that you pay and the kind of memory that you need to process the data. So they might have figured out a better way to encode video to a semantically meaningful kind of like lower dimensional latent space. I would say this doesn't rule out the possibility that they've used game engines or 3D models to to generate training data.

11:50I mean, as we discussed before, I think the quality of the training data is really crucial. And it's possible that they've used synthetic data generated by game engines or nerve -flight 3D models to generate the kind of data they want to see, where there is a lot of motion. They can probably use the internals of the engine to get very good captions about what's happening in the video, so they can get a very good match between the text and the content that they're trying to generate. So any dusting like it's a purely data driven approach, but it's possible that they've used the other pipelines for generative synthetic data.

12:24And when you contrast the fusion transformer approach that they took with sort of the prior generations of many flavors of generative models, whether it was recurrent nets, gans, just vanilla other regressive transformers, why is it that a diffusion model here seemed again to be sort of the uniquely suited best tool for the job. I think prior to the Fusion models, yeah, people were using GANs, they were very versatile networks. GANs are pretty good, they're pretty flexible, but one challenge is they're very unstable to train. So that was actually one of the main reasons we developed the Fusion models in the first place.

12:59We wanted to retain the flexibility of basically an arbitrage neural network as the backbone, but a more principled statistical way of training the models that leads you to a stable training loss where you can just keep training in the model, just keeps getting better and better. Outer aggressive models also have that property and is that they're trying to compress the data and with enough capacity enough compute. In principle, they can do a pretty good job on modeling anything including video. They just tend to be very slow because you have to generate one token at a time. And if you go video, there's a lot of tokens and they've not been the model of choice for that reason.

13:34If you use your models on the other hand, they can generate is basically tokens essentially in parallel. And so they can be much faster. And that's one of the reasons they are preferred, I think, for these kind of modalities. Another more philosophical reason is that if you think about a diffusion model, in some sense, at the inference time you have access to a very deep computation graph where essentially you can apply a neural network, maybe a thousand times. Or if you take a continuous time perspective, like a differential equation perspective, it can even be an infinitely deep kind of computation graph that you can use to generate content.

14:06While at the same time, you don't have to enroll the entire computation graph at training time because the models are trained by score matching and there is this clever way of kind of like trying to make the model better and better without ever having to pay a huge price at training time. And so I will always tell that that is one of the secret weapons of diffusion models and why they are so effective in practice because they allow you to use a lot of resources at the inference time without having to pay that price during training time. And to our earlier point about why did the Sora break through happen so much faster than many expected?

14:38It sounds like the stability of the diffusion transformer model and being able to swap out essentially training time for inference time Which is much cheaper much more parallelizable much more efficient was a big contributor to compressing the training times here Yeah, and there is a question of what's the backbone right when that could be convolution on that word I can be a state space model that could be a transformer there I think they're still just scratching the surface in terms of what works and what doesn't and all possible combinations are possible. Right. Build autoregressive models that are convolutional.

15:05You can build autoregressive models that are based on transformers. You can build autoregressive models that are based on state space architectures. And similarly, you can build the fusion models that are based on convolutional architectures, which is what people tend to do. And now it seems like maybe open AI is pushing towards now. That's just a transformer as the background in a diffusion model. And I'm starting to see people exploring state space models, which might allow for very long context, for example. And so I think there is an exciting space of different combinations that we can try and might give us better scaling, better properties for really getting to the kind of qualities we want to see for these models.

15:43One of the most elegant parts of the transformer backbone architecture is that it works really well with the idea of tokenization, right? And in language models, so much of the scaling laws was work that allowed models like GPT -3 and 4 to be developed so quickly and generalized to all kinds of tasks was that the process of tokenizing language allows almost like a transformation or translation into a format that the models can understand across many many different types of languages whether that's good old -fashioned English or its code or its health records or it's different in some cases multilingual data sets and so the beauty of tokenization It's like this one size fits all process of turning language data into a format that the transformer backbone really understands well and is able to learn on.

16:34It seems like there was a similar key on lock around how visual data was broken down here into small batches, right? They essentially tokenized image and video data into this sort of intermediate representation of a patch. This approach created a meaningfully better output than other models we've seen in the past. That's a great question. Honestly, I don't know the answer. Tink tokenization makes a lot of sense for discrete data, like code or text. I'm less of a fan of tokenization or patchifying images and videos and audio. I feel like you have to do it if you want to use a transformer architecture, but it makes less sense to me, just because the data is continuous.

17:14And the path is a very arbitrary and you're losing some of the structure there by going through a tokenization. You kind of have to do it if you want to use transformers and transformers are great because they scale well and we have very good implementations and they are very hard we're friendly and so maybe that's the way to go. It's a bit or less once again, but it feels like what they have is some kind of latent, again, representation and maybe once you go to a latent space then tokenizing it might make more sense because you've already lost a lot of the structure and so it may be a combination of both.

17:49I mean, it seems like they might have access to a very good latent space where they get rid of a lot of the redundancy that exists in natural data, especially in videos, where I think two frames next to each other are very similar, right? So there is a lot of redundancy in videos, and if they got rid of some of that through a clever encoding scheme, then they apply to organization. I think that starts to make more sense and make things more scalable, less compute, less memory, and just better. We've seen so many texts to video models, but very few were able to actually generate longer form videos, right?

18:23More than a few seconds. And there were often issues with temporal coherence and consistency, even across those short form generation, three to five seconds. And here we've got a model that's able to do one minute long, sort of 60 second generations. And arguably some of the long form generations are actually dramatically better than the short form generations from Sora, too. You start to see these emergent properties of temporal coherence only in the 60 second clips, right? What's going on there? What are they doing differently that enables these videos to have such amazing continuity for long lengths and temporal Co -Hansa of the subjects across those lengths.

18:54I think that was the most surprising element aspect of Sora like disability generate long content that is so coherent and consistent and beautiful I think that was the part that really amazed me because I know it's very hard to do because exactly like you you have to keep track of a lot of things to make things consistent and the model doesn't know what's important to keep track of and what's not and somehow the model they've trained seems to be able to do it. Again, it's not entirely surprising because at the end of the day these models are trained to essentially compress the training data. So if you have high quality training data where there are transitions and of course the quantities consistent with physics and it's consistent that has the right properties that we would expect A real natural good quality video to have Then in order to compress the data as effectively as possible then the model should learn about physics should learn about object permanence Should learn about 3D geometry is and all of that What's surprising is that there is many other kind of like shallower correlations that your mother could discover and what was surprising to me is that it seems like it's really able to learn some of that.

20:05We don't know why I think it's one of the mysteries of deep learning. It's probably a combination of training data, the right architectures, enough scale, but it was amazing. And the 3D properties in their videos emerge without any explicit inductive biases for 3D objects, right? They're purely phenomena of scale. What does that mean? Is physics just an emergent property? Well, it's not inconceivable that at the end of the day, physics is a framework that can help you understand the world. It helps you make better predictions. Like if I understand Newton's law, I can make predictions about what's going to happen if I drop an object.

20:40And it's a very simple formula that allows me to make a lot of different predictions. So if I'm being tasked to compress a lot of videos, if I know Newton's law, if I knew some physics, I can probably do a better job, but predicting what the next frame is going to look like, right? And at the end of the day, these models, although they are trained by score matching, it's possible to relate in a very formal sense the training objective that we're using to train on the future model to a compression based objective. Literally just trying to compress video data in this case as much as you can. And so it's quite possible that knowing something about physics, knowing something about camera views and 3D structures of objects, and and object permanence, but these kind of properties are helpful in compressing data, because they reveal structure that is helpful to make predictions, which means you can compress the data better.

21:30What's exciting is that it emerges just by training the model, that you could imagine other kinds of shallower correlations that exist in the training data, but they're not as useful or as predictive as a new tons law or like a real physics understanding of the scenes and what's going on. And it's hard to tell whether what exactly is going on, like it's possible that there is no real understanding of physics, but it's certainly very effective, right? And at the end of the day, maybe that's enough. What you seem to be pointing out there is if the data that these models are trained on is kind of like their diet and you are what you eat, in a sense, if you're eating a ton of physics, then you are better at physics as a model.

22:06Is that how we should begin to sort of explain other emergent properties? It was a clip that they shared called Blingsoo, which was a single prompt generated video that had multiple transitions baked in without any editing and so on. It was multiple camera shots, almost seemed like somebody had manually stitched together different camera angles, right? How should we explain that? Is that just essentially it mimicking what it's seen in the training data? Or is there something else going on? In that case, I'd imagine that yes. If you've trained it on high quality video data where you can see these transitions across different kinds of shots, again, what these models do is they try to understand what all the training videos have in common, what is the high level structure of these videos, and So sufficiently good model might understand that the videos in the training set, they tend to have this structure where we transition across different views and shots and then we combine them and they're combined in interesting ways and then it's able to replicate.

23:03Again, what's magic here is that it's provably an impossible task in general. There is so many other ways of interpolating between the things you see in the training set and most of them are wrong, right? are generalizations that you don't want to see. And somehow these deep neural networks are able to find interpolations or generalizations that are the ones we want, the ones that make sense. And they discovered a kind of structure that we want the model to replicate, as opposed to the ones that we just thereby chance. And it's not the kind of structure that we want them to pick up. And that's what's amazing and mostly unexplained this point, like we do not understand why this happens.

23:40Right. So we're here now in early 2024. and the answer to when will video models get good enough to cross the uncanny valley has just been breached, right? We've just got to that point. So if we look ahead now, Sora is still in data, but there's several other AI generative video efforts out there already, realistically, how expensive do you think it's going to be to generate AI video at any kind of consumer scale, a readily available scale? I'm sure this release from OpenAI, I set off a lot of competitors to the races and trying to catch up. And I'm sure we'll see developments coming from all the various competitors in this space.

24:20I think there is training costs which are huge. I'm sure they use thousands of GPUs to train for insura and scale was a big part of the success they've seen. And so it's definitely gonna be out of reach for academics, but there is gonna be industry players who will have the resources to try to compete with them and try to replicate what they did. or achieve similar results in a different way. The good news is that now we have an active fact, we have a system that can do it. And so I think it's a lot easier to try to catch up. It's not something that there is a lot less uncertainty whether it's even possible, right?

24:54Now we have an example, it's feasible, and a lot of people will make the right investments to really catch up. I don't know how long it's gonna take, it's gonna be six months, it's gonna be 12 months, but somebody will come up with similar performance, I would imagine, as we've seen in other labs and in other spaces, where... people can catch up. The question is how far ahead will open an IP by then, how much better will the system be in six months or in 12 months? And that's hard to say. The other question that I think you were hinting at is inference, like how expensive is it going to be to serve these models and provide video generation on demand to users or personalized videos, like all these really cool applications that could emerge from a really good video generative model.