Breaking News: Nvidia's Text-to-Video AI Generation Unveiled

0:00Today on the podcast we are going to be talking about the next step in AI we have chat GBTGPT that is doing text generation We have things like Midjourney that are really effectively doing image generation. And now the next generation is video generation. And that is right around the corner. NVIDIA recently just announced and showed off a new text-to-video technology. And today on the podcast, we are going to be diving into this, what its current capabilities are, and what the implications of this are on the AI space at large. So the first thing I want to say is that a new research paper and microsite from NVIDIA's Toronto AI Lab called High Resolution Video Synthesious with Latent Diffusion Models came out.

0:48And that's where I guess this new research paper is kind of what this is all built on. but essentially it's giving us a taste of the incredible video creation tools that nvidia is about to launch um and that are going to be the people are going to be able to uh generate and obviously nvidia is for those that don't know it's a it's a technology company that creates mostly chips um so most of these these big ai models open ai and a lot of these other guys are training their ai models on nvidia chips so it's in nvidia's best interest to help develop ai technology. So A, they could take advantage of it.

1:25But B, my assumption is they're going to, you know, create some good video editing and video creation technology. So that it's, you know, because it's obviously going to be incredibly resource intensive, and that is going to drive the sales of all of the Nvidia chips, AI training chips. So that's my opinion. It's really interesting though, and obviously an incredible breakthrough. So latent diffusion models or LDMs are essentially type of ai that can generate video without needing really massive computing power right like obviously this takes a relatively large amount of computing power but in the past this would have been insane and now it's becoming more manageable so nvidia said that um its tech does this by building on the work of text image generators so in this case stable diffusion and they add a what they call a temporal dimension to the latent space diffusion model so that sounds fancy but really all that saying is that essentially it's generative AI can make still images that you could generate on stable diffusion or mid journey or something like that move in a realistic way and then it will upscale those images or increase the quality using some super resolution techniques and this just means that it can essentially it can produce a 4.7 second long video with the resolution of 1280 by 2048 so that's actually pretty great resolution and it can also do longer ones if you're going to lower the resolution a bit so if you're doing like 500 by you know 1024 um it can do it can make the videos a little bit longer i think a couple minutes or yeah i think uh maybe three or four times as long so you know i think the immediate reaction that i have to all of this and seeing these because they have a couple interesting demos they got one where it's like a stormtrooper and he's on the beach and he's vacuuming and the waves are going and um it looks like there's a vacuum tube coming out the back of his leg and it kind of just attaches to a shadow behind him so that's kind of funny it's obviously like a glitch in the in the video editor or an image editor and then it turned into a video so the vacuum tube's kind of moving but um they have one like that they got one where it's just like this uh you know stuffed bear playing an electric guitar and so I think that this is obviously you know a really big move just because we can see where this is going in the industry as a whole But I think like for where the technology is at right now It's immediately people are gonna use this to make gifts.

3:50I think that's a big thing, right? These are five-second videos that you can generate so immediately I think this is gonna be used to create gifts and then in the future I you know obviously it's not gonna shock us as this Get better and better and people are able to make a lot more robust longer videos So, you know, I like you're able to essentially make one of these videos with a really simple prompt Um, so this is gonna be uh, this is something similar to what you get on mid-journey But you could just say a stormtrooper vacuuming on the beach and boom it generated an image. I'm looking a video I'm looking at now Or you could say, you know a teddy bear is playing the electric guitar high definition 4k kind of like mid-journey where you have like commas And all these extra things.

4:27Um, you could do the same thing Um, and yeah, so I think right now that makes the text of video technology that nvidia is currently demoing It's really just like i'd say it's most suitable for thumbnails and gifs and those kind of smaller things, but obviously going to be useful. And I think this is a really big step in the direction that we would like to go, which is going to be, you know, people being able to generate a lot longer video scenes. And I think that we're probably not going to have to wait too long just given the speed of the industry right now before we start seeing those really complex and bigger videos coming out.

5:04So I think it is interesting and important to know that NVIDIA isn't the first company to kind of show off some of this AI text-to-video generators. Recently, Google made a debut with Fanaki, I think it was called, and essentially they had like a 20-second clip video that they can create based on some longer prompts. So in their demo, it was showing something that was, I think, over two minutes long. So Google did a little bit longer. I mean from what I saw of that it was I would say probably a little bit less quality But it'll be interesting to see where that goes I'm really happy. There's a lot of different companies that are all competing for this because obviously you don't just want one that completely corners the market So I'm happy that Google and NVIDIA and a lot of these guys are all working on it Because I believe that's how we're gonna start to see a lot of cool stuff The startup runway which helped create the text image generator stable diffusion also revealed its Gen 2 AI video model last month.

6:06And so along when they did that, they had like a video that the prompt for it was the late afternoon sun peeking through the window of a New York City loft. And, you know, I watched that video too. It looks a lot like a GIF. It looks kind of glitchy. The shadows are kind of not perfect. but like it really is not uh not crazy to think that this is not very far off before um this is a lot this is gonna be a lot more realistic and a lot longer and you're gonna be able to you know essentially have an idea for you know I want a Star Wars movie maybe with um James Cameron playing the lead role and blah blah blah blah and it's set in the 1920s and instead of lightsabers they have guns and Al Capone is Darth Vader.

6:52Like you're gonna be able to say some like crazy stuff like that and it's going to just create the video. So I think it's gonna be really interesting. Some people that are talking about this are talking about the fact that when some of these video studios release movies around the world, you could have the main character reflected and looking like, you know, a celebrity from different geographic locations or just like a generic person from different geographic locations. So that would be really interesting. There's a lot of interesting use cases this ai technology obviously once it hits video is going to be absolutely insane um and it doesn't seem like it's that far away people are pushing in this direction advancements are being made in this direction so it's really interesting um and i in addition to all of this we have a lot of just like video editing software that's coming out that is integrated with ai we have adobe firefly um which uh just came out that is going to kind of tackle that in programs like Adobe Premiere Rush you're gonna soon be able to type in the time of day or season you want to see in your video and Adobe's AI is gonna kind of do the rest so I think it's gonna be pretty interesting obviously right now right off the bat what it's capable of doing seem more like creating GIFs but I don't think it's gonna be far too it's I don't think it's gonna be a long shot to have these things creating more fully fledged videos so it's gonna be a really interesting space to continue to watch in the future.