This is one of the best real-time AI video models we've seen

9:18Joey:Informational stuff. Yeah, yes, exactly.

9:20Addy:I was just going to repeat after what you told me. You're like, I love listening to my own generated podcast to learn about something.

9:28Joey:Yeah, a deep dive technical paper that I just need explained to me. Yeah, I'm not listening to news on the film industry from an AI thing. I'm listening to The Town and other podcasts for The Daily. All right, yeah, I just figured we should address that. 700 billion views, man, that's crazy. I think it was like in four days on this AI.

9:50Addy:It's very puzzling to me. I don't know how it got that big, but then again, lesser things have gotten bigger.

9:56Joey:Okay, let's try to talk about this model. So this is Lucy 2.5 from Deckard.

10:00Addy:You're running in real time?

10:02Joey:Well, this is my webcam feed. So this was the model. I think we talked about it a few episodes ago, and we just never had time to actually demo it. This is the one that will do real-time video generation. Give me a problem. What do you want to change about me?

10:14Addy:You're in an apocalyptic, desolate jungle with cyborg skeletons strewn all around you.

10:23Joey:Let's change something about me, too. Change the background to apocalyptic, desolate jungle. There is a battle helmet on me with long blonde hair flowing under it. I want to try to showcase something. This is probably a very convoluted prompt. It has changed me. This looks like iMovie style. Move your head around and I want to see if there's physics on it. That's what I wanted to show. Oh, my God. Okay, this is weird. We should go with simpler prompts because it kind of doesn't know what to do. But it will recognize my hand and I can interact with the fake hair on my head. Interact with it.

10:58Addy:Yeah. Can you do, like, 1980s hairband stuff? Yeah. 1980s rock star with long, curly, blonde hair, leather jacket, no shirt, and uh oh let's go face makeup like kiss there we go

11:19Addy:this is fun this is a lot of fun uh can you change the background look oh my god i'm squishing the

11:26Joey:fake hair that's this is awesome yeah this was the thing where it got really crazy was like you can video to video real time but then your hand you can interact with the with the video oh do this

11:45Addy:with like a lot of uh individual parts turn me into a chain link fence

11:54Joey:or um chain mail like put chain mail on your body and then touch the chain mail yeah this is like the best real-time video video i've seen right yeah it's awesome like it's

12:07Addy:interactive okay yeah so now touch your collarbone and stuff oh i put it on my arms look

12:20Addy:i mean it's not that yeah it's not deforming i guess yeah it has to be more loose yeah the uh look even hand and face not not bad not bad at all this is i'll take i mean this could be so much fun for real-time activations.

12:36Joey:Yeah. If you had IP, if you wanted to be Mandalorian or something, you could totally be that. I want to try to give it a reference image. What's a good reference image to test this? Davy Jones. Just go to Google. So I added a reference image of Davy Jones from Pirates. Didn't make me look like him, though. Can you touch your tentacle beard thing?

12:58Addy:Oh, there it is. Nice. And go like this. Like, flick.

13:03Joey:No, it thinks it's rigid. Not great here.

13:05Addy:It identifies hair as deformable, but not this stuff. Not tentacle. Do a very wide mouth open, like, ah, ah. Yeah, that's not bad. I mean, facial articulation's all there. Yeah. So I'm using the webcam.

13:20Joey:I would be curious to push this with, like, running it on, like, your phone or something and seeing how it handles if you were, like, actually trying to film something, previs, is, you know, what does it do when it's not just a, like, I'm sure it is trained on, like, mostly stuff like this, like webcam-y stuff, but I'm curious, I've wanted to push this to see, like, can you use it as a very lo-fi, like, previs or reference capture stuff with, you know, how does it handle multiple people or things like that, but this is really cool for uh this is one of the coolest real-time models i've seen lucy 2 you said lucy 2.5 who makes it deck art deck art shout out to them for coming up with something so cool this is a pretty good yeah this is one of the best real-time models that i've seen very impressive thanks for showing me

14:14Addy:that was very entertaining if we're entertained on this show imagine what professional uh experience makers like somebody who's at imagineering or somebody like that can do with this technology

14:26Joey:oh yeah for sure put it like the end of the ride like put you with uh one of the characters and stuff turn you into one of the characters exactly yeah there's so many use cases these weren't the best demos but i did some other tests with like hair or something on my head and it was just like very natural reactive with your hand moving that was like you can have like a davy professional

14:47Addy:actor who's a davy jones impersonator be at the pirates of the caribbean ride outside on the led screen just like picking people off you know roasting them whatnot they have that with the

14:59Joey:shows right they have the yeah turtle crush the comedy show exactly yeah that thing's impressive too all right what else minimax h3 minimax h3 yes sir this one it's all the rage is cool yes from your favorite word to pronounce hyluo hyluo hyluo 3 um okay so this one the vibe has sort of been new model, can take a bunch of inputs like Cdance, Cdance 2.5, which also has officially come out today, but I haven't seen it on the APIs yet, but in true meaning it's out. Multiple inputs like Cdance, but way cheaper. And apparently a pretty good output.

15:41Addy:Yeah, the interesting thing here is when you throw a bunch of inputs at a video model, right? Like in this case, nine images, three videos, three audios, how does it sort that stuff out? and what c dance has and also what this model has is like a level of intelligence in the in the beginning to figure out your intent and then based on your intent where do those reference assets go and how do they play with each other and what attention mechanisms tie into each of those things um so i i think that's like the secret sauce really is just the orchestration of the generation before the generation.

16:17Joey:Yeah. I mean, completely like, yes, you can give it a bunch of reference images if it has to know what you want and what to do with it. You know, I think we've seen success with gray boxing and like 3D motion references, even if they're very low fidelity, just giving a direction of what you're looking for, marking up an image with camera paths, and then honoring that pretty well. And then prompts and prompt structure and just detailed prompts with time code of, like, I want this to happen here and this to happen here and this to happen here. But, yeah, I mean, I think, you know, prompts are my least favorite way to, like, create.

16:52Joey:So, like, if I can have reference images where it's just marked up, like, storyboard or, like, here's, like, a diagram and I want this and this and this to happen, that would be my ideal. If I could just include that as a reference in the reference images that it takes and then it just understands what I'm looking to do with that.

17:13Addy:Yeah, for me, the thing that I obsess over is camera language and cinematography. So if I can just draw an arrow, like move camera this way, and then a circle for stop camera here, and it understands that. And then timing-wise, I can prompt and say at two-second mark, have it stop at that circle. That would be really useful. So I'm curious if the model is able to handle something like that.

17:37Joey:Yeah. The other big thing with this, it is not out yet, but this will be an open weight model that they will release the weights. That's right. And you could run this locally. You can run it on country. Probably need a very beefy system for that. But kind of crazy that something that is up there with inputs and quality could do with CDNs. Could just be on your desktop. you could be running that locally or building or, you know, um, refining it or, uh, uh, and

18:07Addy:building your own versions of it. Yeah. I wonder if it'll be like a distilled version of the real model or not like just having it run on a single GPU, but I don't know enough about this model.

18:17Joey:I don't either. Yeah. I don't know if it, yeah, if it is distilled or if it's just like, here it is, but you need a server farm to run it. I don't know. Yeah.

18:27Addy:I mean, uh, Hyluo has low key been doing a lot of cool stuff alongside the big boys, the Chinese big boys, right? Like your Alibaba and your Tencent. Them and the guys that make Kling. Kaisu, those are the two that I'm a fan of. It's like when you have a big company with a lot of funding and expenditure and staff, like it's one thing, but when you're like a Black Forest Labs or -

18:51Joey:I was gonna say, yeah, Black Forest Labs flux.

18:53Addy:Yeah. Like it's really something to look out for. So this is cool.

18:57Joey:All right. I know you really want to talk about Kami. Yeah, Kami.

19:00Addy:Yeah, so we talked about Kami K3 two episodes ago when it launched. This is supposedly the most intelligent of the LLMs that we have, supposedly better than Fable 5, and best of all, it's open source. So anybody can download the weights, anybody can look at the architecture, and essentially run it on your own, which is crazy, Because if you think about ChatGPT, Fable, Opus, these really expensive closed models and how much these companies are charging you to use them. We just talked about the whole Sunday reset thing, right? Like, you know, we're spending a lot of money already on these models.

19:42Addy:What if that wasn't the way? What if you just had the model? And the other interesting thing about Kimi that I found was how it's architected. Because it's open source, you know exactly how things are kind of laid out inside of it. It's not a black box in the same way that ChatGPT is a black box and Fable is. And it's really interesting how they divide and conquer a task. And so here's what I mean by that. Kimi is trained on 2.8 trillion parameters, something insane. Like that's how many nodes are in its neural network. And at any given average task, it's just lighting up 1.8 % of that. how it's managing to do that and you know we don't know exactly how chat gpt is architected but um for the most part i think chat gpt is lighting up more or less everything at once so it's like a massive inference task for even the smallest things kimmy is a lot like deep seek where deep seek sort of um pioneered the mixture of experts architecture which is like breaking down that neural network into different categories.

20:51Addy:So like, let's say you have a finance problem. That's one category. Let's say you have an image deciphering or VLM problem. That's one thing. If you have a medical question, that's one thing. If you have reasoning tasks where, you know, things need to be thought out, like let's say for law, that's one thing. So I think altogether, there's about 900 or so experts within this model. And when a prompt comes through, the model determines which of those 900 to light up, and it'll light up maybe 16 of them at the same time. Essentially, just 2 % of it is being lit up for this task. So although this model is massive, it's huge, it's fast, and it's responsive, because it's not working all at once to give you answer.

21:37Addy:It's just activating the things it needs to activate. The other thing that makes this really special is the context window, the memory, if you will, in a traditional LLM. As you know, if you just keep asking an LLM a question over and over and over again, it starts to just compound on itself with memory, right? Like it has to remember what you asked 10 questions ago and then use that as a reference point to answer this new question you asked, and it's starting to build context around you. But what happens at like one millionth question, right? Like, especially when you're coding, you have to remember the entire code base and like repositories and plugins.

22:18Addy:And how do you do that effectively? So what Kimmy's doing is it's building a latent compression around the context window. I forget what it's exactly named, but it's the same type of compression that's used on the neural network itself, right? If we think about video and audio compression, where you go from, you know, one terabyte of raw video data, compress it to an H.264, and now you're at like 10 gigabytes, right? Incredible amount of compression still containing more or less the same information. Latent compression is like that on steroids. So like the reason we can run stable diffusion or image model that has billions of images in it on our NVIDIA GPU that has 20 gigabytes of VRAM is because of latent compression.

23:08Addy:Like it's incredibly good at compressing all that stuff into a very small space. It's using that same technology to compress your context window. So now we can store a lot

23:19Joey:more stuff i guess this is also kind of how it um is able to hit similar levels of performance to like opus with but run without less tokens are much cheaper i'm guessing yeah yeah so like the

23:32Addy:reason this is all the architectural things that you have to do to overcome the fact that training a bigger lm model doesn't necessarily make it bigger right and this is what we talked about The reason the AGI-ASI race has kind of stalled and slowed down is because we're kind of out of ideas on acquiring more data. These models have already scraped the Internet. They've scraped every book that's ever made and every website that's ever been published.

24:00Joey:They scraped every book? I don't know. Apparently, Anthropics is still just slicing and dicing and burning books, according to the Internet.

24:07Addy:Yeah, I can't say for sure, but it sure seems like how do you get trillions of parameters? That's one way to do it. but that's not enough which is the crazy thing and that's what Jan LeCun saying is like it's never going to be enough scale is just not going to be enough it's architecture for now that's going to give you the incremental gains until we move away from a language model all together like this entire thing is just how we started getting into AI it's not going to be the end-all be-all but yeah I just thought Kimmy is super interesting if you guys want to deep dive we'll link a couple of videos here that has more scientific information than i'm able to give you i'm not a scientist

24:47Joey:but if you want to nerd out man yes yeah check out this video can be k3 explained in 13 minutes from caleb writes code shout out to caleb yeah not a not an ai a real person yes uh it's it's so

25:04Addy:it's such an interesting space to watch you know i mean like if you take the geopolitical stuff stuff aside, like if you take the America versus China AI race aside, the fact that there is an actual arms race on the architecture of AI models, that to me is so fascinating because that's innovation that's happening before our eyes. And it is going to be, you know, world changing and have significant impact on our lives very soon. Yeah.

25:36Joey:And also what chips the models are optimized for. I think it was MiniMacs, but I might be mistaken. But I think it was MiniMacs, but also just more of the Chinese models are like they're trying to optimize them for Chinese manufactured chips versus NVIDIA chips.

25:58Addy:They're trying to move away from the limitation that NVIDIA is causing. I think I just read today that the RTX cards, the consumer cards are now going to go up by another 30 % in price. Yeah, that's crazy.

26:12Joey:Crazy.

26:12Addy:I mean, they just can't make it. The price for all this stuff. Yeah.

26:15Joey:Yeah. And just everything going up for, even if it's not directly related to GPUs, just like you saw Apple raise their prices and all these other companies raise their prices.

26:24Addy:Right. Silicon, end of the day, is the limited resource, right? Yeah. I think it comes down to electricity, water, and silicon. Sadly, all the data center stuff is true. It is just all about that. It's just about infrastructure and compute right now. All right, on that note. Sorry. Good place to wrap it up. Yeah, don't worry, guys. We'll be fine. We're just going through some transformational change. But everything will be okay in the end. We're still going to have human podcasters join out. We'll be here. We're going to watch real movies with real scale models.

27:05Joey:If you want a vintage-style podcast, you can just tune in here on Denoise.

27:09Addy:That's right. Only if you want AI filmmaking news. There's still going to be comedy shows and horror movies on Netflix and stuff.

27:19Joey:I mean, could AI replicate the Manosphere? I don't know. I don't think so.

27:25Addy:Do we have to?

27:29Joey:I have come up with the Manosphere. Could you have an AI Seth Rogen? I don't know. Joe Rogen, not Seth Rogen. Let's mix it up.

27:38Addy:Seth Rogen would be offended by that comment. Yeah. Sorry, Seth Rogen. Joe Rogen. The studio, man. The studio. One of the best shows that not a lot of people are watching. I love the studio. Yeah.

27:48Joey:All right. Links for everything we talked about at denoisepodcast.com.

27:52Addy:Thanks for all the wonderful comments on the last episode. we got some interesting ones about how Neil Bloekamp's movie is just not good enough do us a favor, tell Neil Bloekamp that

28:04Joey:no, keep telling Addy that, Addy loves it alright, we'll catch you in the next episode