Green Screen vs No Screen. What is Hybrid AI Filmmaking Going to Look Like?

10:26Joey:Yes, I think that's accurate. Because we mentioned it when Beeple Switch X was showing some new demos of like project shooting with it. And it was this South Korean period film and they were just like shooting in an office. And then like, you know, there's just like stuff everywhere. And then the backgrounds changed and it looked great. And that does make sense. It's like, yes, everything, the data has been trained on a whole bunch of footage. And the more info it has in the frame, it's better at understanding. Like it is not doing a traditional composite where it's looking at the green or the gray.

10:56Joey:And it's like, OK, I know to like ignore that. There are issues where, yeah, there's just not enough data for it to grab onto and understand what's happening. And you do get weird video to video outputs. Even if you were to cover it with like red dots for like marker tracking and stuff. Okay.

11:11Addy:And then the latter, the camera footage heavy, traditional BFX-C hybrid. What are your thoughts?

11:19Joey:That one, you need gray or green because, I mean, first off, it's sort of a hedge of we're shooting this stuff. We want to put it through an AI pipeline. But then if it doesn't work, at least we know we have decent footage with enough stuff to marker track and composite traditionally. and we can, you know, if AI route doesn't work, we can go traditional VFX route or somewhere in the middle and you basically have options with your footage. Because if you're shooting on a very cluttered, noisy background and the AI stuff isn't working or you need to use your camera original footage, you're going to have a very bad time trying to like rotoscope.

11:58Joey:Even with all the AI tools out there, you're going to have a very bad time cleaning up a noisy, crazy background. And so you do need green or blue screen with trackers. That's a traditional route. Gray screen, the idea with gray screen was that these AI rotoscoping tools are pretty good, that you can just select the people and you can get a pretty good rotoscope job. And the advantage of gray is that you don't have to deal with spill as much because you're not getting the spill on people.

12:26Addy:Yeah, the spill is such a big problem with green and blue screens. with gray screens if you have a depth model that is good enough at generating a depth map with a gray screen then your mat your chroma key is essentially solved with a depth model and then you proceed to traditional vfx so now that you put it into two different hybrid buckets right you have a gen ai native very future forward but not quite ready bucket and then you have the hybrid bucket that is pretty much more or less VFX today with some AI elements in it. Now that you put it in two different buckets, I don't disagree with you at all.

13:06Addy:I think you're right. Absolutely. The green blue screen approach is appropriate for the bucket that we're using today. But moving forward in the future, I do think John is right. Like, yeah, you can just shoot anywhere. Like, don't even worry about that thing.

13:21Joey:I mean, I think this goes back to our issue last week of if we get better models that can take these bigger camera files that have all this data and maintain that data, and then when you get your output, it has all that data still in it. Right. Yeah, that'd be great.

13:38Addy:So it also comes down to how good the depth models are getting, and that's neural network-driven depth models, AI models, for the lack of better terms. As far as I know, depth models are still trained on real imagery, so what John said. So even for you to separate yourself from the background, if you're going to run it through depth anything three or something like that, I think it's still better for you to shoot it in a normal environment than a green, gray void, so to speak. However, the bigger question is not which is right, which is appropriate. I think you can get by with either. Like with a good enough artist, you can definitely get what you need out of it.

14:15Addy:The bigger question is, what is the path moving forward? because I don't think the path that we're in today is going to be very economically friendly. And then moving forward, I think the Gen AI native path is the one that's ultimately going to win out because it's just going to solve all the gaps in quality and so on.

14:37Joey:Yeah, I mean, if there's a way you can film, go through Gen AI pass, and then still get your 4K 16-bit camera original, same color science info back in your output, yeah, that could work. Is someone training a model? That would get there. I don't know.

15:01Addy:Well, that's the thing. I think the models are so good already. I was testing Google Omni the other day, and I was trying to do a performance capture of myself and then place it onto a character. Okay, like very, very simple stuff that I think John could do with his eyes closed. What I was really focused on was like, okay, the camera has to be in the right place for me so that the camera for the character is sort of more or less matching. So I'm trying to do camera mash move manually, right? And I was so focused on that that I missed the whole point of mapping performance capture from myself to a synthetic character, which is that you don't need to worry about the camera.

15:42Addy:So I actually did a test where I just did a selfie thing on my phone and I did the line, did the performance like this and then gave it to Google Omni and it remapped it in a completely new camera angle with the character completely in a different pose and everything. And the character had the same expression, same line delivery and like the performance of me.

16:04Joey:But you didn't have to match the camera angle that you wanted. Exactly. You just needed the performance.

16:08Addy:Do you even need to remove the background is what I'm saying. Like you can totally remap the performance without worrying about any of that in the first place. There is no spook.

16:19Joey:Yeah, I mean, that just gets to another level of like, you know, what is an AI stage? And if you have a stage that is doing motion capture or, you know, I mean, I've been seeing new demos of like 4D Gaussian splats where it's like, okay, what if you have enough cameras and you're just literally capturing the entire action of the volume? and then you can just reframe the action and the camera of you know your actors and make any camera you want and you know and change the entire thing um isn't that kind of like what inner positive is

16:49Addy:trying to do right like just find coverage i mean or generate coverage yeah i think that was one of

16:56Joey:the things that they were trying to do i mean that was also it wasn't inner positive related but there was like a paper that came out from my line that was a uh camera reframer but that one is like you have footage you give the footage as an input and you can reposition a camera right i'm talking about you're shooting on a stage you've got uh enough cameras and you're capturing a 4d gauging splat and then you have all of your performances in a 3d you know over time model and you can just have a virtual camera and just remove your virtual camera and reframe anything yeah and ultimately

17:27Addy:like the gen ai native hybrid where a human is a human performance and a likeness of a person is the first and the major input into a series of creative choices i think is going to win because you then you don't have to worry about lighting you can relight in post you don't have to worry about costume or hair or makeup you can do those things in post but the most essential thing is the human performance which as we have tried and tried again is like really hard to get out of synthetic people with prompting and things like that so my guess is like maybe i don't know i think we're still far away out maybe a couple years out that is the standard sort of vfx model is um capture the humans in a regular environment with basic cameras that stuff will get mapped onto that person's synthetic avatar, if you will.

18:25Addy:Like if it's Brad Pitt, it'll just be an AI-generated Brad Pitt with Brad Pitt performance mapped to that. And then off it goes. Why would you do that? Why would you want that? Well, I think because... Why would I want AI Brad Pitt on top of real Brad Pitt? Because the AI Brad Pitt will be in the time period, in the age, in the hair, makeup, in the costume. And the real Brad Pitt doesn't have to bother. So you save a lot of cost. with not doing that to a real A-level actor?

18:55Joey:I think we'll have some pipelines where it's like, yeah, it's fully AI generated. We have AI, synthetic actors. I think we're already seeing that with vertical dramas that are just going full AI. I think, yeah, maybe somewhere there'll be this kind of motion capture avatar-like workflow where you have real actors performing, but they're driving synthetic characters and performances. And then I still think at the, like, whatever, Hollywood, quote Hollywood, a premium level, uh theatrical level and also when you're dealing with unions and and and union rights and stuff it's going to be you're filming with real actors the performance is protected uh we still need to honor that but then elements around them or the environment are changed are generated are synthetic but you still need to preserve their performance yeah no i i agree okay so this is how

19:47Addy:I see it in the future. So imagine you're driving down the freeway, you have the big rig, the big truck that's driving at 55 miles per hour. That's the workflow you just mentioned with SAG overlooking everything and using real camera pixels.

20:03Joey:And that's also the thing where you need to deliver at a 4K theatrical spec. So that's where all of the technical highest quality stuff, you need to match those. So that's like the

20:15Addy:slow moving big truck that That is slowly sort of speeding past you. And it is getting better. It is speeding past you. And then you have this like electric car from the back. Just zoom past that truck and just surpass it in quality in a very short period of time. And that's the native hybrid workflow that I just mentioned for the lack of better naming. Which is that every pixel on screen is synthetic even though it's driven by a real person. and it's assimilating a real person, there is no sort of camera footage that ends up being final pixel.

20:53Joey:Right, and in that case, you're fine shooting the whole thing with your own and just using that as input. I mean, as a camera guy,

21:04Addy:you and me are camera people. We're hardware people. I love my monitors, my cameras, my mics, and my speakers. I even love my cables. Like I love high quality cables. I hate to be in a world where none of that really matters anymore. But the way we're going, I think none of it is really not going to matter anymore. Yeah, you're still going to have the Odyssey and like Once Upon a Time in Hollywood and these like massive big analog quote unquote movies. But then for every one of those, there will be 10 to 15 sub 10 million dollar films that are going to be that speeding electric car that goes by the big rig.

21:44Joey:Yeah, I mean, we're so fixated on standards and reaching quality levels, but then it's like, yeah, just a couple of kids make some, like, cool film on their phone that's like some sci-fi epic thing that blows up online. And then it's like, okay, well, people like that and are responding to it. It's a cool story. Who cares that the quality level doesn't hit, like, standard?

22:06Addy:I don't think it'll be kids. I think it'll be a legitimate Hollywood professional with the assistance of a bunch of kids who are intrinsically just good at Gen.A.I. tools, but the wisdom and the maturity of a veteran filmmaker is needed.

22:23Joey:Yeah, or someone like that case, where they've had a dream fantasy project that was just too expensive to make otherwise, and then they make it, and then people respond to it, and it's like, oh, okay, but it doesn't hit 4K, Ultra HD quality standards for theatrical, and it's like, whatever, it doesn't matter.

22:41Addy:It doesn't matter because it's got a billion views on YouTube, and that alone makes a shit ton of money. Speaking of Brad Pitt, did you see the note that he, or the note, the interview where he was like, yeah,

22:52Joey:AI is just going to unlock the$40 million to$90 million range budget? I didn't see that one.

22:58Addy:Yeah, you can sort of look it up. So the reason I bring that up is because Ben Affleck was talking like that a couple of years ago before we knew he had Interpositive, and he then eventually sold, built an AI company, sold an AI company. So I'm wondering if Brad Pitt has something cooking.

23:16Joey:Maybe he was the one that planted the Brad Pitt roof fight scene the whole time. He was behind it. He's like, oh, you want my likeness? Here you go. That was me.

23:25Addy:Wouldn't that be wild to see Brad Pitt up on stage at like a C-dance event in Singapore or something like that? I think the SAG people have a heart attack back here.

23:35Joey:We'll see. They got the MPA deal. It's all...

23:38Addy:MPA is not SAG. I think SAG is going to be the last holdout. I mean, they're the most protective out of all of them.

23:44Joey:Yeah. Well, I think that's also going back to this hybrid performance. That's where we'll see that play out. The last piece, because this kept coming to mind, what we're talking about, like, does it make sense to film it in a gray box of trackers or not? The gray box, maybe that with the trackers, maybe that feels like this transition period. Like, did you ever see those when electricity was being installed? those um old light installations that was like the top part was like the gas candle flame and then the bottom part was the oh that's that's cute no i have not seen that and it was because like people like did not trust or like electricity yet so they like still wanted their gas option that they like knew was reliable while still being able to like test out electric so maybe of the gray boxes it's like in between technology absolutely where it's like well we we'll try ai but if it doesn't work we at least know we can um we can put this in our traditional vfx pipeline

24:41Addy:and still that's actually exactly the temperature of hollywood vfx today is that yeah we're gonna give these five shots a chance with ai but we have such low confidence that this will be final quality that we're just going to do our own thing anyway so it's very accurate to the gas lamp and the electricity analogy. But I'm telling you, like eventually, not only are we going to get to all electric halogen light bulbs, but it's going to be LED light bulbs. Like very, very quickly, it's going to just level up to the point where, you know, LED is so inexpensive and so affordable and electricity usage is so efficient, yada, yada, yada.

25:19Joey:All right. Mac Mini. Yeah. So update to Mac Mini. No form change or anything, but Mac Mini, Mac Studio now got a big upgrade with Apple's new M6 and M5 Ultra chips.

25:34Addy:That's amazing. Love the M5 chip. I have one. Beefy. I mean, it's a big spec improvement and boost.

25:41Joey:The interesting thing is so much of the marketing and framing around these have been around local AI, your AI agent, not saying OpenClaw, but very much leaning into this, what OpenClaw sort of unlocked when people were just buying up Mac minis to just have these little computers that were just always on could run stuff. It's still like a big leap, and I don't know the specs enough between what you could run on a beefed-out Mac Mini versus if you had something that still had an NVIDIA chip. What kind of models would you run?

26:14Addy:Yeah, I still don't know the exact specifics on how models actually run without CUDA. Like, does Mac have a CUDA emulator or something? Viewers, if you guys know the technical details here, definitely chime in but as far as i'm aware like 90 of the models won't run on non-nvidia hardware

26:34Joey:because they're built on cuda yeah or they just won't run they run well they won't run well yeah right they'll use a single thread instead of like 10 000 threads or whatever and just run terribly slow yeah that's what my understanding was too so i'm curious what is what is a good local model can you get Kimi to run on this and feel the same as if it was running on a system with an NVIDIA?

26:59Addy:Yeah, I'm guessing that Apple has some type of sort of GPU parallelization inside of these things. And then that's what they're calling the neural core, right? I guess M6 has multiple neural cores, two or three of them, something like that. Yeah, neural accelerators inside the GPU. So that would be the equivalent of NVIDIA GPUs having CUDA subprocessors. Like I think they're built, gosh, I forget what they call them. But like within NVIDIA GPUs, there are neural engines in them. But they have like 50 of them or 60 of them, not just like a handful.

27:38Joey:Yeah, this is saying you can max it out up to 24 GPUs on the Mac 5, M5 Pro. Which the M5 Pro is more expensive than the M6. The M5 Pro Ultra is more than the M6. Right.

27:50Addy:Ultra is like the highest level that their chips get to. Yeah. So like a base M6 wouldn't be as crazy as like M5 Ultra. So the base M6 is probably going to end up in like MacBook Airs and Mac Minis and things like that. The Pros end up in MacBook Airs and then the Ultras will end up in like the ultimate sort of use case, I guess, highest level of MacBooks, MacBook Pros.

28:14Joey:Well, I mean, yeah, maybe eventually. I mean, right now it's definitely the Mac Mini and the Mac Studio. That's cool.

28:19Addy:What are your thoughts on the big price hike there? It's like 2X.

28:23Joey:The very base level is$899, I think, for the Mac Mini, which before it was$599 before everything got crazy. So like$300 more.

28:33Addy:I'm so impressed by the M5 chip because I do a lot of video editing and stuff with Premiere and so on. It's totally capable for that. It's so good at that.

28:43Joey:It's great for that stuff.

28:44Addy:And you can set the video monitoring to like 4K and it just keeps up. Like you could hear the fan coming in and all that, but like there is no hiccups or anything. So it's very impressive.

28:55Joey:Yeah. If I was going to pay to upgrade, it would be more so to just get a bigger internal hard drive first rather than like a beefier chip. If I was just trying to get a small computer that's good at editing and other stuff and not running local models. Yeah. I'm very curious how their local model thing comes out. Also, I mean, maybe they're laying groundwork for something that's coming down the pipe.

29:18Addy:I 100 % agree. So remember the fiasco back then, like this was like a year or two ago, when Windows had like this AI agent that runs inside Windows and it just screenshots every few seconds and everything you're doing.

29:32Joey:Was this for all Windows or like if you're an employee?

29:35Addy:So this was for Windows to assist you with your application. So it just needed to know what you're doing. Copilot or something else? I think it was under the copilot umbrella. But anyway, it was like super creepy because it was just screenshotting your own desktop every few seconds. So like nobody wants that. I don't remember this. Yeah. So I'd imagine like Apple would have a more elegant solution to that where it's an iOS agent that kind of helps you run Resolve or run Nuke or whatever, right? right like just very complex hardware or very complex software with the assistance of like a

30:15Joey:llm yeah i mean are they going to come out with a local siri the improved i know we know they're coming out with the improved kind of siri on the iphone but is there going to be like some local version of their own model that is optimized to run on this type of stuff i'm guessing there'll probably be something coming that is optimized to run on their hardware because why make this big push in this whole thing on on ai uh if they're not going to come out with something that is like finely tuned to like really take advantage of this they also know they have like a complete advantage in building up the vertical integration of their own wafers and their own silicon right

30:53Addy:like when the m1s came out a few years ago everybody was like oh my god apple went actually went through it designed their own chip and now they fabricate their own chip they integrated with their own drivers and their os and like the entire stack is theirs and at that time ai was not a big deal and we just thought okay it's it's good for like video editing sure but what else but now it's such a huge advantage because the only other silicon players is like nvidia amd and maybe a couple of small startups like even someone like arm doesn't they don't make their own chips right so they are like one of the three in the world that can actually control the path of which way ai is headed if they wanted to they could build their own cuda or integrate with nvidia's cuda you know what have you and really open up a new revenue stream for ai hardware yeah i mean

31:48Joey:it's totally possible um and yeah i mean they have the advantage of building their own chips i think other companies are trying to catch up like open ai has been and sam allman's been talking about their chip they've been developing.

31:58Addy:Oh, yeah. I saw OpenAI is hiring some camera hardware engineers. I was like, oh, that's interesting. They're actually building this thing.

32:06Joey:Yeah, I would expect something finely tuned to run on, take advantage of this hardware. Also, I mean, it works. Something local works directly into Apple's whole kind of privacy focus where it really plays into like, hey, you know, instead of having all your data and keep going to the cloud, you could just run it locally on your machine and that really fits into their whole brand of privacy anyways.

32:31Addy:Yeah, no, you're absolutely right. Also with like a lot of California legislature and laws coming out to not only protect you from AI data scraping, but also from just data scraping in general. I think them leaning more into consumer privacy and consumer sort of advocacy on that side is a huge plus. So these are big strategic things that we don't quite see yet. You know, like the eight ball was still kind of shaking. But yeah, I think these are all moves in the right direction. I just wish they were a little bit faster, man, because everybody's just moving. This hardware, man.

33:06Joey:This isn't software. It takes time.

33:09Addy:Yeah, I'm sure you can VibeCode hardware design. VibeCode me the M7 chip. Yeah, that's what I'm saying. I think you can. I don't know. I'm just guessing like if there's enough training data on the chip design stuff that they have been doing, feed that into a model and have the model spit out new chip design yeah maybe i'm oversimplifying make me a new m7 chip make no mistakes print it make no so yeah i actually have a background in chip design and stuff that's what i studied in college out of like one wafer if you're if you get like half of them to function then you have amazing success rate like they throw away half the silicon they etch um that's that's how crazy the process of um electrolithography is okay i'm

33:58Joey:nerding out way too much i feel like that's the reason there's only a handful of machines in the

34:02Addy:world that can like make this stuff yeah and that's like the whole like tsmc you know be you know being in taiwan and then china wanting them and america wanting them and like yeah it goes all

34:12Joey:all goes back to geopolitics but it's it's a cool world yeah all right fall h3 before we get to foul

34:18Addy:did you see the claymation video?

34:21Joey:Yes, that's a good lead into that anyways because yes, their announcement video was this cool claymation animation of the Mac Mini, which then they also posted the behind the scenes to be like, hey, it's not AI.

34:34Addy:It was like when Mamdani posted the BTS after his multilingual. I really did

34:38Joey:speak Chinese.

34:40Addy:So that's the marketing move now is you got to post the BTS so you come off as an organic yeah i mean apple's done this before they did this with the apple tv logo and then showed like it was oh yeah motion graphics it was all remember the the liquid glass thing yeah yeah sure but then

34:56Joey:fall in announcement of their new post-trained uh h3 model which is actually cool we'll talk about in a second they posted like we made our own claymation video in 15 seconds um for 90 cents.

35:14Addy:Yeah, that's awesome, but nobody cares. Come on, Sal. That's cheesy to put that up.

35:20Joey:It's a good way to get attention, Addy. It's a good way to get attention,

35:23Addy:but also somebody really worked really hard to make that Apple thing possible, and then you just kind of come up on top of it and say, yeah, you can just, well, of course you can do that,

35:35Joey:but I don't know. We know which one had a bigger cultural impact.

35:39Addy:well that's the thing it goes back to the public acceptance of ai as a whole that that's the that's the wild card that i always think about is yes we can do x y and z and yes you can make a acclimated movie with new characters and that like you can make uh kubo you know you can make uh uh pinocchio today with ai but does it taste good you know what i mean like does the public want that i think that's like a major gray area unknown and miscalculation on most most of us including myself is like we're building things that are absolutely gonna change vfx and animation and live action but at the end like will the public consume it and i think for the most part they will they won't even know it but then again if you can pick between somebody making a clay made an ad by hand or foul generating in 90 seconds i'm pretty sure you're gonna pick the

36:42Joey:handmade one yeah i think the bigger issue with like a lot of the quick ai gen stuff is when it uh hijacks or attaches on to original ideas that were created and then just copies or riffs on that but the work of generating the original idea and the cool aha moment whether it was done by hand or not the fact that it's like i'm making a thing like same with like you know when the ai stuff uses um ip characters and it's like look i made like a batman thing or i made a superman thing and it's like yeah we're also taking a character that already has like tons of investment in mind space and story development and a bunch of other stuff and then just sort of hijacking all that work and not making an original thing.

37:27Joey:And I think that's great. And that is what pisses off a lot of people. Yeah, I think that's what grabs people the wrong way. Yeah. So don't do that.

37:33Addy:Like what I want to see and I think what a lot of people want to see is something completely scratched just leveraging this new technology, but it can totally coexist without this technology.

37:45Joey:Yeah, I think it's an original idea. Something original. Original. Most people are probably not even, Matt Workman made a good point, where he's just like, look, a bulk of people are probably going to see the Apple thing and think it's AI anyways, and just, or CGI, or not even know the difference, or the lingo of the difference. So many people just see CGI stuff today anyways in the movies, and it's just like, oh, is that AI? And they just already think it's all AI anyway. So this is a 1 % us debate. issue of like the actual process to make things but like 99 of people either already think everything's ai or um don't care he's right shout out to that that interview i really enjoyed it

38:27Addy:with matt workman and you having said that i think it's such a critical time period right now with with that one percent of creators right like the world that we're kind of a part of like you no, we're going into the DGA thing and all that. It's like they're still undecided right now and they're still experimenting right now and it won't be for long before they have made up their mind. And that making up of the mind is going to determine the future path of entertainment for the next few years for better or worse. So the jury is still out on how all of this will play out. It is completely to be determined And as much as we think that this is going to be absolutely the way to go, we still don't know for sure.

39:15Addy:Am I just stating the obvious? I feel like I'm really churning here, but yeah.

39:20Joey:A little bit. I mean, I think from an official standpoint, from like the guilds versus what people are actually doing and experimenting and dabbling is a gap. But for like there to be an official position to catch up. Yeah.

39:36Addy:Yeah.

39:37Joey:So like that's really so fast. Here's a good analogy.

39:39Addy:Like there's 7 billion people on the planet and out of them, maybe 1 million of them, which is like a fraction of a fraction of the population is really creating the content that the rest consume. does that is that fair to say like you add up all the big youtubers people that are working in the entertainment industry the guilds and whatever craft and catering people it's like probably a million two million tops right and um i think at a high level yeah i mean if you start talking about

40:12Joey:niche content creators and yeah special interests i'm sure that you know just person on tiktok

40:17Addy:making videos that like has a following the top million people basically make the thing that the 7 billion people consume and i don't mean that in like a class way i just think it's like a niche expertise that they have we're we're in we're in there right you and me like we help with this content creation engine that one million folks have not really fully decided what to do with this they're still trying to figure it out is what i'm saying yeah i that's accurate i'd say i agree with it we'll go with that all right i got two teeth i got two teeth thanks again for having patience with me folks we got there we we landed that eventually it's too hot okay so last bit so

41:03Joey:the thing that follow you the model that they used to make that claymation was their new model that they launched which is i think we'll probably see more of this they took uh minimax 3 yeah and then they post trained it on their servers that basically got better quality out of the better prompt adherence and much faster generations and well because they're posting the api you know quote cheaper it's on their servers so they can kind of set the price at whatever they want is

41:32Addy:that what it was okay i just i just saw a foul announcement and i was like oh new model yeah i

41:37Joey:here the story behind it okay it's h3 max a new post-trained video model by file research they don't say it here but yeah basically it took mini max and then did their own training on top of it and now it's this new model that's faster cheaper only get through fall well i didn't know had like

41:56Addy:data data collectors and model trainers and like high level people like that that's pretty cool

42:02Joey:foul research foul research got it okay top top secret tops it's like area 51 inside the foul headquarters bob lazar's in there what are you doing here so yeah this is interesting i'm curious i mean i know crap was it was it minimax or was it one of the other minor open models that had

42:22Addy:like a laura trainer definitely ltx this might be a full-weight train then sounds like yeah i'm

42:29Joey:curious what this process is like and then how doable that is if you wanted to just make a very specific Minimax for like your own animated style or something. They had someone who was at Fall also training a Minimax that was just sort of trained on, what's his name, Max who was one of the early animation pioneers, why am I blanking on the name? There was Disney and then there was Fleischer. Fleischer? Max Fleischer?

42:54Addy:Sounds familiar. I didn't see this. Well no, I can't find it. I didn't save

42:57Joey:it but anyways they were training uh they were training a model just on like fly share animation

43:01Addy:style which is like that old with his consent i hope he's sorry sorry sorry matt workman and foul guys i i love foul look you guys are you guys are doing a very unique thing in the industry we need

43:16Joey:it yeah foul is great they're training on the animation style um but that's an example of okay if you had a specific look animation style what is the process we should look into this because I'm curious. Just what's the process for the DIY round this is what Paul Trula did

43:30Addy:for that Cucco commercial. They trained on a bunch of hand-drawn Cucco stuff and then the model, I think they trained the Laura, that was it. And then, yeah, they were able to generate.

43:43Joey:I think that one, I mean,

43:44Addy:that was a style transfer.

43:45Joey:But that was just for the images. I mean, this is also like a year ago, so everything's changed. But they were making, they were making a style for, they had a hand-drawn style, they trained the Laura to make the images in that style and then they would animate it. But now it's like, what if you could just train the animation style and then just work off reference to video or text to video, but your text to video, your trained model knows your characters and your style. Yes. And then maybe you don't even have to do first frames or reference images.

44:16Addy:You just describe it. You're exactly describing how feature animation is done, except you're swapping out the computer graphics part for AI part, but the front end is still very human-driven, artist-driven, which is beautiful. So the look development, the production design, all that still comes from people that are really good at that. And in traditional feature animation, it funnels into rigging and designing the character, doing the shading and the material choices and da-da-da-da-da, rendering. But now it's just going into a giant neural network, which is essentially figuring that out and then spitting out a final pixel image.

44:54Addy:I have seen examples Joey of animating on twos do you know what that is no what is it you've seen this so basically animation is done 24 frames per second but if you animate 12 frames per second and then time it to 24 you animate every other frame so it just has like this like this kind of like a stuttery look so if you remember how spider verse looked like that was animated on twos intentionally. I have seen models get on 2's data and then output on 2's delivery, which I thought was incredible. That AI model can figure that out. So even though the output

45:40Joey:of the model was going to be like 24 frames per second, it was able to replicate every two frames is action.

45:49Addy:The little stuttery little. I love on 2's animation. It's like, quote unquote, the vintage classic look. And yeah, AI can definitely replicate that.

46:00Joey:Interesting. Did you want to mention WAN 3? I haven't messed with it yet.

46:03Addy:Yeah, big fan of WAN. I did a WAN 2.2 animated video, I don't know, six months ago. Back then, WAN was open weight. And you know me, I love me some open weights. Hugging face, NVIDIA. So WAN, Alibaba, WAN came out with 3.0. just in time when C-Dense 2.5 dropped, talk about competition heating up. And I threw the same thing into both just to test. And obviously the C-Dense 2.5 stuff was giving me all kinds of likeness alerts and didn't let me generate. But the WAN 3.0 stuff, in my opinion, was pretty close to quality with C-Dense 2.0. So I thought that was really impressive. Okay.

46:48Joey:And you found you had, like, less restrictions and less...

46:52Addy:Practically no restrictions. Yeah, and it was just, like, me uploading myself, just trying to get stuff done, trying to test. Yeah, C-Dense 2 lets some stuff slide, but 2.5 is completely locked down the way I have been experiencing it. Okay.

47:12Joey:Yeah, I've seen some of the outputs. I haven't messed around with it much. uh you know i would say it still feels a little ai compared to some of the other models we have now like cds 2.5 or minimax what if it's focused was like you could just give it a bunch of stuff and kind of how to create a full video i think that was one of its sort of pitches where like you could give it like pdf and images and some stuff and then be like i need a 30 second you know trailer or commercial and it would do like kind of this multimodal stacking together

47:39Addy:thing yeah i don't know why why people want that like complete automation social media stuff it's i think yeah i can't stand it nothing is great i'm saying that's fine it's the feet yeah that's

47:53Joey:why people are into it but yeah one so do you think they'll go back to an open weight model because they had 2.2 that was like yeah but kind of became became the like standard yeah for a long Open weight until, you know, like Minimax and stuff. But then they also had 2.5, which 2.5 was the first API. Closed. Yeah, we covered it on the pod. Yeah. And then now they got 3.0, API only closed.

48:17Addy:I think they can never go back to open weight with the current architecture. Because I think the fundamental limitation with open weight models is they have to fit into a single GPU so the person at home can run it. Not necessarily. Rarely.

48:34Joey:I think Minimax is too. I mean, I guess some people have run it locally, but.

48:39Addy:Yeah.

48:40Joey:Was it Minimax or was it like Kimby? One of those models where it was like open-wave. It was massive. Where like you'd need like a server to run this.

48:48Addy:Yeah, and it just becomes like, okay, there's maybe a hundred people outside of an enterprise that can actually do this on their own, right?

48:55Joey:Versus like millions of people with NVIDIA GPUs who are gamers can do this.

49:00Addy:I think right now, like 26 gigabytes of VRAM, something like in that realm, if it's smaller than that, it should be open weight. But any bigger than that, it's probably not going to be.

49:10Joey:I would think Juan would be in the bucket of like, it made sense for an open weight because parent companies, Alibaba, they have server farms, they have all of these things. Like you could run it on their hardware. So it seems, or train stuff on their hardware. So it seems like the incentive to have something open weight that would then have people wanting to use your servers would make sense oh i see so it's just

49:39Addy:more like a cloud play like a cloud revenue play for all yeah like give it out for free so you

49:45Joey:could use our cloud stuff whereas like mini max i forgot the parent company not tied to like Hylul.

49:50Addy:Hylul.

49:51Joey:Are they, I don't know, do they self-servers?

49:54Addy:They might make cars for all we know. Like Xiaomi. Xiaomi makes phone and cars. We ask this question all the time. Every time an open source model drops is like, what's the point? Why do it?

50:07Joey:You know, I think the open source ones, when the company is also self-servers and other stuff, it makes more sense because they want you to run it on their stuff.

50:17Addy:And piggybacking off that question, why have an open source repository that just got to over$13 billion? It's like, these are the questions we don't have answers to. At least I don't. I don't know if you do.

50:28Joey:So Minimax is a independent AI group in China. However, in March, 2024, Alibaba Group led a$600 million financing round for Minimax. Oh boy, there goes that.

50:42Addy:I guess the only small guy left is Kaisho, who makes Kling. I think they're still a small company. And then shout out to Black Forest Labs as well as the LTX folks, Lightrix. This is obviously Stability AI. Yeah, there's a few smaller model companies left in the world. But then, yeah, it's mostly the Googles and Alibabas and the Tencent's of the world now.

51:08Joey:We had it backwards. Minimax is the company. Hyluo was their text and music generating.

51:15Addy:Gotcha. Okay.

51:17Joey:Okay. So clarify.

51:18Addy:Thank you. Thank you. Yeah.

51:20Joey:Thanks Wikipedia. Okay. Good place to wrap it up. We've rambled a bunch. Links for everything we talked about at denoidspodcast.com.

51:29Addy:Hey guys, on YouTube, there is the hype thing. Can you give us some high points? That would be really nice. Thank you. All right.

51:37Joey:Thanks everyone. We'll catch you in the next episode.