A24's Google Deal, Krea 2 Goes Open Weights, LTX Trainer Arrives

0:00It kind of feels like a slight open source version of Interpositive, Ben Affleck's AI company where we shoot some footage and then we train on the footage and we can make more things with the footage. This feels like an open source version in that realm. Or maybe Interpositive is using LTX Trainer under the hood. It's LTX the whole time. It was LTX the whole time.

0:23Welcome back to Denoised. It's been a minute, Addy. Two minutes. New phone, who this? Two minutes too long. Yeah, Addy was on a nice trip. I got busy. I'm sunburned, as you can tell on YouTube. Yes, it was very hot where I went. I'm glad to be back. And most importantly, I'm really happy to be podcasting again. I miss all of the engagement for our viewers. Yeah, I miss. Yeah, I mean, it has been nice, right? I still run into people, and they're like, I love the podcast. And I'm like, we haven't posted it in a few weeks, but thank you. But we're back. We're back. We had a hiatus. But yeah, Addy, you were gone.

0:59You missed all of the fable fun. You missed the fable excitement. What else did you miss? I was in a very analog world where there was no Wi-Fi and certainly no AI. So, yeah, tell me what's going on. All right. Well, the Fable thing, that wasn't even a story we're going to talk about, but that was an aside. Anthropic released Fable, the Fable model. Oh, yes. I heard that it hacked the NSA in, like, a matter of minutes. They had to pull it back, right? I believe that was Mythos. So they released Fable, which was, like, a safer version of Mythos, their like super crazy model that they're like is too too dangerous too hot to handle fable was released for like a weekend and then the trump administration i forgot the specific order issued an order that basically said it was like it had to be restricted to like u.s citizens only and there's no way they don't have a way to do that so they had to pull the model uh and so the model is off but everyone got to use it for like a weekend it was crazy the thing was crazy i could give it a prompt to build an app and then it just went and did it with like no guidance like it just did the thing and it worked so it's like cloud code on steroids like agentic cloud code it was like yeah i mean i used it for a cloud code use case but it just kind of like ran and did the thing unsupervised for an hour and built the thing it's like i'll be right back i'm sure and then it just goes away crazier uses that happen it was also very restrictive or like it wouldn't do anything that was related to science and some other restrictive cases.

2:29And then they pulled it. I heard that it was actually I'm kidding. I knew I was following along while I was like, I can't stop consuming AI news. But I heard that it was specifically built to exploit vulnerabilities in software, which makes it very usable for hackers. Oh, yeah, for sure. Yeah. Yeah. Zero day exploits and stuff. And then you also Midjourney is now a biomedical company. so you missed that yeah that's crazy and uh at first you know yeah we were all throwing our hands up like oh i guess uh they're kind of broke they need the money from biomedical turns out no their technology is actually really useful for this very specific thing diffusing yeah diffusing images diffusing scans i mean my first thought was like do you trust that if it's diffusing a scan of a human body but apparently with the science that is far more technical than I understand it actually could be very viable and useful.

3:24Yeah, from the article that you sent me, the Cliff Notes was that the way the ultrasonic sound kind of reads your body is just like there's thousands of speakers that blast you with noise, right? So the entire thing is super noisy. So when you get the scan, it's not much different from Gaussian noise that we denoise in image diffusion. so why not just use the same type of mechanism to denoise the acoustic noise instead of pixel noise yeah i'm surprised no one made a joke where it's like we denoised your scan and then it's just like you as like an anime character can you only i i would laugh at that i this is this is the world we're in where i laugh at your jokes you laugh at mine and that's it yeah can can i Can I give you an S ref to style my scan, please?

4:14So my scan looks. It's like, hmm, I need more muscle in my chest. Modern style scan in a minimalist abstract style. Look, I'm just glad that AI technology is being used for something good versus slop for once. So, yeah. I mean, look, if this thing comes out and it's like you could reduce a cost of a full body scan from like thousands of dollars to something. Their vision was like this would just be in a spa, easily accessible, and you could scan yourself much more frequently. What can you discover earlier and treat before it becomes an issue? I just hope it doesn't fall into the shady medical practice category of chiropractors and people that think they're real doctors, but they're not.

5:01They wear the white lab coat and stuff. I mean, hopefully it's an indicator of like, I see something, it's a yellow flag. Let me go talk to a real doctor about this. but at least it helps with earlier detection to then go further to a professional. Exactly. Do you remember the time before weed was legal? You had to go to a shady doctor, and he had the lab coat on. Oh, yes. He's like, oh, you have aches and pain. Yeah, here's a prescription. He wasn't a doctor. He was just a guy from Home Depot. All right. Real stories. Sorry about that, folks. That happened this week, or in the last week or so.

5:41Big one, Google DeepMind and A24 announced a partnership. Google is investing$70 million into A24 in exchange for being a research partner. So if we take that budget into what Obsession's budget was, divide that by$70 million. I mean, we're talking about tons of movies, man. Yeah, I don't know if Obsession's a good model for budgeting. But, you know, the Google already had the Darren Aronofsky thing. They were involved in the Doug Liman Satoshi film as like a tech partner. So they've been involved in a lot of movie stuff for a while. This is called an AI research partnership into A24. And it was clear this is not to like license A24 data.

6:28This is not to like take any of the A24 films and train on them. This was to invest in A24 in one of their latest rounds and then also to help develop tools and stuff in the filmmaking process. The easy go-to one that everyone loves to mention and feels safer is like previs, prep, storyboarding. Sure. You know, which gets people less annoyed when they focus on that. Also, a side thing, Martin Scorsese is also now involved in Black Forest Labs. flux yes with again pitching storyboard in previous he caught some heat on that one online right i'm sure everyone catches heat but then it blows over like i think you just have to like expect you're gonna get like annoying things for like a week and then people move on to the next thing they want to grab their pitchforks over yeah it's the contrast is so stark right you have one of the greatest filmmakers of all time, Scorsese, who's now embracing AI.

7:29On the other hand, the upcoming and emerging filmmakers, I think it's either Curry Barker or Kane Parsons, one of the two newer ones. They're like, oh, yeah, I would never use AI. AI sucks. Yeah, I think it's Kane Parsons. It's great. It should be the other way around, right? The OG guys should be like, nah, I'm good. And the newer guys should be like, yeah, that sounds good. I think, you know, we talk about this endlessly. I think a lot of it's also just like a big misunderstanding or mislabeling of what AI even means, where it's like you say AI and people think, I'm going to go on the computer when you see Dance and spit out a movie from text to video generation.

8:06And it's like, there's a huge gamut of what this means. You know, if we're just talking about previs and it's like, hey, I mean, look, both of those guys, you know, Kane Parsons especially, like grew up with Blender and like figuring out those tools. and it's just like, okay, what if it's just an extension of Blender? What if you could just roughly block out your scenes in Blender and then get a much better previs by video-to-video and getting a higher-fidelity previs out of your thing using a video-to-video model? Things like that where it's like, okay, I think once you get better education and understand that there's a huge gamut of what this means, you come around to it or get on board.

8:44Or they just relabel it to something else. yeah machine learning so let me ask you this like what is the eventuality in your opinion of this partnership between google and a24 like where do you see this few years from now i don't i mean i'm more curious like what's google's interest in this it's not like training better models but you know for what and we've talked about how like the media entertainment or film industry specifically is like pretty tiny compared to gaming and then compared to like autonomous vehicles, robotics, self-driving cars. I mean, you went to Google I.O. Like what is the general theme there?

9:25Right. What is, what are they, what is a trillion dollar idea for them? It's not obviously film and television, but that's a means to an end to get to something bigger. Right. I mean, you know, we've talked about like, I think it just helps with the, it helps feed in or, and, and test out their world models. You know, if people get Omni in their hands and mess around with it and push it and test it, maybe for them it's also a way to just give access to better or more unrestricted versions of the models with trusted partners to then get better data. Because like Omni still is pretty restrictive of what just the general consumer level can do.

10:03Like if I was trying to edit, I was trying to modify some footage with it and then the footage had like a face in it and then And it was like, oh, we're still not doing changes with stuff with faces in it. And so it's hard for me to use it. But I would imagine if they have a studio partner, that they would give them a version of Omni that is less restrictive. And then you can get better learning data off that. the other thing that I'm thinking about and I think in the article that covered this partnership it strictly says that Google is not getting 824 data any of their movies why not I don't know about that Joey I think that's kind of yes the data play is so it's the elephant in the room it's like when you partner up with a studio they got the IP man you want some of that I mean I would believe like they're not going to feed in the archive or whatever also doesn't really i mean they probably know a lot already anyway so does it really matter it's highly stylized highly specific creative intent oriented content which is so far away from you know publicly open source data or stock photo stuff that like that stuff is really a low quality data this is the highest grade of data that you can get your hands on.

11:21I think when they're using it as a tool, like who knows what the training deal on that is, where if they're giving the inputs and the feedback into like Omni or the model, I would imagine that it's like learning off of that. And that's being used as training input to improve the model and also just to learn from it. So maybe not the archive, but like the future stuff they do, maybe that's in the model, or maybe that's being used to train. I'm going to draw a two-year timeline for both companies, and I'm probably going to regret this in two years, but I'll say it anyway. So on Google's end, getting their video models to behave better, more accurately, have better control, all that stuff, it just feeds into their overall AI play of training robots, training cars, delivery vehicles, what have you.

12:13Like all of their big industrial scale stuff depends on having high quality world model, video model, image model. And so that's their play. I think they could care less about being an M &E powerhouse, right? Like Google's not trying to go into the Oscars and grab a SciTech Academy Award or whatever. A24, I think they know they see the writing on the wall that if they don't do it, then Wonder Studios or somebody that's AI native will take their place, right? they'll grow into an A24. So they're sort of doubling down on this, even though there is hesitation internally because they have really indie creators like Kane Parsons who don't like it.

12:56I think as a survival strategy, they're looking at it as a way to sort of just keep up with the times and try to find efficiency along the way. Yeah. I mean, I think as a studio, I mean, we know most of these studios are doing stuff like this anyways, whether they publicly talk about it or not. but like on the very just base level of just having some AI infrastructure for all the operation stuff, for all the previs stuff, for all the other things that might not necessarily be involved in production. And then maybe in some post-production stuff like VFX help or cleanup or other things that would speed things up.

13:30Oh, speaking of Wonder Studios, I mean, did you see Cave-On the Kid's new release last week? I have not. I have not. Shout out to Cave-On, man. that is chef's kiss like but that's what i'm saying is like okay maybe that's not you know a24 grade cinema yet but they have the capability and they can grow into uh like a professional cinema powerhouse right if given the funding or whatever let's say wonder studios gets acquired i don't know by google or somebody and now they're bringing on studio executives and distribution folks and da da da da da and then they figure out the portion of the studio that a24 has and then they have the ai piece covered right like that could well happen in our in the next few years yeah i mean look also you know so we're talking about this like google's interest in having better video models uh you know maybe it's not obviously the film industry is not that big but google has massive ad network with videos and they need tools for people to be able to create ads quickly and fast that are videos, and YouTube, and whatever the AI tools that keep popping up in YouTube and enabling people to make videos where they don't have to use a camera, whatever that ends up shaking out to be.

14:47So, you know, it is useful for Google to have really good video models. Yeah. Oh, it's interesting on YouTube side. Businesses. I thought, I saw something like 20 % of YouTube content is already AI slop. Like it's a problem for them. And I don't think they want the platform to turn into like a 90 % AI content platform. So they're curbing that early. And I think some of their strategies are already in place. Like they still want it to be a human driven platform. AI should just assist. But what's happening is people are automating a lot of content and just putting it like faceless videos and things like that.

15:26Yeah, you could automate an agent right now to just write scripts, generate videos, and upload them endlessly. Like one a day, one an hour, whatever. One an hour, yeah. You could just keep churning this out. Yeah, I know that's been an issue for them, too. Because, I mean, on one hand, they have encouraged AI tools and some other AI tools to help create stuff. And then I've seen complaints from creators, like, having their account shut down by either being accused by Google of using AI too much in their channel. and it's just like where's the line between what Google considers too much AI versus like what they're okay with, especially when they're building tools to help you do this.

16:06But then if you post it and the channels get shut down, what are they? It's a little confusing of what they're looking for, like where the line is between okay and too much. I've also heard of other creators that are like still fully like blender animators being shut down or accused of of AI and then having issues with their channel and it's like I did all this in blender like it's like I'm struggling out here yeah it was all 3D animated yeah yeah obviously we're curious to see how this goes but yeah I mean I feel like if you're a studio you need some AI partner integration to help with definitely just help with operations and like previs and stuff you know if it's like you know nothing about this as part of the plan is like to make a generative back rooms world like that you could explore this is all filmmaker tools that they've talked about so far all right models open models let's do it i love me some open models all right kreotu has been dropping some bangers lately so uh they dropped the well first they dropped the kreotu model like a few weeks ago and I messed around with it.

17:14It was like the closest to kind of mid-journey that I felt in a while. It was very cool aesthetics and very like a fun model to just mess around with and explore different styles. Like an RT model. So that was closed. And then this week, they dropped a Crayon 2 open weight. And I think they dropped it in two pieces, right? So there's a bigger raw model that with the intent with that is sort of to train and build out your own LORAs and then a turbo model for like executing. For just inference, yeah. LORAs are back. Should we do another episode? Maybe we could revisit this with, yeah, CREA 2 LORAs versus reference images or style prompts.

18:00Yeah, I think we were using Z image before. But yeah, look, CREA 2 is obviously going to get you a really specific look and feel. And I think that's the market that they're going after is like we're never going to compete with Nana Banana or GPT Image 2 as far as universal capability, like across the board, generating whatever. But if we can get really specific down to like photography styles or Instagram styles or what have you, I think that could be a really interesting market. And if you pair the fact that this is all very comfy UI oriented where somebody would need to go into comfy because it's all open source stuff and then put the building block together, that goes into then artist driven human in the loop territory so it doesn't feel as inauthentic and agentic yeah i mean the styles i've seen out of this just the aesthetics it's it's it's been really fun to see yeah and look at it's been very um inspiring we're gonna see some of this images and styles here just all sorts of stuff and yeah so you'll be able to train your own lauras to have these more consistent these are all fun yeah i mean like i could just take any one of these.

19:10These are great. Yeah. And just put it on a piece of, like a frame, you know? It's just like a painting. Yeah. And it's open. You can run it locally. You can modify it. You can jailbreak it if you search around the internet. Yeah. And Kriya is known to be real-time and close to real-time. So talk about the distilled model. It's fast. It was fast. I ran it locally. Yeah. The other thing that was interesting, I haven't fully dug into it, but they released this, like, massive technical report that basically detailed, like, how they trained the entire model. It goes into a lot of detail. I haven't finished my Notebook LM podcast explaining this to me yet.

19:47But one of the headlines was that they trained, that it was sort of like part of the issue with AI images where they all look too clean and perfect is because they're all trained on stock images and they're trained on really good looking stuff. CREA trained on good stuff, but also crappy images and bad stuff, or quote bad stuff, which gave it more of this knowledge to create these kind of funkier styles and different visual aesthetics by training on a variety of images, including blurry, crappier, not technically. Where did they get the crappier stuff? That's my question. I have no idea. I'm not going to have that yet.

20:25That is the big question. And honestly, that question is what prevents Kriya from adoption in our industry. right like if the training that data is a question mark then i don't know this is yeah i mean i think it's a good mood boarding style thing reference um also i mean it's just i believe it's just text to image i don't think there's a image image to image or edit image yeah it doesn't yeah so it's like much more for like still photography yeah or just like still imagery or design or mock-ups like Yeah, this feels to me like... I'm not going to be making character sheets or reframing stuff or making first frames in this.

21:11It's like more of a visual explanation. More of a one and done kind of thing. I mean, I guess there's got to be an image to image with Allura if you wanted to give it an input image and change it. I think you can build that in Comfy, like give it structure from a previous image. But yeah, this feels like SDXL or Z image where, you know, sky's the limit to as much as you want to modify because this is all open source and it plays well within Compi. So yeah, Cray 2 has been cool. And now I know I was like not so bullish on Lora's for a while, but this seems like maybe Lora's. Ah, Joey's coming back around to Lora's.

21:47Making a comeback. Also because I don't know if they're technically called a Lora in the next one, but the next update, another open model, LTX, which, you know, we talked about one of the best open source video models. They released LTX Trainer. which is kind of like a video Laura. You can train LTX to kind of just do a specific task in your video and basically to give it an input video and modify it in a very specific way, sort of basically creating these like video. Yeah. And just to kind of clarify here, what we're looking at is some style transfer stuff, which is great. Like, you know, if you want to do a color lot or like a certain look and feel, maybe apply a lens effect.

22:27This is a better example. I saw something really interesting online. I think it was Greg Tiergarten, who we covered in the past. They're using LTX's trainer to do HDR upsampling, which is nuts. That's thinking outside the box for sure. Right. Well, this one, what we're seeing is this is a model trained on just water. And so it's basically giving, it's a water simulation model. So you give it shots without water and then it adds water, whether it's like rain reflections in the street, river babbling brooks in the background. So cool. So this one's kind of cool. That is cool. Yeah, so it's like a very specific physics tool or a simulation tool.

23:10Whatever you want it to be, but it's like you want to modify your video with one specific thing. A look is an easy one, but that one where you're like, make my footage wet or add water is very cool and specific. Yeah, I think six months from now we're going to see something cool like it just hit right and people are going to get creative they're going to experiment trial and error these things take time to figure out those golden use cases out of yeah but i mean i think the coolest thing is where it's like it's this is working in a video video to video space yes and you're able to train it on whatever specific use you're trying to do across and then you could just feed it yeah so probably the funniest version of this was a cross-eyed model where you give it the input video and it just makes everyone cross-eyed which was on display here in this clip from comfy yeah that that goes into scary movie territory doesn't it did you end up going to the theater to see that oh scary movie yeah i did and it's great uh yeah scary movie was hilarious and i remember yes in the last episode i was like i want scary movie to do well because you want white chicks too because i want white chicks too um there was a there was a white chicks too cameo yeah at the end i i saw somebody put their cell phone and posted it on youtube i saw that like at the end right um it was like no it was like we're in the middle it was like or like it was it was in the middle okay yeah but um that definitely got the biggest applause in the audience oh you know decompression wow that's interesting oh wow this is interesting So yeah, giving it some pretty crappy web footage and then it's super sharp.

24:55But that's along the lines of making the image and the frame better, you know, like HDR upsampling or upresing and then de-noising and de-blocking. That's all really useful stuff. And it doesn't impact the image in a stylistic way, but technically it helps you a lot, right? Like as you go along your platform. Yeah. Although other interesting thing is you could train audio with the video too. I haven't seen examples of that yet, but it is possible. All right. I'm going to dig into this more. I'm very curious. But this has been another very cool case. It also feels like it kind of feels like a slight open source version of what they're doing or talking about at Interpositive, Ben Affleck's AI company, where it was sort of like we shoot some footage and then we kind of train on the footage and we can make more things.

25:46with the footage, this feels like an open source version in that realm. Or maybe Interpositive is using LTX Trainer under the hood. It's LTX the whole time. It was LTX the whole time, yeah. I have wondered with a lot of the companies where they spent a lot in R &D for the last two years, and then it's like, we have this cool thing that we can do, and it's proprietary, and then either one of the open source models or just one of the big models comes out with a thing that's just like, eh, we do it in one shot and it's done. Yeah, if I was in the AI startup space now, I would highly prevent myself from building anything from scratch, training any new model from scratch.

26:28Just go out there, see what... Also, how are you going to compete with Google or ByteDance or any of these massive companies that have huge server farms? Yeah, you don't compete with them head on. You figure your niche and your target use case and your demographic and you go after that way harder than they can. Like, I think that's, yeah. Yeah. I think that, yeah, I think you're right. Like, you can't train the model, but you can figure out, like, what's the workflow that gets the results and looks really good. How do you plug all these pieces together in a unique way that other people haven't quite figured out yet?

26:59And then attach some creatives to it and then make them make some cool shit. Also, one other thing that we just saw break this morning. Ta-da! Adobe is acquiring Topaz Labs. Are you going to ask me about it? I mean, I know you don't know anything besides. Yeah, I just found it. So, yeah, if some of our viewers don't know. Nor could you disclose. I guess I haven't officially said it on the podcast, but I do work for Adobe as my day job. And the views expressed on this show are that of my own and not of my employer. I just want to say that. Joey's too. Joey and I are highly opinionated people. Look, we've been a fan of Topaz for a long time.

27:42Joey's covered it on VP Land, right? He interviewed Manji and the team. And they're the closest thing to universal upscaler service that we have in the AI industry today. Yeah. And they really, I mean, it's crazy because Topaz has been around for like 20, 30 years as a... As a non-AI upscaler. Yeah, as an upscaler. Yeah. As a non-AI upscaler. They used GAN models, which I don't understand the math behind that. But yeah, I mean, I used Topaz for years because I worked in documentaries and that was the go to if you had old crappy footage and you needed to try to like get something else out of it.

28:15Topaz was the only answer. And then they really did an amazing job pivoting and building out these generative upscalers with Starlight, Astra and and then have become this sort of like secret sauce, like end workflow piece for like every pipeline where it's like, oh, you're done with your shots. Okay, well, you still want to give it a little extra sharpness or a little extra finish, and you're going to use Topaz with that. And so they're really like that end node in everyone's AI pipeline. No matter what models and everything else you use, the Topaz usually still ends up at the end of the pipeline.

28:52Adobe's had integration with Topaz for a while as one of their... Yeah, like a plug-in type thing, sure. providers yeah and photoshop for operatic images and and firefly uh as one of the partner models to upscale stuff so you know i mean it makes sense that they would want to acquire them and put them in the their ai pipeline yeah i mean i'll just say this like without giving away any any secret sauce uh i don't want to get in trouble but you know some of the ai stuff that we have seen out of Adobe, like Firefly boards, Premiere Pro features like Morphcut, you know, Nano Banana integration in Photoshop, Harmonize, all those things, right?

29:34I think it's nothing compared to the next 24 months of stuff coming out. There's like a whole rethinking of what an AI workflow looks like and what tools are associated with it. So absolutely, Topaz will play a key part in that. But if you think of the Adobe offering in the ecosystem, that is today's ecosystem. That necessarily won't be tomorrow's. It'll be much more embellished with AI in a really useful, creative-focused way. So I'm really excited for that future. I'm ready to meet these guys and see where we are on the org chart and how we can work together. So, yeah. Yeah, that'll be cool.

30:14And, I mean, it's cool. It seems like they're keeping the whole team intact and kind of keeping it as a separate entity. or not separate entity, but keeping Topaz as like a thing. So like Eric Yang, who's the CEO of Topaz, will still lead the team. I mean, yeah, congrats to them. And I think it's a smart play. Topaz is great. They've been around forever and excited to see where this goes. Yeah, same, same. We have some other episodes coming for the viewers. I think that you guys will dig. You know, we want to dig into a specific topic more than just cover the AI developments. So, yeah, stay tuned for that.

30:48Uh, links for everything talked about at denoisedpodcast.com. Yeah. Energize and wake up our YouTube, Spotify, Apple podcast channels again by hitting a like or subscribe. That would be amazingly beneficial. Thumbs up. Five stars. You got it. All right. Thanks everyone. We'll catch you in the next episode.