Reference Images vs LoRAs: Which Actually Works?

0:28All right, welcome back to Denoised. Okay. Look, the reason I wanted to do this is because there is a debate. We've had this debate. And I hear this. We've had this debate. It's like, do you even need to train Allura and do all that stuff to get this style? Allura's dead. When Nano Banana Pro and Flux Contest and all these newer models, you can just throw a couple of references in and get the thing that you want. And so this one specifically, we're kind of trying to target the cinematography style of an existing film, saying if we're pre-pro, we're trying to do some reference images and we want to know we want it to have the vibe of another film.

1:08That's what this experiment is. And so the film we picked that I've been biased all of 2025 as my favorite film is One Battle After Another, unsurprisingly. Yeah, like I was like, Joey, pick a movie. And he picked that. I was like, spot on. Yeah, so shot by the amazing Mike Bauman on VistaVision. But yeah, we just shot deck and grabbed a bunch of frames. And initially, I grabbed kind of like a mix of different frames that just look cool. And then initially, I grabbed a grab bag. And then we were like, it's too varied. So then we're like, OK, let's just go after three sort of distinct looks that happened during the film.

1:46so one look is the first part of the the film which is more of the magenta border patrol raid uh that had this like nighttime magenta look the second look i grabbed was when they were kind of running around sacrament you didn't know what the city was but they shot a lot in sacramento uh downtown sacramento with this sort of neutral office building look and then the third one was the desert area which was a lot more orangey desert vibes which is where you went There is a video and another upcoming video, once we finish it, of tracking down all the desert locations. The second video also has to do with Gaussian Splats and Beeble and a bunch of other random stuff that we're experimenting with.

2:26So that one I'm excited. Once it's finally done. I'm excited to see it. Once it's done. I've been seeing some cool tests, but that one will be a cool video. And I also tracked down the – I've just had a mission to find where the rolling hills were. And I did find them. They're like four hours outside of L.A. But anyways. ways you even found the tire marks on the road which is crazy yeah from the production it was like from the production when the car is like screeched to a halt yes they're there they're still there yeah so anyways we got those shots and as ford of these sort of like okay three kind of distinct looks can we replicate those looks and generate other frames that have the same vibe uh that feel like they might have been part of that scene with that same style cinematography for the idea of experimenting with previs or just kind of planning for another shoot.

3:13Yes, so let's take a step back as to why we're doing this in the first place. The power of generative AI in pre-production before you get into production, that's undeniable. There's absolutely a way to apply this transformational technology to something that's really cumbersome and there's a lot of iterations, a lot of decision making that happens, which saves tons of money down the road. So during production and during post. So if you do pre-write, the idea is you could be way more efficient in production and post. The challenge is, how do you build the look and feel of a film in pre-production?

3:51How do you do it consistently? How do you expand on that universe that's still in your head onto a screen where you can share with others and get creative notes on it? So this is the inspiration. This is the motivation behind why we're having this showdown. Yeah, and so Addy went with the LoRa route with ZImage. And yeah, let's start with you, Addy. So why don't you walk through how you trained the LoRa? Okay, so we talked about ZImage being sort of the sweetheart of the creator community at the moment, right? Like on the video side, you have LTX2 that's doing a lot of cool stuff. And on the image side, a lot of people are feeling ZImage the same way they felt about SDXL just a few years ago.

4:33So it's fully open source. It has strong image-to-image capability. It has ControlNet. You could train LoRa's on it. So you can do a ton of stuff with it. And you can go on markets like Civit.ai. You could buy the LoRa's, download the LoRa's. You can train your own. So I wanted to use Z-Image. It's pretty easy to do, and it's widely supported. So what I did was, instead of doing it in Comfy, which actually I'll do on another episode coming your way, I wanted to just quickly get this done. So I use Foul. And so first thing for me was to figure out what the Laura is actually intending to be used on.

5:12For me, it was the look and feel of the movie. Specifically, it's the VistaVision look, right? Like that film and that camera has a very specific look, even compared to something like IMAX. And also the color grading choices that were used throughout the film. some of the compositional choices, some of the intrinsic landscapes that it was shot in. So like the physical location, the fingerprint of that location would actually come through into the LoRa. So trying to get all those, extract all those creative notes out into an AI model and to repeatedly generate it in that world. So let's quickly go through the training set.

5:54Okay, so as I said, as Joey said, the training set is just from ShotDeck. And again, I just want to highlight that this is for educational purposes only. This is not something I would ever do in a real production. For that, I would actually go out and shoot a small film and then use a lot of that as training data for Laura's. This is one way to just kind of shortcut that process. If you were doing this for real, you would, maybe on the Location Scout, take photos with your phone, color grade them to the look you're thinking of, and then train that and use that as your source. Is that what you're thinking?

6:31You could do that, or you can actually shoot a couple of sequences. A lot of productions just go out and they'll shoot one sequence or two sequence. That could be used for pitching. That could also be used for Loras. Okay, interesting. I'm thinking like, look, when you're in the sort of previous phase, like I've seen a lot of pitch decks and stuff where you're just trying to get the idea across. A lot of times they will just grab images of like things that already exist because it is a similar style or tone of what they're trying to get across. Obviously, there's no that's never going to be released or public.

7:01But in that realm, like I still see a use case where it's like, yeah, you would grab images, maybe not from one film, but from like a variety of films that have a certain look. Multiple films. then train a Laura purely for pitching or reference stuff to figure out how to get. Because even if you're like, oh, we might shoot one or two days of test stuff, that's still a big investment, and you would want to nail down the kind of look or cinematography you're going for before you start spending that kind of money. Yes, absolutely. You're 100 % right. I think it's really a budget question, right? Like what film it is that you're making.

7:34If you're making a$10 million movie, yes, there is budget for test shots and test sequences. but if you're making a million and under, there is none of that. The best you can do is have a camera and go to the location that you're thinking of. Because it's like, okay, we can experiment and figure out the stuff now on the computer when it's cheap, we have time, and we can really dial it in. So then everyone's on the same page and we actually do spend the money and we're making the best use of that money possible. Yes, and to kind of touch on what you said just now, it's really important to mix a bunch of different mood boards into the pre-production phase.

8:07like a lot of people do that really well they've turned it into a career yeah it's like you know you take two stills from star wars two stills from guardians of the galaxy and now you know you're trying to visualize the sci-fi yeah just shout out to shock deck i mean they do a great job at that where you click on an image from a frame it gives you the color uh profile but then it also gives you similar shots just in the sense of the color the composition like how it is but it's from like all sorts of movies so it's also another kind of good way to just do a grab bag of like if you're trying to find shots in the same realm but that are not from the same exact movie.

8:39Yeah, exactly. So again, for educational purposes only, we're just grabbing stuff from Shot Deck. I don't recommend you do this. Check that you can actually do it and you have the rights to train on Laura's. Okay, so these are some of the stills from the movie. You can see on this one here, intense halation around the stadium lights. There's just like, there's really no black, like true black in the frame. Even the black of the night is like this really nice indigo, dark blue color, navy blue color. And then, yeah, just like the reflections are this light blue color. Like, it's amazing. And then, of course, you have a lot of the set pieces that are a little bit brutalist, right?

9:21Like, you don't have a very clean, sophisticated city or environment. These environments are also very rugged, grungy, dirty, dusty. And so we want to extract all of that from these training sets. And then once I have selected the images, I have about, I don't know, 30 images. Did you send this whole group together or did you do it by, because there were like three groups. I put everything into Allura because the bigger your Allura set is, the more versatile, the more useful it is. In the case of like training Allura for a character, like if I wanted to rebuild Joey over and over again, I wouldn't need 30 or 40 images.

10:02I could do away with 10. But I'm trying to build a universe. Okay, so you just trained one Laura. You didn't train a Laura on the nighttime look and the desert look. It's just one Laura? Yes, so you can get away with that with really good captioning. Okay. Because at the end, what you're affecting are the identifiers in latent space. So if you're looking for nighttime, it's just affecting that nighttime vector in latent space or if you're going to modify the desert or the highway right you're just modifying those two things and you can do all of that with a single laura and really good captioning okay so these are my caption files here so for example you know medium close-up of a young woman with a light brown skin and dark curly hair pulled back wearing a white martial arts gi she looks directly forward so i did get the help of gemini with some of the captions and then i actually went in.

10:58I corrected some of that, added some color. I was going to ask if you wrote these or if you ran it through something. This would be where Claude Cowork would come into play. You could just point it to the folder and then have it make all the captions. Yeah, so auto-captioning is a thing and that's what professional AI companies use. For me, I wanted to have just a little bit more control because this is a bespoke quote-unquote film thing. I went into the captions and then I further modified it. So your captions are a text file paired with each image that has the same exact file name? That's right.

11:30So now that I have my pairs all sorted out here, you can see, the next step is to just then compress it into a single zip file. You don't have to do that. You could just upload them one by one. So I'm a big fan of FAL. Then I went into the FAL Z Image Trainer workflow. You can just Google this and it'll take you here. So you can see here you can drop the image one by one or you could just throw in a zip file. I threw in a zip file and then I just kind of kept everything to default. The learning rate is like how much you want the LoRa to affect the main model. And there's different types of training.

12:09So for example, if you're trying to train on a style, I would probably pick this or the actual content of the frame, like a person or an object, then it would be that. But I just kind of left it at default, which is like a balanced. Then the training takes, I don't know, five, six minutes. It's really quick. And then once the training is done, you have these two things that come out of it. A LoRa file, which is a.save tensor file. This can go right into your comfy workflow if you download it. You can see here it's only 82 megabytes, super light. And then the config file. I'm not sure what the config file actually has in it.

12:44I just didn't need to use it. So in order to do inference on this LoRa, I used FAL once again, and you could use Comfy as well. I just use FAL because I just wanted this to go real quick. I was doing this last night. But you could download this file, and if you have the Zimage LoRa workflow from Comfy, you could just add this file to it and run it locally. Yeah, and I'll actually do that for a future video, exactly what you just mentioned. So this file lives on the FAL directory, so I just copied the link to that. And then Val also has an inference workflow for Zimage, which is this one. And this inference workflow takes a LoRa.

13:26So it actually takes multiple LoRa's. I added this one and then going to go ahead and paste that in there. The scale is basically the weight, how much you want things to be influenced. So I'm just going to go ahead. and the litmus test for me, the first thing that I wanted to kind of see if it's working is that desert highway road. So I'm going to go desert highway road as my very first prompt, just to see if the lore is working. Because remember, in my training set, there was a ton of that environment in there, more so than any other environments. So if I was to see the effects of a lore, I would see that there first.

14:07All right, so then I'm just going to hit run. and the nice thing about Z Image, it's a super lightweight model. It's literally like three or four seconds and you get your output. I'm using out of Banana Pro where I'm just like waiting around. This is a very nice thing about that, to see that super fast. Yeah. So Joey, you've been to this exact environment. Does this feel and look like that desert you were at? It does feel and look like it, but now I'm also wondering because I thought my interpretation of our challenge was like We want to capture the lighting, color, cinematography of the place.

14:43Yeah, we do that too. But not necessarily replicate the same exact shots. That's a very fair point, my good friends. And yeah, the showdown's getting intense. What do you got up your sleeve? But yes, this looks like the Texas dip, the big steep hill that was the last one. And I think the area you said is Borrego Springs, right? It is Borrego Springs, yeah. Okay. Okay, so real quick, I'm just going to change it to a 16.9 aspect. It's just closer to the movie. Technically, the movie's VistaVision, which is a 1.5 ratio. I know, I know. All right, all right. Take it easy, film guy. But however, as we look at shot deck.

15:21So real quick, yeah, I hear you on the challenge was to not just replicate a desert environment. That seems pretty easy. But to replicate several shots from the film. So I actually have examples. I'm thinking like a new staging scene, but that feels like it could be part of the film. Love it. Okay. So then we're going to go head to head with the same prompt. How about you give me a few prompts from Nana Banana and I'll run it here and I'll give you some prompts. You run it there. Okay. So I wanted to test what the stock vanilla Z image would look like with the same prompt just to know that, you know, the Laura is really doing its thing.

16:02so I'm just going to take this same prompt here and then I'm going to go to free pick because free pick has Zimage for free if you have the right subscription level so this way I'm not spending any money on inference so again this is what it looks like without Allura and this is with Allura so there's definitely a little bit it's grainier, the vegetation is different the color of the sand is different yeah the other one feels like a default interpretation when you say desert. It's got sand dunes. That's the first thing when you think desert. It's like the Sahara Desert, right? Versus this being Borrego Springs in California.

16:41Which is, yeah, more brushier, rockier kind of desert. Just want to real quickly show you this one handle. You can absolutely go overboard with the Laura. So if I really intensify the Laura weight here and then just run it, you'll see that it actually starts to break. The Lauras are not silver bullets. it's not going to solve your problems. You have to know when to use it. Creative interpretation of one battle after another. Yes, this is the sequel. Okay, so I was thinking it was going to start throwing images of like Sean Penn or DiCaprio in there. Like if you cranked it up too much. I didn't train it specifically on people.

17:20So I wanted to get... Oh, so one of your images had people? It was just... I pulled out, I think I had one or two Leonardo DiCaprios in there just because he was in the frame. But yeah, I didn't, because in order for me to recreate Leonardo really well, I would need like 15, 20 images of him specifically. And that would be its own Laura that I would have to add to this Laura. Not necessarily to replicate him, but more like the way he's lit. And if you had, if you wanted to make a shot of another person, but in the same style of how Mike lit those people. Okay, that's a good test to try out. We should definitely generate some of that.

18:03Yeah. But I'm going to hand it over to you. Okay, so my, let me just kind of show the high level of the workflow. So I went the reference image route with Nano Banana Pro. And so I'm doing this in free pick spaces. So I didn't need as many images as Addy needed to train the Laura. So I kind of just grabbed four images from the same style. I grabbed a mix of images of locations and people, if I felt like the people kind of represented the lighting. Sometimes it would go off the rails and just throw Leonardo in there. Actually, there was one that made a really funky, beautiful montage image of one battle after another.

18:41But I'll load that up later. So I took these images. But before that, I just had Claude write a system prompt. Basically, I explained what I wanted. I wanted it to create a prompt that I could append to a text to image prompt where the nano banana would get reference images and it was supposed to replicate the style and cinematography and color and composition, but not use or recreate the actual reference images. So I had to write up this extra prompt that I would add to the after my deep input prompt of like what I actually wanted to generate. And the cool thing with spaces is you can kind of create multiple text boxes.

19:22Oh, wow, that's really cool. Yeah, so you can create multiple text boxes and you can link it or drag them into the image generator node and then you can sort of call up both prompts so basically you could just kind of it just makes it a little easier to kind of keep things separate so you could just change one prompt with the shot that you want yeah it's more modular sure yeah instead of instead of changing everything and then you just call them up here in the order that you want so like it's probably just combining it at the end anyway yeah it's like a it's a concoctinate like concactinator concactinate but um you don't have to really think about it so yeah i'll these were the outputs that i had and so the according to addy's text which had the parameters of our of our challenge one was desert border town wide shot and so this was using the prompt my simple prompt that just says nighttime exterior of a small town near Texas border, ultra-wide, and there's four input images as reference, and this is what we got, which I'd say that's pretty good.

20:22It got the lighting, it got the vibe. This feels like if we were in a border town next to the detention center, it feels plausible like it's in the same universe. Yes. The lighting, the color palette, the lighting, spot on. The actual assets in there, this feels more like a Wild West border town than modern era border town, but then I'm just nitpicking. Yeah, well, let's try it again. Like, it's a contemporary, yeah. Small modern town, Jared. Let's just go a little sharper, too. I'll go 2K. Ooh, okay. All right, dude. All right. Let's take it easy. It's not going to be as fast as the image, so we'll come back to that.

20:58Joey's really taking this serious. Okay, what was our second shot? Was desert exterior at noon. Wide shot of run-down cars on an empty stretch of road. So sort of like this post-apocalyptic feel. And so these were the images I gave it. And again, this one I gave a mix, again, of some close-ups of people and then mostly the desert wide shots. Yeah, the Dashaun Pan shot really helps set the sunlight to the top of your head, which obviously has big implications on shadows and stuff. And then we got some of these are kind of good examples of the color palette that they had in the desert. Yeah, yeah, yeah.

21:34And then this is the output. Yeah, that's not bad either. I mean, you got the noon, high sun. I think it's a little bit too vegetative too much vegetation I think in this realm too and same with yours as well these are pretty crappy prompts so something that was much more specific in what we were looking for would improve the quality of the prompts but that's not bad at all that's totally usable at a pre-vis pre-production level to convey an idea and to build off also what you could do in Nano Banana and sort of how the interface in FreePick is. Like, I have these four input reference images, and their sole purpose of this is just as that style guide reference.

22:24But I could add additional images, like if I had a specific location that I know we're going to film at, or if I had our actor that we might use, I could add them as an input, and I I could just tag them as well into the prompt to give the prompt specific directions like, you know, close up shot of tag the person's image in a desert or in whatever location we want. Sure. And try to get more specific outputs that I'm looking for. Awesome. Awesome. Very cool. And then the last one was man in distance wearing checkered red shirt entering a pawn shop in a small city. And this one was using the Sacramento kind of more neutral looking government building.

23:06Like an 80s small town. Yeah, or small like mid-city 80s office building look. Yep, yep. And I mean, that one's probably the least distinct style, but. Man, the Nanobin on Nails text. That neon sign is so spot on. I know, the sharpness of this is. Yeah, like that's where Xium had struggled a little bit, which I'll show you. Yeah. yeah uh so yeah this is the output from that dude good job it feels successful let's see if this one worked oh palm tree no bueno i'm just kidding it's my showdown energy this feels this feels more like uh this feels like the town from eddington a little bit but you you have the border wall there that's pretty cool it did put the board yeah i mean again these were not the best prompts we don't really yeah good like that hazy grainy halated light stuff like that stuff comes to so well yeah this is one i was trying to push because it was like i you know i didn't want it to i wanted to see what it would happen to if i wasn't trying to replicate something that already feels like it's from the source images so this one was a prompt that was like a futuristic car driving downtown and that was the base prompt but it really this one also felt like it kept the same style and lighting and vibe of the source images with a completely different scenario and location yeah yeah i think the color palette they nail maybe lacking some magenta yeah like the streets aren't perfectly like dry like i love the wet streets with a little bit of dirt and yeah i got the halation that we see in the halation is so clear yeah like the vista vision stuff is so clear right this was the one that went off the rails that was like which like could just be a good that's a poster montage yeah yeah dude that's like the water wall the chain link fence and everything that's awesome so yeah that was my that was my reference uh my reference attempt awesome so let me show you the other two scenarios what i'll do is actually didn't i actually did generate some but um we'll do it live so here we go that's it we'll do it live screen so i was really fascinated by the nighttime scenario to me like that that was like a big moment in the film so all the revolution stuff really happens at night so like that that was the part that i was that really struck a chord to me emotionally as i was watching the film i was like you know every revolution is done by people and people do like one thing at a time and they chip away and over time that little stuff adds to like a big thing happening right and so in along that same creative thought it's like okay if what if two people are just walking around at night and they're planning some operation out so two people in black hoodies walking down the street at night deeming dramatic lighting with splashes of color i'm gonna just turn down the weight to like i don't know 1.2 let's go ahead and generate so this is what we get uh again uh i think i should specify like silhouette off the bat the lighting it feels like a bit too neutral lighting like i don't have i i was able to get a really good output which i'll show you i saved it how many tries did it take you to get that good output uh two yeah okay all right that's good yeah this is the look that i wanted i wanted the silhouette you know just lighting coming from the front and you could see that the the sodium lighting is is very orange and yellow like that that was the look and feel of the film and because the camera was set to that white color uh white balance to sodium then the other lighting caused significant splashing of color as you could see there um again the the bokeh is white we are talking about film stock sir film stock is correct me if i'm wrong but the white balance is baked into the film stock right the film stock has a set color temperature that's it yes yeah okay damn you're schooling me hard okay uh yeah my uh my film school knowledge that uh yeah coming back it's coming in handy on the showdown yeah all right so this is the laura output and again i could i could influence the weight a little bit so let's just go up to like 1.5 and see how much more of the movie i can kind of squeeze out of the toothpaste tube yeah again it's it's kind of starting to get lost here but like it's starting to get more and more colorful uh which is really nice um so i'm gonna just dial it back to like 1.4 two people in black hoodies walking down a street at night silhouette lighting dimly lit dramatic lighting with splashes of color and then just to kind of see what the laura is doing i'm gonna just run a clean example without a laura here on free pick i mean i will say this is the this is the nicest thing the nice thing about c image is the speed and this is and that's important for the iteration too because if we're in this phase where it's just like we want a bunch of ideas and like brainstorm fast you know the 30 seconds for nada banana gets to be a pain in the ass after a while yeah if especially if you're generating like 100 images in an hour right like you go and yeah directors usually don't have patience right if you're sitting next to fabbro or somebody like that you better be fast so this is plain z image without laura uh-huh this is z image with the laura so there's there's significant influence on the lighting on the color palette for sure I'm squinting.

28:19You're squinting. Come on, man. Look how saturated that yellow sodium lighting is there. You got some of the neon stuff coming through over there. And then look how boring this looks. This is kind of generic. Oh, I think it was too. Okay. Yeah. I think you had two different things flipping up. Yes. This has not a lighting vibe yet. The Laura definitely has the more color palette. This is the one with the Laura. It's just more dramatic. Like it just feels more like a movie where this feels more like a YouTube video. Okay. Over to you. All right. So I ran the same prompt with the same exact setup and here it is.

28:57Yeah. This has got some border town vibes to it for sure. Like from your last generation. To me, this feels more like in the world of the film. Like we have much more of the halation, the streetlights, the color palette. Yeah. I think you win on this round here. Okay. Yeah. This is much closer. I think I won the desert round and you are perhaps winning the nighttime. Give me a specific, give me a desert. Give me a desert prompt. Okay, just a desert highway road. What was your highway? Did you have a highway desert image like this? I had a really good image because I used the white Dodge Charger.

29:34See if you can copy this prompt on my side here. And that was one of them. This was another one. Yeah, that's not like, let me show you here. Oh, because you think there's too much vegetation in these? Not just that, but it just lacks the menacing look of that scene. It's hard to describe it, but here, let me regenerate it here. See, this is where, with the reference images, if you prompt it for a person without describing them and you have reference images with people, they'll creep in. So I just asked for a fruit stand, but it put champagne in the fruit stand. See, that would be so hard to do with Z Image and Laura's.

30:14Nano Banana does it effortlessly. Yeah, that's impressive. Yeah, the text of sharpness. Yeah, it's good. And if you do this at an actual pitch meeting, you're going to get laughed out of the room. Yeah, this was the first one that it came out with. But yeah, I see what you're saying, because there's no cactuses in the reference stuff. And you're going into apocalyptic. not not i mean you probably seen the movie more times than i have i might have prompted it for that okay can you take the apocalyptic stuff out like we don't need record we need regular cars that still run just old cars oh okay run down cars okay get rid of rundown okay so just say old cars or 80s cars because they use a lot of 80s cars yeah so like are they driving around the down the side of the road what was your you know what the scene where leonardo whistles at the people yeah by the fruit stand yeah yeah can we try to recreate that why don't you come up with a high angle wide shot of a rundown 80s car at the intersection of a highway in the desert by a fruit stand and again this is the downside of that i will i will say my you know when you were talking about lauras before i was thinking the the pro of lauras this is a different use than i was thinking you're going to go with like i feel like they are really good if we were trying to do a complete style transform like turn these photos into an illustration or a cutout or like some other distinct style yeah that is the core like bread and butter of what lauras do yes yeah because then you get consistent every time usually if you trained it well i mean if you build if you build a good enough style transfer model image to image model yeah this is this is starting to feel like the movie right here yeah for sure i would be nitpicking if i had notes here but yeah you nailed it so let me take that same that same prompt a high angle wide shot of a rundown 80s car at the intersection of a highway in the desert by a fruit stand so no this is uh vanilla this is vanilla uh z image yeah this vanilla one did a pretty good shot exactly dude yeah don't sleep on z image okay so here it is a high angle white shot of a rundown oh my god another knock out but come on dude look at the lighting look at the color of the sand look at the vegetation like it didn't even get the highway right it's like a passing zone in the highway Alibaba, you're letting me down with Zimage.

32:48Okay, so I'm going to take that same prompt and just throw it into plain Zimage. Zimage vanilla has made a car that does not exist. Well, it's struggling with the intersection part because the intersection is structurally like a cross, like perpendicular lines, and that is influencing the car. It has made the car intersect itself. Yes, that's not good, man. That's not good. Do you know anybody at Zimage I could talk to? I don't. You win this time. If you have done anything we have done and had success with either of them, let us know in the comments. Because this is something we keep going back and forth.