68 | AI As Your Creative Co-Pilot: Transforming Creative Ideas Into Reality with Sachin Kamath

28:02So let me just walk you through it. Now there are multiple ways and tools to approach this And here is the simplest way right within GPT-4. So I always use GPT-4 because it is powered with DALI. And all you need to do is in the next prompt, ask it to generate a sketch for that particular frame. You have already received these breakdown, like panel one, panel two, panel three from your previous step. Now just ask in the next step, GPT-4, to create a sketch out of that particular panel. Yeah. And this is exactly what I've done over here. As you see that DALI generated this beautiful sketch for me, and it is very good at listening to my image prompts.

28:48In fact, it is the best at prompt adherence, even better than mid-journey basics in most of the cases. I want to pause you for just one second to talk a little bit about the two biggest differences between DALI and MeJourney, which are the two tools I use the most as well. I use a bunch of others. I use stable diffusion a little bit. I use ideogram every time I need some text, but mostly I use MeJourney and DALI, which is part of ChachiPT4. And there's two very distinct benefits, one on each side. The benefit of MeJourney is that if you're looking for high-end realistic outcomes, nothing beats me journey.

29:29However, it's harder to get those results. If you look, the biggest benefit of ChatGPT is that it's a part of a chat. It understands the context of what you're trying to do. In this case, it's been a part of the whole conversation, right? Beginning to end. Like this is what we're trying to do. Let's ideate. Let's come up with The idea is it understands what we're trying to do. And hence, the chances that the image is going to represent what you want it to represent is in less steps and less pain is significantly higher. So I will say that if you're looking for the ultimate image, mid-journey is the way to go.

30:12It will be more work. If you're looking for something that's going to be good enough, it's going to save you a lot of time. and some, in many cases, that's good enough is guess what? Good enough. Then you don't need to go further than this. So there's a huge benefit in doing what Sachin just described, which is to, I'm having this chat. And then from within this chat, I'm starting to generate the images because it understands what is it trying to do. Yeah, absolutely. By the way, Midjourney is my favorite image generator that is out there. Every morning I wake up, if I have any idea, the first thing I do is I go to Midjourney and I type it out and I see what kind of visuals I get.

30:48Now, I'm showing GPT-4 over here because this is the easiest way, the simplest way to get it done right within the storyboarding what we are talking about. Because we have the concept right there and you don't need to prompt engineer it to creating very high quality images. Because there is a kind of learning curve involved to taking it from chat GPT to then prompting it in mid-journey. But by the way, mid-journey, you can do a whole lot of techniques. It's not just prompting and creating images. There is a ton of things that you can do. In fact, I deep dive into MidJourney a whole lot in the bootcamp itself.

31:26But also, I love using stable diffusion, comfy UI, all the open source AI models. And there are a lot of things that you cannot do with MidJourney. And I will show you some of the examples. Take, for example, you want to get that exact pose of the person with the exact face all in the same image and create 100 different variations. you cannot do that with mid-journey you cannot create the exact consistent character with mid-journey right there is a bit of trick and kind of workflow which probably takes a lot of time and will not give you 100 of the result every time but and but it's great for high quality photorealistic images and right over here what is the sketch a beautiful sketch which dali generated for me right within GPT-4.

32:11I just told it, this is panel one, right? Create a sketch for me. Create a detailed

32:20storyboard style black and white outline sketch for this image. Usually, the storyboard images are sketches. That's good enough. And why did it generate just a black and white sketch? It is exactly because Dali is very good at listening to our prompts, but it's not very good with aesthetics and photorealism. And this is exactly where we need to go into the next step, what I'm going to show you. So step four is turning these sketches to final illustrations and quickly iterating to land on your final concept visuals. Now in theory, you can just as well at this particular step, stop, right? You have the sketch and you have the storyboard description, you have the story to tell you can stop but we can also go one step further and get those aesthetics in the final image and turn them into finished visuals and this is exactly what we will quickly take a look at and this is what i'm talking about uh what on the screen here right now is on the left side you have the sketch which we generated with gpt4 and on the right side you have the exact same sketch turned into a photorealistic visual which is not possible with mid-journey it's impossible with mid-journey even if you use image references even if you use image weights it's not going to listen to you it's not going to stick to that particular outlines to that particular reference image that i have used and this is exactly why we need to turn to another tool which is you can use the open source models like ControlNet and StableDiffusion.

33:54In fact, I use it a whole lot with my own ComfyUI or Focus, as I call it. Focus Automatic 11.11 and ComfyUI. These are the three interfaces for using StableDiffusion on your laptop, but there is an easier solution that is out there. So there is this web app called Leonardo AI, which is built on top of this open source technology and helps me get these images exactly based on my storyboard sketches. And it is powered with stable diffusion and I can use the control extension all within it. So just like you see over here, I have given it my sketch as a reference image. I want to pause you just for one second for the people again who are listening.

34:36What we're looking at right now is Leonardo.ai. It's a great tool. It's free. It's a really powerful image generation and manipulation tool that everybody has access to. And what Sachin has done, he's using that. The first thing he did is he uploaded the sketch that was created by Dal E inside ChatGPT at the previous step as the initial starting point for the process that we're going to do in Leonardo. Yep, exactly. By the way, I think Leonardo is an amazing tool. It's powered with stable diffusion. and as you see over here you can select different fine-tuned models this is where it's super powerful you can pick from different kind of styles and aesthetics right there and it's free right you get the credits but i'm not sure if you will be able to build this workflow because you need to use this extension called control net and i'm using over here the edge to image model within that and the image guidance which is you have text to image and then you have image to image.

35:38And here I'm using the image to image along with the control net extension. I think you have to pay for this, but I think you can give it a try, right? As you log into Leonardo AI, you can select this and over here, I have input my reference image and I've written down my prompt, which again, you can engineer this prompt within Leonardo by using the prompt generation guidance. Everything can be done within Leonardo AI. So the prompt... I want to pause you just for one second. there is a person on LinkedIn Live that is asking, can we use Playground to achieve the same effect in what you're doing in your narrative right now?

36:12And I don't know if you know what he means. I think if you're talking about Playground AI powered by Stability AI, I'm not sure if it's powered with Stability AI. It might be, but you need to really look for if they have this image to image along with the control net extension. If they don't, you'll probably not be able to do this. It's the same thing as like when you use MidJourney, I can still use this as a reference image, the sketch, but it will not give me the result like exactly the sketches. Yeah, it's going to be its own interpretation versus a high resolution image replica of the sketch.

36:51Yeah, but you can also use open source models on Replicate. There is a control net model. You can do this within it. I found Leonardo is one of the best ways for everybody to access it and to start doing this. All right, here I just use my reference sketch and like I just showed you with the extension and there you go. I have turned the sketch into a photorealistic image within Leonardo AI. And that's the power of using these AI tools and thinking all of this as an AI workflow, right? I think the real power is building AI workflows. Prompting is a small piece of the puzzle, really. This is what I think because prompting is going to become more and more natural natural language power just like Dali is very good at it mid-journey will get there very soon and prompting even though very important right now is going to be a small piece of the puzzle and the real power is in building AI workflows for your particular use cases like model photo shoot right where you want to have particular pose particular you have you want that particular consistent character in every single image you want that specific pose or you want to change the pose very easily and it's listening to your prompts that's what you're that's what you're looking for and you can build these kind of workflows and you can follow these steps for each of these panels each of these frames of your storyboard each of these panels and there you go you have these three panel storyboard ready right you have the script you have each of those panels what it should be you have the sketches for the panels and you have finally the images the photorealistic images.

38:26So I want to pause you for a second. First of all, I think it's brilliant. I think it's absolutely amazing. For those of you who don't see the screen, we have three columns on the page. On the left, we have the detailed description. In the middle, we have the 2D black and white sketch. And on the right, we have a high-end rendering realistic view of the same exact sketch. It looks as if it was a coloring book that somebody who's really good at coloring colored it to make it fully realistic. Like it's literally the same image, just in 3D reflections, lighting and everything. Can you go back for a second for Leonardo?

39:01Because I want to go a step more detailed into the actual step in Leonardo, because I think it's the most technical one we've done so far. So the steps are this, right? You said you upload the image. That's easy, right? You download the image from Dali, you upload the image. You said you write a prompt. What does the prompt describe? Can you read the prompt that you use to create, go from the 2D sketch into the 3D beautiful image? Yeah. So here as well is where the prompt crafting skills come into picture, right? Because you can change this. You need a prompt which will define your final vision as well.

39:39Just a reference image is not enough. If you really want to get that final aesthetic and have that aesthetic consistent across the different frames you want to have to engineer your prompt so my prompt here is the pearl white tesla solar car so i'm defining the color right and i have the subject in there with its sleek and aerodynamic form cuts through the stillness of the deserted highway at sunrise a stunning display of harmony between technology and nature first light of dawn sun rising in the background so i have pretty much described the subject what the subject is and what it's looking and I've described what the surrounding is right and then I'm hoping Leonardo fills in some of the creative gaps for me or you can get a little more specific in there but also keep in mind when you when your prompt becomes very big it starts to ignore some of those details but of course we can deep dive into the prompt crafting funnel the technique that I work with is is definitely a lot more than this and it's very important that you get your prompts to get your final aesthetics to have them consistent across frames.

40:48So I want to ask you specifically about this because when I do stuff like that in Me Journey, I use the describe function. So those of you don't know, in Me Journey, you can upload an image and ask Me Journey to describe it. So instead of doing forward slash imagine, you can do forward slash describe. And then it gives you three prompts that it thinks will describe the image. And then what I do when I want to create images that are similar to other images, I pick and choose components from the three variations of descriptions that Me Journey comes up with on its own. And I craft something that I think is the best description combining all these components together with the weights that I think or the order that I think that will get me closest to what I'm looking for.

41:32Is this a similar process? Is there a mechanism you use to come up with this prompt? Is it just based on the prompt you had in Dali or is it just your experience that teaches you how to do these prompts? I'm just trying to look for the basic ingredients when I'm prompting. Am I describing my subject? Can the machine know what I'm talking about? And I'm trying to make it easier that way, but I'm also trying to leave some gaps for some of the creativity to flow into it, right? Because I don't want to control every single thing. Finally, this is a storyboard and it's a pre-production step i can get much more detailed in the next steps right i am only looking to get those aesthetics that can tell a story from my vision my from my point of view so i have things here take for example you have your subject then you have your color themes anything that describes the mood and feel of the overall scene that's what i want to have over here anything that describes the background like I envision it.

42:32I want to have it over here. And I think having that ambience and that mood along with lighting will just take it to the next level. And I think those ingredients are something that's good enough to start your journey, to start your prompting. And of course, then you can do image manipulation and a whole lot of things with the AI workflows as well. Fantastic. Now I think it's a lot clearer. So you're basically trying to describe the vision you have in your head of what this would look like if this was a realistic image, but leaving enough room for it to come up with different ideas. Yeah, exactly.

43:10I think when you're using AI image generators, any AI tool, even for copywriting, you have to be able to let go of that control, that ultimate control. You will find many different ways, right? You can build AI workflows. In fact, if I need, like I said, if I want a specific pose of a person, if I want that specific outfit, if I want the specific face of the person, I will not work with some of the tools. I will not work with MidJourney because I know it will not work for me. I will use the open source models and build my own workflows to get that done. So I think it's key to realize, and I think that's the skill that everyone needs to learn over the next two to three years, which is really curation of inputs and outputs because the ai models that you choose is going to depend on what inputs do you have and what output you're looking for and that's when you will choose what ai model or what ai workflow to build which you can then use multiple times for very similar projects and i think that's this that's that that's the secret to scaling any part of your creative process and that's that's what i look into that's that's what i'm passionate about and I always look for these AI workflows that can really fill a gap between a creative process.

44:28Fantastic. There's a question on LinkedIn Live. Can you adjust the aspect ratios of the images? I assume that applies both for the sketch created on Dali as well as, okay, now I'm in step four in Leonardo. Can you still change the aspect ratio? Yeah, I think you can change the aspect ratio. like you can make it 16 by 9 in DALI you can tell it here is this is the aspect ratio and it'll do it for you and then you can take that image in here and as you see over here there is also how do I call this there's also a button for aspect ratio and you can choose different dimensions one tip is if you want to use AI image generators like especially stable diffusion your output needs to have the same aspect ratio as your input for image to image otherwise you might find that there might be discrepancies, you will not get that high quality risk.

45:22All right, good. So let's move to the next step. And we just saw this one. And yeah, we have the storyboard from here. So with this AI workflow, now you can clearly pitch and sell your idea or your vision to your client. So you don't have to use words like imagine the scene or that person. You can just show them these images and sell them the creative idea or the creative campaign that you want to design for them. And you can do this all within 24 hours and take that brief and tell them, hey, this is the idea that you have. Do we sign off? Do we have an approval? And of course, there is a ton of things that you can do afterwards, right?

46:03I'm only talking about pre-production where your images are good enough for that particular step. And then a whole lot of AI workflows that you can also build afterwards. All right, I would love to know if you, do you all find this workflow useful? And yeah, if you do, I would love to hear your thoughts in the comments or any questions you have, you can shoot out a message for me. And yeah, I would also like to show whatever we looked over here is just scratching the surface. I would like to give you some more ideas on what is really possible when we deep dive into the AI workflows, right? We are always looking to see how we can push the creative boundaries.

46:42What if we can add animation? And here is what it looks like.

46:58so again those of you who don't see the screen on three panels again on the left there is a kind of like a pencil sketch of a high-end performance futuristic race car in the middle there is a variation of it that is like colored, but still looks like an image. And on the right, there's an actual video of the race car driving. And I assume you've done this either with Pika or with runway, right? Yeah. This is with Pika Labs. And you can do this with runway as well, or stable video diffusion. So it's like always thinking what you can do as the next step, right? Instead of just showing the images, can you turn it into an animation?

47:39And there are a lot of ways to control the animation as well, right? You can do camera controls. You can tell your story in a little bit, take it one step further, tell it in a more compelling manner, and videos always work. So it's always about figuring what really works for you as part of your creative process. And that's one thing I wanted to show. Take, for example, if you want to generate some wonderful, unique concepts for your campaigns, that is possible as well. This is when you have to push the boundaries and start using the right tools because for example mid journey will not give you these kind of results you you'll probably have to work really hard and even in the end you might not be able to create these kind of images this was created with daddy by the way right so you need to really understand what are the tools that work for you what is your input what is your output and what would you like to use as to go through that process now here is an example where you can take control of your image generation and tell compelling stories, right?

48:40As part of your storyboard, what if one of the panels is not working for you and you're trying to control and trying to change things? You can, of course, craft your prompts and do a lot of image manipulation techniques to really get control over your images while you tell your stories. What if you want to explore some specific style and aesthetics and create on-brand images, Right. This is what is super useful when you have a specific mood board and you want to capture those aesthetics and settings and you want to apply that for multiple images that you want to create for that particular campaign.

49:16So, again, to pause here and explain to people what we're seeing, there's a panel with three images on the image on the left has a close up of an astronaut in his flight suit. In the middle, there's a girl in a dress in a field. And the combination of these, so the subject is the astronaut, the style is the girl in the field with blue skies. And the combination of this has an astronaut placed in a field with blue skies around it, which is like combines the subject and the style and settings of the two images into one really unique and interesting and innovative thing, because you don't expect to see an astronaut in the middle of a field.

49:55Yeah, exactly. So if you have some unique creative concept and you have certain inspiration to start with, you can get those done. It's only about figuring out the right process for you. Of course, you can explore, as you already know, like different compositions, camera angles, lighting, all of this start adding to the look and feel of your images to get the fine logboard. And of course, you can also create consistent characters. I want to close this with, I designed this consistent character GPT, and this is top five on the GPT Dali store. I'm super grateful and happy for this one because a lot of people were reaching out to me and telling me they cannot create consistent characters.

50:38Take, for example, for the comic books, right? They want to create comic books, tell a story, and they cannot create consistent characters within mid-journey. and this is exactly why I designed this consistent character gpt and I would love for you to give it a try it's on the if you just go on the gpt store and look for just search for consistent character gpt you will find it and I would love to hear what is your experience with it what do you come up with it all right yeah thank you and if you have any questions you can reach out to me or you can reach out to my co-founder my partner diana's devil you can find me such in comment on on linkedin as well I'm very active on LinkedIn.

51:16And yeah, last thing, we have a creative AI bootcamp coming up, which starts on March 11th. And if you want to deep dive into this AI-powered creative world, and you think this is the right upscale for you and for your work and your projects or your creative agency, then this is the right fit for you as well. So you can reach out to me on LinkedIn if you have any questions about this. Isar, thank you so much. And I really enjoyed walking through this entire workflow. And yeah, what was really happening. Yeah, this was fantastic, Sashin. Before we end, we have a few minutes. There's one more question from the audience.

51:51And I'll just read it word to word. It's from Manalo Torres. He's asking, what would you tell all the people who are not able to run stable diffusion on their computers because they don't have enough GPU power? Is there an alternative to SD plus ControlNet or SD plus ConfUI? SD, for those who don't know, is stable diffusion. So SD and ConfiUI, like the example of Leonardo powered with ControlNet. So are there other options for online capabilities? Yes. In fact, a lot of times when I have to run very heavy, when I have to do quick experiments, I would run stable diffusion on the cloud. And you can find multiple ways to do that.

52:31For example, you can rent a GPU on the cloud and go through the Git installation process. You can take those repositories, upload it, and that's a quick process when you're running it on the cloud. So I do run it on the cloud where I can rent out a GPU. Or you can look for other platforms. I think there are a couple of other stable diffusion, Automatic 11.11 especially, to run stable diffusion. You can, I think it's called run diffusion or something like that. Do look at it, and you don't have to go through any installation process on your computer, and it will not use your GPU performance. is going to use the GPU on the cloud.

53:09Of course, you have to pay it, but the amount you pay is very low. I think you'll have to pay less than a dollar, like 50 cents per hour when you're renting out on the cloud server. Yeah, there are ways to do this and I use that often as well. Fantastic. Another question is, how often do you use weird or chaos parameters when you create these things? I assume earlier in the process. Yeah. Yeah, frankly, I don't use them so much because there are so many ways that you can reach the reached and result. Weird and chaos parameters, they are weird, right? It's very hard to say that there is a specific use case for it.

53:50Take, for example, I think weird parameter will give you a little bit more of realism. Like you're taken, if you want a person's photograph and it makes it look like not synthetic like most of the images that you generate on mid-journey look like highly amazing highly photoshopped highly synthetic and you want to take out that look maybe that's where it's helpful but i would probably use an ai enhancer to get that done um chaos if i need more variations i would probably do that but not so much because i would rather want to keep something consistent and then i'm very heavily into image manipulation of how i can go from point A to the end point that I'm looking for, rather than just getting a lot of different variations which I can use.

54:37I usually need one thing, and I want that level of control, but I also want a certain level of creative spark or creative output. And I think there's a fine balance to strike. But yeah, so frankly, I don't use those parameters, but I know that they are helpful in some ways. So I use chaos pretty frequently, just like you said, especially when I'm ideating and I'm trying to see, oh, what would be cool? Then I use chaos a lot and then I get a bigger variety and then I would upscale one image and then use the chaos again to get variations. Like it just helps me more in the creative brainstorming process than in the generation of the outcome process.

55:17Once you're done with that, then I try to be very specific and then it obviously hurts you. Sachin, this was fantastic. Like it's a very well thought after process. You obviously, as I said in the beginning, the engineering side of you is working very well with your creative side to create really magical process here. Those of you who haven't watched this, we will put the link to this in the show notes, the link to the YouTube channel as well. So you can go and see everything that we're talking about because this one was beyond seeing the process. There's a lot of visual outcome you probably want to watch.

55:50So you can go and do that as well. and as Sachin mentioned, go check him out on LinkedIn. That's how we connected. Go check out if this is what you do for a living or if this is something you aspire to do for a living, absolutely go check out his course. He's the best out there in these kinds of things and really developing AI infused processes for creative outcomes. And so I really appreciate you taking the time. I really appreciate you sharing with us with your knowledge and thank you so much. Thank you so much, Yisar. And yeah, it was a pleasure and yeah, we'll connect again very soon. Thank you everyone.