The whole studio is one guy | Jonathan Brazeau

0:00I had generated 40 ,000 images in Stable Diffusion at some point. And the reason was I was trying to make my own RPG and I had designed the book. You know, my background is in product design. But when I laid out the book, I realized I needed 45 pieces of art and I didn't have it. And so I turned to Stable Diffusion. This was kind of like when stuff couldn't do faces or hands that well. I could draw faces and hands. I would draw over those and make the characters and stuff. And it was really fun. It's just like an idea generator. True art, right, creativity is improbable. It's statistically not probable.

0:34Parth Patil:The Token Grantee Program gives$1 ,000 a week in tokens to high potential creators already deep in AI across film, gaming, comics, print, and digital art. There are no tool restrictions. The freedom to choose is the point. Grantees also get access to my own custom fleet of agents and an ongoing collaboration with me. the goal to close the gap between an idea and the world. So Jonathan, thank you for joining us on Read Riffs. I'm part of this token grant program, which I love as meeting people who are enormously creative or leading the edge of what kinds of things to do. And so with that part, why don't you kick us off with an introduction of Jonathan and how you added him to our fellowship of token grantees.

1:26The fellowship.

1:28Parth Patil:Yes. So Jonathan, super glad to have you both in the program and on today to share your creative wisdom. I met Jonathan almost, I want to say, maybe around eight or nine, maybe 10 years ago now. We were working at a startup in West Los Angeles working. And Jonathan was a designer at the startup. But more importantly, we started, a couple of us at this company, we started playing Dungeons and Dragons together after work. once a week and we would take a meeting room and we would play dnd and jonathan as being a design superstar and that's why i said fellowship for yes yeah exactly exactly into the dungeon right he had his little ipad and he would be drawing our characters throughout the adventures as they changed and like the experience and the different characters that we had like my character was a purple dragonborn and and we just had some such and we played for years actually it was a one of the most interesting campaigns and getting to know his like creative side and and the world building behind him.

2:26Parth Patil:It was just so much more than just a designer. So really excited to have you on to talk about your creative vision, your ambition, and what you think is possible. Now we have some sudden some of the most interesting tools available. Yeah, I mean, no, thank you. I mean, it was super fun. I mean, honestly, I would recommend D &D to anybody. It made me a better, almost better at everything in life, I feel like. So yeah, it was it was very formative for me. So I want to talk about how you first got into playing with AI and what your stance on it was from your perspective. And I guess who you were before AI and what your lens on your creative aspirations were and maybe how that has changed in the last couple of years since the generative AI kind of moment.

3:14Happened upon AI by accident. I actually definitely remember when it was. it was like November 2019 um there was I had just started running my own Dungeons and Dragons campaign um so I was transitioning from a player to uh you know Dungeon Master and I needed like backdrops I needed um landscapes for the game and I was you know we had sessions every week and I was I'm not that fast of an artist um I you know it takes me like 10 hours to do like one piece so I was struggling and somehow I found this thing called Art Breeder and Art Breeder would, I would say, would let you smash two images together.

3:58I know it sounds weird, but you would pick two images and it would combine them to make something new. And you had all these sliders that you can adjust the parameters and weights of things. And I didn't know this was like machine learning or AI at the time. I just thought it was a fun tool and I was making all these backgrounds and, you know, they They were low res. They were very abstract because, you know, this is like early days, you know, Imsgen. And but the cool thing was I could paint over them. And so I would paint castles and bridges and things into the in procreate over this image. And it made me it allowed me to make, you know, these landscapes of this world that were super fast.

4:40And the players loved it. And they were kind of amazed by this tool. and we had no idea what we were doing um but yeah that was my first foray into it was like yeah december november 2019 and then of course things progressed right um we got mid-journey at some point and then i was heavily into um stable diffusion when it first came out like from 2.0 to like xtxl um and i this is gonna sound wild but i had generated like 40 000 images in the stable diffusion at some point. Wow. Yeah. And the reason was I was trying to make my own RPG and I had designed the book. My background is in product design, but I have a design degree.

5:25And so I had designed the book and when I laid out the book, I realized I needed like 45 pieces of art and I didn't have it. And so I turned to stable diffusion and this was kind of like when stuff couldn't do faces or hands that well um and so you know but i could draw faces and hands so i would draw over those and make the characters and stuff and it was really fun and it was like working with another artist it was like working with someone who had um these wild ideas and and so i would i would draw over those and i end up making this whole book with um art from stable diffusion that I had drawn over.

6:06And so that was kind of like how it started to evolve. And of course now there's many, many tools at our disposal, not just image gen, you know, so. By the way, part of the whole D &D background, which Jonathan, you may not know, like I started playing D &D when I was nine. So it was a long time ago. But it's not just obviously the images and the world building and the creativity, all of which is super important, but also of course narrative story, interactiveness, and other kinds of things. And so kind of say a little bit more about how the kind of the choose your own adventure or make your own adventure or make your own world to D &D is not just kind of all the images, mid-journey, et cetera, but also the kind of narrative story, the kind of land of imagination.

6:58Amplify that a little bit too, because I think, by the way, among other things, it'll go with a hat. Yeah, so character building, world building, stuff like that. I think for me, it all starts with an idea. And some people are very creative and they have all these creative skills. And you might have these ideas in your head. What the AI can help you do is like put them together. You just have to know where to point it, right? And so I would use it as like a tool to kind of like, hey, I need, give me 100 names for a magic staff that does X, right? And then you might tell it or, you know, make them esoteric, right?

7:43Or relate them to this poem and get some interesting ideas. And out of those 100 names, none of them might be perfect, but it's enough for you to pull maybe two names together. And now that's the best name for the staff, right? um that's kind of how i would use it um as i'm like building worlds or narratives or things like that um as like almost like a i don't know it's like it's like you're sitting in a room with someone and you're bouncing ideas off of them um that's kind of how i use it yeah so let's go um because this is great and i think the dnd stuff at the beginning but let's now move this to kind of a film or creative project it's almost like you know the the the The film is the dungeon we're going into.

8:28So, you know, what parts of this feel, you know, kind of genuinely creative? Which parts feel more mechanical? You know, and kind of like what are the different jobs around this? I didn't make up this theory, but a good friend of mine did. He's also a genius, and he is deeply into AI. And we were having a conversation about, this was about building products, but about UI UX and engineering. and and but also creative fields like making music making um movies and stuff and um the one thing he said well there are cogs and sparks is what he was talking about there's cogs and sparks and he was saying he's his background is engineering and QA and he was like I think you know UI UX and music making and these creative fields are sparks and that's where the power is That's where the human comes in and that's where the AI can't really go that far.

9:24And the COG jobs are more of like logic-based things. So for my project, I have the vision and I'm kind of setting everything up. I don't want to offload things like storyboarding to the AI. Of course you could. You can ask GPT and say, storyboard me a movie about a dragon or something. um and it will but this might be the wrong theory but i feel like with those kind of things it kind of regresses to the means a little bit when you rely on that creative output like the ai is giving you kind of um the the best average right and um true art right creativity is um improbable maybe right it's it's like statistically not probable and so storyboarding like for me is something I want to do if you relate it back to say like Kurosawa right if and let's look at a movie like high and low right the way he frames a scene is is based off of composition and he might have someone in the foreground someone in the background you know someone in the middle ground and then there's a negative space like right here.

10:40And he's getting you to look at this negative space. For me, I don't even know how I would prompt that, you know, to get the perfect composition. And that's why I want to do the storyboards. That's why I think that's like a spark job to me. It's like that framing is really hard to achieve without like the human creativity part of it. And so, yeah, that's just kind of my theory behind it. So I put things in categories like spark jobs and cog jobs. In the short film that I'm making, the cog job to me is there's a very mechanical part of this film. And that is, I'm trying to make something that looks like 1980s filmation, Hanna-Barbera, Ralph Bakshi, you know, the first Lord of the Rings animated movie, right?

11:34where they are rotoscoping over a person, over a scene. To me, that's a cog job. Once you have the style, right? No artist really wants to rotoscope, right? Because for a five-minute movie, you're going to be drawing over like 7 ,000 frames. And that's not fun because you already have the costumes, you already have the aesthetic. So that's something I offload to like Luma, right? Luma AI. And that way, Luma's doing the rotoscoping, and I don't have to draw 7 ,000 frames because I designed the characters. I designed the scene, and I shot the scene, but Luma is kind of doing the cog job part of it, which is the rotoscoping part.

12:15Parth Patil:So you really think about this more like it's like this AI acts as like a studio that you're directing on your behalf, but then you're taking the like high level, like what's your, you're the director or the, what do you, where do you think you fit in if you view these tools as like doing the execution, the cog jobs? Another example is the song that I use in the film, in the opening. I had played that on guitar 10 years ago. I recorded it 10 years ago. I remembered I had it and I thought, this is perfect for this film. And so I put it into Suno and made it electric with drums and stuff. And it fit perfectly.

12:56I think for me, it's almost more than a director. You're almost like this like all-encompassing creative. I'm not comparing myself to DaVinci, but like imagine like what a DaVinci would do with all these tools, right? It's kind of like that where I think the people that have a lot of creative skills right now are going to be able to almost make anything in their mind. We're kind of getting to that point. And so I don't know what I would call myself. I mean, I guess like creative director, a little bit of everything.

13:28Parth Patil:It's interesting. You get to put yourself in in whatever level of it that you'd like to yeah if you have a breadth of skills i think it's yeah it's it's like you're you're picking your spots where i think you're you're best at and where your creative skills are and i think that's a that's a huge advantage that's a huge thing like there are things that i can do now like making this short film that uh before i would have just stopped at the storyboards of the script and i don't i wouldn't know how i could actually make it, you know, without AI video. I don't think it would be possible. Well, let's kind of amplify this.

14:06Well, you know, go big and not just because we're talking about film. You know, we'll go beyond 11 for the Spinal Tap friends of our listeners. Oh, nice. Yes, exactly. What do you think a five-person-ish AI native studio could accomplish? what might have previously required hundreds? That's kind of exactly what I'm trying to get to. I think if you had that, you could honestly, number one, I feel like you can make any sort of animation that you want nowadays. That's without question. I think we're just getting to the point where we have new seed dance, new cling, all this stuff coming out that's really bleeding edge.

14:53I think you can compete with actual studios at some point. You know, you have to keep the creative spark within the human part of it. But I think you have all the tools to, with a five-person team, honestly, I could probably make any animation possible. I definitely, like, there's not a question in my mind these days about that. If you had asked me this question four months ago, I would have said maybe. but I feel like we're kind of making these leaps right now where I don't have a doubt that it couldn't be made and yeah like you said once you needed a team of like you know 100 animators or something or or like an entire team to create the music or to even do voice acting right in my film we still do the acting almost like you know it's it's like a mo-capped film right with Andy Serkis or something like if you're into video games, like how they made Death Stranding, right?

15:54Where they shot everything with the actors and they're doing the performance. But the greatest thing is you can change everyone's voices now based off of that performance. You can change how they look, you know, and it's still based off their performance. And so like the sky's the limit, right? You just need a small troop almost, like a traveling troop of creators and you could almost make any genre, right? With any characters. Yeah, as long as you have that human creative spark behind there, that curation, that intentionality behind the work, I think, yeah, you can actually make anything.

16:28Parth Patil:Yeah, you kind of already got into this. So tell me more about this project. So you've, this sci-fi project that you've had in your mind, you've had this idea for years, you would tell me about it, but it wasn't really possible until now? Like, what tools are you using? How are you thinking about it? and maybe in just the last four months, like how that has become like crystallized as possible. Sure, yeah. I would come up with these ideas and I would write them down and I would make a script for them. I would even storyboard stuff out. And usually that's where it stopped because I can't get props or costumes or actors or even new cameras to make it.

17:13It would be almost impossible. I have notebooks full of these ideas, by the way, over the years. And I would have stopped there at some point and never made these things. And maybe you can pitch it to a studio or something, but it's like super long shot that anyone would ever give you any money to make anything. I had the storyboard. I had the script. I made the treatment, went out, shot some scenes in situation with some friends of mine. um we wore vaguely you know costumes that vaguely resemble but again they're going to change in in uh you know in in luma um and the way the process worked was i had all the setup already and then go out shoot the scenes you build the characters right in luma there's a whole you know there's there's a lot of kind of like pre-setup that um to get the consistency but that's something you would have done like if you were working at a studio anyway um you know you're going to need to get that character consistency.

18:10So you build it all, the characters in Luma, you can even build them in Higgsfield and other things like that. But I would build them in Luma and then use that as the reference to build the video. And then, so my process is slightly weird in that when you watch those old films, right, like the Lord of the Rings or Fire and Ice, the characters are like hand-drawn rotoscoped but the backgrounds are painted right and it's a distinct feel and so my process is weird in that i take a reference photo for the scene and then i will change that to a painted style and then once we do the mocap and change you know the way the character looks you can also adjust the background to the reference photo to make it look painted so you you can get that feel that sort of like you know 80s like hand-drawn versus hand-painted background feel um and that's kind of like i i that was impossible um even last year uh right um it really was so that's kind of how it changed i the way i look at things when i come up with ideas is almost like what's stopping me from making this kind of as opposed to like previously i would just stop at the character design and storyboard you know yeah that's that's that's

19:29Parth Patil:It's incredible, yeah. Maybe a follow-up to that. So what part of maybe creating this, kind of creating this film, creating these projects, takes a lot more effort than people might assume? Because it's not just one click, right? That's a great question. It's the human curation part of it, I feel like takes a long time that people don't understand. There's a difference between kitsch and art, right? And that difference is genning 200 images to find the one, you know or even 300 right or um genning a scene over and over again it's the curation part of it um in order to get things perfect i think people underestimate how long that takes to but to me that's going to divide the slop from the non-slop right it's that um craftsmanship that takes a long time i would also warn that the character setup takes a while um to get that consistency it's very important but you need to go into it and like fully design these characters and um you know down to like what jacket they're wearing what pants what their boots look like what do their gloves look like you know i'm making a a weird fantasy western right and so the characters have a very distinct style to them and um they're all um like it's almost like in an anime right where like the main character wears red and then the side character wears blue and they like almost everyone has their own palettes and stuff and i think that takes a lot of setup to in in the beginning um so like that and the curation but again it's it's about the intentionality behind the work um you know you're not just pressing go and genning and first gen is perfect right you might have to go back like for us and and and maybe even the cool thing is i could reshoot in my backyard so so i could go back and reshoot a scene in my backyard and make sure the framing is correct, you know, and then do that one, right?

21:24It's that part, I think, that takes the longest. By the way, I think you're right that, you know, curation is a very key word here. There's this kind of question around where curation is all human, individual human, groups of humans, all machine, but actually more likely a combination between, obviously, the notion of how this kind of pulls together in terms of human and machine collaborating, dancing, et cetera, together is really important. How is your perspective on curation changed in this new AI amplified world and where you multiple humans, multiple agents, you know, that kind of thing. And what does that mean for how we get to what is meaningful creative work?

22:08My perspective on it has changed a little bit in that I think we have access to like a lot more um before ai you were a little bit limited in what you could explore i think and that would be like i can only explore things that i was good at illustrating or things that i had on hand maybe you can search the internet sure but i i feel like there was it was very limited um now you can get wild with with production right you can get wild with things um it's one of the reasons i kind of prefer mid-journey in exploring because mid-journey still has a little bit of this artistic wildness to it um that we're like some of the other things like nano banana are like very like pointed and distinct um and so i i like the randomness and I would almost rather gen like a hundred random things to find one good piece out of it so I think our access to ideas is like completely amplified and I think that's that's that's where you know the curation is kind of like evolving is now you have access to so many more ideas than you would have previously it's it's almost like giving you new directions in my opinion um that's almost one thing i kind of loved about the early stable diffusion days was the the image gens were so wild that it was like almost not even what you were prompting but there was something chaotic about it that i loved um and i think uh it just it's just like an idea generator almost

23:50Parth Patil:it it reminds me back to dnd it feels like chaos magic sometimes i'm like you're like it's like not random it's actually not random it's like a probability distribution but it's also like potentially something you've never seen or an idea that no one's ever combined and remixed and then you can kind of just go to latent space and explore the space of possible ideas to find that one thing that no one's ever seen that might might need to be seen by someone like might be worth like showing to the world okay um i'll give you an example when i first started to do this project. I had this, you know, I wanted that rotoscope feel of everything.

24:24When I started Jenning and Mid Journey, there was one out of, you know, I, I, hundreds of images and there was one from my references was slightly different. It was a little more animated. It was a little more exaggerated and it just, it just popped out to me. And so I started making all the other ones like that one and so now it's like it it changed from being like this completely like rotoscoped you know old school style to a little more um a little more almost like anime um but it's like a cross between the two that is something that i would have never found um on my own i i would never stumble on that even if i was drawing it myself i would have never gone down that avenue And so like that's to me where that sort of curation lies and that interaction between like the amount that we can produce and the access to these ideas.

25:25And I think that that combo is I would have never found that without without genning those. So, yeah.

25:32Parth Patil:Oh, man, that's so exciting. Well, Jonathan, I'm so glad we could have you on. And I'm so grateful to have you in on this journey with us and seeing what you're creating. Thanks for joining us today. But before we leave, I'm wondering if you have a question you'd like to pose, either Reid or myself. Sure. Yeah, I do have a question. My question is, there was a part in your book, Super Agency, where it says that true super agency is designed, I mean, is defined by allowing humans to tackle higher order challenges. So my question is, are there some critical higher order challenges that you believe we are currently not addressing?

26:19And what are they? Great question. There's a set of different things. I mean, part of, as you know, the writing of super agency was to try to show people that even though they feel that maybe their agency is diminishing because the world is changing around them, technology is being developed that changes the landscape, they can still grasp agency. They can still do things. And so even though, for example, like Apple builds an iPhone, which you may think reduces your agency, you can use the iPhone and gain a lot of agency and so do that. And that's kind of the parallel for the kind of the call to agency that super agency is.

26:57Now, I do think there are some really important, more subtle questions here. So for example, one of the things is to say, all right, well, even though there's this kind of technological, you know, tidal wave coming in and, you know, you can't choose the tidal wave or not, you can't choose the time of what it is, there's still ways of gaining it. But how do you gain it partially, not just by individual tool assigning tools but collectively like what is this change in terms of our landscape of the social environment or in and part of the reason i use the iphone as a parallel is well sure it enables you you can take pictures and do stuff and have you know maps everywhere where you're going but of course it also has everyone else taking pictures and what does that mean in terms of of the the universe that you're that you're now playing in and what does this mean for our kind of collective knowledge or a collective epistemology, how we parse the world.

28:01And some of that obviously gets to, you know, how do we discern, you know, kind of truths about ourselves, truths about the world that we're in, you know, kind of collectively together. How does that play in to how we gain agency, not just individually, but together? Now, Now, one of my hopes, I think, for AI is this notion where if we've got a good sense of how it amplifies our knowledge of ourselves in the world, like what is true, that then helps us learn collectively better. Like, you know, AI is one of the best learning tools developed in human history to date because it builds upon books.

28:46It builds upon the printing press. It builds upon the Internet. And I think that kind of sense of collective learning, learning together and learning truth and what is truth and what is the epistemology of it, not just outsourcing it to AI, but us learning together is, I think, one of the key elements of that kind of collective agency. yeah i i think you know that's kind of part of the book too that kind of resonated me a little bit um i i mean is that is that the bloomer um the bloomer ethos um yes yeah yeah so um yeah definitely i felt like i was uh part i i mean i'm definitely part bloomer in the book um but i'm also a pessimist in a lot of things so like i have like a doomer aspect to um some of the things i think about.

29:39But we're all conflicted because I'm using these tools every day and I'm creating wild things with them. And so, yeah, when I would read the description of the Loomer, I'm like, well, that's definitely what I'm doing. Jonathan, I actually, and by the way, I completely agree on the super agency that there is kind of Bloomer, Doomer, Gloomer combinations, and it's kind of, it's a more complex thing. So thank you for joining us. And I look forward to seeing some of your creative outputs in the future. It's going to be awesome. I will 100 % send you what I finished. Possible is produced by Pallet Media.

30:19It's hosted by Ari Finger and me, Reid Hoffman. Our showrunner is Sean Young. Possible is produced by Tanasi Delos, Katie Sanders, Spencer Strasmoor, Emo Zhu, Amon Suri, Danny Garrison, Trent Barboza, and Tafadzwa Nima Rundwe.

30:34Parth Patil:Special thanks to Surya Yalamanchili, Sayida Sepieva, Ian Alice, Greg Beato, Parth Patil, and Ben Rallis.