#194 Russell Palmer: How AI is Taking Over Hollywood and Changing Filmmaking Forever

0:00I really think the story element is lacking. I watch all the movies made by Pika's users and on stablevideo.com and Runway TV, and there's probably only a handful of artists trying to integrate story. And so I would just recommend to people, you know, if you want to get into it, don't forget the story element. It's not just about making an AI gimmick film with a bunch of video and moving sounds and trippy visuals. You know, there's a reason people love and watch these story types and these character arcs. and I think without those these AI films will never become just films and that's actually what we want we don't want to see a separate industry of AI films we just want to see regular people making Hollywood blockbuster quality films on small budgets and getting into film festivals and if you remember the first episode of the Simpsons or the first episode of South Park the quality of the animation was about as bad as it can get the people love the characters and the dialogue and the stories.

0:59Yeah, so I'm from Toronto, Canada. I studied electrical engineering and computer science. And my first job was at Microsoft as a PM in the office group. Fun story, that's actually where the profession of PM was invented. I saw the original memo to Bill Gates, which said, we need something between business and the engineers to make sure we're building the right thing. Yeah. So I learned from the best and it was great. By the end, we were really getting into machine learning and big data. So a lot of features like an Excel, I worked in the office group. So a lot of features like an Excel where we do autofill, but you know, powered by AI, more intelligent.

1:43And then, yeah, after I'd always known I wanted to do a startup and always wanted to work more in AI. So after several years at Microsoft, just learning as much as I can about building good product, I moved down to Silicon Valley and joined a startup in San Jose. It was called Viv Labs and it was founded by the Siri founders. So yeah, Adam Chire, Dag Kitlaus, Chris Brigham, and they had left Apple. A bit of a power struggle, I think after Steve Jobs passing and they really wanted Siri to be extensible. So almost merged with the app store. So they started a new AI agent where anyone in the world could teach it to do things.

2:32Yeah, it was great. So we worked there for a while. I got acquired by Samsung to be sort of, you know, the Siri for Samsung devices. And so I learned from, you know, these guys who they sold their first startup to Apple, their second to Samsung. Adam just sold his third startup to Airbnb. So it was an incredible experience learning from them how to found a startup, an AI startup specifically. And then I had a quick stint. I got recruited to JP Morgan. They set up an AI lab on the West Coast near Stanford Research Park. So we were integrating Amazon Web Services, SageMaker and everything into the bank.

3:13You know, JP Morgan's very old. They have like 200 years of data, right? So, but they don't want to lose it, but they wanted to try the clouds. We built a safe system there so they could build loan approval tools to give probabilities of who would pay back. But I kind of realized that finance wasn't it for me. And it was the pandemic and everybody sort of rethinking life and careers, the great resignation, they called it. And so I was taking these Stanford night courses to pass the time online. And a few of them were AI product management. And I had to write a paper on AI disrupting an industry.

3:52So I wrote it with my brother who works in Hollywood as a filmmaker on, you know, we saw an early preview of GPT-2 and tried to make it the right stories. And so we really thought there was something there. Now that robots could speak English and soon draw pictures, we just envisioned how it could help people with their storytelling, writing, storyboarding, animation. And so I quit my job, co-founded the startup with my brother. He's our chief story officer. And we've been doing it for the last three years. Yeah. What I found interesting when we talked before is you were saying that, you know, there's Runway and Sora and Pika and all these video generation tools that are clearly going to have an impact on the industry.

4:48But they're sort of approaching it from the end product backwards. And you were interested in the storytelling aspect and approaching it from that direction. So, yeah, can you talk about that and then talk about the platform? Yeah, yeah. So my brother, he actually took engineering, computer engineering, but realized he didn't want to work in a cubicle nine to five. He paints and writes novels and he works on TV shows around Toronto, Vancouver, like Suits and the Boys as an AD. So he knows how movies are made. They are not made from taking a bunch of two to eight second clips and then working backwards.

5:32Everything starts with story. So writers today, they will plan things on cards, put them on a corkboard. You can almost do weeks of planning and story structure before actually getting to the writing. and when we started in 2021, you know, there was no chat GPT, there were no image generators, it was just GPT-3 with an API and you would type something and it would finish your sentence. So we wanted to start with story, it's the most important part, so we built an app, it's almost like a form and everything, you know, every, my brother had his film school notes, every day is, okay, today's plot, today's characters, today's this and it's very structured, they say, you know, pick one of these character arcs.

6:15Here are common archetypes used in movies. Here are 15 story types used since the ancient Greeks. So we built an app that lets people pick and choose and type in their own ideas. And then anything they're stuck on, they can hit generate, like if they need a B story. Most people forget to do that, or they don't know what makes a good B story. It's the theme reinforced by secondary characters. So our app, it goes and looks at everything you've written. What are all the secondary characters? What's the theme? And then gives you like five ideas. If you don't like it, there's five more, five more, five more.

6:50It never gets tired. And so in around 2021, early 2022, this was before the writer's strike. So people would take our calls and they loved it. Writers loved it. They would say, I've been stuck on this idea for years. And they type it in and click on what they're missing. and they'll say, you know, I'm jealous. Like, why didn't I think of that? It's perfect. It feels like it read my mind. Like, it feels like it's my idea. I just didn't think of it first. And wow, can I really use this? Of course, yeah. So people loved it. And then one month after we launched, the writer's strike happened. So yeah, I'm sure maybe we'll get into that later, but that's sort of the evolution of how we got the app built and launched.

7:42Yeah. And right now, I mean, when we originally spoke, I was thinking that you were more on the video generation end, but you're building basically from ideation through storyboard to, I don't know, the pipeline. I'm not a filmmaker, but it's clear through to video generation. That video generation piece, you will have that. Is that right? Definitely. That's our North Star. We called the original company CyberFilm.ai, even though we knew we were starting with text because the end goal is films. And so, yeah, it was a matter of timing, but also startup bootstrapping. strapping. So when we began in 2021, there was only an API for text, for GPT-3.

8:40So we built out as much as possible to help with plot, characters, acts, beats, and script. And then we had predicted that in a few months, in 2022, the image generators would be released with models and APIs, and they were. So then we added in storyboarding. And it's really easy. You don't need to know cinematography. You just pick common shot types and camera levels. And we say you don't need to learn prompt engineering. You can just type as if what you would say to the artist if you were leaning over their shoulder and they were drawing. And so we do a lot of intelligence there to recognize if you named one of the existing characters, can we go look at their description and the pictures you've uploaded of the actor from that page?

9:30and so we use that to try to solve the character consistency problem and it works quite well. The thing about storyboards is the audience never sees it so it doesn't need to be perfect. You know if you wanted to cast Brad Pitt it looks a little bit like him not so much like him in the next shot but you know it's going to be him so it's actually a great case for for image generators while the quality is the consistency is not quite there yet and then the final thing we've already added it in beta with stable video diffusion where if you're on your storyboard page you can click on any shot and say you know car zooms through the frame or zoom in on the character's eyes and it'll animate it for two seconds so that's not quite long enough So we kept it in beta.

10:20But as soon as Sora comes out, you know, that's just another API we'll add into our app. And each shot can be up to 60 seconds and you can have a full a full movie. I will say, and we talked about this earlier, we don't want to be a complete synthetic video generation company. We want to help with the stuff you need help with. But we still encourage and want our users, and this is what we do for our own movies, to still go out and film real actors giving real performances. You can film it on your iPhone. There's really no restriction as far as expensive camera equipments. So we're hoping our users will intersperse maybe real-life acting.

11:00And then if they need visual effects like a car explosion or something, they can do that synthetically with the app. Yeah. So is the target or the goal to democratize filmmaking so that I can make a film? It's not going to have a commercial release, but I can make something that's credible. or is the market really Hollywood production companies and professional filmmakers? Yeah, it's a beachhead strategy in that we're definitely targeting the first group first. Most of our customers are film school students learning to write, aspiring writers, people who write as a hobby in their evenings and weekends, even people who work on movie sets, but they're not the writer.

11:58They do the camera work or the stunt choreography. But everyone thinks that, oh, I'd love to make my own movie. Maybe not a full career of it, but almost everyone has a fun idea for at least one movie. And we think, wow, if 8 billion people made even just one movie, that's 8 billion new movies. They're not all going to be great. But what we're seeing lately since the golden age of Hollywood is as we open up to more diversity, we see movies out of South Korea winning best picture in China and other countries. So we really do want to democratize. And we'll say we've been using that word for three years.

12:38A lot of our marketing lines seem to be getting picked up on the Internet, and that's great. But some people say democratize storytelling. And the retort to that would be, well, anyone can pick up a pencil and paper and write a story. That doesn't need to be democratized. But some people need help because they can't afford to go to film school. They didn't get into USC School of Cinematic Arts. And so they need help. You know, maybe they're in rural India or somewhere. And there's just no writers or filmmakers around. And most people can't move to L.A. I keep writing, reading these these these articles and variety and Hollywood Reporter.

13:20And everyone's advice is, no, just move to L.A. and you've got to struggle for 10 years. but like most people in the world you can't get a visa to just go hang out in la and hand out screenplays and coffee shops for 10 years so yeah um yeah we really want to democratize it but the final thing i'll say is we built the tools so that they speak the language of professionals um and that all the shot types in our shot list are the ones used in real life and all the camera levels and you know we're going to do color palette palettes next and maybe lighting and so everything even when we're we're going to do our animation you'll see like you know dolly left and pan and zoom and so we really want it to be able to be used by professionals and yeah recently we showed it to agbo the russo brothers studio they made everything everywhere all at once and used runway gen two in the rock scene so they're very open to ai and they used a runway gen one gen two in which scene uh so in every everything everywhere all at once um you know the rocks that had the googly eyes right right they're kind of talking i just i just read that that was runway yeah yeah yeah and and they did it and again it people will bring up if it costs less money does that just mean you fired people but you know they still had a full team of vfx and cgi animators but yeah they were able to bring the cost of a movie down it's up to 250 million dollars for a blockbuster now and i think that's just such a blocker to you know no executives want to green light original stories like parasite yeah it's from south korea it's horror there's no brad pitts i'm not risking my career.

15:08Let's just make Spider-Man 9 and Madam Web. And just audiences are not responding to that. They want to see people from all over the world who have, what is it, the Martin Scorsese line. Everyone's got a story to tell. So we'd love to see people just like you make a movie. Something funny happened in your house last night at dinner. Make a movie. You don't have to put it on YouTube. You can just email it to your friends and family. And how fun is that? It's great. Yeah. A couple of questions here. One is the cost. Do you have a price so that it's really affordable for consumers? Exactly. Yeah.

15:50So we give you three days free and then it's$19.99 a month. So pretty accessible, you know, same price as ChatGPT Plus, except, you know, we say we're going to do text and image and video. and we don't just integrate Dolly 3. We've also got Stable Fusion XL and Stable Video Diffusion. So we're trying to match them on price, but then offer just a whole lot more. And yeah, that's why we hope anyone can do it. We are thinking of adding premium and an enterprise plan. So premium, because if video generation from Sora costs a lot, like it takes an hour to render 60 seconds. So I think people will be making a lot of videos, especially if it's a 90-minute film.

16:38And then we're looking at an enterprise plan. So we've been talking to Agbo, NBCUniversal, Sony. And yeah, they basically just say, you know, we don't pay$20 for anything. So come back when you have a$50 ,000 enterprise SKU for 500 seats a year, and maybe we'll give it a try. So if we do that, we'll use the extra money to build in. And we have encryption, but just, you know, really strong encryption and backup and maybe an offline mode and just feature. I used to work at Microsoft, so there's a lot of features. For example, you know, Outlook email, the consumer version doesn't quite have as many features as the enterprise version.

17:19So we'd look to add a bunch of that. Yeah. I was actually telling my wife about this. She's a writer. Oh, great. right and things and uh she was saying yeah but you know uh if if you got like these all these drop down menus uh for um for sort of i don't know what the term is archetypes uh or story arcs you're gonna you know there'll be a convergence toward the mean right you'll end up with stories that have very predictable and this is already a problem in hollywood yeah uh very predictable story arcs very predictable characters how do you how do you deal with that uh is yeah Yeah, I mean, well, on one hand, to me, it's all about the audience and their enjoyment of movies.

18:27And it seems pretty clear that these archetypes evolve because that's what audiences want and crave and those movies perform the best. I don't think our app does it anymore. If you're not using our app, you're probably reading Save the Cat or Story by McKee or Sid Field Screenplay.

18:50And if you go to film school, this is what they almost force you to do. Our app is a little different because machines are quite a bit smarter than humans. So you can actually go and say, combine the David versus Goliath story, which is really what Titanic is, with the trap with the monster, which is what horror movies are like Jaws. So if you wanted to make a horror movie, Titanic meets Jaws, you could pick those two. And it will actually merge the archetypes and create a new one, which is very hard for human writers to do in their head. So, yeah, we get this question a lot. like is this is ai and chat gbt does this i'll fully admit we we've tested it probably more than anyone to try to write a script because if it could we we might have to give it or start start a new company um and it just doesn't work um it hallucinates it doesn't write enough you're constantly scrolling and it does give very cliche ideas and um and so in our app you know it's been used by people in Hollywood.

20:01The producer of Breaking Bad reached out recently. He's a user and he teaches a class with someone who worked on American Psycho and they teach story structure. And they use our app for that. And they say the outputs are great. That's what machine learning is good at. It learns all of our archetypes. And then it comes up with original stuff. And some of that, especially if you write in the quote, we do a lot of prompt engineering or in the prompt, you know, make sure this is an original idea, something that's never been done before. And our users are seeing that. I liken it to people say, AI, can it be creative?

20:41And so if you've seen the documentary AlphaGo, you know, Go has been played for 3000 years. You'd think humans would have thought of every move ever, right? And then it beats Lisa at all the first game and the second game and the third game, it makes this mistake. And everybody, all the commentators and experts are kind of laughing. Move 37. Move 37. Yeah. And then they all start cluing in. Wow, that was incredible. And Lisa said, it has to be creative to come up with that move. I don't think any human not only has ever done that, but would ever have thought of that. And so I believe that people say AI doesn't have a soul.

21:23It'll hit a limit in writing, but they've said that about chess, go, radiology. It kills us at video games now. It's safer at driving cars. So why not storytelling? Artists hate it when you call their work data or content, but that's what it is, and it's structured. So I really do believe that this tool will give people better ideas than they would have thought of themselves. And I think actually AI will grow and improve the quality of Hollywood as opposed to, you know, reducing the art form to just, you know, industrially processed movie scripts no one enjoys. Yeah. On the various categories that you can choose from, like story arcs, or how do you refer to the David and Goliath?

22:18Yeah, those are the story types. And then we have character arcs. So, you know, the positive, the negative, the flat, disillusion, corruption. Yeah. Where did you get all of this? Is that just out of the movie making handbook, you know, in the sky? There's some list that has it all or has it been, you know, you sort of, your brother think, oh, what about this one? We didn't include this. and then you added in, or are you using AI models, large language models to mine literature and scripts and plays to identify arcs? Yeah. So we made our own list. So that's sort of, you know, part of our IP and moat.

23:10It started with my brother who did, you know, go to film school and majored in creative writing. And then he's read, you know, the top 10 books. I named a few before the hero with a thousand faces and everything. And then he came up with his own list and his own names because we wanted to be original and not copy anyone. Copyright and copying is a big thing in the AI film industry, so we try not to get anyone mad at us. And then, yeah, when we see or experience new ones or get requests from users, we can add them. We did not use, we use, you know, ChatGPT and AI for stuff, but sometimes it's faster to just do it yourself.

23:55And this is one of the things we did ourself. And yeah, it seems to be working pretty well. You can always leave it blank. So you don't have to pick one of ours from the list. And actually we've started adding the ability to enter your own new items in the list. So you enter a story type and define it. Romeo, Juliet, love story, you know, two star cross family lover, whatever. And you can enter it if you think we've missed it. And actually, I just had a call last night talking to a school of Chinese Canadians in Toronto. it's like an LLM specialty class and they said hey where are all the ancient Chinese proverbs and stories and things so I said great send me a list you know like crouching tiger hidden dragon yeah we probably don't have that so send it so we definitely want to globalize it and the beauty of AI is we can just add items to the list and GPT which backs us knows everything already.

25:01So we don't really have to teach it too much. Hi, this episode of Eye on AI is sponsored by BetterHelp. These days, there's a lot coming at us and mental health is more important than ever. When life goes so fast, it's important to take a moment to celebrate your wins and make adjustments for the rest of the year. Therapy can help. You can take stock of your progress and set achievable goals for the next six months. I know because I've been in therapy at different times of my life, and it's been a tremendous help. It's helpful for learning positive coping skills and how to set boundaries. It empowers you to be the best version of yourself, and it isn't just for those who have experienced major trauma.

25:50If you're thinking of starting therapy, give BetterHelp a try. It's entirely online, designed to be convenient, flexible, and suited to your schedule. Just fill out a brief questionnaire and get matched to a licensed therapist, and you can switch therapists anytime for no additional charge. Take a moment and visit betterhelp.com slash IonAI to get 10 % off your first month. That's BetterHelp, B-E-T-T-E-R-H-E-L-P.com slash IonAI. That's IonAI, E-Y-E-O-N-A-I, all run together. So for 10 % off your first month of therapy, visit betterhealth.com slash IonAI. You'll be glad you did. Yeah. But on the idea of, can you use it in the reverse?

26:53Can you give it a script or a novel or a play and have it break it down and say this is you know a David and Goliath story and this is the story arc and this is the these are the characters which fit these archetypes can you do that? So we get asked a lot, you can't do it yet so you can import a script from you know text file or final draft FDX but we don't reverse fill in the characters acts plot page but we get asked to do it so much that we will probably add it um but yeah we have a big a big road map we really want to get video in next and um we're more focused i think the use case for that would almost be like a film school student trying to cheat on their book report right like oh right about citizen kane okay i'll just scan it in and take all that yeah uh and uh so so the the since i do video uh could you put it up on the screen and just show uh those that are watching on youtube what it looks like and just while you're doing that this could be used uh for any form of storytelling not necessarily film right definitely yeah so on our roadmap for this year is to add new templates so i'll show you a demo in a second but our beat page so the act page has you know act one two and three this format applies to everything from an individual tv show episode to the arc of the full season to even 30 60 second television commercials everything tells us even in business they tell you right in your powerpoint presentations tell a story be a storyteller so our acts apply and then for our beat page it's customizable, which I'll show you, which means we gave you a 40-bit default template, which is used in feature film, but you can delete and edit nodes to fit any different format you want, including maybe podcast or short videos or whatever you want.

29:11So we definitely want to expand things out and even have templates for comics and graphic novels. You can make anime and manga and you know um gold marvel golden age style um superhero comics so and and you know presumably uh novelists could use it to to sort of map out their uh their their writing before they start the act of writing absolutely yeah every every story requires planning um it's just it's done in analog today, which is fine. You know, I remember my mom never wanted to give up her typewriter when we got a PC. And I get it, you know, take a typewriter to a cabin, sit by a fire, it's probably nice.

30:01And so there probably will still be people who want to put their stuff on a cork board. But it's the digital age, everything's in the cloud, right? So how do you share that with people? You got to take a picture and they can't edit on their end like a Google Doc. And so we think i mean final draft is the common story uh screenwriting application and it came out in the 90s and so we just think um this will bring forward filmmakers but definitely yeah for novelists um we would love to have a new template and but a lot of the things the plot page the characters page the act the acts page will still apply and we want to have a concept art area so maybe you don't need a full set of storyboards for your novel, but you could do concept art.

30:48So when you pass the manuscript to your agent or publisher, you can pass a book. They can flip through images of characters in the main locations and everything. And most writers can't draw. And they can't or don't want to pay for a storyboard illustrator for$500 a day that you know, will take five or six months. So again, we don't see it as a job replacement because that writer never would have afforded it and never would have paid for that anyways. So yeah, I think it'll be good for novelists. Okay, so this is our app and it starts on the plot page. Now, a lot of our users, they don't want a movie idea generator.

31:31They already have an idea or several they've been working on for a long time. They just need help finishing and they want it to be good and feel like it's their work. So they can type in the title and the log line. If you need ideas for a title, you can generate. We give examples. So if you don't know what a good theme is, these are from real movies. I mentioned before combining the story types. So we do give examples of the dozen or so story types that are commonly used with some example movies. so you can mash a few of them up and pick genres. So to come to the B story here, I mentioned this is something that it's like Adrian's story in Rocky.

32:14You know, it's a subplot. And what it does is when you hit generate, it will look at all the characters, which ones are the protagonist, antagonist, and the secondary characters. And then we'll look at the theme. And then our prompt engineering says, a good B story must reinforce the theme with secondary characters. So you'll see of the five ideas, one's about Raven, one's about Slick, one's about Misha. And so if you come into the characters, you'll see, you know, Misha's role is the ally, Raven's the love interest, Slick is the mentor, and these are the antagonists and protagonists. Continuing on, you know, you could try to invent new character arcs, but these have been around, again, since the ancient Greeks.

33:01You've got Spiral and Corruption. That's sort of like Darth Vader or Michael Corleone, or villains are usually flat. Positive arc is when it has a happy ending. So we're letting you pick things. Again, it's kind of a teaching tool. If you didn't know to type it in, Chad GPT, that's why this is a little better. We show you all the things you really need. For archetypes, we don't just have hero and villain, which ChatGPT seems to be limited to. We have all these types of characters. And even for villains, we just don't have a regular classic villain. We have 10 different types of villains. So you can really develop characters that are multidimensional, but also they fit you.

33:47You're not saying, you know, you tell me what my villain is. You're really picking. and so again you know most characters you'd learn this in film school have an unresolved issue from their past that haunts them you can see some examples here and that makes them believe a lie and then because of that lie there's something they want in act one but they keep screwing up through act two and then what they really need is this by act three to overcome the antagonist so So this is stuff like backstories, especially for the third, fourth, fifth, sixth, and seventh character. Like you're kind of getting tired.