AI Video Killed the Video Star

0:00A few weeks ago, Google dropped VO3, Generative AI Video, but now with generative AI sound to go with it. This is video from VO3. What do you think about the idea that we're just a bunch of prompts? If I'm generated from a prompt, how come I don't have six fingers? So is this. About to do the first plunge into an active volcano. Let's send it. And this. Breaking news. The Secretary of Defense, Pete Hegg Seth, has died after drinking an entire liter of vodka on a dare by RFK. But how are the reviews? A slop monger's dream, says The Verge. It might actually take my job, says YouTuber Matthew Berman.

0:41The world is not ready, says Mashable. We're so cooked, says thousands of people on social media. But are we? Maybe not. That's our take, a today explained.

1:00Weihnachtsschmaus und Silvesterer-Kletz sind gerade vorbei. Da kommt auch schon die nächste Tradition um die Ecke. Das große Opel-Angrillen. You're definitely at your partner at the time. With music, delicious things from the grill and exclusive offers. Like the Opel Frontier, which is already at 225€ a month in Leasing plus a special price. And in the showroom is also the rest of the model palette ready. Up the time to the Opel Grill on Saturday, 24 January at your partner. More on opel.com.

1:41Joanna Stern is a personal technology columnist at The Wall Street Journal. She is not a filmmaker, but that didn't stop her from trying to harness all the latest AI tools to make a short film. So I worked on this film with a close friend of mine and producer named Gerard Cole. He works here at The Wall Street Journal, and he's a seasoned audio and video journalist who just really has become obsessed with testing and playing with AI video tools. We started on this project probably at the end of March. I sort of challenged Gerard. I said, hey, we'll make a film. I think we should try to make something that's like a real film.

2:17We come to this place for magic. I'm going to make them an offer again. I mean, like a two-minute short film. You're not sure. It's not Spielberg here. And what was so crazy about it is that every week there would be new tools that would come out. The companies keep getting in touch and saying, well, actually, we have a new update next week. So you might want to hold off on publishing that video or you might want to hold off because we have a new tool that you can test. And so in May, Google announced VO3, which is their third version of their video model. They also announced a new tool called Flow, which makes it easier to edit with AI video.

2:58And so we kind of had to like uproot the project a little bit to get this going. but this stuff is moving so fast that every night we'd go to sleep, we'd wake up in the morning and there'd be a new AI video tool that we thought we should try. The one that has gotten a ton of buzz over the last couple of weeks is Google VO. And this is from Google. This is VO3. What they did here with VO3 is they just created a new model that really blew people away. Previously with AI video, not only did you kind of have some weird wonkiness to some of the visuals and maybe things didn't look as realistic, but also there was no audio to them.

3:31And now with VO3, you can put in a prompt. You can say, a woman working out alongside a robot. And now with VO3, you have audio. Go for the burn, sweat. When I see the woman boxing with a robot, you hear sprunting. You hear sounds of the robot's mechanics. You hear punching sounds. You hear all kinds of audio to make the scene come to life. Which is to say it just took like a massive jump, this technology, because it just feels a lot more real. Yeah, it feels a lot more real. And I said, OK, well, what if we can see if we can actually tell a story here? We really wanted to see if that was possible.

4:17And we learned very quickly it is possible. It's just really hard and time consuming. The film itself is about my, if we want to say that I'm me in this film, getting a sort of a humanoid robot. And these robots were designed to make people and humans more efficient. Time for your coffee IV drip. Because I thought, OK, maybe we can have some fun playing off of this idea that AI is all about making us efficient and in our jobs and in everything else. I'll let people watch, but the robot lives with me. We have some good times together. We have some not so good times together. He really wants me to keep working.

4:55My ultra sensitive microphones indicate you are not engaged in elimination activities. And then I can't ruin the end. But, you know, let's just say I come out on top at the end. Wait, can you ruin the end? OK, fine, I'll do it. But I don't know. It's not usually how it works with movie interviews. But yeah, I mean, in the end. Spoiler alert. Spoiler alert. I get frustrated with this robot, and I had no other choice, but I have to reprogram him. Joanna, please don't do this. Oh. Yeah. But I will say, there was a lot of constraints to making this. And so you'll notice when you watch everybody, you'll see, like, the robot doesn't talk, right?

5:42The robot has a voice, but it doesn't have mouth movements. And so that was one of the constraints we had. And you'll see, I never talk. Like, my mouth never moves in the piece because we had that technical constraint. You can't really have the dialogue work very well between two people. You can't really make that consistent. And so when you watch with an eye for the technical constraints, you can really see, like, oh, yeah, they kind of had to make something that was like this. Tell people exactly how you made this short film. What exactly are you doing to make this? Because this isn't like shooting a little short film on your phone where you hit record, you capture some footage, then you edit it.

6:21Yeah. No, and I'll take you through as simply as I can, but it is pretty complicated. So we decided we wanted to have two characters, me, and I exist in real life, and this robot, which does not exist in real life. And so we created these digital versions of the characters. The robot named Max or OptiMax 5000. We created using an AI image generator. called Mid Journey. And so we kind of iterated in that. We worked through, okay, what does he look like? What does he look like? And so we finally landed on some images we liked. As for me, I took a bunch of photos of myself, different angles. And so then we went into Runway, which is an AI video generation tool, and we uploaded those photos.

7:01And then we said, okay, create a scene where you see the robot working out alongside Joanna and make it in a suburban background with houses on a paved street. And so then the runway would spit out what we would call the first frame of that. And so we'd have an image and then we would take that image and we'd put it into VIO, Google's tool, and say what we wanted the motion to look like. And here's where things got really complicated. And Gerard really did a lot of this work. but you really have to give the model very specific instructions on what you want to be done. And so he worked alongside Google's Gemini, which is their large language model, to really craft detailed prompts of what we wanted the videos to look like.

7:47And so these were long texts, like hundreds of words that you would put in with the photo and the text into Google Veo, tell it what we'd wanted, and out we would get a bunch of videos. And we'd pick from those videos what would look the best for the scene. When your video dropped, what did people think of it?

8:11What was crazy was how mixed the reviews were. A lot of people wrote in saying they were blown away and they could not believe how real it looked. They laughed because we played a lot of bloopers. So there was a lot of people that really enjoyed watching this. Joanna is so good at doing these and brings in the mainstream in such a great way. But then there was a very loud and vocal group that just hated this. Here are some of the reviews that I read on X or on TikTok. Wow, that was just awful. Ugly, soulless, nonsensical. Garbage. This is an abomination, and you should feel ashamed for making it.

8:53Absolute soulless shit. Wow. Shit from the butt. Shit from the butt? That's my favorite. Why are people so mad at your video? They're mad at AI video. They're not mad at my AI video. They're mad at AI video in general for existing. Can you trust what you see? Because people don't let you know they're using AI. Deep fakes and misinformation could get a serious upgrade. Synthetic video evidence might become harder to distinguish from the real thing. You can also see in the quality right now, it's not really Hollywood level. Is that where there's like a more practical use for this technology right now?

9:33That's the goal of many of these AI companies. I mean, yeah, I mean, that's where it really gets interesting. So some will say like, look, this is a moment to democratize video tools, right? Those folks who aspire to be filmmakers, well, they can now just do this. They can sit in front of their computer and they can make things that they once never would have been able to make before. But then you have the other side of this where what might we see on the big screen that might actually be AI generated. And so we've seen a bunch of AI film studios and production houses start popping up. The goal is for the makers, the Googles, the runways of the world to be working with Hollywood.

10:11Their hope is to start working with film studios to generate stuff that will end up in the films we see on the big screen. Or the small screen, whatever you watch your Netflix on.

10:31You can watch Joanna Stern's short film at wsj.com or on YouTube where it's called We Tested Google, VO, and Runway to Create This AI Film. It was wild. We're heading to Hollywood in a minute at Today Explained.

11:11Support for the show comes from Growth Therapy. Everyone thinks they're a therapist, especially in the new year, when you might be getting advice from every corner of your life, your feed, your group chat, even your barista. But here's the thing. Advice isn't therapy. Therapy is therapy. And if you're looking to start therapy with licensed professionals, Grow Therapy is there to help. Grow can connect you with thousands of independent licensed therapists across the country, offering both virtual and in-person sessions. You can search therapists by what matters most to you, like insurance, specialty, identity, or availability, and get started in as little as two days.

11:50And if something comes up, you can cancel up to 24 hours in advance at no cost. Grow says there are no subscriptions and no long-term commitments. You just pay per session. Whatever challenges you're facing, Grow Therapy is here to help. Sessions average about$21 with insurance, and some pay as little as$0, depending on their plan. Visit growtherapy.com slash explained to get started. That's growtherapy.com slash explained. GrowthTherapy.com Availability and coverage vary by state and insurance plan.

12:31Support for the show comes from BoxyCat. There's a lot to love about having a cat in your home. But there's one thing that even the most ardent cat lover could do without. Stinky litter boxes. Boxy says that they can help. Boxy wants to be the last kitty litter you ever try. They say the difference is in their formula, which includes probiotics that can stop bacteria that cause odors. Plus, Boxy says their clumping power is superior, making it easier to scoop so your litter box can be less stinky without any of the fake fragrances. Here's our colleague, Andrew Melnizik, to tell us more. So my sister is that cat mom.

13:08She calls them her children. So when it comes to trying anything new for them, she's incredibly skeptic and always a little hesitant. That's why I was so excited to hear that she was really enjoying Boxy's cat litter. The first thing she noticed, there was just zero issues transitioning to cats. There was no drama from them, which, you know, cats can kind of push back and be a little bit drama. It was really a miracle. If you're tired of switching litters looking for the one, get Boxy at B-O-X-I-E-C-A-T dot com. It's the last litter you'll switch to. And join 30 % off with code EXPLAINED at boxycat.com slash explained.

13:48That's B-O-X-I-E-C-A-T dot com forward slash explained.

13:59We come to this place. Today, explained. For magic. Because we need that. Devin Gordon wrote a big piece titled What If AI Is Actually Good for Hollywood for the New York Times Magazine late last year. We asked him, how dare you? Here's what he had to say. The premise and starting point was my sense that if you were listening to the discourse about AI and Hollywood, you would either hear that it was going to be the end of Hollywood and wipe out everyone's jobs and turn the future of cinema over to robots, or it was going to be the greatest creative, unlocking, magical wand ever handed to creative pill makers in the history of humankind.

14:55And I had also been hearing and reading these stories in places like The Hollywood Reporter. Everyone is using AI, but they're scared to admit it. It's the dirty little secret. AI is being used for scripting, for shooting and producing movies. You go into a little booth that's 360 degree camera, and you're asked to do 30 different expressions. And so I was like, okay, well, what are people actually using it for? What is actually happening with AI? So I started with a visual effects company that works with AI called Metaphysic. The reason why I wanted to start with them is because everything I kept hearing was that when AI descended upon Hollywood, it was going to hit visual effects first and hardest.

15:47So I wanted to start with a visual effects company. And this particular special effects company, visual effects company, Metaphysic, their specialty was sort of taking the deepfake logic of digitally creating a photorealistic copy of a famous person's face. and applying that to all sorts of aspects of the filmmaking industry, from special effects to dubbing to reshoots, animation, aging and de-aging, etc. And so I went and spent time with them. And one of the first things they did was they sat me down in a chair, pointed a camera at me, and there was a television screen opposite me, and my face was on the screen.

16:34And then the metaphysic guy clickety-clacked a little bit on his keyboard, and suddenly my face had Tom Hanks' face sort of pasted on top of mine.

16:47Tom Hanks, the actor. Yes, Tom Hanks, the actor. My mom always said life was like a box of chocolates. I could see my face. It was still me. And if I talked, it was moving. But I was also very recognizably Tom Hanks. You never know what you're going to get. The reason I was Tom Hanks is because the film project that Metaphysic was then working on was a movie called Here that starred Tom Hanks and Robin Wright. Hey, Dad. I'd like you to meet Margaret. Nice to meet you, Margaret. Nice to meet you, Mr. Young. It was directed by Robert Zemeckis. The team from Forrest Gump reunited again, using AI technology to a degree that it had not been deployed in a Hollywood movie before.

17:34In fact, it was central to the making of it. She's pregnant. She's what? She's pregnant. Margaret is pregnant. You're just 18 years old. In this case, they were using it to enable Tom Hanks to play the same character from the age of 18 to the age of 80. And the way they were able to do that was using metaphysics, AI technology. And one of the reasons why I wanted to focus on this movie here, which is not a particularly good movie. I wouldn't necessarily recommend you Netflix and chill with it. Get the fuck out of my house. But I was interested in this movie because this movie is probably the first mainstream Hollywood movie that would not have existed without AI technology.

18:23And the reason why is because it's effectively a small, domestic, emotional, serious drama. The only reason why this movie could happen is because the visual effects that it required were cheap enough with AI. It's as good as CGI now. And it's a lot cheaper. And it's a lot faster. and it gives directors a lot more creative control on the set. So that's why in the visual effects space, there's such this expectation that AI is very quickly and already is in a lot of ways transforming that industry in good ways, but also in ways that's probably going to cost a lot of people their jobs. I mean, and let's talk about all those people for a moment here.

19:05Let's start with Tom Hanks, because one thing that really surprised me about your piece was that you asked Tom Hanks how he felt about the potential for AI to enable him to star in movies 100 years after his death. And he was like, bring it on, right? Surprisingly unconcerned. He was just sort of like, well, let's just get the paperwork sorted out. Amazing. And I was a little surprised, to be honest, about how cavalier he was. For instance, I mean, isn't it easy to imagine a scenario? Maybe not in the Hanks family. I'm sure the Hanks family is going to—I trust Chet. Do you trust Chet? Big up the whole island, massive.

19:47It's your boy Chet and I coming straight from the Golden Globes, you're what I'm saying. No, but I do trust Colin. I trust Colin. I trust Colin. I trust Colin. You always want to work with good people, and obviously I think my dad's good people. But OK, what about Colin's grandkids? And they're down on their luck. And all of a sudden, 100 years from now, Tom Hanks' legend, his imagery is being sullied because his image is being used to make bucks in porn or whatever. He's not thinking that far in advance. Let's put it that way. I think the takeaway for me, no shots at Tom Hanks, was that it did sort of reflect a class divide in AI worriedness and how worried you should be.

20:35Right, because not everyone is Tom Hanks. I mean, what did you learn about all the people in VFX or, you know, costumes or makeup or what have you that are terrified about what's about to happen to their industry? You know, one of the things that I kept hearing on the makeup front, with AI, is a director going to have to have a makeup department do a character's makeup every single day? Or can the makeup department do it once, right? At the start of the production, that becomes a file that gets saved and mapped onto the character's face later. and now instead of having a makeup artist for the entire run of the set, you've only got the makeup artist for one day.

21:24You go from makeup artists being paid by the day to some sort of almost license or copyright for how many days that makeup work gets used, right? The entire economics of the industry has to change. Does it mean that we're not going to need makeup artists? Of course not. We're still very much going to need makeup artists. They're going to need them as much as ever. But how they work and how they get compensated is going to radically transform. And you can go through every department in the filmmaking process and each of them would have different ways in which AI will disrupt how they work. The thing about all these ways is that none of them are as grandiose as the worst of our imagining, right?

22:07You know, the people who were the most skeptical about AI's ability to overtake human creativity that I spoke with are the people who understand AI the most and use it the most. They understand its limitations and also how to best use it. They understand how to use this tool. When we're talking generative AI, when we're talking creative orientations or applications of AI, they understand how indispensable the human mind is to that equation. It just doesn't work without it. The notion, the theory, who knows if this will come to pass, but the positive theory, the flip side of this, is that AI lowers the barrier of entry to so many more films that even though the size of the crew and production is shrinking because of AI, the amount of productions that can exist grows.

23:08because more people can afford to make more movies. You can accuse that of being sanguine and overly sunny. I would say in the defense of the sanguine people, the indie film movement does provide an interesting parallel here, right? When filmmaking went from very, very expensive, limited film in the 90s to small handheld digital filmmaking where anybody could make cinema quality movies, all of a sudden you did have a lot more movies, right? You had a lot more movies being made for a lot less money. So there is a test case, right? Can AI do that? Well, I feel like in some ways that brings us back to our friend Joanna Stern at the Wall Street Journal.

24:02To her haters out there, I think you're missing the point. I don't think that Joanna Stern is in any way trying to make a film that could go air on ABC or air in the movie theater. What she's trying to demonstrate is how easy it is for even someone like her to effectively sit there and make something that looks at the worst a bad knockoff. But look at all the things that she can do without having anybody. Exactly. Or experience. Anybody. Or experience. Yeah. Right? Right. And now take that capacity out of her hands and put it in the hands of people who actually do this for a living. Right. And the question is, how dangerous does this get?

24:55How many people is this going to replace? And I just don't think we know. I don't think we really know. In some ways, what Joanna's film leaves me with is both fear and relief.

25:19Read Devin Gordon's great piece on Hollywood and AI at nytimes.com. This episode was made by humans. Their names, Peter Balanon-Rosen and Gabrielle Burbay. Amina Alsadi and Abishai Artsy Patrick Boyd and Andrea Christensdottir and I'm Sean Ramosverham and here are some more humans who didn't work on today's show Noelle King, Miranda Kennedy, Jolie Myers Hadi Mawagdi, Miles Bryan, Victoria Chamberlain Devin Schwartz, Denise Guerra We use music by Breakmaster Cylinder and Laura Bullard as our senior researcher Today Explained is distributed by WNYC The show is a part of Vox You can listen to this podcast ad-free by signing up for a membership at vox.com slash members.

26:02Right now, you can pay 30 % less than normal for that membership. So get in there while you can. Thank you and have a weekend.