Robot Cinematographers: Is This The Future of Filmmaking?

0:00In this episode of the DeNoise podcast, we're going to talk about robot camera operators, The latest on AI copyright battles. And an innovative new tool for virtual production from Stage Precision. Let's get into it.

0:14All right, welcome back everyone to Denoise. Hey, good to see you. Good to see you. Welcome back, everybody. So first story, I had texted you an image that was going around the other week of Kim Kardashian doing a photo shoot with Tesla robots. And one of the robots was holding a either FreeFly or some sort of camera stabilizer. It looked like as a prop. And I was like, ha ha ha, look, the robots are operating the camera. Ha ha ha. Joke. Now that's not the case anymore because we just saw a demo, not from Tesla robots, but from WPP partnership with Boston Dynamics. They released this whole video showing how they were using a humanoid robot as a camera operator on set.

0:47So that is direct job replacement of a cinematographer. That is happening right now. They clarified many times in the video. It is not to replace. It is to augment creativity. But the robot was the only cinematographer on set. I didn't see a real one. The robot was the camera operator. Okay. All right. So let's break down this video, but also let's back up a little bit because robots in film production are not new. No. We have been using modified industrial robots, the robotic arms. Like the KUKA or the Bolt. Right. Mark Roberts motion control, a whole fleet of robot arms. We have a whole video of Steve Giralt from the garage.

1:18He has an army of these robot arms. They are great. They can move very fast. They can do repeatable movements. They're used a ton on advertising shoots. They kind of got to resurface recently because they shot the opening sequence of Severance with a variety of these Mark Robert motion control robots. So, and then the behind the scenes clips started going around the internet. And some people are like, oh, robots, cool. Yeah, my favorite use of robot cinematography was on Kendrick Lamar's humble video where it does the quick, you know, one, two, three, four. And these things are just not humanly possible.

1:51No, you cannot do that. Anyway, the super fast movements with a high speed camera, you can just do really crazy moves. So taking this to the next step is a humanoid robot. Right. And so WPP had been using the Boston Dynamics, more famous for their dog looking like robot. That's also a bit terrifying. Yes. In the way it moves. And there have been adaptations of putting cameras on the dog robots, both for creative uses, but also there's a whole fleet of practical industrial uses for putting cameras on robots for non-cinema things of like surveying or sending robots into dangerous things that are too hazardous or dangerous to send a person.

2:25Yeah. They've been putting cameras on the dog robots with Boston Dynamics and then Basta Dynamics was like, well, hey, we've got this humanoid robot. Right. Let's see what we can do with that. And I think they have a new generation humanoid robot. I believe it's called Atlas. They released a video of it recently. And Joey, when I was watching that video, that robot was so elegant and graceful in its movements. It wasn't hunky-junky. You know, it was just like in control of its balance and just precise. I thought it was an Unreal render that was composited on top of the light plate. Yeah. I was like, this can't be real.

3:00This robot is just too good. Yeah, it was moving smoothly. And so the demo video was a robot on a virtual production stage kind of shooting like a fake car commercial or something. And the robot is holding a free fly movie rig. Yeah. Stabilizer with two hands. Yeah, two hands. Just give it the stabilizer and it's just moving around and filming. Wasn't quite clear if it's completely autonomous, if the move was programmed if there's like a remote operator who's operated in real time that wasn't quite clear and i think that opens up the questions too of like how could we potentially use this and like does the robot does the robot know what it's shooting and is it looking at the viewfinder or is it just right walking an arc that i'm gonna guess this is all probably programmed i'm gonna guess too it's just too early for that i mean they did talk about too that they're using the NVIDIA real world training model, which we kind of seen more for like car applications, but it's just a model that's an understanding of the real world.

3:58From my understanding, Cosmos is very applicable to humanoid robots because it replicates basically a human vision system. If you and I look around this room, we recognize every single object and a robot is supposed to do that, but where is it going to get the sophisticated AI model to pull from? And that's what Cosmos is. having said that is cosmos aware of led tiles how a film set works right how a film set what a viewfinder is uh-huh dolly track don't chip on dolly track right this is the this is on camera this is off camera this is behind this like knowing you know something that a crew person knows was like don't be in front of the camera because you're gonna be in the shot film sets are so complex especially a vp film set that you would need its own ai model just to decipher that Like you would need a Atlas robot that's specifically trained on Cosmos film version of AI.

4:47Yeah. And maybe that is what they're doing because they said that they are training or working on training additional models or building on Cosmo. Interesting. So maybe that's what it is. So you just feed the special data set or training on film set specific stuff. Yeah. But where you get all the BTS footage of film sets, you would need thousands of hours of it. Or I don't know if that's what they're doing by having the robot on set and running through it. I don't know if that's what that means. Oh, gotcha. I don't know. Okay. Maybe. I mean, if they want to tell us more about it. I mean, taking a step back, this is really cool.

5:18It's cool. So at first when I'm watching this, I kept going back and forth between like, this is dumb and this is cool. And this is dumb and this is cool. So in my dumb category, yeah, please elaborate category with a lot of applications, like humanoid robots are kind of popular because they look like us, but they're not the best form or shape for doing tasks that we're asking the robots to do. So when I was like, Well, a humanoid robot seems kind of silly if you're trying to do something that's like a dolly track. Like a robot with like wheels or something seems like, oh, maybe that would make more sense if you're trying to replicate something that could be on set and replicate a dolly with like arm movements.

5:54Yeah, like bolt is on some kind of track or wheels. You can't put it on a track. You can put it on wheels, right. So imagine the bolt bottom with like a humanoid top. That would make way more sense on set. Something like that, yeah. Or something bigger or something different. it where it's like i guess it depends on what the shot is but then i'm thinking more it was like well we're going down these rabbit holes of like these specific use cases right the humanoid robot we know because all the gear is already built for human hands exactly so you can just bring it on set like with the bolt you gotta forklift that thing in or some other like use heavy machinery you gotta bolt like if you're doing the tracks then you have to like bolt it to the ground yeah in some cases it has to be super level and it's massive and heavy yeah this thing i mean it's going to do the range i don't think it could do the same speed as a uh a bolt but it can hold up to 20 kilograms which is about 44 pounds that's a huge payload yeah and it can hold it in awkward spots that for a camera person with its arm out where a human can't probably do that most humans can't yeah and for long periods of time and it's a robot so it could have programmable repeatable movements and it could i'm thinking in theory also being extremely still so it could tippy toe left or right.

7:00So you can mimic that dolly movement with the robot and still have it locked off super solid. I wonder what the charge time and sort of the battery life on this thing is. Yeah, I know. Yeah. I mean, presumably too, you probably run a cable into it. Oh, yeah. Yeah. If you're on set or something. Yeah. So I was going back. So I'm like, okay, well, actually that does make sense. I saw another shot too in the video where the robot was holding the Leica Geosystems RTC LiDAR scanner. Brilliant. Which we have a video coming out with a whole deep dive of the scanners. And so I've learned a lot about them too.

7:28At first I was like, wait, that seems silly because the LiDAR scanner, you don't move it, you just hold it in place. But then I'm like, oh, that actually could make sense. It's a robot. It can stay completely still. To scan a huge environment, you have to do multiple setup positions. So you have to have a person with a tripod, plop it down, scan, move the tripod. The robot, you give the robot the scanner. Walk, walk, stand. You're like, hey, here's the area you got to cover. It just walks, holds it up, freezes for two minutes, scans, moves to the next spot. Brilliant. Scans. Yeah. So I'm like all right that's cool so i i am on the same page with you uh i was watching the video and i was like this is stupid that's cool this is dumb that's cool so i i had my back and forth as well yeah so what are you seeing with this yeah so what came to mind i'll just get the dumb thing out of the way first like the robot has no freaking clue what it's pointing the camera at so that is the biggest setback yeah a cinematographer is really valued because of the creative decision that they make with composition, blocking, framing, all that stuff.

8:25So that's completely missing. And the bit from like Rob Legato goes in my mind too, where he went on, like in the multiple interviews I've had with him, where he's like talked about the artistic finesse of the camera operator, even in a virtual environment. Yeah. Like every step, every move, every punch in is intentional and it ties back to the creative and the story. And it's beautiful how high level cinematographers do it. Having said that, what is cool is let's say you're a YouTuber and you're shooting outside at the Century City Mall and you have one human cinematographer, but then you need coverage, you need four or five cameras, dude, get like three of these robots and just have them dumbly point at you and follow you.

9:06I mean, these would be great for our NAB coverage. Yes, exactly. I got Justin, my camera operator, but if we had, yeah, a couple other cameras, just follow us. Yeah, and even as you're walking the NAB show floor, you just have two robots behind you that are just constantly pointing at you and they're targeted to you, right? Yeah. That's easy coverage. I think there'd be a constant horde of people around watching you today. Yes. But five years and after that, I'm telling you, this is going to be normal. Yeah. So that's one. The other thing I was thinking of is like when you're in a concert venue, big, let's say the O2 arena, and you're shooting up at the stage from, typically in a concert venue, you have like five, maybe 10 camera operators.

9:46and all they're literally doing is just getting coverage of the star that's performing robots just put maybe half of them way back into the audience and because they're on their feet and not on a complex you know track system or dolly or whatever they can just walk to the venue wherever they you know seat 17b and just from there point their camera and do their thing yeah it's like the next level of ptz cameras the pan tilt zoomer it's like a remotely app controllable camera but you can only move it as far as your tripod wherever it's mounted to yeah but if you have that on a robot yeah and then same thing that we were talking about before with a concert venue yeah there's no if it's a humanoid it just shows up you don't have to lay track you don't have to do any like crazy builds yeah in the venue for it you just unpack your robot and you know how exhausting it is to like sit through like a four or five hour concert with like a 20 pound 30 pound camera pointed at the stuff like that's insane so a robot can do that easily and the other thing you're saying with the Leica scanner is this robot is both a tripod and a person.

10:49Yeah. Because it can stand still. Yeah, it can freeze. Yeah. So, if you have a really heavy, complex cinematic rig, what it could do is walk around with you and then just hand it to you. You shoot it and then hand it back to it and it'll hold it indefinitely. Yeah. It'll be the new, the Steadicam assistant, come hold my Steadicam rig while I'm taking a break. There's definitely a place for a humanite robot on a film set. Yeah, this is interesting. I was thinking too, when we're talking about like the, you know, it's not understanding what it's shooting. Is it someone operating it? I mean, another potential use to that merges the humans and the robots is we do something with like the phone where you like record your shot, human operator of like, this is the shot I want to do something that we kind of do in a virtual environment.

11:32I love that. But then the robot just replicates it. Yes. And repeats it repeats it over and over just do the movement with your phone or your ipad or something with with sensors and use a natural language model to instruct it to say do that slower do that two feet lower da da da da and it'll just keep going yeah all right now i'm brainstorming now all right now the idea now it's way cool now cool is high dumb is low see all right i judged you too early wpp yeah this is very interesting very curious to see where this goes and the robots and stuff we've talked about a bunch and it's just funny to see this come out now after we've been talking about this on denoise there is no shortage of robot coverage.

12:07So stay tuned for more robot coverage. Next story, kind of a grab bag story is coming back to copyright law in the US. There's a few things going on here. One was OpenAI, Google, and a bunch of other AI companies petitioning the government to loosen copyright laws to basically put usage of copyright material into AI training into the fair use bucket so that basically they could train on copyright material and not, and it counts as fair use and not have faced these existing lawsuits or future lawsuits that they are facing. Right. And then on the flip side, there was an open letter from 400 or so people in Hollywood petitioning the Trump administration to protect copyright, existing copyright on creative material that the US has.

12:49So it's like from both sides. Yeah, basically have the companies being like, hey, let's loosen up copyright law so we can train. Yeah. And they framed it in the sense of like, we need to be competitive against China because China is training their own models and they don't have the same protections or relation with copyright law that we do in America. And the Hollywood creative industry is like, we've kind of dominated the world culture, you know, through human creativity. Yep. We need to protect that and not just like give it all away for free to AI companies. I mean, this is such an interesting time we live in and I kind of agree with both.

13:22And I also disagree with both. Yeah. All right. So where are you at them both? Okay. So first I'll start with the open AI Google ask is a loosening fair use law. So, quote unquote, their models are ethical and they can scrape the internet forever and not be penalized for it. Their reasoning is, well, China's doing it. China's going to be ahead and they're going to beat us. But then my argument is like, OK, so China's current human labor laws are probably terrible and they can make an iPhone cheaper. so is a phone manufacturing company in America saying like hey let's delete OSHA and all the federal laws that have made for a humane work environment because China is just doing it cheaper like you can't compare a country that has no respect for law and just general common sense rules just to make your stuff work at that level you have to fundamentally believe that the The laws and regulations in the United States exist for a reason, and it builds the society that we thrive and prosper in.

14:23And because of that, we are then further incentivized to be more creative and better at our job, and we'll have a superior product. That's my take on it. It's a super philosophical take on it. But I finally think that OpenAI and Google is in the wrong here. But at the same time, I could see their point after DeepSeek, like, what's next? What is China going to come up with that's going to completely disrupt what we're doing? I'm in the middle of it and it's new more nuanced than a yes, no question of like, should we allow a copyright for free or should we like have to pay? Should they have to pay the same as if they were like a streamer licensing the content or something?

15:00Right. Which would probably be cost prohibitive or too expensive for what they're trying to do. I feel like there's more nuance in this because there's multiple parts of AI and it's like part of it is like, okay, just the training and the training soup or equivalent of like, okay, just they just throw all the data into their model. and train it. And it's just kind of like a big bowl of soup. And you can't really pick one element out. It's just all thrown in there. And the more you kind of throw in there, then predictably, the better your model is going to be creating stuff. But then there's what you do with that of the output.

15:28And so there's like, and we talked about this before of cases of like, okay, you're kind of just brainstorming or previous or you're thinking, and you're just like spitting stuff out. And you're not going to really do anything commercial with it. Or you're not, you're just kind of never going to be final pixel. Yeah. And you're just kind of brainstorming with that. And that's one case. But then there's the other case where it's like, okay, I'm trying to spit something out and i want this to be final pixar i want this to be like a shot that i'm going to use in a final film and then there's like there's the gauges of what's in your prompt and we've talked about this before but of like is it something that's you're just bringing your own creative prompt and you're trying to adapt something that's your creative vision and help speed up the process to spit something out or are you doing stuff like give me a pixar style like animated character and you're like using very established creative intentionally stealing or copying or just laziness of like using someone else's ip characters to use the ai to spit out it could be a batman short film right and you're spitting out existing ip which would if you were to do this by hand and try to like let me hand draw my own batman cartoon and try to sell it you couldn't because you're violating copyright law well there is the parody law yeah i miss a a non-spoofy bat like i'm trying to do a legit bat yeah i'm trying to do like i'm just gonna draw Batman as a legit Dark Knight character and try to make my own short film.

16:43Yeah, you would get in trouble. Yeah, it'd be like, hey, yeah, and it's not a spoof. You're violating copyright law. So there's so many, I feel like there's like, oh, okay, there's all of these nuances and applications of what you can do with this. Where, who should pay and where should you pay? It's more nuanced than, okay, they should just pay to train on the data. I'm thinking like my, okay, this would be great idea or this would be better is like if there's better tracking of like what goes into the AI training soup, and then some form of like token call out or something where it's like, if you start calling out Pixar or specific words, you got to pay more for that.

17:15And pay Pixar. And pay Pixar, right. Some sort of trackable licensing. I don't even know if that's possible, but some sort of like being able to have better content authority or track like what goes into the soup, what comes out of the soup. And I think as, I mean, it would be like pennies are, you know, like Spotify streaming or something like that, but something that's tracking and seeing like whose material is getting used or and how can you compensate for that there is something that is in its infancy adobe introduced that is akin to what you're saying the content authority that's it yeah yeah that was also to on the flip side of all of the deep fakes and other issues to prove like that something was legitimately shot like yeah a partnership with like canon and other cameras manufacturers to have a certification embedded in the footage it's the blue check mark equivalent of AI.

18:02Yeah, like this was shot with a real camera. This was unaltered footage. Yes. So that's in the right direction. I know it seems complete. Right, and that could be the footage and that could be tied. Right now it's to prove that the thing was authentic, but that could also maybe be if you did want to train or license your footage to train, it's trackable. Yes. That came from you and your authorization and it's good to go. Yeah. It seems super impossible right now to picture a world where everything you just said about tracking the training soup and then tracking the prompting and the inference, it seems impossible right now because we're in its infancy.

18:33We're in the infancy, but also we're like two, three years into this and cats out of the bag, Pandora's box is open. Yeah. Yeah. I think even a year or two ago, the ad companies were asked like, well, can you like, if like the New York Times is like, we don't want our data in there. Can you take it out? And they're like, no, it's GPT 3.5 or whatever's already built. It's being used by millions of people. Yeah. And it's not like, oh, we can just pull your data out of the training set. It's a soup. So then this Trump petition thing for the Hollywood elite, it seems like maybe it's too little too late, right?

19:09Like, it's just a show, song and dance. Yeah, I mean, I understand the concerns. I mean, on the flip side, I'm thinking too, it's like, well, the studios that own your content are probably trying to make their own models too with their own libraries that they control. Like if Ben Stiller shoots a film for Warner Brothers, Warner Brothers owns the likeness of Ben Stiller in that film. Like that character. Yeah, like Zoolander, whatever. Yeah, yeah, yeah. And then if Warner Brothers wants to make a Zoolander, it's a terrible example, but a Zoolander AI model. The blue steel look. The blue steel look, right?

19:43They have every right to do it, and Ben Stiller probably can't say anything about it. Right, I mean, obviously these all go down to the details of their contracts and stuff in each individual person. But yeah, for the most part, the studio owns the IP of the character. But then what if Warner Brothers licenses Zoolander amongst a hundred other films to open AI to create a film and TV chat GPT model? Warner Brothers makes money by providing the training data. Open AI get legit Hollywood training data. So what's the actor supposed to do in that case? I don't know. Right. Exactly. At this point, too, what benefits would the AI companies get from accessing the copyright material that they believe a lot of them have sort of trained on copyright material anyways?

20:29And I don't know if this is like a ask for forgiveness kind of deal, because it's like, well, what other with the letter with OpenAI and the other AI companies saying that the copyright law should be changed more in a fair use? What other stuff are they looking to get and how much benefit is that going to add at the point that we've gotten to now? That's a great question. Like, what is down the pipe for them that we can't see? yeah like what else yeah what else how much better will this stuff get or what you need by having access to some to some of the stuff and even if it's like a final edit movie it's not really going to be a benefit to you and that ties into another one of the pieces puzzle the parallel that i'm making here uh it's kind of a stretch but hang with me okay so in the early 2000s you and i are probably i definitely was downloading stuff on napster and what and torrents and stuff right yeah music so uh as a youth i was transitioning out of physical media like cds into early digital files super legal right and but everybody in my age group they were doing it because uh we had a very crude mp3 players at the time our cars that that unit we just swapped it to mp3 units and so like one cd can now hold a thousand songs instead of 10 super exciting times as a teenager and uh we knew like it was sort of wrong and you know but we were just kids at the time fast forward to today their physical music media is like it's niche spotify apple music youtube music uh these pretty much expected you could just listen to anything right any music that ever happened all the stuff that you mentioned with like rights management licensing, it's been worked out for the most part with something that was as wild as early streaming and torrents were.

22:12Now, the issue is Spotify is not paying their artists well, right? We all know like the artists made way more money with physical media and all that stuff. So Daniel X, CEO of Spotify came out and said recently something to the effect of paraphrasing here is like, look, you guys love to blame Spotify for not paying artists, but we pay the record labels very well. And the record labels are the ones that license the music and own the rights to the music, not the artists. So if the record labels are not paying the artists, then we take it up with them because they're taking more of the cut than they...

22:46Kind of a fair point. So that goes back to what I'm saying here with Ben Stiller and then Warner Brothers and then OpenAI, right? Like they should take it up with the studio to negotiate a training deal or something on their behalf. At some point, the studios will turn themselves into training data gold mines. I mean, I think this is also, I know this was an issue with the strike and this is probably just be a future point in contracts and deals and stuff of like, okay, we make the film too and you're on the rights, but training on AI data is like another revenue backstream, just like licensing to Netflix or VOD or all the other revenue backstreams for movies.

23:20Training, licensing data will probably be another stream. You take a show like Friends, which has thousands of hours of footage and let's say Netflix wants to make the next generation of Friends, can it build an AI model from the existing Friends data set and then just start to output some insert shots here and there just to kind of fill in the gap and lower production cost? Probably. And is the Matthew Perry estate and Courtney Cox and all those people who are in Friends are going to have issue with it? Probably. Would they have any power to change that? Probably not. Yeah. We got it. Yeah. So these are, these are things that are going to come up on us very fast.

24:03And it's a world where visual media is not there yet where music and the audio industry was there 10, 15 years ago. Yeah. And I know, yeah, different actors have had different responses to it. And I know there are some that have leaned in and have created or licensed their likeness to, I know there are some with the voices that license their voice to, I think it was like one of Meta's AI voice chatbots. And like some of the actors licensed their voice to that. I know, I believe too, not necessarily with AI, but digital likeness and also sort of having like a, like a full body scan as kind of insurance.

24:38I remember like after Paul Walker passed away and then finish the fast, whatever that number seven Fast and Furious movie was or six. And just so the estate can keep making money. Yeah, but also just a way to, I mean, in that case, it was like he passed away and had to finish the film. And I believe there was like a whole process where they kind of sourced because they kind of did some digital replications like with his brother and then some face swapping. But this is like years ago and they had to like pull, dig up material. And so I believe now it's like, okay, actors, just big name actors have a full, like let's have a digital scan that the studio wants to have as like that insurance.

25:10And I think that'll be the next step up to the actors and the agreements of like, what's licensable, what's trainable, like and how they're gonna be compensated for that. Yeah, if I was an actor, I would absolutely have my digital scan done and be in such high quality that if I pass, that could be used to generate revenue for my family. Yeah, that too. Yeah, it's like capture it all, save it now, and be able to have something that could be monetized. Like look at Ian Holmes. Ian Holmes was in the Alien Romulus movie, you know, as the, like, well, I don't want to give the movie away, but he has a major role there.

25:43And they used his body scans and facial scans from Lord of the Rings to build all of that. He passed away a few years ago, but he was in the movie. And I believe Ian Holm's estate profited from Alien Rhymes. Yeah. So there is another story. That's why I've been scrolling because I'm trying to find the name of the company. So there are some companies who are trying to be data brokers from creators to AI companies to source together footage that people are okay with and license it to the AI companies. One of the companies is Trovio. I think one of the other companies is Moon Valley. and yeah moon valley just recently dropped that uh clean ai model okay okay uh that esterea films is using supposedly it's a 1080p native like it's super high quality models so i'm really excited about this company moon valley yeah and so they're going more towards uh youtube creators and licensing the footage the interesting thing was the stuff that's usually more valuable for the companies is not necessarily the published youtube video but the raw footage the b-roll all the b-roll clips all the outtakes you know because it doesn't have edits it doesn't have text on the screen right it's just good raw high quality like 4k data yeah and that's what a lot of the companies are really interested in i mean for like every every uh take in a film you and i know this well there's probably 20 takes on that media card that was never used exactly yeah so it's not necessarily I mean, and obviously that is sort of a bit of a security proof too from the AI companies just going online and scraping stuff and training anyways.

27:19It's like, okay, well now that's kind of another, literally a paywall where it's like, well, if you want the good stuff that your AI really wants of just like raw, unedited footage, that's not going to be available anywhere for you to scrape and try to beg forgiveness later. Like you're gonna have to pay up for that. And so that could be another middle point or revenue stream for, I mean, it's definitely revenue stream now for some creators. apparently it could go from anywhere from uh they only work with creators that have at least like a thousand hours of uh footage yeah and it can go i thought it was kind of actually like that's surprising a dollar one to two dollars uh per minute or as high as six dollars a minute that that could be some serious money there yeah i mean it's not uh i mean like during the beginning days of the internet who would have thought data would be the biggest gold mine right like where you and i eat shop drive to like all that data that google and apple maps is collecting that they're selling that to advertisers, I'm sure.

28:08Same way, you know, who would have thought that generative AI training soup would be the next goldmine? Perhaps it will be. Yeah. Yeah. So yeah, there is a middle ground here where it's like people are paying to license this footage. Yeah. And the unaired stuff is safety. So yeah, maybe with Hollywood movies too, it's like, okay, well, let's take all of that terabytes of high quality raw data. Yeah. And that could be another revenue stream. Those studios have like these giant magnetic tapes backup storage that, you you know, magnetic tape to like go back 10 years. You have to like rewind the tape for a long time, but it's all there.