War on Reality: AI Voice and Video

0:00This episode is brought to you by Forethought. Most companies build their customer experience in pieces. Sales in one system, support in another, and onboarding somewhere else. Forethought brings it all together. Forethought is an AI system made up of advanced agents that handle sales, onboarding, support, and retention. Each team manages its own agents. The customer sees one unified experience. Forethought powers over a billion interactions every month for brands like Scale AI, Cohere, Airtable, and Upwork. Learn more at forethought.ai.

0:41Hey, everybody. Welcome to the Cerebral Valley podcast on the newcomer newsletter. We're getting ready for the Cerebral Valley AI Summit in London on June 25th. Thrilled to be back with Max Child. Hello. Glad to be back. And James Wilserman. Hey, Eric. Good to see you. We are talking about Max and James's favorite topic, voice and the future of video, though I guess you guys are getting in both. You got to get the branding right. It's voice AI. OK, now voice AI and then video AI. I tweeted at you. I was like, I forget what we were fighting about, but I was like, I noticed you add the AI to the company.

1:19Yes, that's correct. We were pioneers in the field of voice AI and I've added myself to the Wikipedia. All right. We're in the Hayes Valley Wikipedia. Do you know that? I have seen this. Yes. Yes. Okay. One of our proudest accomplishments. Clearly, you know, VEO3 is crazy. Like video creation has gone insane. We want to cover a couple of things with this podcast, where we think video is going. And then we're going to review some videos. We just had a lot of technical difficulties trying to introduce an AI video co-host. It seems like the problem was that fundamentally, ChatGPT is down because they introduced new models, which is honestly the most disappointing failure mode for having an AI video co-host since just ChatGPT working feels like it should be the minimum.

2:11What are they spending the money on? I don't know. Apparently not server uptime. This has happened to us pretty often, by the way. We run a lot of our games on GPT APIs, and every 100 days or so, you get nailed by this. They just dropped the prices, right? 80 % on O3 or something? Does that affect you guys? O3 is five times cheaper. I don't think we run any of our games on O3 right now, but we do run our games on the real-time voice API, which is, I think, what we're here to talk about today. The Samantha Herr model, as we called it a year ago before Scarlett Johansson sued OpenAI. All right. So a direct question.

2:49Where do you think video is in five years? We love predictions on this podcast. And I think specifically, I want you to get to, do you think TikTok or movie studios like loom larger in this world? And do you think AI is intertwined with human actors or do you think we're pure AI movies? There's a lot of layered questions there. I think I would first say that anyone who's seen this sort of viral Google VO3 model output, you know, on Twitter, which has been very popular over the last week or two. You know, there's a new viral sort of version of this VO content where it's a day in the life of a Star Wars stormtrooper.

3:31And they're making little joke logs of how they've crashed on Tatooine and that kind of thing. And, you know, it, it looks, I think, as good or better than many scenes in certainly the television versions of the Star Wars universe. I guess you could argue maybe the film looks slightly better if you're on a 70 foot screen in a theater. But, you know, it's, it's close enough in my opinion to sort of cut to the chase where you're like, this could be a part of certainly like a star Wars miniseries or something like this. And so it seems hard to believe that YouTube slash Tik TOK isn't going to take over this world.

4:05Um, I don't know how much of this type of content you'll see on the movie theater screen, but I think what's more interesting is like, given that you now have kind of on demand video that's of the quality of, uh, you know, Hollywood, does everyone get a personalized Tik TOK feed of their own AI generated videos that are perfectly targeted at them and perfectly dispensed like their little, you know, their little piece of candy that, that they love. Are you betting that or not? I feel like that's one of the key forks here. You're betting on long tail custom video. I am betting on ultra long tail.

4:35I am betting on like the barbell curve or what, you know, the U curve, right. Where on one end we'll have like monoculture. We'll still probably have like, you know, the Taylor Swift's and the sort of ultra popular films, you know, Barbie Oppenheimer or whatever. And then most content consumption will be super long tail, ultra personalized. Everyone gets their own TikTok feed. That's like AI generated exactly for them. I think like everything in the middle is going to have a lot of trouble. Like, uh, like our podcast, you know, until we put our co-host on. Yeah. Yeah. We're probably in the middle right now.

5:07So we need to get our AI co-host taking over. All right, James, what's your prediction here? Well, I think that maybe podcasts will be safe for a while. I think it seems hard to imagine that we will have a big market for people listening to podcasts purely hosted by AIs for a long time. Because the reasoning through line isn't there? Is there just going to veer off? Yeah, I guess like I think people mainly listen for the hosts and the sort of personalities of the hosts and the relation to the real world. You're like, we're so charming. How could a no, I don't know. If we're talking five years, I mean, I guess anything's possible.

5:45Five years. Anything's possible. The Silicon Valley point of view. Yeah. I mean, I just think that podcasts in general are just not that big of a business. I mean, it's especially if we get self-driving cars, I don't think people are going to listen to that many podcasts in general. I mean, I think you already see video eating podcasts. Yeah. Yeah. We're a video podcast. I don't think there's this hard line. Okay. Well, I guess I'm saying like the concept of a talk show or whatever, right? A long form talk show, which is sort of what James is talking about. I think, I guess I really disagree. I think it will totally get eaten by AI generated content.

6:14I listened to the Bill Simmons podcast. I feel like the personal relationship I have with Bill Simmons is, is of course like, you know, meaningful because I like that podcast more than other podcasts, but I totally believe there's an AI out there that could create a better show that I would like more. And then I wouldn't care if it was a human or not. Like there's nothing quintessentially human about the Bill Simmons podcast. That's like keeping me around, I guess. Maybe I'm underestimating Bill Simmons' abilities. It's possible. Would you predict that the amount of content generated by AI podcasts makes it uncommon to ever listen to a human-generated podcast in five years?

6:48That's what your sort of prediction is, or would you go that far? More airtime for human podcasts or AI podcasts in five years? I say human. I'll say AI, if we're including all video talk shows or whatever. James, you're you're on the hardcore human side right you sure my my prediction is based on like podcast players as they are today like you know whether that's youtube or uh i'll count youtube this is some kind of edge case here all right i think apple podcasts and overcast uh everything will be human all right all right i do believe max is on the right track here when we're thinking about short form video when we're thinking of you know whether even hollywood video will be mostly AI generated, like anything fictional where you're already looking for kind of a fantasy type experience or world, like then obviously AI is just a faster path to generate that.

7:42You can still have human written scripts involved. I'm sure that will be AI impacted as well. But yeah, I think that would be where I would see this changing faster. All right, Eric, what's your thoughts on this podcast situation? I think what video hasn't had is fast iteration. And so I think the biggest, I mean, there are lots of things that are going to be interesting about AI video, but on the sort of like long tail of videos, but even popular videos like you get on TikTok, AI generates a short video, sees how it's doing and then sort of dedicates itself to videos that are getting heat. Like I think this sort of just A-B testing a video and seeing what's working, what's interesting.

8:20I think that's going to be a path for just like novel video creation and is going to be really powerful. So to me, I think TikTok and YouTube are much more important than movie studios for video five years from now. I also think the movie business, it's going to be IP wars. You know, it's like Studio Ghibli is like the top thing on opening. I like I just think that if you actually try to make a movie that went to theaters about it, you'd have a lawsuit. And I think the movie studios with the actual IP have all these employees that are really hostile to using AI to generate their IP. So I think largely YouTube, TikTok and this sort of user generated type stuff is just going to be AI generated.

9:05And so we're going to see a lot of the competition come there. Yeah, that sort of gets into a question I have about the future, which is given that the training data sets of all of these AI is sort of where we are today or where we are up through 2020. and also given that you know the only sort of rallying points for monoculture are sort of nostalgia and ip to your point like are we sort of in the like the end times for new content in some ways where like forever going forward like just the content universe of the 2020s will extend permanently into the future because it'll be so deeply embedded in the training data of the models that like sort of became you know became agi or whatever you want to call it that like you know the 1990s and 2000s will be like weirdly important for the next 200 years because it's like the last original content on the internet that the internet had killed sort of generations or that it was so self-aware that you didn't have this like we all dress this way until we all you know yeah and so that banding was already sort of on the way out we see with new technologies does open up the opportunity for new creative ip right i mean mr beast didn't exist before youtube and you know mr beast the ultimate derivative he's doing the hunger games yeah i've kind of i've got it with eric here i think mr beast is gonna not not really uh last the test of time but i think like obviously that that era of the internet and youtube like created new stars that wouldn't have existed without the internet right so i guess i could imagine something similar happening with ai where there's new types of musicians right that are fully invested in ai there's new types of filmmakers and directors that are like go 100 % in on AI maybe new podcasts I mean right now we're constrained in a couple ways like one VEO3 what it's like eight seconds at a time so people and humans end up having to do a lot of the work because they're connecting all these different sequences if they want to do it so we're not even really seeing true like AI creation, right?

11:15At scale. And then the other thing, Sam Altman just wrote a new blog post. And I think he predicted that innovation or novelty was like an upcoming feature of AI. And so like in our five-year window, I have a lot of optimism that novelty could be coming. I think it's the big question. And obviously if there's novelty, then there's new IP. And then we're like, wow, they came up with the new Disney. Like, this is perfect. You came up with Pokemon in 20 seconds. Like that would then blow us up. Then we're like every movie, it's like, we get high on it. I mean, we could even have this period where it's like, we're so inundated with it.

11:53And then we sort of pull back and we're like, oh, we're fatigued of the AI movies. That starts to ask the question of like, does it become a monoculture or is super hyper-targeted and everyone has a different Pokemon that they're watching, right? I, at the end of the day, I know this makes me an old, but like I, I, I watch Tik TOK. My wife watches Tik TOK. We're lying in bed. We think we're so unique. The Tik TOK comes up and it's like, Oh, it understands me so well. Then she gets the same one. Like I think sometimes the super niche Tik TOKs are like, actually, no, there's a lot of common humanity.

12:24Like we're all interested in the same things. Like I think some of the long tail stuff is overrated. Obviously we have hobbies and it's like, you know, chess or league of legends or whatever. So I'm not denying that, But I think there's always going to be this urge for a shared culture. Yeah, I think you're probably right. I think the ultimate question is how many different human interests really are there? And like, especially ones with any sort of density, like more than a thousand people share that interest or more than 10 ,000 people share that interest. And there's probably not that many, maybe on the order of hundreds or thousands.

12:56So ultimately, there might be just a convergence around that. And maybe the tail isn't that long. But I do think like Andre Capathy, who made like the original vibe coding tweet, had an interesting tweet this week about VO3 where he said this will be the first time in history that video is what he called like directly optimizable meaning like you could just run like millions and millions of iterations on the same video you know and keep tweaking it for almost no cost whereas historically tweaking video has been sort of prohibitively expensive to to do more than a few variations or more than a few kind of iterations and tests and so video is going to become a lot more like, you know, text or something like that, where you can iterate and test it so rapidly that you can converge on sort of the like optimal solution for a given problem.

13:44Even if like the optimal solution is just like a very particular version of a meme or whatever. And so it'll be interesting to see what those shared pieces of content look like in the future and what the sort of hyper-optimized version of something that, that was merely popular when we were saying it's cheaper. Not yet. But like, there's going to be a lot of costs. Like I do think another reason to be bearish on hyper individualized video in the next five years is just everybody's like, oh, wave away costs, wave away costs. Actually, it's like, ah, make one expensive AI video for everybody in the beginning.

14:18And then once we get costs down, you know. Yeah, for sure. I do think five years is such a long time in this AI era that we live through, though, that it's really hard to predict where costs end up for anything. I mean, like we just said, OpenAI reduced costs for 03 today, 80 % down from what it was yesterday. So I think things just changed dramatically. I think there will be a successful Pixar style AI company. Like I think if we look at MidJourney, part of why MidJourney has been successful is it did sort of, it's like, okay, we like this design aesthetic. They're sort of playing Kingmaker on certain looks to make it work.

14:56And so I think there will be sort of like AI studios basically trying to make movies. Given that your wife is in the filmmaking industry, what is your sense on is Hollywood going to adopt these tools for special effects or for certain shots or to edit certain things? How much blowback is there against it versus how much excitement is there? My wife is in documentary. I think she's a little more open to it given what I do. But certainly in those circles, you can feel like they're way, you know, go to their parties. They're way more hostile to AI than beyond fathomable to us. So, yeah, there's definitely a lot of hostility in Hollywood.

15:35I mean, Tom Doton has been doing some reporting for Newcomer. And I don't know, my sense from his story is it's like Hollywood secretly, you know, the execs are going to use it. It's going to happen. And there will be this sort of clash with the rank and file. Well, I mean, documentary, given it's about the real world, you know, has its own reasons to be sort of not as AI generated, but it'll continue to create sort of this blur between real and fiction. And I mean, we kind of talked about this earlier, but like, I already think a lot of this AI video looks more real than like an effect shot in a Marvel movie.

16:09And I think Hollywood obviously moved in the direction from practical effects in the 70s, 80s and 90s to 100 percent CGI for the most popular movies today. And I just think that it's sort of inconceivable that you wouldn't take this technology that lets you create at least like effect shots for like one one thousandth as much money, like one one hundred thousandth as much money and like probably looks better already in 2025. But it's still hard to control the camera, right? Like there are people working on that for sure. We're really close, I guess. I have a question. I just think that we're dancing around that we haven't gotten to, which is just like we're predicting that all this video is going to be generated with AI.

16:49We're predicting that Hollywood studios will adopt it for special effects. I'm still kind of unclear. What what will the human role in any video production be? Will it be writing? Will it be acting? You know, someone's likeness, their voice? Will there be anywhere humans are part of the process? I think storyboarding will, I mean, I think the script might hold on the longest or like ensuring coherence across the script. I think that's where I'm at. James and I have been making a game and definitely today the script is the thing that is the hardest to get the AI to do a good job on. Like we're doing an opening cinematic scene for our game.

17:30And James and our content team have devoted weeks and weeks and weeks to iterating on the script and reshooting the script and everything. so it does feel like the current thing that's not good but you know if you throw the script into clod 4 or 03 and say hey like clean up the script and like give me a better version of it like it gives pretty good notes and pretty good feedback so maybe it can't hold a coherent script across 120 pages today but it's doing pretty well at a six-page script right now in terms of at least like editing and understanding the narrative arc and everything so i just and And does it have like payoffs and stuff later and like the sort of classic stuff?

18:05You can ask it for payoffs. Yeah, it's kind of like you as a human have to know what you're looking for and then it gives you good ideas for these things. But yeah, it can't do everything itself. But I do think that's key. You're like, oh, does it have the key element of story that like there's some emotional payoff or something? You're like, oh, hand wave that away. The humans will make sure. Like that's what I'm saying. Which is why short form video maybe. You're saying it's like a script producer or something, or it's like the editor of the script rather than the writing. I don't know. A year ago, it was really, really effing bad at this.

18:39And now it's like getting kind of good at it. And so you're like, I see where the curve is headed on this. I don't know. I think I think a year from today, it could be pretty good. I have a prediction that within five years, video games will be completely AI generated at the pixel level, effectively using video generation models that are interactive as opposed to current strategies to render in a game engine 3D environments. Just for context, the way most games are made today is inside a game engine. You put a bunch of shapes in an environment and walls and light sources, and then you try to figure out how the light bounces off various things and what camera the player is looking out of and such.

19:20You're saying the future is we're basically just going to take VO3, but like it generates the whole environment in real time all the time and so instead of watching a little eight second clip of something funny you can walk around the world that vo3 has generated for you and it'll all just change as you're walking around doing this with minecraft already right i mean i've seen weird demos of it but it doesn't seem real yet yeah they're they're doing demos where there's minecraft levels being generated in front of you as there's anything What do we know about demos that every tech demo that has ever been given has turned into a real product that someone can deliver?

19:57I don't think it'll replace everything. I think this is the classic, like, all or nothing false dichotomy. I mean, I think it'll be interesting. It doesn't seem like the Switch just came out. First of all, like, Nintendo is, like, building games on, like, five generations old technology, it seems like. It's not even, like, OLED on the device. So I just think you're like overestimating how quickly an industry that's not AI will come for it. Like new, I think new companies fundamentally and new studios will have to create these games and they will have to disrupt the existing studios. I don't think the existing studios will necessarily embrace it quickly.

20:32And the existing studios have IP. And so it's going to be a slow transition. Five years is really fast for this kind of thing, to Eric's point, just from like a business model and technology perspective. I also just think that I've seen all those demos that you've seen of people generating Minecraft with video and like it's not clear to me what they're better at. Right. OK, they're more expensive. They're slower. The world looks like crap. The walls change all the time. Like, you know, there's no other players in there. There's no game mechanics like they're worse at everything. Right. OK, that doesn't mean someday they won't be better at something.

21:00But I've never like heard the pitch for why it's a better way to build a game world. And you can do anything. You could do anything cheaper. You can prototype everything. in seconds. You can have any game you want, right? But you can do anything in a game engine. You can drop whatever shape you want in the game engine. You can make any character sprite. I mean, maybe it's easier and faster someday, but it's also extremely non-deterministic, right? You're like, hey, I want a wall there, and then sometimes the wall moves or whatever. I don't know. Again, I believe there is a world in which they're better at some things, but I've never heard which game you would want to build in this thing, even as sort of an imaginary awesome version of it.

21:38It might come back to this like long tail argument where you can make custom worlds for every player and somehow that's more interesting than you know a more structured game world that you know everyone lives in the same one i mean fortnite is a huge game even if there is one game that is made this way that is like sure it's the everything game because it changes you know as you develop it as you go through it through it you know but that's a much smaller claim there will be one super intense game where you can do this or maybe there will be a whole category i just think if It's technically possible.

22:08It seems like it will be better than waiting, you know, five years to build a new GTA or something. It'll just be kind of laughable that you would invest in a studio to do that. I don't fully agree. But one thing I will say is like, and this isn't games purely, but if you like think about Harry Potter, right? And they're psyching themselves up for this like series on Harry Potter. But there's so many assets. Fans have already created so much. It feels like given a little bit of AI juice, like the fans are ready just to create their preferred version of Harry Potter and all the like fandom can fight with itself about exactly like how everybody should be portrayed.

22:48And I don't know. I think that's almost more exciting. I think this Harry Potter series is coming at the worst time. Like this is the time where the fans are about to be empowered to really make a full length series themselves. And I'm super excited about that type of thing. Let's dig into some actual real videos instead of just imagining the world that could be, the world that is, the insane world that is. Let's pull up some videos and get your reactions. All right. We're watching an AI video from seven years ago. You won't believe what Obama says in this video. Wink. We're entering an era in which our enemies can make it look like anyone is saying anything at any point in time, even if they would never say those things.

23:37Moving forward, we need to be more vigilant with what we trust from the internet. That's a time when we need to rely on trusted news sources. My first reaction was this was better than I thought it would be. Better than I thought it would be from 2018. Yeah. It had that effect of the mouth moving off to the side because it basically was like someone cut a hole in Obama's mouth and then just like put someone else's mouth in the video. I think that's kind of one thing. It's a voice actor. You're like, OK, that's not AI. So the TTS voice wasn't there. I mean, it was it was pretty good. It comes back to our point about the monoculture versus the long tail here, which is Obama, obviously very monoculture figure.

24:14It was worth the investment to cut a little hole in Obama's face and have him make silly things because you knew that was going to go viral. Whereas in the modern era, we can do that for anyone. I'm sure Buzz would spend a ton of money on this video. More than you would think, probably. All right, next.

24:33Ah, that's hot. That's hot. Uncle Phil, come try this. Fresh pasta of Bel Air.

24:45why do we think that this particular will smith eating pasta video went so viral in 2023 i know now it seems like who cares i mean this is definitely the argument for like wow things have really improved this is a horrifying and disgusting video and i think people enjoyed sharing that look how terrible ai is i feel like there might have been an element of dunking on how bad in some ways it's like this badness is what's useful about it because it suggest OAI is not going anywhere. All right, now we're moving on from old BuzzFeed videos and the stable diffusion era not too long ago to pretty recent history, Sora.

25:31Yeah, so there's two dogs podcasting on what looks like an alpine hillside or something like that. that one looks pretty realistic. I mean, I would say like, I don't think it passes the Turing test as it were, where I'm like, that looks like a real video. There's just enough issues with it, but you know, it's, it's starting to get into the era of like, oh, this kind of looks like, you know, a realistic image, I think. Whereas obviously what we saw before is nowhere close to that. What's interesting to me also is that they had these watermarks on there, which I think now they've kind of allowed you to remove the watermark.