In search of the perfect movie recommendation

0:00When will AI finally make work easier? How about today? Say hello to Gemini Enterprise from Google Cloud, a simple, easy-to-use platform letting any business tap the best of Google AI. Retailers are already using AI agents to help customers reschedule deliveries all on their own. Bankers are automating millions of customer requests so they can focus on more personal service. And nurses are getting automated reports, freeing them up for patient care. It's a new way to work. Learn more about Gemini Enterprise at cloud.google.com.

0:39Welcome to The Vergecast, the flagship podcast of interest clusters. I'm your friend David Pierce, and this is the third and final episode in our series all about AI in the real world. We talk a lot about AI right now, including, obviously, on this show all the time. but it somehow all feels really abstract and big and kind of jargony and technical most of the time. So we wanted to find some places where AI is actually showing up in our real lives. So far, we've talked about using AI as a tool to give everyone perfect memory, and we've talked with an artist who uses AI to make us think differently about privacy and surveillance and life itself.

1:19Today, I want to talk about movies. And more specifically, why it's so hard to figure out what to watch. This is a problem incredibly familiar to anyone who gets their entertainment through streaming, right? You sit down, you turn on the TV, and then you spend somewhere between 2 and 2 ,000 minutes looking for something to watch. No matter where you look, it's just stuff all the way down. So I open the Netflix app, and the first thing it shows me is Mucho Mucho Amor, The Legend of Walter Mercado, which I feel like, no thanks, probably not for me. No offense to Walter Mercado. I just, I don't know who that is.

1:54Then we get my continue watching list, some new stuff on Netflix, top 10 shows, because I watched Girls 5 Eva, into that. Binge-worthy TV shows. Today's top picks, for me, like eight rows down, it's giving me personalized recommendations. Then we get some romantic favorites. My list, quirky TV shows, goofy sitcoms, your next watch, critically acclaimed TV shows. These all sound like the same thing. What are we doing here? I don't get it. Let's try something else. okay we go to hulu and hulu starts by taking a long time to load and then we get dress my tour no chance that's for me uh we get a bunch of live stuff i don't really care tv for me which dress my tour again bunch of crime documentaries that's my wife uh house shows sure then movies for me continue watching emmy nominees because i watched how i'm at your mother top shows another recommendation for the same top show trending newly added tv full series comedies newly added movies these are like just things that hulu has none of this really feels like it's actually directed towards me at all okay now we try max max first shows me a cnn show and then hard knocks with the new york giants that i will watch and will be very into uh recommended for me is up top appreciate that continue watching today's picks which is actually game of thrones and justice League and Kite Man and a spy documentary.

3:17This feels like me. Then a bunch of top tens, live, true crime, designing home, just at it. Okay, this feels closer to being me, but it's still just kind of like, here's a bunch of stuff that we have on Macs. That's the vibe here. Like, do you want to watch this? Here's some things. Like I said, it's all a lot of stuff, but it doesn't feel like anything here is just telling me what to watch. And actually, this seems to me like the kind of thing AI ought to be able to do really well. It's low stakes in the sense that if the model doesn't recommend the objectively perfect thing, or if it even makes up a movie because these systems tend to make things up a lot, that's not a huge deal, right?

4:00I just want to be able to tell a chatbot what I want and get some recommendations. To be fair, there are actually lots of ways to do this already. You can do it with ChatGPT. What's a good movie that's kind of like Mission Impossible or The Bourne Identity but isn't either one of those movies? You might like Skyfall. It's a James Bond film. I've seen all the Bond movies. What else? In that case, you might enjoy The Man from UNCLE. It's a stylish spy film with plenty of action and and a good sense of humor, similar to Mission Impossible and The Bourne Identity. There are also a bunch of plugins in ChatGPT, the custom GPTs, that are explicitly designed for this, though they're mostly just kind of better formatting on top of the same answers.

4:46All right, I'm in MoviesGPT, which calls itself a go-to movie encyclopedia. So I'll just do, I just watched Crazy Stupid Love and loved it. what's a similar movie i probably haven't seen

5:07says if you enjoyed crazy stupid love you might like the five-year engagement this romantic comedy drama directed by nicholas stoller stars jason siegel and emily blunt follows the ups and downs of a couple's prolonged engagement blending humor with heartfelt moments the film explores relationship dynamics in a way that's both touching and funny similar to crazy stupid love it's a pretty good answer i'll take that you can also use other ai systems like Google's Gemini. What are some classic 1980s action movies that I need to see? The 1980s was a golden age for action movies, churning out some of the most iconic characters and quotable lines in cinema history.

5:42Here are a few classics you can't miss. Die Hard, 1988. Bruce Willis redefined the action hero with this game changer. Okay, yeah, we don't need to hear all that, but it recommends Die Hard, Raiders of the Lost Ark, Terminator, Predator, Beverly Hills cop. Then it says, this is just a small sampling of the many great action movies from the 1980s. Still, not bad. I don't know that I learned any new movies from that, but it's a decent start. These models will happily give you movie recommendations, and you can ask for them in pretty natural language, which is cool. So you don't have to just search for a movie or like a genre.

6:17You can actually say, I want a romantic movie that's under two hours, that is funny and not sad, and you get, How about Crazy Stupid Love? It's a romantic comedy that's funny and heartwarming without being sad. I just watched that. Great recommendation, but can you give me a different one? Sure. How about 10 Things I Hate About You? It's a funny and charming romantic comedy, and it's just under two hours long. It's a modern take on Shakespeare's The Taming of the Shrew and has a lot of great moments. Not bad, right? It's not exactly groundbreaking new title ideas there, like crazy stupid love and 10 things I hate about you, you'd find in one Google.

6:55But they're good recommendations. And there's something important in that interaction, I think, that is actually a signal of why AI might be a big deal here. So for years, I've been asking people about recommendations. And for years, I've gotten roughly the same answers. Recommendations are hard to do really well. Really, really hard. And the most effective way we've found over time is to basically say, okay, you like this set of things, which we know because you've either rated them highly or watched them a bunch of times or done something else that makes us confident that you like them. Maybe you gave it a five-star review.

7:29There are some other people out there who also like that same set of things. Oh, and they also like this other thing that you haven't seen. So we bet that you will like that thing too. It's simple. It's how recommendations have worked forever. But it does kind of work. But AI has the potential to approach the problem really differently. So there are two components that generally AI seems to be like a good tool to solve, right? That's Diana Popescu, a developer and entrepreneur who recently started an AI movie recommendations tool called MovieVanders. We are pretty fans of Harry Potter after this, right?

8:07And we said, we were joking, saying, you know, the movie has to choose you. It's not only that you are choosing a movie, right? So we said, oh, that sounds like the Ollivanders from, you know, the wound is choosing you as a wizard. So we said, okay, magic plus that kind of concept. Movie Wonders sounds really good. This is how the name show up. I love a Harry Potter name. But back to the two components. The first thing AI does really well is just ingest and organize data. So, so, so much data. all the genres and actor names and everything that you'd normally associate with a movie, but also all kinds of other stuff.

8:47Every feedback that people is maybe giving in, I don't know, Reddit, Twitter, Facebook, whatever, like, you know, social media, we give more information than we think, right? Every click we are doing, it's information that goes somewhere and it's somehow interacting with other information and producing some kind of behavior that could easily be incorporated into an artificial intelligence to learn from. The second component is that AI can begin to structure that data for you and turn all those articles and reviews and tweets and Reddit posts and everything else into something kind of like a usable database of information.

9:26So if I'm saying, I want to see a movie that has a dog and it's not sad, please, because I'm a very sensitive person, and all these things, somehow that is interacting with the information that is already sort of flying in internet and how those generative AIs are working, we are not exactly sure. The reason Diana and others are excited about AI for making sense of all this information about movies and shows is that otherwise it doesn't really exist. And to explain what I mean, we have to talk a little bit about data and metadata. As I've talked to folks in this space over the years, I've learned that there are basically four useful kinds of information about a show or a movie.

10:09The first is the metadata, which is just information about the thing itself. Its name, who's in it, who directed it, the poster. That's the sort of stuff you can find on a search with IMDb. The second thing is watch data. The single best signal someone can give you about how they felt about something is how they watched it. Did they watch it all the way through? Did they turn it off right away? Did they watch it every night, 12 nights in a row? Did they get halfway in and then bail? That's all really, really useful information. But that information doesn't exist at any real scale, at least not for streaming services.

10:44And it definitely doesn't exist publicly. Netflix knows what you do on Netflix, Hulu on Hulu, Max on Max, Peacock on Peacock, but they don't know about all the others. And watch data, even those lists of most popular titles that you see from these services sometimes, is as a result a very closely guarded secret. It's crucial data and no one wants to share it. The third thing is just stuff to recommend. This is sort of an underrated one, honestly, and not really an AI story at all. But the recommendations you get are only going to be as good as the stuff there is to recommend, right? Even if you know exactly what I'll like, it only matters if you have that thing to offer me.

11:26And most streaming services have a few thousand shows and a few thousand movies, and honestly, sometimes not even that. And in theory, even if someone did have the whole available library of TV and movies ever, it's not a very big library. IMDb's library includes 685 ,000 movies and 265 ,000 TV shows, which is a lot. It's more than you'll ever watch in your life. But it is nothing compared to the amount of content being uploaded to TikTok and YouTube and Instagram reels every single day. The fourth piece of information about movies and shows that matters, And maybe the hardest, and maybe the most interesting for AI, is what you might call traits.

12:05Deep information about the thing itself. Not just who's in it, or whether it's a comedy or a thriller, but one or two or three levels deeper than that. Is it a sad movie or a happy movie? Does it move fast or does it move slow? Does it take a while to get into, or does it capture you right away? Is the soundtrack awesome or does the soundtrack kind of suck? Is it a good movie to watch while you're looking at your phone? Is it a good movie to fall asleep to on the couch? Is it a movie you should only see in theaters? This stuff really matters, and it's information about the title that is hard to know unless you really understand the thing itself.

12:40I think the easiest way to understand what that might look like in practice is through music, which has actually gotten a lot of this stuff really right. Back in 2021, I interviewed Gustav Soderstrom, who is now the co-president of Spotify, about the company's recommendation system. He actually described the whole idea of it kind of perfectly. If you actually have non-real time, if you have time-shifted on-demand content, then you have the benefit of actually being able to machine-listen to the content, transcribe it, and understand it. Then the problem is actually quite similar to text recommendation, if you think about it.

13:13Then you have a few things. One big difference from music. In music, we had these 4.5 billion playlists where people had sat and manually organized the world's 70 million tracks into how they go together. there is nothing like that for podcasts and not even on the competitive platforms right so we kind of had to bootstrap it somehow and so what we started with was to build what is called a knowledge graph quite similar to an artist graph like this artist has this discography of these albums with these songs and you know belong to these genres so we started building a knowledge graph a traditional knowledge graph quite early on before we had any listening data to bootstrap it And then we started adding listening data to that.

13:53So we could see that people that listen to this also listen to that. So we started with actually a pretty simple system of a sort of hard-coded graph. And then after a while, now that we have lots of listening data, we started inferring with more machine learning-based techniques. Both the quote-unquote traditional collaborative filtering, people who listen to this also listen to that, which is kind of what the playlists were about. People who playlist this also playlist that. But we also started adding much more advanced techniques like actually machine listening to the context and summarizing them so that we can do things like podcast topic search.

14:29And there we're actually just we're not just looking at the headline. We're actually listening to the content or listening. We're reading the content that's transcribed, summarizing it. And then we put it into one of these embedding spaces, as they're often called, to understand how it's similar to other things. That's the magic right there. That combination is where everyone seems to agree the best stuff happens. And when Gustav talks about machine listening, by the way, he's talking about technology that Spotify has that can actually break a song into its component parts automatically. So with that technology, Spotify's computers can recognize that the song is 110 beats per minute.

15:08It has a driving guitar line. There's a violin, a female lead singer, angsty vibes, and lots of other things. Tons and tons of different categories and classifiers for what this song actually is. And that actual understanding of what something is at a content level is really important in getting recommendations right. And that just hasn't existed in video before. It hasn't really even been possible. But then, just recently, something changed. We got to take a break, and then I'll tell you about it. We'll be right back.

15:44Support for this show comes from LinkedIn. When you're a small business owner, your business is on your mind 24-7. So when you're hiring, you need a partner that works just as hard as you do. That hiring partner is LinkedIn Jobs. When you clock out, LinkedIn clocks in. LinkedIn makes it easy to post your job for free, share it with your network, and get qualified candidates that you can manage all in one place. LinkedIn's new features can help you write job descriptions and then quickly get your job in front of the right people with deep candidate insights. Either post your job for free or pay to promote.

16:18Promoted jobs get three times more qualified applicants. At the end of the day, the most important thing to your small business is the quality of candidates. And with LinkedIn, you can feel confident that you're getting the best. Based on LinkedIn data, 72 % of SMBs using LinkedIn say that it helps them find high-quality candidates. Find out why more than 2.5 million small businesses use LinkedIn for hiring today. Find your next great hire on LinkedIn. Post your job for free at linkedin.com slash track. That's linkedin.com slash track to post your job for free. Terms and conditions apply.

16:57Welcome back. Let me tell you about an AI breakthrough. So in February of this year, Google launched its newest AI model. It's called Gemini 1.5. Gemini 1.5 was all the things that new AI models usually are. Faster, smarter, does well on benchmarks, all that good stuff. But its big new feature was a new context window, which basically refers to the amount of information that the model can take in and consider at a time during a single query. So like one token is a few letters. Gemini 1.5 could do a million tokens. That's like a book. You could just feed a book all at once. Or, as in one demo that Google showed, it's a movie.

17:35The company demoed Gemini 1.5 with a 45-minute Buster Keaton movie called Sherlock Jr. The movie came out to 696 ,417 tokens, if you're curious. And just listen to how the demo goes. In Google AI Studio, we uploaded the video and asked, find the moment when a piece of paper is removed from the person's pocket and tell me some key information on it with the timecode. Then the whole thing processes for a minute, 57 seconds to be exact, and it answers. The model gave us this response, explaining that the piece of paper is a pawn ticket from Goldman and Company pawnbrokers with the date and cost. And it gave us this timecode 1201.

18:15When we pulled up that timecode, we found it was correct. The model had found the exact moment the piece of paper is removed from the person's pocket, and it extracted text accurately. I remember the first time I saw this demo and being totally blown away by it. This is an AI model processing an entire movie for the first time and identifying one thing inside of it in under a minute. There was one other demo in there also. Next, we gave it this drawing of a scene we were thinking of and asked, what is the timecode when this happens? The drawing is like a crude stick figure drawing of what I think is a water tower.

18:52Big round thing on two legs, person below getting sprayed by water. It's either a UFO abducting someone or a water tower. Anyway, here's how it goes. The model returned this timecode, 1534. We pulled that up and found that it was the correct scene. Like all generative models, responses vary and won't always be perfect. But notice how we didn't have to explain what was happening in the drawing. Simple drawings like this are a good way to test if the model can find something based on just a few abstract details, like it did here. This is obviously just a tech demo, and most things are not this easy and this perfect.

19:29But the possibilities with that idea are huge. Suddenly, we're heading towards a world in which you can upload a movie to an AI model, and that model can start to understand things about it. Granted, identifying a water tower is easier than figuring out the mood and vibe and pace of a movie, but this is still a big leap down that path. It is, by the way, a leap filled with copyright violations and complicated legal issues, but just for the sake of our episode today, I'm going to leave those to the side for right now. So much of what's coming in AI is going to be decided in court, but until that happens, this is what we're going to get.

20:07And this is where we come to the biggest, maybe thorniest question in the whole AI recommendations universe. What actually makes a good recommendation? And actually, no, there's a question even bigger. Why do you like what you like? There's an example I always give for that one, which is The Crown. Like the first couple seasons of that, like, was it, it was a great, I don't know why, but I loved it. It was great. But then like, you know, I noticed on, like whatever streaming service it was on, like it started to recommend like British shows to me, which like, or British royalty, which like I have no interest in British royalty shows.

20:40There was something intangible, I don't know what, about the first couple of scenes of The Crown that caught me. That's where it's trickier. That's David Sanderson. He's the CEO of a company called RealGood. RealGood is one of those products that tells you where you can stream a given show or movie, but it's also becoming a really important data provider in the streaming industry. Like I mentioned earlier, there just is no good universal source of data about TV shows and movies, what they are, who's in them, who made them, and where you can find them. I know that sounds nuts, and it is, But it's true.

21:13That data just doesn't exist. So Real Good has its own consumer product that you can use, but it's also the database behind a lot of the other streaming search providers out there. But think about that example, The Crown, or any other show or movie you like. Do you like it because of the star? Maybe. And maybe that means you'll watch anything with that person in it. That is an easy recommendation problem. But how do you account for someone liking Breaking Bad but not Better Call Saul? or being super into selling Sunset, but really only liking the first season. If you liked Game of Thrones, does that mean you'll like House of the Dragon or any other show that has dragons?

21:50Or was it the palace intrigue that you liked or just the naked people or one of the actors, but not any of the other actors? Or did the thing with the Starbucks cup make you like it or something else entirely? Like they say, there is no accounting for taste, but frankly, I hate that. And I want AI to fix it for me. So Real Good has been experimenting with AI a lot recently, mostly as a way to try and explain whether you might like something. It's less proactive recommendations and more a way of you saying, I'm interested in this. Do you think I'm going to like it? In their app, you go to a title and you tap on the icon and it takes a couple of seconds based on what it knows about you to analyze whether you're likely to enjoy it.

22:30All right. So I'm in the Real Good app and one of the recommendations it has for me is Eric, which I've never heard of before. Vincent, a grief-stricken father whose son goes missing, finds solace through his friendship with Eric, the monster that lives under Edgar's bed. Sure, that's something. Looks like Benedict Cumberbatch is in this. Anyway, I hit the button that says, should I watch this? And it says, it's analyzing it just for me. And it says, with its low IMDb audience score of 6.9 out of 10 and real good score of 79 out of 100, Eric may not be the best fit for you. However, the show's crime and drama genres align with your interests in mystery and drama.

23:08The show's suspenseful storyline about a father's search for his missing son could keep you engaged, but the monster under the bed element might not be to your liking. And then if I hit a button that says suggest similar, it just shows me a bunch of other things that I may or may not like, including the bear and presumed innocent and sunny and house of the dragon, all of which I very much do like. So maybe I need to watch Eric. When you think about the four parts of recommendations, metadata, watch data, stuff to watch and traits, real good is a really interesting one. It has lots of metadata.

23:38That's the database it's been building for years. It has very little watch data because Netflix and the others don't share that. RealGood and every other platform like it, Just Watch and Likewise and all these other recommendation systems are desperate for you to tell them which shows you like and which ones you've seen so that they can try and back into some of that watch data, but they'll just never have it like Netflix has it. On the flip side, RealGood has way more stuff to watch and more things to offer you because its library includes lots of streaming services. And when it comes to traits, actually understanding the content itself, well, that's complicated.

24:13Here's Pablo Alessia, who runs engineering and data for real good. It's not the same thing as looking at a genre or a tag, right? It's a completely different problem. It's a way more holistic view of looking at, okay, what are the archetypes? For example, I've seen some analysis that's made by actual people, like analysis of movies and shows where they're looking at the archetypes and the psychology of them. And that's really maybe the reason and why we like movies, we like the archetypes of the characters, I mean. I would say LLMs are probably the best equipped at looking at problems like that because you would need to look at such a breadth of knowledge at the same time to be able to do that analysis like humans do, but I would say that we're pretty far away from getting to that level of analysis.

24:53Even in the best, most powerful scenario, Pablo says he's skeptical that we're going to be able to teach LLMs to completely, deeply understand movies and shows and also understand why people like them. I know there's people that do. LLMs are the way humans think. I don't think that's the case, but let's say it is. Let's assume that for a second. We're still not like, LLMs don't perceive the world in any way, shape, or form, close to the way humans do because we're giving them data in such a different way. And also the data that we give them is so limited and lacks so much context. Even if you give them a thousand people, they're still individual, a thousand people.