OpenAI's New Video Generation Model, Zuck Returns Fire, Tesla's Fatal Crash

0:00Let's talk about Sora, OpenAI's new video generation model, along with plenty of AI news this week, including the stagnation of ChatGPT, Meta's new AI model, Gemini 1. 1.5 and Edison if that is not enough we'll also break down Mark Zuckerberg's Vision Pro response and the wisdom of putting$50 ,000 in a shoebox all that and more coming up right after this welcome to big technology podcast Friday edition where we break down the news in our traditional cool-headed and nuanced format wow wow wow massive week of AI news and plenty of other good topics to talk about we're joined as always by Ranjan Roy of Margins Ranjan welcome I try to underplay big news weeks in AI, but I think this actually is probably the biggest week, at least this year, but in a long time in AI news.

0:50Absolutely. So let's start with Sora, which is OpenAI's model that instantly generates eye-popping video. It's a video generation model. And this is what the New York Times says. OpenAI has unveiled a system that creates videos that look as if they were lifted from a Hollywood movie. A demonstration included short videos created in minutes of woolly mammoths trotting through a snowy meadow, a monster gazing at a melting candle, and a Tokyo street seemingly shot by a camera swooping across the city. Now this is what they say the implications are. The technology could speed the work of seasoned movie makers while replacing less experienced digital artists entirely.

1:30It could also become a quick and inexpensive way of creating online disinformation, making it even harder to tell what's real on the internet. What do you think about that assessment? Do you agree with that? Like they're editorializing pretty big in a news article right at the top. I think we need to break this down into kind of like different categories. You have the actual technological achievement, the coolness factor, the impact on jobs, and then disinformation. And I really want to start with the technological advancement and the coolness factor, because, oh, my God, if you saw those demos, I am so excited.

2:10This is what I like. This is probably the biggest advance I've seen in a while, because, again, you go from mid-journey version five to six, Dolly two to three. All of those are still, I don't want to say incremental because they're still big leaps, but this is the first time in a long time that wow feeling that when I first used ChatGPT and I saw the text starting to stream across and actually answering my question and generating text. This is the first time I've had that feeling in a long time. And my favorite part of all of this is how OpenAI probably, I think they're throwing a bit of shade at Google on this one, that they make clear that these are all real demos and this is not just some research paper.

2:52Because every time Meta and Google release this stuff, it's always just a new research paper and they'll get their headlines. This is real. Okay, but let me ask you this. Is what we're seeing, okay, it's definitely a breakthrough. But is what we're seeing computers able being able to understand more than, you know, they have in the past? Or is it just as simple as like we trained a dolly like model with video instead of images and they can do images because you saw that they were able to like, for instance, put pirate ships in a cup of coffee. So do you think these models understand the fact that coffee is a liquid and a pirate ship is a solid, and therefore it would basically sail on the coffee ship, the coffee cup, the same way that it would on the water, and let's do it this way.

3:37So do you think this is increased understanding by the models, or is it just simply them taking the same process they have all the time? Because that is a very big difference. Yeah, no, no, that's exactly, that's perfect. I think there's two layers to how big of an advancement this is. as you said, now physics are involved. Before the kind of physics understanding of how things move and what's happening in an image is obviously not nearly as important. Still, how the pirate ship would render on the coffee cup and balance or whatever in that still image still mattered, but now how things move, even the video they released, the first tweet with a woman walking in tokyo how they're walking how things move around her that's huge and then you start to think about the processing power and kind of like overall processing advancement here generating one still image is one thing generating in a one minute video clip essentially i don't know however many thousand frames with one prompt and doing it i don't know how long this will take how long it'll take to render this stuff but i mean that that's a massive jump there was some like really fun stuff also with the mistakes that i made like there were a bunch of people trying to place a beach chair down in the sand and it just kind of floated and wobbled and it's like wow like the what does the model think a beach chair is made of and what does it do like it does under it does show i think some emergent understanding of physics like you mentioned and that is really the interesting thing we're going to see where it goes i think i'm a little bit more freaked out now about what ai is and what I can do than I was at the beginning of the week.

5:18And I'm curious, speaking of people being freaked out, what do you think Hollywood should feel about this? I wish we had Sharon Waxman with us again from the rap talking about this stuff, because the reaction from all the Hollywood trades is like, it's over people, pack it up. But that being said, like, come on, like, this is not something in my opinion that can start to replace the feature length film. Yeah, you're not going to get acting. Like I think the entire art of acting is the most human thing imaginable. And, you know, conveying human emotion in original ways is not going anywhere. I do think I think all these kind of things, advertising is going to change a lot faster than feature films.

6:02I think this is the stuff that gets me excited that small brands and creators are going to be able to create videos in ways that they never would have been able to before. And pretty quickly, because there's not as much need. That's more just about doing it, setting the scene, communicating visually whatever you're trying to communicate. So I think feature length films are nowhere near, you know, changing. but okay, I take that back because I think some changes you can see is maybe for B-roll, you know, like the little bits of film that, you know, are not involving an acting or an actor, I think that stuff might start to get generated and it'll actually make movies potentially better because you don't need to fly across the world to get one shot and you'll actually be able to storytell in a better way.

6:53I feel like they can already do that with special effects though. yeah no no i think that's why the stuff has already been changing a good amount so yeah no i agree you know your half the stuff is shot even stuff about new york city is often shot in hollywood so right exactly yeah i agree so so that's why i don't think it's gonna be just pack it up hollywood you're done i think i think this this has a much bigger impact among smaller players people all that kind of stuff the big people big players have already had a lot of this technology and access to it. It just makes it more readily available.

7:29And I think we should also note that this is not going to be available anytime soon, at least from OpenAI. I mean, basically, they were making the announcement showing that they got ahead of this model. I mean, their model is definitely far ahead of the other models that are out there, but they are also saying that they're trying to red team this, basically find holes in the technology, and that the general public should basically shouldn't hold its breath about getting it anytime in the near future. Yeah. I think they're, to their credit, you know, you always say OpenAI is in the hits business.

8:01Yes. They're pretty good at product marketing. And like, you know, again, going at Google and saying, look, these are real demos people created. It's here getting that excitement going, but still making sure and working on the product. I think they've shown that they can release products pretty well, certainly better than Google has recently. Meta doesn't even really have any consumer-facing products in the same way. I think they have an image generator now. They do. Yeah. Microsoft has been all right, but OpenAI has still, in this whole wave, been the master of product releases and marketing.

8:38So I think they've started this one pretty well. Yeah. And I do think just going back to our original reaction to it is that this is incredible technology and seeing some of the videos and the way that it, you know, video can really trigger emotion in people and seeing the way that it was triggering emotion in people across Twitter yesterday. yesterday. And of course you had some people farming for clout, but like seeing a cat rolling around on a bed with someone was just like totally unbelievable. And if there was any question of whether the progress at NAI is going to come to an end or slowing down, it doesn't seem like that's the case.

9:15It seems like we are really in for a year, another year of breakthroughs. And this is, I think the first big one on the agenda. So what you're saying is NVIDIA's valuation makes sense. No, I'm not saying that. No. I think I agree. I think this is a big week. I agree that there's been a moment, the last few months almost have felt calm in a good way that you can actually start trying to really use the technology and trying to figure it out. And then suddenly now it's like, oh, here we go. 2024 is going to be just as big. Exactly. But, and there's always a but, right? You mentioned that I think that OpenAI is in the hits business, and here's why.

9:59It's that this technology can become equaled, and because it's so new and novel, it's not always going to take off. And there's some data that I broke in big technology this week with SimilarWeb, which is an online analytics company. Just looking at where ChatGPT stands, because I think you'll remember in the summer, ChatGPT stagnated a little bit. It had a couple of real big dips in usage. And people said once students come back, it's going to go back again and surpass its highs. Well, the data that I saw from SimilarWeb, which measures web traffic across the internet, found that not only have they not reached their peak, they're now 11 % off the peak that they hit in May 2023, stagnant from mid-2023.

10:47and they declined in both November and December this year, key months where students are in school and the usage dropped. So basically my perspective on this is that the ChatGPT's growth has flatlined. And if you want to read more and see the data, I definitely encourage you to go to bigtechnology.com and check it out. And Ranjan, I'm just kind of curious what your reaction is to this. I mean, we thought for a minute it was the fastest growing consumer product in history. It had 100 million users in about two months. And then we thought that, okay, it's such a cool technology that maybe it will continue to grow.

11:22But it hasn't. What do you think that means? What do you think the implications are for OpenAI and ChatGPT? Yeah, I think that's why this video, text-to-video, maybe that's why they're pushing it even more right now. Because personally, and we talked about this the other week, I have perplexity. I have Microsoft Co-Pilot. and let's all remember that Microsoft Co-Pilot for free gives you access to GPT-4 and Dolly 3, whereas with ChatGPT, you have to pay for ChatGPT plus 20 bucks a month. So they have a lot of competition and I'll admit, I've been Gemini-ing a little bit in the past week. I have.

11:59Doesn't it feel great not to say barding? Oh my God. Let's not beat a dead horse. No, but I'm going to do this all year long. All year long. Thank you. And I'll never forget the name Bard because Gemini is great. I feel good saying Gemini and it's working. It's working pretty well and it's integrated nicely. At first, I was a little surprised they did not release a separate app for it. And it's actually funny. I asked Gemini, does Gemini have an iOS app? and still in a good old school bard fashion it returned to me gemini a crypto the crypto trading app and then another gemini it wasn't even self-aware about i was asking if itself had an ios app but then they did the way they released it is in the google ios app gemini is kind of you can like uh hit a toggle and switch to gemini mode i actually think it's smart because it gets you into the Google app itself, uploading images and these things are already built in, like it still really entrenches it in the overall Google ecosystem that everyone's already, you know, used to.

13:11So I actually think it could be a very, very smart consumer play. And a lot of people already have it on their phone anyways. And that means that they are going to be a direct competitor to chat gpt and so you have so much competition coming from every single angle that just being the first it might not be enough for them so okay so one of the aspects might be competition but the other aspect might be just the format is getting tired i think that people were initially so excited about how well these bonds performed but started to lose the ability to find new things to ask them and the use cases like they're they're like there's like a desire to use chat gpt but but also just sort of a confusion in terms of what to do with it after that initial novelty wore off.

13:57And I think that's another explanation. It's not just the competition because yes, Bing has ticked up, but basically you look at any of these bots compared to, let's say a Google and you're just single digits, low single digits. Like for instance, ChatGPT's web visits are a tiny bit more, maybe a few hundred million more than Bing's, but still Bing is just 3.4 % of the total search market. So I wouldn't go as far as to call it a niche product, but it almost is there. Yeah, I think the chat interface itself, I agree, is limited and not people, unless you get in the habit of understanding how to use it in different ways, I think it is a limiting factor.

14:42And again, going back to Gemini for the win, possibly integrating again as the chat part is just one part of the overall feature but getting people to take a picture with their phone and directly tell you all about it google has had google lens for a long time and integrating the gemini model directly into that is smart i think starting with that blank cursor the way chat gpt has you do is is definitely going to keep it more niche than it otherwise was so yeah i think They need to, but they have been better at integrating, again, image generation, image understanding with text. But to me, the more I thought about it is, again, there's the using large language models for knowledge retrieval or information retrieval and then using them to generate new content.

15:33And the first half is just search. And perplexity is doing well. Copilot's doing well. Google is building their own search generative experiences. So that's something that will ChatGPT compete on? I don't think so. Right. I'm going to push back on this idea that this multimodal thing is something that people actually want to do. Google's had LensUp forever. It has not broke through into the mainstream, which is that app that you could take pictures of and Google will search and stuff like that. And maybe there'll be some moments where you'll, you know, drop an image in and ask these bots to have a conversation with you about it.

16:10but it's hard to see that really being like the super use case wait what are you basing your google lens is not a mainstream product just like how many people do you actually see talking about lens have you just anecdotally yeah yeah yeah traveling traveling i've seen i've used it it's a kind of a common thing i remember it was actually a friend who told me about oh when they're traveling especially like in countries with non-English alphabets, they go straight. Any menu you open, any sign you look at, you pull out Google Lens right away and get it to instantly translate things for you. And after I heard that, that's how I use it myself.

16:50I'm Google Lensing all the time. Okay, all right. So you versus me. I'll take your word for it. I'll take your word for it. So also on the Google front, just a quick note to hit. They released this new model, Gemini 1.5, already. It's interesting to me. They have said that, you know, they're already. Then they just released Gemini 1, and now they're already releasing 1.5. And the interesting thing about 1.5 is that it has 1 million tokens compared to 128 ,000 for GPT-4 and 32 ,000 for Gemini Pro, right? And so basically, it's, let's see, this is what Sundar says. it's about 10 or 11 hours of video, tens of thousands of lines of code.

17:37And the context window, that's the tokens, means you can ask the AI bot about all that content at once. That seems big to me, but it got basically no attention while everybody was looking at the Sora news. What do you think about that expansion at Gemini? Yeah, I think it's tough because, well, first of all, I agree that guys take a breath. You're doing well with just the whole Gemini rebrand. We don't need 1.5 just yet. Save it so you can build some buzz around it, but they are doing it. So we'll talk about it. The token windows are interesting to me because one of the big changes that happened even in a lot of the work I've done is as these context windows and the amount of tokens you can use get bigger, that means in the past, you would fine tune a model, meaning you would like train a model, a foundation model in a very specific way.

18:32And it took work and you had to write code and it was difficult. Now you can literally make a prompt that's the size of a book and say, here's 200 examples. Now using those, recreate this kind of text in this style. And, but yeah, so, so it changes the way you actually build the entire process that now it's called prompt tuning. It's that you can actually get it to do exactly what you want and provide all that instruction just in the prompt alone because they're so long. So you can put in a whole book, but even in GPT-4 and use that to essentially on the fly train that prompt and get it to respond in a specific way.

19:10The difference between that and 32 and 128 ,000, I don't even know what 128 ,000 feels like. And you had a quote in here that was really good from Sundar, or he says it's about 10 or 11 hours of video, tens of thousands of lines of code. So what exactly you're inputting, I can't even fathom, like, are you putting in 10 or 11 hours of video? I don't know what you do. Maybe you can put in entire code bases and get that to train to output code in a specific way. That gets kind of interesting but but the size of it has been a major change especially gpt4 then claude all of these have been pushing the limits on how big these context windows are and it changes the way we all can use llms in a pretty powerful way but is this out yet because there was uh somebody from founders fund that tweeted this guy john coogan google launches a new ai tool i go to check it out wait a minute this isn't a working product this is just a blog post every time are we in that scenario again?

20:14Thank you, John Coogan, for outlining exactly what I've been trying to say in a much pithier way, in a good way. Because yeah, when we're talking about product marketing and releases, again, Google always does this. They announce something way, way before it's ready, and they just fall behind even more. So OpenAI still for the win on product marketing. But OpenAI also this week just released a blog post and everybody's going to go nuts. So you can't be selective about that. No, no. But they were very specific that these are real demos created by real people. This isn't like a research process that we validated and something.

20:56This is, folks red teaming this are the ones who created these prompts. Can I tell you what my favorite bit of AI news this week was? Nothing. not text to video and the complete transformation of reality? No. It was VGEPA. Did you see this about VGEPA? So VGEPA, let me see what it stands for. It stands for Video Joint Embedding Predictive Architecture, VGEPA. And it's from Meta. And they released this model or they've announced this model. And they say it's a first step toward artificial general intelligence. Those are bold words. Now, why is it? Okay, so basically what it does is, this is from Fast Company, it learns by processing unlabeled video and figuring out what's probably happening in a certain part of the screen during the few seconds it was blacked out.

21:50So basically, if you have somebody playing guitar, like Mark Zuckerberg posted a picture of this, and you blacked out the part of his hand that's actually strumming the chords, and you ask the model to create what's in that blacked out area, it can show the hand strumming the chords. And this isn't just like, you know, total video generation, it requires a true understanding of the world. And to me, that is very interesting. It's, you know, effectively saying that, like, you know, I, the video, I, the AI model, you know, I'm not just generating, but I am understanding and that to me, and I think that's different from just generating a full video, it's figuring out what's missing.

22:31to me sounds pretty crucial and very interesting. And that's why VGEPA to me is the big news of the world. And this is what Fast Company says. It is not a generative model. It develops an internal conceptual model of the world and then acts based off of that. I mean, if VGEPA is working, if it actually is for real, that is a change because, you know, again, large language models with text work with trying to predict what is the most statistically probable next word or word that will fill in that blank space in the gap images similar what's the next pixel but to try to do that for video is uh if it can get that right that does seem pretty profound and maybe that is like a breakthrough as big as the 2013 computer vision breakthroughs but but I don't know.

23:27Jan LeCun, give us a demo. Come on on the show and give us a live demo. I know, I did invite him on the show. They said he was, I think he was somewhere making some speeches or something like that. He won an award I saw like in Dubai or something like that, but it was some like a global AI award. Listen, Jan, if you're listening, less awards, more big technology podcast. Less awards, more big technology podcast. Simple as that. Show us VJPA. It's the formula for success. Here's what we can do. Black out the image of Jan LeCun accepting that award in Dubai or wherever it is. And then the model will figure out that it's him.

24:07Have real Jan LeCun come here and demo VJEPA. But it is interesting because these models, this is weird, right? We're talking less about predicting the next word, more about reasoning, more about understanding the world conceptually, more about learning physics. like where is this going to end this is kind of getting scary creepy exciting all of the above i mean it's interesting because jan lecun i feel and meta overall has been kind of actually in a pretty smart safe responsible place especially versus open ai and others where google might be too conservative or you know they're like open source build guardrails directly in recognizing the need to be responsible while still progressing.

24:53Overall, I think they've positioned themselves pretty well in terms of how these things develop. But yeah, and as we started this conversation, especially in this election year, how this stuff plays out and in the disinformation realm is going to be pretty scary, interesting. We're going to be living in real time the potential impacts of these things these tools i'm way less worried about the disinformation and more of like what happens if this thing becomes self-aware which like i don't know it sounds more and more like that that might happen in our lifetimes where before it seemed far-fetched i'm more concerned about disinformation in the election than self-aware artificial general intelligence choose your enemy choose your enemy it's only a matter of what kills you first yeah i mean i don't know do we really need disinformation when we have tucker carlson out in a russian supermarket um talking about how great the cart return system is that's fair that's fair it's the algorithm it was always the algorithm yeah by the way that is a very innovative system that cart return you put the coin in return the cart i didn't say i didn't actually even watch the video i tried to block out the entire i saw some of the memes but i i really tried to block out uh sometimes you have to make like like a very concerted effort to not partake in something or engage with something.

26:17And this is Tucker and Russia was one of those things for me. I mean, I love how Putin said that. It was like Putin goes, yeah, it was a weak interview. Oh, did he say that? Yeah. It's amazing. Okay. Anyway, let's talk about another comeback video, which is Mark Zuckerberg coming back and basically giving the Vision Pro a tongue lashing, saying that not only does he think that Quest is a better value because it's seven times less expensive, but also that it's the better product in general, period. And he makes all these different comparisons, saying the Quest weighs less, its controllers are better, it is more immersive, all of the above.

27:02And it was very interesting seeing the reaction there, people really liked Zuckerberg taking that swing against Apple. Did you think this was a wise move? Do you think it was accurate? Have you tried the Vision Pro yet? Give us your thoughts. I have. I have. I have. And I actually, I don't think I'm going to buy one just yet, but it was incredible. I think I'm in the same exact place as you are, where I'm going to let it breathe a little bit, give it a little bit of time, but it was spectacular technology. I've tried the Quest 2. i've not tried tried the quest 3. um and we can get more into vision pro but on zuckerberg's video i have to say and i have a lot of issues with meta overall and their products and it's but i respected this video it was it was great marketing it was great like the way he did it casually sitting down and just talking it was a reminder that like mark zuckerberg and i've heard this i've read about this many places like he's a competitor and you know apple is coming for them in many ways and you've written well about how maybe apple's actually helping them inadvertently but but he's here he's here to play and he's sitting on his couch and just looking in the camera and talking and i do find it kind of like you know rich that when there are headlines like zuckerberg coming out saying quest 3 is better than vision pro because of course why wouldn't he But I thought the overall format was good and great, in fact.

28:33I do also find it interesting that I've been using threads more, and somehow my algorithmic feed in threads for a couple of hours was literally all fawning praise of Zuckerberg's video. And I really do wonder how Meta is tweaking that algorithm the same way Elon did with Twitter to favor themselves in particular ways. I'm sure there's something there. So, but, okay, here's the counter argument about the Zuck video, right? So, M.G. Siegler, who's a former Google, or I think he might still be at Google Ventures. He says that Zuckerberg is running a real risk of this all coming back to bite him in the ass.

29:16Steve Ballmer laughed off the iPhone, but actually his stance at the time was quite reasonable. The iPhone was too expensive relative to the market, so much so that Apple rather quickly dropped the price. But no one remembers that. They just remember the dismissive laugh. Is Zuckerberg in another Ballmer situation? He could be. I think he totally could be. But I think that, I don't know, he wasn't as dismissive. I thought he was like well rehearsed, at least spoken about, you know, here are the very concrete reasons I think this and not kind of laughing it off more because he can't laugh it out.

29:52He can't say that headsets are stupid because the whole argument is our headset is better. So I don't think he's fully at risk. Do you? I think that's that's totally right. I think it is much more sensible for him to take this position than it would have been if he was in the Balmer spot. And they never really competed against the iPhone, right? I mean, they had the Windows phone, but that went nowhere. So they have the device that's been in market, and it's been selling. Did you ever see a Windows phone? I have. I have seen them, for sure. Okay. They were not good. No, there's a reason they got sunset.

30:30I saw a few Zunes. Remember the Microsoft Zune? their iPod computer. Yeah, that was also bad. Yeah. Look, there's a reason why they fell behind and had a lost decade. They just continued to ship total shit for years, and that was a problem. They've rebounded. But now Satya is just rolling right now. Okay, last thing on the Vision Pro, the Apple fans are returning them to the Apple store. This is from The Verge. It's no coincidence there's been an uptick on social media. a Vision Pro owner saying they're returning their$3 ,500 headsets in the past few days. Apple allows you to return the product within 14 days of purchase, and for the first wave of Vision Pro buyers, we're right about that point.

31:15Comfort is among the most cited reason for returns. People have said the headset gives them headaches and triggers motion sickness. The weight of the device and the fact that it is front-loaded has been another complaint. Okay, here, this is amazing. The Verge's product manager said he thought that using the device led to a burst blood vessel in his eye. At least one other person noted they had similar experience with redness. Do you think this is just people blowing social media chatter out of proportion, or is this a real problem for Apple? I think that it's definitely a problem if it's happening at any kind of scale.