The Fourth Paradigm Shift: Metaverse 2.0

0:02Hey, everyone. If you like this podcast, go behind the paywall to get privilege access to the smartest minds in finance. Visit realvision.com slash rvpod and use the promo code podcast10 to get 10 % off our essential membership for the first year. Join the Real Vision community and learn how to become a better investor. And now to today's episode of Rao Pal Real Vision. Robert Scoble, I am so excited to bring you to Real Vision. How are you? I'm doing good. That is great to hear. And as I say, I'm just deeply excited about the conversation that is going to unfold right now. For many out there, you will need no introduction, but you are a legendary tech blogger and strategist writing about technology and this internet revolution we've been living through at your blog, Scobalizer, since the early 2000s, ancient history in internet terms.

1:00A true Silicon Valley insider, interviewed almost every technology leader, you know, I can think of many of whom you count as friends. I've been reading your work forever. You really are an OG of this game of right thinking about and writing about the connected revolution, the internet revolution. You helped inspire me to to start to try to do the same um let's dive in is all i can say let us dive in before we start there's one thing i read that i have to ask about is it really true that siri the launch of siri happened at your home studio yeah that is a fact is it yeah how did that come to pass slipped it several months too because uh when they first came to show it to me uh we found several problems that it wasn't working on.

1:52And they opened up the back end, Adam Shire and dad came to my house and they opened up the back end and showed me how it works and tried to fix it right in front of me. And they're like, oh, we got to come back. Yeah. Yeah. So they actually came to my house twice, right? Second time to do another show, like a few months later after they fixed the problems. And back then that was pretty much the most advanced sort of consumer facing AI conversational agent out there. Boy, have we moved on since then. I actually have an interview up on YouTube with the VCs who funded Siri because they recognized I was the only one who got it other than Steve Jobs.

2:34So that's the behind the scenes story of what was going on at the launch of Siri. Right, right. The breadth of your work on technology, as your readers will know, and your readers on Twitter or X will know because you're prolific there too. The breadth is vast. So just to put a kind of loose structure on this conversation, I'd love us to just paint a portrait, or you primarily, to paint a portrait of what is coming. You know, you believe a massive wave of change is coming. I want to paint a portrait, a brief history of our very near future. And let's start with what strikes me as your overriding thought, the framing thought you put on a lot of your analysis, which is something you call metaverse 2.0.

3:21Sometimes you call it the fourth paradigm. Just introduce us all to that. Or the holodeck. Or the holodeck. We just released a paper this morning on the holodeck, which is metaverse 2.0. We're seeing a range of new 3D technologies being developed and improved. I'm reading all these AI papers. This week, there's a big conference and you saw a bunch of neural radiance field papers or Gaussian splat papers. It's like, what are those? These are new 3D scenes that an AI can generate around you, right? So if you're wearing a pair of glasses, augmented reality glasses, or a headset like the Apple Vision Pro in the future, you're going to be able to talk to an environment, have the environment change, and have a virtual being in that environment that you talk to.

4:16And I'm just laying out, if you search on x.com for metaverse 2.0 in quotes, you'll find all these posts that I have been sharing of anything that's moving in 3D technology, basically. And there's a lot coming. So is this fundamentally about bringing computing into the physical environment? Is this Zuckerberg's internet that we can step inside? Is that a description you like? Yeah, spatial computing is computing that you, a robot, or a virtual being can move through, right? The old way of computing, we look at, like right now, I'm looking at you on a 2D screen, right? Flat screen. In the future, you might be sitting in the room with me, right?

5:02Virtually. In 3D. and soon we're going to be able to, you know, when we move our hand up, a whole user interface could be on our arm, for instance, right? Particularly if you're wearing a watch that has a sensor that would do that, right? And so there's a lot coming in this new spatial computing, a ubiquitous computing world or whatever you call it. But yeah, we're soon going to be, the computing is going to happen all around us. The wall could change into, you know, Taj Mahal or a material. If I'm a material scientist and a chemical structure could appear on the wall, right? Or in 3D space around me, right?

5:50Yeah. And that's already happened. I mean, we have apps like Nanome and VR where we could do precisely that. It's just it it hasn't quite gotten to consumer level yet because you need a fairly expensive PC and a cord and a fairly ugly, you know, big headset to do it. Well, all of that in the next three, four or five years is going to change. We're going to end up in five years. We're going to end up with a pair of Apple glasses that's very lightweight. Right. Yeah. Yeah. And I want to dive in. I mean, let's dive into Apple because it's such a natural progression from that overarching thought of bringing computing into the physical environment, dropping a digital layer over the physical environment.

6:34You have been talking about and writing about Apple's incursion into this space for years. You've been saying they're cooking something up at Apple. I wrote two books on a topic. Something is coming, right. And I've been reading it going, yes, maybe this year, maybe this year, well, finally something happened. The Vision Pro. It took a lot longer than I was expecting. It took a little longer than you thought. But hey, directionally, you were absolutely correct. How do you think that the Vision Pro, talk about, for our viewers, talk about what the Vision Pro does fundamentally, but also what part does it play in Apple's broader strategic vision?

7:16Because I know you think there's a much bigger vision at play that Apple are The whole family of products coming. Yeah. I mean, what we're about to see is computing is going to move from 2D screens to 3D environments that you are in, right? And to do that, you have to wear something on your face. Yes, I know that there's new kinds of 3D monitors. I have one in my family room, right, from Looking Glass that let me see a 3D object on a 2D screen. But that's not what I'm talking about. I mean, I have a HoloLens, right, up here. And with the HoloLens, you can play a game called Fragments, for instance.

7:55And in Fragments, aliens blow holes in the real walls around you. And then you have to shoot them before they get through the holes. Or if they get through the holes, they start flying at you. And you have to shoot them in 3D space, in the air, before they get to you. If they get to you, you're dead, right?

8:15So where are we going with this? The Vision Pro is that, but unlike the HoloLens, the Vision Pro has beautiful chips in front of your eyes. So let's just start with that. You can watch a movie on it. You couldn't do that in previous VR headsets that only had 2K monitors. They're just not good enough to compete with a 4K TV you get from Best Buy or something. Right? Yeah. And this new headset has two 5K chips in front of your eyes with high dynamic range, high color depth, right? High difference between black and white, which is not true on Zuckerberg's Quest 2. Now, tomorrow he's going to announce a new headset, but even the new headset only has 2K displays, not 5K.

9:06And that's just not good enough to work in. I can't use TweetDeck, for instance, inside a Quest because it just isn't sharp enough. It's not high res enough. Yeah. It's not high. And so that's why for the last seven years, meta has been mostly focused on video games. Because video games, you don't need the resolution that much, right? If you're playing Beat Saber and swatting with your hands and seeing some fun things, you don't need that high a resolution. But if you're working, like if you're trying to do email or anything modern on a computer, you need a better screen than that. And the screens are just now getting there.

9:51The other thing is Apple has a lot of AI inferencing on your head. That's a huge shift. from just VR. If you compare to any of my VR headsets, whether it's the Vive or the Quest or whatever, they have screens, but they don't have AI inferencing. So they can't do this new generative AI that's about to hit. You can't put a large language model in there, so you can't do a real virtual being where you're going to talk with the thing or a non-playable character in a video game. These things are about to become human-like. where you're walking down the street, you're going to be talking to AIs who are meeting you in the street soon and talking about some pretty deep things because it knows all of music and all of art and all of politics.

10:47It's crazy. This is Apple's pitch. My point of view would be, this is Apple's pitch to own Metaverse 2.0, to own the fourth paradigm that you talk about. How do you, and I think I'm right in saying, well, you tell me, you think Apple are going in a more fruitful direction than Meta? Because the big company, many of our viewers will think about immediately when they hear the word Metaverse, obviously it's Meta and Zuckerberg, spending billions of dollars to try to own the fourth paradigm too. Why do you think Apple is going in the right direction and not meta? Here's one. I went down and visited John Carmack, who's a famous video game developer.

11:36He worked at Meta back then. I went and visited him in Dallas, and he was like, I think Meta is going to need to do a phone. I'm like, what? You already tried to do a phone. Nobody bought it. Why do you say that? Well, because Apple has in the phone a whole bunch of GPUs. They just added, I think, six new GPUs in the new phone, right? A rewritten semiconductor stack with a lot of new graphical processing, which is also really good for AI, right? A lot of GPU means you can do AI inferencing there. And it also has the radio and a big battery and a big heat sink, right? So you can heat up the phone pretty warm and it's sitting on the desk in front of you while you're computing in a pair of glasses.

12:30Zuckerberg doesn't have that, doesn't have this, right? So he has to develop a lot of custom silicon for the glasses and do a lot of inferencing up in the cloud, where he's going to have banks and banks of NVIDIA cards in a data center. And all the AI inferencing is being done up there, shoving the data straight down to the glasses with custom silicon so that glasses don't heat up and are lightweight and are affordable, right, and easy-ish to manufacture for these companies, right? And I'm not sure how that's going to go because that brings into play a whole new privacy problem. Because I was playing VR one night in my Quest 2, right?

13:21and it has four cameras on it. Now, it's not doing a lot of AI inferencing yet because that's an earlier version. But the next version, I bet they're going to do start doing a lot of computer vision and AI inferencing. That's all being done in the cloud. My wife walked into the room naked while I'm playing VR with these cameras blaring, sucking up everything, right? So if you don't think about where that AI inferencing is being done you really will soon because you're soon gonna be into these glasses i mean right you know it's apple and meta are both spending tens of billions of dollars and this is going to come out with a product someday that it excites people because the the computing around you in a pair of glasses like this is insane compared to looking at a little tiny phone hey everyone we're going to take a quick pause and hear a word from our partners.

14:19We'll be right back.

14:24So Apple have been like essentially weaving AI chips through the physical world, viral, but billions of devices. And it's almost like, you know, and I, I believe this because I've, I've read it, you know, you've been telling this story for years. It's almost like Apple have planned this all along. They put so much AI capability out there into the world. Now, you The first iteration headset is in place, dropping this digital layer over the world around us. How long, no one has a crystal ball, but how long until that technology truly can be just a pair of glasses that is this easy to put on? Three years.

15:04Okay. What leads you to believe that? For me and you, three years. Early adopter, five for the normal consumer, ten for the late adopter, right? So sometime in the next decade, everybody's going to have a pair of these things on. Whether it's Apple or Snap or Meta or Google or Amazon or Tesla or some other company, right? You're going to have a pair of these glasses on if you want to be part of modern society. And that's sort of scary for people to hear because they don't like hearing that they have to change and adopt a new thing. And they don't like wearing glasses. I hear that over and over.

15:42on people. I hate wearing glasses. Why do I have to wear glasses? Now, you and I have to wear glasses just to see. So we're already ahead of those people. But most people don't like to wear it. They're going to resist. They resist this idea, right? But the second they put it on, and by the way, that's the real goal of the first device. Half of every Apple store in the world that's going to be rebuilt to show off the Vision Pro. It's there to get a demo. You go to the store, you try it out for 10 minutes, and you see, holy shit, computing is fucking insane. I'm sorry. It's really insane. Because now instead of a small little monitor in front of you to watch a movie, you have a wraparound IMAX theater around you, right, in the headset.

16:32And once you see that, then you start going, oh, I want that. And you're then open to, oh, when Apple finally gets a pair of glasses that's lightweight enough and cheap enough that you buy in. And when billions of people are wearing this technology, what does that change? I mean, everything. because, I mean, if you're in San Francisco right now, you see an autonomous car goes by like every fifth car. It's crazy. And you start thinking about all the data that that thing sees and is sucking in and could use on a digital twin of the real world. And then you start thinking about, oh, let's say every fifth person has a pair of glasses on that's walking around that digital twin and adding more data on top of it.

17:26So you're going to start seeing this digital twin, this real-time copy of the real world, the mirror world, Wired called it, start to really get utility, right, where you're walking around a shopping mall someday and it's going to tell you everything about the shopping mall, everything about the products in there, and everything about, hey, how do I get some help over here? You know, a robot will come up and start talking to you or a human will, right? Humans aren't going away anytime soon. This is so, you know, we're going to get into robots and autonomous vehicles in just a sec because they're a massive part of the portrait, the picture you've been painting for the last few years.

18:11It's the same technology underneath, right? NVIDIA's Omniverse, which is their corporate strategy. It's a hydra that sucks in all data. It runs autonomous cars. It runs augmented reality glasses. It runs virtual beings. And it runs robots, right? One database, one digital twin to rule them all, right? Yeah, yeah. And so your robot that's going to be in your house in 10, 15 years is going to use the same data structures as your glasses are. And you can interact with them, right? Yeah. Yeah, I mean, okay, before we dive into robots, you know, it seems to me - And I'm not the one who says that. Adrian Kaler actually worked on, he built the computer vision system for Stanford's first autonomous car.

18:57Then he worked at magically building augmented reality. Then he built a humanoid robot company. And now he's building a conversational AI, right? So it's all the same. It's all the same kind of technology, right? Is it the end of reality? Is it the end of a shared reality when we can walk around in the world around us with this layer of data and digital information dropped over it that we can personalize, we can choose as we see fit? We can kind of opt into our own realities then. Am I right? I mean, that feels to me part of where we're headed. Let's go to the best reality, right? Let's go to Yosemite National Park.

19:36Go to the top of where it's called Glacier Point, right? and where you can look down at the valley across the way is Yosemite Falls. Over there is Half Dome. Down to the left is El Capitan. You put on the glasses, and it makes that reality better. Now, the reality, if you're standing there, is beautiful. It's analog. It's a smooth wave. Human beings like it, right? And that's why you go to Yosemite or someplace like it, right? Because it's beautiful, and it's fun. It's nice to be there. But you put on the glasses and now you know the scientific names of all the plants around you. Or you can look at the sky and say, hey, Siri, replace the sky with the Hubble telescope view.

20:20Now all of a sudden you see all the stars, not just a thousand stars, but all of them. And you see them all labeled, right? So you know what you're looking at. You know where Mars is. You know where Betelgeuse is, right? Up in the sky because it's telling you, right? And it's telling you what the names of all the mountain ranges are around you. So is that reality? It's sort of an enhanced reality. Right, right. And it feels to me that in the end, the distinction between what we call physical reality, the stuff of atoms, and informational reality, the boundaries between those two things will sort of blur and become somewhat irrelevant.

21:00Yep. And we're already seeing this happen, right? Apple's maps and Google's maps are adding augmented reality. I just used it in San Francisco the other day using my phone to look around. And I was looking around and, oh, man, it changed the side of the building and put the street names and told it. And it had an arrow like this is the right place to go. Right, right. On the side of the building, right? You've written about Apple and essentially their mirror world, a 3D map of a one-to-one, essentially simulation of the entire world. Is that still a technology that's part of the picture here for Apple?

21:39Are they still working on that? Yeah, yeah. They have a digital twin of the real world. That's how they're doing that. So the phone, when you open up the phone, you don't have to wait for the glasses to see some of this happening, right? You go to New York or San Francisco. You open up Google Maps or Apple Maps. You turn on augmented reality piece of that, right? And then all of a sudden it knows where it is in 3D space and it knows what it's looking at. Like, oh, you're looking at the pyramid building right now, right? And then it can change the pyramid building and put, you know, a name on it.

22:12Or, you know, if you're walking around the city, it can tell you, hey, go down this aisle to alley to get to the Chinese restaurant you're trying to go to. Right, right. And this and the key thing is, you know, this technology is about to jump from screens, you know, an experience like this to just an experience like this, an immersive experience. And that's going to change, as you say, you know, I mean, consumerism, learning, our relationship with knowledge and information, just just everything. Just it's me. It's an incredible revolution coming. You believe in the next three, five, 10 years, billions of people wearing these devices.

22:52And it's partly about solving the hardware challenge of just getting it this easy to put on. We do that every morning. Billions of others can. An incredible wave of change. But let us dive into robots. I mean, there's an incredible video right now circulating on X of Tesla's humanoid robot sorting, autonomously sorting different shaped and colored objects. You know, just something extremely useful autonomously that it's now able to do. I was pretty skeptical when Elon got on a stage, you know, whenever it was and said, we're going to do this humanoid robot. And a man in a suit came out and did a dance.

23:31Yeah. Look how fast they've gone to something that looks extremely credible and extremely powerful. They aren't starting from zero, that's why, because they have the autonomous car ready. Talk about that. Well, my car, when I go pick up the kids, it has eight, nine cameras on it, right? And it's watching the kids playing. You ever think about where that data is going and what it's being used for, right? It has to know what human beings are doing in an autonomous car because it has to predict what your next action is before it hits you, right? It needs to know that, oh, you're riding a bike and you're probably going to keep going in a straight line, right?

24:21But maybe you turn really fast, so we have to be prepared for that, right? And it's watching you do that. And it has the database of videos, of millions of videos of people doing various things in the street around you. So it can use some of that data to do an end-to-end neural network. A modern AI now learns from watching videos, right? So they have the videos. They have the simulator. They have the auto-tagger. They have a lot of infrastructure at Tesla to deal with the data that's coming off the cars. Well, now the robot comes along and it's a little bit more complex than a car, but not a, well, it is more complex, but it's not a huge new thing, right?

25:09It's using 26 electric motors with cameras that see the world in 3D, right? And microphones that you can talk to and an AI computer in the brain and a bunch of batteries in the arms. Right. And this is Tesla's unique advantages. They're building this incredible machine vision technology via the cars. And they're now going to use that AI technology to make their robot move and to make it understand the world. How soon are humanoid robots coming to people's homes? And talk about why you believe pizza delivery, because I love this, why you believe pizza delivery is the first interaction that like millions of people have.

25:49I mean, if we have a robo-taxi network where the car actually can drive without a human, we could argue about what day that's actually going to get finished. But GM has crews driving around San Francisco, and Google has Waymo driving around San Francisco, and Zoox from Amazon driving around San Francisco, right? And my Tesla is driving around the world pretty well. It still makes a few mistakes, but, you know, it's doing pretty well. you can say it's the next two to four years, right? Where it's going to be worldwide and it's going to be fairly good. Maybe I'm wrong. Maybe it's another two to four years.

26:29So eight years. Let's see. But what's fascinating for me is sometime in the next eight years. Yeah. I mean, what's fascinating for me is the business model you imagine where essentially we have a network of autonomous vehicles delivering these humanoid robots to homes. So I, as a consumer, don't have to own it, right? It's sort of sent to me on demand. Yeah, it makes a lot of sense, right? Because if you start having a robo-taxi network without humans in the car, out on the street, right, driving around, well, you're going to want to do things like, can you go to the local Home Depot and pick up a screwdriver for me?

27:08Or can you go to the local Chinese restaurant and get me some lunch? or can you go to the laundromat and pick up my laundry? Well, how would that happen if it's just a car? Now you have to have a lot of infrastructure to get somebody to know the car's waiting for your laundry and take it out to the car. No, just put a robot in there, have the robot go into the store, tell the person, hey, here's the receipt for Robert Scovel's clothes and I'm picking up the clothes, right? And then drives back home And when it gets to the curb, right, it gets out of the car and takes my clothes up to the door and rings my Amazon doorbell.

27:49In fact, it probably already rang before it even got there because the doorbell has this stuff. But, you know, it can go, oh, your robot's here. And that's the business model, right, because I'd pay 20 bucks for that. It's a lot better than me driving down to the laundromat and picking up my clothes. That takes 20 minutes each way, and that's 40 minutes of my time. I'd pay 20 bucks to have 40 minutes of my time back and sit here and look at Twitter more. I cannot wait for that to become a reality. And so I can just pick up my phone or by this time I'll be immersive computing and probably tap a tile in my environment or something like that.

28:29Order a robot to my house to tidy the house and run to the store for me, pick up what I need and bring it back. Here's another way to look at it. Beyond demand economy. Yeah, the car is hard to make because it's big and the line to make it is long. For instance, I visited Detroit and went and saw how Ford makes F-150 trucks, F-120 trucks, right? And it's a very long line. It's like a snake through this big, huge building, right, with a lot of people doing things, putting an air compressor in or something, or an air conditioner, and a lot of robots. Robots are putting the windshield in and gluing them in, right?