A New Era of VR: From the Court to the Classroom

0:02Hello, hello, hello, welcome back to the A16C podcast. This is your host, Steph Smith. And look, we are always trying to experiment here, and this week is no different. In the past, we've taken you straight to the people building top AI models or even into self -driving cars. But today, we take you into the metaverse. Or whatever you want to call it. Because today we take you into an episode that was fully recorded in virtual reality. We actually recorded this episode in anticipation of MetaConnect, which was around a week ago, or Meta announced a bunch of things, from releasing their first mixed reality headset to AI -powered smart glasses.

0:42Plus a pretty incredible podcast recording, or Mark Zuckerberg joined Lex Friedman in the Metaverse, using new, hyper -realistic VR avatars. And like many of you, I listened to that full episode, or one line stood out in particular. We're kind of sitting in a dark room, which is familiar for your podcast. But I think part of the vision for this over time is not just having this be like a video call. I mean, that's fine. It's cool. It feels like it's immersive. But you can do a video call on your phone. The thing that you can do in the metaverse that is different from what you can do on a phone is like doing stuff where you're physically there together and participating in things together, and we could play games like this.

1:23I couldn't agree more. As this technology evolves, there are going to be so many new experiences that are unique to mixed or virtual reality, things that cannot be done on a phone and that benefit from that extra dimension. So we wanted to give you a glimpse of these very real world experiences that actually already exists on many of these devices. In particular, this episode will show you two examples. The first is with Jim class VR, one of the top rated applications on the quest, where I actually get to hop into VR with one of their co -founders Paul Katzen. Now Paul was a great teacher, he taught me how to play basketball in virtual reality, which means I dribbled, I did a layup, I even dunked for the first time.

2:07And of course, you can't really get the full sentiment when you're listening only an audio. So if you do want to catch the full video, including my attempt to dunk, you can head on over to our YouTube channel. But we'll play you a quick sneak peek. People's movements and physics, it's all synced in real times. If you kind of walk up close and then just can't give it that shooting motion. Wow. First shot. First shot. That is actually kind of rare. After my VR basketball demo, Polonize sat down at the beach, again, in VR, to chat about the implications of this technology, including the social judgment, the technological requirements, and of course, how AI plays a role.

2:50Make sure to keep tuning in past Polonize interview because we also talk to another founder who's trying to disrupt education with VR. Another very real -world example about how this technology is being deployed today. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16z fund. Please note that A16z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details including a link to our investments, please see A16z .com slash disclosures.

3:37Hello! Hey! Good to see you at the beach. I know. Yeah, this is, and then people typically go in the gazebo and then that's where they hang it up. Okay, should we go over and have a chat? Talk about the future of VR. This is the first time. I can say that for sure that an A16z podcast has been done in VR. So let's draw this line in the sand. I hope we'll do many more. Let's do it. That sounds great. First thing I want to ask after I'm going to go to the beach, I'm going to go to the beach doing that for the first time is just you mentioned that it's one of the most popular West apps currently.

4:09What goes into that? Like why do you think people are coming back? Because there are so many apps that kind of fail or people try once and then you know there's also a suite of apps that people have just never heard of. Yep, I mean I think there's kind of two things. One, like when we first launched it, you know there was very little in the experience like you couldn't even actually to move around or dribble a ball. But our hypothesis was like, if you had a shared interest in basketball with other folks, and we could put you in an environment with eight other people where you could hear them, they're all kind of going through the same experience, they would want to come back.

4:42And that actually ended up happening. We made a post on Reddit, and there was a few hundred people that tried it out, and all of a sudden they were in a discord being like, how can I move around yet? Why can't I dribble? They were asking for a lot of things related to basketball, but they were coming back just to chat with people and hang out with someone who might be in another country, but they share this common interest. And so that's definitely one of the biggest drivers within VR today, like most of the top and 13, 15 most played VR apps on the quest, they're social. It's ways to share experiences as people from around the world in a really immersive and expressive way.

5:19There's no other technology outside of VR today that gets you as close to expressing yourself like you would in real life. It's not all the way there, but it's close. And that allows for way more interesting social interactions. Yeah, it almost sounds like social is effectively the killer app that we've been looking for with VR. And I mean, even on Zoom, I'm constantly thinking about the fact that this cannot be the end state, like this box that we have like floating heads. Like that cannot be the best that we get to in terms of communication and digital social interaction. And so you're saying that basically this is a step ahead.

5:56But I'd love for you to speak to where we are in that trajectory of VR because I mean, I think it has taken a while and I think a lot of people think we're still in this trough of disillusionment. And so I guess first would you say that we're still there? And then also what will it take really for VR to be the everyday household activity or not even activity, right? Something that just integrates into your daily life the way that an iPhone might or the way that, you know, your computer and watching Netflix might. Over the long term, I think the expectations that people have are these headsets, VR, AR headsets, are going to replace a smartphone and they're going to be something you always have on.

6:33It creates digital layers on top of your everyday world that all of a sudden makes everything digital and it brings the internet everywhere. And I think over the long run, something like that is going to be inevitable, but there's all these steps along the way to get there. And the way to kind of replace a smartphone isn't to just kind of build a device that's great for gaming. It's a device that has value to you in all parts of your life so that you might want to use it in all parts of your life. So I think the way to get there is all of these different use cases that bring you back wanting to use this device.

7:07Just like with a smartphone, you have an app for your camera, you have an app for navigation. All of these things that might have required special hardware before it became an app. And I think with these headsets, what we're turning into apps are experiences. So things like hanging out under a gazebo with someone for a few thousand miles away, you know, getting your downloading of an NBA basketball court to your house, you're not downloading a camera, you're downloading a whole experience. And so I think the big idea is like there's gonna be an app for all of your experience. Maybe we can speak a little bit to two other things that maybe relate to the arc of VR.

7:42One is the social side and one is the technical side. And I think in both lanes, there's somewhat of pushback in terms of like socially. Tell me what you're seeing. Like are people already adhering to this and saying this is wonderful or are their parents who are like, no, take that thing off, like going out in the quote unquote real world? You know, if someone's taking the R very seriously, let's say, and they're setting up some cones and doing some drills, and we have people that are running coaching clinics. Oh really? The folks are all over the world, and if you see that on TikTok, the most common comment is like, just go outside.

8:20Or like, why don't you just do that in real life? You'll get better at this for it. And the funny part is, we don't see that as much for someone playing a console basketball game, or a console sports game, or a mobile. That's kind of like, what gets me excited, isn't that, you know, VR and these technologies are gonna be replacing everyday life. Like we're already spending most of our lives on our phones, playing games on TV, watching content on TV. And those are just such, like the humanity and the expressiveness and how social those moments are, are just so much worse than if you got off the couch and instead of learning a bunch of controller combinations to get better at a basketball game, you're actually moving.

9:02And all of a sudden that can also expand the sport and make you feel a lot more fulfilled, less lonely you're connecting with people on a much deeper level so I just see VR and these technologies is a way to level up. These really kind of like you know low impact digital experiences that we're getting used to and make them feel a lot more like real right. Right. And a lot more personal. Maybe we can jump to the technical aspect because there's been you know the Apple Vision Pro has the announcement has come out soon we're gonna hear from Meta in terms of Meta Connect and what they're doing next.

9:35Tell us a little bit more about what you see on the horizon technically and if you see any important unlocks that would allow us to have just you know much better experiences and allow more people to want to participate. Yeah so I think there's technologies that move the ball forward on use cases that we already know work in VR and AR. So new devices especially like new VR devices like the next class and you're going to see a lot more they could do a lot more in terms of performance and resources that they give a game. So like for instance, like Quest 2, it's incredible. We could run an NBA core.

10:09We got crowds, all that kind of stuff. But it's running on an old smartphone chip. And the fact that any of this is running sometimes kind of blows my mind. We've got customizable avatars. People that can hop in and connect with each other and sync that physics in real time. The next devices will just allow us to create more immersive experiences. We'll have more resources to create larger environments, bigger worlds, have more people in these worlds. You have much more content around. You could imagine a car driving by and all this kind of stuff that you expect from console experiences. We'll be able to start leveling up these spaces, make them more social, make them more immersive.

10:54So every new device basically allows us to get more resources to create better experiences. But where things get really interesting is some of the new use cases. So even for us, if you have a device where you can see your surrounding environment, and maybe the hand tracking is much better, maybe it has real -time object detection, all of the sudden you could put your controllers down and you could have an interactive experience with a real basketball. And you could start tracking how many shots you've made,

11:24you could do that we're trying to match your motion to them. So all of a sudden, that unlocks a training use case, which is like a really big market. And a few deeper fitness use cases. So I think piece by piece, we'll just see these new devices unlocking very valuable use cases that I think it'll give you more reasons to use the device every day and start to have it really be a replacement for the smartphone as well. Maybe you could speak to some of those opportunities. I mean, you're obviously working hard on Jim class and it's going well. It's one of the top rated apps in the quest store, but what else do you think is just like an untapped opportunity given?

11:59Let's say the next three to five years of technical advancements that you see coming You know with a social technology like this. There's a lot of you know companies and meta's doing this They're kind of going after how do I build kind of like this big platform of open worlds where everyone can create whatever they want And maybe that'll lead to this metaverse But I think the best way to go about it, and this is kind of like the dimension that we're going about it, and it's like, we're focused on one interest, which is basketball right now, and that's the reason why you download, and then you stay for the people and the relationships you're making, and kind of like the back and forth interactions that you get.

12:33And so I think the biggest opportunity that folks is like, make an interest and create the simulation experience where people can really go in and really dive deep into it and make friends around that interest, and make it as realistic as possible. So it feels like you're having that experience. And from there, you could expand to so many different things, or just focus on that interest. And you're kind of building almost like a digital online representation of all the things that happen within that interest in real life, like with basketball, you know, fashion culture, and people, you know, people want new shoes in here and new clothes and all that stuff, like all of those things start to become opportunities for you digitally.

13:14And so I think that's a big area and just really just picking something specific and making it great and making social There's there's just kind of like endless opportunity there with space. Let's end off on on a question that I feel like I have to ask in in this era But where does AI come into all this like I feel like people think AI and VR are two separate things But to me as you were talking there about the different problems to be solved and the worlds to be built that AI I have to be part of this. How are you thinking about that new technology integrating with what you're building? Yeah, well, when I think about AI, I think a lot of what it does is help you create more content and experiences and personalize content, content experiences at scale, especially with games and social media.

13:57And on our team, for instance, we are building, for moderation, there's systems that we have there. For rec, we have a lot of the standard things, like recommendations that we're building. And we also, our art team, for instance, uses a generative AI for concepting and designing things before they build them. So I think right now, at least our team for the past year has been in that phase of starting to move just existing use cases as well as we're seeing new generative tools to be able to move certain parts of the art, work flow and engineering workflow to those tools. VR, though, is really, really interesting because one, you know, you could imagine being able to create endless content.

14:42Like if you think, you know, 100 years in this technology it develops, it really becomes something like an experienced machine. In some ways, if you're hopping on and you're like, whatever experience you want, you'll be able to have it. And maybe it's with real people, or maybe it's with, you know, agents that are kind of like bots that come up to you. All of those things are ML problems. And it's, again, very personalized. That's what you often want. and you want a bot to be able to respond to you, to move, you know, and kind of respond to a scenario, to say something that makes sense. Like all of those become really, really interesting problems to solve within a 3D space, expanding content and things that you can do.

15:20And then where things get really interesting is really motion. And that's where we're focused on in the short term. Like the way that you move, imagine kind of like your dribble move or kind of like your crossover, you know, if you could somehow capture that, and then you maybe control that over mobile and start to move like you and behave like you. Like those are things that are really big opportunities, I think, in terms of where VR is going ahead. In the short term, I think it's all going to be about how VR connects to mobile, whether through content or gameplay, and that's where I think Apple's in a really great spot.

15:51And in the long term, I think, you'll imagine endless personalized content that you want to figure out how to go swimming or go fishing in an crazy part of the world. Now there's an app for that. And you could progress. So you could get better, you know, you could kind of discover incredible things in those experiences. Absolutely. Even the note on AI, like it seems like right now you haven't gotten the avatars to like move, like the facial expressions, the mouth moving. I saw this thing on Twitter recently where, you know, you could record a video. It would translate it into another language.

16:24And then it would also modify your mouth movements with AI. Match the new, the new language. So it looks like you're really speaking French or German. And I can imagine, you know, that's like an immediate implementation to your point of just making everything look more real, to make everything feel more intuitive. Like if I do these movements, the relationship which being like what I'm doing and what the quest is picking up, it's just more intuitive. So I'm really excited for that, Paul, this has been awesome. I think I'm gonna have to go practice by the way. And then I'll see you back on the court.

16:58Let's talk back on the court. Alright, I hope you enjoyed that interview, which again was recorded fully in VR. That's also why you heard all those seagulls because that was part of the game. But now let's pivot to a completely different story and application of VR. Together with Prisons VR founder Anna Rupa Ganguly. Now instead of using the extra dimension to sing some hoops, Anna Rupa is using it to teach children mathematics. And that was after she had her aha moment that one of the best indicators of math proficiency with spatial reasoning. So why not use spatial software? Plus, I think many of you will be just as surprised as I was by this statistic.

17:39Today about 70 % of US eighth graders are not proficient in foundational math. Let's kick things off with the beginning. Prisms VR, you are a solo founder, you're doing this alone. How did this all begin? Thanks so much for having me. So, prisons was very much born out of my lived experiences as a teacher and a district leader. After a decade of teaching and leading math instruction across Boston and New York, I became convicted that we don't have the tools to close the learning gaps at scale. You may know this, but today about 70 % of US 8th graders are not proficient in foundational math. I did not know that.

18:13That's a stat. It's a stat. And though it's, it dropped 8 percentage points pre -pendemic. You're looking at 60 % of US 8th graders were not proficient. and despite billions of dollars going into this problem, the numbers were static. And so as I began to kind of look more deeply into why this was happening, I discovered that the top indicators of success and post -secondary STEM are number one, your ability to rotate 3D objects in your mind and maintain perspective. It's what you roughly call spatial reasoning. The second indicator of success is your ability to abstract. And all that means is being able to go from like physical human experiences and describe language notation and build mathematical models, not from text on a page, not from word problems, but from your life experiences.

18:57So I kind of looked at those two things going, huh, we learn spatially, and we learn by abstracting up from our day -to -day lives. We have not found a way to scale either of these methodologies. So prisons was born to scale these two key competencies, and hopefully fundamentally and radically, and quickly, close the achievement gap in math and science. Right. Is this widely understood? Because I mean, you said this was discovered in 1970. Yeah. Why haven't we seen this translate into the classroom before? Well, there are a couple things there. It's been widely understood in the research community.

19:29And I think the last few decades in education reform has seen a huge lack of translation of what the research has shown to actual tools and solutions. That's been primarily dictated by the tech. So, so far, computing devices have mainly been on 2D screens. That's right. We were digitizing learning mechanisms that were possible using the computational devices we had. And now with the advent of spatial computers and mobile VR and XR technologies, we now finally have the natural interfaces where we can scale learning with your body using six degrees of freedom, using a multitude of tactile tools before jumping to an equation which is very abstract.

20:08Why is this critical to turnaround? Let's talk about that. If we are seeing math scores stagnant or declining, why do you feel like this is so important for us to turn around? I think that math education has been under attack for a long time of why do kids need to learn algebra? Why do kids need to know percent and ratios? And just let's set the record straight. Bob Moses had shared this into the civil rights movement. Algebra one is a civil rights issue of our time. I'm a math ed person, so from my perspective, it's essential, not only is it tied to salaries over time and earnings over time. It's of course tied to just the types of jobs that you have access to.

20:44You don't do well in algebra one You are not taking chemistry physics biology If you were not succeeding in chemistry physics biology you are not going on to the applied mathematics or the medical sciences So you are just kind of cut off from a very large swath of jobs in our economy. Yeah, I'm pretty early And very early on yeah because students take algebra one at the eighth grade level Yes So then you get tracked at the high school level and you know great seven to twelve is when you are training for or whatever you're going to kind of contribute to and apply for in your senior year of high school.

21:11So let's jump back to the why now. It does feel like we're at an inflection point. I feel like what was a week or two ago, Apple released their vision pro, or at least, showed the world what was coming. And it does feel like maybe, maybe, we're at an inflection point with VR. But tell me more about how maybe we're at the right time, where this technology can really change things. So with the release of mobile VR, which is this idea that you don't need to be connected with an umbilical cord to a bulky piece of hardware. In its previous generation, it was really hard. Like every kid had to have this like big laptop connected to their headset.

21:43They couldn't move around. Computational abilities were pretty limited. Yeah. Movement tracking, hand tracking, all of that just made it very clunky and frankly uncomfortable. So that was never going to scale. And so with just all the work that's been done for the first time, VR is comfortable. It's safe. There's a psychological safety that's now come with all of the work that's been done around UI and accessibility. And then I would say with respect to Apple's announcement, what they've now done is taken a lot of the standard text interfaces and UI that we are still comfortable with using our iPhones, using our laptops, and made it spatialized.

22:18It's mixed reality where we are able to kind of be in our physical space and use a multitude of interfaces in a spatialized environment, which I think is just critical to getting a lot of people very quickly using the medium. So I'm now able to kind of like control my level of immersion. So if I want to be in VR, fully immersed, focused, shut the world out and be in a new space. I can do that and I can turn my dial and then move to augmented, come back to my space, move to MR, where I can kind of have my face time up and be in my living room. So I think that the power of now is the seamlessness through which we can move through the different modalities, whereas before VR was more niche because it was VR or bust.

23:00It felt very binary, right? Headset or no headset. Then even when we talk about spatial computing, it was this paradigm of like, is it going to be AR or is it going to be VR? And it feels like what you're pointing towards is it's not binary. There's a spectrum and you know, for different applications, you're going to use one or the other. Right now, presums is in VR, so you're fully immersed if I understand that correctly. And you have developed this application and it's in schools already, right? So kids are using this. What are you seeing in terms of whether it is actually changing that data point that we were seeing before about math comprehension, math reasoning?

23:36Is this spatial paradigm that we are now in changing that equation? The short answer is yes. We launched the schools in fall 2021. In that time, with a very small team, we're already across about 160 school systems across 30 states. And so when I think about the greatest value that our students are shared with us is relevancy. When you get to immerse yourself, like leave the four walls and go to the Himalayas and India to build new elementary schools that create shaded regions as an architect, they're able to fully build an identity, which you cannot just by kind of looking at things in a highly abstract way, because you aren't doing it.

24:14It's not with your body. So that's like the first thing that people have shared with me around just the value of pure VR. The second kind of big value that our communities are sharing with us is the ability to physically derive these math equations that meant nothing. Why is it equal to MX plus B? Where did they come from? Kids have no idea where it came from. So what this is kind of culminating to is West Studies, our research partner, we've conducted two big studies with them. One was a feasibility at scale study. That showed a couple things. One double digit growth outcomes on algebra one proficiency, which we just talked about an important problem to tackle head on.