Seeing the real world inside a virtual one

0:00Support for this show comes from MongoDB. You're a developer who wants to innovate. Instead, you're stuck fixing bottlenecks and fighting legacy code. MongoDB can help. It's a flexible, unified platform that's built for developers by developers. MongoDB is ACID compliant, enterprise ready, with the capabilities you need to ship AI apps fast. That's why so many of the Fortune 500 trust MongoDB with their most critical workloads. Ready to think outside rows and columns? Start building at mongodb.com slash build.

1:01BurgeCast topics in a while. Super fun. Go find it. Today, for our last episode in the series, our producer, Liam James, is here to tell us about Vision. Liam, welcome to the show. Hello, hello. I'm so excited we get to have you on instead of just you yelling at us for taking too long to go to ad breaks. This is great news. Yeah, you finally convinced me to come on the show. So we gave you what I would assume is like the easiest topic in video gaming, which is sight. How has this experience been for you? I definitely thought I lucked out when I drew the long straw and got vision and then quickly realized, wow, there's just so many roads I could take with this mission.

1:46I do remember you got really excited about TVs at one point. You were like all up about wanting to do a whole thing about televisions. I did. I am a huge display nerd. I find new tandem OLED screens to just be, to me, that's my sports. But David wouldn't let me do that, listener. Lucky for you. Nothing makes great audio like deep dives into display technology. That's kind of how I feel. Agree to disagree. Agree to disagree. But yeah, the journey took me back to my childhood, which was a very big surprise. And it turns out the thing that was the most advanced, most spectacular visual experience you could have back when I was a kid is arguably still the same thing.

2:28And that's flight simulators. Yeah. Back when I was a kid, flight simulator was the thing I wanted to try, but couldn't, didn't have the computing power. So I just kind of admired it from afar and stopped paying attention to it once I got a little older. But fast forward to this episode and looking around at what the visual landscape is in gaming. And I was blown away by what you can recreate in a flight simulator today. It's just astonishing. So I reached out to Polygon. I said, who's the flight sim person that I should be speaking to about this? And of course, they partnered me up with Charlie Hall, who I had a great conversation with about all this.

3:05But what I really wanted to find out was in 2024, what is the best way to experience a flight simulator? Is it creating a wall of monitors? Is it strapping a VR headset on? Is it something in between? And the answer was not as obvious as I thought it would be. And Charlie really helped me understand what is a gimmick and what is actually useful when you're trying to fly a virtual airplane. Okay. So before we get into it, just give me the like super high level. What is it about Flight Simulator that is so sort of uniquely visual? Like when I think of really visual games, you think about the sort of big, huge open world, like No Man's Sky kind of games with like beautiful graphics and it's always in space and everything is happening.

3:48Flight Simulator seems otherwise like it's just like a bunch of buttons and you're in a cockpit. Like, what is it about Flight Simulator that felt so like specifically visual to you? Absolutely. The more fantasy-like games, in fact, even flight simulators that take place in space, are actually a lot easier for us to create because they're not based on something in real life that we can compare it to. Whereas in a flight simulator, like, for instance, Microsoft Flight Simulator, you've got models of real planes at real airports. You're actually flying a real life recreation of planet Earth. You can literally fly all the way around the Earth.

4:25And if you were to look at a bird's eye view of what you're seeing, it is one to one exact to satellite imagery that Microsoft has been building over many, many years. And if I'm not mistaken, there are a lot of people on the Internet who like make it their job to find when that is wrong. And that like the stakes for screwing up the length of a runway at the airport in flight simulator are actually like pretty high. Very high. Yes. Everything from the hat switches to the airports, to the airplane reflections on the outside, they try for the most accurate representation as you can get. And that's a lot harder than just creating some wacky looking spaceship that you'd see in a sci-fi movie.

5:07Got it. Fair enough. All right. Well, before we get into it, I should say, I haven't really been a flight simulator guy, and still I'm sitting here looking at my one measly 24-inch monitor, and I feel like this episode is about to cost me a tremendous amount of money. So with that as a warning to everybody, here's Liam and Charlie. Charlie Hall, welcome to The Verge Cast. Glad to be here. So I was lucky enough to be the producer chosen to do the Vision episode for this gaming series that we've been running the last three Sundays. I couldn't help myself. I went back to being an eight-year-old kid with a Packard Bell 486 computer wanting to play video games.

5:46And the one I wanted to play the most but couldn't actually run on my machine was Microsoft Flight Simulator. Yes, yes, yes. The original crisis for PC gaming, right? Does it run Flight Simulator? Absolutely. This was 1993 or 1994. And yeah, I couldn't run Flight Simulator. And probably because of that, right, I wanted to play it so badly, even though as a third grade kid, I probably wouldn't have had the patience to sit through the pre-flight checks and actually get a plane off the ground in flight simulator. Didn't matter. I wanted to play that game because it just felt like something above and beyond a normal video game that you were flying in the air.

6:31But it also felt as close to real in 1993, 1994 graphics that you could get. Absolutely. So fast forward to today, working adult with a salary, I could actually afford things like a gaming PC that could run Flight Simulator. I wanted to see what does Flight Simulator feel like and look like today? And quite a bit has changed in Flight Simulator in 30 years. It has. So maybe you could tell me a little bit about your experience with Flight Simulator. I know that Microsoft's been doing this for quite some time, but there was some time in between where there was a dearth of Flight Simulators out there.

7:11What's the deal with Microsoft Flight Simulator? Well, much like you, I started playing Microsoft Flight Simulator in the late 80s, early 90s. My dad actually worked at Bell Laboratories here in northern Illinois. So he was able to bring home kind of his almost like company surplus, these fairly high-end switching terminals, basically. So I had a PC 6300 and 8086 board. And so I was playing an old version of Microsoft Flight Simulator. I had the AdLib sound card, right? So I had all the good stuff. And I had my Gravis joystick there. Oh, wow. You had a Gravis. I did, but it didn't have the twist rudder, right?

7:53So I had to do rudder with the keyboard. Anyway, and so I was able to take off and get up and going for MIG's Field, you know, back when there was a MIG's Field before Mayor Daley put a couple of eggses through it. And that really gave birth to this kind of lifelong gaming obsession with flight for me. I went deep on Flight Simulator. I got all the mods as they were going up for clearance at Toys R Us, right? I remember a science fair project that I did in eighth grade where I combined SimCity and Microsoft Flight Simulator and took the teacher on a tour of my SimCity that I had rebuilt in Microsoft Flight Simulator.

8:34Like, I was way into it as a kid. I just need to pause. I just need to pause for a moment to just tell the listener that the face I just had on was of complete awe. I mean, those were the games of my childhood SimCity Flight Simulator that were so impressed. So impressed. Go on. I apologize for interrupting. But the way that Microsoft Flight Simulator was built back in the day was entirely by hand. The John Hancock building, the Prudential building, all of those famous buildings in the Chicago skyline were built by hand. So if you went too long in your approach and you hit the Adler Planetarium there at the end of Meigs Field, like somebody put that there to impede your progress in the game.

9:19But that is not necessarily the case in the modern version of Microsoft Flight Simulator. And we can certainly get to the modern version of Microsoft Flight Simulator. But, you know, for a while there, Microsoft kind of kept the heartbeat going, right? And then kind of some new competitors came into the space. X-Plane is another really popular platform that kind of grew up during the early 2000s and kind of came of age in kind of the 20 teens. But then there was this big surge of space flight simulation as well. This Chris Roberts, one of the co-creators of the Wing Commander franchise, kind of brought that back to life with Star Citizen.

10:03This massive, ongoing, decade-long project to recreate kind of the 90s dream of Wing Commander inside that game. And then, of course, Elite came into its own with a brand new version called Elite Dangerous. And I've also had the experience in both of those. But after both of those products came to market, that's when Microsoft came back to it, basically, with this new vision for Microsoft Flight Simulator, which, while there are some things in the game that are kind of made and rendered and done by hand, the vast majority of the game is not done by hand. It's actually done with a very sophisticated, almost kind of photogrammetry, which we can discuss, too.

10:48So yeah, 2020 rolls around and Microsoft releases what, to a layman's eyes like mine, looks like photorealistic flight simulation. How did they pull that off? Have GPUs just gotten so good? What is it that's making this look so pretty? Well, you've got to remember that Microsoft is a very large company, Liam. And in addition to having in its portfolio Microsoft Flight Simulator, which I don't know if you're aware, is its oldest piece of published software. It's older even than Windows. In addition to owning Microsoft Flight Simulator, well, it also owns this little thing called Bing. And Bing is, in addition to being a very capable search engine, it is also an entire suite of mapping tools.

11:39So just like you can go to Google Maps, you can go to Bing Maps, and you can find your way around. But Bing has done something a little bit more aggressively than Google has, and that is capturing satellite information of the entire world. And so what Microsoft and its partners at Asobo Studio, a French studio that actually did a lot of the heavy lifting on Microsoft Flight Simulator, what they were able to do is take, I don't know if it's two or four or five petabytes of this satellite data that it's getting, that Microsoft owns effectively as part of Bing Maps. And then it bolted Microsoft Flight Simulator to this data.

12:23So when folks received Microsoft Flight Simulator in the mail or when they downloaded the client, they were really only downloading a tiny fraction of what they needed to run the game. You get the applet, you get the client, you get it started up. And then it starts downloading, I think it was 120 gigabytes on launch day. It was one of the largest downloads on launch day that I'd ever received. And that was just the basic information. After you get that 120 gigabytes, then it starts streaming data directly from Bing. So if you go, like I went to Egypt to be parked, you know, 2000 feet over the Great Pyramid at Giza, it's streaming in satellite data, photogrammetry information, all of this sophisticated information from Bing Maps and putting it directly on the screen in front of you using all kinds of technological wizardry.

13:22And it's literally applying that satellite data to other data sets that it's pulling in. It has topographical information that it literally skins the ground with satellite information. At launch, it was really weird. It looked kind of like there were rivers that were kind of crawling up the sides of mountains. They hadn't quite gotten it dialed in quite perfectly. As the game has progressed since 2020, that resolution, the difference between the topographical map and the satellite data has become even better. You're literally able to fly over the entire surface of the entire world, and it looks like it does in Bing Maps.

14:04It actually looks better because it's putting in trees, it's putting in houses, it's putting color on the roofs of these houses, and it's doing all of it automatically without really any human intervention on that landscape. It's really remarkable. Yeah, I have wondered a couple of times why the map companies, the Google Maps, the Apple Maps, Bing, don't just copy Flight Simulator because it is that much just prettier looking and more realistic than the 3D maps we get now. They go further than that, though. They bring in real world air traffic. So you are flying in the sky next to other planes that are really in the real sky.

14:44They bring in real weather data from actual weather data in real time. You can actually bring in other players as well to act as like air traffic controllers so that they can control the airports that you're flying around. It's an incredible multi-layered, multiplayer simulation. Yeah, simulation is truly the only word for it. Okay, I want to get into flight simulator peripherals, but first we're going to take a quick break to listen to some ads.

15:17Support for this show comes from Shopify. One of the most exciting parts of starting a business is when the sales start coming in. If your business is looking to start selling online, Shopify has everything you need to create your online store. Shopify is the commerce platform behind millions of businesses around the world and 10 % of all e-commerce in the U.S., from household names like Mattel and Gymshark to brands just getting started. With hundreds of ready-to-use templates, Shopify helps you build a beautiful online store to match your brand's style. And Shopify is packed with helpful AI tools that write product descriptions, page headlines, and even enhance your product photography.

15:56And best yet, Shopify is your commerce expert with world-class expertise in everything from managing inventory to international shipping to processing, returns, and beyond. Turn those dreams into and give them the best shot at success with Shopify. Sign up for your$1 per month trial and start selling today at shopify.com slash vergecast. Go to shopify.com slash vergecast. Shopify.com slash vergecast.

16:29Okay, we're back with Charlie Hall from Polygon. So Microsoft has made us this unbelievably realistic world to fly within. But then it's kind of up to the user at home to try to recreate a workspace that puts them, you know, immerses them in that flight world. And there are so many options you can go down, right? We could spend three hours today just talking about different control devices that you can use for things like the rudder pedals, things like joysticks for different types of airplanes. But what I want to talk about, because this episode is about what we see when we're playing games, is what some of the gamers in this community have done to create visual experiences that are a little above and beyond what maybe a typical gamer has with a 16x9 computer screen in front of them.

17:26What are some of the things that you've heard about, or maybe you've even personally tried to kind of go beyond the 16 by 9 box that constrains a normal flight sim experience? The most easy and frankly, the most popular next step is to go with multiple monitors. Some folks will do two, but then you have that bezel kind of right in front of you there. So the kind of the standard is three monitors. And so there are companies that will make triple monitor arms with the intent being that you tell the program, be it Microsoft Flight Simulator or X-Plane or Star Citizen, whatever you're using, I have three monitors.

18:09And then you kind of calibrate it in and you've got a view out the front of the canopy on your middle monitor to whatever degree works for you, a view out the left side of the canopy and the right side of the canopy. And at its most fundamental, this lets you do things like line up your approach when you're landing, when you're making that final turn. You can see the runway. You can time that turn appropriately without doing any crazy stuff with a hat switch or your keyboard. So that is kind of the most typical adaptation. The next step up from there is actually my preferred way to fly. And that's with something called the track IR or track infrared.

18:46So if you remember the Wii, right, the dearly departed Nintendo Wii, it had that remote control and there was an infrared emitter on the front of the remote control. Then you'd put that bar up on the top of your TV and the bar would track where you're pointing the Wiimote, right? Sure. Track IR works basically the same way. You get three little IR emitters. You clip those to your headset or you clip it to your hat. You just make sure it's on your head. And then there is a receiving panel that you put on the top of your middle monitor. And it just kind of sits there. And yeah, as you move your head, the panel on your monitor can see how those three points of light move in space.

19:31And it interprets that as head movement inside the game. So when I'm using my track IR, I can just look ever so slightly to my left. In the calibration of the track IR, it will accelerate my head movement inside the game. So even though I'm still looking at my middle monitor on the screen, what I'll be seeing is fully behind me. Very cool. And so you can calibrate this to meet your needs for a number of different games. I use it also for Arma 3. I used to use it for DayZ. And it gave me a neck, which is not something that you get in most first-person shooters. Tremendously helpful when you're looking down a gun sight to be able to look to the left.

20:10look to the right without taking that gun off your shoulder. But in a flight simulator, it's just situational awareness. And it's increasingly important when you're doing any type of combat flying. So when I'm playing in Elite Dangerous, it's very easy to lose my quarry, to lose my bogey, lose the thing that I'm trying to shoot at. And I can look around really with my actual head and the view changes on my screen and it is delightful. An alternative solution to that, and it's something that I've had less success with, it's eye tracking technology. A company called Tobii does that, or Tobii. I don't know how to pronounce that company correctly.

20:49But it's more in line with the form factor of the old Wii. It kind of sits on the bottom or the top of your monitor, and it literally looks at your eyes. And as you move your eyes, it changes the view for you. So it seems like, Charlie, peripheral vision in flight is essential, right? Absolutely. So, you know, a casual gamer like myself, I can't help wondering, why not just throw a VR headset on your face? Have you tried that at all? I have. I have. Actually, in Elite Dangerous, I traveled from the bubble, which is the area right around the Sol system, in the unfashionable western spiral arm of our Milky Way galaxy.

21:27And I went all the way across 65 ,000 plus light years to Beagle Point, one of the furthest reaches in that game. And I spent four months nightly making that trip. And I did it all in virtual reality with kind of, I think it was a launch Oculus, if I remember correctly. So you have a lot of experience in VR and I would imagine a very strong neck. Yeah, yeah. It was daunting, man. Probably like two to four hours per week night. Yeah, it was a lot of travel. I was doing probably like 80 jumps a night to try and get where I was going. And anybody who's played Elite knows that that's a lot of pushing the throttle back and forth.

22:06But I've also done it in Microsoft Flight Simulator. I've done it in, I've tried to do it in Star Citizen. And I've done it in a few other flight simulators as well. Okay, yeah. I mean, as an apartment dweller, I'm looking around at my space thinking, two or three screens, that'd be hard to fit on my desk. But a VR headset seems, at least at first thought, to be the perfect solution for this. If I move my head up or down, left or right, it's going to show me what I would be looking at in a cockpit. But there have to be some downsides other than the cost. Tell me about that four months in VR. What was it like?

22:42I mean, Elite Dangerous is a real pleasure in VR. And there's a reason for that. You're often not looking at very much. It's space, Liam. There's a planet. It's very far away. Looks like a star. There's a couple thousand of those. And that's it for the majority of your trip. Now, you get closer to a single celestial body. It's very easy for that circa 2014 client to get the visuals right as you're coming closer to it. So, rarely do I have issues with frame rate in Elite Dangerous. Microsoft Flight Simulator is real hard, though. And we're talking about, I think I had an i7 or an i9, 32 gigs of RAM, a 2060 card I was working with.

23:29And Microsoft Flight Simulator was still very challenging frame rate wise. And when you are flying, you do not want to have a delay in your frame rates. You need to know where that airplane is relative to air traffic around you, relative to the ground, relative to the runway at all times. So aside from just, you know, the nausea that can be induced by the frames locking up or skipping, not having that situational awareness is going to end your playthrough very quickly. But the other big issue that I had on my trip to Beagle Point and Elite Dangerous was, well, I couldn't see the keyboard. And Elite Dangerous is unique.

24:09They actually model the Psytec X52 joystick into the game. And so it was a partnership between SciTech, recently purchased by Logitech, and Frontier Developments to put their product in the game. So I looked down in real life at the SciTech X52, same controller I'm using in the game. But it doesn't move the same way all the time because I have my bindings set differently than the game is expecting me to have my bindings set. So when I use the rudder, it shows my hand twisting. But no, my feet are moving because I have rudder pedals. So that's not one-to-one. But there's a lot of buttons in the cockpit, and I need this keyboard in front of me in order to hit those buttons.

24:51And so that required a lot of remapping of the controller. I was able to remap what I needed to do all those light speed jumps, to do all the landing, to do all the maneuvering, to the buttons on my Warthog throttle and the buttons on my Warthog. I think it's an F16 stick that I swapped out for the traditional Warthog stick. But I was able to do everything I needed to do using the buttons on the joystick, including reset the center of my VR. But I couldn't shut down the game. I couldn't pause the game for that. I had to take off the helmet. I had to look down at my keyboard. And it takes you out of the experience a little bit.

25:32But mostly it's a performance issue. Like if the frames are not where you need them to be, it's a very uncomfortable experience. It's not productive, frankly. It's better, I feel, to have a higher frame rate, to have it on a monitor in front of you. And I prefer to use TrackIR over VR because of that. Well, it seems like the community agrees with you. I watched many, many hours of people excitedly trying out headsets from the MetaQuest 3, the Valve Index, of course, the HTC Vive. And one in particular stood out to me that a lot of flight sim enthusiasts were excited about last year, which was the Pimax Crystal.

26:12And it seems like a very similar story to what we're seeing with even the Apple Vision Pro, the kind of consumer-focused version of one of these VR headsets. And that, you know, there's a huge amount of anticipation and excitement. And then, of course, you get the thing and you try it on and you see all the edge cases and issues with it. And you now have this very expensive device that you can strap to your head, but you don't want to anymore. It seems like from what I'm gathering from the Flight Sim YouTube community is that a lot of people have tried these VR headsets and gone back to that three monitor setup that you're talking about.

26:52Because in the end, that just still provides a better experience. Would you agree with that? I would agree with that. It's absolutely been my experience. It's been my experience in high-end Flight Sim YouTubers that I follow. The folks like the Grim Reapers. The majority of their team, I think, does the one or two or three monitor setup. And so do the ground training centers for the F-35 and the F-22. Like when you look at the military training regime for those latest generation fighters, you know, it's a tactile experience for them. They need to know that the buttons in the cockpit are where the buttons are going to be in the actual cockpit.

27:38And that's the majority, I think, of what they're training to. I need to hit this switch without even looking at it. That's the work. And they should already know how to fly the airplane by that point. But yeah, technologically, it seems to be a better experience just working with screens. Okay, we're going to take another quick break. Then we'll get into whether VR will ever be the future of flight simulators. We'll be right back.