This Company Mapped the Entire World in 3D. Here's Why.

21:34Grant:Yeah, I like that. Yeah, let's actually unpack the Tensor Globe at a high level. There's Cortex, Forge, and Nexus. We kind of covered Forge. Can you explain Cortex and Nexus for us? Yeah, for sure. So I think, you know, a great platform always emerges from your own operations. And I like to think about, you know, whether it's the iOS App Store and Apple writing those first couple mail apps and you know, working with Google on YouTube and Google Maps, you know, those initial apps are the thing that becomes the platform. I think the example for us that's a little bit more close to home is the example of AWS, where they didn't set out to build AWS at Amazon, they set out to build amazon.com.

22:15And to build an e commerce system in the early 2000s and late 90s, you know, you had to build storage, you had to build commute, you had to build a whole networking layer, you had to build routing systems and DNS entries and all this infrastructure that they built to power amazon.com. And the really inspired thing they did is then they productized that platform, right? But, you know, I think when I think about Tensor Globe, the platform we're launching and launched last year, it's really productizing the core operations of Vantor. So what we do every day with a team of, you know, 2000 people across our whole operation center doing constellation scheduling, production, quality assurance, distribution, exploitation, the whole sort of what we call in the industry, like the TC-PED intelligence cycle of tasking, collection, production, exploitation, dissemination.

23:09That whole cycle is really what we've done inside of Vantor for 20 years. And what we're doing with Tensor Globe is sort of productizing it so that other organizations can do it. So, you know, with Cortex, that's really a system that's designed for constellation management. managing our constellation, managing our constellation, working with other constellations, and trying to understand if you have a goal, like you want to monitor 150 mines for activity or 250 electricity plants for activity, right? We can schedule that constellation across a bunch of different individual sensors and individual satellites.

23:44So that's really the cortex piece is that constellation management element. Forge we talked about is really that fusion element. So taking data from lots of sources and producing a 3D globe. Nexus is really that API and UI layer for looking at the globe, for accessing the globe, for accessing the raw materials and sort of the evidence behind or, you know, the raw data that is being used to produce that 3D world. And then over the next couple months, we'll be launching another component of that platform, which is really focused on looking at the globe, right? Understanding what's happening in the world.

24:18And this, I think, is something we've been working really closely with the Google team on and working with a lot of their foundation models and agent development kit to really apply reasoning to that globe. You can almost think of it as, you know, that that real time strategy look at the world where you're trying to understand what are the patterns happening across very different parts of the world that are actually connected, whether that's a supply chain or, you know, some sort of order of battle workflow in a defense context. yeah you know that that's really that that fourth piece is is providing understanding

24:52Grant:i'm sure you've seen some of the demos on x like of people trying to do this without your level of ability but like where they'll have like these like booty maps of the globe and it's like here's the map of all the different conflicts happening like currently and and it's it's really cool to see that but it'd be even cooler to see it with your level of sophistication and data yeah i think that's right. And I think I think those are amazing. I've loved looking at them. I, you know, we work with some of the people who've been working on them. And it's been awesome just seeing the creativity of, yeah, you can do with these tools.

25:25I think, you know, one of the things that is is true is that, you know, we've really standardized for the last 25 years on a zoomable map as the sort of fundamental substrate that we think of when we think about digital mapping. And, you know, the thing that's very odd about AI is that it can look at the whole world. And we can't. Like, we have to zoom in and zoom out and zoom in and zoom out. And I think a lot of the exciting opportunities, you know, is really applying AI at that very granular level and then rolling up some of these, you know, observations or alerts to a global view, sort of men in black style or, you know, like situation room style.

26:06but the thing from an uh you know when you zoom out in the world it's very low resolution obviously that's sort of how it works is you're compressing the data as you zoom out so that you just see africa and you just see you know the americas you're not seeing every single one of the hundreds of millions of pixels that makes that up and i think you know as a human we just

26:26Grant:can't do that it's like we even have a compute compute cluster strong enough to do that at the resolution we would want? I mean, I think this is like with many of these agentic systems, product development and technology development are different. And I think a lot of the art of product development is using the technology, seeing where the technology is going, and then sort of cheating to build something that actually works today and is actually useful and tricks the user a little bit. There's a lot of art as you zoom out today in the way that the pixels decompress and recompress and the way the vectors sort of do line simplification.

27:04There's tons of technology there that makes it feel like all the data is down there. Yeah. In terms of compute clusters, this really is part of, you know, why we were so excited to partner with Google is you really, I think, can't overemphasize how much money these companies are spending on compute. And from our perspective, you know, we own a constellation of satellites that works in the public cloud environment. And being able to run AI in a public cloud as opposed to in a classified environment, that's just a real change over the last 10 years in terms of, you know, the amount of compute buildup is, you know, orders of magnitude, roughly the same size is the U.S.

27:50defense budget today. Like the careback expenditure from the top five, 10 hyperscalers is probably 70, 80 % of total U.S. defense spending. And so, you know, one of the things we're excited about from a corporate perspective is just, you know, our sensors, our satellites that we've built and put on orbit over the past 20 years, you know, those are directly, you know, we can plug them right into a Google TPU and running next generation frontier models on the really high resolution data. And so, you know, to your earlier question, like, do we have a compute cluster big enough? Who knows? But I think we definitely have very big compute clusters that are capable of processing really, really high resolution data.

28:31And I think that just is a huge unlock in terms of, you know, what can be done. For so long, you know, I've been in this industry for a while. We've been so focused on computer vision algorithms. And I think that so much of the like exciting shift to spatial intelligence is really, you know, what happens after you identify all the vehicles in a parking lot, right? That's always been the task that felt tractable. And so I think as a technologist, we've put a lot of time into trying to solve that problem, because it was like a well specified task that was close enough to the current capabilities that you could actually go after it.

29:07It was like a little bit crazy to imagine building a talking computer that would look at the pictures and tell you what was happening. And that's not crazy at all anymore, right? Like that is actually totally in scope, but it's a really different ambition than I think what the industry has been focused on for 15 years, which is really much more image segmentation, sort of traditional computer vision algorithms versus these more advanced sort of reasoning algorithms.

29:34Grant:So going back to your earlier point about, well, there's a, there's the cool thing that you're building now where you can use the the i guess the layers of all this information but then to your earlier point about the actual layers of the information you know you can do it seems like you can map a lot more than just the number of cars that are in a built like you can you could almost get to the point where you're mapping like the chemical makeup of everything if you had sensors strong enough where you could say like this car is made yeah you don't want hyperspectral sensors for that yeah which is in our specialty but but yeah i mean at some point though right you could if you're if you're able to embed all this information you can just keep stacking layers and layers of information to the point where you can almost make a realistic 100 complete um yeah there's a famous borges uh short story about sort of these map makers who tried to make the most accurate map in the world eventually i think it's called inexactitude in science but eventually they made a map that was perfect but obviously it was the whole size of the world.

30:35So it was leather that wrapped around the whole world and people were wandering around seeing like pieces of the map falling apart. The nice thing about the digital world is you can truly map the whole world at incredible fidelity and incredible precision to your point. In terms of the use cases, yeah, we really think about a lot of what we're doing on the insights and AI side is, you know, developing a language and a grammar for spatial data. So if you think about, you know, image stack of pictures of the same location over time, you know, you can almost think of those as words that are then forming a sentence, right?

31:07And maybe you have another stack over here, that's another sentence. And so you're creating paragraphs. And I think a lot of what we're focused on are sort of what are those higher level abstractions that we're trying to build towards, obviously mediated by language from a human perspective. But I think for a lot of machines, these are going to be, you know, subspaces within embedding space that have some semantic content.

31:30Grant:There's a great paper from Sandia. This is an old paper, like 2014 or something about like geospatial semantic graphs around like, well, if you see a track next to a building next to a parking lot, that's probably a high school, right? And like, that's a pattern that is like high school, even though it's made up of four or five sub patterns. And I think as you sort of think about mapping the world, going back to that New York subway map analogy, like I'd love for us to be able to have, you know, the dynamism of a zoomable map with the semantic relevance of a lot of these features that, you know, pop.

32:12And I think a lot of the experience that Google and Apple and a lot of these zoomable maps have really nailed is like, as you zoom out, California replaces San Francisco and, you know, the labels change. And that's all based on, you know, hard coded relevance and semantic systems underpinning it like a knowledge graph effectively. But I think, you know, as we think about this vocabulary or grammar or language for events happening over time, you know, you have these ribbons that are fixed, you have ribbons that are moving around, you know, the world is made of entities that move around it. And I think a lot of the language that we're going to develop from a spatial intelligence perspective is going to be, you know, anchored on describing those patterns and formalizing them so that, you know, they become things that both computers and humans can sort of reason about objectively.

33:02Grant:Okay, I'm going to, I'm going to try and, and I'm going to posit an explanation here and I'm going to see if I got this right. So if you do that well, and based on where you're at today, that look like equivalent to zooming in and zooming out, like we think of it on the Google Maps example. But as you zoom out into like a 3D space, it starts to see, it's able to describe things like the San Francisco, California equivalent. But like, let's say you're really zoomed in in this 3D version on like the tire of a car. And it's like tire. And then you zoom out and it's car. And you zoom out and then it's street.

33:35Grant:And then it's zoomed out. And then to the point where you could then, no matter where you are in that level of zoom, you can kind of make predictions and explain what's at any layer of that stack? Is that more or less what you're building towards? Is that what you have today? Is that accurate? Yeah, I think that's right. That would be almost like a semantic hierarchy. And I think one of the things that's always hard when you're designing a taxonomy or an ontology is when is something a difference in degree and when is something a difference in kind? Like when is it a lion and a tiger versus a cat, right?

34:07And we build these taxonomies that help us understand the world, help us communicate about the distinctions. And I think one of the things that's really critical as we think about, you know, the new words and new terminology we're going to need in this future, right? The word molecule didn't exist. The word chemical didn't exist. Like people invented words as science and technology advanced, right? Computer used to be a job, not a machine. It was someone who tabulated numbers. And I think as we think about the next generation of sort of words, we are going to need words for these sort of like spatial abstractions.

34:41I think that's exactly right. In terms of like, maybe what you were getting at, in terms of predictive, you know, this is something that's that's pretty new for us. We haven't done any, you know, deep research on it or deep development on it. But, you know, as we think about, you know, image generation, I think there's a real opportunity with world models to generate synthetic imagery and say, okay, we're planning to take a picture of this location in three hours, let's generate the image that we think we're going to take, right? And then let's diff that image against the image we actually get.

35:14That's really when you think about learning, it's sort of making a prediction and then falsifying it with an observation. That's sort of the classic scientific method is, hey, we think this is going to happen. We're going to run an observation or an experiment, and then we're going to validate that. And I think a lot of what we're going to start seeing from an intelligence tradecraft perspective is a ton of in silico image generation, almost running a synthetic war game where we're generating all the images that we anticipate over the next week. And then we're diffing them against various courses of action to understand as those observations come in, you know, are they corroborating our hypothesis or are they rejecting the hypothesis?

35:53And, you know, we can even use that for benchmarking your models too, right? Like you could use it

35:57Grant:for benchmarking your prediction models. You could use it for benchmarking even your image models where you're saying like, we think that, you know, know, if this world model is accurate and capturing the physics, we think it'll look like this when we go out and take this picture of like, you know, a cannon firing at a, you know, you know, a tree or something. And then you go out, you take the picture, and then you see how accurately it got, you know, the image models. Exactly. And I think, you know, as humans, we can launder information from high resolution into low resolution, we can launder information from low resolution into high resolution, like, we're built with a world model built into our brains, right?

36:30Like, I'm not a neuroscientist, I don't know where it is. But, you know, fundamentally, like, from a very young age, humans are developing a sense of physics, they're developing a sense of causality, they're developing a sense of sort of how things occur, and some basic intuition around like, if this, then that in a physical world, senses of gravity, of sort of permanence of objects, right. And when you think about the projection of sort of in silico versions, or synthetic world models, You know, you can imagine them being able to do very similar things, right? If I take a picture of a car on one day and it's a red car at really high resolution and I know it's license plate and then I take a low resolution picture of a red car, I can infer based on my understanding of patterns, my understanding of what's happening in the world, how things occur, that they might be the same.

37:19Now, I'm not going to set that at 100 % the same, right? I'm going to apply an Asian discount to that. I think that's sort of the thing that's really important about anchoring all of this on a sort of unitary digital globe is ultimately the world is happening, whether you observe it or not. And a lot of the work of intelligence analysis, a lot of the work of spatial reasoning is trying to figure out what's happening when you're not taking a picture, right? Like what's happening between the frames? And the idea of being able to generate synthetic frames, have analysts produce key frames in the middle and say, well, based on this and this, I think something happened here.

37:56Generate me a video of if this happened here, what was the full video of the world? Right. I think that's really interesting, sort of similar to a lot of the generative video techniques, similar to a lot of generative images techniques. But that's really useful for like forensic analysis, right?

38:11Grant:Like if you're like, like, you know, you're in a legal case, you could use this model and say like, OK, this is what we think happened in this in this instance based on the evidence. Exactly. I think, you know, digital forensics is, I think, a really interesting use case for this where, you know, you collect a ton of evidence. You have dozens of video files, camera files, people who saw pieces of the event, people who saw, you know, who got a, you know, body camera view of only a little bit. But there's a really cool, ProPublica did a great story where they took all of the video files off of Parler during January 6th.

38:44And they sort of arranged them spatially and temporally so that rather than, you know, in a traditional video file, it starts at zero and it plays forward to 30 seconds or 45 seconds. What they did is they took all those and they aligned it with like 12.14 p.m., 12.19 p.m., right? So it played from 12.14 to 12.14.30 and 12.19 to 12.19.30. So they arrange them in time, they arrange them a little bit in space. But, you know, I think a lot of our vision for what we're building is a world where you never look at a video again. You're only looking at a map. And the video is on the map, but you can zoom out and you can zoom in.

39:21And you're really thinking of the video more as a viewport of the map versus an actual data file that you're like clicking on and looking at.

39:29Grant:Yeah, love that. So that's so many branching ideas here to talk about. But I want to go back to one that I was thinking of earlier when we were talking about the idea of compressing and how humans, you know, we have sort of like this idea of like we can abstract it on a high level and at a low level and we have a predictive model built inside of us. Could you ever compress the world model that you've built or any of the versions of your world model to the point where it would be something that a robot would run on board? Maybe it's not necessary for it to do that, but like, could this ever be something that's fundamental in a AGI like system, for example?

40:06Yeah. So as part of we launched a product last year called Raptor, which is really designed to be a GPS resilient positioning technology. So, you know, the way GPS works right now is satellites, you know, essentially have atomic clocks on them that are highly accurate. just say this is the time this is the time this is the time you receive times from multiple clocks and you sort of do the trigonometry to figure out where you are in the modern battle space people are spoofing those all the time or jamming them and so the gps can go down often does go down i think if you're in an urban canyon in a sort of normal commercial environment i'm sure you've experienced that thing where your gps is like i'm not quite sure where i am i'm like on this road but i'm actually a road over not the end of the world the way it would be in a battle situation you know annoying and certainly bad raptor is really a product we designed to you know navigate the way humans navigate which is looking around and seeing the 3d world and understanding okay based on my knowledge of london and my observations through my glasses of london this is where i think i am and so as part of that we do compress that world model down we do what's called chipping and shipping where we cut out piece of model at high resolution other pieces of the model at low resolution and then we give that to the device the robot or the machine so that they essentially have a map that they can look at and compare what their camera is seeing in 3d to what our world map sees in 3d and that gets you positional accuracy very close to what you get with gps and with the added advantage of you know where you're looking so with the gps system you know where you are but you don't know where you're looking and yeah and that's a huge deal exactly so i think that's a huge, you know, form of compression that we use today.

41:51And I think a lot of the challenge of this, again, going back to sort of what makes it hard to build these systems, you know, there's obviously an AI model development element, which is not my personal area of expertise. But there's a huge sort of data orchestration element of moving data, updating data, making sure you have the right clocks, the right update semantics, sort of this multi master version control system of, you know, how do you take all the data that's coming in and updating the representation of the world and then make sure everyone understands what version of the world are they looking at?

42:24And, you know, you can really think of in the same way that a Git repository or a version control system would do that for a piece of code. You know, our globe would do that for these physical systems, which are both looking at the world and observing it, but then also ideally working together so that they can tip and queue and sort of operate off of a shared digital replica. That's awesome.

42:44Grant:Would it ever be possible to give that kind of level of feature control to like an iPhone app or something? Totally. So, I mean, this is we, you know, a lot of people, you know, at Vantor have worked for a long time with some of the folks at Niantic. Last year, we had a partnership with them really specifically on, you know, using their augmented reality system against our globe so that you could do augmented reality globally. I think, you know, one of the things about GPS, it's a global positioning system. And so while it's possible to do sort of inset mapping or mapping of particular urban areas, if you want to go off road, literally, you need a global map, a global three.

43:27You need a local positioning system, basically. Exactly. We call it a terrain positioning system, a TPS. Yeah, yeah. That's exactly right. And so taking their VPS and making a global, you know, requires them to look at a global representation of the world. And that may not be particularly, you know, important in certain consumer enterprise use cases. You know, if you're Walt Disney World and you're trying to build awesome augmented reality, you know, experiences for Walt Disney World, you don't need that app to work across the whole world. But, you know, if you're a company trying to build AR experiences that do work globally, you know, I think that's really the genesis of that partnership was trying to bring some of their awesome technology.

44:07We've built technology that helps positioning in the air, but they really are the experts on the ground. And so we've been working with them on sort of connecting those two pieces. That's awesome.

44:17Grant:I want to talk a little bit here before we wrap up about like how businesses can use this. But before we do, it sounds like you have essentially you've been focused on trying to make all of this information and data that this world model that you've created, the ground truth world model, as we've called it, accessible to like today's language models and reasoning models. But what about the vision models and the world models? Like, like, will there will there ever be a version of this data where, like, let's say, like an agent system would actually be going inside of the 3D representations that you've created?

44:49Grant:And how do you think about building for that use case as well? Yeah, I think that's definitely right. There's a bunch of simulation workflows along those lines, you know, whether it's trying to train AI pilots or, you know, robotic systems to maneuver in the digital world. You know, if you if you talk to people at the at the self-driving company, most of the miles driven are synthetic miles, right? They're not actual miles driven on the highway just because the number of surprising events happening in the real world, you know, you can't get enough of a corpus. So you want to create synthetic environments that really push you from a development perspective.

45:23So, yeah, I mean, I think that bringing, you know, let's say models that are trying to understand the world into that digital world, I think is really critical. We currently do most of our perception layer was what we call it is really that sort of segmentation, computer vision. That perception layer today is mostly done in 2D space. A lot of the work we've been doing over the last year is being able to take a single image or a single stereo pair image from our satellite and create an on the fly 3D image. And, you know, have been doing a little bit of work now on the on the AI perception side to see, you know, how much better does the perception model get when you give it a 3D image versus when you give it a 2D image?

46:03And I think that's a particularly interesting area of research where, you know, so much of the perception technology stack has been about perceiving 2D images, segmenting 2D images. But obviously with the third dimension, you're getting a ton more information. You know, certainly when I think about analyzing 2D images, a lot of what I'm doing is if I'm metacognating and really thinking about like, how do I interpret a 2D scene? A lot of it is by constructing a 3D scene under the hood, right? Totally.

46:33Grant:Yeah, I think all living intelligence like lives in a 3D world. Right. So like, I think that that at the end of the day has to be a component of any sort of agentic AGI system is that it has to be able to think in 3D space at a certain point. I think it's like it's it's always tempting to anthropomorphize things like but we do. Yeah. Prior to three years ago, there was one entity in the world that spoke natural language. Right. Like maybe parrots spoke a little bit of it. Like there was one, maybe 10 neural architectures that had like some degree of of language capabilities. And there's a new one.

47:07I think that's very exciting. It's likely there are going to be similarities. It's likely they're going to be big differences. But I think from a sort of pure information theory perspective, providing a perception model with a 3D representation, I think is likely to provide much better like f1 outputs in terms of extraction of vectors

47:25Grant:and semantic understanding of the scene yeah and then I think taking that a step further there's a lot of people who think that for an AGI like system to actually be true AGI has to be an agent that takes actions and reacts that can make decisions and that sort of things then then you know you would you would probably want to train that type of system in a 3d environment yeah yeah Yeah, I mean, I think that's right. I will say, you know, as you zoom out, the world becomes more two-dimensional, right? Like if you take off on a plane, like you look down, it's pretty 2D. You're getting ready to land.

47:57It becomes much more 3D. And so I think it also depends on sort of the vantage point you're operating in. There are a lot of domains in which we, as humans, operate intelligently in effectively a 2D space, right? Whether it's certain types of surgical routines, certain types of workflows, there are certain types of technology that have low-dimensional inputs and low-dimensional outputs that we operate intelligently and interact with in a sophisticated manner. I think our computer has a sort of one-dimensional interface in terms of the keyboard, and we're able to do quite a lot with that one-dimensional system.

48:36That's fair, yeah. I don't know that I buy that, like, this, that you have to be operating in 3D space to be intelligent. But I think that there's a class of very important actions in the world that happen physically that are going to be mediated robotically to some degree over the next five, 10 years. And those actions do require physical intuition. And I suspect that those actions are some of the most important actions that end up getting, getting, getting taken in the world. I just think, for example, like, you know, stock trading, you don't you don't need to walk around Wall Street to do stock trading.

49:09Grant:Yeah, that's true. That's a good that's a good counterpoint. I like that. Well, I do want to touch on before we wrap up here, how can people use the tools that you're created? How can they work with you? What's we talked about a couple of the tools in Tensor Globe? But yeah, just walk us through like how how someone could work with you if they wanted to. Yeah, I mean, I think that the best way, especially for folks in this audience, probably is really, you know, getting access to some of the data as a service products, 3D vector products, really sort of getting access to the mapping data. We host that in a cloud system that's accessible through API and have a good licensing structure.

49:46We also have a lot of publicly available data through our open data program for a lot of sort of wildfire events, things that are of public note. And so I think that's a place where people can play around. We run a number of sort of machine learning exercises and have made pieces of our archive accessible to various university groups and other people who are sort of training models, working on particular tasks. I think we did a SAR model a couple of years ago. We've done some interesting image segmentation, prize opportunities. That's, I think, a natural way. But yeah, I mean, I'm on Twitter. I'm on LinkedIn.

50:21Feel free to reach out to me directly. I'm happy to put you in contact with the right people. Yeah, has been an awesome conversation. So feel free to reach out and I'll be sure to reply. Awesome.

50:33Grant:Thank you very much, Peter. Thanks for joining us. And thanks for walking us through all of this and what it actually takes to build that ground truth foundation model, like ground truth world model. It's really awesome what you're building. Thanks. Yeah, hopefully that language kicks in because I think it'll be, you know, it's always good to lock in the right terms when you're building technology. And I think right now the world model is just it's a little bit too big in terms of it could be it's sort of in the eye of the beholder. And it's also like it can be it can mean a perception model as well.

51:02Grant:Right. Like I think I think of it as like you're generating the world, but that could just be like my model of the world. So you're right. It's totally out there. Right. So we'll see. I think this is going to evolve a lot in the next year or two years. A lot of really good people working in this space. So it's definitely exciting, exciting place to be spending some time. I guess all of this conversation, you know, you brought this up off camera and it makes sense to me, reminds us of the metaverse conversation, right? And I think you have some interesting thoughts there if you want to touch on the metaverse.

51:29I mean, I think we've lived in this weird part of history where the dominant flow of technology has been sort of from the consumer space into the enterprise space, which is really historically anomalous, right? Like if you think about how did technology get developed in the 1800s, you know, a lot of technology was developed in the enterprise space and sort of percolated into the consumer space, right? um and and i think you know one of the key questions is sort of is spatial intelligence an enterprise technology or consumer technology and what's going to be the the diffusion arrow is it going to go from consumer to enterprise the way like an iphone did or a personal computer did or is it going to go sort of from enterprise to consumer the way you know a more traditional military technology did let's say like the internet moved from sort of military to consumer The GPS system moved from military to consumer.

52:22Airplanes moved from military to consumer. Like a traditional pattern of technology development historically has been that, you know, the defense industry has pushed the frontier. And then that technology has diffused, whether you look at the Apollo program or the Manhattan program. Military technologies have been then repurposed in the consumer world to, you know, drive productivity there. And I do think that augmented reality in particular is a technology that's likely to be developed in the military context, or at least in the sort of physical enterprise context, right? It's a very challenging technology to develop.

52:57It requires, you know, people who are really interacting with the world in complicated ways, doing sort of physically difficult tasks for it to really be useful. and I think you know it's really that shift from you know do people really want virtual reality or do they want augmented reality I think augmented reality yeah I mean there's use cases for virtual

53:17Grant:reality but I think we can leave virtual reality for the machines I think what we would like is to all of the information we've been talking about is like accessible to you like if I'm looking around my room you know you and I both wear glasses so we're comfortable with wearing glasses and I think people who aren't maybe are a little less comfortable but if I could just look around the room and see like have the ability to like focus in on something and it give me all the information about that object i think that would be really cool or being able to manipulate you know on my computer in a 3d space instead of a 2d space i think that'd be really cool i totally agree and i think you know ultimately i think one of the hardest things about building products is you have to sort of hold two things in your head at one time one the product is important and other the product is not important.

54:04And so if you're working on Netflix, you're coming into work every day. You're like, Netflix, Netflix, Netflix. It's 40, 60, 80 hours of your day. As a user, it's just Netflix. It's just Uber. It's just a product. And I think especially with augmented reality, which is this really persistent part of your life, I recently got a new oven. And the oven really wants me to use its app and make its app the center of my life. And the app has all these features. And it's like, you could get recipes, we could integrate, it wants to be the center of my life. And I think with augmented reality, it's almost even more dangerous, because it's, it's something where you almost have to be very subtle about the way in which you you create the user experience.

54:45Because, you know, as much as you as the developer, as the developer might want to sort of like, be like, Look, I know all this stuff about the world, you're looking through the glasses this is a dell monitor this is a macbook this is expo what it's like i as a user don't want it showing me all that like i think no that'd be way too busy yeah human computer interaction and i think a lot of this is going to be sort of what's the click effect what are the persistent parts of the augmented reality experience versus the transient parts and i think developing that in a defense in a sort of hardcore industrial setting you know that's where a thing like like Apple Vision Pro, you're like, the market for that is not people sitting on their couch watching Netflix, right?

55:26It's people building stuff, people trying to learn carpentry, people trying to like, fix their car, teach their kid how to play baseball, right? Like, it's people doing stuff in the real world. Like, that's the market. And I think when you look at the marketing of these technologies, it's truly dystopian. It's like people sitting on their couch alone, watching slop. And I just don't think that's the right way to position the technologies. I don't think it's the way the technologies are going to have a great impact on the world. And I also think it's a really depressing vision of the future.

55:55Grant:Well, I'll give you an alternate take, which is imagine AR replacing YouTube. So I was, for example, I was using a Rubik's Cube recently, and I was like watching YouTube videos to try to memorize like exactly how to do it. But imagine if you, instead of watching a YouTube video that you have to go back, click play, you can combine AI with an AR interface. And it's like, just shows you a little arrow, like turn it this way. Now I'll turn it that way. I'll turn it this way. And then I can be like, why am I doing this? And it could talk me through it. And that would be so much more intuitive. And I would never have to watch that video.

56:26Exactly. And I think it's sort of the video is that pixel representation. What you're describing is a vector representation, right? Where you're reprojecting those pixels into your pixel space. And whether it's learning Rubik's Cube or fixing your toilet or, you know, doing a carpentry project, like having lightweight cues. You know, I think about when you're putting together Ikea furniture, you're like constantly looking from pixels to furniture, from pixels to furniture. And, you know, I think it would be an incredible experience to have Ikea in my glasses saying that's, you know, object 12, put object 12 in object 11.

56:59Like, that's actually a really cool experience you could imagine. But it really, again, requires you to almost drop the pixels, use the pixels for the machine and produce vector cues for the end user in the augmented reality space that are sort of reprojecting the content from a YouTube video or from millions of YouTube Rubik's Cube videos into this idea that like Grant is holding a Rubik's Cube and that Rubik's Cube is, you know, rotatable in these ways. and I'm going to produce cues from his glasses that know how far his hands are from his face. Like, that's a really tough problem.

57:34Grant:And then there's a whole other layer as the product developer where you want to have a predictive model that knows exactly when Grant needs those cues, right? Where it's like, he's confused, he's looking at it, like, now I should add them in. I think that's right, and I think that's where, you know, it's going to be really hard. And I think a lot about the Alexa problem of, you know, it felt like I had to go to Hogwarts and learn all the spells of like Alexa, timer, Alexa. You know, I never knew what it could do, what it couldn't do. I think the affordances of a voice-based or vision-based system are really tricky compared to the affordances of either a physical or a screen-based system in terms of teaching the user how to use the technology.

58:15And to your point, the worst version of it is like, hey Grant, like, did you know you can set timers on Alexa? You're like, shut up, Alexa. Like, I don't want that in my room right now. And so I think that's going to be a really challenging Like I suspect for these technologies, there's going to be some sort of almost, you know, in a video game where you start and you learn how to jump and you learn how to there's like a training part of the video game. I think for both augmented reality and just voice technology generally, you need some sort of training environment where you as a human user are like learning the affordances of the system and it's constantly updating you as it changes.

58:49And I think that onboarding piece is really hard. And I actually think people haven't nailed it that well. And it's going to probably be one of the hardest parts of deploying this technology. Is that something you're building or something you want to partner with people to build? I think that would be something we'd probably partner with. You know, that's much more on the sort of like, I think of it as like the first person shooter view of the world versus the real time strategy view of the world. And a lot of what we're trying to do is join those. But I think a lot of that sort of like FPS experience is probably going to be more of like an app store experience.

59:21You're going to have you're going to want a proliferated set of developers. Maybe there's a Rubik's Cube app you install on your glasses. Maybe there's a kind of plugins like Co-Works really pushing plugins. Exactly. Yeah, because I think part of that is, you know, one of the powerful things about iPhone or Android is that, you know, based on the apps you install, you're taking an action to like choose the Marriott app and choose the Chase banking app and like you're sort of configuring it to your world and I think similarly it's going to make a lot of sense to be like install the Rubik's Cube app you're going to know that you did that you're going to know that that affordance is available versus just having like a Rubik's Cube app pop up out of nowhere right but I do think like you know maybe to to close this out a little bit I do think one of the things I'm most bullish about is like really using the physical world for app discovery.

1:00:10You know, you go to Chipotle, your glasses have like a little button to install the Chipotle app. You go to McDonald's, your glasses have a little button to install the McDonald's app. Like, I think over time, you know, these physical, the physical world is going to be a much more important part of digital product discovery. As the digital world gets more and more confusing, I think the physical world, you know, you already see this in sort of channel marketing, where people are marketing in the physical world in order to generate impressions in the digital world, right? The like New York subway takeovers.

1:00:40The purpose of that isn't necessarily 100 % to have people on the subway download the app. It's almost to like take pictures of people on the subway with the posters to post on the Instagram to generate, you know, some sort of salience for the users of Instagram to understand that you cared enough to buy out the subway. And so I think more and more like the exclusivity and rivalrousness of the physical world is going to be a key sort of part of, right? It's like sort of that real estate thing of like location, location, location. I think as the digital world gets less and less exclusive, there's more and more content.

1:01:17You're going to see more navigation experiences in the digital world mediated by the physical world, whether it's QR codes, which I hate, but I think are very effective. Other sort of forms of almost like digital waypoints that are positioned in the physical world.

1:01:33Grant:And a lot of that could be technology that is like agent to agent, right? Like it could even be that like the human isn't even the interface, isn't even the interaction layer there, but it's like it exists so that the agent can prompt it for the human. Exactly. Like I think a great example is if you think about the future of a fast food restaurant, right? You know, will you be ordering, you know, will your agent talk to their agent? How will it know which agent to talk to? You're like, well, probably know based on where you are in the world, which agent to talk to, right? You don't want to talk to McDonald's.

1:02:01You want to talk to the specific McDonald's store that you're in. And probably a lot of that routing is going to be physically mediated, even in information space. I love it. That's awesome.

1:02:11Grant:All right. I don't want to take more of your time, even though I could talk about this forever. Me too. All right. Thank you so much. Yeah, of course. Good to see you. Take care.

1:02:28You