Google Maps Leaders Talk About Its Biggest Update in 10 Years

12:43Logan Kilpatrick:So, you know, I like to use this phrase, we bring out the human voices in this age of AI.

12:49Miriam Daniel:I like that.

12:50Logan Kilpatrick:Okay. Because we have 500 million contributors around the world, that's a whopping big number, right? But it's spread around the world. We get so many contributions from users. And off late, everything has become rich and visual, right? We actually had a push to make things rich and visual. So people have been uploading real photos of the food as it arrives on their table, right? and real photos of the restaurants they're in, the vibe that they're feeling. Sometimes even videos are being uploaded. And we've been using AI to actually correlate between the menus that people are uploading and the dish photos they're uploading and actually even naming the dishes.

13:31Logan Kilpatrick:So what you might see as part of these responses is very real, right? It's not just those marketing photos and videos.

13:38David Cronin:I love that. We hear from our users, actually, one of the favorite kinds of photo or video that we post is just the panorama, the 10-second panorama of what it's like at happy hour, right? What's the vibe? What is the noise level? How are people dressed? What the energy level is like? And so that kind of that authenticity that you talk about looking for is what a lot of people come to Maps looking for because they don't want the marketing version. They want to know what it's really going to be like when they go there. Yeah.

14:04Logan Kilpatrick:And if I may allow to plug something in, local merchants, the small merchants are very important to us, right? And to the users in the community. So we've actually been encouraging local merchants to record these panoramic videos, you know, or, you know, working with us to help promote that business. They're hidden gems, local gems. So you'll see some of these local and hidden gems come out in the answers as well. And, you know, our immersive view, which was a feature we launched two or three years ago, shows the vibe video in real. It's not a marketing video. It shows a real, what does this restaurant look like?

14:40Logan Kilpatrick:What is the vibe? do I want to go there?

14:42Miriam Daniel:I love that. My second feature request is the thing that my girlfriend actually does when we go to restaurants is we're like looking at the pictures to try to like correlate that to the menu. And I feel like if there's some way to do this in maps, this is the dream for me because I'm like, I can't decide what to eat unless I can see what the food looks like. And so you have to like do this manually. And I'm like trying to see like, do they say what they ordered in this picture so that I can attach that to the menu picture that they uploaded. I feel like this would be a great Ask Maps use case to be like, here's the menu.

15:17Miriam Daniel:Can you tell me what's on the, what user uploaded pictures correlate to those items? I feel like that would be a killer feature.

15:24Logan Kilpatrick:Stay tuned.

15:24Miriam Daniel:Thank you. I'm excited. Absolutely. We'll do it for IO. Awesome. Well, I want to see the new Maps experience. So maybe we can try out some demos.

15:33Logan Kilpatrick:We have some ready to go.

15:34Miriam Daniel:Let's do it. I'm excited.

15:36Logan Kilpatrick:So you see this, It's the home screen of maps that you're familiar with. You see the little chip. It's the first chip now, Ask Maps. You tap on it. You enter this conversational experience. I'd like to show something that might really apply to you. This is a query that I just tried this morning. I need a gym that is open at 6 a.m., has clean showers, and is along my route between home and work. And there's a lot of little nuances here, you know. So the first thing is timing. Clean showers, which is not easy to know unless somebody recommended the gym to you. And somewhere between home and work, right?

16:18Logan Kilpatrick:This is if you are somebody who's a Maps user, you know, you've probably put in your home and work address. Okay, so, you know, we're looking at the answer here. The first thing I want to point out is it's Maps forward. You know, you actually see the map with the places on it. It's a very contextual map, not a one-size-fits-all map. It's very relevant to your response. But what it's showing you is part of the answer, right? It's found a couple of places. And really important is, since you asked about clean showers, it's telling you that reviewers consistently praise their locker rooms, they're spacious, and it has great clean showers and steam and sauna and everything.

16:56Logan Kilpatrick:This also...

16:57Miriam Daniel:I like these reviews from users. Somebody's saying, described as downright luxurious. That's the kind of the the the validation you want.

17:06Logan Kilpatrick:Yeah And you also see You know some additional information, you know if it has a gym a sauna this that and everything Logan, how about we try something that is really relevant to you? Yeah, if you were to ask ma 'am something, what would it be?

17:23Miriam Daniel:Oh Dave was poking fun at me about my there's no good restaurants in the Bay Area comments, which there definitely are So let's let's see if we can find some of them. I'm vegetarian I don't like loud restaurants. I would ideally like to avoid the things that everyone else is going to, though maybe everyone's going to all the good restaurants. So those are my three prompts. I mean, I don't know. What location?

17:49David Cronin:Like what city?

17:50Miriam Daniel:Yeah, yeah. Try it. We're in Mountain View right now. So like hopefully something within like 20 minutes so that I don't have to sit in traffic for an hour.

17:59Logan Kilpatrick:Great. I just captured it like you said it.

18:02Miriam Daniel:I like it.

18:02Logan Kilpatrick:Okay, let's run that.

18:06David Cronin:I mean, this is really to the point we were talking about before where a power user could probably figure out how to do this with maps. Yeah. But it's a lot of work and most of our users aren't power users. Yeah.

18:17Logan Kilpatrick:Apparently that was a very easy query.

18:19David Cronin:Nice. Yeah, yeah.

18:20Logan Kilpatrick:It says, you know, finding a quiet restaurant with veg options is easy. Well, actually, I've got a

18:25Miriam Daniel:follow-up for this, which is I don't want a vegetarian restaurant. What I found is like, actually, this is my problem often, which is like, if I go specifically a vegetarian restaurant, you just get, it's too nuanced. I want like normal restaurants that have like a few good vegetarian options. Okay.

18:44Logan Kilpatrick:Okay.

18:45Miriam Daniel:I wonder if that all... Sounds good.

18:48Logan Kilpatrick:Now we could modify it, But it's very possible that these are not all just vegetarian restaurants. But let's try what you just said.

18:57David Cronin:They look pretty veggie.

18:58Miriam Daniel:They do. They do. Wild, yeah. Wild seed, very classic barrier of vegetarian restaurants.

19:03Logan Kilpatrick:For restaurants within 20 minutes.

19:08Miriam Daniel:And I assume this conversation is like it's multi-turned. So does it have context about this?

19:13Logan Kilpatrick:It's absolutely multi-turned, yeah. and with good reg options.

19:23Logan Kilpatrick:And table for two, since you mentioned your girlfriend. Not too loud.

19:29Miriam Daniel:Not too loud.

19:32Logan Kilpatrick:It might already know that you don't want it too loud.

19:34Miriam Daniel:I like that. That's good. Let's try that. But I think this is fun because I can imagine trading off. Like, I'm actually willing to, I wonder if I can tell them all this, like I'm willing to trade off a little bit of noise if it really is the best place. Like, I'm willing to compromise slightly.

19:51Logan Kilpatrick:Since you're in a back-to-back conversation here, it actually knew finding a quiet restaurant with great vegetarian options.

19:58Miriam Daniel:I like that.

19:59Logan Kilpatrick:So Wild Seed still came up, and it's actually showing you that it's actionable. If you like what you're seeing, this table's available at these times. You can just select and turn plants into action.

20:09Miriam Daniel:Can I do one more follow up on this, which is I want places that don't have lots of reviews. This is my other new filtering mechanism, which is like if the place has almost too many reviews, I'm like, it means it's the place everyone's going to. I want to try to find something like a little newer, a little. Yeah, I want the hidden gems. Yeah.

20:28David Cronin:Our hidden gems algorithm actually considers that.

20:30Miriam Daniel:I like that. Perfect. I love this.

20:39Logan Kilpatrick:How about that?

20:40Miriam Daniel:Yeah, let's try it. It's also fast. This is awesome. It's like we're getting answers quick. Launching five expeditions. I like it.

20:53Miriam Daniel:Hmm, this is good. This is good.

21:00Miriam Daniel:I like this. This is my comment before of the fusion of the marketing image and the real image. And this one on the right looks like a great example of the marketing image next to the real actual image of the user uploaded version of it, which is cool.

21:14Logan Kilpatrick:Actually, almost all the photos you're seeing here are user uploaded images. Nice.

21:18Miriam Daniel:That's awesome.

21:19Logan Kilpatrick:Yeah. They're all coming from our corpus of user generated content. I love it.

21:25Miriam Daniel:I'm going to have to check out some of these places. Some more quiet finds.

21:28Logan Kilpatrick:And here you go with a very helpful map.

21:30Miriam Daniel:Nice. This is awesome. And it does, does it do multimodal queries right now? Like do people like upload images of like menus or things like that and be like, find me something that has food options similar to this?

21:45Logan Kilpatrick:Not right now. We'll take that into consideration.

21:48David Cronin:We had all the feature requests. I'm excited.

21:50Logan Kilpatrick:Lots of feature requests today.

21:51David Cronin:Yeah. Awesome. When we work on maps, we're used to the feature requests. We get it from the friends and family. All the time. I love it. Other examples?

21:59Miriam Daniel:Do we want to show anything else?

22:00David Cronin:Things to do would be another one if you want to branch out from dining.

22:04Miriam Daniel:Yeah, yeah. Maybe just more open-ended. I'm curious what the...

22:08David Cronin:Like, what's up this weekend? Or, like, I want to go for a hike and grab the coffee afterwards. Queries like that.

22:13Miriam Daniel:Yeah, let's see if there's anything fun to do in San Francisco. And this is pulling from, I assume there's more, there's grounding with Google search in some of this as well, being pulled in?

22:23Logan Kilpatrick:It's definitely looking at all the information, public information that's available. And it's looking at making sure it's grounded in maps local data.

22:33David Cronin:Oh, interesting.

22:34Logan Kilpatrick:You know, so, you know, we can't send you somewhere if it doesn't exist or, you know, if it's not open on the weekends or something like that. So we make sure it's grounded in our data as well. So looks like there's a Chinese New Year's parade.

22:47Miriam Daniel:Nice.

22:49Logan Kilpatrick:Something to do with the opera.

22:50David Cronin:We're going to the Warriors game.

22:52Logan Kilpatrick:Yeah. Wood tape. And some little local stuff like cab crawl.

22:55David Cronin:Mary J. Blige at Chase Center. If you're 90s, I would be.

23:00Logan Kilpatrick:I kind of snuck in the best ways to get there because we wanted to show you what it's like in terms of looking up both should I take the train or drive.

23:10Miriam Daniel:Yeah, yeah. I need this. Sitting in a waymo between Mountain View and SF takes a long time, so I need to get better. I'm leaving at rush hour all the time, and I need to ask maps and start planning my commute better so that I'm not doing that.

23:25Logan Kilpatrick:Totally. And see, it gave you both driving and Caltrain because we're sitting right here in long view. But depending on where you are, you might see different options.

23:33Miriam Daniel:Yeah, yeah. This is awesome. I'm excited. And does this go and the next step is if you were doing the research and then ready to go, it's just one click and then I like -

23:44Logan Kilpatrick:One click and it's actionable. So you would simply, let's say you decide to do this. Yeah. You just look at directions and it's taking you to public transport. Nice. So you just pick which line you want and you're off you go.

23:57Miriam Daniel:That's awesome. I love it. Very cool.

24:00Logan Kilpatrick:Just like whether you wanted to reserve a table, navigate somewhere, select public transit, you're looking for schedules and timing. It's all just a tap away. Awesome.

24:10Miriam Daniel:I'm super interested. Obviously, Maps powers more than just Maps. We've got Maps in the Gemini API for developers. We've got the whole Maps developer platform. Maps is also powering a bunch of the experiences for local recommendations in the Gemini app, I think. And maybe there's something interesting we can talk about, like parts of that experience or things we learned. I don't know, this timeline sequence of this, like things we learned doing that, which then carry over to Ask Maps.

24:38Logan Kilpatrick:Yeah, I'm really glad you asked because when we see Maps, it's not just Google Maps. It's actually all of this local data and imagery and information about the world, and it does power Gemini app. it actually powers your local queries and search as well. And so, you know, for example, we've been working very closely with our friends on the Gemini App team to make sure that the local queries really shine inside of the Gemini App. So when you see a map forward response, we're detecting that you're asking something about your local area. And, you know, we pull forward a map with all the places to go to.

25:14Logan Kilpatrick:It's grounded in real world data, the trusted information that we have in maps. so you can feel confident that what you're seeing there is a real place. And what we're recommending is something that will be open for you to go to when you want. So this is part of our collaboration with both Search and with Gemini to make sure that all of the local needs of our users are met regardless of where they go.

25:40David Cronin:It's also like a great way for us to try out some of our ideas, right? To see kind of what works in Search and what works in the Gemini app. And, you know, it's so hard to design these things in Figma, in Mox. You kind of have to do it in the real world with real queries. And so it gives us a great platform to see what's working. And we get inspired by things we think are really working in the Gemini app or in AI mode and bring those into Maps, too.

26:02Miriam Daniel:Is there a deep research Maps sort of like integration? Or I assume deep research can just like use Maps in some way. Do you find users who are like maybe trying to be like exhaustive? of like, give me the bucket list 10 item, and thinking about my Bay Area use case. I'm like, do all the research, find me the top 10 places in the Bay, and I'm gonna go do that as my bucket list item through maps.

26:24Logan Kilpatrick:We do see a lot of itinerary planning. People are thinking about planning out a trip, even if it's a little further away. I recently just came back from Argentina, and I actually used it to plan out all of my stops, places to go, you know, even what times I should catch some of the shows, you know, things like that.

26:45David Cronin:Yeah, for sure. Planning a summer vacation with my family, going to the Alps, looking at the Dolomites versus the Swiss Alps. Definitely dug into deep research on that, checked out how AskMaps work. And it's a great way to kind of understand like, well, what are the kind of answers that make sense to me that I'm going to find useful? And it really, really, really makes it a lot more efficient to plan a trip.

27:07Miriam Daniel:I want to make sure that we get to talk about immersive navigation and hear more about sort of like the story of like how we actually got here today, launching it now in the, I think in the U.S. to start. So maybe we can talk about sort of just the whole, what's the experience today and also sort of the, all the work that it took, the technical work to actually get to the launch.

27:28Logan Kilpatrick:Maybe we'll start with how, how it was born. When our teams came together, we actually talked about a lot of the real world pain points, to be honest. A lot of what our users have been telling us. They want more intuitive guidance. When they look at the map, they want to clearly understand where they need to go when they're facing complex interchanges on the road. And road networks have changed dramatically over the last few years. They're not what they look like 10 years ago. And they're also looking for, how can I arrive with confidence? I'm going to an unfamiliar destination. So there was a lot of these little pain points that I feel myself every day.

28:06Logan Kilpatrick:and we were all discussing. So that's where immersive navigation was born and where we set out to solve these real user pain points. And there are many components to immersive navigation, but I want to let Dave talk about the canvas. A big part of it is the navigation map.

28:23David Cronin:Yeah, we start with the visualization and kind of our goal with it, driving can be really stressful for people. So our first goal is to kind of create this calm, comforting experience for drivers to help kind of alleviate some of that stress. And so the first most important thing is obviously making it easy as possible for people to follow the route. But it gets complicated pretty quickly, which is you're going through these freeway interchanges, complex intersections, there's buildings, you have to understand what is the right place to turn. And so we quickly realized is we needed to depict the world around the route in intricate detail.

28:57David Cronin:So someone could understand where they are in relation to where they need to be and where they need to maneuver. And so we kind of looked at adding dimensionality. How do you bring in buildings so you can kind of recognize where you are in a city or an urban environment? How do you depict roads in a way so you understand when you're going under an overpass versus up on an exit? And we've added shadows and a lot of dimensionality kind of helped someone spatially reason about that. And then you kind of realize, oh, we've added some buildings and the route goes behind some buildings around a left turn.

29:30David Cronin:So now sometimes the buildings need to be transparent. So we sort of start with this basic idea of the route should be really easy to follow. And you start building on top of that, and you end up with a lot of design problems to solve in order to get it just right. It was quite a process. You know, we kind of start with sort of a numerical understanding of like how contrast ratios need to work to make the polyline of the route most visible to users. but a lot of it is art. A lot of it is using our eyes and judging kind of what makes sense in certain situations. And we kind of start with a set of, you know, what we call golden or platinum routes, where we knew these routes were complicated and difficult to follow and really designed so that we were confident users could follow those routes.

30:16David Cronin:But that's not the real world. And then, so from there, we quickly had to get this experience onto our phones, into our cars, and spend a lot of time just driving and testing it. One of the coolest parts of the process for me has been we've had an active dog food chat going for several months now. People across Google were testing out this experience and Googlers have really showed up to be helpful. Like all around the world, showing us what they're seeing through their windscreen, what they're seeing on their phones, pointing out places where it's working, where it's not working, where it's beautiful, where it's not beautiful.

30:50David Cronin:And that's really helped us hone the experience and make it what it is.

30:53Logan Kilpatrick:So there's this beautiful, vivid map of the world. It's like it's dimensional. It's got all these beautiful colors and layers and things. And then, as Dave was just saying, we had to make sure that we're adapting to the drive that that user is having, right? Like, you know, if you're on a long stretch of road, you don't want anything distracting. You just want to zoom out and see the full round. But if you are about to come to a complex interchange and you need to take a left and then a right, you want to see more of the route. So, you know, the camera actually tilts and pans to show you more of the route.

31:28Logan Kilpatrick:So you can look around those corners, you know, so you never have to miss a turn. You know, that was our goal. You know, if we do a great job, you never have to miss a turn. But then we also realize that it's not just the camera and showing the route. It's the voice guidance that you give right at the right moment, you know, when you're preparing, not when you're at the intersection and you're, you know, or find yourself in the wrong lane. So we have to tell you a few hundred meters ahead of time, you need to be on this other lane so you can take this turn. We had to sell it in a human understandable form.

32:00Logan Kilpatrick:Rather than say 500 meters, we would say after the gas station or go past two lights. So it's really understandable and intuitive. And even as you're arriving, it would say, stay on the left side, parking is behind the building or parking is on the side of the building or underground. So you want all of this kind of really intuitive voice and visuals guiding you in concert. So that's like the full navigation experience. But I should also mention the world is not static. It's changing all the time. You have traffic situations. You have incidents on the road. Sometimes it's construction. And once again, just as we were talking about our contributors in Ask Map's side, we have hundreds of millions of people contributing to us and telling us about real-time disruptions on your route.

32:51Logan Kilpatrick:So we overlay this driving experience with all of that information, real-time information of what's going on around your routes. We also offer you alternate routes when we find that you are going to be stuck in traffic for some time and there is a better way for you. We offer you choices. So your route, your way, your choice. So when we talk about immersive navigation, it's this full experience. The goal is to make sure that you get to your destination confidently and stress-free.

33:21Miriam Daniel:I love that. I'm actually super curious. I've seen the visual and the new UI experience. How do we actually get that? It's impressive to think about how we get the level of fidelity. Is it just satellite imagery from Google Earth plus a bunch of other stuff? I'm like, I don't even understand how it's possible.

33:40Logan Kilpatrick:There was a lot of hard but fun stuff that went on behind the scenes on that one. So obviously we start from our street view imagery, our aerial imagery. We have mapped all the road networks, several hundreds of millions of kilometers of it. But when we actually do the rendering, we found that we didn't have a renderer capable of doing all of that high fidelity stuff. So we actually worked in-house with Google Labs on building out a brand new renderer that would power some of these capabilities. And it's called Impress. The same renderer is actually found on our XR devices as well. Anytime you have to have this vivid, beautiful view of the world, we use this sort of a renderer for abstract maps and abstract views of the world.

34:24Logan Kilpatrick:And so that renderer and co-developing it with the labs team is what helped us bring it to this level of fidelity. But how do you know it reflects the real world? So that was another challenge. How do you know that we modeled it correctly? right? So we actually use Gemini to do these evals.

34:44David Cronin:Interesting.

34:45Logan Kilpatrick:So what we did is we would take our street view as ground truth and we would take our rendered versions that we are automating and simulating a drive and we would have Gemini compare the two and do automated evals. There's a lot of human raters and evals too. There's a lot of field testing that we're doing with operators as well. Dogfooting, as Dave mentioned. But if you want to do this at scale, you need something of far more automated. And that's where Gemini was really able to help us, tell us, hey, you're missing crosswalks here, or there's a median there, or this road has to fork here. It really has accelerated the pace of our innovation.

35:25Miriam Daniel:That's so cool. I feel like in all these conversations that I have with folks across Google, it's like this thread of Gemini truly being an accelerant for the pace of progress from a product perspective. And even the model itself perspective is very salient. But I'm super excited for more folks to get their hands on this experience. We've got lots of feature requests from this conversation. Thank you both for sitting down. Congrats on the launch. Excited for more folks to get their hands on this. And thank you, everyone, for watching this episode of Release Notes. We'll see you in the next one.