Gemini 3 and Gen UI in Google Search

0:00Today we're joined by Rhee and Robby. We're talking about AI in search. Spiritually, what we believe with search is that you can truly ask anything you want. I think search has kind of like had that googly-ness, the quirkiness from the beginning. So I'm excited to see how that manifests in a search experience for AI. With Gemini 3, it can do everything from reasoning and very complicated math to even coding you up like little simulations. I'm a big believer that a lot of AI in the future is going to take a much more visual form. It just sort of helps you understand data just completely differently to how you would have if it was just in a table or a chart.

0:34Getting that model in front of as many people as humanly possible is like the manifestation of what Google's mission is. It's so awesome to think about, wow, the capabilities that we just discussed, like coming to millions of people who use search every day. So cool.

0:55Hey, everyone. Welcome back to Release Notes. My name is Logan Kilpatrick. I'm on the Google DeepMind team. Today, we're joined by Rhi and Robbie. We're talking about AI in search. I'm super excited for this conversation. And actually, specifically, we're talking about Gemini 3 in search. So let's dive in. Let's do it. I mean, I'll just say that in general, there's a really special moment to be able to launch a frontier model in search to lots of people day one. I think we've been working up to that moment. So with Gemini 3, it can do everything from reasoning and very complicated math to even coding you up like little simulations to help solve your problem.

1:28Like if you need a little calculator or a widget like on the fly. Because I think spiritually what we believe with search is that you can truly ask anything you want and get effortless information. But really that's hard to do because people ask pretty hard questions. And so it's kind of the purest ability to really allow us to achieve that mission. And I think we've all rallied around it to make that possible. Yeah. The only thing I'll add, I think for us is just like the teams now have access to all these capabilities. On day one, it's just like, you know, you get the opportunity to think just more creatively, think about the core use cases that those capabilities can help users with.

2:00And so it's been great to have that on day one. Yeah, it's wonderful. It's fun to build a product when you have such a good model to build around. It makes it a little bit easier because you could be more ambitious. The Gen UI story is also really interesting. So I don't know if I still fully grok the gen. Like I intuitively understand what's happening, but maybe we can sort of talk Also, if folks haven't experienced this before, either one of you want to give the high level of what folks should expect in GenUI? Basically, what GenUI is, is you think about the model being able to have more control over not just the response, like the text that it sends back, but also the page it constructs.

2:39And so what you can do is you can tell the model, hey, for certain graphical information, you should consider graphing it. And here's a graphing library, and here's how it can look, and here's the styles. you can use this as a primitive now. It's like, oh, that's cool. I'm just going to start throwing graphs in. And so it's just one example, but you can kind of teach the model to think like a designer and kind of work through some of these decisions. And we've done sort of things like this before in search, but it hasn't been like automatic by the model. Like it was bespoke. Like I think we talked before about like real time information that gets pulled in, all this stuff, like the one box.

3:11I think from a design perspective, what's been so great about it is that originally, you know, we would create these sort of like static experiences You know, there would be tension between like the designers and the model because you'd be like, why can't you make this bold? The spacing doesn't look quite right. But we would work on the system instructions to get it super dialed. And then now with generative UI, it's like having a script and then having basically an improv stage. It's like we give the model all of the different components. And then we give those components a set of system instructions as well.

3:42So that are based on how a designer might lay something out. What's been so great to see, actually, in this is that the designers now are designing these experiences kind of like on the fly with the model, like to the point where designers are writing like system instructions that say, OK, here's a sort of set of components that you can have for a response that's like this. So it's like a carousel or here's like some imagery or like, you know, this is how we would lay out typography or a list, et cetera. And maybe here's some data visualization, which we should also talk about. And then the team will create like a set of design rationale system instructions.

4:15So it says, OK, for a sizing spec, we would say, model, you need to look at, hey, is this a primary piece of information that needs to be displayed or is it secondary? And so then the model can make decisions on how to actually lay certain pieces out based on core design rationale and then also then the user's needs and what a user might need in a response. What's crazy is you couldn't do any of this, very little of this, like three months ago even, six months ago definitely. You're talking about the bespoke experience. you'd have to kind of retrain the model. Like it would be something where you have to kind of take the weights.

4:48It can do certain things. And then you'd probably train it, maybe even post-trained. So like, oh, if you see data, the model just learns almost in the training process that it's better to put a graph in there than it's not. Whereas I think what's happened with increasing intelligence and reasoning in Gemini 3 is instruction following and reasoning. And when you can do that, you actually can just kind of say like, hey, here's rules. Like this is graphical information is best if in this way. And by the way, here's a link to a spec that has all these principles. So the more you can encode things in natural language and create specs like you would for another person who's a designer on your team, the model can kind of do that.

5:24So that's a way that you kind of need – what does it mean to have a conversational search experience? You need to kind of find your own way there. And I think we've been slowly finding what feels good for search. And the other piece is obviously what search does and is the key part of search is bringing you close to the web and the richness of what's out there. And so from the first design, we realized having AI with links within, but also like on that side right rail, like this kind of rich representation of like how the web and what kind of brings you outside the universe of this very tunnel vision AI.

5:54And it also kind of makes the experience feel more balanced, I think, too, and rich kind of worked. And so that was another piece that we thought was really important, I think, in the product side. Yeah, just reinforcing basically to a user that they're still using search. And so all of the things that Robbie just mentioned really helped, I think, just orienting users to AI in search. What's the breadth right now of the different, back to our meme of use cases and search queries, what's the breadth of what it is? Like, is it very, like, focused in certain, it's like a small set of domains today, and GenUI will sort of expand to potentially every sort of question somebody might ask will build some bespoke UI for them?

6:36Or, like, how is it? There's kind of two pieces that's manifesting right now. One is in the layout itself, in the primitives. Whether you have a table, an image, a graph, it's deciding whether to do those, what images to put in, and how it looks. Now, it's given instructions in our design language that it looks good. And actually one of the problems early on is that if you ask the model to make a page, it makes like a crazy page that just like, and every page looks different. So how do you make a consistent and predictable user experience while also giving the model a control? And so a big thing we had to figure out was how to put it on the rails and be like, look, this is like our design language.

7:07Like you're a new designer joining our team. There's a design language and there's a design system and here's what it looks like and here's the color palette we use and here's the typography we use. And you have to do that or else it kind of goes, it makes sense, right? tell a designer to just design something for me. They're just going to design whatever they think. That's one thing. And the other thing that it's able to do is it's able to code up these little simulations and inject those as their own primitives. And those are really neat. There's little interactive experiences where you can teach someone.

7:35Like I was teaching my daughter about lift and I said, I asked it to create a simulation or a visualization for it. And it made this crazy little window with like vectors, like arrows running over a wing through these sliders. it would adjust the wing and then show how much lift was occurring, like where the arrows would start going under the wing and pushing the plane up. Super cool. And that's the kind of thing that's hard to describe to a person in text, but through the visual medium, it's super clear. And so I'm a big believer that a lot of AI in the future is going to take a much more visual form.

8:05I think we've been innovating a lot in this space and think about what we've done with shopping and visual search in AI mode. But this takes it further. These are all versions of the model having control over what it's showing you. Yeah, I mean, the additional layer of motion also is just a game changer. I had a similar example. I was playing around with it the other day with Ollie and my daughter, and we were looking at, like, she had asked how cars work, and so we started looking at simple engines. And, you know, it creates, like, a full sort of, like, piston system. It shows you how, like, fuel works, the ingestion, the exhaustion.

8:36I mean, it was just really, like, remarkable, actually. And I don't think you would have gotten that from just, you know, a static graphic either. So I think the ability for these things to become just like truly interactive also. So there's like aspects that you can hover over and get more information. I think that's, you know, easily where it's going. I do think one thing, though, just to touch on that Robbie was kind of talking about around like, hey, when you give the model all of these components, there does still need to be like a layer of just like taste and quality and craftsmanship that I think needs to exist.

9:03And this is definitely something that, you know, we we kind of have like almost a visual QA process like with the model. So we're actively working right now. What does it mean for us to evaluate these things from a design perspective? And so do we have separate evaluation processes for that? Do we create a system instruction for a VizQA process? And starting to see some really amazing results with that, actually. So I'm super excited about that because those things are going to come very soon. And I think it's going to just take everything up a notch. Rhi, your comment about like design taste and how that's an essential part of the GenUI experience, I think, hits home.

9:40I feel like some of the other constraints, just like the efficiency latency piece of it. So I'm curious, like how maybe from a design perspective, or just like the product constraints of like, building a simulation is obviously, like takes time to do that. So like, what do users see? And like, what's the? Yeah, I mean, there's definitely like a latency design component that is required. Like we need to design the latency sort of experience. And so, you know, we need to make sure that users know that there's something that's being generated. And so we think about that. I think we also are working very closely with engineering.

10:10Are there things that we can do here to create reductions in latency? Are there certain components that actually don't need sort of to be regenerated? So we're talking a lot about that. I also just have seen our capacity to reduce latency. It's kind of second to none at Google. It's like one of the things that I think search has always just prided itself on. And it's like, we just want to get you the information that you need as efficiently and as quickly as possible. And I just have so much confidence in our ability to solve for those things over time. So, yeah, I see nothing but opportunity when that's concerned.

10:45I know it's going to get better. What do users see right now when, for example, a car engine simulation is being built? Is it just like... Yeah, so the same way as thinking steps, which is, you know, I think thinking steps is really interesting in my mind because it's an opportunity for us to use that latency to communicate to a user what the model is doing. And so when you do have these moments of latency, we can create sort of like a representation of like, hey, there's things happening in the background here. we're calculating this, we're drawing this, what have you. And so right now it's just sort of a relatively straightforward overlay where the image will appear or the data visualization will appear that represents what the model is doing on the back end so that users know to wait for something.

11:30It's a little longer than maybe you would want it to be right now, but I know it's going to get better. Obviously, the three stories started with Pro. It was available to AI mode customers. and obviously the teams worked super hard on Flash. So I'm curious for you both and just like for AI and search, like what the Flash story means. And obviously lots of hard work to make Flash happen from a search perspective. Yeah, couldn't be more excited to bring, you know, the frontier model at the kind of speed and availability that people need for everyday use to search. And I think that is one of the most exciting things.

12:06And so I think what you'll get is you have these lineages of these model series. So you have the three series, which I think will help people ultimately tap into much more sophisticated reasoning, problem-solving skills. And also, on the generative side, actually help create things for you, whether they're these widgets over time or help you understand your data and generate a graph for you about it. I'm excited to bring that to many more people through these models that can be run at a larger scale and in a faster way. Yeah, it is awesome. We were playing around a bunch of our evals in AI studio.

12:39It's like for some of the use cases, Flash is like three times faster. And I feel like the Flash story feels very much like the search story, where the timing matters, the quality matters. And getting that model in front of as many people as humanly possible is like the manifestation of what Google's mission is. Robbie, something you talked about or something. Actually, I get my AI mode, search AI updates from all your tweets. and something you tweeted about recently or announced was the experiment of bringing AI mode to the bottom of AI overviews. I don't know if there's a better way of saying it than that.

13:13But I'm curious about that experience and how the fusion of the different AI search experiences coming together is playing out and what the high level idea is. Yeah. I mean, I think in general, the high level desire from the user is you just put what you're thinking into Google search and just ask. You can drop huge amounts of code in, You can ask a really specific question. You can get advice. Just put it into Google. And if AI we think will be helpful, you'll get this kind of generated experience at the top. And if you expand it on mobile, we're experimenting with that just opening up an AI first experience kind of all the way into the screen.

13:48And now you can have a follow up because you have a follow up box at the bottom. And now you're in a conversation. So we're trying to do is make it really fluid to get to AI in the first place. And then when you tap in, be in a more conversational mode, which is basically AI mode and will just naturally take you into AI mode for your follow-up questions. And so that users then need to think, where do I have to put my questions as an AI mode thing? Do I put it into the main search engine? We want to bring these things together so that it's just as easy as possible. And of course, for power users who kind of know when they want AI, we're seeing them just go right to AI for those really hard questions.

14:22But I think users ultimately will have that choice. Yeah, something interesting, actually, I was talking to Josh about this. This was maybe like three or four months ago. And he was saying that like 2026, one of the things that's top of mind is just like this model routing story. And I think more loosely defined, like we actually like Google has lots of different models now, like we've trained many iterations of Gemini. Actually, in some cases, they all have a different set of trade offs. I'm curious as you all or as you both like think about the search experience and like all of these different models.

14:54Like does it feel like there's I'm sure you want better and faster models. But is there anything interesting that like you wish you had a model that could do something or is it like you're actually getting what you need right now. I'm curious because we can pass the feedback the feature request on to the model team. You get your wish. I get whatever I want. Whatever you want. I think one of the things that has been a work in progress, like everything here, is just around the personality and the persona of the experience. There's just so much opportunity for us to just be more personable. Nobody wants Search to be super chummy, but I think that there's just opportunity for us to be there for users in a different way than we've been before.

15:35I think we've made good progress. We've got experiments that we're running. I think that ability for a user to sort of build a relationship with us as their like knowledge companion that we talk about all the time is like one of the things that, you know, we're actively working on that I feel like is a priority for us to get right. Yeah, that's a great example. Has that been something that like search has thought about this like persona idea historically? Yeah, yeah. And we're working closely with like teams within, you know, Google DeepMind and within Gemini also to like understand like their learnings and then how we can bring some of those things to search.

16:05But we also want to have kind of like our own flavor. you know, sometimes I think about like search has always like had these moments of delight, like the Easter eggs, you know, the validation of sort of super fans, like whatever it is, a Taylor Swift album. And, you know, I think that whatever we design here, and it is a design exercise because you're designing a, you know, a persona or a personality or model behavior in some way, I'd love to infuse it with some of the things that I think Google is kind of known for some of this Googliness, the quirkiness. And I think search has kind of like had that, you know, from the beginning.

16:36So I'm excited to see how that manifests in a search experience for AI. Yeah, what's interesting is the search has, I think, the persona of search. She kind of existed a bit, but not through language and not through an AI paradigm. So it's like you think about it, it's kind of like, it's obviously like an intelligence service. People think about it with information, but it's also, so it's kind of has this kind of science-y, futuristic vibe. So it's like we celebrate scientists, right, for doodles amongst other things. And it's got a quirkiness and kind of an unexpected thing where, you know, I think if you could, you could like throw a, what, a bouquet of flowers at like your favorite team and people thought that was funny.

17:12Or, you know, like there's like a nerdiness and a fun kind of jovial spirit there. But like, if you're talking to that thing and you say, hi, like what does search say back to you? Right? Yeah. Like someone just says, I'm feeling sad down today. Like how would a search, how would search, like respond to something like that. And those are the kind of questions we're thinking about now because people are asking about advice. They're asking personal questions. That's been a particularly interesting part of this product taste and shaping is like, you don't really think of your job as potentially thinking about those kinds of problems when you're working on a technology product, but those are actually as important as anything else we're doing.

17:50Yeah, that's awesome. That is super fascinating to see that. And I feel like people definitely, it's maybe one of the shifts in user behavior of how people are using search. Do you not have anything that's on your wish list? Yeah, there's no wish list. Robbie's like, I got what I need. Very long wish list. I think for me, it's more about, I think the models are becoming more, like I said before, about capabilities to do things versus doing things that I like specifically for me. But I think that one is, it'd be really cool if the model just kind of naturally could understand how all of Google systems worked.

18:27Think about how, at least as a developer internally, it would be pretty neat to be like, okay, the model just naturally knows how to use. It could crawl your code base and it could maybe work for any company and just know how every API and every system worked. So you could be like, hey, model, you now should be able to use everything that search uses for Google Finance or something if you're a search. And then all that information is just perfectly available now to the model because it can go just figure it out by itself. I think that would be super cool. So the more the model almost agentically learns your own kind of systems and then can and can learn and can build that capability that kind of allows us to, you know, make it even more helpful for you, I think.

19:09On the model story, one of the models I've been super excited about is obviously the Nano Banana models available in AI mode and then I've been on a pro for even more sort of like deep factual stuff. I'm curious how both of you have been thinking about like what that experience means for for search and AI and search. Yeah. Yeah, I think what we're starting to see like the opportunities are for like data visualization with Nano Banana in particular, we see opportunities for users to look at data that they might have in like completely different ways. So one of the use cases that we've seen as exciting is a sports one where you have like a two basketball players that you're super fans of and you want to visualize their stats and we can create, you know, an infographic for you.

19:53and like that you know that never existed before and and so like this idea that we can just visualize information for you in in these new ways but it's amazing to sort of watch like you know how the data that it can pull and then how it manifests that data in this visual way that is just sort of helps you understand data and just completely differently to how you would have if it was just in a you know in a table or a chart. I think actually that what's really neat is what Ree's describing is really the kind of unison of the most powerful model with search honestly because if you think about it like each of these things requires the reasoning of the model but also the knowledge of search and so to use these tools to look up these sports facts that are like real-time information or if it's trying to build you a product thing it's like finding shopping data it's pulling images it's like browsing to see what reviews have um it needs to kind of pull that in and then you're combining that reasoning with tools but then this like visualization thing you kind of need all of that working together.

20:48It's like a few pieces and that makes these really magical things where you get a game recap visualized with graphs and stats that are like live just for you. And I think you're starting to see a lot of that magic happen now because of these pieces coming together. And it happens in lots of facets, not just for Nana Banana. Like you can do shopping on AI mode now and it'll pull a gallery with images and with live prices and you can follow, ask follow-up questions and say, I like the black instead of the green pants and then it'll switch them all the color. So I think you're seeing these combinations more and more in the system.

21:18Yeah, that's interesting. I wonder, I feel like there's an interesting thread to pull on. I feel like the next time we're going to talk, it's going to be about the even more of those combinations coming together and fusing all the different parts of search into an experience. So thank you both for sitting down to talk about this. And thanks everyone for watching. We'll see you in the next episode.

21:43Thank you.