Gemini app: Canvas, Deep Research and Personalization

0:00We're going to take a deep dive into some of the latest Gemini updates. We announced Gemini Canvas. Anything you can imagine, you can now build. So you can see here it's made this interactive web app, even with some animations built in, which is pretty cool. I feel like people have been losing their mind over some of the deep research updates. We're fired up and we're going to make these better and better. Trying to take all that sort of AI complexity and sort of abstract it away into an experience that sort of just feels like magic. A really truly personalized vision for what a Gemini app and AI assistants can be.

0:31I'm excited for folks to get their hands on it.

0:58On this episode, we're chatting with Dave Citron. a PM on the Gemini app team. We're going to take a deep dive into some of the latest Gemini updates. Welcome, Dave. Thanks so much for having me. Dave, we've had a series of really awesome Gemini launches in the last few weeks. Do you want to give us sort of a rundown of everything that's launched? Yeah, absolutely. So, you know, we announced Gemini Canvas, which is amazing for collaborating with the model to create awesome, beautiful docs and actually write code and build Beautiful Web Apps, and that's all powered by Gemini 2.0. And then we also announced an upgrade to Deep Research, where we are really excited to have the latest and greatest on 2.0 thinking models.

1:41And it's also now free for everyone to try. And then we announced our new personalization feature, where you can opt into connecting your Google search history and start playing with a really truly personalized vision for what Gemini app and AI assistance can be. It's really exciting. I love that. I feel like people have been losing their mind over some of the deep research updates, but let's maybe start with Canvas mode. Do you want to sort of just give us the lay of the land of like for folks who haven't used it yet, like what does that product experience actually look like? Yeah, absolutely.

2:15So, you know, we noticed a lot of people were using Gemini app to create content, all sorts of content, but it can be a little bit annoying once you get on to maybe turn four or five. You know, you get something that's pretty good, but you want to tweak maybe paragraph three. And so you'd have to tell the model in your prompt, hey, can you update paragraph three, but make sure not to touch the rest, et cetera. And so Canvas basically gives the user this really great interactive, familiar UI to actually, with the mouse and keyboard, collaborate with the model directly. So you can not only sub-select specific paragraphs and ask the model for feedback.

2:54But you can actually make edits directly like you would in, let's say, Google Docs. And this all happens within the Gemini app, both on web and in mobile. And then beyond Docs, we also have the ability to create code and to, again, collaborate with Gemini to build all sorts of amazing coding experiences, and similarly be able to sub-select specific pieces of code to iterate instead of having to churn out the entire iteration over and over again. And then maybe most excitingly for Canvas, you can preview the web app in line. So you can create all sorts of really incredible experiences, even with zero coding experience.

3:30And not only preview in line, but actually publish it and share it so others can play with your creations. I love that. I've got a bunch of deep dive questions on this. And actually, you'll show our first sort of live demo on this podcast show conversation, whatever we call it. But so for folks who are listening in audio, you can watch on YouTube and Dave will be showing a bunch of really awesome stuff. But my really quick tactical question is, how do you think about the user journey for people starting in the Gemini app and using Canvas versus in what cases would I start in Google Docs as an example?

4:08I'm a Google Docs 20 times a day, daily active user. How should I be thinking about this? Am I starting for certain use cases in docs versus Canvas mode? Or what's the breakdown between those? Yeah, it's a really good question. The truth is you can start wherever you want, and we're going to bring amazing Gemini AI to you. I think what we're finding is a lot of users, they come to Gemini and not necessarily knowing exactly what they want to create. Oftentimes, these kind of creation sessions start out by brainstorming. and then all of a sudden you have that spark that you needed to create something amazing, whether that's, again, a doc or some sort of code-based experience.

4:46And so what's really great about starting from Gemini as well, if you choose to start there, is once you've decided to create something like a doc format, let's say it's an essay or a script for a podcast, whatever you want, you can easily export it to Google Docs and then continue right where you left off and then have all of the amazing Google Docs features. So, you know, you really can't lose either way. That's awesome. A great mental model. Do you want to dive in and we can actually take a look at Canvas in action? All right. Let me show you how it works. So I'm here at the Gemini web app and I have a couple of props saved just to kind of speed things up a little bit.

5:26But for this scenario, imagine that I want to help my daughter study for chemistry by asking Gemini Canvas to build an awesome study guide. So I am going to write a prompt, basically just saying, please help me build this study guide. Click this new Canvas button and fire off the prompt. And you'll see in just a second or two, the new Canvas editor pops up. And I have this great study guide that's being produced basically instantly. And so what's really cool about this is it's actually a live editor, almost like Google Docs, where I can, if I wanted to, make edits directly to the title. I can have a couple of different formatting options, like changing the font size and styling.

6:10I have undo, redo. And I can also collaborate with Gemini in real time to make refined tweaks. Let's see, if I wanted to make changes across the entire document, I can also use these controls at the bottom. Here's one where it allows me to change the length. So I want to, in this case, make the whole document a little bit longer. and it's going ahead and doing that. And so you can quickly see how gone are the days of having to tell the model with every turn, only update paragraph two and then make this a little bit shorter, but not the title and kind of struggle to have a chat UI and make great artifacts.

6:47In this case, I'm actually having this interactive UI and working with the model to refine it. So we're really excited about the doc editor. Yeah, I love this. This editing experience is super cool as someone who's using Google Docs in too much of my daily time. You mentioned before that Canvas Mode also has code. What's the experience like to go from like a Google Doc to like a code artifact? Well, let's just play this scenario forward a little bit and imagine that I now want to go from helping my daughter with a study guide to actually building a full interactive periodic table. Let's imagine that the exam is all about the periodic table of elements.

7:24So again, in the same session, and it uses all of the context that's generated so far with this doc, I'm now asking the model to build a beautiful web app. And so again, the code version of Canvas popped right up. And whether you know how to write code or not, you can see here, it's actively busy trying to build me a beautiful web app. And again, I didn't prompt it very much. This is maybe two sentences worth. And I'm getting this incredible, sophisticated web app that you'll see in a moment. is quite impressive. This is awesome. I thought my days of looking at the PuriR table were over, but I appreciate getting to see this come back to life.

8:06And the code looks like, it looks like a lot of code, honestly, so I feel it's impressive. Yeah, I mean, you know, definitely wouldn't have been able to write that much code in five seconds or whatever it was. And again, all of that happened by just basically sending a couple of sentences of instructions. So you can see here it's made this interactive web app, even with some animations built in, which is pretty cool. And now I can hit share and actually share this with anyone. They don't even have to have a Gemini account, and now they can play with my interactive web app. So we're really excited about the ability to basically make everyone a web app developer, and we're really excited about the kinds of things people are going to be able to make with this.

8:46Yeah, this is super cool. Dave, do you have any more examples of Canvas in action? Yeah, so we've been piloting this with a bunch of trusted testers, and we've gotten back some amazing and mind-blowing web app experiences. And many of them were written entirely through prompting from people who don't know how to write code at all. So I'll just show you two of my favorite ones. This one is a full solar system visualizer. So it turns out, you know, with web platform technology, you can do all sorts of amazing, sophisticated 3D experiences. And the Gemini canvas with coding can actually help you produce all these things.

9:29So in this case, it's a full, let me actually zoom in here, and I'm going to take advantage of the fact that I'm using a touchscreen laptop. But I can actually go now and interact with the entire solar system, see how the planets are moving. I can jump to a specific planet. You know, again, you don't have to know how to write code, and now anyone can produce something like this. It's just incredible. Here's another example. It's basically like a particle simulator test. So I can now, you know, click a bunch of different settings and see how various different particle systems will manipulate with gravity settings and repulsion and turbulence.

10:12Again, it's kind of like this precursor to a video game or a physics simulator. Anything you can imagine, you can now build in just a prompt. It's just incredible. Yeah, this is wild. This is super. I am horrible at physics, and I remember how painful it was. But I feel like this is the kind of thing that actually brings it to life in a way that would make it a little bit more fun to learn. Can you talk about what are the limitations of this? Like, is it just like any like external package that's available on the internet I can use? Or is there like a specific subset of different things that the model could like actually generate code to do?

10:53Or like what's for the code sandbox environment, like what are the limitations for that? Yeah, you know, we're starting, you know, we're planning to basically continue to build on this platform again and again and allow you to build more and more sophisticated web apps. And so to start, imagine that it's mostly going to be kind of contained sandboxed type examples like the ones that you've seen here. Over time, we wanted to hook into more and more APIs. And basically, the sky's the limit, enabling you to build any sort of web app you can imagine. I love it. And is it also, is it single, it's like all single page apps today?

11:28Or can you do like a much more like multi-page comprehensive application? Yeah, so it really just depends on your prompting skill. So like fundamentally, the URL that you can share is a single URL. But you can instruct the model to build all sorts of comprehensive navigation structures inside of the page and dynamically update. So you can build a navigation tree and all sorts of different subpages. It would all be contained in a single URL, which also makes it really easy to share. But again, you can go as complex as you want. Let's spend some time talking about deep research. So we actually had Jack Ray on to talk, who's one of the co-leads for Gemini Thinking Models.

12:10Can you sort of talk about the combination of this new model with deep research, which we launched back in December? And it seems like people are super excited about the combination of these two things coming together. Yeah, absolutely. We were, you know, just really excited to pioneer this new category of deep research back in December. It just saves people an incredible amount of time in situations where you really want to ramp up on something quickly, but you want a deeper look. You know, the kind of couple bullet point responses that you're seeing from Gemini before we launched Deep Research were good for some things.

12:46But if you really want to go deep, you'd still have to do a lot of that research manually yourself. So Deep Research basically is one of our first long running agentic features that takes that, you know, and sometimes hours of Deep Research and does it all for you. And, you know, just a few days ago, we hooked it up now to the 2.0 thinking model, which dramatically improves the output quality, depth of research. Ultimately, the thinking model took the different agentic steps and up-leveled almost every one of them, from the planning phase to how it was searching the web to how it was synthesizing and kind of figuring out second-order questions and continuing to fire off even more searches.

13:28And so what we're seeing from our testing and early user feedback is that people are loving the reports even more. And again, this whole space is moving so fast. We just launched V1 a couple of weeks ago, and now already we're dramatically improving the output quality. So we're fired up, and we're going to make these better and better. But the thinking model was a huge step forward. Yeah. And Dave, one of the things that's top of mind is when Deep Research launched, it was using 1.5 Pro. And as you all migrated over to 2.0 Flash Thinking, any sort of weird challenges or like how much did the prompts have to change in order to make that migration work?

14:10Yeah, it's a great question. There were definitely modifications that we had to make. A lot of the prompting we used in the original version was trying to kind of push the model to do thinking without actually having test time compute. Once we're shifted over to 2.0 thinking models, a lot of the work kind of happens automatically because the model is a reasoning model. So it's taking the prompt and kind of what you get for free is this kind of reasoning step. So there's definitely prompt evolution. There's also, I think, less need to do supervised fine tuning, to kind of post-train the model with specific curated data sets on what makes a really good research report.

14:52Part of that was just upgrading to the 2.0 class of models in general. But then when you add in thinking test time compute, you just really get a step function improvement in the ability to synthesize these great research reports without having to curate a bunch of data to kind of teach the model what makes a good research report. Dave, do you want to show us deep research in action? And while you're showing us, I'll make my random obligatory comment that the first time I saw this was actually one of the demos that was going viral externally around like a thousand plus pages visited on the web to answer someone's query.

15:30And I was like, that is literally incredible. It's so wild to see it like actually very tangibly doing work for you. And I think when you see like the number of pages visits, like I think about how long and how much time that would have taken me in order to do. Yeah, I mean, that's a really good point. One of the things that we're taking advantage of here to deliver this cutting edge feature is Gemini's long context windows. And I think that's really, if you think about what deep research means in the Gemini app, it really is filling up that context window, which is actually quite hard to do normally, especially as a human copying and pasting a bunch of text.

16:09Basically, agentically having the model go out and go as deep as it needs to. And that large context window really shines here. No, I love it. Let's see a demo. Okay, awesome. So again, the experience of using Deep Research is very similar to what we launched in December. We've added a pretty handy shortcut directly to the bottom bar here. So I'm going to have a prompt saved just to speed things up a little bit. In this case, I have this fancy question about researching integrating AI agents. And I'm just going to click the Deep Research button and then hit Submit. And again, this is going to fire off and the model is going to recognize, OK, this user is asking for a long-running agentic task that is deep research.

16:57And it's actually going through a little bit of a pre-planning step, and it's going to output to me all the different steps that it thinks I'm going to want. And if I want to, I can make changes before it goes off. So, you know, if this was kind of more of a quick answer, we wouldn't even ask the user if this was exactly what they wanted. We would just give them something very quickly and and then allow the user to keep riffing. But in this case, because it's going to take a couple of minutes, we really want to make sure that the user, in some sense, has an approval step that this is actually the research plan that they're looking for.

17:30So if I wanted to, I could just type to edit the plan, or I could click the Edit Plan button and say, actually, no, I want you to focus on this, or spend a little bit more time on this, or maybe focus on academic sources. But in this case, this looks really good. And all I need to do is click Start Research. and boom, the research report is going to start. What's really cool about this new version of deep research is as the model is actually doing all of this agentic work, using the thinking model, we can now see the thoughts of the agentic task as it's happening. So you can see here basically step-by-step what the model is doing, thinking, you know, what web pages it's checking out, what it's thinking about those webpages, what secondary questions it's going to now ask itself and uncover.

18:19And this will ultimately conclude with producing this amazing synthesized report. So you can see here the first set of websites that it's decided to research. And in fact, I can even just click through and open them directly if I wanted to. But the great thing about deep research is you don't have to sit here and babysit. It's all under your control and your approved plan. And so now you can go off and do other things. In fact, you don't even have to stay on the device. You can switch to mobile if you wanted to, and the Gemini app will notify you when it's complete. So again, I'm going to not wait for this thing to finish.

18:52Right before we started chatting, I, in a different tab, actually let this exact same report complete. And so you can see here now I have a very Canvas-like editing experience where I can actually go through and read this amazing comprehensive report. And again, let's see how many, this is, you know, hundreds of sources probably would have taken me, you know, half a day if I really wanted to read all of these different sources and then construct them all together into a synthesized report. And I no longer need to do that. I can have Gemini do it for me. Is it doing inline citations for a bunch of the sources that are being visited?

19:34Yeah, absolutely. In fact, you can see it here. We actually corroborate at both the paragraph level and even the sentence level. So you can see once I expand sources for a paragraph, I can actually go through and get little indicators of exactly where we got each piece of information. And so it's a really good question because both we want the output to be as trustworthy as possible so you actually can leverage this for real for all sorts of really important workloads. But also we want to make sure to showcase great sources on the web. This has been a remarkable way to discover whole new websites for topics that I'm interested in that I've been able to discover through deep research and just actually looking through the citations and then the sources links at the bottom.

20:20Um, so, you know, um, uh, yeah, it's, it's an incredible new way to think about using the web. My, my really quick obligatory question, because I get this every time I post anything about deep research is developers really like this. Um, and they want this in an API. I don't know. Um, I think maybe it's on us to go and potentially make an API, but I'm, I'm just curious, like to get your like high level thought about, um, how this type of like product, you know, building it into the Gemini app versus like something that's available to developers? Like, you know, could a developer go and build this themselves today?

20:57Or is there a bunch of like special magic that we're doing that requires, would actually require us to put this in an API? Yeah, it's a really good question. I think, you know, we talked earlier about what it took to kind of port the V1 version on the 1.5 Pro model to the new thinking model. And, you know, the amount of work it took and the kind of like, you know, as an example, supervised fine tuning data that we had to curate and collect and, you know, kind of post-train the model in the first version was no longer needed in the second version. And I think even some of the prompts got a little bit simpler.

21:29And so I think, you know, there's still quite a bit of special sauce that we've bundled into the feature on top of the basic model. But the trend line is, I think, the important part for developers, where every model iteration, it's getting simpler and simpler to build more and more sophisticated agentic experiences. And I think that the reasoning power of this class of thinking models is really the big breakthrough, especially if you're trying to build these kind of longer running agentic experiences. And so I can expect whether it's a deep research specific API or not, it's going to get easier and easier as we continue to update the base models to build experiences like this.

22:10Yeah, Dave, this is awesome. How are you thinking about sort of, you know, the amount of content that's being shown for like people on different devices, like the mobile story for this versus for deep research versus like the desktop experience? And I sort of think about how I'm, you know, searching for stuff on the go versus, you know, sitting at my desk, like doing like actual deep work. Yeah, it's a great question. I mean, it kind of reminds me of this YouTube, you know, metaphor for model picking with respect to like giving you the right content for the moment. You know, we're really excited also by the ability to create audio overviews now in Gemini.

22:48And you can do that now for any content, any content type. You can upload files, you can chat with the thing, you can do deep research and all of this can be turned into now an audio overview. And I think that's the perfect on-the-go consumption mechanism. So oftentimes my use case has turned into kickoff of research, you know, maybe on desktop, maybe on mobile. But either way, I kind of cue a couple of them up. And then on my drive into work, I have all this amazing, rich audio overview content to listen to. I can spend hours listening to this content. And it's just amazing. You can kind of infinitely now generate research and then really incredible content that you can listen to on the go.

23:27So it's all at your fingertips and it's all now available. I should also mention too, Deep Research is now available for anyone to try. So you don't need to have Gemini Advanced to try Deep Research. And so again, all these features are coming together and we're really excited about everyone being able to try this. Is my understanding correct that the audio overview is like the same sort of audio overview that's powering the notebook LM experience that people love? Yeah, absolutely. We worked really closely with the Notebook LM team, and it's almost identical implementation. So if you've used Notebook LM or you've heard about their great audio overviews feature, we're really excited to be bringing it to Gemini app.

24:11I feel like the Gemini app is becoming this interface to all of the stuff that you do with Google and all these different technologies, which is actually a perfect segue to talk about all the personalization stuff that launched in the Gemini app. and I'll give my sort of bad explanation of it. And then let's do a deep dive and actually look at some demos. But my understanding is starting with Google search, you can actually now sort of bring in a bunch of your search history into the context of the Gemini app and have that sort of inform some of the searches. And we were talking offline about how this like intuitively doesn't seem like it would make a big difference, but actually for a bunch of user queries, like you end up getting like this really magical experience.

24:52So do you want to talk us through and show us this? Yeah, absolutely. Well, so just to start out, I kind of want to level set on our vision for personalization in Gemini App. You know, today when you use Gemini App or really any chatbot on the market right now, it's a very transactional, almost an incognito mode style interaction where it knows nothing about you. You know, it's almost starting from scratch from the very beginning. And this can be quite exhausting. You know, prompting is super powerful, but having to kind of remind Gemini App again and again, this basic context like, hey, I work at this place and here are my preferences.

25:28It can be quite exhausting. And so people oftentimes are just not doing that. And they're learning to kind of have a very limited and transactional interaction with the product. We rolled out a couple of features over the last couple of months where you can actually, when you tell the model to remember something like that I'm a vegetarian, it will actually remember that across all of your sessions. And then you can go into settings and edit those specific memories where the model can actually look up previous chats on the fly. So it gives this feeling that as you're chatting with Gemini app, it's learning with you and you can reference, hey, you know, yesterday we were working together on a project.