Koray Kavukcuoglu: “This Is How We Are Going to Build AGI”

0:00Gemini 3, we're sitting here. Reception seems super positive. The vibes of the model are good. I'm very excited about the progress. I'm excited about the research. We actually pushed the frontier on a bunch of dimensions. This is how we are going to build AGI. We want to do it the right way. And that's where we are putting all our minds, all our innovations. It's not like it's this purely research effort that's off in a lab somewhere. Like it's a joint effort with us in the world. This is a new world, right? There's a new technology that is defining a lot of what users expect. We are in some sense like co-building AGI with our customers.

0:32So all of a sudden you enable a lot more people to be builders. Bring anything to life. Bring anything to life, right? Yeah, I feel like the next six months are going to be probably just as exciting as the last six months and the previous six months before that. We are lucky to be living in this age. It's happening right now. It's very exciting.

0:57Hey everyone, welcome back to Release Notes. My name is Logan Kilpatrick. I'm on the DeepMind team. today. It's an honor to be joined by Koray Kavachulu, who is the CTO of DeepMind and the new Chief AI Architect of Google. Koray, thanks for being here. I'm excited to chat. Me too. Yeah, very excited. Thanks for inviting. Of course, Gemini 3. We're sitting here, we've launched the model. Reception seems super positive. I think we went out and we obviously had a hunch about how good the model was going to be. Leaderboards looked awesome, but I think putting in the hands of users and actually getting out is like...

1:27That's always the test, right? Like, I mean, we have been like benchmarking is the first step. And then we have been doing tests. We have been like with trusted testers with priorities and everything. So you get a feeling that, yes, it's a good model. It's capable. It's not perfect, right? But like, I think I'm quite pleased with the reception, really. People seem to like the model and the kinds of things that I think we found interesting. They also found interesting. So like, that's good so far. Like, this is good. Yeah, we were talking yesterday. and the thread of the conversation was just around appreciating this moment that the progress isn't slowing down, which I think resonates with me.

2:07And as I was reflecting back to the last time I sat next to you, we were at I.O. as we launched 2.5 and we were listening to Demis and Sergey talk about AI and all that stuff. I feel like the progress has not slowed down, which is really interesting. When we launched 2.5, it felt like a state-of-the-art model and it felt like we had actually pushed the frontier around a bunch of dimensions. And I feel like 3.0 delivers that again. And I'm curious what the scaling conversation of can it continue, continues to go. What's your sense right now? Yeah, I mean, look, I'm very excited about the progress.

2:39I'm excited about the research. Like when you are actually there in the research, there are a lot of excitement in terms of like in all areas of this, right? Like I mean, from data, pre-training, post-training, everywhere. We see a lot of excitement. We see a lot of progress, a lot of new ideas. at the end of the day like this whole like this this whole thing is really running on innovation running on ideas right the more we do something that is impactful that is in real world that people use you actually get more ideas because like your surface area increases the kinds of signals that you get increases and i think like the the problems will get harder the problems will get more varied right and with that like um i think like we will be challenged and these kinds of challenges are good.

3:27And I think that is the driver for going towards building intelligence as well. That's how it's going to happen. I feel like sometimes if you look at one or two benchmarks you can see squeeze, but I think that's normal because benchmarks are defined at a time when something was a challenge. You define that benchmark and then of course as the technology progresses that benchmark becomes not the frontier. It doesn't define the frontier. And then what happens is that you define a new benchmark. It's very normal in machine learning, right? Benchmarks and model development is always hand in hand. You need the benchmarks to guide the model development, but you only know what the next frontier is when you get close to it so that you can define with the new benchmark.

4:10Yeah, I feel this way and there was a couple of benchmarks like HLE originally, all the models were horrible on and doing like one or two percent. And I think now the newest with DeepThink is like 40-something percent, which is crazy. ArcGi2 was originally all the models could barely do any of that. It's now like 40 plus. So it is interesting. And then it's also interesting to see, and I don't have the context on why, the benchmarks that are static that do have a little bit of like the test of time, if you will. And like, I think they are probably close to saturating, but like GPQA Diamond, as an example, like continues to stick around, even though we're eking out 1 % or whatever.

4:47There are really hard questions there. Yeah, yeah. Like, I mean, and like those hard things we are still not able to do. Yeah. Right. And they still test something. But if you think about like where we are with GPQA, like it's not like, oh, there's a, there's like you are at 20s and you need to, you need to go to 90s. Right. Like, so you're getting close there. So the number of things that it defines as unsold is like, of course, like decreasing. So at some point, it's good to find new frontiers, new benchmarks. And defining benchmarks is really, really important, right? Because if you're going to think about, like, if we think about benchmarks as the definition of progress, which does not necessarily always align, right?

5:29Like, there's this thing between, like, there's progress and then there's the benchmarks. In an ideal case, it's 100 % aligned, but it's never 100 % aligned. like to me the most important measure of progress is like we have our models in real world and scientists use them students use them like lawyers use them engineers use them and then like like people use them to do like all sorts of things writing creative writing emails easy or hard right like that spectrum is important and different topics different domains if you can actually continue delivering larger value there. I think that's progress.

6:07And these benchmarks help you quantify that. Yeah. How do you think about, and maybe even there's a particular example from 2.5 to 3 or whatever, we could choose whichever model version change you want. Where are we hill climbing? And actually how in a world where there's a zillion benchmarks now actually of you could choose where you want to hill climb, how are you thinking about for just broadly Gemini, but also maybe the pro model specifically like where do we try to help climb for that i think like there are there are several important areas right like one of them is instruction following is important the instruction following is where like the model needs to be able to understand the request of the user and to be able to follow that right like um you don't want the model just like answering whatever it thinks it should answer right so that instruction following capability is important and that's what we always do.

7:00And then for us, internationalization is important. Google is very international and we want to reach everyone in the world. So that part is important. And I feel like 3.0 Pro, at least, I was talking to Tulsi this morning and she was remarking about how incredible the model is for languages that historically we haven't been really good at, which is awesome to see. So continuously, you have to put the focus on some of these areas. They might not look like, okay, it's the frontier of knowledge, but they're really, really important because you want to be able to interact with the users there. Because as I said, it's all about getting that signal from the users.

7:36And then if you come to actually a little bit more technical domains, function calls, tool calls, agentic actions, and code, these are really important. Function calls and tool calls are important because I think it's a whole different multiplier of intelligence that comes from there. both from the point of view of the models being able to just naturally use all the tools and functions that we have created ourselves and then use it in its own reasoning but also the model writing its own tools right like you can think that like the models are in a way the models are tools in themselves as well so that one is is is a big thing like obviously like code is because not just because we are all software engineers, but also because we know that with that, you can actually build anything that is happening on your laptop.

8:28And on your laptop, it's not just software engineering that happens. Bring anything to life. Bring anything to life, right? So a lot of the things that we do right now happens in digital world, and code is the basis for that, to be able to integrate with anything that happens, pretty much anything that happens in your life. Not everything, but a lot of things. That's why these two things together, I think, like, makes up for a lot of reach, like, for users as well. I give this example of, like, wipe coding, right? I like it. Why? Because, like, a lot of people are creative. They have ideas, and all of a sudden you make them productive, right?

9:05Like, going from creative to productive in a way that, like, you can just write it down. and then like you see the application in front of you and like it is like I mean most of the time it works and when it works it's great right like I mean so cool and that loop I think is great so all of a sudden you enable a lot more people to be builders like building something like I mean like it's great I love it yeah yeah thank you for this is the AI studio pitch I appreciate we'll clip this part out we'll put it out online um one of the interesting threads that you mentioned is like how and actually the as part of this Gemini 3 moment we launched Google anti-gravity a new agent encoding platform how much do you think about like the importance of having this product scaffolding to hill climb on quality from from a model perspective obviously yeah tool calling and coding yeah yeah it's like to me it's very very important and I think like like Like antigravity as a product itself, yes, it's exciting.

10:04But like from a model perspective, if you think about it, so it's like double-sided, right? Like let's talk about first the model perspective. From the model perspective, like being able to have this integration with the end users, in this case software engineers, and learning from them directly to understand where the model needs to improve is really critical for us. I mean, it is important in areas. Like Gemini app is important for the same reason, right? like I mean understanding users directly is very very important. Antigravity is the same way, AI studio is the same way. So having these products that we work really close a bit and then understanding and learning that getting those user signals I think is really massive and Antigravity has been a like a very critical launch partner.

10:49It hasn't been long that they have joined right but in the last two three weeks of our launch process like their feedback has been really really instrumental. The same thing with search AI mode, right? Like, I mean, AI or we use even that we get a lot of feedback from there. So, like, to me, this integration with the products and getting that signal is the main driver that we understand. Like, of course, we have the benchmark. So, like, we know how to push the STEM, the sciences, the math, that kind of intelligence. But it's really important that actually we understand the real-world use cases because, like, I mean, this has to be useful in real world.

11:25Yeah. In your new chief AI architect role, you're now responsible for also making sure that we don't just have good models, but the products actually take the models and implement them and sort of build sort of great product experiences across Google. How much, obviously I think this is the right thing for users, like getting Gemini 3 and all the product services on day one is like an awesome accomplishment for Google. and I think even more so, hopefully more product services in the future. How much additional complexity from the deep mind perspective do you think it adds to try to do it? In some sense, life was simpler a year and a half ago.

12:02But we are building intelligence, right? A lot of people ask me, you have these two roles. I mean, I have these two titles in a way, but they are very much the same thing. If we are going to build intelligence, we have to do it with the products, through the products, connecting with the users. With the Kaya role, what I'm trying to do is make sure that the products in Google have the best technology that is available to them. We are not trying to do the products. We are not product people. We are technology developers. We develop the technology. We do the models. And of course, just like everyone is opinionated on anything, people are opinionated.

12:38But the most important thing for me is making the models, making the technology available in the best way that is possible, and then work with the product teams to enable them to build the best product in this AI world. Because this is a new world, right? There's a new technology that is defining a lot of what users expect and how the products should behave, what information that they should carry over, and all the new things that you can do with this new technology. So to me, it's about enabling that across Google, working with all the products. I think that's exciting, both from the product perspective, from what users getting perspective, but also from the point of view of, like, as I said, that's our main driver.

13:23Like, it's really important for us to be able to feel that user need, to be able to get that user signal. That's critical for us. So that's why I wanted to do it, that, like, this is how we are going to build AGI. This is how we are going to build intelligence, like, with the products. Yeah. That's how I think it's going to happen. This is a great tweet for you to put out at some point, because I do think it's interesting. I share this perspective that we are in some sense co-building AGI with our customers, with the other PAs. It's not like it's this purely research effort that's off in a lab somewhere.

13:56It's a joint effort with us in the world. And I think it is actually a very trusted, tested system as well. It's a very engineering mindset that I think we are adapting more and more. And I think it's important to have an engineering mindset in this one. because when something is nicely engineered, you know that it is robust. It is safe to use. So we are doing something in the real world and we are adapting all the trusted, tested, in a way, ideas of how to build things. And I think that's reflected in how we think about safety, how we think about security. We try to think about it, again, from that engineering mindset of think about it from the ground up, from the beginning.

14:42not something that comes at the end right like we don't like so when we are doing post-training models when we are doing pre-training when we are looking at our data we always have this like everyone needs to think about this like do we have a safety team obviously we have a safety team and they are bringing in all the technology that is related to them we have a security team they're bringing in all the technology but enabling everyone in Gemini to actually also heavily be part of that development process that is that is taking this as a first principle and those teams are themselves part of our post-training teams, right?

15:14So when we are developing these, when we are doing these iterations, release candidates, just like we look at like GPQA, HLE, those kinds of benchmarks. We look at its safety security measures as well. Like that's, I think that is a very, like that engineering mindset is important. Yeah, I completely agree with you. I think it also feels natural to Google, which is also helpful because of how collaborative and like big, how big the effort is now to ship Gemini models out the door. I mean with Gemini 3 I think like we were just reflecting on this like to me one of the important things is like this model has been a very team Google model.

15:49We should look into the data it might be like one of the I mean some of the like maybe the Apollo, NASA programs had a lot of people but like it is I think this massive Google global also global effort across all of our teams to make happen which is crazy. Every Gemini release like takes people from like this continent, Europe, Asia, all around the world. We have teams all around the world and they contribute. Not just GDM teams, all teams across Google. It's a huge collaborative effort. And we sim-shipped with AI mode. We sim-shipped with Gemini app. These are not easy to do because they were together with us during our development.

16:30That's the only way that on day one, we can actually go all together out at the same time the model is ready. and we have been doing that. When we say across Google, it's not just like people actively trying to build the model. All the product teams, they're doing their parts as well. Yeah, I have a, maybe this isn't a controversial question, but, you know, Gemini 3 were sort of soda on many benchmarks, a lot of benchmarks. We're sort of sim shipping across, you know, across the Google product surfaces, our sort of partner ecosystem surfaces. The reception is very positive, sort of the vibes of the model are good.

17:02if you sort of fast forward, knock on wood, if we sort of fast forward to like the next major Google model launch, like are there things that you are like still on your list of you wish we were doing X, Y, and Z? Like how does it get better than the gem or should we just enjoy the moment of Gemini 3? I think we should do more, right? Like we should enjoy the moment because like one day of enjoying the moment is a good thing. This is the launch day and like I think people are appreciating the model so like i'd like the team to enjoy this moment as well right but but at the same time every area we look at we also see gaps right like is it perfect in writing no it's not perfect in writing is it perfect in coding it's not perfect in coding i mean especially i think on the on on the area of like agentic actions and coding i think like um like i think that there's a lot more room there.

17:57That's one of the most exciting growth areas and like we need to identify where we can do more and we'll do more, right? Like I think we have come a long way. The model is like I would say pretty much like maybe 90-95 % of the people who will engage with coding in some ways. Are they software engineers or these are creative people who want to build something? Yeah. I'd like to think that this model is the best thing that they can use, right? But there are some cases probably that is, we still need to do better. Yeah. I have another sort of pointed question for coding and tool use. What do you think has it just been, if you sort of look at the history for Gemini, and obviously we had like a very multimodal focus for 1.0.

18:36And I think for 2.0, we started to make some of the like agentic infrastructure work. Like, do you have a sense of like why we, and I'll make the caveat that like, I think the rate of progress looks really strong, but like, why has it just been like a focus thing? Why we haven't been like state-of-the-art and agentic tool use from the get-go. But for example, multimodal, we have been, literally Gemini 1 was state-of-the-art and multimodal, and we've sort of held that for a long time. I don't think it was a deliberate thing. I think it was like, I mean, honestly, I think, like if anything, when I reflect back, I tie it to using the models, the development environment being closely tied to real world.

19:12The more we are tied, then we are more better understanding these like real requirements that is happening. And I think like in our journey in Gemini, we started from a point where, of course, like, I mean, like the AI research in Google is a huge history, right? Like the amount of amazing researchers that we have and the amazing history of AI research that has been done in Google. I think it's great, but like Gemini is also a journey of moving from that like research environment into this, like as we talk, this engineering mindset and getting into a space where we are really connected with the products, right like when I look at the team like I have to say I feel really proud because like this team is still majority formed by people like including me right like for five years ago we were writing papers like we were researching AI and here we are actually we are at that frontier of that technology and that technology you are developing it via products with the users it's a completely different mindset that we are building models every six months and then we are doing updates every month, month and a half.

20:16It's an amazing shift and like I think like we walked through that that shift. Yeah I love that. Gemini 3 progress has been awesome. Another thread that was top of mind is just generally sort of how we're thinking about like where the GenMedia models which I think historically have like not been a huge, I mean not that they haven't been a focus, they've always been interesting but I feel like we've had with VO3, VO3.1 with the Nano Banana model we've had like so much success from like a product externalization standpoint. And I'm curious how you think about in this like pursuit of we want to build AGI.

20:51Sometimes I think sometimes I can convince myself that like a video model is like not part of that story. I don't think that's true. I think in general you can sort of, you should understand the world and physics and all this other stuff. So I'm curious how you see all these things intertwining together. If you actually go back like 10, 15 years ago, genitive models were mostly on images, right like because like we could we could much better inspect like what is going on in terms of and also this idea of understanding the world understanding the physics as the main driver of doing generative models with images and and and then some like some of the exciting things that we have done with generative models like date back to like 10 years ago like maybe like 10 years ago feels like 20 right 20 years ago we were still doing image models right I mean that's I was hesitating a little bit but during my PhD we were doing like generative image models right like everyone was doing those at that time we walked through that like I mean we had uh we had things called like pixel cnn's right like they were like image generative models in a way what happened was um I think it is also it was also a big realization that text actually was the better domain to have very fast progress but I think it is very natural that the image models are coming back and like at GDM we have had really strong image video audio models for a long time I think that's what I'm trying to explain maybe bringing those together I think is natural so where we are going right now is we have always talked about this multi-modality right and of course naturally like we have always talked about like input output multi-modality and that's where we are going right and when you look at it as the technology progresses the architectures the ideas in between those two different domains have been merging with each other.

22:37It used to be that these architectures were very different, right? But they are getting together quite a lot. So it's not like we are forcing something in. What is happening is naturally the technology is converging. As the technology is converging, it is converging because everyone understands where to get more efficiency from, where the ideas are evolving, and we see a common path, and that common path, I think, is getting together well. So, Nanobanana is one of those first moments, right? Like where you can iterate over images, you can talk to the model. Because what happens is that text models have a lot of world understanding, right?

23:11Like from the text. They have a lot of world understanding. And then the image model has the world understanding from a different perspective. So, like when you merge those two, I think you get exciting things. Because I think people feel that this model understands the neons that they want to get through. I have another question about Nanobanana stuff. Do you think we should just have goofy names for all of our models? Do you think that would help? Not really. Look, I mean, like, I think we didn't do it on purpose. Gemini 3. If we didn't name it Gemini 3, what would you have called it? Something ridiculous.

23:41I don't know. I'm not good at names. I think I like, I mean, it was Rift Runner, right? Like, it was Rift Runner. Like, we actually use Gemini models. Those are code names. We use Gemini models to come up with those code names, too. And Nano Banana was not one of those. Like, we didn't use Gemini, right? There's a story about it. I think like it's published somewhere. Yeah. I mean, as long as these things are natural and like organic, I think I'm happy because I think the teams who are building the models, it's good for them to sort of like have that connection. Yeah. And then when we release them, like I think that just like, I mean, that happened because we were testing the model with the codename, right, on LM Arena.

24:23And people loved it. And I think, I don't know, I'd like to think that it was so organic that sort of it caught on. I'm not sure if you can create a process to generate that. I agree with you. That's my feeling. If you have it, you should use it. If you don't have it, it's good to have standard names. Yeah. We should talk about Nano Banana Pro, which is our new state-of-the-art image generation model built on top of Gemini 3 Pro. And I think the team, I think actually, even as they were sort of finishing, Nano Banana sort of like had early signal that potentially doing this in a pro capacity, like you could sort of get a lot more performance on a bunch of like more nuanced use cases like text rendering and world understanding and stuff like that.

25:09Anything sort of top of mind for, I know we're a lot of stuff going on. I think like this is like probably where we see this like alignment of different technologies is coming into play, right? Like, I mean, because with Gemini models, we have always had like every model version is a family of models. Like we have the Pro, Flash, Flashlight, like this family of models. Because at different sizes you have different compromises in terms of speed, accuracy, cost, those kinds of things. As these things are coming together, of course like we have the same experience on the image side as well. Yeah.

25:42So I think it's natural that the teams like thought about, okay, like there's the 3.0 Pro architecture. can we actually tune this model more to be like generative image using everything that we learned in the first version and increasing the size and i think like where we end up with is something a lot more capable understands really complex like some of the most exciting use cases are you have large set of really complex documents you can feed those in we rely on these models to ask questions you can ask it to generate an infographic about that as well and then it works right? So this is where this natural input modality, input output modality just comes into play and it's great.

26:21Yeah, it feels like magic. I don't know, hopefully folks will have seen the examples by the time this video comes out but I think it's just, it's so cool seeing a bunch of the internal examples being shared around. It's crazy. Yes, I agree. Like it's exciting when you see that all of a sudden, oh my god, yes, like that's sort of huge amount of text and concepts and like complicated things explained in one picture such a nice way yeah like when you see those things like it is it's nice right like you you realize the model is capable and it's yeah and it's it's the there's like so much nuance to it too which is um which is really interesting i i have a parallel question to this which is uh probably december of last year uh december 2024 before, Tulsi was promising how we were going to sort of have these unified Gemini model checkpoints.

27:14And I think what you're describing is like actually that we've gotten really close to that now where like the architecture historically was unified in terms of like image generation and oh I see. Yeah yeah and I'm curious do you think like that I assume that's like a goal is like we want these things actually mainlined into the model and there's like natural things that stop that from happening and I'm curious like if any like context or sort of high level. Yeah look I think as I said the technology the architectures they are aligning. Yeah. Right so we see that happening. At regular intervals people are trying but it's an hypothesis and like you can't be ideology based in this right.