Google DeepMind's Vision for AI, Search and Gemini with Oriol Vinyals from Google DeepMind

0:05Hi, listeners, and welcome to KnowPriors. Today, we're talking to Aureel Viniels, the VP of research at Google DeepMind and technical co-lead for Gemini. His storied career in machine learning includes leading the AlphaStar team, which built a professionally competitive and pioneering StarCraft agent all the way to today. And we're really excited to get his historical perspective on where we are in machine learning. Welcome to the show, Oriol. ORIEL KERNIKARZIHRIIJANI Yeah, amazing. Thanks, Sarah, for the invitation. And likewise, thanks a lot for hosting me. ORIEL KERNIKARZIHRIIJANI Yeah, thanks for joining.

0:33LESLIE KERNIKARZIHRIIJANI Last year was an eventful year at Google and DeepMind. How is that research effort organized now? What do you think of the mission as internally? Yeah, so sure. I mean, I'm happy to obviously discuss the different phases that research organizations have gone through in the last many years. But focusing on last year, two major events happened. One was that the Gemini project was formed as a result of having two sort of parallel efforts on LLMs, mostly led by Google Brain and what we now call legacy DeepMind. So earlier in the year, there was an effort to merge the two projects.

1:14And that's when sort of Jeff and I came together and brought the two teams together to create the very first Gemini model, which was eventually released later in the year. Then the second big event was to take all the organizations that were doing AI research or AGI research and also form a singular organization. That's what today is called Google DeepMind. And it comes from Google Brain and Legacy DeepMind coming again together under one roof. Obviously, Gemini being a very large and very important project within that organization. And really the goal of Gemini itself is to create an awesome core model to power the technology that, of course, LLMs today are powering all around the world.

2:02And we obviously expect this to all increase. How do you interact with the rest of the company and like Google as a business? And I'm like, I feel like I have to ask you, does AI replace traditional search? So even running that from a research standpoint is super interesting, right? There's two major centers, one in California, one in London, given the organizations that we come from. So that in itself is very interesting. In a way, we have the project running 24-7, which is helpful when you train these large models. And then you have to do a few things, right? One of the things we do, of course, is trying to build state-of-the-art technology, showing from sort of a research, knowing where the field is coming from and when it's going to, trying to really showcase from our own sort of intuitions and ambition what might come next, right?

2:54So a prime example of this was, for example, the long context that we released earlier in the year. Millions of tokens now have been able to be processed by our models. But then, of course, we also sort of take into consideration all the different needs from the different products that we work with. Google has a lot of product areas. So we try to focus, of course, initially, especially to form the project. we try to focus on critical projects. And you see that very much by how Gemini is first surfaced to users or to enterprises, right? So obviously cloud and enterprise is very important. Developers as well.

3:37Super cool to put these models in the hands of creative minds that are going to do things you didn't even anticipate these models could do. And then very important, formerly known Bart, now Gemini App, which is sort of the chat bot surface of our models. And then maybe the last very important piece indeed is Search, which is trying to integrate, of course, this technology into their product. And of course, has a lot of users. So it's extremely exciting to think, well, the decisions you make at modeling eventually, and eventually means just maybe a few couple months after or so, will make it into the users that maybe are signing up for a beta, etc.

4:17So super exciting. And it's obviously connected. It's the core of the company, really, especially for the products that require very intelligent AI systems like the ones we're creating today. How do you think about the various types of use cases that fall under chat-based model versus search-based models? Because I remember I was at Google many years ago at this point. And at the time, a lot of the different types of search queries were broken into different chunks. There's navigational queries. You're trying to get to some other site. And it just kind of helps direct you there. There were sort of strong intent commerce queries.

4:52You know, you could kind of break it down. There's medical queries. And so you can kind of map out the world of different types of things that the users are actually trying to do with the searches, the interface to get there. What do you think will move more towards chat? And what do you think remains in the domain of more traditional search based approaches? We might not know the answer. And like we're sort of experimenting in a way. Right. So you can kind of think an LLM first experience like that's the chatbot. Right. So so in there, search has a role to play because it can be seen as a tool that enhances the answers of your chat experience.

5:24Right. It provides citations, a bit more reliance. We know that, of course, language models can and do hallucinate. So there's that sort of LLM first point of view, which we are building up sort of from it's kind of more of a new product. And then Search itself, which is obviously embracing sort of enhancing some of, as you said, different query types that could be useful to be sort of do a bit more like summaries, like AI summaries. But there's a lot more coming. I think IO actually showcased this year quite a lot of a good breadth of vision of what Search is trying to do with language models.

6:06Right. So some of it is not ready. but it's being tested and obviously like feedback is very important. But I think for now, it's hard to think of a convergence even or that one sort of dominates the other. I think both right now seem useful in different ways. I as a user use both certainly. But I think what is very clear though is that even if search is sort of the initial point where you have a query and you want to research something, it just feels like that experience will tremendously be enhanced by these models. So the search product itself is trying to figure out how to integrate the LLM answers or their capabilities, reasoning capabilities, etc.

6:53So I think we're going to see a lot of that. And I kind of call like from search integrating LLMs and of course the vice versa is very obvious as well. And then as a core model project like Gemini project is, we have an open mind. We don't really need to decide one or the other. And one of the things of operating at the scale that Google operates is that, yes, we are the research team that builds the models. But then, of course, the PAs are the ones that are going to drive the strategy, of course, influenced by what the models can do. And with input from many of us that have been dreaming about this world.

7:32But it's kind of cool to also have the very experienced folks, right, sort of iterate with their users and their use cases. That makes a lot of sense. I mean, I think one of the things that's very understated or forgotten about Google is the degree to which it really was the world's first AI first company, or ML first company, as it used to be called, right? I mean, the entire product, both in search and ads, was very AI-driven in the very earliest days. And part of that, I guess, is also inferring user intent through action and then algorithmically adding that as an outcome. And so I guess to your point, you could continue to use search as a primary UI or interface or entry point.

8:08And then if it routes to more of a chat-based thing or a chat bubble could pop up or something else, it could just happen organically based on the type of intent that the user is exhibiting. And so I think you're raising really interesting points about the capabilities of Google and all the rest of it. But I guess related to that, what has been the most surprising thing about how users or companies have been interacting with Gemini? Yeah, there's quite a few, right? I guess maybe the ones that surprised me the most, because initially I even thought this was just a number that you could report, is the fact that infinite context length is coming eventually.

8:48So I thought, look, I mean, this is interesting, right? We come from a world where we had recurrent neural networks and LSTMs that actually had infinite memory, although it was not very capable. The models in practice, they never remember more than a few hundred words or so. So that was kind of first that we could make the context length so long. And then seeing the use cases just emerge even internally when we were first just trying the model. So that, I don't know, that seems very trivial now in hindsight, but, you know, putting a whole one hour video and just ask anything and it feels superhuman, right?

9:26You just literally put the video in and after 10 seconds, 30 seconds, I mean, it does take some time to process the context, but you just can't ask anything, right? And I mean, thinking of computer vision as a field or video question answering some of these data sets that we come from. I mean, they all seem very dwarf then compared to the capability that was in our hands. And then we put it in the hands of developers and we saw like, I mean, amazing, obviously, demos and things that people could do, even as mind blowing as you point the camera to the screen directly and not even the code, right?

10:02It's just the letters that appear on the screen and you can debug like that, right? So you can imagine how future interfaces will be effortless. I mean, you just need to point the camera, ask a question and you get answers. It's an interesting research problem, of course, but maybe it's not going to be that useful to now thinking, wow, that's amazing. Yet, it's not very mainstream either, right? So we are kind of still trying to discover, right? What does this enable? We showed Project Astra where, of course, memory, you get the phone and you just interact with it as if it is an agent. Memory is very important, but it's still not very clear.

10:40in a few years what this might be, although you could imagine, well, the whole web is in the context or all your personal data because it's remained in your device, because it's the working memory of the model rather than the weights in lots of applications, but still fairly early days. So it surprised me, yet, of course, in a way, it hasn't taken off fully, gone fully mainstream. Although it does feel magical when you start interacting and realizing you can ask anything, you get an answer like this without watching any of these movies or many books you can upload, et cetera. MARK MANDELAVYSCHENKOVICHEIASCIA.

11:15What do you think is the time frame for very long context windows really being in broad-based use? This is not a specific to Google question, but it seems like multiple people are on a trajectory to add this at scale. and you're also seeing it crop up in biology models and other things like that. So I'm just a little bit curious in terms of the timeframe for which it actually hits either a large enterprise or consumer use cases. And to your point, you could imagine ones where the company is doing something on behalf of the consumer by adding all the context from their device or from their interactions into the context window.

11:44Or it could be an enterprise uploads a large folder, including a bunch of legal documents, and then it gets incorporated into some query or something. If there is a comparing use case, the technology is not far away from being able to be deployed at scale. And of course, hardware is also being updated based on what I guess the research developments are. Right. So certainly like one, two years, this might be quite the context that is a commodity will be definitely enhanced by a factor. I mean, I don't know, 10x or so everywhere. and then, I mean, extremely long context, I think it's going to be a motivational drive for definitely from a research perspective and then deploying it at scale.

12:30Some techniques, like many have been explored already, like hierarchical memories and so on. And even RAC is pretty common, right? So we're going to combine these and probably thanks to the use cases, I expect, I mean, order of one, two years, you might see, wow, We went another order of magnitude from both state of the art and, of course, what's considered a commodity. So I'm pretty certain about this. That's, again, model of finding use cases that will be compelling to serve the model that requires more memory. There are some certain limitations, of course. But these will be figured out from a technological standpoint if there's the motivation for sure.

13:09In the very quickly coming era of infinite context, like what is the relevance of retrieval architectures and more hierarchical memory? And like, you know, I think you can make an argument for this being continually relevant just from an efficiency perspective. But, you know, how do you think about it? Yeah, I think definitely the efficiency argument to hierarchical memories to make context even longer make a lot of sense. And even from just efficiency of learning and, of course, of retrieving the memory in a sort of course-defined manner like, you know, an intelligent being like ourselves might do makes sense.

13:48So I think that even the quality will motivate this sort of solution regardless. And we do have a lot of experience, of course, of retrieval-based methods, definitely at Google. And then combining them with neural-based methods. I think it's a matter of time and finessing the details and the use cases of how much the problem with, of course, retrieval-based methods is they tend to simplify things to say, hey, like this whole book is just a single vector. Whereas if you just upload a whole book into Gemini and ask questions, it can really reason about every single word, right? So probably finding the middle grounds for different use cases is needed.

14:31But I think to me, it seems like a feature, not a bug, that we do have a bit of a hybrid mode, perhaps going into the future. And research will be driven like this. How do you contextualize this moment in time, just in terms of, you know, what the biggest limitations are for current state of the art LLMs and like what's worth working on? I mean, there's one reflection that even many years ago with friends who were kind of early, quote unquote, in the game, they said, well, get ready. Lots of brilliant people as this gets mainstream will enter the field. And you certainly see this, right, with open sourcing and a bit of a random search event, right?

15:09It's not like, I mean, you just selection bias, like someone does something random, but people actually want that. And then that becomes sort of viral in a way. So I think there's the sheer size of the field that is one aspect that I think we were sort of anticipating. But to me, that's one of the biggest changes that I've seen, that there's more brains, more different backgrounds coming into the field. And that is combined. I usually tend to assign credit with what has happened to, of course, the scale of data and compute. Algorithmic advances that you can simplify, but there's certainly been some that have been important in the last 10, 20 years or so.

15:53And then actually the accessibility, right, the software, the open sourcing efforts, those have been quite critical to then create these sort of exponentials or linear trends in log scale that we're seeing. Now, a bit more into sort of how I see the field from maybe like I tend to call maybe the 2000, let's say 10 to 20, like deep learning era. So what that era did, right, is it took a set of algorithms that were general, right? The algorithms are like stochastic gradient descent, deep learning, neural networks, right, reinforcement learning. And you could think of these are ingredients. They're common.

16:35You expand them over the years, but they're certainly the same. And then you just apply them to a domain and you get extremely good at that domain. Right. So we have, you know, mastering the game of Go, beating the ImageNet challenge, becoming state of the art speech recognition, a state of the art image generation. Right. So so that decade is kind of the models themselves certainly are not general, but the algorithms are general. You could just take the same algorithm and then change the data set and, of course, tweak a couple of things and voila, you get like protein folding really enhanced from the traditional methods.

17:14And I think then the greatest insight, of course, came from realizing that. And I think that's a lucky factor for us because it makes communication with these entities much easier. But modeling language turns out to be such a powerful abstraction for generality. So, of course, the GPT-2 paper especially posts that sort of, in the abstract, it says, look, you can solve every task, not one task, by modeling language. And then perfecting that, the whole field, of course, building up from a lot of years of research, created what is not just the algorithms are reusable, but actually the model now becomes general.

17:51So that's why I think like, OK, HGI is getting closer that, you know, we have powerful models first in 2010 to 20. Now we have general models. And I think multimodality has been more recently another amazing breakthrough that these techniques expand not to language, but also to vision, sound, videos, etc. et cetera. So that means that we have very powerful general models that from an AGI definition standpoint, it starts to tick many boxes. Reasoning capabilities of the models are there, but I don't think we've perfected sort of making the reasoning very crisp and accurate so that these models would not hallucinate or would not, you know, the model might solve an Olympian mathematical problem yet failed to then discern a very basic puzzle, right?

18:44And I think what you do with the model, this reasoning step, there's a lot of ideas and a lot of experience and a lot of algorithmic advances we've also done in the last few years over like search and so on. But we haven't quite perfected this. And then of course the question is, you know, push the frontier and move forward in certain domains at least. But yeah, we've come a long way. I think the investment And luckily, these models are finding usefulness. So the resources that go into the training, the models, right now, there's a good sort of feedback loop, right, of revenue and then reinvesting, certainly from the biggest players.

19:24So as a researcher, of course, I mean, we welcome that. What is the difference between having reasoning capability and having reasoning capability that's crisp and accurate? Sometimes the distinction between probabilistically solving something, right? So right now, you know, you get these models, they still assign probability mass over every sequence of, let's simplify and think of not multimodal, but words. So every single sequence of words, it will assign a probability distribution over those. So then you, of course, are absorbing all the knowledge on the Internet and then sharpening those models around following instructions, being aligned with humans.

20:06but you still have this probability distribution that will assign non-zero probability to certain things that would not deem to be correct. Although in language, of course, there's so many ways to say the same thing correctly. So that's why these models shine in the end of the day. They are very efficient ways to integrate over all possible sequences, right? Now, it's possible that, let's say now you predict when it's a hard problem, right, a tricky question that requires deep knowledge, you might be at 95 % accuracy. But of course, this will create errors. Even if it's small, this is deployed to the whole world.

20:47So certainly you will get to see the mistakes. So one thought would be like, you just keep making the models larger. you keep improving the algorithms, and you're going to hit a point where the probability of a mistake vanishes. It's possible. We will obviously explore that. But to accelerate that sort of progress, you want to start sort of really exploring what's the reasoning the model has. And by making it more redundant, more logical, by iterating more on these kind of ideas, You could imagine generating a very small program, right, that runs slowly with the language model at the center and then getting that 99.99 % faster.

21:37And of course, you're going to do both as an ambitious lab and so on. But the crisply means that this probability of error diminishes. And of course, we can always put more compute power. Humans will make mistakes. We get tired, et cetera. But I mean, these models are powerful. We can put more hardware that inference. So the hope is that in that sense, they become at least as good as humans. You know, one way to frame this problem is that even DeepMind or any large lab or, you know, the human race, we have some limited amount of compute toward this problem. Right. And it's like there's there seems to have been a strong shift of how much of that compute should be scaled at training time versus at inference time with, you know, test time search or some of the techniques that you describe in system two.

22:29What is your prediction of like what that mix is of compute at training versus inference time compute, you know, let's say two or three years from now? What we're kind of all aiming to discover is to make the bitter lesson from reach out and through, right? The bitter lesson states like use skill learning and use skill search. And then that's all you have to do as a computer scientist. I mean, it's controversial. I certainly like simplicity. I'm a deep learner at heart. So I don't disagree with that. Certainly the learning scalability part has been tested and proven quite heavily recently. The search, at least when you do not have access to a perfect reward, which is this is the current, I think, current problem we have in language.

23:16That is, you know, one of the key research areas. How do you assign reward fastly to statements that, I mean, not even, again, you and I might agree to be true, right? I mean, is the sky blue? I mean, I don't know. At night, it's not, right? It's quite an interesting kind of point that assigning truth or one or zero, which is required in games, is not so applicable here. Now, historically, if you look at AlphaGo, which actually followed quite closely the recipe of you pre-train your model on all human data, you then use RL to make it better, and then you do some search at inference time, the compute there was very skewed for the middle step, the reinforcement learning step, right?

24:04So I don't even know the exact numbers, but certainly the majority of compute was spent on the training of the self-play, as we called it at the time, self-improvement loop. And if you look at today, that's clearly not the case. Most compute is spent in pre-training. And in fact, you will see over and over that you overfit. So you have to actually stop the reinforcement learning process to overfit to these imperfect reward functions. Otherwise, you start sort of doing a bit of adversarial search against a reward function that might be perfect because you have a data set of human preferences and all of a sudden the model might discover, hey, you can issue lots of emoticons and this reward function thinks that's great.

24:50Right. And clearly, like we have a problem of reward functions not being as accurate as in the game of Go or chess and whatnot. not. And then there's the third component, which is now you have your model trained. How much do you let it ponder? Which in AlphaGo, again, using the same example, well, the rules of Go say, don't quote me, but let's say roughly a game must last four hours of compute time from human's perspective. So we know we had quite a limited inference time because we obviously couldn't go over time. So of course there was parallelism and so on involved, but the compute there was certainly not as big as the one you used to train.

25:28So to me, that balance feels correct. Like some are on pre-trading and here we're trying to learn every task. So certainly that's going to be, you know, let's say it can be as high as 50%, not as high as over 90 like today. And then the rest mostly on reinforcement learning or if we get access to good rewards, that would be kind of the next maybe piece of compute, but much bigger than it is today. And then inference time, I mean, system two is slow, but it's not terribly slow. So unless you ask the model, I mean, soft protein folding, then probably you can go for a vacation for a month and the model will come back with a solution.

26:07And I think a few seconds of compute is OK for inference. So that would be the rough. So that's a small percentage probably compared to the amount you spend in training. Of course, you're serving many queries for billions of people. And then that means research is needed, especially on the middle bucket of reinforcement learning step that currently feels still reasonably in the early stages of research. How do you think about scaling the reward function beyond games once you really hit superhuman performance of the model? I'm thinking of things like the older, like MedPalm 2 models and things like that, where they outperformed human physicians in terms of output relative to physician expert panels.

26:48And then obviously you could then do some post-training with physician experts is the key. But at some point the machine will be better than that. And so how do you keep scaling reward functions? Traditionally, right? You just get, it's supervised learning, right? Reward function means good or bad. So we can scale that process as much as we have so far. I mean, obviously, many, many players are realizing the power of human annotation. And in fact, deep learning comes thanks to amazing like Fei-Fei Li and lab effort to label a data set of a million examples. Right. So that way of scaling is one.