Jeff Dean & Noam Shazeer – 25 years at Google: from PageRank to AGI

29:33I mean, and obviously the idea is not to just pile up some giant pile of money but it's just it's create value in the world you know and so much more value can be created when these when these systems can actually like go and do something for you write your code or figure out problems that that you wouldn't have been able to figure out yourself and to and to do that at scale. So I mean, we're going to have to be very, very flexible and dynamic as as as we improve the capabilities of these models. Yeah, I guess I'm I'm pretty excited about kind of a lot of fundamental research questions that sort of come about because you see something that we're doing could be substantially improved if we tried you know this approach or things in this rough direction and you know, maybe that'll work maybe it won't but I also think there's there's value in seeing what we could achieve for end users and then how how can we work backwards from that to actually build systems that are able to do that.

30:36So as one example, you know, organizing information that should mean any information of the world should be usable by anyone regardless of what language I speak. Yeah. And that I think you know we've done some amount of but it's not nearly the full vision of you know no matter what language you speak out of thousands of languages. We can make any piece of content available to you and make make it usable to you. And you know any video could be watched in any language I think that would be pretty awesome. And you know we're not quite there yet, but that's definitely things I see on the horizon that would should be possible.

31:12Speaking of different architectures you might try, I know one thing you're working on right now is longer context. If you think of Google search as like it's got the entire index of the internet in its context, but it's like sort of very like shallow search. And then obviously language models have like limited context right now, but they can like really think it's like dark magic like in context learning right just like can really think about what what it's seeing. How do you think about what would be like to merge something like Google search and something like in context learning. Yeah, maybe I'll take a first step at it.

31:46I mean I could have thought about this for a bit. I mean I think one of the things you see with these models is they they're quite good, but they do hallucinate and you know have factuality issues sometimes. And part of that is you know you've trained on say tens of trillions of tokens and you've stirred all that together in your tens or hundreds of billions of parameters. Yeah, but it's all a bit squishy because you've like turned all these turquoise tokens together. And so the model has like a reasonably clear view of that data, but it's sometimes like gets confused and we'll give the wrong date for something.

32:22Right. Whereas information in the context window in the input of the model is like really sharp and clear because we have this really nice attention mechanism and transformers that the model can pay attention to things and it knows kind of the exact text or the exact frames of the video or audio or whatever that it's processing. And so right now we have a model that can deal with kind of millions of tokens of context, which is quite a lot. It's like hundreds of pages of PDF or you know 50 research papers or you know hours of video or tens of hours of audio or some combination of those things which is pretty cool.

33:03But it would be really nice if the model could attend to trillions of tokens, right, could it attend to the entire internet and find the right stuff for you, could it attend to all your personal information for you, right. I would love a model that has access to all my emails and all my documents and all my photos and I'm when I ask it to do something, it can sort of make use of that with my permission to sort of help solve what it is I'm wanting to do. But that's going to be a big computational challenge because the naive attention algorithm is quadratic and you can kind of barely make it work on a fair bit of hardware for millions of tokens, but there's no hope of making that just naively go to trillions of tokens.

33:46So we need a whole bunch of interesting algorithmic approximations to what you would really want to make a way for the model to attend kind of conceptually to lots and lots of more tokens to trillions of tokens and attend to your tokens. Maybe we can put all of the Google code base in context for every Google developer, all the world's source code in context for any open source developer. That would be amazing. It would be incredible. Right. The beautiful thing about model parameters is they are quite memory efficient at sort of memorizing facts. Maybe you can probably memorize order of one fact or something per model parameter, whereas you have some token in context.

34:38There are lots of keys and values that every layer could be kilobyte, a megabyte of memory per token. Jacob, you blow it up to 10 kilobytes. Yes. Yes. So there are some, there's actually a lot of innovation going on around, OK, how do you minimize that and the, OK, what words do you need to have there are there better ways of accessing bits of that information and, you know, just seems like the right person to figure this out. It's not like, OK, what is our memory hierarchy look like from the SRAM all the way up to data center worldwide level. I want to talk more about the thing you mentioned about look, Google is a company with like lots of code and lots of examples.

35:32If you just think about that one use case and what that implies, so you've got like the Google Monorepo. And if you maybe you figure out the long context thing you could put the whole thing in context or you find tune on it. Yeah, basically like, why why why why why hasn't this minority done and you know, because you can imagine like the amount of code that Google has proprietary access to just like me, even if you're just using it internally for it to be developers more efficient and productive. Oh, it'll be clear we have actually already done further training on a Gemini model on our internal code base for our internal developers.

36:11Yeah, but that's different than attending to all of it. Right. Because it sort of stirs together the code base into a bunch of parameters. And I think having it in context is makes things clearer. But even the sort of further trained model internally is incredibly useful like Sundar I think has said that 25 % of the characters that we're checking into our code base these days are generated by our AI based coding models with kind of human kind of. How do you imagine trying to do it? And the way in which you're enacting these models in Europe, what does that look like? Well, I mean, I assume the way we'll be we'll have these models a lot better and hopefully be able to be much much more productive.

37:07Yeah, I mean, I think one one of the. In addition to kind of researchy context like anytime you're seeing these models used, I think they're able to make softer developers more productive because they can kind of take. Which sort of a high level spec or in sentence description of what you want done and give a pretty approximate, you know, pretty reasonable first cut at that. And so from a research perspective, maybe you can say I'd really like you to explore, you know, this kind of idea like similar to the one in this paper, but maybe like let's try making it convolutional or something. Like that, if you could do that and have the system automatically sort of generate a bunch of experimental code and maybe you look at it and you're like, yeah, that looks good.

37:51Run that. Like that seems like a nice dream direction to go in and seems plausible in the next year or two years that you might make a lot of progress on that. And seems under hyped because you've got like it's you could have like literally millions of extra employees. And you can immediately check their output that employees can check either each other's output. They like immediately stream tokens. Yeah, sorry, I didn't mean not to hype it. I think it's super exciting. I just don't like to hype things that aren't done yet.

38:25Yeah, so let's. I do want to play with this idea more because you know it seems like it would be a deal. Like you have something like kind of like an autonomous software engineer, especially from the perspective of a researcher who's like, I want to expect build the system. Again, there was OK, so you'll literally would say, yeah, like somebody who has worked on developing transformative systems through your careers, did yet that instead of having to code something like whatever the today's equivalent of map reduces or TensorFlow is just like here. Here's how I want like distributed AI library to look like right enough for me.

39:03Could you do imagine you could be like 10X more productive, 100 X more productive. I was pretty impressed. I think it was on Reddit that I saw like we have a new experimental coding like model that's much better at coding and math and so on. And someone external tried it and they basically prompted it and said, I'd like you to implement a sequel processing database system with no external dependencies. And please, please do that and see. And from what the person said, it actually did a quite good job, like a generated a sequel parser and a tokenizer and you know a query planning system and some stored format for the data on on disk and actually was able to handle simple queries.

39:47So, you know, from that prompt, which is like, you know, a paragraph of text or something to get, you know, even a certain initial cut at that seems like a big boost in productivity for software developers. And I think you might end up with other other kinds of systems that maybe don't try to do that in a single, you know, in semi interactive respond in 42nd kind of thing, but might go off for 10 minutes and like might interrupt you after five minutes saying, I've done a lot of this, but now I need to, you know, get some input, you know, do you do care about handling video or just images or something.

40:26And that that seems like you'll need ways of managing the workflow if you have a lot of these kind of background activities happening. Yeah, actually, could you talk more about that? So what interface do you imagine we might need if we have, if you could literally have like millions of employees, you could spin up hundreds of thousands of employees, you could spin up on command who are able to type incredibly fast and who. So it's almost like you go from like 1930s, like trading of like tickets or something to now modern like, you know, change or something, you know, you need to better, you need some interface to keep track of all of the sets going on for the AIs to integrate into this big monitor repo and leverage their own like strengths for human's keep track of what's happening.

41:11What is it like to be a Jeffernome in three years working day to day? It might be kind of similar to what we have now because we already have sort of parallelization as a major issue because you know, we have like lots and lots of really, really brilliant machine learning researchers and we want them to work our work together and build AI. So actually the parallelization among people might be similar to parallelization among machines. But I think there definitely it should be good for things that require like a lot of exploration, you know, like come up with come up with the next breakthrough because, you know, if you have a brilliant idea that it's just certain to work, you know, on the ML domain, then, you know, it has a 2 % chance of working if you're brilliant and, you know, mostly these things fail.

42:14But if you try 100 things or 1000 things or a million things, then you might hit on something amazing. And we have we have plenty of compute like modern, you know, top labs these days have probably a million times as much compute as it took the train transformer. So yeah, actually, so that's a really interesting idea. If you have like suppose in the world today, there's like on the order of 10 ,000 AI researchers and this community coming up with a breakthrough. Probably more than that there were 15 ,000 in Europe, so 100 ,000 I don't know. Yeah, maybe. Sorry. No, no, it's good to have the correct order of attitude.

42:56And the odds that this community every year comes up with a breakthrough on the scale of a transformer is let's say 10%. Now, suppose this community is a thousand times bigger and it is in some sense, like this sort of parallel search of better techniques. Do we just like get like a break through every year or every day? Maybe sounds, sounds potentially good. But is that feel like what M .O. research is like is just if you have, if you are able to try all these experiments. It's a good question because we, you know, I don't know that folks have been, haven't been doing that as much. I mean, we definitely have lots of, lots of great ideas coming along.

43:40Everyone seems to want to run their experiment at maximum scale, but I think that's, you know, that's a human problem. Yeah, it's very helpful to have a one one thousand scale problem and then vet like a hundred thousand ideas on that and then scale up the ones that are this team promising. Yeah. A quick word from our sponsor scale AI publicly available data is running out. So major labs like meta and Google DeepMind and OpenAI all partner with scale to push the boundaries of what's possible through scale's data foundry. Major labs get access to high quality data to fuel post training, including advanced reasoning capabilities.

44:20As AI races forward, we must also strengthen human sovereignty. Scales research team SEAL provides practical AI safety frameworks, evaluates frontier AI system safety via public leaderboards and creates foundations for integrating advanced AI into society. Most recently in collaboration with the Center for AI Safety scale published Humanities last exam, a groundbreaking new AI benchmark for evaluating AI systems expert level knowledge and reasoning across a wide range of fields. If you're an AI researcher or engineer and you want to learn more about how scales data foundry and research team can help you go beyond the current frontier of capabilities, go to scale .com slash Dwarkech.

45:02Alright, back to Jeff and know him. So I think one thing the world might not be taking seriously. People are aware that it's exponentially harder to make like to do this skate like make a model that's 100x bigger is like 100x more compute right. So it's like people are aware that it's like an exponentially harder problem to go from Gemini 2 to 3 or so forth. But maybe people aren't aware of this other trend where Gemini 3 is coming up with all these different architectural ideas and trying them out and you see what works. And you're constantly coming up with these algorithmic progress that makes training the next one easier and easier.

45:37How far could you take that for you back, Lou? I mean, I think one one thing people should be aware of is the improvements from generation to generation of these models. Often are partially driven by hardware and larger scale, but equally and perhaps even more so driven by major algorithmic improvements and major changes in the model architecture and the training data mix and so on. And that really make the model better per flop that is applied to the model. So I think that's a good realization. And then I think if we have automated exploration of ideas, we'll be able to vet a lot more ideas and bring them into kind of the actual production training for next generations of these models.

46:19And that's going to be really helpful because that's sort of what we're currently doing with a lot of machine learning research, brilliant machine learning researchers is looking at lots of ideas. Because you know, winnowing ones that seem to work well at small scale, seeing if they work well at medium scale, bringing them into larger scale experiments and then like settling on like adding a whole bunch of new and interesting things to the to the final model recipe. And then I think if we can do that, you know, 100 times faster through those machine learning researchers just gently steering a more automated search process rather than sort of hand babysitting lots of experiments themselves.

46:58That's going to be really, really good. Yeah, the one thing it doesn't speed up is like experiments at the largest scale because you still end up doing like these n equals one experiments in there really just try to put a bunch of really brilliant people in the room and have them stare at the thing figure out why this is working. This is not working. Hardware is a good solution and better hardware. Yes, yes, we're counting on you. So, um, okay, and naively, I would say so there's a software. There's like algorithmic side improvement the future they ask and make. There's also the stuff you're working on on a full chip, I'll let you describe it.

47:40But if you get into a situation where just from a software level, you can be making better and better chips and a matter of weeks and months and better. You can presumably do that better. Basically, I'm wondering, how does this feedback loop not just end up in like a Gemini three takes a two years and Gemini four is like a six or the equivalent level jump is now six months. Then like the level five is like three months, then one month and you get to like superhuman intelligence. Pretty much more rapidly than you might naively think because of this software both on the hardware side and from the algorithmic side improvements.

48:22Yeah, I mean, I've been pretty excited lately about how could we dramatically speed up the chip design process? Yeah. Because as we were talking earlier, the current way in which you design a chip takes you roughly 18 months to go from we should build a chip to something that you then hand over to TSMC and then CSMC takes four months to to FabIt and then you get it back and you put it in your data centers. So that's a pretty lengthy cycle and the FabTime in there is a pretty small portion of it today. But if you could make that the dominant portion so that instead of taking you know 18 month, 12 to 18 months to design the chip, you could shrink and with you know 150 people.

49:07You could shrink that to you know a few people with a much more automated search process exploring the whole design space of chips and getting feedback from all aspects of the chip design process for the kind of choices that the system is trying to explore at the high level.

49:32Yeah. And that would be great because you can shrink that time, you can shrink the deployment time by kind of designing the hardware in the right way so that you just get the chips back and you just plug them in to some some system. And that will then I think enable a lot more specialization and it will enable a shorter time frame from the hardware design so that you don't have to look out quite as far into what kind of a logarithm to be interesting. Instead it's like you're looking at six to nine months from now what should it be rather than you know two to a half years and that would be pretty cool.

50:09I do think that that fabrication time is if that's in your inner loop of improvement, you're going to like how long is it the leading edge nodes. Unfortunately, you're taking longer longer because I have more metal layers than previous you know older nodes. So that that tends to make make it take anywhere from three to five months. Okay, but that's how long training runs take anyways right so you could potentially do both at the same time. Yeah, okay, so I guess like you can't get sooner than three to five months, but the idea that you could get like, but also yeah, you're like rapidly developing new algorithm ideas right that can move fast.

50:45That can run on like existing chips and explore lots of cool ideas. Yeah, so this isn't that like a situation in which you're like I think people sort of expect like there's going to be a sigmoid. Again, this is not a sure thing, but just like is this a possibility the idea that you have like sort of an explosion of capabilities very rapidly towards the tail end of human intelligence that you know gets like a smarter and smarter to more and more rapid rate. What possibly yeah, I mean, I think I like to think of it like this right like right now we have models that can take a pretty complicated problem and can break it down.

51:23You know internally in the model into a bunch of steps can sort of peep puzzle together the solutions for those steps and can often give you a solution to the entire problem that you're asking. But it you know isn't super reliable and it's good at breaking things down into you know five to 10 steps not 100 to 1000 steps. So if you could go from yeah 80 % of the time it can give you a perfect perfect answer or something that's 10 steps long to something that you know 90 % of the time can give you a perfect answer or something that's 100 to 1000 steps of sub sub problem long. That would be an amazing improvement in capability of these models and you know we're not there yet, but I think that's what we're aspirationally trying to get to areas.

52:09Yeah, we don't need new hardware for that. We but I mean we will take it. Yeah, exactly. I've never looked new hardware in the mouth. One of the like one of the big areas of improvement I think you know and then your future is is this entrance time compute like applying more compute you know at entrance time and I guess the way I've like to describe it is that. You know a like even some giant language model you know even if you're doing say a trillion operations per token which is you know more than more than most people are doing these days. You know operations cost something like 10 to the negative 18 dollars and so you're getting like a million tokens to the dollar right so I mean compare that to like a relatively cheap past time like you you go out and you buy a paper book and read it you're paying like 10 ,000 tokens to the dollar so it's so like talking to a language model could be like you know is like 100 times cheaper than reading a paper back so there is a huge amount of headroom there.

53:22To say okay if we can make this thing more expensive but but smarter because we're like to like 100 X cheaper than reading a paper back we're like 10 ,000 times cheaper than like talking to customer support agent we're like a million times more cheaper than then you know hiring a software engineer talking to your doctor or lawyer like can we add you know the yeah add computation and make it make it smarter so like I think a lot of. A lot of the takeoff that we're going that we're going to see in the very near future is of this form like we've we've been exploiting and improving pre training a lot in the past and post training and those things will continue to improve but like taking advantage of you know think harder at at inference time is going to just be an explosion.

54:17Yeah and an aspect of of inference time is I think you want the system to be actively exploring a bunch of different potential solutions you know maybe it does some searches on its own and get some information back and like consumes that information and figures out oh now I would really like to know more about this thing so now it kind of iteratively kind of explores how to best solve the high level problem you pose to the system and I think having a dial where you can make the model. I'll give you better answers with more inference time compute seems like we have a bunch of techniques now that seem like they can kind of kind of do that and the more you crank up the dial the more costs you in terms of compute but the better the answers get that that seems like a nice trade off to have because sometimes you want to think really hard because there's a super important problem sometimes you probably don't want to spend enormous amounts of compute to compute you know one plus what's the answer to one plus one.

55:17Maybe the system should be 100 comes up with like new actions of set the idea of the calculator to a lot of you know very large language model. Are there any impediments to taking inference time like having some way in which we can just linearly scale up inference time compute or is this basically a problem that sort of solved and we know how to sort of like 100x compute 1000x compute and get correspondingly better results. Well we're working out the algorithms as we speak so I believe you know we'll see we'll see better and better solutions to this as these many more than 10 ,000 researchers are hacking at any of Google.

56:01I mean I think we do see some examples in our own sort of experimental work of things where you if you apply more inference time compute the answers are better than if you just apply you know x you know for lie 10x you can get better answers than x amount of computer. And that seems useful and important but I think what we would like is when you apply 10x to get you know even a bigger improvement in the quality of the answers that we're getting today and so that's about you know designing your algorithms trying to approaches you know figuring out how best to spend that 10x instead of x to improve things.

56:40Is it look more like search or does it look more like just keep going in the linear direction for a longer time. I mean I think search is I really like rich sentence paper that he wrote about the bitter lesson and the bitter lesson effectively is this nice one page paper but the the essence of it is. You can try lots of approaches but the the two techniques that are incredibly effective are learning and search. And you can apply and scale those algorithmic or you know computationally and you often will then get better results than any other kind of approach you can apply to to pretty broad variety problems.

57:17Yeah and so I think search has got to be part of the solution to spending more in front of you want to maybe explore a few different ways of solving this problem and like you know that one didn't work this one work better so I'm going to explore that a bit more. How does this change your plans for future data center planning and so forth where if you know can this kind of search be done asynchronously does it have to be online offline how does that change. How how big of a campus you need in those kinds of considerations. I mean I think one general trend is it's clear that inference time compute you know you have a model that's pretty much already trained and you want to do inference on it is going to be a growing and important class of computation that maybe you want to specialize hardware more around that.

58:09You know actually the first TPU was specialized for inference and wasn't really designed for training and then subsequent TPUs were really designed more around training and also for inference. But it may be that you know when you have something where you really want to crank up the amount of compute use it inference time that even more specialized solutions won't make a lot of sense. Does that mean you're going to accommodate more asynchronous training training or inference. Or just you can have the different data centers don't need to talk to each other you can just like have them do a bunch of.

58:44Yeah I mean I think I like to think of it as is the inference that you're trying to do latency sensitive like the user actively waiting for it or is it kind of a background thing and maybe that's. I have some inference tasks that I'm trying to to run over a whole batch of data but it's not for a particular user is just I want to run inference on it and extract some information and then there's probably a bunch of things that we don't really have very much of right now. But you're seeing inklings of it in our deep research like tool that we just really I forget exactly when like a week ago. Where you can give it a pretty complicated high level task like hey can you go off and research the history of renewable energy and all the trends and costs for for wind and solar and other kinds of techniques and put it in a table and give me a full eight page report and it will come back with an eight page report with like 50 entries in the bibliography is pretty pretty remarkable.

59:40But you're not actively waiting for that and for one second it takes like you know a minute or two day go go do that. Yeah, and I think there's going to be a fair bit of that kind of compute and that's the kind of thing where you have some UI questions around okay if you're going to have a user with 20 of these kind of asynchronous tasks in the background happening and maybe each one of them needs to like get from our information from the user like I found your flights to Berlin but there's no non stop ones is you are you okay with a you know a non stop one. How does that flow work when you kind of need a bit more information and then you want to put it back in the background for it to continue doing you know finding the hotels in Berlin or whatever.

1:00:23I think it's going to be pretty interesting and inference will be useful. I mean there's also a compute efficiency thing in inference that you don't have in training and that you know in general transformers can use the sequence length as a batch during training but they can't really in inference because you're when you're generating one token of time so so they're there there may be different hardware and inference algorithms that that we design for the purposes of being efficient to the inference. Yeah like as a good example of an algorithmic improvement is like the use of drafter models so you have like a really small language model that you do one T token at a time when you're decoding and predict like four tokens and then you give that to the big model and you say okay here's the four tokens the little model came up with check which ones you agree with and if you agree with the first three then you just advance and then you've basically been able to do a four four four.

1:01:26So you're going to have four token with parallel computation instead of a one token with thing in the big model and so those are the kinds of things that people are looking at to improve inference efficiency so you're not don't have this single token decode bottleneck right basically the big models being used as a verify or it's a person. Yeah that's a generator and verification you can be hello how are you that sounds great to me I'm going to like advance past that so a big discussion has been about. You know you're already tapping out like nuclear power plants in terms of delivering power into one single campus and so do we have to like have just like even like two gigawatts in one place five gigawatts in one place or can it be more distributed and still be able to train a model.

1:02:15Does this new regime of inference scaling make different considerations they're plausible or how are you thinking about multi data center training now. I mean we're already doing it so we're pro multi data center training I think I think in the Gemini 1 .5 tech report we said we use multiple metro areas and trained with some of the computer in each place and then a you know pretty long latency but high bandwidth connection between those data centers and that works fine. Yeah it's great actually training is kind of interesting because each step in a training process is you know usually for a large model is a few seconds or something at least so the latency of it being you know 50 milliseconds away doesn't matter that much.

1:02:59Just the bandwidth you know yeah just bandwidth as long as you can sync you know sync all of the parameters of the model across the different data centers and then accumulate all the gradient so it's in the time it takes to do one step your your pretty good. And then we have a bunch of work on in you know in even early brain days when we were using CPU machines and they were really slow so we needed to do asynchronous training to help scale where each copy of the model would kind of do some local. Computation and then send gradient updates to a centralized system and then apply them asynchronously and another copy of the model would be doing the same thing you know it makes your your model parameters kind of wiggle around a bit and it makes people uncomfortable the theoretical guarantees but it actually seems to work in practice.

1:03:48It's not worth it to work. It was so pleasant to go from a sync to sync because because your experiments are now replic replicable like rather than like every like your result depends on like whether there was like a web crawler running on the same machine. It's like what if you're confused so I am so much happier running on my GPU. I love asynchronous just like to use two iPhones and Xbox or whatever. Yeah, what if we could give you asynchronous but replicable results. So one way to do that is you effectively record the sequence of operations so like which gradient update happened and when and on which batch of data you don't necessarily record the actual gradient update in a log or something but you could replay that log operations.

1:04:41So that you get repeatability. Then I think you'd be happy. At least you could debug what happened. Yeah, but you wouldn't be able to like compare to necessarily two training runs because I made one change in the hyperparameter but also like I had like a like. That crawler.

1:05:05And there were like a lot of people screaming the Super Bowl at the same time. I mean, the thing that led us to purple thing that let us go from asynchronous training on CPUs to fully synchronous training is the fact that we have these super fast GPU hardware chips and then pods which have incredible amounts of bandwidth between the chips and a pod. And then scaling beyond that we have like really good data center networks and even cross metro area networks that enable us to scale to you know many, many pods in multiple metro areas for our largest training runs. And we can do that fully synchronously as known said as long as the gradient accumulation and communication of the parameters across metro areas happens, you know, fast enough relative to the step time your golden you don't really care.

1:05:54But I think as you scale up, there may be push to have a bit more asynchronous in our systems than we have now because like we can make it work. I've been you know our ML researchers have been really happy how far we've been able to push synchronous training because it is easier mental model to understand. You know, you just have your algorithm sort of fighting your rather than the synchrony and the algorithm kind of battling you. As you scale up, there are more things fighting you, you know, like there's yeah, I mean that the probably right that's the problem with the you know with scaling that you don't actually always know what it is.

1:06:32That's fighting you is it you know the fact that you've pushed like quantization a little too far in some place or another or is it your data or is it maybe it's your adversarial machine M you QQ 17 that is like. So setting the seventh bit of your exponent all your radians are something right and all of these things just make the model slightly worse so you don't even know that the thing is going on. So yeah, that's actually a bit of a problem with no that says there's so tolerance of noise you can have things set up kind of wrong in a lot of ways and they just kind of figure out ways to work around that or learn and despite you could have bugs in your code.

1:07:12Most of the time that does nothing some of the time it makes your model worse some of the time it makes your model better and discovered something because you never tried this bug it scale before because you didn't have it didn't have the budget for it. What practically does it look like actually to debug or decode what the like you've got these things some of the which are making more better some which are making it worse. You what you going to work tomorrow you're like all right what's going on here how do you figure out what the most alien inputs are right I mean well it's small scale you do lots of experiments so so I mean there's I think one part of of the research that involves OK I want to like invent these improvements or breakthroughs kind of in the isolation in which case you want a nice simple code base that you can fork and hack and like some baselines.

1:08:07And you know my dream is I wake up in the morning come up with come up with an idea hack it up in the day run some experiments get get some initial results in the day like OK this look this looks promising these things work these things work didn't work and I think that that is that is very achievable because OK that's all scale at small scale as long as you keep your you know keep a nice experimental code base and the experiment takes an hour to run or two hours or something not not two weeks it's great it's great so so there's that part of the research and then there's some amount of scaling up and then you have the part which is like integrating where you want to stack all the improvements on top of each other and see if they work at large scale and see if they work all in conjunction right how the track right you think maybe they're independent but actually maybe there's some funny interaction between you know improving the the way in which we handle video data input and the way in which we you know update the model parameters or in that interacts more for video data than some other thing you know there's all kinds of interactions that can happen that you maybe don't anticipate and so you want to run these experiments where you're then putting a bunch of things together and then periodically making sure that all the things you think are good are good together and if not understanding why they're not playing nicely.

1:09:35Two questions one how often does it end up being the case that things don't stack up well together is it like a rare thing or does it happen all the time. It happens happens. I think most things you don't even try to stack because they they you know the initial experiment didn't work that well or it showed results that aren't that promising relative to the baseline and then you sort of take those things and you try to scale them up individually and then you're like oh yeah these ones seem really promising so I'm going to now include them in something that I'm going to now bundle together and try to advance you know what and combine with other things that seem promising and then you run the experiments and then you're like oh well they didn't really work that well like let's try to debug why.

1:10:22And then there are tradeoffs because you want to keep your like integrated system you know it's clean as you can because you know conflux code base code base and algorithmically conflux the you know conflux the hurts conflux the makes things slower introduces more risk and then you know at the same time you want to you want it to be as good as possible and of course every individual researcher wants wants his inventions to go into it. So that there are that there are definitely challenges there but we've been working together quite well. My sponsors Jane Street invented a card game called FIGGI in order to teach their new traders the basics of markets and trading I'm a poker fan and I'd say that FIGGI is like poker in the sense that there's hidden information but it's much more intense in social in poker you're usually just sitting around waiting for your turn whereas in FIGGI you spend the whole time just shouting bids and ask the other players the game is not of such that there's a winner in the end of course but during each turn you are incentivized to find mutually beneficial trades with the other players and in fact that's the main skill that the game rewards.

1:11:33FIGGI simulates the most exciting parts of trading Jane Streeters enjoyed so much that they hold an inner office FIGGI championship every single year. You can play yourself by downloading it on the app store or you can find it on desktop at FIGGI .com. All right, back to Jeff and Noam. Okay, so then going back to the whole dynamic of you find better and better algorithm thinking improvements and the models get better and better over time even you take the hard part out of it. Should the world be thinking more about and should you guys be thinking more about this there's one world where you just like a eyes a thing that takes like two decades to slowly get better over time and you can sort of like refine things over you know if like you kind of mess something up you fix it and it's like not that big a deal right it's like not that much better than the previous version you released.

1:12:25There's another world where you have this big feedback loop which means that the year that the two years between Gemini 4 and Gemini 5 are the most important years in human history because you go from a pretty good ML researcher to super human intelligence because of this feedback loop to the extent that you think that second world is plausible. How does that change how you sort of approach these greater and greater levels of intelligence. I've stopped cleaning my garage because I'm waiting for the robots. So probably I'm more in the second camp of what we're going to see a lot of acceleration.

1:13:05Yeah, I mean I think it's super important to understand what's going on and what the trends are and I think right now the trends are the models are getting substantially better generation over generation. And I don't see that slowing down in the next few generations probably so that means the models say two to three generations from now are going to be capable of you know let's go back to the example of breaking down a simple task into 10 sub pieces and doing it 80 % of the time to something that can break down a task a very high level task into 100 or 1000 pieces and get that right 90 % of the time right that's a major major step up in what the models are capable of.

1:13:45So I think it's important for people to understand you know what's what is happening in the progress in the field and then those models are going to be applied in a bunch of different domains and I think it's really good to make sure that we as society get the maximum benefits from what these models can do to improve things in you know I'm super excited about areas like education and health care. So I think we're making information accessible to all people but we also realize that they could be used for misinformation they could be used for you know automated hacking of computer systems and we want to sort of put as many safeguards and mitigations and understand the capabilities of the models in place as we can and that's kind of you know I think Google as a whole has a really good view to how we should approach this you know our response is really good.

1:14:41So I think Google AI principles actually are a pretty nice framework for how to think about trade offs of making you know better and better AI systems available in different contexts and settings while also sort of making sure that we're doing the right thing in terms of you know making sure they're safe and you know not saying toxic things and things like that. I guess the thing that stands out to me if you were like zooming out and looking at like this rear of human history if we're in the world where like look maybe if you do post -training on Gemini 3 badly it can do some misinformation but then you like fix the post -raining and like it's going to stop doing them is it's a bad mistake but it's a fixable mistake right.

1:15:21Whereas if if you have this feedback loop dynamic which is a possibility right then the sort of like mistake of like the thing that catapult this intelligence explosion is like misaligned is is like not trying to write the code you think it's trying to write and optimizing for some other objective. And on the other end of this very rapid process that lasts a couple of years maybe less you have things that are approaching Jeff Dean or beyond the level or known as your beyond level and then you have like millions of copies of Jeff Dean level programmers and anyways that seems like a harder to recover mistake and that seems like a much more salient like you really got to make sure we're going into the explosion.

1:16:10As these as these systems do get more powerful you you have you know you got to be more and more more and more careful. I mean one thing I would say is there's like the extreme views on either end there's like oh my goodness these systems are going to like be so much better than humans at all things and we're going to be kind of overwhelmed and then there's the like these systems are going to be amazing and we don't have to worry about them at all. I think I'm somewhere in the middle and I've been a I'm a co -author on a paper called shaping AI which is you know those two extreme views often kind of view our role as kind of laissez faire like we're just going to have the AI developed in the path that it that it takes.

1:16:51And I think there's actually a really good argument to be made that what we're going to do is try to shape and steer the way in which AI is deployed in the world so that it is you know maximally beneficial in the areas that we want to capture and benefit from in you know education you know some of the areas I mentioned healthcare. And steer it as much as we can away maybe with policy related things maybe with you know technical measures and safeguards away from you know the computer will you know take over and and have unlimited control of what it can do. So I think that's an engineering problem is how do you engineer safe systems I think it's kind of the modern equivalent of what we've done in kind of older style software development like if you look at you know airplane software development that has a pretty good record of how do you rigorously developed safe and secure systems for for doing a pretty pretty risk risky task.

1:17:58The difficulty there is that there's not some feedback loop for the 737 you like put it in a box with a bunch of compute for a couple of years and it comes out with like the version 1000 I think the good news. The good news is that analyzing text seems to be easier than generating text so I believe that the sort of ability of language models to to actually analyze. Language model output and you know and figure out what is what is problematic or dangerous you know will will actually be the solution to to a lot of these control issues we are definitely definitely working on this stuff we've got a bunch of brilliant folks at Google.

1:18:53We're working on this now and you know I think it's just going to be more and more important both from you know both from a you know do something good for people standpoint but you know also from a business standpoint that you know you are a lot of the time like limited in you know limited in what you can deploy based on you know based on keeping keeping safe and it's. So it becomes very very important to be really really good at that yeah I obviously know you guys take the potential benefits and cost you're seriously and you guys get credit for but not enough I think there's like there's so many different applications that you have put out for using these models to make the different areas you talked about better.

1:19:44But I do think that there what again if you have a situation where plausibly there's some feedback loop process on the other end you have like a model that is as good as no I'm sure you're as good as Jeff Dean if like if there's an evil version of you running around and suppose there's like a million of them yes I think that's like really really bad yeah that's not that could be like much much worse than any other risk maybe sort of like nuclear war or something like just think about like a million evil Jeff Deans are. Or do we get the training. Yeah but to the extent that you think that's like a plausible output of some quick feedback loop process what is your plan of like okay we've got Gemini three or Gemini four and we think it's like helping us do better job of training future versions is writing a bunch of the training code for us from this point forward we just kind of like look over it verify it even the verifiers you talked about of looking at the output of these models.

1:20:43Will eventually be trained by or you know a lot of the code will be written by the AIs you make. You know like what do you want to know for sure before we like have the Gemini four help us with the research we really want to make sure we want to run this test on it before we like let it write our I quote for us. I mean I think having the system explore algorithmic research ideas seems like something where there's still a human in charge of that that gets exploring space and then it's going to like get a bunch of results and we're going to make a decision like are we going to incorporate this particular you know learning algorithm or change the system.

1:21:21In into kind of the core code base and so I think you can put in safeguards like that that enable the system to get the benefits of the system that can sort of improve or kind of self improve with human oversight without necessarily letting the system go full on self improving without any. Any notion of a person looking at what is doing right that's the kind of engineering safeguards i'm talking about where you want to be kind of looking at the characteristics of the systems you're deploying not deploy ones that are harmful by some measures and some ways in you have an understanding what its capabilities are and what it's like we do in certain scenarios.

1:22:07So you know I think it's not an easy problem by any means but I do think it is possible to make these these systems safe yeah I mean I think I think we are also going to use these systems a lot to check themselves check other systems you know it's I mean even as a human it's it is easier to recognize something than to generate it so. So one one thing I would say is if you expose the models capabilities through an API or through a user interface that people interact with you know I think then you have a level of control to understand how is it being used and and sort of put some boundaries on what it can do.

1:22:51And that I think is one of the tools in the arsenal of like how do you how do you make sure that what it's going to do is is sort of acceptable by some set of standards you've set out in your mind. Yeah I mean I think our goal is to empower people but you know so for the most part you know we should be mostly letting letting people do things with these systems that make sense and you know closing office few parts of the space as we can but you know yeah if you if you let somebody take it. So I think that's a great thing to make your thing and create a million evil software engineers that that doesn't empower people because they're going to they're going to hurt others with a million evil software engineers so I am against that.

1:23:33I'll go on. All right let's talk a little few more fun topics. Yeah I'll make it a little like I heard. Over the last 25 years what was the most fun time what period of time we do you have the most nostalgia over. I mean I think the early sort of four or five years at Google when I was sort of one of a handful of people working on search and crawling in search and indexing systems and our traffic was growing tremendously fast and we were trying to expand our index size and make it so we updated it you know every minute instead of every you know month or two months. Something went wrong and seeing kind of the growth and usage of our systems was really just personally satisfying you know building something that is used by you know today two billion people a day I think is is pretty pretty incredible.

1:24:23But I would also say equally exciting is sort of working with people in the Gemini team today and I think the progress we've been making in what these models can do over the last you know year and a half or whatever is is really fun. People are really dedicated really excited about what we're doing I think the models are getting better and better at you know pretty complex tasks like if you showed someone using a computer 20 years ago what these models are capable of. They wouldn't believe it right and even five years ago they might not believe it and that's that's pretty pretty satisfying and I think we'll see a similar you know growth in you said to these models and impact in the world yeah I'm with you.

1:25:04Early days were super fun you know I mean part of that is just like knowing everybody and you know in the social aspect and the fact that you're just building something that that that millions and millions of people are using and you know same thing today we got that that whole nice micro kitchen area where you get like lots of people hanging out with you know so I love being in person it's work with a bunch of great people and build something that's helping me. Millions to billions of people like yeah what could be better what was this micro kitchen. We have a micro kitchen area and the building we both sit in it's the new greedy so named gradient canopy.

1:25:45It used to be named Charleston East and we decided we needed a more exciting name because it's a lot of like machine learning researchers and I research happening in there. And there's a micro kitchen area with that we've set up with you know normally it's just like a espresso machine and a bunch of snacks but this particular one has a bunch of space in it so we've set up like maybe 50 desks in there and so people are just hanging out in there. You know it's a little noisy because people are always like grinding beans and espresso but you know you also get a lot of like face to face ideas of connections like oh I've tried that like did you try to think about trying this in your idea or you know oh we're going to launch this thing next week like how's the load test looking.

1:26:30There's just like lots of feedback that happens and then we have our Gemini chat rooms for people who are not in that micro kitchen you know we have a team all over the world. And you know there's probably 120 chat rooms I mean related to Gemini or things and you know this particular very focused topic we have like seven people working on this and there's like exciting results being shared by the London colleagues and when you wake up you see like what's happening in there or it's a big group of like people focused on data and there's all kinds of issues you know happening in there it's just fun.

1:27:04What I found remarkable about some of the calls you guys have made is you're anticipating a level of demand for compute which at the time wasn't obvious or evident. TPUs being a famous example of this or the first TPU being example this. That thinking you had in I guess 2013 or earlier if you're if you think about it that way today and you do an estimate of look we're going to have these models that are going to be a backbone of our services and we're going to be doing constantly inference for them we're going to be trading future versions. And you think about the amount of compute will need by 2030 to accommodate all these use cases.

1:27:42Where does the Fermi estimate get you. Yeah I mean I think you're going to want a lot of inference compute is the rough highest level view of these capable models because if one of the techniques for improving their quality is scaling up the amount of inference compute you use then all of a sudden what's currently like one request to generate some tokens now becomes 50 or 100 or a thousand times as computationally intensive even though it's producing the same amount of data. And you're also going to then see tremendous scaling up of the uses of these services as you know not everyone of the world has discovered these chat based conversational interfaces where you can get them to do all kinds of amazing things you know probably 10 % of the computer users in the world have discovered that today or 20 % as they.

1:28:38That pushes towards 100 % and people may keviger use of it you know that's going to be another you know order of magnitude or two of scaling and so you're now going to have you know to order the magnitude from that to order the magnitude from that models are probably going to be bigger you get another So you want extremely efficient hardware for inference for models you care about in in flops the global total global inference. 2030. I think just more is always going to be better like like if you just kind of think about okay like what fraction of world GDP will be you know well people decide to spend on on AI at that point and then like okay what do the AI systems look like well maybe it's some sort of personal assistant like thing that is in your glasses and can see everything around you and has access to all your digital information and the world's digital information and like maybe it's like your Joe Biden and you have the European companies in the cabinet that can advise you about anything in real time and solve problems for you and give you helpful pointers or you could talk to it and you know it wants to analyze like anything that it sees around you for any potential useful impact that it has on you so I mean I can imagine okay and then say it's like your okay your personal assistant or your personal cabinet or something and that every time you spend 2x is much money on compute the thing gets like 510 IQ points smarter or something like that and okay do you would you rather spend like $10 a day and have an assistant or $20 a day and have a smarter assistant you know and not only is it an assistant in life but an assistant in getting your job done better because now it makes you from a 10x engineer to a hundred X or 10 million X engineer I mean okay so let's see from first principles right so so people are going to want to want to spend some some fraction of world GDP on this thing the world GDP is almost certainly going to go way way up to like orders of magnitude higher than it is today due to the fact that we have all of these artificial engineers like working on improving things probably we will we will have solved unlimited energy and and like carbon issues by that point so we should be able to have lots of energy we should be able to have millions to billions of robots like building us data centers like let's see what's it like the sun is what 10 to the 26 lots or something like that you know I mean I'm guessing that the that the amount of compute at the you know being used for AI to help each person will be astronomical I mean I would add on to that I'm not sure I agree completely but it's a pretty interesting thought experiment to go in that direction and even if you get part way there it's definitely going to be a lot of compute and this is why it's super important to have as cheap and hardware platform for using these models and applying them to to problems that I'm described so that you can then make it accessible to everyone in in some form and have you know as low a cost for access to these capabilities as you possibly can and I think that's achievable by focusing on you know hardware and model code design kinds of things and we should be able to make these things much much more efficient than they are today is the is Google's data center build up plan over the next few years aggressive enough given this increase in demand or expecting I'm not going to comment on our future capital spending because our our CEO and CFO before I go to the late but I I will say you know you can look at our past capital expenditure is over the last three years and see that we're we're definitely investing in this area because we think it's important and that we're you know we're continuing to build new and interesting innovative hardware that we think really helps us have an edge in deploying the systems to more and more people both training and also how do we make them usable by people for interest.

1:33:31One thing I've heard you talk a lot about is continual learning the idea that you could just have a model which improves over time rather than having to start from scratch. Is there any fundamental impediment to that because theoretically you should just be able to keep fine tuning a model or what is that future look like to you? Yeah, I've been thinking about this more and more and I've been a big fan of models that are sparse because I think you want different parts of the model to be good at different things. Yeah, and we have you know our Gemini 1 .5 pro model and other models are mixture of expert style models where you now have parts of the model that are activated for for some token and parts that are not activated at all because you decided this is a math oriented thing and this parts good at math and this parts good at like understanding cat images.

1:34:24So that gives you this ability to have a much more capable model that's still quite efficient at inference time because it has very large capacity but you activate a small part of it. But I think the current problem well one limitation of what we're doing today is it's still a very regular structure where each of the experts is kind of the same size. You know they the paths kind of merge back together very fast. You don't sort of go off and sort of have lots of different branches for Matthew things that don't merge back together with the kind of cat image thing. And I think we should probably have a more organic structure in these things.

1:35:05I also would like it if the pieces of those model of the model could be developed a little bit independently. Like right now I think we have this issue where we're going to train a model so we do a bunch of preparation work on deciding the most awesome algorithms we can come up with and the most awesome data mix we can come up with but there's always trade offs there like we love to include more multi -lingual data but that might come at the expense of including less coding data and so the models less good at coding but better multi -lingual or vice versa. And I think it would be really great if we could have like a small set of people who care about a particular subset of languages go off and create really good training data train you know a modular piece of a model that we can then hook up to a larger model that improves its capability in say Southeast Asian languages or in you know reasoning about a Haskell code or something.

1:36:06And then you then also have a nice software engineering benefit where you've decomposed the problem of it compared to what we do today which is we have this kind of a whole bunch of people working but then we have this kind of monolithic process of starting to do pre training on this model and if we can do that you know you can have 100 teams around Google you can have people all around the world working to improve you know languages they care about or particular problems they care about and all collectively work on improving the model and that's a kind of a lot of things that we can do. And that's kind of a form of continual learning that would be so nice you could just like glue models together rip out pieces of models and stuff like that kind of thing or like you just touch a fire who's new suck all the information out of this model yeah shoving into another model there is I mean the counter they're selling interest there is sort of science in terms of like okay we're still in the period of rapid progress yeah so if you want to do sort of controlled experiments and okay you know I want to compare this thing to that thing because that is helping us figure out okay what you want to build so for thing you know in that interest it's often best to just start from scratch so you can compare one complete training run to another you know to another complete training run sort of at the practical level because it kind of helps us figure out what to you know what to build in the future and it's less exciting but but does lead to rapid progress yeah I think there may be ways to get a lot of the benefits of that with kind of a version system of modularity like I have a frozen version of my model and then I include a different variant of some particular module and I want to compare its performance or training a bit more yeah and then I compare it to the baseline of of this thing with you know now version you know and prime of this particular module that does has school interpretation actually that couldn't lead to faster research progress right you've got some system yeah and you do something to improve it and if that thing you're doing to improve it is relatively cheap compared to training the system from scratch then it could actually make yeah it could actually make research much much cheaper and faster yeah so okay and also more more parallelizable I think yeah okay because you across people okay let's figure it out and do that next yeah so this is the idea that is sort of casually laid out there is actually it would be a big regime shift yeah this is the way things are headed this is like this is a sort of like very interesting prediction about you just have this like blob where things are getting piped line back and forth and if you want to make something better you can do like a sort of surgical incision almost right or grow the model add another little bit of it here yeah I've been sort of sketching out this vision for a while in the sort of pathways yeah under pathways name you've been building the building infrastructure for it so a lot of what pathways the system can support is this kind of twisty weird model with like asynchronous updates to different pieces yeah we should put that in front we're using pathways to train our Gemini models we're using or not making use of some of its capabilities yet but yeah maybe we should maybe there have been times like you know like the the the way the TPU pods were set up I don't know who who did that but they did a pretty pretty brilliant job you know the low level software stack and the hardware stack that okay you've got your you know you've got your nice regular high performance hardware you've got these great Torres -shaped interconnect and then you've got the the right low level collectives you know the all reduces etc which I guess came from super computing but it turned out to be kind of just the right thing to build to build distributed deep learning on top of the okay so a couple of questions one suppose you do figure suppose no makes another breakthrough and that we've got a better architecture would you just take each compartment and distill it into this better architecture and that's how you keep some proofing over time yeah I mean I do think distillation is a really useful tool because it enables you to kind of transform a model in its current model architecture form into a different form you know often you use it to take a really capable but kind of large and I wield the model and distill it into a smaller one that maybe you want to serve with really good fast latency inference characteristics but I think you can also view this as something that's happening at the modularity at the module level like maybe there'd be a continual process where you have each module and it has a few different representations of itself it has a really big one it's got a much smaller one that is continually distilling into this the small version and then the small version once that's finished then you sort of delete the big one and you add a bunch more parameter capacity and now start to learn all the things that the distilled small one doesn't know by training it on more data and then you kind of repeat that process and if you have that kind of running a thousand different places in your modular model in the background that seems like it would work reasonably well.

1:41:38This is really what you're doing in front of scaling like the router decides how much do you want the big one. Yeah you know multiple versions and like you know this is an easy math problem so I'm in a router to the really tiny math distilled thing and oh this one's really hard so. At least from public research it seems like it's often hard to decode what each expert is doing in mixture of expert type models if you have something like this how would you enforce the kind of modularity that would be visible and understandable to us. Actually in the past I found experts to be relatively easy to understand I mean I don't know the first mixture of experts paper you could just like look at the actual.

1:42:18I'm only the mentor make sure. Yeah like yeah you could just see okay like this expert like we did like you know a thousand two thousand experts okay and this expert like all of the was getting words referring to cylindrical object and you know like it's one super good at dates yeah yeah talk about time was actually pretty pretty easy to do but I mean like not that you would need that human understanding to like figure out how to like work the thing and it's like that. It's a really good thing to do is that you can just run time because you you just have like some sort of learned router that's looking at the example and I mean one thing I would say is like there is a bunch of work on interpretability of models and what is they doing inside and sort of expert level interpretability is a sub problem of that that broader area.

1:43:08I really like some of the work that my former intern Chris Ola and others did it and Thropike where they could kind of they trained a very sparse auto encoder and we're able to deduce you know what characteristics does some particular neuron and a large language about so they found like a golden gate bridge neuron that's activated when you're talking about the golden gate bridge. And I think you know you could do that at the expert level you could do that at a variety of different levels and get pretty interpretable results and is little unclear if you necessarily need that if the model is just really good at stuff you know we don't necessarily care what every neuron in the Gemini model is doing as long as the collective output and characteristics of the overall system are good.

1:43:54You know that's one of the beauties of deep learning is you don't need to understand or hand engineer every last feature. I mean there's a really interesting implications of this that we could just keep I could just keep asking you about this one implication is currently if you have a model that has some tens or hundreds of billions of parameters you can serve serve it on like a handful of GPUs in this system where any one query might only wake it's making it a little bit more interesting. So it gets way through a small fraction of the total parameters but you need the whole thing sort of loaded into memory the specific kind of infrastructure that Google has invested in with these TPUs that exist in pods of hundreds or thousands would be like immensely valuable right.

1:44:40I mean for any sort of even existing mixtures of experts you you want the whole thing in memory. Yeah. I mean basically if you are I guess there's kind of this misconception running around with like mixture of experts that okay the benefit is that you know the you don't even have to like go through those weights in the model you know some expert is unused. It doesn't mean that you don't have to retrieve that memory because really in order to be efficient you're serving at very large batch sizes. So independent request of independent right independent request so it's not it's not really the case that okay at this step you're either looking at this expert or you're not looking at this expert because if if that were the case then when you did look at the expert you would be running it at batch size one which is like massive.

1:45:36So it's not necessarily inefficient like you've got like modern hardware right the operational intensities are whatever hundreds or you know so so so that's not what's happening it's that you are looking at all the experts but you only have to send the small fraction of the batch through each one right but you still have a smaller batch at each expert that that goes through and in order to get kind of reasonable balance. Like one of the things that the current models typically do is they have all the experts be roughly the same compute cost and then you run roughly the same size batches through them in order to sort of propagate the very large batch you're doing it in front time.

1:46:18And you can have good efficiency but I think you know you often in the future might want experts that vary in computational cost by factors of a hundred or a thousand or maybe paths that go for many layers on one one case and you know a single layer or even a skip connection in the other case. And there I think you're going to want very large batches still but you're going to want to kind of push things through the model a little bit asynchronously for at inference time which is a little easier than the training time. And you know that's part of kind of one of the things that Pathways was designed to support is you know you have these components and the components can be variable cost and you kind of can say for this particular example I want to go through this subset of the model.

1:47:08And for this example I want to go through this subset model and have them kind of the system kind of orchestrate orchestrate that. It also would mean that it would take companies of a certain size and sophistication to be able to like right now you know anybody can train a sufficiently small enough model. But if we if it ends up being the case that this is the best way to train future models then you would need a company that can basically have a data center size data center serving a single quote unquote blob or you know model so it would be it would be interesting change in paradigms in that way as well.

1:47:49You definitely want to have is at least enough enough hbm to put to put your whole model so depending on the size of your model most likely that's how much that you know that's that's how much hbm you'd want to have the minimum I mean yeah. But it also means I think you don't necessarily need to grow your entire model footprint to be the size of the data center you might want it to be a bit below that and then have you know potentially many replicated copies of one particular expert that is being used a lot so that you get better load balance. Right so like this one's being used a lot because we get a lot of math questions and this one on you know maybe it's an expert on Tahitian dance and it is called on really rarely that one maybe you've been paid to D -Ram rather than putting it in hbm but you want the system to kind of figure all the stuff out based on load characteristics.

1:48:47How right now language models obviously like you put in language you get language out obviously it's multimodal but you could imagine the pathways blog post talks about like every sort of like so many different use cases that are not obviously of this kind of autoregressive nature going through the same model. So could you imagine like basically Google as a company the product is like Google search goes through this Google images goes through this Gmail goes through it just like the server the entire server is just this huge mixed or experts specialized. I mean you're you're starting to see some of this by having a lot of uses of Gemini models across Google that are not necessarily you know fine tune they're just sort of you know given instructions for this particular use case in this feature in this product setting.

1:49:41So I definitely see a lot more sharing of what the underlying models are capable of across more and more services. You know I do think that's a pretty interesting direction to go for sure. I feel like you're listening might not sort of register how. Yeah like how interesting how interestingly prediction this is about where it's like sort of like getting like no monopod because in 2018 and being like yeah so I think like you know language models of the thing it's like. This is where things go this is actually yeah that's incredibly interesting. Yeah and I think you might see that might be a big base model and then you might want customized versions of that model with different modules that are added onto it for different settings that maybe have access restrictions like maybe we have an internal one for Google use for Google employees that we've trained some modules on internal data and we don't allow anyone else to use those modules but.

1:50:37We can make use of it and maybe other companies you add on other modules that are useful for that company setting and serve it in our cloud APIs what is a bottleneck to making this sort of system viable is it is it like systems engineering is it ML is it. I mean it's a pretty different way of operating than our current Gemini development so I think you know we will. We will explore these kinds of areas and I think makes and progress on them but we need to sort of really see evidence that it's the right way you know that it has a lot of benefits some of those benefits may be improved quality some may be.

1:51:17Sort of less concretely measurable like this ability to have lots of parallel development of different modules and I think that but that's still a pretty exciting. I think that would enable us to make faster progress on improving the models capabilities for lots of different distinct areas I mean that even the data control modularity stuff seems like really cool because then you could have like the piece of the model that's just trained for me. It's good news like a personal module for you would be useful another thing might be you can use certain data in some settings but not another settings and you know maybe we have some some YouTube data that's only usable in a YouTube product surface but not another settings that we can have a module that is trained on that data for that particular purpose.

1:52:05We are going to need like a million automated researchers to invent all of this stuff. Yeah. It's going to be great. Well the thing itself you know it's like you build the blob and it like tells you how to make the blob better and blob 2 .0. Or maybe they're not even version it's just like an incrementally growing blob. Yeah. Okay Jeff. Um, motivate for me big picture. Why is this a good idea? Why is this the next direction? Yeah I mean I guess this kind of like notion of an organic like kind of not quite so carefully mathematically constructed machine learning model is one that's been with you for a little while.

1:52:49And I feel like in the development of neural nets like the biological analog they artificial neurons you know inspiration from biological neurons is a good one and has served us well in the deep learning field. And we've been able to make a lot of progress with that but I feel like we're not necessarily looking at other things that real brains do as much as we perhaps could and that's not to say we should exactly mimic that because silicon and you know what where have very different characteristics and strengths. But I do think one thing we could draw inspiration more inspiration from is this notion of having different specialized portions part sort of areas of a model of a brain that are good at different things.

1:53:37So we have a little bit of that in a mixture of experts models but it's still very kind of structured and I feel like this kind of more organic growth of expertise and when you want more expertise of that you kind of adds a more capacity to the model there and let it learn a bit more on that kind of thing. And also this notion of like adapting the connectivity of the model to the connectivity of a hardware is a good one. So I think you want incredibly dense connections between artificial neurons in sort of the same chip and the same HBM because that doesn't cost you that much. But then you want a smaller number of connections to nearby neurons so like chip away you should have some amount of connections and then like many, many chips away you should have a smaller number of connections where you send over a very limited kind of bottlenecky thing the most important things for that this part of the model is learning that for other parts of the model to make use of and even across multiple TPU pods you like to send even less information but the most salient kind of representations and then across metro areas you like to send even less.

1:54:52Yeah. And then that emerges organically. Yeah, I like that to emerge organically. You could hand specify these these characteristics but I think you don't know exactly what the right proportions of these kinds of connections and so you should just let the hardware dictate things a little bit like if you're communicating over here and this data always shows up really early. You should add some more connections. Yeah, then it will make it take longer and show up at just the right time. Oh, here's another interesting implication potentially. Right now we think about the growth in AI use as a sort of horizontal so suppose you're like how many AI engineers will Google have working for it.

1:55:31You think about like how many instances of Gemini three will be working at one time. If you have this whatever you want to call this like blob and it can sort of like organically decide how much of itself to activate then it's more of like you know if you want like 10 engineers worth of output it just activates a different pattern or a larger pattern if you want a hundred engineers about it's not like calling more agents or instances it just calling different sub subsets. Yeah, I think there's a notion of like how much compute do you want to spend on this particular inference and that should vary by like factors of 10 ,000 more really easy things and really hard things maybe even a million.

1:56:16And it might be iterative like right you might make a pass through the model and get some stuff and then decide you now need to call on some other parts of the model as another you know aspect of it. The other thing I would say is like this sounds super complicated to deploy because it's like this this weird you know constantly evolving thing with maybe not super optimized ways of communicating between pieces but you can always distill from that right like so if you say this is the kind of task I really care about. Let me distill from this giant kind of like organic thing into something that I know can be served really efficiently and you could do that distillation process you know whenever you want once a day once an hour that seems like it can be kind of good.

1:57:02Yeah, we need better distillation yeah and you went out there and that's amazing distillation techniques that instantly distill from a giant blob onto your phone that would be wonderful. How would you characterize what's missing from current distillation techniques. I just wanted to work faster yeah a related thing is I feel like we need interesting learning techniques during pre training like I'm not sure we're extracting the maximal value from every token we look at with the current training objective. Like maybe we should think a lot harder about some tokens you know when you get to the answer is maybe the model should at training time do a lot more work than when it gets to the right right yeah right there's got to be some some way to get more from the same data make it learn it forwards and backwards and what like which way like hide some stuff this way hide some stuff that way make it infer from like part of the way.

1:58:01So you know these kinds of things I think people have been doing this envision models for a while like you you distort the model or you hide parts of it and try to make it guess the bird from half like that it's a bird from this upper corner of the image or the lower left corner of the image. And that makes the task harder and I feel like there's an analog for kind of more textual or coding related data where you want to you know force the model to work harder and you'll get more more interesting observations from it. You know the image people didn't have enough label data so they have to mental this yeah like they better drop out was invented images but we're not really using it for text mostly that's one way you get a lot more learning and a more large scale model without overfitting is just make like a hundred epochs over the world's text data and use drop out.

1:58:57Yeah, but that's pretty computationally expensive, but it does mean we won't run it like even though people are saying oh no we're almost out of like textual data I don't really believe that because I think we can get a lot more capable models out of the text data that does exist. I mean like a person has seen like a billion tokens yeah and they're pretty good a lot of stuff. Obviously human data efficiency sets a lower bound on how or guess up or bottom one of them on it's an interesting data point yes. So there's a sort of like modisponents modisp tolands thing here of one way to look at it is look elements of so much further to go therefore we project you know orders of magnitude improvement and sample efficiency just if they could match humans another is maybe they're doing something clearly different to given the orders of magnitude difference.

1:59:50What's your intuition of what it would take to make these models as sample efficient as humans are. Yeah, I mean I think we should consider changing the training objective a little bit like just predicting the next token from the previous ones you've seen seems like not how people learn right. It's a little bit related to how people learn I think but not not entirely like a person might you know read a whole chapter of a book and then try to answer questions at the back and that's that's a kind of different kind of thing. I also think we're not learning from visual data very much you know we're training a little bit on video data but we're definitely not anywhere close to thinking about training on on all the visual inputs you can get you know so you have visual data that we haven't really begun to train on.

2:00:38And then I think we get extract a lot more information from every every bit of bit of data we reduce you know I think one of the ways people are so sample efficient is they explore the world and take actions in the world. And observe what happens. Right like you see it with very small infants like picking things up and dropping them they learn about gravity from that and that's a much harder thing to learn when you're not initiating the action. And I think having a model that can take actions as part of its learning process would be just a lot better than just sort of passively observing a giant days is got to the future then.

2:01:14You something where the model can observe and take actions and observe the corresponding results seems pretty useful. I mean people can learn a lot from thought experiments that don't even involve extra input and like Einstein learned a lot of stuff from thought experiments or like Newton like when it to quarantine and got an apple dropped on his head or something and invented gravity. And like mathematicians like you know map didn't have any extra input chess like okay like you have the thing play chess against itself and it gets good at chess that was deep mind but also like all it needs is the rules of chess so like there's actually probably a lot of somehow a lot of learning that you can do even without external data.

2:02:06Yeah and then you can make it in exactly the fields that you care about. Of course there is learning that will require external data but probably maybe we can just have this thing talk to itself itself and make itself smarter. So here's a question I have. Yeah what you've just laid out over the last hour is potentially just like the big next paradigm shift in AI that's like a tremendously valuable insight potentially. How do you know in 2017 you released the transformer paper on which tens of not hundreds of billions of dollars of market value is based in other companies not to mention all this other research that Google has released over time which you know you've been like relatively generous with in retrospect when you think about divulging this information that has been helpful to your competitors in retrospect is like yeah we'd still do it or would you be like that.

2:03:06I didn't realize how big deal of stress or was we should have kept it indoors. How do you think about that. It's a good question because I think probably you know we didn't need to see the size of the opportunity like often reflected in what other companies are doing and also it's not a fixed pie. Like this is like the current state of the world is pretty much as far from you know fixed pies you can you can get I think we're going to see like orders of magnitude of improvements in GDP help well and anything else you can think of. So you know I think it's definitely been nice that the transformer has got a got around and you know thank God Google's doing well as well so you know these days we do we do publish a little less of what we're doing but you know.

2:04:12I mean I think that there's always a straight off and of you know should we publish exactly what we're doing right away should we put it in you know the next stages of research and then roll it out into like production gemini models. And not publish it at all or is there some intermediate point and for example in our computational photography work in pixel cameras you know we've often taken the decision to develop interesting new techniques like the ability to do. You know super super good night side vision for for low light situations or whatever put that into the product and then published you know a real research paper about the system that does that after the project is released and I think you know different techniques and and developments have different treatments right like so some things we think are super critical we might not publish some things we think are really interesting but important.

2:05:13For improving our products we'll get them out into our products and then make a decision you know we publish this or do we give kind of a lightweight you know discussion of it but maybe not every last detail and then other things I think we publish openly and try to advance the field in the community because that's how we all kind of benefit from you know participating you know I think it's great to go to conferences like neuropse last week with like 15 ,000 people you know all sharing lots of lots of great ideas and you know we publish a lot of papers there. As we have in the past and you know see the field advance is super exciting how would you account for.

2:05:50So obviously Google had all these insights internally rather early on including the top researchers and up now as of 2024 your you know Gemini 2 is out we didn't get a chance much to talk about but people know like it's a really great model. Yeah, as we say around the micro kitchen such a good model. So is top in LMSIS chat about arena and so now Google's on top but how would you account for basically coming up with all the great insights for a couple of years other competitors had models that were more that were better for a while despite that. I mean I think yeah we've we've been working on language models for a long time you know the.

2:06:42Nomes early work on spelling correction 2000 and one the work on translation very large scale language models in 2007 and seek to seek in word to Vic and you know more recently transformers and then Bert and things like. The internal mean a mean a system for that was actually a chatbot based system designed to kind of engage people in you know interesting conversations we actually had an internal chatbot system that Google is good play with. Even before chapter PT came out and actually during the pandemic a lot of Googlers would enjoy spent you know everyone's locked down at home and so they can join spending time chatting with with me during lunch because it was like a nice and big yeah you know lead partner.

2:07:31And you know I think one of the things we were a little you know our view of things from a search perspective was like these models who listed it a lot and they don't get things right correctly you know a lot of the time the time and that means that they aren't as useful as they could be and so we like to make that better. And you know from a search perspective you want to get the right answer at every time the time ideally going to be very high on factuality and these models were not near that bar. But they I think what we were a little unsure about is that they were incredibly useful and they also had all kinds of safety issues like they might say offensive things and you had to work on that aspect and get that to a point where we're comfortable releasing the model.

2:08:19But I think what we kind of didn't quite appreciate was how useful they could be for things you wouldn't ask a search engine right like help me right to know to my veterinarian or like no can you take this text and get a quick summary of it or whatever and I think that's the kind of thing we've seen people really you know flock to in terms of using the chatbots as amazing your capabilities rather than as a pure church. And so I think you know we took our time and got to the point where we actually released you know quite kickable chatbots and have been improving them through Gemini models quite a bit and I think that's that's actually not a bad path to have taken.

2:09:02Would be like to have released a chatbot earlier maybe but I think you know we have a pretty awesome chatbot with awesome Gemini models that are getting better all the time and that's that's cool. Yeah so we've discussed some of the things you guys have worked on over the last 25 years and there's so many different fields right you start off with search and indexing to distribute systems to hardware to yeah algorithms and generally there's like a thousand more just go either of their Google scholar pages or something. What is the trick to having this level of not only your career longevity where you're having you have many decades of making breakthroughs but also the breadth of different fields.

2:09:49Both of you would have ate any of the order available with strict career longevity and breadth. Yeah I mean I think one one thing that I have that I like to do is to find out about a new and interesting area and one of the best ways to do that is to pay attention to what's going on. Talk to colleagues like pay attention to research papers that are being published look at the kind of research landscape as it's evolving you know be willing to say oh you know check design I wonder if we could use reinforcement learning for for some aspect to that and be able to dive into. To a new area work with people who know a lot about different domain.

2:10:28And or health a after health care is on the act on a bit of work right you know working with clinicians about what are the real problems is you know how could pay I helped you know wouldn't be that useful for this thing but it would be super useful those getting those insights and often working with like a set of five or six colleagues have different expertise than you do. And it enables you to collectively do something that none of you could do individually and then some of their expertise runs off on you and some of your expertise runs off on them and now you have like this bigger set of tools in your tool belt as an engineer and researcher to go tackles next thing and I think that's that's one of the beauties of you know continuing to learn on the job it's something I had to measure and I really like enjoy time you do things and see what we can do.

2:11:15I'd say like probably a big thing is like humility like I'd say I'm like the most conval ever.

2:11:26But seriously you know there's you know to say hey you know what I just did is is nothing compared to what I can do or or what can be done and to be able to like drop an idea as soon as you see something. That are like you hear somebody you know with some better idea and you see how maybe maybe that maybe what you're thinking about what they're thinking about or something totally different can you know it could you see the way work better because I think there's a there is a drive in some sense to say hey the thing I just invented is awesome by give me more chips. Particularly if there's a lot of tough down research assignment but I think we also need to you know you know incentivize people to say hey this thing I am doing is not working at all let me just drop it completely and you know try and try something else which I think Google brain did quite well with we have a very kind of bottoms up.

2:12:39The UBI kind of the chip allocation work. Like basically everyone had one credit and you could pull them. Yeah and then John and I I mean it has been like mostly top down which has been very good in some sense because it has led to a lot more collaboration and you know people working together you less often have like five groups of people all building the same thing you're building interchangeable things. But on the other hand it does lead to some incentive to say hey what I'm doing is working great and then then like as a lead you hear like hundreds of groups and everything is different. So you give them more chance there's less than set up to say hey what what I'm doing is not actually working that well let me try so let me try something different so I think going forward we're going to have you know some amount of top down some amount of bottom up so as to incentivize sort of both of these behaviors collaboration and like flexibility because I think both those things lead to you know a lot of innovation.

2:13:54Yeah I think it's also good to kind of articulate interesting directions you think we should go and you know I have an internal slide deck called go Jeff wacky ideas. I think it's like there's a little bit more like product be oriented things of like hey I think now that we have these capabilities we could do these right you know 17 things and you know I think that's a that's a good thing because sometimes people get excited about that and want to start working as you want. And you want to want to more of them and I think that's a good way to kind of bootstrap you know where we should go without necessarily ordering people we must go here.

2:14:37Yeah this is great. Thank you guys. I appreciate you taking the time and it was great great to have you.