Yossi Matias on the golden age of research

24:06Yossi Matias:That's actually what science is about. It's about building this cathedral of knowledge, these layers of knowledge. In order to build those, you need to actually have validation. That's why peer-reviewed publications are significant, so you can actually trust that what has been discovered doesn't only look good, but it was actually validated to the extent possible by the experts in the field. When we talk about the more prestigious, if you will, publications such as Nature, then obviously the importance is that it tests the merit of the scientific results in a very rigorous way, both in terms of its contribution to science, in terms of the validation of the claims made there, to make sure that experimentations weren't done in the right way, etc., etc.

24:53Yossi Matias:So that's why these scientific milestones make a difference, especially in time that you have so many publications out there. Yeah, yeah. Very much related when you think about the proliferations of publications. And by the way, not only peer reviews. I think that publications in archive is one of the amazing phenomenons that we see in recent years. The fact that you no longer wait for the innovation until it's actually appearing in a conference or published in a journal. Sometimes it will take months. Yeah. whereas somebody, actually, they have the result whenever it's ready to share. They just put it out there and it can be shared.

25:30Another acceleration of this, both the research cycle, if you will,

25:35Yossi Matias:but also the magic cycle of research between identifying learning and applying it. But it's also created a huge challenge because with these proliferations of papers, if you're a scientist, how do you actually learn about everything which is going on? How do you actually deal with all this literature? How do you actually build on that for the next one? I feel like it's impossible. Like, is it not? It's very difficult. For a human, it's impossible. So, of course, we're doing our best on, you know, every person with their own techniques of how to try to get a grasp of what's going on. And, you know, reading as many abstracts as possible or at least as many titles as possible.

26:12Yossi Matias:and here enter one of the capabilities of a system that we called AI CoScientist, which is asking ourselves, can we use AI to actually help us with first literature search? Sifting through all the literature out there for any research question that I'm posing. So I want to have some drug repurposing for a particular condition. And I want to scan, I want to read the entire literature that may be relevant. By the way, importantly, not all the relevant literature is obviously relevant. Sometimes you need to actually read through to look for those hints. Also importantly, sometimes the literature that is relevant for your research is from not necessarily only your domain.

26:59Yossi Matias:And in fact, quite often the magic happens when you bring together insights from multiple domains, which is quite difficult to do in real life. Yeah, because especially among scientists, people are encouraged to actually be experts in one area, they're not two, in order to be the front line. And then when you want to connect disciplines, it's actually quite difficult and there aren't too many polymers out there, you know, in the state of the art of more than one discipline. With Eicoscientist, which is a multi-agent system that is meant for a given research question posed by a scientist to do literature search practically as vast as needed.

27:36Yossi Matias:But then importantly also, given the question, the research question, start creating hypotheses, generating hypotheses that are relevant to that question. And then once you can start doing this hypothesis, obviously you may have a plethora of hypotheses. So an important question is how do you actually filter? How do you actually decide which ones actually are worth exploring? And also how we rank them. Because, you know, it's like search results. It's not only about how many pages you find, but what are the top 10 for consideration for the scientist. So ranking is important. Validation, of course, is important.

28:11Yossi Matias:Here's an hypothesis. To what extent can I see that it's actually consistent with literature that I find? Co-scientist is doing that. It's actually doing the literature search. It's doing the hypothesis generation. So think about it as a virtual lab that is actually doing all this work that typically a very senior researcher would do for more junior researchers, helping them, grad students, postdocs, and others. And what we found out is that this can actually be a huge acceleration for science. So in early partnership that we had with Imperial College, they were studying certain bacteria for years, actually.

28:54Yossi Matias:It took them almost a decade to come up with a certain hypothesis. They didn't publish it yet. And then with AI co-scientists, they got the same hypothesis in just a couple of days. And importantly, AI co-scientists also presented additional hypotheses they were not considering. And I was speaking with Tiago, the professor in Imperial College, who were actually interacting with the system. I asked him, so how was it to work with the co-scientists? And his response was, well, it was like working with an amazing collaborator. And similar response, you know, we have this from our partner in Stanford, we're looking to liver fibrosis and other partners on drug repurposing and other areas.

29:35Yossi Matias:So we have multiple actually research partners. Some of them already published actually papers based on research that was made possible with hypothesis generation. How much of this like the hypothesis generation sort of moves? I had a conversation with Jeff Dean the other day about sort of like where the bottlenecks move to. And it feels like if you can sort of do hypothesis generation with high fidelity, the bottleneck moves to this verifiability loop. And so I'm curious if we've seen more traction in domains where, like computer science, where you could actually do some of this validation or some other domains like that.

30:10Have we seen any early signs of that?

30:12Yossi Matias:In the world where you can actually generate hypothesis, and some of them, by the way, a very novel hypothesis. Again, should not come as a full surprise given that the amazing milestone of Move 37 were so that you have, when you have a powerful AI that can actually sift through opportunities and try to find something that could actually be good, some of that is going to be extremely novel. A lot of the effort is going to move into validation and into actually also asking the right question, evaluating alternatives, because, you know, we typically tend to ask ourselves, here's a question, what's the answer for the question?

30:49Yossi Matias:In reality, really an important area of the research is not only answering the question that is being asked, but also come back and say, you know what, but if I change the question a little bit, here's a much, much better answer. So being able to actually applying the judgment about the questions, about the interaction, about the validation is becoming increasingly more important. I think that AI should play a role also in that. In fact, if anything, the scientific method is more important than ever. And applying it in a very rigorous way is actually more important than ever because we cannot afford actually having papers or anything that is not validated.

31:30Yossi Matias:We're, in fact, looking into how to use AI to help out with some of those capabilities. We have this experimental tool called Paper Assistant Tool, PET, that is about getting a paper, actually providing back feedback to the authors. In fact, we worked with three major conferences, ICML, STALK, and NeurIPS, making it available for authors before deadline. And we had about 10 ,000 or so uploads. And many actually came back and said, hey, this was really helpful. It actually identified a gap or actually suggested another experiment to run to make the paper better. So this is a step in that direction.

32:08Yossi Matias:but I think there's going to be more opportunity and need in order to have this kind of validation. It's going to be extremely important for this kind of efforts. If I think about the eco-scientists and other tools, think about a world in which everybody can have their own virtual lab. Everybody means including junior researchers, including postdocs, including gradients. Think about the future where every person could have their own virtual lab doing this literature search, hypothesis generation, good hypothesis validation to some extent. At some point, it's going to be connected perhaps to lab experimentation that is going to even do more validation in an automatic way.

32:47Yossi Matias:This actually opens up some opportunity for everyone to ask bigger questions, to move faster, and to do that earlier in their career. Of course, there's some work to be done because if you ask yourself, if you're a junior researcher, do you know how to actually run such a virtual lab. Well, we still have work to do in order to train people to actually build on these kind of capabilities, which is true also for software engineering, as we know, and many other areas. So there's more work for us to do as society, as academic communities in order to prepare for the world in which actually everybody can be empowered to actually do this kind of scientific work in an accelerated way.

33:25Yossi Matias:I love that. I forgot to mention the punchline of my hope for 2028, or hopefully before 2028, We have these major scientific breakthroughs announced at I.O. And I think specifically it's going to hopefully come from Stanford and Harvard and all the great institutions will be doing important work. But I think the coolest thing will be if it if it's coming from folks who we would not have expected. It's going to be like, you know, a cure for a major disease. You know, theoretically, that's like not from somebody who like had 50 years of experience or like with a team and a domain. It's going to be a random grad student, master's degree student in Iowa who sort of has had a bunch of ideas and it somehow worked out.

34:03Yossi Matias:Definitely so, because what's really happening is that when I think about AI empowering or amplifying human ingenuity, it actually takes various aspects that today we expect actually every person to have all the knowledge and the capability to actually solve all these items in order to even participate, to have a seat at the table of research. The world expert with the best intuition about a particular scientific domain may not necessarily have all the other skills necessary in order to actually build these models. Which brings us to another important innovation. Because if you think about the scientific process, even once you have a hypothesis and when you want to actually test it out or you want to actually test out correlation of, say, COVID events and other factors.

34:51Yossi Matias:These are all scientific problems that require building models. And building models, as we know, is actually very tedious work. It's actually something that contains days or weeks or months. And improving on them, finding a better fit and all of that stuff is something that not everybody can do. So this notion of how to actually build those models is an important aspect and increasingly important part of the scientific progress. And here is where ERA or Empirical Research Assistance actually can play a hugely important role. ERA is really about if you have a scoreable problem, namely a problem that you know what you're trying to compute and how to measure that, and you have the input.

35:33Yossi Matias:ERA is about how to actually do this kind of search and tuning of models to try and solve it for you. So ERA can actually replace work that could be weeks or months or longer to actually try to complete. And when we build it up, and it can search through thousands of different models or parameters, space, etc., and actually give the result to you. And we already actually had just ERA again was published in Nature last week. But at the same time, we also shared about eight or so new papers, actually, that were enabled, made possible by ERA in various domains. I think from cosmology to epidemiology to engineering to economics, which is empowering, actually, to accelerate this kind of research.

36:19Yossi Matias:So now we have this hypothesis generation, and now we have also this ability to have model discovery with ERA. And also we have Alpha Evolved that was created, you know, amazing tool at Google DeepMind. And so we actually announced last week Gemini for Science, which includes these tools, you know, hypothesis generation built with scientists, computational modeling and discovery, computational discovery built with ERA and Ultra Evolved, and also literature insights built with Mold Book and Lem. So these are all tools that hopefully will accelerate scientific discovery. And now just imagine the world where you can have anybody who's passionate and learns about the topic and they want to actually drive research on that.

37:09Yossi Matias:Of course, the opportunity to be a researcher is even greater than ever because now you can ask bigger questions. Yeah. You can actually have a virtual lab helping you. And once you want to actually build models, you actually have this computational discovery to help you do that. Quite often, the magic happens when you bring in different disciplines. The beauty is that when you think about these AI co-scientists, it's not only having this amazing collaborator, but you practically have a polymath in your pocket because it actually reads through literature across all disciplines. That's quite powerful.

37:45Yossi Matias:It is. It's a good tagline, too. I feel like AI co-scientist, a polymath in your pocket is sort of catchy. I like it. I like it. Where do I get it, right? Yeah, I think there's actually still a big gap between an interesting hypothesis and starting to get early signal. If you don't know, if you're not in a place where you have access to those types of resources, do you think we'll actually build or someone is going to build out the physical infrastructure in order to do that in a more scalable way? Yeah, I think that the world is heading towards automating many of these areas that can then accelerate stuff.

38:19Yossi Matias:So various labs. I mean, obviously in the world that you can actually generate hypothesis faster. then the economics actually make it make sense to actually have labs that can actually then test things faster. Yeah. Right. I mean, that's becoming the bottleneck. Yeah, for sure. So I'm sure this is going to we are going to see even acceleration on that one. We need this from GR. We need physical labs to make this happen. Cool. Well, let's talk then about all the architectural improvements that are coming out of GR. Interesting novel model techniques we were talking before about like speculative decoding.

38:51And we should describe what it is for the audience and maybe for me as well. to sort of go through the details. Generally, that whole class of like, it's not just science. It's not sort of like the applications, but it's also actually like fundamentally like model techniques to improve Gemini and everything else that we're doing. Yeah.

39:07Yossi Matias:So first, perhaps if we can take the overall view for what we're looking for, what kind of problems are we looking at? Obviously, a lot of what we do is actually helping transform Google products for the benefit of its users. And this applies to search, YouTube, Geo, Cloud, and of course, Gemini. And working very closely with all teams at Google, who have a lot of collaboration with Google DeepMind, cloud, search, and so forth. So these are actually an inherent part. And part of the magic actually working at Google is to work across organizations, both on the research side as well on the side of actually bringing it to reality.

39:45Yossi Matias:Yeah, we need more GR AI Studio collaborations. That's my pitch. So we definitely, we talk offline. We talk offline. We should definitely do that. We should definitely do that. In fact, we're looking into JR Lab. Yeah, yeah. Offscript. I love it. One of the big questions, of course, with General TBI is how to actually drive research on foundations of Gen AI. Some of the obvious questions are how to make them more efficient. How to make them more factual. Have them in multiple languages and so forth. Perhaps we can start with efficiency because efficiency is, it's funny because as a computer scientist working on algorithms to make things more efficient has always been one of the, you know, the holy grails of things.

40:30Yossi Matias:Obviously, with Gen AI, it's becoming more important than ever. It has direct impact of what we can actually do there. A couple of years ago, we actually developed this technique algorithm called speculative decoding, which looks into how to actually accelerate the inference in our lamps. And what we found out that you can algorithmically, in fact, double or more than double the efficiency of the inference. And the efficiency is both in terms of latency and throughput and without any loss of quality. And, you know, somebody said, hey, there's no free launch principle. I said, no, no, with algorithms, actually, that's where we're creating free launch.

41:11Yossi Matias:That's what we try to do. And, you know, it helped us actually accelerate our products right away and actually save a lot of computation. in fact, practically became industry standard. So one can actually argue that today the entire world is running LLMs as if we had at least twice the chips. Yeah, yeah. Just because of technique and its variants and our many innovative kind of variants of speculative decoding. So that's an example of actually how to use algorithms to actually, you know, accelerate and then, of course, have it apply to our products and obviously to models. Yeah. What about sort of new architectures?

41:51Is that also something that y 'all are exploring?

41:53Yossi Matias:So as much as I'm excited about, you know, having twice the performance, and actually recently there was a team who actually applied it to Gemma 4 and got the times three performance. I think I saw that. We published this, right? Yeah, I saw this. So as much as I'm excited about it, I'm still waiting to see the times 10 or times 100 improvements. Algorithmic or architectural, so obviously looking into various techniques and research to see how can we accelerate stuff. And by the way, looking for that from the entire community. So innovation novelty can come from everywhere. And I don't see a reason why we shouldn't see a step function at some point of efficiency just by algorithmic or architectural modeling approach.

42:40Yossi Matias:And suddenly we're doing our own share to look into how to actually accelerate. Yeah, I love that. Through this conversation and through when we were talking off camera, I think the thing that is so interesting about GR is it really is the full stack of it. It's like everything from making inference more efficient to doing foundational research to actually how do we bring new sort of product UI experiences to customers. And so I saw, you mentioned before, I had a conversation with Robbie and the search team about GenUI, which is actually something that your team worked on. So do you want to sort of give context for folks?

43:14Yossi Matias:It's hard to believe that just it was a year ago when I had a conversation with Robbie about how do we actually bring it to search. Yeah. And here you go. You know, GenUI is one of those examples where you put a small team, you ask them to come up with, let's come up with some crazy ideas. And you see a demo and wow, it's about how to use generative AI to not only create the content, but also to help decide, given the content, what is the right presentation of that content that actually make it most effective. It feels like a very Google problem. This is like showing the world's information. Right.

43:49Yossi Matias:Totally. So, and again, another important thing is not only a brilliant team coming with a brilliant idea, but then also using AI to accelerate, to do the demo quick, and then to implement it very quickly. Yeah. And then to actually have it out both in Search and Gemini and build on that. and many of the announcements on generative UI in search and German app actually are based on this foundation of this demo that was actually iterated on. So again, this is an example for a magic cycle of actually coming up with the research, with the prototype, iterating on that, solving a few more problems along the way and then applying it to reality.

44:28Yossi Matias:And now we have a new experience of actually consuming content with AI actually helping us to, with the presentation itself. Yeah, I love that. I feel like that also has to be one of the, or I'm curious actually where it stacks up as far as you mentioned before this the magic cycle is getting faster and I feel like GenUI is maybe the maybe one of the fastest examples I don't know if there's been something faster but um you know probably the order of like months to like actually getting it into some of these experiences and then like more widely rolling out at io this year and and actually more widely rolling out a few months ago and uh when search announced it so crazy well Well, actually, it rolled out within months, actually, on the first kind of, if you will, publication, or a first kind of demo.

45:10Yossi Matias:And then within a year, actually, we have now a large scale. So I think that this acceleration is there. So again, a combination of innovation, very strong technical kind of capabilities to actually move things. And of course, the drive to actually make a difference here. So this is a good combination. Yeah, I love it. Josie, I feel like we could have this conversation, or we will actually have this conversation every couple of months, because I feel like GR is touching so many different things across Google that I think you have a very interesting perspective, like seeing all the things that are happening.

45:39Yossi Matias:On a personal level, my interests have always been both on driving research. And again, I had the opportunity to be in Bell Labs in their heyday in the Mathematical Center, where Shannon and Heming were there many years before me. And Paul Erdos was actually visiting us every year. At the same time, my passion is really about how to build products and have impact. So, you know, over the years, I was actually going back and forth between scientific research and entrepreneurship, building products. And at Google, actually, much of my time, in fact, over a decade, I was at Google on the search leadership, building products in search, Google Trends, autocomplete and so forth.

46:18Yossi Matias:We are in this amazing time where actually this notion between driving the research breakthroughs and then applying them to products and working with product teams is tighter than ever. Yeah. You should do some writing about this. I feel like I'm not sure. I think you're spot on with this. And it's not clear that the broad ecosystem really is grokking this point. And it's so worth double-clicking on that. I totally agree. Totally agree. And in a way, for example, I never use the term technology transfer. It's never about the transfer. It's really about identifying the problem, solving the research, applying it to reality, and circling.

46:57Yossi Matias:Because it's never one time. It's never one directional. And that's actually where the magic happens. It's really this iterative approach on improving on all things. So personally, since, in fact, even when I was in Bellabs, by the way, my best-known theoretical research was motivated by a real-world problem, which happened to be what was then the largest data warehouse in the world, and actually then helped create a field for, you know, data streams that practically enabled big data analytics. But it took years actually to do this kind of cycle. And now actually part of our job is actually to do both.

47:34Yossi Matias:And one of the strengths of Google Research is actually in this, doing this actually both. Same team that actually pushed the research on floods is the same team that actually built the system that is now, you know, providing prediction to 2 billion people. Same thing I would ping actually what's going on in that place. same thing as thinking about the next project. And it's a fun fact, I recently had a conversation with one of the research scientists there, and I learned that he was actually looking into a particular model that he, for a long time, was unable to make progress on. And then with ERA, he actually made some good progress on.

48:13Yossi Matias:So it's all connected. Let me give another example for research we've been doing for several years. In fact, when LLM just started back in 2021, we actually started looking into research on factuality for LLMs. And again, part of what we're trying to do is to not only think about the problems that everybody care about today, but to take bets on what may be important in the future. In 2021, very few people actually cared about the future of LLMs. Factuality, but we actually thought, you know, I thought that factuality is going to be more important than perhaps anything else if we ever want to use LLMs.

48:49Yossi Matias:So in fact, one of the pioneering papers on factuality for RLM was published by our team in collaboration with academic collaborators back in 2021 about measuring consistency of RLMs. And even in 2022, we already had a benchmark out published. We called it TRUE. When we started putting out our language models, we started actually taking some of our research and applying it to our language models, you know, and to Bard and others, to Palm and other models, to Gemini, of course, and keeping on the research and iterating on that and then working very closely. Everything we do, of course, on Gemini is a very close collaboration with Google DeepMind.

49:33Yossi Matias:And so working with the teams and actually bringing together factuality research into making Gemini even more factual, publishing the fact leaderboard that so is to encourage actually other, you know, models to also test themselves against benchmarks and keep iterating on that. So this is an example of our magic cycle that has been going on since 2021 on the factuality of RLM. And in any given moment, I would say that, hey, in fact, much of the work is still ahead of us. Because factuality of RLM, in order to trust large language models, we really need to know that it's working to the best possible.

50:13Yossi Matias:And the better it is, then our expectations are going higher and more nuanced. the nature of factuality is changing. It's a model. It's taking context into account. It's addressing also more nuanced questions. In fact, it's sort of doing things on our behalf. Exactly. It's authentic. And by the way, one of the areas that I think is extremely important is also for not only the question of how factual things are, but to what extent do we know how factual we are? What's the level of confidence? That's an extremely important aspect from my point of view about the factuality of LLMs or any system for that matter.

50:51Yossi Matias:Yeah. The beauty of technology is that what is only possible for very few in very few years becomes a commodity. Let's call it technology commodity or a scientific commodity because everybody can now use it or build on that. It's like nobody thinks that there's going to be surviving cars. It's an inflection point. It's only boom. Yep. 20 companies. Of course. Not only that, but look into what's happening on the street with Waymo. It's amazing. The amazing success of the Waymo team. And again, it wasn't that long ago that people were skeptic. Yeah, yeah. So the beauty is that technology innovation and scientific progress is not linear.

51:32Yossi Matias:That's actually a common mistake. People are extrapolating. That's what we're doing again and again the mistake. In fact, going back to Google research, our job is to look into the horizons and around the corner. is not to extrapolate. That's why quite often I would take bets. And that's part of my fun is actually, you know, I started actually doing AI. You know, I did Duplex. Remember Duplex? I don't think so. Google Duplex? No, it sounds super familiar. Google Duplex was the big announcement in 2018 of the AI system that's called the restaurant reservation. Oh, yeah, yeah, yeah. I do remember this.

52:10I do remember this.

52:11Yossi Matias:The biggest AI moment in 2018, arguably. Yeah. So this was my pet project. I was actually in search at the time. Yeah. But at the time, I thought about, well, in the future, actually, conversational experience is going to be the most important one. So we should actually look into that. And the question I was asking there, are we going to get to a stage that everybody can just talk to the computer? This was a question I actually put publicly in a talk in TEDx. One is actually saying, hey, can we actually just talk to a computer like we talk to ourselves? Feels so common now. Yeah. And my answer was, yeah.

52:42Yossi Matias:I think that anything that we can, if we can dream it, we can build it. Yeah. So, and again, today, if you ask anybody, of course, yeah. I mean, yeah, of course I can take it way more. And of course I can talk to my computer. I tweeted this morning, actually, that in the age of intelligence, sort of like you can outsource your thinking, but you can't outsource your understanding. There's like this conflation between the two things that I think is actually going to become ever more present. And that's why I'm like even more excited to hear about sort of what's top of mind on education. So, you know, when I think about a few years ago, somebody asked me, hey, if you had a magic wand for helping climate, but you cannot just use it to fix climate, but use it somehow, how would you use it?

53:25Yossi Matias:And I was thinking about it and my answer was, well, I would actually use it in order to make sure that we're getting the right education for every kid because they're actually going to solve all the other problems. Yeah. Yeah. So obviously, having the right to education is the single most important thing for any society. Obviously, for societal reasons, but obviously also because that's actually the most important resource that we have, this human ingenuity. And with AI as a possible amplifier of human ingenuity, that becomes a multiplier. So the question is, how can we actually do a better job in education?

54:00Yossi Matias:And also, how do we need to change it, given what's happening, right? with AI, it's actually enabling new capabilities in education, but also creates new objectives for us, how to actually educate the next generation so we can actually use AI. You know, AI is not something that happens to us. It's something which we should build on. Yeah. And same applies for education. Like in any system, the first question is, how can we build models that are better suited for that? You know, we did work on LearnLM, which essentially is how to adapt model. so that it knows that it can support various needs for education, such as, for example, re-leveling, such as quizzes, basic stuff for education.

54:44Yossi Matias:And we actually made it eventually part of Gemini, and now we can have applications built on that that can leverage on the capabilities. As an example, I was visiting a high school in Ghana where a teacher was using an app developed by a third party actually built on leveraging and learning them. and they told me, hey, now students are getting response to assessments every day instead of only once or twice a week. And the teacher got back five hours. Yeah, yeah. So obviously, this is about how to make things more efficient. Now, when you think about it, and we did some efficiency studies and looking into other ways in which you can actually use it.

55:22Yossi Matias:But when you really think about it, the real opportunity is for using AI not to just make things more effective, but actually to change things. Why can't you have assessment that is more personalized, that is real time, as if you have a tutor sitting next to you and actually tapping you on the shoulder every time you need to do some correction? That's actually the real opportunity. Another comment here is that the role of the teacher is more important than ever. In fact, you know, a few months ago, I spoke about AI for Education in a conference. And then I actually pointed out that, you know, I'm forever indebted to my own high school math teacher and I was actually fortunate to have him in the audience actually there about to be there and the role everyone probably every one of us have their own no teachers that influence their lives how can we actually make it even more so for everybody else who perhaps didn't have those teachers and to me it's really empowerment it goes back to how do I use technology so we can empower the teachers so they can actually focus on being the inspiration of actually making those change.

56:31Yossi Matias:Another important comment is that when we think about how to use AI for a system, the first thing is how to make things more effective. I would say how to amplify human capability, if you will. That's only the first step. The more important impact is how to do things that previously were not possible to do. That's really the amplifying of human ingenuity. And that applies to education. It applies to doctors. It applies to scientists. It applies to all of them. So when I look into education, really the question is, how do we first make sure that we can empower teachers with technology? How can we make things more effective?

57:11Yossi Matias:For example, you have this experiment called Learn Your Way, which essentially asks the question, how can we reimagine textbooks? Can we take a textbook and actually make it in multi-modalities and in a way that is personalized? How do you teach gravity for a girl who's 10 years old and loves soccer? You do that differently than for a 16-year-old that loves tennis. So these are opportunities to do. But also looking into how can we actually have programs such as AI Quest, which is working with Stanford Education Center, how to actually expose kids to the opportunities of AI. Think about it as AI skilling.

57:51Yossi Matias:Talking about skilling, one question that I think related to your comment, what are the skills that are most important for the teacher? So we can debate to what extent do you need to learn the basics. Personally, I think that, for example, thinking how to code basic computer science, etc., is going to be more important than ever. Agreed. I mean, I always thought that computer science is not learning a language. It's not learning how to think. Yep. So in a way, it's going to be more important than ever for the same reason that math is important even when calculators became a commodity. But obviously, there are skills that are going to be more important than ever.

58:29Yossi Matias:For example, collaboration. If the role of the person is going to be more importantly to ask the right questions and to validate the answer and to ask what is good, it means that certain humanities and certain capabilities that perhaps sometimes we call them soft skills, but they're actually skills. How to collaborate is a skill. These are going to be more important than ever. So how do we teach them? In fact, we actually launched an experiment called Vantage, where we're actually trying to measure those skills, this collaboration with NYU. And we put it out as an experiment to see, can we actually look into how to measure it and how to skill it.

59:10Yossi Matias:To me, these are important areas that we need to actually double up on in order to see how do we teach the next generation the right skills for the future? How do we keep making sure that they are learning the basic skills? Because when you think about AI becoming a kind of a tool for everybody to build in, we need everybody to actually be elevated so that they can actually use those tools. It's true in science to run your own virtual lab if you think about it you need to be do the stuff that you need to be very advanced scientist to do that if it's a teacher to actually build on that if it's a business person et cetera et cetera if you're an healthcare worker essentially to be able to operate as if suddenly you have this kind of capability in your hands so I think this goes back to education to the early questions and education across all ages how do we actually give those skills how do we empower them and how do we use the AI in order to do a better job at that yeah Yeah.

1:00:08Even upskilling folks who are sort of like not in the traditional education system, I think is really interesting. Like I was just thinking, as you were saying that I barely eked my way through calculus. It was very difficult for me for whatever the reasons were. Things didn't click in many ways. And so one of my like personal benchmarks is like, I want to go back and like try to like from scratch learn and see how long it would take. Like actually using these tools, using a lot of like actually notebook alum, I think is one of the products that hopefully a lot of the research that y 'all are doing ends up making its way to and just like personalizing the educational experience for people based on what they get excited about and that was always my challenge was like you sort of learn a lot of these things in a vacuum and even actually computer science like you learn computer science sort of and and there's like a distant hope that you maybe will do something that's interesting um and and i feel like what's what yeah what's gotten me excited in the last couple of years about what's happened with software is like the distance between the exciting thing you can sort of like taste it now and you can feel it and you can experience it and I feel like in you know in math and other disciplines it like wasn't like that you were sort of you had to get so far down the path in order to like experience it and I feel like it's been it's been cool to see that change and hopefully get people more excited about doing

1:01:22Yossi Matias:that type of stuff yeah I mean if you think about it a few observations is those who actually got the right tutor and the right teacher, then they actually got this opportunity to do stuff that others perhaps not. I think that human capability is perhaps one of the most, this is the most important resource that we have here. How do we actually enable everybody to reach their own potential is an opportunity that we have here. And on education system, you're totally right. I think my own experience even as a tutor, I remember when many years ago I was actually tutoring kind of bright kids. And I couldn't understand why they are actually failing in math when they are so bright.

1:02:08Yossi Matias:And actually, after a little kind of investigation, learned that actually they have a gap to fifth grade math, which they were carried on all the time. And it was actually all the time and the way of actually doing things the right way. third grade. I got my appendix removed and I was out for a couple of months and I'm sure that that's what it was catching up to me in calculus. Now, all these matters are fixable if you have the right attention and the right time. Hopefully with AI, we can actually fill many of those gaps. Yeah. Because some of them are actually not too sophisticated if you put the right intelligent reasoning on them.

1:02:46Yossi Matias:And I think with AI, we can actually get pretty good reasoning to actually address many of these problems that today, as if you had suddenly an abundant of tutors, very smart assistants helping you, research assistants, if you will, helping you on practically any topic. So I think that's actually some of the opportunities that are opening up. And again, going back to what our role in that is really to think about some of those opportunities and to build into some of those, to the extent possible, work with others. So I'm quite optimistic about how we can actually make a real difference here on things that really matter, really having societal impact on those.

1:03:29Yeah. You mentioned before we won the Nobel, a bunch of the researchers won the Nobel Prize for quantum. That shined a light on a lot of the breakthroughs that have been happening, but folks who have heard bits and pieces, maybe thinking quantum is science fiction. What is Google actually doing today in this space?

1:03:43Yossi Matias:So I think quantum is an interesting story. Obviously, some of the early pioneering thinking came from Richard Feynman about, you know, making the case of why quantum could be a big deal one day. The big breakthrough was Peter Shore actually showing a real practical algorithm that today is perhaps some people would say too practical of actually factoring numbers. and this was by the way Peter was his office was just a cross-mining bell app and he showed actually with a quantum computer you could actually solve a problem that is assumed to be hard for classical computers actually it's so much assumed to be hard that it's the basis for many of the crypto systems that are used in the industry and he showed that if one day we're going to have a quantum computer we're actually going to be able to solve it now by now we have actually a dozen or so perhaps 70 or so kind of known applications of where quantum could actually make a significant acceleration over classical computer.

1:04:46Yossi Matias:But importantly, there are some additional expectations that if we'll have quantum computers, we're going to be able to actually have a better understanding of materials. We can actually simulate materials. We can have quantum sensing. And for simulating materials, as an example, it's just like some of those simulations are just like so computationally intensive that... Not only that, you need to have the intermolecular, you need to have the quantum phenomenas. So this was actually part of the statement by Fanland that, hey, in order to actually simulate a quantum world, you need a quantum computer.

1:05:18Yossi Matias:Yeah. This was kind of the thesis. But importantly, my projection is that when you think about what is known, that quantum computers are going to have advantage over the world, suppose what will happen if you take thousands of bright people working on actually developing more applications? because think about all the big number of amazing talent working today on AI and machine learning, and you see proliferation of progress. Just think about the fraction of that actually working on quantum. I'm sure we're going to see many more opportunities to actually unlock because it's a new computing paradigm.

1:05:55Yossi Matias:Now, the real question is then, can we build a quantum computer? And what is the technology to do that? Now, some of the breakthroughs that are enabling that go back to the 80s. And we're really thrilled with the Nobel Prize in physics to Michel DeVore in our quantum AI lab. And also John Martinez used to be part of that lab. And also John Clark, the collaborator from Berkeley. The three of them were recognized with Nobel Prize from their work on the 80s. Now, we built our quantum AI lab in 2012. Actually, really quick. Like why, why the gap? And I'm not super, I saw the announcement and stuff.

1:06:32Why the gap between the work in the 80s to being recognized in, I think it was 2025 or something. It was just, it like took that long for the, for it to be like shown actually that like the work was going to prove out or it had been proven out. And then folks just are, you know, quantum is becoming more real. And so there's like more like scientific interest in the ecosystem or like why, why such a big gap in that?

1:06:54Yossi Matias:Well, I cannot speak for the considerations of the Nobel Award committee. So the big question over the years was will we be able to build a quantum computer that actually can do the work? There are many questions including one of the big technological hurdles was in order to have a scalable quantum computer you need to solve the problem of errors and you cannot just apply error correction like you do with classical computers so this notion of quantum error correction is increasingly important. Now over the years we did have significant milestones. Our own milestones just last year was actually the Willow chip, which had a couple additional milestones with that.

1:07:36Yossi Matias:One was an error correction that was tremendously better than previously known, that actually showed a promise that we can actually keep scaling quantum computing with error correction. And quantum error correction, just to double click on that, is really when, because the basic operations are dealing with qubits and in this physical world not all operations actually work and then you need to somehow compensate for that so how do you actually deal with faulty qubits so error correction is helping out to do that much in the same way that things happen also in the regular computer and then you use error correction on classical computer in order to compensate for that we had a few The most recent one is called verifiable quantum advantage, namely a problem that we could actually apply a quantum computing that we could verify and compare it to the best classical computer that could solve it.

1:08:34Yossi Matias:And we show that a particular problem, we could actually do that 13 ,000 times faster. Wow. So this is kind of a verifying in a problem that is different than a mathematical Schroer algorithm that, hey, there's a certain computation that with a quantum computer, we could actually do it much, much faster than classical. Now, it's still in the premise, and we still need to keep building the systems. We had very clear roadmaps of how to reach those. And again, it's a combination of an amazing hardware team, applications on quantum, and we keep actually having these milestones. And the premise there is quite significant, obviously also accelerating AI in some capacity.

1:09:12Yossi Matias:So it's not going to replace classical computers, but it's going to complement them and for certain problems it's going to unlock opportunities never seen before such as generated data for example for which we can possibly build AI models that can help us better understand say materials or biology information or fusion and so forth. So anyway these are some of the premises and really exciting to you know to see how we're making progress on that and how this is actually based on work done in the 80s but keep iterating on that so the magic cycle of research here it takes longer but it's quite significant there's so much stuff happening keeping track of this i don't know if we put a metric on the pace of the magic cycle and sort of watch as things happen like the speed up this would be a fun benchmark i don't know how to do this but uh this might push for you is you should make that i feel like going to be super interesting to see.

1:10:05Yossi Matias:All the work that I've been talking about is made possible by amazing teams that are working together across great collaboration, obviously within Google Research, but also with our colleagues in Google DeepMind and across Google. And a lot of that also with the academic community. So this notion of actually having smart people that are driven by making real impact, working in collaboration, then applying it on some of the best systems in the world, of course, and applying it to products that are actually serving billions of people, helping in so many different ways, and then iterating and thinking about what's next, about what's around the corner, what's in the horizon, how to take those bets that we can actually drive things that perhaps may seem impossible, how to actually make them possible.

1:10:54Yossi Matias:Yeah. That's actually a pretty amazing opportunity. that's why it's really more exciting than ever to do research it's really a golden age of research yeah it's very exciting well jose thank you again for sitting down and talking about sort of the golden age of research i feel it and i think it's sort of our obligation to help tell the world this story so thank you again well thank you thank you very much for having me here it's always great to have a conversation and uh and thanks for amazing one of course yep and thanks everyone for tuning into this episode of release notes we'll see you in the next one