Can AI Advance Science? DeepMind's VP of Science Weighs In

27:26It would be huge. So I think there's the arc of that and I think we're just at the very beginning. Definitely. We talked about AlphaFold, which is very exciting and maybe the most familiar to folks, but Pishmeet, your team has also created a bunch of other papers that touch this intersection of AI and science, or you could say AI and math or AI and physics. And those are things like materials, graphcast, which has to do with weather forecasting, fun search, alpha geometry. And so I'd love to hear from you again on this probing of, are we moving the frontier forward with these different models?

27:58What are you seeing from some of these other projects that your team is working on in terms of AI helping us actually uncover new science? Essentially, what we have entered is basically an age where a single human mind cannot comprehend the data that we are gathering about the universe. And this is true in any field, you know, encounter. It is true in biology. No biologists can reason and analyze all the biological data that will be gathered. No physicists can look at and analyze all the high energy physics data that is being gathered. it, and even mathematicians cannot sort of look and analyze all the large scale mathematical simulation data that we can now compute and simulate and find out.

28:41And I think what's happened is AI is not sort of nice to have. It's basically of almost a necessity for us to make sense in reason about any problem that we are now looking at. I have examples in pure mathematics where work on topology, you describe a not in two different sort of There was a generic definition and there was a geometric definition. And mathematicians understood these characters, but never understood the connections between that. And what we showed in one of our sort of works is basically we generated a lot of data for nodes in these two characters. And somehow, Yasen even let's what can you make predictions about one character from the other?

29:22And the idea was well down there should be no. But in fact, it couldn't make predictions. And when we drill down, we found a very nice conjecture that nobody had encountered. And we work with mathematicians who then not only make that conjecture, but actually prove that there was a very elegant, nice relationship between those two characterizations. So this is like completely fundamental discoveries in mathematics that were completely unknown to mathematicians now being uncovered by a machine learning and AI model. And we are seeing this across the board in any of the scientific areas that we are looking at.

30:02We are discovering new insights, new sort of patterns that were not expected, just because the techniques to analyze the raw scale of data did not exist. I think amongst biologists, especially maybe 10 years ago and Frinter back, I think there was often a belief that biology is just so complex that it's just incomprehensible that there's no way to even understand it. The only thing you can do is run the experiment that's going to happen. And I think we're seeing the beginning of a shift where people are starting to think, well, there are complexities and there's a lot we don't know a lot to learn, but that AI actually can gather all that together and start to decipher this and to be a natural language for biology.

30:44And I think there's going to be this really fun cultural shift where 10 years ago, people would say, I was ridiculous, a computer could try to do these things. I think 10 years from now people will be like, what's a ridiculous effort, a human being to that? Like you can't, like load all these numbers in your head. That's just ridiculous to even say that. And we've seen this in other places, like chess. It seemed like impossible that a computer could be the grandmaster. And then now there's not even worth trying. It's table stakes. Yeah, yeah. And we saw it with Go. We saw it with all these other things.

31:12So I think that's just a cultural shift. But I don't think that's a bad thing. I mean, Fort Lift can lift much more than the strongest weightlifter and we view that as a positive thing. It's always going to be us and them. I think the interesting question will be is once you can do these things that we can't do, well, what do we do together with that? Yeah, and what can we do? I mean, one of the most amazing things, I think, is that deep -mind for the most part has given these models or the results of them to the community and so researchers have their hands on them. And so maybe we could talk about that, how are researchers leveraging these new breakthroughs?

31:47There's all kinds of stats around we don't have enough cancer drugs or they're insortages and those are very real things we want to fix. So Pishmeet maybe we'll start with you. What are you seeing and your team seeing in terms of this technology being deployed and how are researchers using it? Yeah, so this was another sort of fascinating journey of growth. I was not not from the natural sciences. So working on Alphabot was a learning experience, but then actually releasing Alphabot to the community was even the bigger sort of learning experience. So Alphabot database, when we were sort of building it up, we wanted it to be available everywhere in the planet to all the sort of scientists.

32:25But the scale of science was unprecedented. I was not aware of it. The Alphabot database today has been accessed in 190 countries. And there have been 1 .6 or 7 million users of the Alphold database. Now, if that is not a positive statement about the planet, then I don't know what it is. There are 1 .7 million people interested in 14 structure prediction. I'm really happy about that. I mean, all the things that are happening in the world. And in terms of the impact, it's again, like in the amazing sort of spectrum, We saw Alpha Fold being used in path -breaking, fundamental, biological discoveries.

33:07Like my personal favorite in that domain is the nuclear pore complex, the structure of basically the pore complex, like the way a nucleus controls how a material gets into the nucleus and out. I mean, that fundamental structure of that complex was not known. And the searches used Alpha Fold to structures to be able to piece together the whole complex. A recent paper from the Feng Lab showed how you could develop a molecular syringe. And again, they used Alpha42 in designing that. And there's so many other sort of areas where people have been using it for developing new vaccines and working on new antibiotics against antimicrobial resistance and synthetic biology.

33:47Like one of the key partners at the early stages was a university here in the UK, which was using sort of alpha -4 to develop and think about enzymes that could decompose plastics. So you have this whole spectrum of fundamental biology drug discovery to even synthetic biology and enzyme development that has been impacted by alpha -4. And so it was very difficult to even predict what would be the uses of the two. I think there's also just within biology, there's become a shift that I think people are sort of wrapping their heads around prediction a bit better. I think before experiment was the gold standard and that was all people wanted to hear about.

34:30I mean part is also just the zeitgeist of the time when you deal with large language models, you're basically dealing with predictions of what comes. And I think people have understood the pros and cons of predictions, but that there's massive value in having it. And I think it's It's funny that we would talk so much about the technology, but I think it's the human shifts, so the cultural shifts are the things that we're going to really need to push. And I think what gets me most excited about what Prishmits has just been talking about is the fact that I think that's the sign that we're seeing this cultural shift as well.

34:59Maybe something else you could speak to VJ that's just coming to mind as both of you are sharing more about these researchers. How does this change the economics of a lab, right? If you think about what we talked about before, is like uncovering a structure it could have taken a whole PhD. Now we have new tools and we're seeing these economics change in some of the more consumer fields and those are very obvious. How does this change the economics of research overall? One of the sort of fantasies that one of my former colleagues talked about was what we call beach biotech, where you have let's say one person at a laptop, but presumably on the beach, wherever you want to be.

35:35And you've got CROs, these contract research organizations to do the experiments, you have some AWS cloud or whatever, some GCP clouds somewhere to run your calculations, and that one person with AI, I think we're not quite there yet, but I think that's an intriguing fantasy to think about. And I think on the way to the one person's sort of aspiration is smaller teams doing way more with much less capital outweighs and building startups, I think much more efficiently, and where they get to results much more rapidly. That challenge is going to be, what I mentioned is four, is that getting to the clinical trials, speeding that up will be nice, but I think the big financial return will be on the clinical trial side.

36:17But I think the expectation is that AI for biology and understanding targets and so on, based on human data, that would also help on the trial side and in addition to anything else there. So I think put together, I think we can get to these therapeutics faster, cheaper, and hopefully better. Yeah, and maybe push me, we could tackle that directly. If you could give a sense for folks who aren't these researchers who aren't already leveraging these tools, how much does it really cost if someone does want to get a protein structure prediction or use some of the other models that we've talked about again, graphcast or materials, etc.

36:52Like what cost are we really looking at? Yeah, so for the ultimate goal database, it's literally free. We just go to the full database, sort of, to find the protein that you're interested today, now from the 250 million sort of proteins and make it and it's there. It's for free for everyone on the planet to use. So really it has democratized things in a way that scientists in Latin America or India who was working on sort of neglected tropical diseases, for instance, who had no way they could get a structural of a protein that they were interested in can now get access to these structures at the sort of click of a button.

37:33Of course, a lot of research needs to be done to take that work and towards a more focused outcome and a lot more investment is needed if you are trying to finish and accomplish the vision that Vijay has had outlined. The other four structures are start but you really need to think about how does it bind to the how do you do the liquid design, how do you solve the co -holding problem. So there is a lot of investment that is needed to make these models and make these predictions and refine them for specific applications. And we have a spin -off from DeepMind Isomorphic Labs, which is now investing in this area as well.

38:14At the same time, we are continuing and work on the foundational sort of side of things and have now released an announcement or an update on the next generation of alpha -fold, which goes beyond proteins to other biomolecules to nuclear acid, like DNA, RNA, BDM, small ligands, and so on. I think it's amazing that you've opened this up to the community. And I think something I'd love to hear both of your takes on is really the relationship of these models and them being open sourced. I mean, it's a big debate with an AI at large, but I think especially when it comes to science, there's, I think, both ends of the spectrum in a way, right?

38:56I think there's nothing more that people get excited about about this idea of curing cancer, like solving poverty and agriculture crisis, but at the same time, people also get very scared, right? I think that's where people's sci -fi nightmares come to be, right? Where they're like, oh, someone can engineer a molecule that can kill us all. And I guess starting with you, Vijay, what's your take on this relationship of AI and science and why it should be open source? I think the beauty of open source, and we see this open source for AI and biology, but AI more broadly, is that people can build on top of each other.

39:33And I think what's really remarkable about the AI field, I would say, over the last five, maybe possibly 10 years, is that it feels like an amazing result comes out once a week. And that the key part of that is that it comes out with code or GitHub repo. And that you can check out immediately. You don't even have to just believe the results you can run it yourself. people have even open sourced for his tests of things. So essentially we're building like a skyscraper where each person builds a new floor and we're going up really fast. And that's what open source can do. In the past, if it wasn't open source, I'd have to read the paper, I'd have to code it myself.

40:07And sometimes the paper may be a little vague for some detail. So I might not bother, right? And I'll just go do my thing. And so I think what open source allows us to do is to build on top of each other and build rapidly. Now, certain parts won't be open source. I think you unfortunately can't open source a drug compound because then no one's going to pay for the trial and certain things like that. Just the economics doesn't make sense given these hundreds of millions of billions of dollars and so on. So, certain parts will be closed source and there's hundreds of startups in AI in biology and AI drug design that will maybe take advantage of what's been done, develop their own methods and build them top and then that's where I think the drugs will come from.

40:46You talked about also the concern for how because this is so powerful we could maybe do a certain dangerous things with it and that's where everything there's a bit of a misconception because actually there's a huge asymmetry between the complexity of drug design for treating disease and that's a really hard problem to do but it actually turns out to be really easy to come out with chemical that actually are dangerous and toxic. In fact that's why we have phase one trials because like even the things that you thought would really hopefully not be toxic at all, turns out to be toxic. So it's actually very easy to make toxic things.

41:21And Google will teach you actually how to get rice in and how to get all this other stuff for better or worse. So I think there, the asymmetry is that if we get rid of AI for drug design, you lose all the good and you don't prevent any of the bad, which is already here. I think that's a good point that a lot of people don't think about. Fish meat, maybe you could just speak to why DeepMind has chosen to open source these models, which isn't necessarily the norm across different AI companies. There was a lot of deliberation within the team and within the company on this. I think there were a few different things that went into the final decision.

41:57One was, we wanted to, like, out of all wars, that foundational there. It was so foundational. It would, if we had kept it close source, the impact of it, like, fully leveraging them back for society, I mean that would have been difficult. It was because it's so fundamentally sort of foundational and it's very hard to even predict what are the potential sort of applications of it. Just to give you an example, when we launch Alpha Fold, couple of days later, somebody did an analysis on the uncertainty associated with the Alpha Fold predictions and figured out that in fact Alpha Fold was even though it was not trained for that was the best predictor for predicting disorder in proteins.

42:42So that was something that we would not have come up with, right? If we had kept it close so as someone barely interacting with the model in the community figured that out. So when we were thinking about it, there was of course how to maximize social impact and scientific impact of the model. The second world was responsibility. And we consulted a number of experts from structural biology, from chemistry, from doctors' recovery to figure out what is the right and responsible and safe approach here. And even considering the malicious sort of use cases, and after we had done all the due diligence that we felt that this was safe to release, and the impact of releasing it and open sourcing it in a wider sort of way would outweigh any costs that we would need to sort of model.

43:34It was decided that we should open sourcing it. And I think the decision has been validated by the impact that our whole do as had in the community. Now of course that's not true for all the different models. In fact, subsequently we have had models which we have not open sourced. But I think in the case of alcohol too, the decision was very, very clear. in favor of sharing it with the world in the most free way possible. For the ones that you haven't chosen to open source, if you're willing to share, how do you make that decision? There are a number of different factors. Both what will be the social impact, the scientific impact of releasing things versus what is the commercial cost of releasing something while leveraging it for commercial purposes or even the safety sort of argument.

44:18So just to give you the example, one of our recent models that we announced last year was alpha missense. And this is a model for predicting effect of missense variance. And what the model does, it produces state of the art accuracy in making predictions about whether missense variance are denied or path, what could be pathogenic. And in this particular case, we found that the predictions of the model for the human genome, for the human distance variant, like the 71 million of them, if we release that, that would serve most of the purposes, that a clinician or a biologist would be interested in.

45:01So we just release the predictions rather than the model, because the model had many other sort of users, you could run it on different organisms, there were other sort of commercial considerations. So it was felt that we could release the predictions, we could share the methodology but we will not sort of open source that approach. That makes sense. And I think at the very outset, you shared so many different projects or areas of scientific study that your team is working on. I'm just so curious because it sounds like there's been success across many. Are there any areas of science or mathematics that you've tried to address with this approach of using machine learning and AI that's not quite working, whether it be because we don't have the prior data set as VJ has spoken to that sets the foundation.

45:46I'm just so curious if there's limitations emerging in any of these fields that your team is running into. One specific area that I would love to have impact on, right? And I think yeah, I would eventually have impact on a system's biology. It's an incredibly important sort of problem to really understand at the system level how biological systems behave, it's just the data and the evaluation is not at a place where it is for maybe genomics, functional genomics, or for structural biology. Before we actually start an initiative in any of these areas, there is a huge due diligence process that we need to undergo because essentially you're making a very long term commitment and the careers and the impact of some of the best scientists and engineers that we have are being committed to that area.

46:40So we take that responsibility very seriously. And only when the impact, when we are confident of the impact of the problem, we are confident that we have a good evaluation metric to track progress and we have the raw material. the data or an assimilator to get good data, only then do we make that long term commitment towards a specific topic. The highlight the data issue, I think one of the biggest differences between AI for, let's say, language models or AI for video and AI for biology or for healthcare is that I think most of the interesting data in biogen healthcare is either dark, that there's all these medical records and so on that you just can't access on the internet, which would be very useful for understanding the healthcare side, for trial side and so on.

47:30It's either dark or it's never been measured. And that, oh, we need to do the experiments. I think having the data could be paramount. And that I think that's gonna be different than other places. Other places maybe the algorithms can really drive things because everyone has the same data more or less. I think here people will be differentiated by their data. And so the innovations will be innovations in AI I combine with innovations in data collection. And there are obviously things I've been in a interface for active learning and how can you use the data more efficiently and so on. But the data game I think is going to be huge.

48:02Absolutely. And Vijay, I'd love to just get your take. You've spoken to a few examples already, but what different areas do you wish that more attention was being allocated? Or do you just think there's a set of grand challenges that can and will eventually be solved with some of this technology? The fun thing about CASP, this critical assessment of structure prediction is that I think it also inspired all these other prospective trials and prospective studies. So there's a ton of that stuff to do and I think there's tests for predicting binding of small molecules. I think we'll see in time these types of methods do extremely well in those assessments.

48:39But the Holy Grail is in my mind being able to predict clinical trials is something where you know to understand how a drug works in human biology And that's a push me's point is that's a systems biology problem at the largest skill. And so that is the holy grail and I think we'll probably do it in parts. You could imagine even like models for specific organs or models for specific parts of the body and then we put them together. Make sure as of experts is pretty common these days and maybe that would be one approach. But however I guess done once that gets done to the point where these models are better than the animal models, I think that's where there's really going to be a tipping point and up point where we can just move much more rapidly, where we can sort of not get stymied with having to run these animal models which takes a long time, it's very expensive.

49:25And even there's crazy things like right now, there's a monkey shortage because monkeys are in such type of man to run these experiments. So I think there's all along roads to get there where these models of humans are more predictive than the alternatives, but I think once we get there, that will be a major intersection point. Wow, I did not know there was a monkey shortage, But I mean, it really is important to know, right? As a to your point, hopefully we get to a future where some of the things that we're doing in research today seem just so incredibly outdated because we just have better options.

49:56Push me what's next up for deep mind in terms of areas of interest. I mean, you're already working on so many things, but would love to just get a pulse on what's exciting for you too. I think what is fascinating about science and like in any of these fields is that there's so much more to work on. I mean, even the on -structural prediction, I just mentioned that the latest version of the Earth and Paul, the work there is on extending it to general biomolecules, like DNA, understanding RNA, understanding the interactions between small molecules, begins and proteins, like bigger complexes, antibodies.

50:33There's so many things that we can extend in genomics. We have worked on both gene expression, the coding part of the genome, like with the Bissens variance, and the non -coding part of the genome, right? Or like, critically gene expression, we have made progress, but we are not completely at the end of it, right? So there's a lot that we are doing in all these areas, in material science. You mentioned this model known, which was able to predict 400 ,000 normal stable compounds, which expands the number of stable compounds known by more than order of magnitude, right? But how do you now take those sort of compounds and then reason about their specific properties that would be useful in a particular application?

51:18So in any of these disciplines, we are not targeting one specific milestone. You're just saying, here is a topic and the long term sort of road map is to think about a paradigm shift in how science is done in that area and moved towards a more rational modeling based approach and tackling some of the problems that are encountered here. So there's a lot that needs to be done. And we are just trying to focus on some specific areas and then new areas come up if the raw materials are there in terms of data. And if you have to, on the valuation metric, we are constantly reviewing them as well. That's amazing.

51:56I haven't done as much research as UVJ, but I did do a summer of battery research and materials research where we were trying to discover new sodium ion transition metal materials and my summer was literally, I mean, this is when I was in college, so I wasn't very advanced, but it was literally like finding a paper that documented how to synthesize this material and the kiln, mixing it up, creating a little battery, doing it in the glove box and running it and just seeing how effective it was and obviously in cases, it was very ineffective, but every so often we found a material. It was truly just trial and error, trial and error, trial and error.

52:33And when I see papers like this that do things in a completely new way at scale, way cheaper, you don't have all of these university students just in a glove box day and night, it's so exciting. The end point for me is like, as we talked about, we're kind of in the middle of this journey and this technological journey, this cultural journey, in these cultural shifts and that it's going to feel like the big goals that I've laid out, let's say clinical trials, things and systems biology, that's so far off, right? And it's going to take a while. But we can get a lot done in 10 years collectively, 15 years, you think about where we were five years ago, 10 years ago, 15 years ago, now 15 years ago, people weren't really even talking that much about deep learning or just beginning.

53:14So the goals that we have are lofty, but I think we're right in the think of it and all that I think is very doable is just now building that tower once every time. It'll be fun to have this chat again in five years. Hopefully sooner. I think one sort of thing that having been exciting in the last few years is the rise. Of course, there's a lot of excitement about LLMs and foundational models and so forth. If you look at the impact that's going to have on science. Now, in most of the projects that I was talking to you about, you're working with structured data, data, either which was collected or in the case of some of our fusion work data that was simulated.

53:55But with the rise of foundation models and elements, that opens up the possibility of now using unstructured data to feed these models. And so that really opens the goal for a large scale ingestion of scientific knowledge into the models. And that is a very exciting direction that will I think bring a number of other problems now in the feasibility zone, which previously were not there. Of course, there are challenges with understanding uncertainty and sort of hallucination and all these sort of technical problems need to be sort of addressed, but once that is done, I think the impact that's going to have on models for scientific discovery would be amazing.

54:41So that's another reason to be excited for the future. Absolutely. And all of the problems you just mentioned are also opportunities for people to go and fix and be a part of that whole ecosystem. So this has been really wonderful push me VJ. Thank you for as you said, getting people excited about what's to come because I think these two fields intersecting what a time to be alive here in 2024 to kind of be a part of it like you said, VJ we're in our equivalent 1920s. So hopefully people in the 21 20s will look back at this fondly. Absolutely. Yeah. If you liked this episode, if you made it this far, help us grow the show.

55:19Share with a friend or if you're feeling really ambitious, you can leave us a review at ratethispodcast .com slash basic ccd. You know, candidly producing a podcast can sometimes feel like you're just talking into a void. And so if you did like this episode, if you liked any of our episodes, please let us know. We'll see you next time.