AlphaFold: Grand Challenge to Nobel Prize with John Jumper

0:07I think we'll get this ability to poke the cell in exciting ways to interrogate it. And every time we develop that, we'll develop a more interventional understanding of the cell that we will bring forward to medicine and synthetic biology. Welcome to Google DeepMind, the podcast. I'm Professor Hannah Fry. Now, today we are talking about AlphaFold, one of the most extraordinary technological breakthroughs in modern science. A tool that has been described as the most useful thing that AI has ever done. And in truth, that might be an understatement. This is a Google DeepMind AI system that solved one of biology's grandest challenges, predicting the 3D structures of proteins, the fundamental building blocks of life.

0:50Its latest version, AlphaFold3, can now model the structure and interactions of all of life's molecules with unprecedented accuracy. And the impact has been seismic. AlphaFold has mapped hundreds of millions of protein structures and more than 3 million researchers across 190 countries now use its database. It is transforming drug discovery. And in 2024, the Nobel Prize in Chemistry was awarded to Google DeepMind's Demis Osalvis and John Jumper, who is our guest on today's podcast. Now, this is a story that we have been following on this podcast since season one, which was nearly eight years ago, long before it hit the headlines.

1:31So if you are coming to AlphaFold for the first time and wondering what all the fuss is about, you can find our previous Explain It episodes linked in the description.

1:43Welcome to the podcast, John. Oh, it's exciting. I don't think I've interviewed you since you won your Nobel Prize. Where were you when you found out about it? I stayed home because I was nervous enough I thought there was a chance, like one in ten a chance. And so I figured I would be disappointed at home. And I was kind of just sitting in the bed. my original plan was I'll sleep through it and a phone call wakes me up, then I've got the Nobel, but I couldn't sleep. Because you knew the day that the phone might happen, right? You know the day. I knew, in fact, the kind of time that it was scheduled to be announced at 11.

2:14I knew that winners were called about an hour beforehand. So by about 10.30, I said, oh, well, I guess not this year. And I told my wife and she goes, no, no, wait. And as she's telling me to wait, my phone lights up with a phone call from Sweden. And thankfully, it was not the world's meanest prank call. And yeah, it was just kind of this extraordinary thing. And you answer and they say, is Dr. John Jumper available? Yes, I have some wonderful news. Great. Can you please hold? Right? So they get, I think, Hansel will agree. Well, they make you hold. Well, I think they were trying to read. Part of the problem was they didn't have either Demis or my phone number initially.

2:54So anyway, they ended up calling us very late, But then they were finally arranging and they pulled the person on and he says, you know, I have some life-changing news. And they don't say the word Nobel for like 60, 90 seconds, which was the longest minute of my life as there is no other explanation for this call. And I remember the very first thing I did is run to get a shower because I knew I was going to get no time for the rest of the day. But after that, you know, it was announced. You come in. You see the team. You have this amazing kind of celebration. We bought the local waitress out of sparkling wine.

3:26And only the best will do. It was, I'm not a connoisseur. And we were celebrating with friends. And there was just this incredible kind of party just across the floors of our building. It was amazing. The thing is, it's an extraordinary story of you as an individual, right? Because your first PhD, your physics PhD, you dropped out, right? Yeah, yeah. And so going from that, which I think must have been quite a hard experience to live through, to like being a Nobel Prize winner and having your tool being used in tens of thousands of academic papers? I mean, I will say dropping out was a very lucky thing for me.

4:05I was doing the wrong thing. I didn't really want to. And so I just left. And because I left, I actually fell into this computational biology group that was doing amazing work on custom computer chips to simulate proteins. And then I go back and I do my PhD now in chemistry by another set of accidents. and I didn't have those great computers. So why not get into AI? Why not try and use sophisticated algorithms to make up for a lack of computer? I have to be the first person to get into AI because of a lack of computational capability rather than an abundance. And then I got lucky enough to kind of find a job that had something to do with everything I'd ever tried to do in my past.

4:44And it worked out and I get a Nobel. Do people react differently to you now then? Oh, I mean, well, I think there's all sorts of people. You know, they're the people that I did my chemistry PhD with who knew me as a, you know, pretty good physicist and a lousy chemist. There was, you know, the people that I work with every day. And I'm still, I think, just John, but now John with the Nobel, so he's busy. But then there are all the people I meet. I mean, I would, you know, I'll get on phone calls. And a surprising number of my phone calls start with, it's such an honor to speak with you. And I sometimes think, and also with you.

5:17There's a certain type of deference or at least excitement, and it's a symbol of this giant AI world and what it can mean in terms of applying AI to solve real-world problems. And then you're a Nobel Prize winner, so you're allowed to have an opinion on anything, and it's supposed to be a bit valid even if it's not. So people want you to show up to things just so that you can symbolize that a Nobel Prize was won, and then you're done, which is not a very satisfying thing to do as a scientist. but you have this platform that maybe you can use to affect how the public thinks about science, how it funds science.

5:50So all of these things kind of roll together in this wild combination. I'm, I would say, roughly at the midpoint of my career in terms of kind of time since undergrad and time since rough retirement. And so I've got to figure out what to do in the second half. And that's a, you know, that's a fair amount to live up to. Yeah, absolutely. A lot of pressure going The thing is, is that we're, I mean, we're still only five years on from that CASP breakthrough, really, when Alpha Fold 2 smashed the prediction challenge. Did you realize at the time the potential significance of the work that you were doing?

6:26We were sure of two things and totally unsure of some others. I think the two things we were pretty sure of, we were very sure it worked even before we entered CASP. We had measured very well. We knew about how we would do in CASP. That we understood and we were careful. We knew that we had solved this grand challenge, but the normal thought in a grand challenge in science is that you'll solve it and it'll be a great celebration. And then you will go build effective, useful systems that use the ideas that enabled you to solve the grand challenge and that this was kind of the beginning of an era.

6:58I think the real shock to me is those weights that we train, that system, that piece of computer software has been so incredibly practically important to scientists working in this field to this day that the actual bit of software is used that makes this difference in all these different application areas, all this different type of science published on top of this as a black box computer program. And the extent to which that has entered into scientific practice has been really, I think, beyond my imagination. Yeah. I mean, it's really difficult to overstate the genuine significance that this has had.

7:34I saw one thing where AlphaFold was described as the most useful thing that AI has ever done. That hasn't sort of landed with the public yet, has it? I think it's hard for people to appreciate, and you work in science communication, how very hard science is, how very hard curing disease is. we have to work extremely hard to get smaller bits of knowledge about how, say, the cell works, how the body works, for protein structure prediction or protein structure, what alpha fold does. I think it's hard for people to appreciate this process takes a year in the lab. I've seen PhD theses that are progress toward determining the structure of X.

8:14And that doesn't mean they finished it, just they feel like they're a little bit closer and they need to graduate. Of one protein?

8:48And I think the right kind of way to think of alpha fold is not certainly that we've solved all problems in biology. We very much haven't. I think for this slice of biology that cares about what structures in the cell look like, structural biology, maybe we've made it 10 % faster overall across the whole thing. We've amplified this enormous effort and societal expense. And then ultimately we will have transformative science. And there are certain narrow areas, say protein design, that are just being transformed by this understanding. I think one of the ways for me that really demonstrates just how important this was as a breakthrough was the way that biologists reacted when you published the 200 million protein structures.

9:28Just tell me a little bit about that because you just put it out there, right? Oh, yeah, yeah. So the original release was a bit smaller, but was still I think it was like 400 ,000. What I remember was there was maybe a week in between when we had put out our code and the real experts were playing with it. And they were like, this really works on hard problems. But all the other biologists were, no, no, no, these can't be real hard problems like I work on. And then, of course, we put out this huge database, AlphaFold database. And people are like, well, let me just see how dumb the AI engine was and click on their protein of interest, I think, expecting to make fun of it.

10:00And then they sat there and they were amazed. I saw this comment from someone on Twitter saying, how did they get a copy of my structure? How did DeepMind get this thing that I had done and not yet published? Like they couldn't believe that this was like literally a machine doing years of painstaking work like this all at once kind of a flash. And what was I think also amazing about it is how rapidly this turned into a community understanding of what AlphaFold does, what it doesn't do, for example, not sensitive to single amino acid changes and how to build this into their workflow and do work on top of it.

10:42I thought it would take kind of years for people to really figure out what's the right way to build it in. How do I make sure that I look at the confidence measures where AlphaFold is saying this looks like a reliable answer, this doesn't. This happened within a matter of months, that science is a community, that people developed this incredibly rapid, not totally perfect, but actually really good understanding very, very fast. And so people were doing excellent science on top of AlphaFold. You know, we released this in, I think, July or something like that. And people were almost immediately doing really excellent science on top of it, that really cool work was coming out by the end of that year.

11:21And I think that's just a testament to how much scientists are looking for really effective tools that help them push knowledge forward. And then when they find it, they use it well. How do you stay on top of the work that people are doing with AlphaFold now? I mean, because it's so embedded in the way that people do biology now that, I mean, presumably they're not sending you an email every time. Thank goodness. I actually end up, I still pretty often just put the word alpha fold into, you know, searching on whatever X and just see the random work. What I love is the random things that pop up that says, oh, we use this to do this weird thing.

12:00I think the other way, one of the nice things about being at a company is if really cool things happen, someone will notice and they'll post it to this chat room that we have to collect various, you know, things that people find cool. And so it's so valuable to collect that experience. And it's so much fun, right? You feel this kind of vicarious ownership of a little bit of that work. Well, OK, tell me then, what is the most random, unusual use of alpha fold that you've come across? One that I really love is this protein in bumblebees, and they're trying to understand bumblebee populations, you know, reproduction, but their biology to try and, you know, enable pollination and understand things like colony collapse.

12:40And so there were some important proteins involved in the honeybee life cycle that they were studying with alpha fold. And you can see ultimately how this kind of leads on to bee conservation. And I think it's so interesting to see the structural biology of this echoing into all these things that we care about from food to industrial production to everything else. They're all connected because it's all the same biology, right? You know, plants, animals, they have basically the same proteins. We were definitely thinking most about human health. We weren't thinking, how am I going to help honeybee populations?

13:16But here we are. There was another really nice story that people were trying to understand human fertilization, when an egg and a sperm meet and come together and eventually fuse. Right. So they want to find the exact proteins that were involved in sperm sticking to eggs. In sperm sticking to eggs. And they, I think, had the full picture of all the egg proteins, but not of all the sperm proteins. And in fact, there were two independent groups that did this. And they said, well, there are only 2 ,000 proteins that we know that are on the outside of sperm. Why don't we just try all of them and see which ones stick to the proteins that we know are on egg?

13:50And if you think about doing this experimentally, that's like, well, I'll spend the next two millennia, you know, a year at a time,$100 ,000 each in the next millennium, and I'll get a nice paper in Nature. That's not a feasible approach. But AlphaFold is pretty fast, and they had some computers available, so they tried all of them. And then they both came out with this one protein, T-MIM something, I can't remember the number. But this was the one that they didn't know what it did before, and now they find out. AlphaFold says it sticks to the egg and that this is how kind of the first step of fertilization.

14:23And so, of course, they don't just trust AlphaFold, right? It's a computational system. So they went and they said, well, what happens if I remove that protein or if I change that protein? I find if they change that protein or remove that protein, then sperm and egg will get close, but they won't fertilize. So you go from this broad hypothesis. There's some protein on the surface of sperm that does this. AlphaFold says, I think it's this one. And then you go do your detailed experiments to confirm. And now you can think about questions like infertility. Now if you see mutations in that protein, maybe that's a cause of infertility.

14:56Maybe we can think about treating that. And we go from kind of rough hypothesis, alpha fold in the middle, confirm with experiment, and now maybe we can think ultimately about something like drug design on top of this. But we have to get this biological understanding first that we bring meaning to all those pieces in the cell. And that's what alpha fold, I think, really helps with in the early stages is bringing meaning to the parts of the cell. And then later, companies like Isomorphic use it in order to build small molecules that have targeted effects. We should talk about the difference between alpha-fold 2 and alpha-fold 3, though, right?

15:32Because, I mean, alpha-fold 2 was like predicting the structure of proteins from these strings of amino acids. But, of course, biology isn't only about proteins, right? You've got all of these other biomolecules. You've got DNA. You've got RNA. You've got, like, small drug molecules, for example. You've got ions, you know, charged particles, et cetera. All of these are interacting with each other. So how early in the process did you know that you needed to change the fundamental model of alpha-fold-2 in order to incorporate those additional molecules? So even before the world knew about alpha-fold-2, we were sitting there and dreaming.

16:07And we were dreaming for two reasons. One is that we had a lot of, for example, proteins that exist naturally in what are called complexes, multiple proteins stuck together. and sometimes there's no real way to predict their structure without predicting them all together. So we're already thinking about this multi-protein problem. But we also almost immediately said, well, a lot of proteins, for example, as you say, bind drugs, small molecules, maybe 20 atoms. You can think of aspirin, right? It sticks to a protein. And we knew that this was really important. We said, but later. And we started to talk about this kind of dream about this goal we would call whole PDB, right?

16:41So the protein data bank, the PDB, is the data source we use. but we take it in and we threw away a lot of the things. Oh, this has RNA or DNA attached to the protein. Well, let's throw away the RNA or DNA and just have alpha fold predict the protein. Because you can't handle those extra molecules. We couldn't handle that complexity that we were very like driven by, we have 20 amino acids that produces 20 types of structures. And then we will predict and all our code was kind of based around that. And we're like, it's a challenge for later, but eventually we'll start doing it. And one of the things we almost immediately realized is a lot of the decisions we made in AlphaFold were very good and very helpful and very annoying to extend to more complicated things.

17:22The other bit of work is that we were trying to figure out how to simplify AlphaFold. And we thought, OK, well, AlphaFold is complicated, but maybe there are some things we can remove. How has the architecture shifted from AlphaFold 2 to AlphaFold 3 then? So there's a lot of changes, but I would say there's two big themes of changes. When we were trying to handle much more of the kind of the DNA, small molecules, etc., we adopted this thing called a diffusion architecture, a different way in which we handle our uncertainty. And I can tell you more about that. But then I think the other one was really thinking a lot about the role of evolution and evolutionary data.

18:00Well, let me ask you about that then, because this is one of the things that I remember being quite key to AlphaFold II, this idea that actually proteins have evolved. in lots of different creatures numerous times. And actually there is like some clues about the evolutionary history that will indicate where amino acids are likely to end up in the final folded shape. So that even if you're starting with a string of amino acids, you're not going in totally blind because I mean, this sort of stuff's happened before, right? And like that ended up being quite a key part of the model, but it was also, I think, potentially one of the parts that made the model quite, I don't know, like inflexible to other molecules.

18:37Is that fair? So AlphaFold 2 used evolutionary information in this exuberant way. At kind of every part of almost every block, it was saying, and here's the evolutionary information in case you need it. But a lot of what we studied in AlphaFold 3 that we knew we were moving toward didn't have evolutionary information. So we were shouting at it with nothing, and we were kind of worried that this was both slowing down the network but also possibly leading to some bad dynamics in how it works. And so we decided to just take that out of most of the network and otherwise emphasize the geometric information, the thing that really is always there.

19:15And that turned out to work exceedingly well, actually better than we expected. I wonder if there's an analogy that we can use here, okay, for the difference in this architecture, right? Let's imagine that you're planning a wedding and you've got to do the seating plan for the entire wedding, right? And you have all of these guests, those are your amino acids, and you have to sort of work out where each one's got to sit. And there's a couple of different ways you can think about this. So you could think about pairwise interactions. So like this person sitting next to this person, is that a good interaction?

19:42But then what does that mean for this person over here or that table over there? But you could also potentially think about the history of what you know about those people. How am I doing so far as an analogy? I can go for this. I think, you know, you know, some people went to school together. some people used to date and had a terrible breakup right those might be don't sit them together you probably don't want to sit them right next to each other and unless you're really looking for sparks but yeah I think before we just talked about where the wedding guests sit now we think about where the flower arrangements are that's nice right you know that we think about all these other things that come together to become the reception dinner in in this analogy then alpha Fulphal 2 was very focused on the history of the guests, right?

20:28It was sort of like continually checking, looking at where they might best fit based on their past. And that's great for proteins, but it's difficult once you try and include other elements of the reception, like the flowers, like the food. And once you start bringing in other biomolecules, you don't want to focus so much on history. I think that's all true. I think one thing, though, I would say is that we kind of always made this history available, this evolutionary history and the analogy kind of what we know about their past. And what we would find is that AlphaFold, we think, was not relying on it much other than at the very beginning, that it was like saying, oh, well, these people should probably be together.

21:06These people should probably be apart. I know a couple of things. But then it kind of trained itself to ignore it. And so by inspecting and seeing that we're probably not using that information, maybe we should stop attaching it constantly into the processing. But then as a result, you managed to massively simplify the model. I wouldn't say we massively simplified, but we made it massively more accurate. And suddenly we were doing new problems. And in fact, we made light adjustments and then we made a much better model. And then it turned out that even that protein-protein problem, something that has nothing to do with ligands or nucleic acids or anything else, even that protein protein problem got massively better from this kind of science and improvement.

21:45Also in AlphaFold3, you have this diffusion element. And I mean, I know that that comes up a lot in video generation, for instance. How does that fit in here? So diffusion is this different idea in how you train a neural network. The AlphaFold2 system was really heavily based around the kind of shape of a protein backbone. In AlphaFold 3, we went to diffusion where you basically say, here's a blurry image of the protein. I kind of took all of the protein and added some noise, some error, like you looked at it in the wrong prescription glasses. And then guess the right answer. And you have it constantly refined.

22:21And so what this gave us was a really great understanding of local geometry, of how to make things extremely precise, because that's what it does at small scales. and this way of tackling big systems. And that gave us a kind of new approach that we didn't have to get so involved in the details of exactly how proteins look because they're different than DNA. They're different than RNA and small molecules. And the upside is that it made it really, really easy to kind of handle this wide universe of things that we study. The downside is that it led to a higher rate of hallucination, of weird stuff appearing.

22:57And so then we needed to handle that in different ways. Well, this is one of the big differences between AlphaFold 2 and AlphaFold 3, right? That you have this introduction of the stochasticity, the potential for hallucinations. How much should people be concerned about that when they're using it? Is there a danger that they think because AlphaFold 2 was so on the money that they think of AlphaFold 3 as though it's some kind of oracle? I think one of the wonderful things about biologists is that, you know, as scientists, they're deeply sceptical of their tools. AlphaFold 2 did have an advantage that wrong answers often look stupid.

23:30They didn't – no one looked at that and said that's definitely a protein, whereas wrong answers in alpha fold 3 are sometimes more plausible. But I think that people have gotten really good, not uniformly perfect, but really good at saying, well, alpha fold 2 is also telling me how accurate it thinks it is in the confidence measure. I should also use that. And so it's this kind of social knowledge. There's no experiment or tool that scientists use that is without limitation, right? even experimental structure determination has all these known faults. And so scientists, I think, use it relatively well.