Arc Institute's Patrick Hsu on Building an App Store for Biology with AI

0:00One of the things that we, you know, that the field of computational biology is often asking is, you know, if you have a genetic mutation in your genome, if I sequenced you, whether that's via, you know, 23andMe or, you know, or some other genetic tests, right? You'll find mutations in your genome. How do we actually interpret those and understand, you know, what the functional consequences are, right? Sometimes you'll get a rare genetic disease. Those are causal genetic mutations that are known to cause a devastating disorder, that might be muscular dystrophy or cystic fibrosis or breast cancer.

0:35But most of the mutations that you have, they're sort of this, we call them variants of unknown significance, which is fancy kind of scientist - You don't really know what the hell is going on, right? And it turns out the model has an opinion about those mutations and what the hell is going on with them and it turns out it's sort of state of the art in doing that. [♪ OUTRO MUSIC PLAYING [♪

1:11Today we're joined by Patrick Xu, a pioneer in genome editing, CRISPR technologies, and the emerging field of generative biology. He's the co -founder of the Arkansas Institute. where cutting edge AI and biology converge to reimagine scientific discovery. Patrick and his collaborators created EVO2, a revolutionary biological foundation model that can interpret and generate genomic sequences across all domains of life. By training on the fundamental information layer of life, DNA itself, EVO can identify patterns from genetic code at scale and predict effects of both coding and non -coding mutations that can mean the difference between health and disease.

1:52In this episode, we'll hear how Patrick's vision goes beyond creating better drugs to building comprehensive understanding of biology at all scales. Patrick, welcome to the show. Thank you for coming. Thanks for having me on. Excited to spend some time with you today. I think maybe the most obvious thing to start with is, you know, people have heard about CS and Bio for the longest time. Now it's all about AI and bio. We're the results. Like what should we actually be expecting to see? And - Where are the drugs? Why are we not seeing the drugs yet? It takes time, right? Well here's the thing. Even if we had perfect drug design molecules coming out of these pipelines and fancy models, you can design a trillion molecules, ten trillion, right?

2:42But you still have to actually test them. Right, initially in animals and then in people, right. And so that's the real bottleneck. And even if you pack top of funnel with all the things, it just takes years to actually go through the regulatory apparatus, right. And so I think there are a few intermediate checkpoints along the way in order to kind of realize this potential. But it might be worth taking a step back and just saying, you know, this is a bit of a soapbox of mine that ML for bio is not just drug design, right. This is actually ultimately, I think, a very important, but narrow part of the potential of biology and not just as a field of STEM and in the way that affects human lives, right?

3:21And do you mean basic, just like understand the human body or where else do you think the applications are? We'll end drugs that treat all of us. Yeah, one of the things that motivates me academically is the idea that we actually have a unifying theory for biology, right? So unlike the physicists who have been kind of scrimping and kind of poking for one for a century, right? We have this in biology and it's so obvious that we Find it, you know, sort of an just an obvious force, right? This is of course a solution, right? And so it acts on biology across all of its different length scales from entire planets Right biology can for example, terraform planets, right all the way down to you know ecosystems and populations to individuals, to our tissues, to individual cells, to individual molecules.

4:11That's this unifying force that is actually very deep and rich and actually you can learn a lot from. How do we activate that unifying theory? How do we put it to work? We've been thinking about this in the lab and recently have been training a series of models that we call EVO inspired by these forces of application that tries to connect biological sequences using this sort of modern sequence modeling paradigm directly to biological function, with the idea that evolution passes down its effects of natural selection throughout generations of life via DNA mutations. And so last year, of course, multiple Nobel prizes were awarded for AI, and for AI in biology, in particular for protein design, and for predicting the structure proteins to David Baker and to Demisosabis and John Jumper.

5:05But if you read those citations, they both explicitly state for proteins. And we love proteins. These are, of course, some of the most important and fundamental molecular machines, but our realization, for me, as a genome biologist, if you will, the ideas that proteins are encoded in DNA, along with RNA and with regulatory DNA, and all of the things that you need to make life. And so we asked, could we train a model on genomes with a long context models that it could reason over all the different bases and molecules are embedded inside of genomes to learn about the molecular interactions and how they lead to biological function?

5:48Now, that was very scientific or academic, right? But we can talk through specific examples of what we were able to actually do with this in a way that grandma can understand, you know, like predicting the effects of breast cancer causing mutations, right? It actually is best in class at doing this, right? Or being able to design new CRISPR gene editing systems. Or, you know, I think, you know, in addition to zero shot capabilities, I think people are building really an app store for biology on top of all of these kind of foundational layers, one of the kind of funny things and modeling today is how everyone's model has to be more foundational than someone else's model.

6:32There's a bit of a pissing contest. That's happening, right? But maybe our model is more foundational than the other models. Because you're DNA versus protein. And then below that, there are these all -atom diffusion models. So maybe those are even more fundamental. I don't really know. I think what matters are the capabilities and doing something that actually feels useful and think, you know, those are some examples of what we thought was cool and useful from the model. Can you actually walk through a couple more of those use cases like which are some of the most exciting ones? And why is it possible today with EVO but wasn't possible before?

7:06In the model's open source and so where has it been picked up? Like where are people running with it with some of those use cases that Josephine mentioned? Yeah, so the model, I can just maybe talk about what the model is. Yeah. So, you know, it's an auto -aggressive sort of multi -convolutional hybrid model, right? But you can think of it like, you know, just a really efficient long -context model that's trained at least in this version, auto -aggressively, right? And basically, it does this next token or next base prediction, and it turns out just like in natural language or in vision or in robotics and embodied intelligence, this general machine learning paradigm is able to find higher order patterns.

7:47And so just like if you're doing next word prediction, you can learn about grammar or color and world navigation. You've seen to learn some rich set of representations about biology by predicting the next base or the next amino acid residue or the next gene. The model learns something about the molecular logic that gives rise to itself. And so one of the things that we, you know, that the field of computational biology is often asking is, you know, if you have a genetic mutation in your genome, if I sequenced you, whether that's via, you know, 23 and me or, you know, or some other genetic test, right?

8:30You'll find mutations in your genome. How do we actually interpret those and understand what the functional consequences are? Sometimes you'll get a rare genetic disease. Those are causal genetic mutations that are known to cause a devastating disorder. That might be muscular dystrophy or cystic fibrosis or breast cancer. But most of the mutations that you have, there's sort of this, we call them variants of unknown significance, which is fancy kind of scientist. Yeah, we know what the hell is going on, right? And you know, it turns out the model has an opinion about those mutations and what the hell is going on with them.

9:08And it turns out it's sort of state of the art in doing that. Interesting. Wait, what's an example of one of these mutations and what, what did the model discover and how do you verify that what a discover was accurate? Yeah, yeah. So, so, you know, one example that we showcase in the paper is a gene called Brocka one, right? It's sort of a famous gene that's known to cause breast and ovarian cancer. And if you have the sort of specific causal mutations in Brocka 1, many women elect to get double mastectomies. This is obviously a serious and major life decision and medical decision for you and for your family.

9:42And the question is, if you don't have one of the known to be benign mutations, so you're fine, and you just go ahead and get an annual mammogram and just check and monitor, right? There's this entire middle distribution of these VUSs or variants of unknown significance. And there is this kind of gold standard database from the scientific literature it's known as ClinVar and it basically has a list of all the different genes that are known to cause disease and which of those mutations in those genes can cause disease state or not. And we can basically use this as a ground truth database to assess the predictions of the model for new mutations that you introduce into the gene and whether or not those would be pathogenic.

10:29And so when you develop a new type of model, you have to create a lot of the e -vows as well. And so that was actually something that we put tremendous effort into. And obviously you guys see this horizontally across AI in many different domains as folks will build things to the benchmarks. No one really, I think, likes building benchmarks. It's really gory. It requires a lot of taste. It takes a lot of time. You have to continually update them as the models get better. And we dealt with a very similar sort of challenge here, which was making eVals that would be similar to the AGI or intelligence eVals, where it actually feels meaningful when you're actually able to do it like Amy or Putnam problems or things like that.

11:16what would be the equivalent of demonstrating true biological understanding that a cell biologist would feel emotion if you were actually able to solve that, right? You know, what would it look like to make all molecular biologists feel what the NLP people felt a few years ago, right? That's sort of the core of, you know, what we're kind of noodling through. And I think you mentioned this briefly, but you know, there's, you guys are working on the DNA layer and you guys didn't actually do anything in the lab. you didn't have a lab in the loop, like there was no RL, talk us through the decision to do that, and then kind of the decision for people who are doing protein models or affinity models, a lot of them have a lot more lab in the loop, talk us through the differences between some of those models too.

12:00Yeah, so we started with DNA because we think it's the fundamental information layer of life, right? The second is it's also just pragmatically where we have the most data. Like data, yeah. Where does the data come from? It comes from the entire scientific community, right? And so there are these open source government funded and maintained databases known as the sequence read archive, where basically when you publish a paper, you have to submit your data, and all of the sequencing data that's been created over the last 25 plus years goes into these databases. And that has all the genomes that the community has ever sequenced for bacteria, for bacterial phage for viruses for humans monkeys fish flies you know the entire Noah's Arctic or menagerie whatever right we've got all those genomes and you know so so the experiment that's already been done if you will is the experiment of evolution right that you know there are different mutations that are what make us different from each other that's human genomic variation but also the mutations that make us different from chimpanzees or from worms or from bacteria.

13:14And the model can just look across this trillions of tokens large data set and learn those patterns. And that was sort of the insights of this sort of evocere of models that we've been training at ARIC is to kind of ask if you could predict that next base, right? then that might be the difference between being healthy or having a sickle -cellenemia mutation, right? Or it could predict the next amino acid residue, not could be the difference between having a catalyptically active binding pocket for a key enzyme, right, in your, you know, body physiology, or that being a null mutation where that thing doesn't work anymore.

14:01Or it could also be the next gene. So these are different levels of abstraction that completes some biosynthetic pathway, where it's a different gene that's been removed by some transposon or jumping gene mobilitarian element that's excised is some sort of viral interference, if you will. So there's just across large databases, you find new patterns, and it turns out those seem to be biologically meaningful. So if you're going from DNA and you know the function, in many ways you mentioned even the DNA model can actually predict binding affidimities. Do you even need the structure, the protein structure models at all as intermediate step, or can you just go straight from sequence to function?

14:47So structure is another way to have an abstraction of function. So you have concepts of convergent evolution, for example, where something has similar function, but they have different sequences and slightly different structures that act out the activity or function of that protein. And so this sort of sequence, the structure, the function, token, or mapping of protein language modeling think is very beautiful. It takes advantage of the central dogma of molecular biology. The full series. Right. Well, it takes advantage of just our supervised textbook understanding of how biology works. Right.

15:30Now, the interesting oxymoron with these models for me is the way that we use them is, you know, just like you use chatch -e -b -t, it's text in, text out. You know, Evo is D -N -A -N, D -N -A -Out. Yeah. And it turns out, we don't speak D -N -A very well. This sort of imagine if you were using a model like chat chatt -t -b -t in Russian, right? But 1 % of the words were in English. Right? That's kind of what it feels like. That's the vibe of using Evo is actually you don't really know what's going on. And so you have to build lots of annotators and... Interpreability. Yeah. The like techniques to try to interpret and read what's happening.

16:11And so the way that we even use and prompt the models is really primitive, right? And so the way that we do fancy prompt engineering workflows to, you know, get more utility out of these models is something that we're just in the very early innings of exploring how that works with these biological language models because we speak DNA with an extremely heavy accident. Who do you think will do some of that work? Because the model is currently open source. Do you envision a company formed around this? Will it be individuals who are at these former companies who learn how to prompt this? Like how does that ecosystem evolve?

16:47Yeah, I hope everybody, right? First of all, right? And I think the tools become useful when they meaningfully lower the energy barrier adoption. It's like a it's a catalysis type of activity where everyone uses blast for sequence alignment. Everyone uses alpha -fold to look at protein structure. Everyone uses CRISPR to do gene editing. Everyone does NGS to read DNA or RNA. So I think there will be a zoo of different models for not just modeling molecules or mapping sequence to function, but I think every step of the scientific method. Right, and so I think, you know, if 2025 is the year of AI agents, right?

17:34I think you know, there's lots of interest in agents for science and not just agents for interpreting molecules but also for doing the meta aspect of how scientists work and operate. Yeah, and we're also very excited about that at ARC and recently released some of our first AI agent work. What is the most important role for ARC to play in all of this? Um, we started the institute to be able to have this mothership that is able to attack long -term research capability breakthroughs and to be able to have the long -term thinking and the multidisciplinary expertise in order to actually execute on these goals, right?

18:13I think if you look in biology, one of the interesting things is that a lot of the biggest mechanistic or basic science breakthroughs do happen in a university context. I would say that's interestingly kind of in contrast to what happens in AI. Yeah. RCS, yeah. Yeah. What do you think that's the case? That might be a 10 -hour podcast, right? One minute. I have a lot to say about this, but I think in short, it does happen in universities. I think, you know, 30 years ago, the questions that basic science was interested in and industry was interested in were quite different, actually. And I would say today they seem to heavily overlap, right?

19:06And there are some things that, you know, folks don't typically do an university lab, like, you know, people on tennis study, PK or talks or CMC, right, or, you know, hardcore drug manufacturing type things, but folks are interested in molecular glues, induced proximity, and degraders, and new drug concepts. Folks are also interested in new machine learning models, and new delivery mechanisms, and inflammation, and stress, and all kinds of things like this. So there's much more heavy overlap, but I think the way that the type of product that that selects what you do upstream is very different.

19:45They're structurally different. You know, end of the day, you have to optimize for grant funding and first or co -first author papers and, you know, that type of stuff in the academic setting, whereas, you do have to make a drug and have, you know, dozens to hundreds of people lying behind a molecule or a program to reach the Holy Land, right? And I think that does lead to differences in strategy. I don't know personally, by the way, I think people really over emphasize what's academia and what's industry and the realities these are heavily overlapping distributions today and help people operate.

20:25I think the differences are a little overblown, but they do have fundamentally different incentives that drive different behaviors. A lot of what we try to do and blend our academic side of the house with our technical staff side of the house which is built much more like you'd see an industry. It has been part of what we hope will make ARCA model that others want to copy or replicate or propagate. Yeah. It makes sense. You've had some fun people come through on the technical side of the house, including people like Greg Brockman. Yeah, no, it's been a joy. Yeah, so Greg joined us during his sabbatical from OpenAI.

21:07We're really the first vacation that he had ever taken since starting OpenAI. Of course, it's to work for him. So actually, the funny story about this, hopefully Greg will allow me to tell it. But when he first came by Arc, and we were talking to him about biological language modeling and how the machine learning breakthroughs of all these other domains might just pour over directly lead to understanding molecules. He was really jazzed by this and also by the idea that his very specific capability and expertise would be really meaningful in actually making this happen. But initially, he was saying, this is really the first vacation I've ever taken.

21:57I promise too much. I might not be able to spend so much time on this. I need to take Anna on vacation. And then Anna gets up and goes to the bathroom. In the middle of this very long meeting, and then he's like, all right, here's my email. Get me on the repo. Of course. Of course. Yeah, just built different. It was such a joy to learn from him. What have you found works well in getting the different disciplines to work together as one unified team. Yeah, I know it's an interesting question, and we've thought about this deeply at ARC as a convening center, you know, not just between the three flagship research universities here in the Bay Area and Stanford and Berkeley and UCSF, but also between basic science and the biotech industry, but also biology and the technology sector.

22:53For example, you know, our CTO Dave Burke just started a few months ago and has really kind of been leading the computational modeling of virtual cells where we're trying to simulate human biology with these AI foundation models. And Dave used to, he actually is a PhD in biomedical engineering from many moons ago. It was most recently ran engineering at Android and Pixel. And so I think we have also built an entire operational side of the house from finance to legal to the lab ops, to facilities, to university relations and academic affairs, and we run around space, around administration, around ops.

23:35And we try to do much of that like a tech company, right? And we've also, I think, recruited in on the ops side of the house many people who maybe ordinarily wouldn't work in a basic science or discovery setting, but are kind of motivated by the mission to be able to take fundamental breakthroughs and have a product sensibility where we can get these out into the real world. And we're not optimizing it arc for nature and science papers, right? If you give Berkeley or Stanford professors millions of dollars to do more science, that's almost the default expectation and output, right? And what we really care about are things that could be tangible, right?

24:18And the real world. So does success look like just more people using the products you create or what is a success for our Institute. I think there are lots of ways that you can parse scientific productivity. Journal publications is of course a important and fundamental part of sharing work with the community and pure reviewing it and all of that good stuff. But technical blogs, code reposts, protocols, are just platforms that people can use. And I think we want to make technologies and platform capabilities that are broadly useful to be able to create new mechanistic insights, but also actually try to cure some diseases.

24:57And I think over time, if we can be an Edison shop that's inventing or finding lots of cool things, there hopefully will be real world valley in those things, and there are lots of partners who specialize in this. So you can be curing diseases earlier talking about even if you had perfect drug design or infinitely accessible, infinitely intelligent drug design. There's still a long process that has to happen after that before you can make an impact on humans. Can you talk a bit about where you see opportunities in that value chain? And if you had a magic wand and you could just accelerate progress by 10 years, which of those bottlenecks might be alleviated by things you see coming down the pipeline?

25:40Yeah, so happy to talk about this in the pharma context, which are large decentralized sprawling bureaucracies, much like universities, or the Congress or the SFCD governments. And I think some parts are incredibly functional and then I think everyone would agree in these different organizations. Some parts are less efficient. And so I think the first thing that hopefully we can see is that we can have models that can and prove efficiency in discrete individual steps. So can we have a more efficient process for target ID in particular? Can we have a more efficient process for data analysis? Can we have a more efficient process for information and literature review and summarization for molecule design, absolutely, making a better binder?

26:37And then figuring out the drug properties of those different molecules. That could be selectivity, that could be pharmacokinetics, half -life, expression, manufacturability, what have you. And there will be models or model guided approaches for each of those steps. And then there's, I mean, I think if you look at how AI is actually being used today in pharma companies, a lot of them are actually just taking massive regulatory documents, summarizing them, and then using AI to help them write more of this. So there's this compression and decompression of information in a structured fashion that has actually been leading to enterprise adoption in this setting.

27:28And I don't know, I think that says something about the process of where people find things useful. So, one example is if you talk to some pharma execs, not the drug discovery organizational leaders, but the budget people who hold the enterprise per strings, they'll say, well, I don't spend that much money on drug discovery. I actually spend most of my money on drug development. So tell me something about drug development, which is really where most of my dollars go. How can AI help me with that? right? And I think that is actually like a very deep comment right? Because it says something about where money is spent and where value can be found.

28:15You know the first thing that realizes you know our industry probably success is like 10 % right? And so you know I think a lot of the things that people talk about or complain about or comment on in drug discovery and development falls out of that fundamental statistic, right? Like, why does the FDA heavily regulate? Why is it so focused on safety? Well, if 90 % of the time it doesn't work, they're gonna care a lot about safety, right? And I think the promise of AI is, if we can go from 10 % POS to 20 or 30 or 50, as you move to those steps, can we? Do you think we can? Over time, I think here's the thing that I find kind of interesting is we have done a lot of biology with what is not that far from guess and check.

29:11If you look at what happens in the wet lab, like the actual experiments that are happening, you're just kind of in the arena trying to. Yeah, trying out random hypotheses and seeing what happens. Yeah, and this is like the missing, reasoning trace in the scientific literature is you don't know what didn't work. Everything is narrativized and written in a story of, you know, an inexorable logic and vision leading to scientific breakthrough, right? But everyone who makes the sausage knows that's not what actually happens most of the time, right? And, you know, if you actually work with the, you know, research, you know, kind of folks at the bench, right?