Basecamp's Phil Lorenz on Combining AI With Biodiversity Data - Ep. 223

0:10Hello, and welcome to the NVIDIA AI podcast. I'm your host, Noah Kravitz. The intersection of AI and biology is one of the most fascinating and promising areas of modern technology and research. My guest today is working at the leading edge of this field in his role as CTO of Basecamp Research. Basecamp, who's a member of the NVIDIA Inception program for startups, is leveraging their unprecedented knowledge of the natural world to create better food, better medicines, and better products for the planet. Basecamp has collected an unprecedented data set capturing orders of magnitude more diverse biological data than any public resources, and they're leveraging this data for deep learning and Gen I applications.

0:50Here to shed light on what that means for his company and for all of us is Phil Lorenz, Chief Technology Officer at Basecamp. Phil, thanks so much for taking the time to join the podcast. Thanks so much for having me. How's your trip been so far? We're recording on, I guess this is day three of GTC, so you've been in town for a few days. How's the conference? It's amazing. It's great to see some friends who I haven't seen in a while. So that's actually been great meeting with folks from NVIDIA that we've been working with for a while. And yeah, I mean, lots of great networking opportunities, including people that are really far outside the life science industry.

1:24But it's great to see what everyone else is doing. So really exciting. It is. It's nice to be back in person after several years here. So let's start with the basics. Maybe you can tell us what Basecamp Research is, how you were founded, what you do. Yeah, of course. Maybe just take a step back with respect to why we're doing what we're doing and how we thought of this. I guess if you think about kind of the life science, it's probably one of the most exciting domains to apply AI to, I think. And we obviously have a lot of kind of human clinical data collected in the last few years and decades.

1:56But when it comes to kind of life on Earth and biology as a whole, we actually haven't because there's probably about 10 to the 26 species out there, which is a lot. and we've sequenced a few million and so if you kind of make that comparison in terms of what we know about life on earth that's about five drops of water compared to the atlantic ocean which is what we don't know so if that's the kind of place you're starting with and everything the life science industry has ever built is based on that tiny knowledge of life on earth that kind of slice we thought that if you want to do deep learning for the life sciences really well there's exciting algorithms being built, exciting architectures that we can do.

2:34But at the same time, we feel like, okay, there's actually a big data problem to begin with. And so to do this from first principles, what we've done over the last two or three years is we've built partnerships with nature parks across five continents, including places like the Antarctic and rainforests and volcanic islands and you name it. And we have professional explorers that go to these places and they sequence the biodiversity in these areas. A lot of microbes, because that's where the greatest diversity of life on Earth is. And we do this in partnerships with these nature parks. Right. When you say sequence the biodiversity, am I getting that right?

3:13Sequence the biodiversity that they found? What does that mean for kind of a layperson? Absolutely, yeah. Realizing that I'm a heavy life scientist. I'm saying the layperson, I really mean me. No, that's completely fair. Sequence basically means every organism has a genome, has DNA. And so sequencing the genomes of all of these unknown organisms. Okay. And out in, is that, just to kind of take a tangent for a second, can that be done out in the field remotely? Are we at the point with gene sequencing where you can do that on site, or how does that work? Yeah. So when the founders started Basecamp, they actually did that on site.

3:47They were on an ice cap in Iceland with a solar-powered tent. That's how the company started. And now because we are doing this at scale, we are extracting the DNA there and then we're sequencing the DNA at a much larger scale in Europe. Yeah. And I think what's exciting is, again, this is underappreciated, the vastness of unknown life on Earth. We now have a database just within two years that has samples collected from the Antarctic and volcanic islands and all the life that exists in these places. That's now a few orders of magnitudes more diverse than all public data combined. And what we're doing on top of this is not just collect hundreds of millions of new protein or DNA sequences, but also the chemical environment, the geological environment, and connected all of that information together in a knowledge graph that now has about 6 billion relationships.

4:42And so that really gives us a good information of entirely new, never seen before information that has never existed before. So to go back to your analogy about drops of water in the ocean, if we were at sort of, you know, five drops of water compared to the ocean of knowledge, how much more knowledge have you been able to accrue in these past couple of years? Yeah, we're probably still a few orders of magnitudes away from the Atlantic Ocean. We're trying to be maybe a cup of water. Cup of water. Getting close to that. That's kind of the goal in the next few months. That's a lot of progress. do with the data?

5:19There's several things. I mean, we have a huge, there's just a huge engineering effort to just annotate all of this data, to organize all of this data, because that's, you know, it's now bigger than most public database, and that's actually a big undertaking. The exciting application really is to see, okay, there's some architectures for deep learning in biology, such as structure prediction of proteins. And now that we have so much more diverse data, what we can do is actually leverage some of the algorithms or architectures that exist and apply that to our data advantage. And so one thing that we've now built is called BaseFold, which is using a similar architecture to AlphaFold, but we can actually be up to six times more accurate because we have so much more diverse and additional sequence information.

5:58And so that's something that's really exciting because there's obviously a lot of work and effort being done on using, you know, a different algorithm, different methods. But actually data, especially in the life sciences, makes such a big difference. So that's something that we're really excited to use. I think the right way to get into this is, I want to ask about what your sort of day-to-day life is as the CTO, but then I'm also curious what happens. Are you working with partners across academia, industry to leverage the data, to create better medicines, better foods, those kinds of things we talked about at the intro?

6:32So either way, talk us through kind of what you do as CTO, and then maybe from there we can talk about some of the partnerships. My role is I do a lot of very different things at the same time. The one thing I'm not doing anymore at all is coding. I did that at the start, but that's kind of the one thing I'm not really doing. But that's probably a good thing. I think my role has kind of three, four main things. The first is actually making sure that the data collection process and how we enter that into our database. We have a genomics team that's amazing, and they're doing incredible work dealing with all of this data and having really high quality annotations for that.

7:10That's kind of one thing. Then the data engineering, how we organize all of our infrastructure, the deep learning work, applying this data advantage to the most exciting AI applications, and then the product team. That's actually what you mentioned with respect to what our partnerships look like with therapeutics or biotech companies. Some of them, they will ask us, oh, do you have a protein that can do this function, like an enzyme that can break down plastic? and then we work with them on that. Or sometimes gene editing systems that will cure genetic diseases. And we have these in our database occurring naturally.

7:44But because, again, of our data advantage, we can use generative algorithms to actually generate these assets. And then they license them from us, and then they use them for downstream clinical applications or whatever that might be. I'm familiar with the basic idea of gene editing, but not much beyond that. if a company comes to you and, for instance, says they need an enzyme that can break down plastics, and let's say, I don't know if that occurs in the natural world or not, but let's say it doesn't, what can you do then? How does that work? So there's kind of multiple ways in which we look at this.

8:20The first one is, let's say someone wants an enzyme to the great plastic. In many cases, that assumption of, let's say, oh, we're not sure whether this exists in nature or not, is actually, that assumption is made on what we know from public data. And actually, we are making new biological discoveries based on our data set all the time. So that's kind of one argument. But then there's another way of thinking about this, is that because of the data advantage, when we use deep learning to optimize or generate these enzymes for a specific function, because we have explored sequence space and evolution so much more, we can actually understand how to get towards even these unnatural reactions much easier in many, many situations.

8:59Right. So you said it's been about two, a little over two years that you've been gathering the data sets. How much has AI, the technology that's available that, you know, compute as well as data, how much has that world changed in those couple of years relative to, you know, you thinking about, well, we're collecting, you know, this enormous amount of data and it's fantastic and it can open so many doors, but do we have enough compute? Do we have the right algorithms to be able to work with it? From the outside, and especially we talk a lot about generative AI these days, for the mainstream, that world has exploded in that time period.

9:39But for the kind of work you're doing, has the change been as dramatic? Absolutely. There were a couple of situations a few years ago even where we had maybe not the same size of the data set, but the same foundational architecture with a long genome context and all of this metadata that we collect, where I was actually thinking, oh, damn, I don't think there's an architecture out there that can deal with our data. And that is slowly starting to change. And I think one of the most exciting things in kind of the deep learning applied to biological tasks in the past few months and years is that what a lot of people have done is thinking about what additional biological context can I include into my language model architecture or something.

10:21So not just use something from a different domain and force that architecture onto a biological question, but think about how can I change that model architecture in a way that it represents biology much better. And I think that kind of trend in the last few months is really exciting and it yields much better results as well. And that's great. Are you taking off-the-shelf models and fine-tuning them for your own use? Or are you building models from scratch? How does that work? We're doing both. So on the folding problem, we use pretty much AlphaFold's architecture that exists because I think it's pretty good and it works.

11:00And for that, it's purely just doing a much better job because of the data that we have. We've also built our own architectures and models, one, for annotation. There's a lot of what are called functional dark matter sequences where we have a sequence, but we have no idea what it does. And if you have, you know, sequences from the Antarctic that have never been seen before, it's actually important for us to be able to computationally say what they do. And so for that, we've developed some contrastive deep learning algorithms to annotate them at pretty high accuracy. And we presented that at New Ops last year.

11:31So we kind of do both. It depends what we feel like is worth building something from scratch versus just leveraging our data advantage. But even when we built something from scratch, we're always leveraging our data advantage as well. So it's kind of, we're doing both, whatever works. I'm speaking with Phil Lorenz. Phil is the Chief Technology Officer at Basecamp Research. And we're speaking high above the show floor here at GTC 2024. Our podcast recording area has a nice window view of the show floor coming to life this morning. Phil, you came from an academic background. You were at University of Oxford before joining Basecamp.

12:08Maybe you can walk us through your journey a little bit, and then we can talk a little bit about what's the same, what's different from moving from academia into your role now. Yeah, definitely. I'm a traditional life scientist at heart. You have a lot of people in healthcare life science now that have a computer science background working through that. And that's amazing. That's incredible. I'm a little bit from the kind of molecular biology traditional background, which is what I did during my undergrad. and then moved towards more kind of deep learning applied to genomics sequencing data for my PhD.

12:41I discovered a couple of new human genes and transcription start site using deep learning doing my PhD. So that's kind of where I worked on for a while. One thing I always really cared about whatever you do in the life science of healthcare industry is thinking about the problem you're trying to solve first and then going backwards and thinking, okay, what kind of technologies tools can you use to address that problem right that's kind of always thinking about what you're trying to do in that kind of order of events um and that's still how i think about this now even though i'm i wouldn't really say i'm a traditional life scientist anymore but always thinking about what you're trying to solve and then what do you need to build second uh that kind of philosophy is still something i i think about uh quite a lot even though it makes sense yeah my phd was quite applied so it wasn't wasn't too academic which is maybe a good thing um but That's kind of how I came into what I'm doing now.

13:31Did you learn to code when you were younger, kind of out of an interest in computer science and learning how to code? Or was it more of a, in your work in the life sciences, you kind of hit a point where you realized, oh, this will be faster if I learn how to write scripts. It was almost, I started coding almost out of a necessity. When I got my first kind of big data sets, and at some point I was like, yeah, I'm not going to use Excel for that. So it was almost kind of, I wouldn't say I was forced to, but at some point I was like, well, it's just going to make everything more efficient, faster, and so on.

14:03So it was kind of great because I felt like I was coding always with a purpose to do something specific for what I wanted to do with my project. So in that way, I kind of always felt really motivated to go after it. So that's kind of good. So how long have you been at Basecamp now? Basically from the very beginning for almost three years now. Okay. And how big is your team now? How has it grown out over that time? The company as a whole, about 32 people. Okay. My team is 15, but a bit less than half the company. Yeah. Can you tell a story or kind of explain a discovery along the way at Basecamp that will blow our listeners' minds?

14:43Is there something about, you know, sequences from Antarctica or, you know, something undiscovered about the way, you know, ecosystems work in the desert or for that matter here in San Jose? I don't know. But something that just really sticks out is like, you know. Yeah. I mean, one thing that I still think is a nice thing to share is actually from the very, very beginning, the very first few days of Basecamp was – I mentioned this earlier. but the two founders of Basecamp, Glenn and Ollie, and big kudos to them having this kind of vision. They love exploring the world. They go out in the wild.

15:20They go climbing. They go up mountains, whatever. I'm more fragile. I like to stay behind a screen. And so at some point, a couple of months before they started Basecamp, they spent over a month on an ice cap, fully off-grid in Iceland. And Glenn did a lot of sequencing, DNA sequencing, during his PhD. So he brought these kind of mini flow cells with him, these portable sequencing devices. And he was just sequencing ice caps and see what was in there. You probably don't expect much life to be happening there. And so they came back with this data and they realized, oh, Phil, you do a lot of this analysis and you do coding in your PhD.

15:55Can you have a look at what's in there? And I analyzed this data. I annotated this. And something like 97 % of it has never been seen before in any public databases. It was completely novel, had 0 % similarity to anything that's ever been seen before. And that was just a random spot glinted somewhere on an ice cap. We didn't look for something new. It was just random. And so from that, we just realized, oh my God, the vastness of life on Earth is just so huge. And the opportunities lost in the life science industry by not leveraging this data more systematically, that was a great kind of origin story for us to realize, let's do this systematically.

16:31And with deep learning in mind, because a lot of the data that we have in the life sciences is us, you know, having kind of all these academic endeavors sequencing here or there. And fingers crossed that they've collected the right data, right? And so we've kind of made this systematic and in partnership with all of these nature parks, which is exciting. Yeah, that's amazing. From the technological side, we talked a little bit before about the infrastructure, the compute, the techniques, sort of, I don't want to say catching up to, but sort of keeping pace with, you know, the size of your data set along the way.

17:04What are some other AI machine learning related challenges that you've encountered at Basecamp that you got past or perhaps that you're sort of, you know, still grinding on now? Yeah, I mean, one thing that I am most excited by that has been kind of addressed in the last few months is how to deal with bigger context sizes. So that's kind of something in the language model field. There's a couple of architectures that were developed, especially I think in Stanford last year, like Hyena and Mamba, that I'm super excited by. Because what we're collecting is not just protein sequences, but these really long range genomic context windows that's not really that common in other public data.

17:44Can I ask you to explain what that means? Yes. I mean, when people sequence environmental data, not like a strain from a Petri dish, but kind of these wild environmental samples, you might have 10 ,000 species in a tiny piece of soil or something. When you sequence that, usually what happens in public data is you get maybe a few thousand base pairs. And if you're lucky, there's one gene on there. What we've done is we've really tried to optimize this process in a way where we get near full genome. So the entire genome of every single organism of that. So hundreds of thousands of base pairs with tens of thousands of genes or whatever that might be.

18:20And so with that, we're actually understanding a lot more about the interaction of all of these genes, what they do to work together and also understand more complex behavior. For example, how you can use this for gene editing or therapeutic applications. But modeling this with language models hasn't really been that straightforward because there wasn't really that many architectures out there to deal with that kind of information. And with Hyena and Mamba, we're now really excited that this is now possible and that we have the data set that we can apply this to. So that's something I think in terms of dealing with long context, probably the most exciting development for very selfish reasons, basically.

18:55But I'm super excited about it. Very cool. Very cool. So you kind of hinted at this in that answer when you mentioned gene editing. Yeah. What are some of the applications going forward for, you know, the work that Basecamp's using? And then, I don't know, and I'm not asking you to compare to, you know, competitors or what have you, but as the available data sets, you know, from sampled from biology, from nature grow, as they continue to grow, So what are some of the implications and applications for, you know, everyday folks like me downstream from the work that you're doing? No, I think the reason I think gene editing and at some point gene writing technologies is going to change not just medicine, but health in general.

19:39You know, we accumulate millions of mutations every day just by existing, by breathing and eating. And some of them are non-significant, some of them are bad, some of them are maybe good. But for us to be able to accurately change them or write new DNA into the human genome to make a change, that's really, I think, kind of the next wave of big therapeutic changes that we can make. And the machines that can do this actually often are derived from the way bacteria and viruses fight with each other. So bacteria and phages, those are viruses that infect bacteria. They kind of have biological warfare going against each other in the wild, in nature.

20:15It's not something you can measure in a Petri dish with a sterile strain or whatever, but in nature, they have warfare against each other. And those machines that enact this warfare, those gene editing systems, CRISPR-Cas nuclease is like one of the kind of major headlines that came out of that. There's hundreds of millions of these systems that haven't been discovered yet. And leveraging them in a way that we do, but also in a way where at some point, by having enough of these that we can generate them or design them through language models, for example, That's something that, a development that I think is really exciting.

20:48You know, I'm abstracting to the level that I can comprehend. Absolutely, yeah. The last bit that you said, I was thinking, so my kids, maybe my grandkids might be able to prompt a model and edit their genes or rewrite their genes. And I know it's maybe not quite like that, but is that a future we're headed, Charles? I think, I mean, I live in Europe, so there's always a lot of regulation to think about maybe. Of course. I don't know. I can't promise all of it. No, but I think, joking aside, I think one of the things I can definitely imagine is that if the way we monitor our DNA and our mutations and the way we can address these mutations, let's say in 20, 30 years time, is something we can do in real time, I can imagine, where because of, you know, sequencing in the body as we live and breathe, where through some device we detect a harmful mutation and being able to fix it within two hours.

21:42Like, this sounds crazy, but I do think this is kind of where this is going in 20, 30 years. And that's kind of the science fiction scenario for therapeutics and gene technologies. Amazing. NVIDIA Inception, you're part of it. I'm not asking you to plug anything, but how's that been? And, you know, being sort of a startup on the leading edge of life sciences must be, you know, in some ways similar to other startups with similar concerns around growth and funding. and keeping things running and all that kind of stuff. But I'm sure there's something unique to being a startup, working on discovering novel science.

22:22What's that like? And what's it been like working with Inception? It's been amazing. We've been working with NVIDIA for two years, almost two years now. I think it's really exciting because a lot is happening. And so, I mean, sometimes I open BioArchive or PubMed or something like, damn, can everyone please stop publishing? Right, right, right. There's just so much happening. But I actually think it's exciting because everyone has their strengths. Everyone, and by having these networks of lots of life science companies, and everyone has a different product, they have a different strategy. And so actually, some people think like, oh, are these, you know, startups all competitive with each other?

22:59And in some cases, maybe. But I'm actually a lot more excited by the fact that what's really happening is we're all growing the field. We're all growing the market. And so some people offer a software. Some people offer an asset. Some people will offer a service, whatever it might be. And so actually the products are different, the technologies are different. And so just the space growing as a whole is something that's super exciting. And NVIDIA and Inception, they're connecting everyone and making it happen. And so that's something I'm super excited about. That's fantastic. So you mentioned coming up with something of a traditional, I hesitate to call it old school because as they're sitting at the table, I'm not going to guess, but I know you're a fair amount younger than I am.

23:35So if you're old school, I don't want to think about what I am. But coming up with more of a traditional life sciences background and then kind of moving to a place where you moved into applying it and using technology in that way. Jensen said something in the media a couple weeks ago about giving advice to young folks to focus on a domain and develop domain expertise because the computing language of the future is just speaking, right? It's natural language. And so the tools will progress so that you can leverage them in your domain. What advice would you give to a young life scientist, somebody studying biology in high school or going into undergraduate work in this age where everything's moving so fast and technology is such a big part of it?

24:22I guess, yeah, I mean, I speak to a lot of kind of biologists, but also computer scientists that are flying through basically. To the biologists, my main advice is always do what you care about. there's a lot of biologists that have something they really care about but then they go into oncology because that's where big farmers oh yeah yeah and that's great but i actually think there's so many there's so many um areas in the life science industry where the problem is not that they're not relevant the problem is that they're not relevant yet because by making more discoveries we're always going to find something that's clinically relevant like gene editing was found through studying bacterial immunology which is like right no one ever thought was going to be relevant therapeutically a few years later right so i think it's always better to do something that you're passionate about and make it relevant rather than trying to find something that oh this is what therapeutics cares about and just running after that that's my advice to biologists for kind of computer scientists the main thing i find is uh when when we have people apply to base camp or interviews and so on often what i hear is um people kind of saying oh i really care about this specific type of diffusion model and i want to apply this and my counter argument often And it's kind of, let's discuss what we're trying to solve first.

25:33And then does that help? Or should we think about something else? Or should it be a language model? Or should it be, you know, regression? I don't know. But basically, always, this is often what, when I speak to people from kind of a more technical background, always. And this goes back to Jensen's point about the domain. To interact well with the domain, the problem they're trying to solve. And then absolutely, yeah, talk about the technology and what it's going to address. But identify the problem, the mission first, and then apply the tools. Yeah, it makes good sense. One thing I haven't really talked about much is you probably know about things like the New York Times suing OpenAI because they didn't ask for their data.