AI Will Cure Us All!! Or Not!! - EP 84 Abhi Mahajan

0:00Eryney Marrogi: All right, here we go with one of the most important questions of our age. Will all of this AI technology that is unfolding before us improve biology and biotech? Will we be healthier? Will we live longer? Will we have cures to all kinds of diseases? Silicon Valley in the biotech world would try and convince you, yes, there are about 47 ,000 AI biotech companies. They say they're folding proteins, coming up with antibodies, coming up with therapies faster than ever before. I think there's a camp of people that would say, no, biology is messy. It's going to stay that way. You've got regulation, trials, all kinds of things to do that will keep things running at the pace they just about always have.

0:49Eryney Marrogi: To discuss this, I am stepping aside. I'm turning the microphone over to Arne Marogi. And he's on his way to becoming a doctor. He is a accomplished scientist and he writes for Core Memory quite frequently. And he is going to be interviewing Abhishek Mahajan. He now works at the OpenAI Foundation. He has also worked in biotech. He writes the OWL posting substack where he writes in detail about biology and biotech. Wonderful detail. So these two guys are going to go at it. Arne wore a suit and everything to be respectable and show me up. I can't think of two smarter, better, more knowledgeable people to be discussing these issues.

1:38Eryney Marrogi: I think you'll find it fascinating. I should note their views are their own and not representative of the OpenAI Foundation or whoever Arne is beholden to in his professional capacity. It's just these two gents thinking big thoughts and walking us through this incredibly important question that may be one of the most profound and important topics of this century as we know it. So I am Ashley Vance. This is Core Memory. Here are two other guys on the show.

2:31Eryney Marrogi: I'm Aaron A. Marogi. This is Core Memory. I know it's not Ashley talking, but we have a special episode today about biology and AI. Our guest is somebody who is quite well known in the space. Some people know him as the Owl Poster. I personally know him as Abhi Mahajan, who I worked with at Dino Therapeutics for a couple of years before he went off to a company called Noetic. And now most recently, he started at the OpenAI Foundation. So it's a pleasure to talk to you, Abhi. Pleasure. Pleasure for me as well. So, you know, I guess my goal from this episode is to engage with some of the ideas I think around whether, you know, sort of maybe develop a taste for whether I actually believe that AI can accelerate the pace of biology by throwing unlimited intelligence at the problem.

3:26Eryney Marrogi: So can you just sort of think about stuff really hard? And does that get you far enough along to move faster than the speed at which we've been moving to, I don't know, the last maybe 500 years or so? Sure. That sounds like an interesting discussion. Okay. All right. So just starting us off here then, I have a pretty maybe simple question, which is, do you think that the bottleneck in biology is intelligence or a sort of physical grounding of reality through experimentation?

3:57Abhi Mahajan: I guess it depends on what stage of, at least for the problem of drug discovery, I guess it depends on what stage you're operating at. I think at the preclinical state, I feel like you can make an argument for every side has kind of a place to stand on. I think for the intelligence maximalists, they very well could say that, yeah, like doing things in the physical universe requires like time and people, none of which are replaceable by intelligence. But at the same time, you could imagine super intelligence could state what experiments are most worth running. And that perhaps like lets you do the preclinical experiments that are perhaps best.

4:38Abhi Mahajan: At the clinical stage, probably intelligence is not all that useful. It's primarily like a human coordination problem, which I don't know, like maybe in the aggregate, super intelligence helps to like plan out the built environment in such a way that it helps. We had like a friend of ours who went to China and said, like, the big advantage of running clinical trials there is that you can recruit from kind of anywhere in the country because there are trains that will get you to the major medical centers. And so in that case, maybe super intelligence helps plan that out better. But you can make kind of contrived arguments as to where intelligence is useful.

5:12Eryney Marrogi: I guess you could say the physical grounding in that way is like, how many trains do we have running between Chicago and MD Anderson in Texas? Yeah.

5:19Abhi Mahajan: I think for the most part, it is primarily a human coordination problem. I think when you get to post-approval, I think superintelligence actually is pretty useful in discovering, what was that drug? like Vioxx, that like it was an antihypertensive that turned out to have like major cardiovascular events a few years after its approval. I think in cases like that, maybe intelligence really is good for like roaming through, what's that like self-reported database, Aphorus, something like that? I don't know about that one. It's like something run by the government where like people can, it was a big thing during COVID where people can.

5:59Abhi Mahajan: Oh, self-report like the side effects. Yes. Like what you're supposed to do in phase fours. Yes. Yeah, yeah, yeah. Okay. And maybe superintelligence is really great at like roaming through that and discovering things that humans could discover, but we just simply don't have enough epidemiologists on hand.

6:13Eryney Marrogi: Yeah, I guess, you know, like a follow up to this. I'm not a mathematician. I don't think you're a mathematician, but I think you probably have more of a sense for math than me. so you know I think a couple of days ago maybe it was yesterday I'm not quite sure I don't quite remember the days blend nowadays there was a big GPT I think 5.7 soul thing where they had like 10 theorems that they solved or I don't know I'm sure like mathematicians are gonna be really upset at the way I frame that but supposedly the really big deal and I was sort of reading some something on Twitter, which we know is like a grounded reality, that any number of these could be like a fields level discovery.

6:56Eryney Marrogi: I guess like in my mind, seems like super intelligence, at least in a mathematical sense to me, at least like definitionally, people have always been around like, could you have like a thousand von Neumanns in like a data center, right? Like kind of seems like we did. And so we got this outcome, right? I guess the question is like does 5.7 do anything to biology right now it feels like we actually do have it maybe maybe you can make arguments that it's not quite super intelligence but we have something pretty close to something that looks like super intelligence do we think this is going to like tomorrow right we should be picking up the pace really quite rapidly if super intelligence since it's a bottleneck.

7:38Abhi Mahajan: I think the common counter argument here is that math is an extremely verifiable domain. And so your feedback loops are extremely tight. And actually, I think the essay, I think a lot of that is written by an ex-co-worker of ours, Stephen Molina, who wrote this thing about fast feedback loops. And I think if you can imagine a future in which there are robots in a lab that can plan out their own experiments and like learn from them immediately and then plan out the next set, maybe that is like really the substrate for genuine superintelligence and biology in the same way you see it in math. I think today those feedback loops are roughly as slow as they were five years ago.

8:22Abhi Mahajan: Whereas like in math, the feedback loops are always tight. It was just like intelligence genuinely was the bottleneck. Now the bottleneck is like both intelligence and also like human hands to do everything. um and for my conference i wrote this article about like lab automation a few months ago and i got the impression that like the the the problem with like aggressively automating a lab is simply that most experiments are not worth automating there's like a few select experiments that um like like mixing compounds doing like uh like hepatocyte uh drug screening those things sure you can scale up uh um an infinite amount of times but they aren't also aren't super informative The most informative experiments are kind of just like you tinkering around and trying things.

9:05Abhi Mahajan: And maybe we'll eventually arrive to the point where a robot in a wet lab has the dexterity necessary to do all those like super fine mixing experiments and like swabbing a cell plate with stuff. But today we're not really there.

9:19Eryney Marrogi: Yeah. I guess, you know, when I read that article, that was fairly recent. I want to say like six weeks-ish maybe. you know when I read that article I think in my mind it's probably worth considering like there is lab automation which is like maybe translating making a you know PCR strip that you're you know you're putting all the reagents whatever translating that process into like a plate-based assay that a CNC like machine basically is doing or liquid handlers we call them in lab biology. You know, there's that kind of lab automation, which I think speeds you up maybe on the order of like less than 10 folds.

10:00Eryney Marrogi: Maybe if you run things 24 seven all the time, that gets you some acceleration. But actually, you know, what I think about is interesting. So one problem we worked on together at Denotherapeutics was the idea of like, can we speed up the evolution of proteins, specifically viruses. Prior to, I would say prior to next generation sequencing, this was something you did manually, right? So you had to like make a virus, you had to, you know, you have to design the virus, you have to make the virus, you have to transfect the cells, you have to infect the monkey or mouse or whatever, get the data out and be like, okay, did it work or did it not work?

10:36Eryney Marrogi: And you do this maybe on the order of like dozens-ish viruses, because you have to have some reporter and those senses. Maybe you attach a GFP to one, RFP to another, something like this, maybe immunostaining, but ultimately you're gated pretty quickly by the assay. So not the automation. You could have automated that process. And maybe instead of doing 12 viruses, you could get 48 viruses or 64 viruses, but you're not getting 11 million viruses. but the thing that kind of changed that was next generation sequencing all of a sudden you could get you know in some cases to some of our virus viral libraries billions of viruses in one pool that to me is actual like data acceleration that's nothing to do with automation um but at the same

11:26Abhi Mahajan: time then like the bottleneck moves to a new place now your problem is that um the mouse the mouse uh like blood brain barrier receptors are not the same as the ones in nhps which are not the same ones as humans. And so even if you scale up an infinite amount of the NHP, you don't get all that much closer to solving the delivery problem for humans. Yeah.

11:45Eryney Marrogi: I mean, that I'm completely with you. I feel like maybe that's actually a good segue into my next question. So reading one of your essays, I actually didn't know about this history. I've read the word. I've never said the word before. I'm going to try. Tigit. Tigit? Is that how they say it? T-I-G-I-T? Oh, Tidget. Tidget. There we go. Tidget. All right. Okay. You learn new things every day. so like looking back at TIGIT so there was kind of like a maybe to put some context there was a sort of sense maybe in 2010s or in 2010-2015 that there could be this new pathway for killing cancer cells where there's this thing TIGIT and if you knock it down you could unlock the sort of immune system of the body to go and kill cancer cells and there was some interesting preclinical data maybe like 2020 and this then moved into a bunch of companies investing into clinical pipelines in this um and all of the pre-clinical data was actually pretty rich and was pretty concrete in saying oh this is gonna this is gonna work this is gonna be it we're gonna be able to move past checkpoint inhibitors this thing is gonna change the game for cancer and all of the trials failed, all the phase three trials failed.

13:03Eryney Marrogi: Maybe one way of putting this question is if you took that data, all the data that existed, both maybe all the preclinical data, and you fed it to GBD 5.7, or maybe Mythos, I don't know. I have no idea how good that one is, but you feed it to the most frontier genius model with unlimited credits, API tokens, do you think we get a different assessment for whether TIGIT is going to work for cancer in a human?

13:35Abhi Mahajan: I forget which article I did this in, but when O3 Preview came out, I ran this exact type of experiment where there was this major drug failure that I forget the exact name of it, but it happened in the early 2010s. I basically asked the model, pretend you don't know anything about the future. Was there sufficient evidence for you to predict whether this was going to fail? And I did not find the answer very convincing. It was in the pre-training data that the drug had failed. So it a little bit distorted itself to say, yeah, there was sufficient evidence, but it didn't really seem super convincing.

14:17Abhi Mahajan: Maybe 5.6 or whatever the Astra version of 5.7 is, maybe that changes it. But I'm skeptical. And I think this is also true for Tidget. I think the argument that superintelligence would have against Tidget is that this is mouse data. Be careful. Don't spin up 12 clinical trials based off of those results. But I think if you were to put yourself in the headspace of someone at, I think it was like Roche. I think Roche was the first one. Like this is like the golden goose, you know? Like the whole point of blockbuster drugs is that, or I guess like a large part of pharma is like venture style bets and that you do kind of need to go all in the second you discover something that has some promising preclinical signal and it empirically has just like a massive market.

15:09Abhi Mahajan: And so maybe from the standpoint of Roche, even given the failure of TIGIT in the year since, it wasn't actually that bad of a bet from an expected value standpoint. And maybe it'll pay off someday. And so I remember I wrote this article about will AI be all that useful for drugs currently in clinical trials? And I talked to a lot of clinical trial consultants for that piece. And this was before this was last year or so before the current state where the models are at. And the general impression I got was, no, they're not going to sway anyone's opinion for drugs already in phase one through three because they've already like made a bet.

15:51Abhi Mahajan: You know, they're not going to they're not going to shift away from that bet just because of what a model tells them. And you could argue that maybe two, three years down the line, people are sufficiently convinced by the model to turn away from it. But a lot of the times these trials are ran more on emotional conviction rather than scientific ones. And I guess my overall point is, one, I don't think superintelligence would have predicted it, predicted the failure. Two, even if superintelligence had predicted the failure, I don't think people are well rewarded for actually trusting it. And they would have just ran it anyway.

16:24Abhi Mahajan: I see.

16:25Eryney Marrogi: Yeah. it is interesting to consider whether uh you would have been rewarded for going like oh you know they're pointing out something very you know it's useful to consider that this is a mouse and so maybe we shouldn't do it um you know i guess like there there's a component the dangerous part about the super intelligence thing right is like people can always change they can shift the goalposts and be like well super intelligence will simply be able to know what the relationship is between preclinical data and how likely it is to succeed in a human. So I should say, maybe from this point forward in our conversation, I'm going to assume that that's not going to just happen by just training on more internet data or more RL training.

17:05Eryney Marrogi: I don't think it's going to necessarily work out in that way, but maybe I'll be wrong. To maybe sort of change the direction of the question that in this case, based on your little experiment, and maybe we can apply it to TIGIT as well, what kind of data do you think was missing that would tell you that this actually was not going to work in a human? And maybe the obvious answer is human data.

17:30Abhi Mahajan: I think the obvious answer is human data. I think it's a little bit of a siren song to think that you can just magically discover human relevant data from these super scalable assays, either like in vitro ones or like ones done in animals, and it just immediately transfers to humans. Empirically, that just rarely turns out to be the case. It did vaguely turn out to be the case for like PD-1 drugs and like regular checkpoint inhibitors. It did turn out to be the case for chemotherapies. like not not not too insane to imagine that like tidget was also there um and i don't know like i think the the original uh like thesis for tidget was like one it seems to be highly uh elevated in like particularly aggressive tumors and two knocking it down in mice seem to lead to some responses not not a terrible pitch for for a future cancer drug um especially if like like I'm sure if you looked up Tidget around the time prior to this pivotal paper being published, you would have found a bunch of like, hey, Tidget is implicated in all these other immune processes.

18:46Abhi Mahajan: And I mean, like your eyes kind of filled with dollar signs at that. And you just go forward with it. I think it feels like rarely the case that blockbuster drugs have this like super clean story to it. and I in terms of like what data would have actually convinced people I'm not I'm I'm not sure anything would have suffice other than like taking it slow and like doing a phase zero with like an academic center first but then there's the obvious incentive against that is maybe you run that you get positive results um like this was done in 2020 so maybe this wouldn't apply but like if you do today like now China is like running like multiple phase ones and phase twos at the same time as you, and they beat you to the punch.

19:29Abhi Mahajan: Back then, your competitors would be still slower, American and European pharma's, but still, you'd be worried about them catching up to you. Yeah.

19:40Eryney Marrogi: I guess it sounds like the thing that protects you as an American company in this case is that a Chinese company can't just take a drug and say, oh, we have all this data that works in China. We want to sell it in America now. You have to actually run the trial again in America. At least today, that's the case. I expect this will probably remain true for a while. So there's some, maybe call it like the, we will once again say the verification layer is the thing that protects AI from, or whatever system from like really accelerating us. But I think the thing I'm personally just trying to develop a sense for is what kinds of problems actually are interesting for AI when it comes to biology.

20:24Eryney Marrogi: I know that you particularly worked on something at Noetic, so the company that you worked at after you left Dino. I'm going to try to explain it to my understanding. I'm on the left tail of the IQ curse. You correct me where I'm wrong. My understanding of things is that the belief system here is you can take cancer cell lines and you could introduce drugs and develop these perturb-seek type models that are basically tells you generally like how a particular cancer cell line is going to behave to different perturbations. And so when you give a new drug, maybe you can actually do some like latent space.

21:05Eryney Marrogi: Like if you have this model, you can actually just explore sort of explore similar areas around the drug that you are testing. And you can get a sense for like, oh, actually, this will work or this won't work. And that might speed up your ability to like maybe speed through some of the screening of the preclinical. Do you feel like, okay, I guess the first question is, do you think that's an accurate depiction of what you did and what no addict is all about?

21:29Abhi Mahajan: I do not think that's an accurate description. I want to hear from you then. The thesis of no addict is you can take a bunch of biopsy samples from a bunch of real like cancer patients, like resections. You can collect a bunch of high-dimensional data from them, so specifically spatial proteomics, spatial transodomics, HNE, and an exome panel. And you can feed all of that into a model as like a self-supervised learning task. So the end result is that you have a model that can create a sort of like embedding space of all the tumor samples that it sees. And one way to ensure that this embedding space is well organized is that, hey, maybe squamous carcinoma is here, adenocarcinoma is here.

22:18Abhi Mahajan: They're separated by known legible human categorizations. Plus, within them, they also contain categorizations that are only known to the model itself. And so once you have this, now one interesting thing you could do is fine-tune it on real clinical trial response data. Because ultimately, clinical trial response data is the rarest type of data that's out there. And so unless you have a model that already has a pretty good understanding of cancer, it's difficult for it to just like generalize to this new space. And so what the company is interested in is taking biopsies from patients who have been through this investigational like phase one, two, three, running through the model and seeing if it's fine-tuned on the set of like seven responders and like six non-responders, can it learn some notion of what was so special about the responders?

23:09Abhi Mahajan: And can it place responders here in the embedding space and non-responders here in the embedding space, such that there's a clean separation between the two? You don't know where this inclusion rule comes from. Only the bottle can really know, but maybe there's clues. Maybe you can see that as particularly high levels of this particular protein, high levels of these set of transcripts that are immune-related. And so from there, you can do kind of one of two things. One, you can go to the company and say, hey, you should probably run this in whatever future clinical trial you run. You should probably add in this inclusion criteria to ensure that your objective response rate in future trials is really good.

23:50Abhi Mahajan: Two, if the drug has failed and the company simply doesn't want it anymore, we, or they, Nordic, could simply buy the drug and run it in a clinical trial that has a really high expected value of success. And there is this like separate like perturbation platform that is you do the exact same thing as I described, but with mouse data instead of human data. And you apply like perturbations because it's a mouse. You can apply perturbations to the tumor microenvironment and you can like similar to Dino, you can multiplex them. So you can have like like I think dozens of tumor cell populations inside of a mouse lung.

24:31you can take that out and um the one of the interesting we wrote a blog post about this but

24:38Abhi Mahajan: one of the interesting things we learned is you can take this model that has been pre-trained on only human data and run the mouse data through it and if you run the mouse data through it you kind of get this projection of mouse data into a human landscape and there's like a few reasons why this might be interesting i think it's uh like like the the value proposition is still like a little bit fuzzy, but it's interesting that you kind of can learn, because what comes out of the system are like human transcripts. They're not mouse transcripts. Mouse transcripts have been mapped to some degree of humanness.

25:13Abhi Mahajan: And when we, you mentioned this thing about like, oh, you can like perturb this, like the model itself. One interesting thing about this whole setup is that it's not just like a auto encoder. It's a generative model. So you can take the input and say, like, there, there's like an elevated level of this particular gene, maybe it's tidbit, maybe it's something else. And you can see how the predictions themselves change. Maybe if you elevate like these five genes, you can see that a bunch of these other, like, inflammatory markers also go up. So probably a good thing. And so like, maybe those represent really good targets.

25:50Abhi Mahajan: and the spatial mouse perturbation platform acts as sort of like a ground truth for how it actually affects at least mouse tumors. I think the perturbation part is the least well understood. I think the pre-training to fine-tune on clinical trial stuff is the most well-fleshed out part of it. No.

26:09Eryney Marrogi: Okay. Yeah, no, I appreciate your redirection. All right, what do we do at Core Memory? We cover innovative, fast-moving, forward-thinking companies, which is why Core Memory is sponsored by Brex, because Brex is the intelligent finance platform for many of these companies. 30 ,000 companies from startups to the world's largest corporations rely on Brex's technology for their finances. They've got smart corporate cards, high-yield business banking, and expense automation tools that are fantastic. I hate doing my expenses. and Brex's AIs and software. Run right through those expenses, figure out where we're spending money and take care of so much stuff for you so you don't have to waste your time on it yourself.

27:00Eryney Marrogi: Go to brex.com slash core memory to learn more and just, you know, get with the program. Let's get going. Let's get out of this archaic finance software and move toward the future. Core memory and Brex. I'm wondering if you, you know, In my mind, it makes sense to at least approach the problem. If you're going to try to address the main issue, or I guess what we're trying to say is the main issue, which is a translation from preclinical to clinical, it sounds like really you're just trying to map mouse, which we'll just say broadly is the most common preclinical data you can have. If you're trying to map that into existing clinical data sets, so like phase one, two, three trials, four trials data, it would make sense you might learn something.

27:47Eryney Marrogi: and it might increase your hit rate. I don't know. Do you have a sense for like, maybe roughly, this is increased from like 5 % is kind of the number that people throw around when it comes to preclinical to clinical.

27:57Abhi Mahajan: So importantly, the mouse data is almost entirely for like novel target discovery. It's not used to necessarily connect it. Because like if you have clinical response data, you don't need to even bother with the mouse data. Of course, yeah, of course, yeah. In the case where there is no investigational drug that hits these particular sets of targets, then the mouse data becomes valuable. So it's almost like a backup. Yeah.

28:22Eryney Marrogi: Yeah, yeah, I see what you're saying. So you're saying like it's to explore a new space, which I get, and you're hoping that you can like sort of learn these representations of maybe drugs or some molecular interactions. Oh, and so your question is like,

28:36Abhi Mahajan: in the case of like novel targets, how well do you expect the mouse data? Yeah, exactly, exactly. I think the answer is like unknowable. I think, yeah, I think it's like - As with all things, it's unverifiable.

28:47Eryney Marrogi: It may be difficult to verify.

28:48Abhi Mahajan: I think this is the problem with AI and bio. And I think especially, I think like the protein people and the small molecule people are very lucky in that they have identified interesting hills to climb. So like improving IPAE in the protein world, good. Like it's like it's a good metric. It seems to like correlate well with like in vivo efficacy to at least some degree. So yeah, let's pump those numbers up and we'll get some more interesting. I think with regards to in vivo efficacy of like immune, like IO drugs in general, I think the hill to climb for in silico models is not well defined. So it is very much reading tea leaves.

29:34Abhi Mahajan: And I think that's just like kind of, that's kind of the reality where you have low end. you do kind of need to like, like squint your eyes and hope for the best. And this, this might sound like a horrible thing, but the, the counterfactual case is not having the model at all. So like, maybe it will be better. And then the worst case, it does nothing at all, which is exactly where we begin.

29:56Eryney Marrogi: Yeah. I actually think the counterfactual is like, are you better off taking like$1 trillion of investing and putting it into something else? I don't know. I don't know. That's true. That to me is actually the counterfactual. But to be clear, I actually do. I obviously believe in AI. I do work at an AI and bio company. I've kind of worked with an AI in biology and medicine. So I do believe in it. I think the more interesting question, I think it's undeniable that you can find narrow verticals where it's like a useful thing to work on. protein binding and, you know, making binders, you, I think were like the first one to really convince me that we solved maybe binder generation because now people can make binder libraries that have like 10 % hit rates, which is way better than like the 0.1 % hit rates we were getting 10 years ago.

30:46Eryney Marrogi: So, you know, there are these narrow slices where I think you totally can apply ML in very, very accelerationist directions. Like at Dino, we were making extremely good viruses much faster than say like the 25 years that preceded, you know, maybe the AI and bio landscape or field. I think that the challenge is like, you know, you and I had a discussion, I think at dinner earlier this past week, where we talked about like, is techs actually just like the only thing you need is text it actually the only thing you need to like learn about stuff and in the case of biology that text data can be protein sequences it can be dna rna it can be like transcription data it can be a lot of different types of text um but does this text actually tell you everything you need to know i don't know it's a i'm not really sure it's like it's hard it's hard me to say um but it's seemingly like this is the most obvious but it is like the important question really returning to the foundations of like are we generating the right kind of data to actually make massive leaps in biological discovery i think it's a i'm not sure if anyone has necessarily

32:05Abhi Mahajan: tried doing this yet but i've been i've long been because like drug discovery existed before computers, you know? And they were these like very smart med chemists who had like very strong intuition as to how drugs ought to be designed. Like it was like Lipinski's rule of five, I think. There are like a bunch of these like tiny little chemistry rules that seem to directly map to like, you're supposed to have these particular number of like hydrogen atoms, you're supposed to have these particular chemical motifs, and that seems to work really well in practice. And I think it is a really fun idea to imagine making super, super human medicinal chemists who, okay, like maybe they aren't super, super human, but they are these like scaled up versions of these extraordinarily rare humans.

32:53Abhi Mahajan: And so like, imagine, um, uh, like I, I've never, I've never like met one of like the Baker lab protein wizards before, but I have to imagine that if you had just a billion of those in a data center, you would get something useful. Like maybe human intelligence, like human intelligence as filtered through an LLM is like a very good regularizer almost in the sense of like a pathology of a lot of small molecule foundation models is very easy to get them to overfit on a particular slice of chemistry. But the LLMs have like almost an incentive, like this is almost like a tortured analogy, but they have an incentive to be broad and they don't necessarily overfit in the way that we think of overfitting.

33:41Abhi Mahajan: And maybe if you encode medicinal chemistry knowledge, if you encode drug campaigns as like you start with a smile string and then you include some reasoning text as to why you need to move on to this different smile string, then you move on to another one. Maybe encoding reasoning traces like that into natural language models is a really interesting direction for how you make superhuman medicinal chemists. And maybe you get like really crazy leaps as a result of that. I'm sure there's like, there's a ton of things in structural biology and toxicology and on and on, which are not encodable via this format.

34:18Abhi Mahajan: But it feels like there's like an underappreciated fraction that is.

34:23Eryney Marrogi: Yeah. So, okay. So my knowledge of chemistry sort of ends after like physical chemistry in college. There's like a limit to what I know. My sense from taking chemistry in college was that we kind of answered so many of the interesting questions there were to answer. And maybe physics, we actually did this as well. And now we are returning to a place where like it's harder to come up with simple rules for these things. um you know like a good example is like if you think about small molecules there are rules for synthesis that you can just apply and actually so simple that we teach them to every 20 year old pre-med who wants to become a doctor in this class called organic chemistry um and part of that is just the nature of like small molecules being simple i mean they're flat they're obviously not flat, but like they are kind of flat.

35:22Eryney Marrogi: You can really flatten how much complexity there is to them. And yeah, you will have difficult things here and there, but so many, so many of the principles of chemistry are actually genuinely just things you can like think kind of hard about and then be like, yeah, that kind of makes sense. Biology feels weird because even like proteins have this sort of, yeah, there are rules to maybe how they fold, but that doesn't always tell you like how they're going to behave. And maybe we will figure out that, oh, it's actually like this really simple thing. You just got to look at like the 14th amino acid and how it relates to the, you know, third to the last amino acid.

36:01Eryney Marrogi: And that tells you how it's going to like move in with a ligand or something. I don't know, whatever. Maybe there is a principle and maybe we just don't know it. But it seems unlikely to me that there will be one that's like this simple. And so your only other option is to invest like all this money into say molecular dynamics and just be like, Oh, we're just going to generate all this cryo EM data and all this MD representations of like what proteins look like when they flop and fold. And then we'll finally understand, well, you'll be able to predict with 10 % accuracy instead of 0 % accuracy, how a, you know, antibody is going to function in a, in a, in a plate.

36:38Eryney Marrogi: And so there was a lot of words there, But maybe the question that I'm trying to ask you is whether you feel like there's been something that you've seen that you're excited about that points at the direction of, actually, maybe we can just learn principles. And that is not AlphaFold, because AlphaFold is the one that everyone goes to. Because it's like, oh, AlphaFold, we solved protein folding. As you know, we didn't solve protein folding, because obviously, we don't have just antibodies you can print off the shelf.

37:06Abhi Mahajan: There's a friend of mine, a bunch of collaborators, made this really interesting benchmark that is basically trying to assess how well natural language models can interpret variants or SNPs in particular genes. and like the way that the benchmark is set up is I think they provide the wild type sequence and they provide the mutated sequence and they ask some particular question. I think it's like, how much does this, does it break the fold of the protein? And they compared it to like leading models and it gets pretty good. What do you think that tells you? What do you think that tells you? Do you think there's a lesson in there?

37:58Abhi Mahajan: Well, okay, okay. They're still not as good as the models. And I was initially super skeptical of the results when I first saw them. And I asked codecs, use simple metrics to see how well, use Blossom64, use just more substitutions is worse, more deletions is worse, and just create a linear predictor of that, see how well it compares to the models. And it's still worse than the natural language models. So clearly, like, the models understand some vague sense of what makes for a good protein and what makes for a bad protein. And it is at least somewhat at the level where, like, in an ideal world, there was, like, a baker-luck person who could also, like, run through this.

38:44Abhi Mahajan: And we have, like, the peak human benchmarks to it. But I don't know, like that, like that, that was like convinced me a little bit that the natural language models are actually doing something really interesting.

38:56Eryney Marrogi: I know there's no, there's no, you know, there's no argument for me in terms of their ability to give you something interesting. And there's tons of evidence for that, right? I mean, yes, we haven't solved zero shot prediction of antibody behavior, but maybe we've improved it sufficiently that like Abcci or all these, you know, antibody design companies, NABLA or whatever, are going to start printing antibodies in the next five, 10 years. Viruses, obviously, there's been a lot of progress there. Enzymes, maybe we're getting some momentum, but actually it hasn't gotten that far. And a big part of that is like, it's not just about the prediction.

39:34Eryney Marrogi: it's about predicting how things are going to work in dynamics yeah and that's the part that i feel like i haven't seen too i've obviously seen academic papers here and there where people like will you know like we we predicted how this like uh uh this this one like cofactor is going to be involved uh in this enzyme but we haven't seen necessarily of people i mean you and i wrote something for us i'm impressed about this but there hasn't been so much where i'm like oh yeah you know what we're gonna just we're gonna start zero shotting enzymes i can like turn a piece of wood into hard plastic uh if you just add it to a bucket or something i think i think enzymes is

40:15Abhi Mahajan: particularly hard and it's actually like it's one of those like one amongst many areas in biology where like despite having written that article i feel like i have a very bad understanding of like enzyme ml but i i do think at the end of the day like like nabla and chai i think if they end up failing or if there are any major pathologies in the company, it's the fact that the stuff they're predicting is a few steps removed from what actually matters. And so maybe, yes, they could create really binders and they happen to have really good developability properties by virtue of what's deposited on the PDP.

40:50Abhi Mahajan: But there's a bunch of these other PKPD steps and admin properties. They're not really optimizing for it today. But eventually they will get there. Eventually the data substrate necessary for that will be created and then they'll climb that hill. And maybe there'll be another step after that of like, can you get cells to manufacture this at like high scale? And maybe they'll hill climb that as well. I think one of the adages of biology is like there's many bottlenecks present throughout the drug development process, but I also don't think, I don't think any of those bottlenecks are impossible to hill climb.

41:27Abhi Mahajan: I think they're just difficult too. And right now people aren't optimizing for the things that are at least, like if they can raise money today on one particular climb, they'll raise money on that and they'll move on to the next one. I do think a lot of, like whenever people are pessimistic about the future in bio ML, they often point to these like, like these virtual cell papers that are like,

41:54Eryney Marrogi: they're no better. We don't need to name, we don't need to name anybody. So yeah, we're friends of the show and all that, yeah.

42:00Abhi Mahajan: Like, they'll point to these papers that are like, oh, it's like no better than linear baselines. And I think that's, like, those are all correct papers. I think they're very good papers. But I think it's a little bit like, it's almost like, of course, they can't be that good. They're barely measuring the cell. Their input is like a smile string and their output is like, or sorry, their input is like the cell line. The chemical they're applying to it with the genetic perturbation they're applying to it. And the output is like single cell RNA readouts. There's a lot that isn't being measured there.

42:38Abhi Mahajan: And maybe if you did actually like accrue all of those measurements, both input and output and train them all on that, you would actually see this like crazy, crazy out of generalization performance. I don't think like I think for a long time, I looked at the virtual cell literature and thought like what what a what's the word? Well, like what a misallocation of resources. but I think it's just like early in many ways. They just like, the problem space itself hasn't been flushed out enough.

43:05Eryney Marrogi: So I kind of have an answer but I want to hear your answer first which is why do you think that there's all this data that's not being measured?

43:13Abhi Mahajan: I think it's just hard to measure a lot of it. Like what do you need to measure a cell? You, like single cell RNA readout is one. You'd probably benefit from ATAC-seq as well. You'd probably benefit from what is being expressed on the surface of the cell and the intracellular proteins as well, you probably benefit from knowing the glycans that are floating on the cell surface. You'd probably also benefit from understanding, like if you had a bunch of cells together, you'd probably benefit from understanding how cells are communicating with each other, like the secretome. I guess, yeah, that's a full set I can think of.

43:54Eryney Marrogi: Yes, I think you're right. I think your sense is correct. Maybe from our conversation, it feels like the aspect I would be most excited about seeing AI help with is actually maybe the same thing that I expect a lot of hardware people are hoping as well, is that, you know, extremely cheap intelligence allows you to make new ways of working within the environment and maybe put in a more specific way. I feel like the most exciting problem you can apply AI to is like, can we actually just find new ways of like creating assays that collect a ton of different data, right? And so like the specifics of what you just discussed, when you have a cell, you can actually only, They're like just by definition of like how different assays are done.

44:46Eryney Marrogi: There are limits to what you can do at once from the same cell. Obviously, you can do batches, right? You can produce, you know, 10 liters of cells and you can partition one quarter of it to like RNA transcription. One quarter of it, you can be like, I want to look at the like DNA here. And maybe some of that stuff can be shared. Maybe you want to do like a quarter of it where you want to do like live imaging within the cell of like compartments. Maybe you have to do some immunohistochemistry. You can't get all this data from the same exact cell. You can get this data from batches. And maybe the thing we don't have right now is a really exceptional assay similar to next generation sequencing.

45:31So NGS really unlocked a ton of stuff in genomics

45:36Eryney Marrogi: and, you know, DNA, working with DNA. Obviously, a lot of like RNA technology really relies on, or I should say like RNA-seq really relies on like the basics of what we learned from NGS. And then protein engineering, as I discussed with viruses, can really take advantage of NGS as well. But it's like a specific assay that really broke that open. It wasn't like, yeah, let's just like make, let's just buy more OpenTron's OT2s and let's just like make them, you know, screen more plates. Like that will get it for us. And that's kind of where we are right now. People are like, well, all we need to do is just like scale up the number of OT2s we have running and make it sufficient so that they can run 24-7 and you don't need people to be introducing things.

46:19Eryney Marrogi: This feels like a losing strategy to me. It feels like the interesting problem is how can we apply intelligence? How can I get like get a co-scientist who's actually like a co-engineer to help me? you know i'm a biologist you know i work in medicine now but uh i i don't really know anything about hardware i don't really i'm you know i've done things in ml but like as you know like i'm not some exceptional software engineer either there are limits to my skill set uh good thing claude code is an excellent uh you know uh coder and uh so uh codex is also really great at this um can it actually also be really great at helping me like make a new device that doesn't exist right now.

46:58Eryney Marrogi: Like that seems like the actually important problem to solve. And if you get their unlimited tokens at something, that feels like the thing you want to throw unlimited tokens at. I, I do, I directionally agree with everything you said. I think, um, Oh, that means you, that means there's still, you disagree on it. I say that, and I think everyone says that we're like actually disagree.

47:16Abhi Mahajan: No, I, I, so I agree with it from the standpoint of, I think better or more clever instrumentation or assays is probably the single most important thing you could do. If you were trying to improve the state of biology R &D, better methods will be scaling up bad methods every day. I think what I'm most unsure about is whether the intelligence that is available via the LLMs today are all that useful for that. I have no idea. I do think instrumentation and assay research is one of those like black magic, uh, bio, bio R and D areas where it's a little bit unclear to me. Like, um, like I think the, the only ones that I've like, like really deeply studied is that, uh, yeast mating thing that A Alpha Bio does.

48:06Abhi Mahajan: And like, I'm not sure if you've read that people, but it's like, it's a very clever idea that like takes advantage of the fact of like, this is how yeast cells mate with each other. They're, they're, they're male to female to like, you can, you can take advantage of that property to do really large end-by-end protein-protein interaction screens. Like, would that pop out of the models? I don't know. I genuinely have no idea. It very well may be the case that the models are so capable of, like, they're tireless, they are relatively creative, that maybe if you did try to churn out, like, a billion memos, or, like, maybe, like, a few hundred memos of, like, propose this interesting assay to measure this part of biology that's very difficult to measure and you just hired a bunch of postdocs to like look through all of those and figure out like which ones are most promising to move forwards maybe that's actually like a really

48:54Eryney Marrogi: great use of resources i the good news is i actually think this is kind of a verifiable theory or at least right so what you could do is you take whatever your favorite flavor of like frontier model is and you can uh and you can you can try to see like can we can we some of these are very recent right like it's not like these are 70 year old uh technologies that we're applying you could say like in the last five years or maybe even 10 years ago could we have like taken everything we know and um predict sort of like the next step in the technology jump like could we have learned whatever they're doing from the data that existed five years before they started doing it um this is i think this is not like necessarily a fresh idea but i'm actually surprised that people don't try to explore this in a serious way um my guess is actually you would be able to because like what is it you know you ever like talk to like a really a really bleeding edge biologist and you know i had the pleasure of working with many of them in the church lab and also at dino there were plenty of them as well um you can kind of like feel like their brain like it's kind of always like you read a paper it's maybe like a kind of mediocre paper maybe it's a really great paper or whatever but you always like have this takeaway of like what does this matter to my work?

50:08Eryney Marrogi: And it feels like that is a mindset that an LLM could actually take on, where like reads, you know, you tell it like I work in the space, and you're like, are just giving it papers, and you can kind of say, like, what ideas do you have to like help me from like this field? What ideas do you have to help me like, apply to my field? I don't know, these are like, these feel like the kinds of things that are interesting to do with LLMs. And like, truly, if you had like an unlimited token budget, what could you actually do in an interesting way? Like this is what I would do. I do hope someone actually does these things rather than like me talk about them on like a podcast.

50:45Eryney Marrogi: Yeah. I feel like I have had like,

50:50Abhi Mahajan: we're friends with like a few people who run like semi-instrumentation bio companies. I feel like I have seen them say a few times like, oh, like Cloud Code 5.6 just like helped me solve this particular problem in the wet lab. and that like it it does directionally feel to me that um i know when you when when i say directionally it makes you think i'm disagreeing with the event i directionally understand what you're saying um but i i think the the whenever i hear that i i do get like brief glimpses into this future where the models genuinely are aiding the future of whatever uh interesting instrumentation is about to come out but are those like big leaps i so i've used models for

51:32Eryney Marrogi: troubleshooting things um especially like when i'm doing an assay for the first time yeah but they're not like really making these massive leaps for me it's like uh my protein express i've never really used this column before what do you think is going on like maybe it's like oh you gotta just you gotta hit flip the switch or like you gotta run the the buffer through it first

51:55Abhi Mahajan: i'm like oh duh i like i like i'm sure there are some assays they're a little bit functionally impossible, like maybe not impossible, but at least very difficult for the model to come up with. And like maybe one example of this is PCR. Like you needed the TAC polymerase. You needed the existence of a heat resistant polymerase to be able to attempt that assay. And would a model have been able to say, go find a polymerase that is heat resistant so you can like perform this like thermocycling?

52:22Eryney Marrogi: Maybe you could have, but - You need the idea of the polymerase. Like it's not even just heat resistant. You have to know like polymerases matter. Polymerases are actually the thing that stringed together DNA, but it's actually not even just that. You need to understand that like, well, actually we build these things one nucleotide at a time. And there's like this templated process and blah, blah, blah. There's so many steps. It's hard to know exactly where like you can hit start on and say like, progress forward. I do take your point. It's hard to know, obviously, like so much of this discussion is like, who knows, maybe.

52:53Eryney Marrogi: I do think a lot of aspects of our discussion here are actually verifiable. All right, pod friends, I am here to tell you about Send, Cut, Send. They are a manufacturing phenomenon. Whether you're working on your own car or you're a giant automotive company, a rocket maker, if you need metal cut, machine, or bent, you're going to Send, Cut, Send. They'll do it for you right, and they will do it for you fast. go to send cut send.com slash core memory for a 15 discount on whatever you're trying to make we love send cut send go visit them today there is something more specific i want to ask you about which is really to a recent article you wrote with organoids so switching a little bit to models do you feel like predicting organoid behavior perfectly say literally 1.0 right do you think that's a$1 trillion company?

53:53Abhi Mahajan: It depends. It depends on like, yes, yeah, I do. Because empirically, it seems to me that I have an upcoming article about like the clear success stories of organoids, and there aren't that many, but there are at least a few in the cancer space where patient derived organoids do genuinely seem to be pretty predictive of patient response. and I think the reason cancer organoids, I think, I could be very wrong about this, but I think the reason they aren't heavily used is because there's this sort of lag time between collecting the patient's patient cancer cells, growing them into an organoid, running the drug over them and screening them, and then like using that as like a treatment benefit, like stratification for the patient.

54:42Abhi Mahajan: I think that lag time is on the order of weeks to months and that's kind of too long. And I think if you could compress that to like a single yes or no answer, then I think that is like, maybe not trillions of dollars,

54:55Eryney Marrogi: but it is quite useful. Well, I guess trillions of dollars, everyone's worth a trillion dollars these days. So maybe it is trillions of dollars. I take, yeah, it sounds like, yes. It seems like your sense is like, yes, it is worth trillions of dollars.

55:07Abhi Mahajan: I think if you take like the maximal view, there's this really interesting line of work from the Mark Davis lab out of Stanford, which basically said that they can model some interesting facets of immune elicitation via tonsillar organoids. Tonsillar? Tonsillar organoids. Oh, yeah, yeah, yeah, yeah. And so like, it's a little bit like a torture definition of organoids because they're like, they're neither stem cells, like neither iPSC nor tissue stem cell derived organoids. They're more so biopsies of tonsils. So they take the biopsy and like expand them? Yeah, yeah. and those contain like germinal B centers.

55:47Abhi Mahajan: So they are able to model some facet of immune modulation. And if your question encompassed that as well, then I think that does become a trillion dollar company.

55:57Eryney Marrogi: Yeah, my question does encompass that. So it sounds like it could be a trillion dollar company. It's sort of, you know, so I, yeah, that's an interesting insight. I guess I don't know that much about this. My sense for organoids has always been like, there's so much batch to batch variation and not even just that it's like within the lab actually i was just talking to a graduate student um at sun kettering yesterday um or i should say on friday she is a t cell or an nk cell person and she is working with a new organoid um uh model that she's never worked with before and she has to like learn how to like layer the cells like in the right way and even though it's like yeah it looks right yeah like yeah you know her results don't necessarily match like what a postdoc who's doing the same exact thing does which is troubling it's it is

56:49Abhi Mahajan: bizarre that like your your primary functional readout for organoids often is either these like very coarse set of protein markers or it's just like they look wrong yeah um yeah and i think that

57:01Eryney Marrogi: that's a bad thing yeah i this but this is i guess this is what i guess what i'm trying to get at where it seems like an interesting problem is could you actually turn the model's attention to like resolving particular steps in the process from going from like, all right, man, we just like, we want to make a model that's able to like tell us if a drug is going to work or not in a human before we, before we do it, where the steps look like, well, you're going to have like, I got to have a good computational screening. Some, some sense maybe you have to have a good receptor knowledge, Okay, so maybe we've actually solved this problem to some extent with all these different PLMs and whatever.

57:42Eryney Marrogi: Okay, great. We solved that problem. All right, the next thing you need to know is can we predict mouse behavior? Great. Can we predict mouse translation to an organoid, or can we skip the mouse and go straight to an organoid and so on? It feels like you can actually split up the superintelligence, the burden of superintelligence into these discrete tasks, and then attack it that way as opposed to like we'll just dump a ton of biological data into like the largest model we can and hope that it can just like figure it out i guess maybe maybe the high level question um is whether you feel like the lessons of um the you know the whether the bitter lesson truly does apply to biology in the same way uh to me it feels like it could the ultimate thing that i'm not as quite sure about is like with transformers it was just like we need more text data yeah uh with biology it's like well what actually is the data that we need what's actually the important data and i don't know that anyone has necessarily found that

58:42Abhi Mahajan: yeah i i think uh biology i i was i was i've been trying to work on this essay for like months now title, was trusting the biological exponential, where I try to ponder what is the hill we actually need to climb with regards to both biological data sets, but also the instruments that are necessary. And I've been struggling to finish it because it's hard to come up with a right answer. I do think the sort of breaking things down into discrete problems and then throwing intelligence of that, solving that problem, and then moving up the hierarchy is a good way of approaching it, like the decomposition.

59:22Abhi Mahajan: And I think, like going back to the organoid problem, maybe you are able to perfectly predict, like you have a model that can perfectly do in silico organoid behavior. Then the next step becomes, how well does this perfect ability to replicate organoids transfer to human behavior? And now you need to set up an experiment where you have like organoids derived from humans in one branch and the humans themselves in another branch. And you have to compare those two. So you need to have like some notion of gold standards for every proxy data set that you're creating. Right now, people seem to be exclusively either scaling on super coarse readouts from humans or they're scaling on super deep measurements from proxy systems.

1:00:09Abhi Mahajan: And there's never any connection in between them. So like one example is like PDP. Like very deep biochemical knowledge is available there, but it has no knowledge of like isoforms that commonly exist in humans. No understanding of like the often massive multimeters that exist in humans either. Someday that data set will be created, but today it's not there. Right.

1:00:35Eryney Marrogi: Yeah, I guess everybody's trying to figure out like what their data says. The data set that they think is like the one that's missing. Um, and maybe there's not like one right answer. And I suppose like hope, you know, maybe the anthropic life sciences team thinks it's one thing. Maybe noetic thinks it's a different thing, at least for their problem set. Uh, maybe deep mind thinks it's a different thing. Maybe they actually all think it's the same thing. I don't know. Um, and, but ultimately I think your, your point is one that I agree about or agree on, uh, which is like, you can just separate discovery into discrete steps, even though We don't like to think about yogurt spoiling leads to fixing metabolic dysfunction in a baby born in Pennsylvania.

1:01:26Eryney Marrogi: Maybe actually you can just break that down into discrete steps. And if you think about it hard enough and yet enough people to think about it with, like say in a data center with like 10 million similar instances of yourself, maybe you could actually just get there. I do want to shift gears just a bit to talk about my favorite topic, biosecurity. Because you can't talk about AI without talking about biosecurity. I'm going to ask you a very open-ended question. I will just say, I'm going to say this first. my personal stance on the biosecurity question is is really just that for some reason we have we have really lost the plot a bit and we are uh focusing on the wrong thing with biosecurity and maybe definitionally to me biosecurity should not be like can we prevent uh bad actors from doing bad things to us.

1:02:30Eryney Marrogi: Maybe biosecurity, actually, to me, the definition should be something like, can we as humans have agency to prevent biology from harming us in whatever capacity that is? And so my first question for you is, do you feel like that definition is a fair one?

1:02:48Abhi Mahajan: I guess I struggle to see the difference between the two definitions.

1:02:52Eryney Marrogi: Well, one of them is about humans hurting humans. The other one's about humans maybe getting hurt by humans, but It actually could be about humans getting hurt by natural evolution of viruses. It could be about insects harming other insects, insects harming cattle, like with screw worms. It could be about fungi ravaging tree populations in the Appalachian Trail. It could be about all sorts of things that have nothing to do with AI. The trouble for me is AI just takes up all this oxygen when it comes to these things.

1:03:27Abhi Mahajan: I agree more with the second definition, but I guess my stance towards biosecurity is like, I am aligned to your point of probably good to focus on almost the pathogen agnostic tools. I think for UVC, concretely a good thing. If not for someone releasing mega Ebola, just to reduce airborne infections in general. Good thing. I do think the people who are working on these sort of like like being worried about mirror bacteria being worried about uh the like the mega evolved um viruses that maybe like evo 3 or evo 4 will bring about i don't like agree with their worldview but it's the sort of thing where like i'm glad

1:04:15Eryney Marrogi: someone is thinking about it like i'm yeah i i get your point i'm glad someone's thinking about it you and i had dinner we talked about something that actually i feel very passionate about i I worked on gene drives, right? I worked on gene drives, I worked on this idea that we as humans can have agency over biology to eliminate something that's very harmful to us, called malaria. And unfortunately, that work hasn't necessarily translated into impact, not because the science doesn't work, but I think because there's a lot of regulatory, there are obstacles over it, sort of preventing that. And I wonder if it is fair to say that the screw worm problem gets not enough attention relative to, say, like all the red teaming projects that, you know, get probably tally up like over hundreds of millions of dollars towards how many college undergrads can we put in a room to see if they can make a virus.

1:05:18Eryney Marrogi: And actually, there was something I learned specifically from your article. I read it. I read it with the hope of being convinced that I'm wrong. Or I should say, like, I should reframe. And there's a study, you said it in there, where there are like 150 college undergrads, I think. But it's like novice biologists who are split up into LLM versus internet. And their task was make a virus. and you know the citation was like five percent success rate for llms six percent with internet yeah uh and i'm not saying that obviously lms do nothing it's just that seemingly doesn't make any difference whether it's the internet or the llm sure um and then the point that people bring up about ohm shimrikyo and like oh they produced the wrong thing it seems like the thing that actually stops all this from working is actually like physical reality like you have to move stuff with pipettes so you got to order the right stuff you got to have it come to your lab like there are agents you you can't order you simply cannot order them uh we have screening which is good i'm glad someone's doing viral or nucleic acid screening uh to me it feels like this is actually a solved problem we don't need to think about it too much anymore like let's move on let's actually put money into stuff that matters and you know i brought up the screw worm because something i mentioned to you so it's like only one lab like in the whole country we have a country of 300 million people two percent of them have a phd in bio or a phd in something uh so you know we have millions of people with phds hundreds of thousands of them work in biology um and yet we have like one lab of like a dozen people that work on screw worms yeah and they got like a huge grant of 500k to work on solving this screw worm problem and then i see all these startups who kind of are getting dozens of millions of dollars each to work on AI red teaming for biosecurity.

1:07:15Eryney Marrogi: Yeah. It feels like an asymmetry.

1:07:17Abhi Mahajan: I agree. I agree. There's an asymmetry. I do think there is a sort of like a bias towards, I think there's two things. I think people like sci-fi things a lot. And I think it does influence a lot more of their decision making. Than they want to believe it does. Yeah. Yeah. I do think there's also this bias of like, maybe this is connected to the sci-fi section, or sorry, people like sci-fi stuff. They also think the sci-fi stuff is more likely to be existentially risky than the stuff that's actually real today. I think if you really wanted to save a lot of humans, the most legible way to do that today would probably be to improve the number of vaccines that are delivered to people in like lower middle income countries i think the the biosecurity people would look at that and say yes but uh what if what if they like the megaviruses come about and they wipe out all of humanity so like someone ought to be working on that sort of stuff i like again like i don't i don't agree with it but at the same time i also do view a lot of these things as like dual use where if someone is helping create like um like something i i i posited in the article that um like here's here's how ai might help with like the defense problem in biosecurity and a lot of what i talked about feels pretty applicable to a lot of therapeutic areas of like we probably should be learning how to fc engineer antibodies such that they last months in your bloodstream so we don't need to discover a new immunogen for every single like crazy virus that comes about which like yeah maybe the crazy viruses are um like designed by north korea but maybe the crazy viruses are also just like tuberculosis right um and uh we probably want to invest in really good biomanufacturing like like have a really good biomanufacturing um like set up in the united states so we can defend ourselves against bioterrorism but we also want to have that probably so we can distribute these to other countries for their more banal viruses.

1:09:27Abhi Mahajan: Like, I do think like the pure red teaming of like, like, how do you ensure the model doesn't teach people how to make buyers, the buyer, buyer weapons? Like, I think I could go back and forth as to like how useful that is. I do think the therapeutic biodefense companies are like a lot, like fairly dual use. So they are sucking up, I guess, like the intellectual oxygen. but the actual fruits of their labor i think apply in many different areas i do take your point about like agricultural biothreats are very under focused on and i like i talked about it in the article and i would define like put screw worm underneath that as well i don't know why this get doesn't get that much attention other than it's like it's like i like wrapping up to like the

1:10:13Eryney Marrogi: sci-fi thing i think it's a boring topic to think about no yeah i i yeah i take your point yeah i And yeah, maybe to be most clear, I am glad that there are people working on things. I'm glad that Stripe has their Intercept fund and they're going to try to work on many dual-use things that are about trying to reduce the global burden of viruses. This is good. I'm glad people are doing this. I am glad that like there are some people like Red Queen who are working on finding the boundary of like what should and shouldn't be possible with AI. the challenge for me is really just towards that last bit that like and maybe it is just about like sci-fi maybe we actually just need to like hire like hanu to like only write like books about uh you know fungi that like kill all the trees and therefore we uh don't have any national forests anymore uh and then people will care about it um but it is it is troubling to me that so much of the conversation and I know that conversations happen online that are not necessarily every conversation that happens offline right it's not always the same people but as we know a lot of people who are about to become extremely wealthy and deploy a ton of capital are online and they do really they read all this all this stuff that I'm sort of referring to and they take that in.

1:11:49Eryney Marrogi: I've certainly had many conversations where I'm like, wow, I totally can't even fathom why you would believe that other than you read it on a sci-fi website. But people believe these things very deeply and in such a way that it's dogmatic. You can't convince them that you're wrong the same way that maybe you can't convince me that I'm wrong. At the same time, I do see there is some data. through actual studies. So like Om Shemrikyo is an anecdote, but we now actually have studies where we've tried to compare the impact of LLMs on acceleration of bioterrorism, so to speak. Turns out it doesn't really accelerate.

1:12:34Eryney Marrogi: And maybe you can always say like, oh, well, that was, oh, that was GPT 5.1. Really, we need to use GPT 5.6. And then the, you know, if it ends up being head to head the same, it'll be like, oh, I actually need GPT 6.5. Like that'll be the, And this is sort of the obnoxious component of this argument, but it feels like at some point you have to say, like, actually, the thing stopping all of this is that one, people F up in the lab constantly, even really highly trained people F up in the lab. Two, ultimately, state actors are going to be able to behave however they want to behave. And you can't stop them.

1:13:09Eryney Marrogi: Big countries, small countries, regardless of their size and their wealth, they can do whatever they want. you know al-qaeda was not necessarily a state actor but they were state level in terms of their financing they could afford as much bioterrorism as they want to wreak uh if they if they want to do that and you can't stop them right um so like what are the things you can stop i think the dual use like finding those dual use problems are are the thing right if you can like make a pill you can take and it makes you immune to like not just this season's flu but it actually makes you immune to like all the viruses from this year or like it jacks up your immune system in such a way that it makes you extremely resilient to viruses or something like this or maybe you can undergo like a genetic surgery like the most sci-fi world you undergo a genetic surgery to like remove all these viral elements from your genome like maybe these things are are really good and i agree with you i'm glad people work on stuff like this really maybe the point i'm just trying to make very selfishly on this podcast with a lot of people who are very well off, who listen to this, is that I think really trying to explore deployment of capital to things that have nothing to do with AI, but are focused on biosecurity, is maybe where the attention should go and say, like, we've actually put enough money into AI and biosecurity.

1:14:40Eryney Marrogi: Like, we can focus on other stuff.

1:14:43Abhi Mahajan: I do think the frontier labs themselves have a pretty strong incentive to ensure their models aren't spearing out instructions on how to do bioterrorism. I would almost trust them to take care of that. And most of the like this like third wave of philanthropy should go to places which like kind of like cut the difference of like it's useful for public health is also useful for the edge case of like some terrorist group decides like they want to work on the virus. I do think, like I posted this meme on Twitter once about how like I think biosecurity discourse has this like repeating midwit curve of like there are ways to think about biosecurity that are very dumb, that are paranoid, and there are ways to think about biosecurity that are very dumb.

1:15:39Abhi Mahajan: They're also not paranoid at all. And I think the most correct way to think about it is broad security measures are always a good thing. And broad antivirals, good. Someone probably ought to be working on phage therapy, just in case. You and I maybe disagree on this one a little bit. We probably should be fitting buildings with our UVC. and also the boring parts of right now, there's not really building regulations for what defines clean air. There are HVAC standards, but they're not super biosecurity-pilled from even a public health standpoint. And it's probably good to develop those despite everything else.

1:16:27Abhi Mahajan: I think the... And so one thing I do want to say about the active site study is that people are split on what that study actually means. What are they split on? They're split on similar to how we were talking earlier about like, are these AIs useful for developing new instrumentation technology? They probably wouldn't be useful for like me who is using it, who's like trying to think like, how can we make better mass spec? Because I barely know how mass spec works in the first place. Wait, wait, hold on. I actually disagree with this. But yeah, OK. I want to come back to this. I think AI for interesting scientific discovery is usually best ran by people who are like experts in that particular domain because they best know when the AI is like leading them in a wrong direction or like has bad taste in ideas.

1:17:17Abhi Mahajan: And I think a better active site study would have actually used experts for a harder wet lab task.

1:17:23Eryney Marrogi: I that I mean I think is fair right but isn't that kind of like the point where it's like does an expert even really you know not I would never do this but if if let's just say there's an RNA prime he exists on earth prime it's not me not earth uh and he wants to make smallpox prime like I I guarantee you I can make smallpox prime and I wouldn't need to use chat to be to do it I guess like it might it might help me do it like in a couple weeks instead of like a month but I guarantee you I don't need I don't need Chachibutti to do it like what if you uh

1:18:04Abhi Mahajan: people are people are gonna get upset about the maybe we should we don't okay okay all right um but like I I think it feels very likely to me that there are people who are kind of on the like so there are a limited number of RNA primes on Prime World, you know? No, I think there are a lot. Like I said, I want the left tail. I think like most of the RNA primes are going to be working in like well-to-do jobs where they have like little reason to hate society. And they'll keep this knowledge in their head where kind of it belongs. I think there are going to be plenty of people in the world who have some reason to hate this particular other section of society.

1:18:54Abhi Mahajan: And they're kind of on the edge of knowing all the steps they need to do something that is very harmful for the world. I also think there is a sort of a... You can imagine scenarios in which you want to improve particular facets. Actually, this is not a brain worm because this is documented in the essay. But like, they failed with their initial virus release attempts because whatever encapsulation they use for their virus just like prevented it from. Yeah, they use the vaccine strain, I think. No, that's one thing. But another one was like they had the correct strain. They released it, but they released it in like broad daylight.

1:19:43Abhi Mahajan: And so like UV immediately neutralized it. At least that's what they think. Who knows? This stuff is a little bit unknowable. and maybe if there was an LLM helping them throughout that process they would have very reasonably said to probably release this at like 7 p.m or 8 p.m instead of what do you think

1:19:59Eryney Marrogi: so I like to be honest with you I think the challenge with LLM slightly switch general use is and the reason why being an expert or at least knowledgeable in the domain you're prompting about is like you kind of know roughly what you might not know exactly the fact but you might be like doesn't uv kind of affect you know dna stability i don't know like and you might and you might prompt it maybe you wouldn't ask like what time right but and again this is a little bit of an info hazard but it is documented but there's a little bit there's an element of like the expert kind of knows enough sort of like you know i'm not an expert i'm not an expert software engineer I used to do pair coding with Steven Molina, a friend.

1:20:47Eryney Marrogi: And it's like, there are things that he would say to me where I'm like, oh yeah, okay, sounds good. Because it just didn't occur to me, right?

1:20:53Abhi Mahajan: But that's why I think an ideal active site study would focus on experts, because experts do kind of... But do they need to? I guess I view it as experts have a lot of things they need to be concerned with. Right. And maybe the LLM just helps as a thought partner. I guess so.

1:21:11Eryney Marrogi: I mean, like, so you're saying like, oh, the real, the really good study, the really good study to assess the capability of a model is not a novice, but an expert, because that differentiates, call it like, it differentiates the high level athlete from the elite athlete, right? And it differentiates like the bubble roster player in the NBA from like LeBron James. That's kind of like what the study is, like, can you actually just, is, Is Chaijiwiti a steroid from already like really good scientists?

1:21:41Abhi Mahajan: I think of it as that. And at the same time, like Om was kind of in this unique position where they had like a little bit infinite shots on goal. They had a lot of members. And like the Japanese government didn't care about what they did. Right. But you can imagine like the thing I said in the essay that I'm much more concerned about like terrorist groups who have a limited number of shots on goal. They know like every time they like poke their head up and they try to do something crazy, there's a chance like it'll get caught by someone. And so they really need to make sure each individual shot is as good as possible.

1:22:13Abhi Mahajan: And I think the LMS are helpful for that.

1:22:15Eryney Marrogi: Did you read the article about the terrorist group in I think it was Burkina Faso, the Chachubiti, or maybe it was Mali? Oh, vaguely. Yeah, this article. they were using chat chubiti basically to see like oh can we do terrorism better um and i won't go into the details just you know read the article right but there was a specific scenario where they described like oh we asked chat chubiti how to like improve our ability to do this we'll call it performance stunt because it really is like a uh evil knievel type stunt and if you read the article you'll take you'll understand what i'm referring to and then you know later and everyone I saw this screenshot and I was like, oh my God, it's like you're enabling terrorism.

1:22:58Eryney Marrogi: And then it's like, oh, did this increase your ability to do the thing? And it was like, oh no, it didn't work out. It was just fun to talk about. It was just fun to talk about doing Evo Knievel stuff with your boys and a bunch of explosives. And it didn't work out. I'm not saying, I understand what you're saying, right? It is a very reasonable and somebody should think about these things. Yes, it's possible that I'm actually wrong and maybe we need to focus more on screening my knee-jerk reaction on screening stuff is like it just introduces screening specifically meaning nucleic acid screening my knee-jerk is that whenever you introduce

1:23:40Eryney Marrogi: regulation and if the regulation is not doing a lot it's probably not a good idea yeah which is why generally speaking I'm like kind of against regulation for DNA synthesis beyond like the company's already doing what they're doing right now. Like why introduce regulation? They're already doing it. And the ones who aren't doing it are in countries that are not our own country. So we can't stop them from not doing it. Right. Maybe you can say like, we can only do business with countries that do it or whatever. I don't know. So it's possible I'm wrong in this case, which in all, you know, in that instance, I would say we've addressed some bottleneck, which is like synthesis.

1:24:18Eryney Marrogi: And maybe the next bottleneck is like the fact that you can order a benchtop synthesizer maybe we should make licenses for this i don't know whatever i disagree with that but maybe this is actually a reasonable middle ground what else do you need to do okay like i i also don't think it's like

1:24:34Abhi Mahajan: the the most onerous thing in the world to have like some degree of know your customer

1:24:39Eryney Marrogi: you know that's fine i mean i think that's fine but this does exist right go try to buy right now go try i would like you to buy uh hgh go buy hgh go start uh owl posting llc and you know spin it

1:24:55Abhi Mahajan: up and go buy hgh but i guess like you can buy that off like the like the the deep uh like like

1:25:02Eryney Marrogi: silk road you can do it illegally sure you do it illegally so right so there are illegal markets

1:25:07Abhi Mahajan: for everything no i get that but like if i try to go buy like a pound of uranium or enriched uranium probably a bit harder to do, right? Because, like, we have really strict, like, standards as to, like, like, if any uranium is ever lost, there's, like, a worldwide manhunt. Right. I feel like...

1:25:22Eryney Marrogi: So you're saying we should have a worldwide manhunt for, like... I think for, like... My primer that amplifies, like, you know, tack polymerase.

1:25:29Abhi Mahajan: I think for at least... So I can... Never mind, go ahead. I do think, like, the whole thing where you're going to, like, like plasma fragments to create like horsepox virus. Like that did make me downweight the value of having any sort of regulation over bench top synthesizers. But I don't know, I did get a little bit pilled on the Swiss cheese security model of like, you make it hard for people to do the dumbest version of their plan. And maybe that just prevents a lot of tragedy. And so I think we aren't there today, but I think the second enzymatic DNA synthesis really like hits prime time.

1:26:11Abhi Mahajan: I would support like a kind of like be careful where these things end up.

1:26:16Eryney Marrogi: So you're saying like right now, so I guess just for context, what you're saying is right now you have to go through these companies like Twist or Agilent or your stuff. But maybe if people succeed with enzymatic synthesis where it's more like a protocol you can run in your own lab, then we should prevent people from doing that.

1:26:37Abhi Mahajan: I think I think there's like like. Or we should find some way of controlling who gets to do it. Either there's a very strong know-your-customer setup, or there's some sort of hardware lock on it that has a very good... It needs to be connected to the internet at all times. It will screen whatever thing you ask to make via some data set of... Both public and private data sets that have been curated by, say, SecureDNA, by whatever you're trying to create. and like maybe that's not perfect but like it's also like not that onerous for the people who are actually like trying to do their synthesis and like that's maybe completely like legal uh and like it places all the onus on the people who are actually selling the bench shop synthesizer as like like you have to you have to have these things in place to ensure that your machine is not going to be used for evil in the most dumb possible way i think it's fair to look at that and say, like, you've gotten rid of, like, the worst possible cases.

1:27:41Abhi Mahajan: What about, like, the more clever people? I think it's fair to, like, throw our hands up at that point and say we'll find other ways to deal with those people.

1:27:48Eryney Marrogi: I guess, yeah. So, I mean, towards the second group, I think, yeah, maybe it should come closer to your argument where, like, we should do more to stop, like, the dumbest person on the planet from, like, doing something bad. Maybe we can do more for that. I don't know. It seems like screening kind of does that right now. And we didn't need regulation or we don't need regulation to do that. But maybe a synthesizer says changes, in which case we need some regulation. That seems reasonable to me. The second group, and I've had this discussion with very many smart people, smarter than me. But I feel like I can't quite get it.

1:28:23Eryney Marrogi: But you don't think that like the reason why there's so much of a fascination with like Ted Kaczynski is because it's so absolutely unique to have somebody be both extremely intelligent and also like evil and like basically on a mission to like end society's, you know, stability and whatever. There are really not that many people like this. I mean, I like I. And the ones who do will just like you can't stop. You cannot stop people from coming up with these plots that involve understanding social dynamics. It's just like it is impossible. I do think as to the topic as to why agro-terrorism or bio-threats to agriculture are less focused on,

1:29:11Abhi Mahajan: I do genuinely think there is sort of like an optics problem of like, it's a lot scarier to proclaim that someone's going to design a virus to make you bleed out of your eyes rather than someone's going to design the virus to depopulate wheat when you live in a first world country

1:29:25Eryney Marrogi: and think to yourself like America or wheat somewhere else. We'll just get it from another country, which, by the way, has a natural virus that makes you bleed out of your eyes.

1:29:34Abhi Mahajan: I do think a lot of these things come down to optics. And I do think there isn't a strong rational reason to trust. Like a lot of biosecurity is actually one of my favorite tweets. A lot of biosecurity sentiment has a sort of like Pascal's Wager vibe to it of like the downside is so infinite that you have to trust me regardless of how atlantic the thing I say is. I also don't think they're entirely wrong.

1:30:07Eryney Marrogi: I think there is a kernel of truth there. Yeah, I guess so. I mean, the thing that's and, you know, this this is this is not rational of me. I, you know, this is irrational of me. But I will say it is curious to me that we apply this constraint to biology when you can actually do so much more damage currently without any sci-fi scenarios with really good coding models. Sure.

1:30:34Abhi Mahajan: I think we're plenty paranoid about cybersecurity.

1:30:37Eryney Marrogi: Right. But like we kind of let whoever wants to use cloud code use cloud code. Sure.

1:30:43Abhi Mahajan: Yeah.

1:30:46Eryney Marrogi: I agree that it's weird. It feels like an asymmetry. This is all about asymmetry. I'm not like, well, you know, you let him do it, so why can't I do it? That's not really it.

1:30:56Abhi Mahajan: I do think part of this, and this is also something I don't understand very well, is clearly there is probably a really huge economic downside to not allowing coders to just use cloud code. You don't think it's an economic downside? No, this is the thing. I'm a little bit like I do hear biologists talk about like a lot like like like biosecure shown exist uh like like xyz biosecurity regulation showing exist because it slows down my work I've never worked in like like that particular type of wet lab like does it actually slow down

1:31:27Eryney Marrogi: their work I will say like maybe it's hard because all these like biosecurity legislative pushes are newer um but there are call them like health data and other types of legislation that are in the same vein like it's all about protecting people from the frontier of like biology and medicine uh or at least the downsides of those things those things legitimately do slow you down not because in the moment they slow you down but it's like in 30 years when it no longer is like a like i think like a basically the argument that these people are making and like i certainly would probably be one of these people as well um is that it's not necessarily that making it harder to order dna today stops me from doing anything because ultimately like you know at least i haven't or i haven't ordered dna like a couple you know 18 months or so but when i was ordering DNA, it was like, I'm making, you know, uh, plasma or protein binders.

1:32:27Like none of those

1:32:29Eryney Marrogi: protein binders are going to be things that I expect are going to be troublesome. And if they, if it gets flagged because it's like close to something that I missed, it's like, I, I shouldn't be working with that anyway. And that's totally fine. Um, it's, I think it's more like, what does it look like in say 10 years when the specifics of the law that you wrote now start touching on some new technological development. And now you're doing funky things like, well, we can't collect data from this state because of the way that their law is written. Actually, a great example is the state of New York, which we're in New York City right now, has quite backwards laws when it comes to patient biopsy samples.

1:33:13And a lot of trials just go like,

1:33:17Eryney Marrogi: yeah, we're just going to exclude New York. Like, even though we have this massive city with tons of patients and tons of money and, oh, Sloan Kettering and NYU and New York Presbyterian, it's just a hassle. We're just not going to do New York. We're out. That's really the concerning thing most people have. Yes, there is a component of, like, the libertarianism that I think a lot of people are drawn to, which is like, I want the government out of my industry because they don't know what they're talking about. that is maybe like the heart the most hardened stance in this particular case but i think a softer stance is like fine make the law but make it expire in five years and if you think it did anything you know renew it yeah but don't have it last forever and this is a general principle regulation right it's not like a biosecurity specific thing and then the second aspect i'll say, and again, like every, every person who holds these stances probably can speak to a different thing they care about, but, um, it's like, well, who's going to actually dictate who is an acceptable person to order DNA?

1:34:27Eryney Marrogi: Is it the academic? Is it the industry person who has a PhD from an academic institution that's approved? Is it, uh, an industry PhD who's allowed to order these things? Is it going to be an MD? Is it going to be someone who got their PhD in a foreign country, but moved to the US? Is it going to be a permanent resident? Like, these are all things that, you know, these are details that matter. And these things actually end up decelerating biology or decelerating technological progress. And I think a good example of this is every AI person freaked out when the US government said, you know, we're going to limit access to like fable, I think, because Anthropic has a ton of employees who are not American citizens.

1:35:15Eryney Marrogi: And I think justifiably, they were upset because a very specific instance of federal government legislation applied to their industry. That is why biology, biologists who actually do biology, who actually manipulate DNA, who actually do things and make drugs that help people live longer get a little bit upset and paranoid because yeah i mean it sucks when somebody comes into your industry and makes

1:35:43Abhi Mahajan: it harder for you to do things even in a theoretical way yeah i do think i guess like there is the um today i think like the biologists are like absolutely correct i do think if you have like your AGI pilled hat on, you can imagine a future where maybe lab automation really kicks off. And the number of biologists necessary to have in the world to make biological progress move is actually far lower than it is today. And you can actually have these few trusted individuals who are shepherding in the future of biology. I don't think that's feasible. in like five to 10 year time horizon. But I imagine that's what's going through the heads of people who are very biosecurity pilled.

1:36:34Eryney Marrogi: And it is worth saying that I absolutely understand the reason why people care about these things is because they are very worried about, as you said, the unlimited downside scenario. And I can understand, like if you can have any capability to model someone else's worldview, you i can understand why somebody who has worked in ai who saw scaling solve their you know their problem that they care about even are all with like self-driving cars whatever like i can understand how you'd be like well this is going to happen to every industry now obviously all the worst things now become possible i can get that i guess i don't understand why these people don't also understand the flip side which is like they're upset when their industry gets regulated why are they why can't they understand that other people might be upset not even just biologists right chemists uh environmentalists uh petroleum engineers whatever everybody in their own field

1:37:36Abhi Mahajan: gets upset i guess like a part of my worldview is that is probably good the united states try to regulate the models yeah like i purely because like i don't think like like fable or mythos is necessarily like destabilizing to society but i think at some point the models will be and so it's probably good for us to get some appetite for like eventually the government's gonna like poke their

1:38:00Eryney Marrogi: head in and ask like what's going on here which which is totally cool i guess the difference is the frontier labs don't well the frontier labs were not really limiting who got access necessarily then eventually they did no uh the good news is twist is perfectly happy to screen the dna and they didn't need the government to tell them not to do it.

1:38:22Abhi Mahajan: But like there are, and this is actually why I think like, I think there are pathologies to buy or secure, but I think like, sure, Twist, US-based company, they're happy to do it. As like the DNA supplier in China, are they willing to also do the - Some of them do,

1:38:39Eryney Marrogi: but they also don't really believe in this as much. It's worth considering why they don't. And I think like we don't, you know, we don't have the, this is something you could probably talk about for hours and hours and hours, but it's worth considering why they don't.

1:38:50Abhi Mahajan: I guess like if I were to put my like, yeah, like put the AGI pill hat back on and be in the position of someone who's actually like reading the transcripts that people are putting into a cloud or chat. I have no doubt in my head that like some of those chats like look really scary. And they do push you towards this like maximalist worldview of like, we need to like shut things down this instant. and i think like uh i think in in an ideal world there is like a biologist sitting alongside these people saying like some of the some of the stuff isn't as scary as it looks but like i can understand their reactions and i sympathize them with them a lot like i think um a lot like when i wrote my biosecurity article there were at least a few people who messaged me saying like you're not paranoid enough like like the world the world's gonna explode in a few years and like i don't know I get it.

1:39:43Abhi Mahajan: I get where they're coming from. I hope they continue screaming as loudly as they can. I also hope the other side continues screaming as loudly as they can. And hopefully we'll end up, I guess like it's a boring position to be like, everyone has a point. But I genuinely think everyone has a point.

1:39:58Eryney Marrogi: I do think everyone has a point. Again, if you consider their mental model of things the the the maybe the question is is the thing we should focus on like stopping the low-hanging fruit sufficient to also be stopping like people who are actually doing important work um is that really the best approach and i don't know i don't know and maybe maybe maybe it is and maybe they have maybe they do have access to like all these prompts from like people that i don't interact with who are like asking chat gbt like to order them like you know uh from bomb.com order me a bomb right like i have no idea maybe people are doing that but um in the absence of that evidence apparent to me i maybe my hope is that people can understand or at least build in an element of, call it, deprecation to some of these laws.

1:41:08Eryney Marrogi: And that's the thing that I think everybody wants about regulation in general. Let's actually do it with biology. You know what? We're all smart people. Most of them are, a lot of them are smarter than me. I think that they can understand the merits of that, at least. So I don't know. There is a middle ground component of things. I think a really interesting blog post for someone to write, and I hope someone does this, someone who's listening, is let's say biology did get heavily regulated in whatever way the biosecurity people really want.

1:41:41Abhi Mahajan: What actually changes to the bio industry? I have no idea. You clearly have some idea, but I would love to see a quantitative breakdown of here is what research immediately becomes almost impossible to do. And what is the economic impact of this?

1:41:56Eryney Marrogi: Can I present the future that I expect to happen if you go with the maximal regulatory? It's the one where China actually just destroys American biology. That's the future, actually. That's actually the future. Yeah. Because they don't care at all about this stuff. And you could argue that they should care a little bit, but they don't care. They don't even think about this stuff whatsoever. And maybe today it doesn't matter because their models are worse than ours. But, you know, you read the same stuff I read. Their models are getting quite good. At some point, and they have a lot of highly, highly technical, exceptional scientists.

1:42:34Eryney Marrogi: Maybe they start demolishing American biotech such that it doesn't matter that you solved HGI. And this is why I would like to see that blog post. I could totally imagine that your version of the future where we pass all these regulations,

1:42:51Abhi Mahajan: it cripples our biotech industry even more than it's already crippled and China like takes the

1:42:55Eryney Marrogi: lead. To be clear it's not just regulations right it's like even the idea that we should screen bioprompts and like not every other type of prompt. Sure yeah I think I think there is

1:43:07Abhi Mahajan: like to some extent I feel like like like China's like a little bit already like on route to the future that you mentioned. We can hit the we can hit the exponential if you want to. Yeah um I do take your point though and i don't know i i hope the smart people involved in this discussion find some like pareto optimal place i do i do wonder if like there will be some flip in china at some point where they do start to care a little bit because like you like uh like like china wasn't really like a big force in the world after like during the big anthrax attacks but like maybe there will be some big biosecurity event that happens in the next five to ten years and china does also start to get a little bit spooked about it i think it feels unlikely to me that the the paranoid people are so clearly off base and that there will never be some like major biosecurity threat in the future.

1:44:13Abhi Mahajan: I think it probably won't take the shape they expect it to. But I think it will happen someday. And China will also be a similar degree of concern. As like, not perhaps like not to the same extent the United States is, but I think they'll lose something. It just feels like impossible. Like one example of this is - How about COVID-19? Oh, that's a good point.

1:44:39Eryney Marrogi: Right. Do they do anything? Well, they shut down the country. Their countries, their cities were shut down. But they also had a, like, you know, maybe this is probably, I wasn't in China, right? I was in America. I was in Boston, which, like, was a completely dysfunctional, it was a mess, it was a nightmare. My understanding of things is that in China, they, like, shut down the cities, and everyone had to stay in their home, and, like, food would come get delivered to them. The army would, like, deliver food and stuff. they at least took biosecurity seriously if the application of it is like basic public health preventative measures.

1:45:13Eryney Marrogi: So like once the problem arose, then they implemented it. And went into a sort of like reactive mode immediately. And maybe in America we have to care about this more because we have poor institutions. Maybe that's what I'm learning right now. And maybe this is one of these like East versus West things where it's like, well, in the West, we just think really hard about it and therefore doesn't happen. And in China, it's like, well, if it does happen, we know we can handle it real quick. I don't know. Maybe actually this is it. Maybe this is it. And maybe this is like a philosophical thing that is embedded in my mind, maybe in my genome from my parents being maybe from the Eastern block of the world.

1:45:57Abhi Mahajan: Because I don't know the full details about this, But I do know China has made their, like, investigator-initiative trial things a little bit more stringent over the past year. I know, like, I'm not sure if I'm, like, prompting any article. Like, I do know it's not—

1:46:15Eryney Marrogi: I mean, there was just the AAV-9 trial where the kid died because the PI was being negligent.

1:46:21Abhi Mahajan: And so, like, a lot of regulations are historically written in blood. And like, I know, at least in aviation, this is true in the United States. I think we're like jumping, jumping the gun for biology. I wonder if China will learn from like first principles, like which, which regulations are worth taking seriously and which ones aren't. Whereas we're going to speed run like everything all at once and then we'll slowly dismantle the things that seem to be too onerous.

1:46:46Eryney Marrogi: Yeah, I don't know. Maybe, maybe there's, there's so maybe one way of putting it is in America, we trust anthropic and open AI to surveil us, our behavior. And in China, the government is like, we have the optimal surveillance machine. So we know how to stop people from doing bad stuff because we're always watching them. Maybe this is just it. Maybe that's the answer. And maybe we do just need to say, all right, I'll just keep using Claude Opus 4.1 while Iranian hackers use Fable 5 to destroy the American water system. And maybe that's the way, well, we generate a lot of money from writing software, so it's okay.

1:47:35Abhi Mahajan: I think a lot of this wraps around to maybe regulation isn't necessary if we had a sufficiently good therapeutic apparatus for dealing with the problems when they arise,

1:47:46Eryney Marrogi: which is why - This is the accelerationist view, right? This is like AI should maxillate us. And like, yeah. And maybe this is the question is more like, is biology like attack or defense? You know, what is a favor attacker defense? Well, it clearly favors, like, okay, actually,

1:48:04Abhi Mahajan: I think it clearly favors attackers. I think there are some defenses as to why defense also, like, doesn't have like that. Human immunity is a pretty good defense. Immunity is a pretty good defense. if you get really like there are such thing as broad viral protectants there are like physical barriers if you deploy glycol vapors and fire uvc everywhere like maybe like rendering the internal environment continuously hostile to pathogens actually is a really good defense and like attackers would have to spend like enormous amounts of money to like hurt anyone um i think if we got that whole apparatus built out i think like defense actually becomes pretty

1:48:47Eryney Marrogi: easy yeah i mean i guess you know i remember having a discussion with with us with an open ai employee i won't i won't name them here but um about like what you know what happens if you just completely let down your guardrails and you say like ai models are allowed to do whatever and like let's really think of let's just like digest this a bit because this has this has now been like something in the ether so i don't think it's too dangerous to discuss but let's just say there's a virus historical virus and it is um you know some percentage of people who get infected with it die and it's bad um didn't we just learn that you can just quarantine maybe in america people don't quarantine didn't quarantine during covid because of like all these public health messaging things and like the politicization of COVID.

1:49:44Eryney Marrogi: But maybe if it was like virus X, which I won't name, that is historically known to be super dangerous, people will stay indoors. And now you've stopped super spreading. And now we can wait, genuinely wait for the spin up of vaccines. I don't know. Like this is the other view of it. It's like, can you sufficiently capture? Like that seems to be what China's bet is. Can we bet on the same thing in America?

1:50:10Abhi Mahajan: I think for COVID, yeah, like it wasn't like known to be like incredibly dangerous. So people were willing to go out and about. I think if it was incredibly dangerous, people wouldn't have gone out and about. And maybe the virus would have like burned through people quite fast. So it would have like died pretty quickly. I did have an interesting conversation with someone who I'm not going to name for the for the biosecurity article who told me, like, thank God COVID did not target children. Because if it did target children, people simply would not leave their house, one, but two, would also not be working on vaccines.

1:50:52Abhi Mahajan: They simply wouldn't leave. They would just stay in their home and they don't have the authoritarian government that is China, like pressing down on them to let go.

1:51:00Eryney Marrogi: This is not true. How many scientists have kids? At Harvard, none of them had kids. At Dino, I don't know. We had 150 employees. When I was there, maybe like 10 % of them had kids.

1:51:16Abhi Mahajan: But importantly, you're still cutting down on the workforce and some meaningful capacity, because it's not just scientists, you know, it's people who are working at the GMP places. Yeah. And I think if you're like, like, cutting your work workforce by a substantial fraction, I don't know what that fraction is, like that, that does slow down therapeutic development a fairly high amount.

1:51:38Eryney Marrogi: Yeah, yeah, I, you know, yeah, you got me. I don't know. I think there are there there there biology is a is a is a is a sort of field of confrontation sorry this is kind of like the thing ultimately it's like you have to either be ready to roll around in the mud and maybe uh accidentally infect yourself with some AAV in the process of making extremely exceptional therapeutic viruses uh or you go like well we'll just like operate at a speed that seems acceptable like i will always slow i've never spoken to someone who

1:52:15Abhi Mahajan: works in a bsl3 or 4 lab i'd like to get their thoughts as to like like maybe this is chill like they're they're they're they're happy with doing therapeutic development even in dire times like uh when there is like a super bug going around and this wouldn't actually slow them down at all.

1:52:33Eryney Marrogi: I don't know. N95s are pretty good. Wearing gloves is pretty good. But I guess like... Using Purell is pretty good.

1:52:39Abhi Mahajan: It gets into your head. I imagine. Like, I'm not even talking about, like, the physical impacts. I'm talking more about emotional impacts. Yeah, man. You can imagine, like, viruses that have much more, like, physical displays of, like, suffering that actually does, like, again, like, cut down on the workforce again.

1:52:57Eryney Marrogi: Yeah. Maybe it makes you bleed out of your eyes. Yeah. Yeah. I, yeah, I would say we have really, to be clear, when I started, when I imagined this podcast, I actually didn't imagine us talking about bi-security this much. I really was like, really make this a platform to talk about. Like screw worms. Screw worms. Like a scientist whose lab I think is like highly underfunded, Max Scott at NC State for any listeners who have the means, like by all means, please do look into his work. I think it's extremely important and extremely underfunded, especially right now. I didn't expect us to go into all this biosecurity stuff, but I'm glad we did it because I have versions of this conversation with so many people in AI.

1:53:43Eryney Marrogi: And I would say, and I don't mean this in an insulting way, they know as much about biology as I know about Frontier LLM infrastructure, which is nothing.

1:53:54Abhi Mahajan: I do think the life, have you spoken much to like the life science people at the Frontier Labs?

1:54:01Eryney Marrogi: Yes, I have. And I would say even then, like they don't have that many people who actually have like put their hands on a pipette before.

1:54:10Abhi Mahajan: I, yeah, I'm also like reasonably far away from a pipette. My impression is that things are getting better over time. I think like when the LLMs first popped up in like 2022 to 2024, I think people were freaked out all the time for reasons that I couldn't quite grasp. They didn't have a good theory of change for how this stuff actually ends up impacting humans negatively. I think as time goes on and the frontier labs start to gain like a stronger life science team who has actually worked in a lot before.

1:54:45Eryney Marrogi: Typically, I think they do employ very strong people. And there is a question about whether like you need to actually work in a lab to be like good at biology. But I think that there are issues that crop up that require you to do them with your hands to even know. But go ahead. I think their opinions are becoming more grounded in reality as time goes on.

1:55:07Abhi Mahajan: And they're learning what is a real problem versus what isn't a real problem. I do think there has been this, maybe you don't see it in like I'm talking to a very select group of people. But when I talk to people in biosecurity, they do seem pretty, like the people who work on pure LLM work, they're freaked out about everything. The people who are working on biosecurity XLLM, they do generally agree with like broad defenses are better. Let's not get too paranoid about like mirror bacteria. Yeah, it's like I think as time goes on, the like the culture in certain fields becomes more and more sophisticated and matches up with what actual domain experts think.

1:55:50Eryney Marrogi: Yeah, absolutely. And, you know, final final note here is like I'm not trying to straw man the position of Frontier Labs. I think the final thing is just I'm glad that people are thinking about I'm glad that people are thinking about these things I hope that there's more room for the integration of like differing opinions because it's some extent right now it does seem a little bit of like well we were right about scaling yeah so we're right about your stuff too and I think that can be a kind of it's both off-putting and I think also dangerous to thinking in general. In the same way that I've shifted quite a bit towards the position of pro-sum measures through discussion.

1:56:37Eryney Marrogi: So thank you very much, Abhi. We discussed a lot here. Maybe the big takeaways are that we hope that people will work a little bit more on tooling and instrumentation to generate data. And we hope that people do a little bit more when it comes to environmental hazards and funding goes towards that stuff, which is also really important. And also that you should give money to Max Scott's lab at NC State. Do we agree with all three of those things? Yes. Okay. And then, okay, so you're leaving New York. And so we have something, you know, you're a guest of the show. So we want to make sure that you remember New York when you're in sunny San Francisco.

1:57:26Eryney Marrogi: So here you go. If you want to just open that up. Here you go. Oh, you're going to be disappointed by the bag.

1:57:43Eryney Marrogi: Did you pick this up on the way here? I picked it up yesterday. I picked it up yesterday. Wonderful. I passed by. I was like, you know what? He needs one of these.

1:57:53Abhi Mahajan: I feel like I lose hats very frequently. So this is a welcome edition to have.

1:57:59Eryney Marrogi: Thank you. Thank you very much. Thank you. All right. Well, thank you very much, Avi. It was a pleasure.

1:58:19Thank you.

1:58:49Thank you.