🔬 Automating Science: World Models, Scientific Taste, Agent Loops — Andrew White

28:56That's not always the case. Yeah, I've actually seen that myself. Yeah, but there's a lot of gotchas and I think people can miss those, but I think they're actually pretty good. And so I'm not as worried about hypotheses that can fail fast by an expert looking at them. I think now the filtration process really happens in literature. And I think the filtration process happens in looking at like, you know, like a biobank data or like, you know, what do we know from GWAS or something? You know, other sources of existing data as much as you can draw upon. Yeah. So, yeah, with regards to existing data, another contrarian take is that oftentimes the hardest part is just understanding the context of data and where it comes from and how do you interpret it.

29:37I can also think from my own life, multiple cases where the data in some sense was there and you had two people who were both experts and very smart people who looked at it and drew very different interpretations. In fact, when we were interviewing Heather Kulik, she had some fun stories about using LLMs and she would find that there would be raw data in a paper which wouldn't agree with the conclusions of the actual paper. And it's straight from the paper. It's not even like cross paper talk or something. Man, I'm going to be a really boring interviewer and be like, yes, you're right. You know, like this is a hard question.

30:16I think, you know, to give you something concrete, we have a bioinformatics benchmark we call Bixbench. Bixbench, like we put it out. We've updated a few times. It's in some frontier alums when they release their system card, they'll mention Bixbench. It's like one of the things they test on. And, you know, we're getting to 60%, 70 % correctness on Bixbench. And we found that actually we're at the point where humans disagree at this level. Like humans only agree 70 % of the analysis. And so it's true that like when it comes to analyzing data, like humans do not agree 100 % of the time. There is a certain amount of like choice that goes into it.

30:57And, you know, we try to, so Edison is a for-profit company. We like trying to sell some of this stuff to the companies and we'll go to some companies like, oh, we never impute data. Imputing data is bad, like, or, you know, whatever. And like, well, we'll have to change our ages so we don't impute data for them. But then some of the companies like, oh, yeah, we impute data. It makes everything easier. Right. Or in and, you know, you want to know what the real modern dark arts are that like AI resistant area of the world is like medicinal chemistry. That is like a spot where like, you know, there's so much superstition.

31:26Oh, yeah. Yeah. Everyone. Yeah. Everyone is like pseudo religious. yeah exactly you have to be the survivor i feel those burnt out but the religions never agree two medicinal cannabis will have completely different viewpoints about like a functional group yes exactly and i remember this is a talking to somebody who works at cro and they're like oh whenever like company x orders anything we never put boron on any of the compounds because they hate boron because there was one program that was killed because there was a boron and you know somewhere in the core and it led to some toxic side effects so no boron for this company this company they like love things to be fluorinated or something because they love think it's great for the AdMet properties, right?

32:02And so there's like all this stuff where you reach the point where you're at, I don't know, human bias level or human disagreement level. And I think we're getting to that point in data analysis. And so of course you will see then that if I take the raw data from a paper and I analyze it myself, I will get a different conclusion. One of the cool tricks you can do is this back to this brute force thing is that I can go to our agent and I can run it a hundred times and I can take the consensus like analysis. Or I can say, even if you make these three different choices in your data analysis, you get the same conclusion, right?

32:31Or this conclusion is somehow sensitive to those choices. And then you can, there's even like words, it's like epistemic versus aleatory uncertainty, right? It's like, this is aleatoric, which means like, I think it's noise from the data or this is epistemic uncertainty, which means like, I think there's some choices that are being made. There's some model differences that lead to the disagreement. Anyway, there's like a Donald Rumsfeld formulation of this as well. Like, no, no, no. It's an aleatoric, epistemic. debate there. Interesting. This kind of digging into your cosmos. Yeah. So I glanced at the paper and one of the things that jumps out is that there were certain class of problems for which it was only 50 some percent accurate.

33:16Oh, yeah. Yeah. And can you talk a little bit about that and how that like, OK, so if I'm just raw getting 50 percent accurate answers and then I'm going into the wet lab and being like okay try this and then it's like ah like the the stupid thing did told me to do it well how do you i would say first of all that 50 it's actually pretty good because it's rare that experiments in the lab are actually coin tosses right they're usually a lot more outcomes than you know than than binary yeah sure okay yeah but but that particular number was uh human agreement in the interpretation of the results okay and so we asked people to evaluate different aspects of cosmos we had them evaluate like the data analysis decisions we have people ask it to evaluate the literature like is these do you agree with its finding the literature that number that was 50 that came from cosmos's interpretation of uh some of the analysis yeah so like it might go in literature and find this result and then would say wow this is super exciting this is amazing or my do data analysis like this is a novel discovery really excited about it yeah and then people would disagree that's actually not interesting or like i don't agree with the interpretation of it so it's like picking bad problems maybe yeah in the the negative of class and so i think it's like that that 52 or 55 whatever it is that's um interpretation and so i agree i think that's where like i was saying i think the frontier right now is scientific taste yeah and so that's what we're working on right now is how do you get that interpretation to match could you step back and just introduce cosmos on a high level yeah yeah um i would actually be in even curious to hear starting from like chem crow and uh you know you have uh paper qa avery ether zero yeah yeah i'd like to hear a little bit of the the lineage and how those different decisions were made what were the key learnings and how did you get to where you are now yeah so i could retcon and tell a really great story about how we arrived at cosmos but i will say that like to a large extent we just try a lot of stuff and sometimes it works and sometimes it doesn't okay you know i'll say that we're very i'm i'm a builder like i like to like build things piece by piece.

35:19I'm probably some fancy word for it, but I'm like a Lego guy or something. My vision was that we would make an agent that does this part of the scientific process, an agent that does this part of the scientific process, whatever. And so we had like ChemCrow, which is going to help us with setting up our medicinal chemistry work. We had ProteinCrow, which we haven't released. I don't know if we will ever release, but ProteinCrow is like designing proteins we might need for some part of our workflows. Or we had a data analysis agent. Is that LLM or that's a... It's an agent, so an LLM plus tools.

35:47Okay. Or we had Ether Zero. It was like, okay, we noticed that the frontier models can't work with molecules very well. So let's make a model with intuition for medicinal chemistry. And that was what led to Ether Zero. But then Sam actually really pushed on us to like, let's just even do the whole thing. Let's just try to build an AI scientist. Let's just try the whole thing. And that was what led to Robin. and um robin was like let's just take these agents we already have and we'll just put them in like a a work basically it's like you could express it in a concise python file of like you know try a whole bunch of ideas then go see if they all filter through literature or if they've been disproven and then go like uh come up with experiments that you could do in a wet lab yeah and this is our inventory list and then go analyze all the data then go back and repeat the process right so that's like what Robin was.

36:30And, um, we like, we came across cosmos. We're trying to like, understand what is the process that Robin is, is automating. And it came from this idea of like a world model, which is that when we first started Edison, we were thinking like, what do we, what do we want to change about this? Like what is new here? And so we spent some time thinking about, well, the scientific process, like what is actually going on in like my brain, which is that I have some understanding of, of the world or the phenomena I've studying. And that's my world model. And then a lot of the actions I take are about trying to update that world model.

36:59And it's something that changes over time. And so this is like this ability to change over time, but it's also something that is practical. Like I can use it to make predictions about, I know from this experiment, this will happen. That's why it's like a model and not just like, you know, memory or like a bunch of like papers, just like that. It's like, it's supposed to operate. In Cosmos, we tried this idea out and actually Ludo, who's the first author on paper, We tried a whole bunch of ideas around world models and we kind of thought they weren't really appropriate. Like, well, we tried a lot of different ways to do this.

37:33We tried, you know, method A, method B, method C, and they're okay. And so we all just had to take a break. Ludo, like his project didn't work on trying to do this world model stuff. He's like, I'm going to keep trying it. Ludo's a very stubborn person. So he tried it for like, I don't know, a week or two weeks. Then he was kind of like quietly. He's like, hey, can you guys come take a look at this? And we're like, wow, this is actually really cool. And then we like started building on it and jamming really. And I think what Ludo figured out is that you have to get this like experiment loop thing.

37:59You have to be able to let it in the data analysis agent is what got us in the loop. So if you put that in the loop of like, it can really update this world model because the, we were trying to build it around literature before. And when you build it around literature, there's just like not really experiments you can do and then see the results for. That was like our surrogate was, was literature. It just wasn't working. But data analysis actually really lets you explore ideas. And so that was what led to Cosmos. And so in Cosmos, we basically, we had all the pieces sitting around. We've been working on world models, we've been working on a data analysis agent, we've been working on a literature agent.

38:27And then we've been working on, you know, we built a platform for scientific agents. So we had things that can write a LaTeX report. We had things that can make nice plots. Then we put that all together and like a world model was like sort of the glue that allowed it to fit together. An analogy is like encoding agents. Like GitHub is sort of the glue. Like there's some shared repo and everyone works on the repo and like software engineers have spent whatever, lots of brain cycles thinking about what's the way to coordinate, you know, and organize working on code together for a long time. So the world model is actually like a memory system?

38:59Yeah, you can think of it as a memory system. We think about it as a model. So like it actually, you can put in input and it will output predictions. We think about calibration. But like really it is a set of like a big bundle of information that we accumulate over time that's distilled in some way. And that is like what allows us to do this. And I think you can think about like a GitHub repo is like, it's a distillation, right? Like really there's a long graph of commits that lead up to it. And like the current file system in that GitHub repo, or keep saying GitHub, I'm such a corporate shill here.

39:31Your Git repo is like a distillation of all of the work that people put into the PRs, into the commits. And so I think there's a nice analogy between a Git repo and what a world model is. I see. And I think that's just sort of what allows us to automate scientific discovery so well. Can you talk about like kind of how you implement a world model or is that sort of like secret that's our like secret sauce right now yeah that's fine yeah no it's fine people have asked so one thing that's notably missing yeah is the like simulation right yeah or dynamics or or or like uh bolts or yeah i want to help you guys pump up your views here so i i think molecular dynamics is overrated in fact coming from someone.

40:17Yes. That goes in the intro. In the thumbnail, you know. And DFT is overrated. In fact, DFT may be even more overrated than electronics. I think these methods... You mean for materials or for biology or for both? For materials. Okay. And I can explain more about that. Basically, MD and DFT have consumed an enormous number of PhDs and scientific careers at the altar of the beauty of the simulation. Also, random interjection. Once I did an estimate, I think pre-ChatGPT, something like 20 % of the world's computing power just went to simulating water. Oh, my fucking God, water. Yeah, yeah. I had to deal with so many water simulations.

40:57I did DFT simulations of water, and they are so annoying. I used these big computers from the Department of Defense, and I spent, like, I don't know, five months. And by the way, this is pre-LLM training days. Five months of compute is actually a really long time. I simulated water with quantum effects with a grotus mechanism for how a proton hops through water. And it's on YouTube. It's my number one YouTube video. And it represents like... Until now. And it represents like, I don't know, a million CPU hours of compute. It was one of the biggest computes that I... Probably the biggest one I've done in my life so far.

41:33Maybe Ether Zero is bigger, but it took a lot more work. Anyway, and what's the point? What did you learn? All I learned was like what set of hyperparameters reproduce some physical effects of water, but none of it was de novo. Right. And this is the this is the issue with with molecular dynamics and DFT is that they don't model the world correctly. And so we have to invent little stories we tell ourselves about we're like making good inductive biases and then it models the world more correctly. Like in DFT, you simulate water at 330 Kelvin when you want room temperature water. Is room temperature 330 Kelvin?

42:07No, it's not. That's a little too hot. Right. And so the issue is that people just make up these things or like, I don't know, GGA or B-LIP or B-3-LIP, all these different methods people. They're clearly empirical. And then they bolt it on to DFT and they say, look, it's a first principles method, right? But actually you made a whole bunch of choices and you overfit to the validation data to get this to work. And that's, I think MD and DFT are like that. Because if you go look at the catalysts, you know, what catalysts change the world, none of them are single crystal materials that are really well suited for DFT.

42:45They're always like they have grain boundaries, they have dopants, they're complicated, right? And you never capture DFT. So I think this is one of the fundamental, I don't know, dichotomies of the world is that simulations stimulate really boring things really well. They don't simulate interesting things very well. And so that's why I don't do DFT and MD anymore. What about somewhere like the machine learning stuff like AlphaFold and... AlphaFold was trained on x-ray crystallography data. And I think, you know, this is the story of MD is that MD was supposed to be the protein folding solution.

43:22There is a great counterexample. There's a, I don't know what there's a word, but the counterfactual is basically a group called DESRES, D.E. Shaw Research. they had you know similar funding to deep mind um probably more actually they tested the hypothesis to death that md could fold proteins they built their own silicon they built their own clusters they had them taped out all themselves they burned into the silicon the algorithms to run md they ran md at huge speeds huge scales yeah i remember david shaw came to a conference once on MD and he flew in by helicopter and was like this pretty famous guy kind of rich and um he gave an amazing presentation about the special computers and special room and outside of Times Square and like what they can do with it is beautiful amazing and I always thought that protein folding will be solved by them but it would require a special machine maybe the government would buy like five of these things and we could fold you know maybe one protein a day or two proteins a day.

44:20And when AlphaFold came out and it's like, you can do it in Google Colab, you know, or on a GPU or desktop, it was so mind blowing. I forget like that protein folding was solved. I always thought that was inevitable. But the fact that it was solved and on like your desktop, you can do it was just completely floored, changed everything. Like the bitter lesson on steroids. Yeah. I don't even know what it is, but it's like, imagine chat GPT came out, but instead it was like, oh, you can just run it on your phone or locally on your own desktop. Like that's the level of like shock that came out and it gets down to this thing that humans are really bad at estimating problems that aren't human-made problems protein folding we all thought was like would require a huge amount of compute very challenging problem was hardest problem in the world right and it turns out that you can actually do it on i don't know i think the numbers are now like 10 000 gpu hours you can train a good protein folding model it's actually turned out to be barely an inconvenience therefore why not oh oh therefore protein folding was highly efficient based on experimental data They took X-ray crystallography.

45:14That's what DeepMind did is they took X-ray crystallography data. Desiree has tried the first principles method. And it's like a nice head-to-head comparison. Two very well-resourced groups. They both tried different ideas. And the machine learning on experimental data beat out first principle simulation by a very large margin. And so why isn't like Bolts or whatever inside of Cosmos? Like why isn't there a tool that can run Bolts? Oh, we have Bolts inside of... We have Bolts Gen. yeah yeah we have that inside of cosmos okay it is i mean i think in the version that we have uh for people to just sign up and use it's not in there yeah but like uh you know you can imagine that you can just modal or lambda or tamarind or 310 there's all these companies that basically wrap a lot of these these like um uh deep learning protein design tools or chemistry design tools they wrap them in an api you just give that to give it to cloud code if you want you can give it to cosmos and you can be like hey you know if you want to design a protein for x use these tools Your mechanism, it sounds like, or one of the primary mechanisms that has been successful is like it like enumerate a whole bunch of possibilities and filter.

46:19Yeah. And so how do you think about serendipity and out of out of distribution thinking and getting there and how far have you gotten and what's left? Yeah, that's a great question. I think I guess the short answer is that there is very. So this is the domain of seaborne. So chemical, biological, radiological, nuclear weapons or I don't know, safety. Yeah. This domain has been explored a lot in history by a lot of organizations. And I would say that there was a big question mark for us a few years ago was like, how much of this stuff is intellectually bottlenecked? Like how often are people like, oh, wow, I want to cause harm, but I need to know like some facts.

46:59And could LLMs make that easier or go faster or anything like that? I think, you know, the first set of answers in 2023, I think was basically no, is that like, you know, you can go find the synthesis route for many dangerous compounds on Wikipedia. People know what are the targets in the human body that like are targeted by most biological weapons. It's not really that much of a mystery. So I don't think there was a lot of like, there's a lot of new ground when LLMs first came about. And then there's a lot of concern about like laboratory protocols is that could agents or LLMs reveal some tacit knowledge that like maybe people couldn't find on Wikipedia or like maybe for making something, there's some technique that is acquired when you scale it up in size or something.

47:45Or maybe there's like some way to get around like tracking lists by ordering different compounds. So in that, I think, was really well tested by a few different labs. not me, but there were some groups that spun up that started making tests for this and labs pay attention to. I think it's really been put into process where LLMs will shut down or be filtered in those scenarios. But I think that is actually an area where there is some risk. And so I think that's something that people pay attention to for open source models. And there's still, I think, some discussion there. But I think to a large extent, it's not really been greatly accelerating in practice, or at least I haven't seen much evidence of it.

48:22And again, I think it comes down to the fact that it's not really available, but if you look hard enough, you can find most of the information you would need to get up to no good in the public domain already. But then I think now is the next frontier is like, can it somehow help you with real-time protocols, troubleshooting more in the loop and more, especially in the computational side of things? there are some scenarios that are now coming into focus that could be more dangerous or more intellectually bottlenecked and so i think people are trying to pay attention to that to some extent there was like a first wave that we thought this could unlock a lot of stuff and i don't think it came to pass yeah i think there's now an emerging sort of second wave of like there are some actually new scenarios that were just too far-fetched to consider two years ago that i think are now realistic um some smart people are paying attention to it but i don't think it's solved yet.

49:15I don't know. It's very vague. So I guess like one kind of differentiator, there's a lot of talk about AI safety in like the broader LLM, you know, ASI space. And, you know, there it's jokes about paper or paper click maxing robots or something. But like the core threat here is more like a malicious actor using this as a tool to accelerate something dangerous. And like kind of the first order hypothesis is that you basically already have to be an expert to effectively create a bioweapon or a chemical weapon and a non-expert or an expert already know how to do this yeah i think you know so so each of the categories in the cbrn they're all a little different but i think to a large extent it's a lot of like pushing material around you know the classical example nuclear is like it's a lot of a lot of centrifugation yeah a lot of ultra a centrifugation, a lot of high pressure or high RPMs.

50:10And so it's just, you can maybe get smarter about how to set up, you know, the economy of scale to do that with an LLM. But to a large extent, I think you can call your friend and country X and they can tell you what are the steps. It's not, I don't think it's that much of a secret. It's just a lot of like moving material around. And I don't think it's accelerant, meaningfully accelerated. Now, with that said, there are all kinds of like, you know, dumb dual use things of like, maybe you want to call a company that makes centrifuges and you want to make sure that they sell you them and they go through some KYC steps and maybe an LLM can get you through the KYC faster.

50:46And that's like a dumb thing that like, okay, like, yes, like, you know, email makes it so that you can order centrifuges off the internet more easily. Is email like a dual use technology? Like, yeah, to some extent it is. And so I think there's a lot of like weird second order things that we don't pay attention to in AI safety. of like, does it make KYC easier? Does it make it easier for people to know where to order this from? Or like, what is the expected price? Or like, what should you order first, right? All those like sort of simple logistical things, I think are accelerated by AI just as like a consequence of AI being an accelerating technology.

51:19But certainly, I mean, shit guys, there's some scary stuff. And I try not to think about it too much. Yeah. I don't know, I guess, I don't want to get too political, but I do think that right now the United States government is maybe taking a slower, less intensive look at safety. But there's definitely people, I think, in other spaces than the U.S. government thinking about it hard. Do you think it's a thing people need to spend more time on? I do get waves of angst about AI. I'm sure many people living in San Francisco do get a little bit of waves of it. And sometimes I think that there isn't enough work being done on it.

52:00And then sometimes I think, wow, I need to mellow out And like, you know, we have lots of time to think about it. What is my opinion on it then? I don't know. I think my opinion is not formed fully. Yeah. You and Sam have done a lot of thinking about funding science and future of science. You have you've been vocal about the reproducibility crisis and other things. First question, why this focus research organization or for it? Yeah. FRO. Yeah. Yeah. What does that get you that you don't get from academia or, you know, big lab or whatever? A nice network of people. And I think Edison is like a real, of course, I think Edison's going to be great, but I think it's a mystery of what's going to happen.

52:46So I don't think we've had as much friction there as you might expect. But yeah, this is all stuff that Sam and I think about all the time. It's like, how do you balance stuff like this? How do you balance the economics? um you know there are some there are some venture-backed companies that are having cash salaries over a million dollars and it's like insane to me yeah that you would use all of your cash from your equity financing you know in these insane salaries but they can in terms of like total spend on gpus it can still be a total a small fraction of your burn so sometimes it kind of make sense yeah yeah that's that's one way to think about it so so like you this is a good uh lead-in to you are automating science yeah in some capacity yeah so where does that leave scientists so i think um this is uh jevin's paradox we can try here is uh um so uh let me start with the contrast here is that, you know, if we automate, you know, taxi cab drivers, there's not going to be an increase in people needing to go places.

53:52Maybe there'll be somewhat an increase, but like there is a finite amount of like time people will be spending in cars. And so there's an upper limit. So when you automate that, that's like a scarcity thing. It's basically you're displacing jobs when you automate driving. In science, I don't think there is a finite appetite or a finite capacity for science. I don't think science is like a scarcity thing. Like there's, you know, 100 more discoveries left to be made and then we'll be done. And so like we're displacing jobs. I think instead, actually, if we can make science go much, much faster, there will be no, there will be no decrease in demand.

54:25There'll be actually, I think, an increase in demand that will match whatever automation amount we have. And so my vision for what a scientist would be in the future is that they will be, I don't know, like agent wranglers or cosmos wranglers of like, okay, they're exploring 100 ideas simultaneously, or they're like working with systems like ours to make 10x discoveries 100x discoveries, because I think there's an unlimited amount of scientific discoveries to be made. So there's no like scarcity set where basically we will displace them all. Now that's kind of like, this is what I would tell when I talk to a first year PhD student yeah everything's gonna be just fine you know but then when it gets into the nuts and bolts i i do agree that this is going to be like a really hard thing where like if i am ceo of a company that makes science like a pharma company or material science company or something like that or a r &d arm at ibm i think well i could spend you know a million more dollars on on compute for the ai scientist or could hire 10 more people i might just choose to go with the ai scientist is because, you know, to a large extent, like hiring people is hard, right?

55:30And hiring an AI scientist is probably a little bit easier. And so I think that there could be some friction. But another thing is like, science is in some ways closer to art in the sense that like, there is a large number of people who just appreciate good science. Like if you get published in Nature, it's not because it's really going to be world changing. Of course, that's part of it. But it's also because like people are like, wow, this is really interesting science. So I think the enjoyers of science are also scientists. And so I think that it's kind of hard to imagine a scenario when there's not scientists as the consumers of science.

56:05And so I think if they're going to be consumers of science, they're also going to be some of the producers who are involved in the process by itself. I don't know if that makes any sense. Yeah, you've touched on this. The question in my mind is just what does a scientist do then? There's a great short story by Ted Chiang, I think in like 2003 or something. and it's about like well at first scientists were displaced and they became like the uh interpreters of like what the ai scientists are doing like the scientists read the ai scientists like papers and then you know translate them for whatever popular science or something and then after that like they couldn't read the papers anymore and so they were left behind and so they had nothing to do and they just sat around and but the problem is that science is like you know you you have to translate science to make any impact like science cannot exist by itself i do agree there's like engineering can exist by itself like if you give some kind of system a goal of like making me a material that i can make a space elevator out of you could be not participating in the beginning the process in the middle of the process and you just come by the end and be like okay all this recipe like science of like what's the origin of life or like is there water on other planets or, you know, why is some catalyst better than another catalyst?

57:14That has to be hitting human eyes and human brains at some point. So I think a human has to be involved in the process. Don't want to be contrarian, but yeah, be contrary. Why does a human have to be involved? Why does a human have to be involved? Well, a human has to be involved at least some point to be like, yes, this is good science or this is bad science. Okay. So it goes back to taste. Yeah. But I don't know. Maybe you're right. Maybe there is no point for humans. Maybe we'll be like no what is it sora uh you know like the ai slop app but i think in sora there's still humans at the end clicking the videos or something yeah so so the the sort of analogy kind of brings up an interesting point like is it possible that like due to the biases of ai science if we really go full in science that you know there still is a market for kind of boutique human science like you know there's still people who want to you know paint things the old-fashioned way but But more to the point, does it become even more important to have a human who is actively doing their own exploration?

58:10Because there will be like large blind spots and biases due to the models that just you'll never be able to overcome because this is sort of baked in now due to your training data. And without a human, you'll always get stuck and there will be a blind spot that will never. bio which is a company in oakland um or in emeryville and they do really cool stuff with automation i think they're going to be testing this theory of like okay maybe if that's the bottleneck we can see evidence of it because they're going to start doing really well um it could be true i still though want to say all of those i in my mind are still sort of scoped in terms of like r &d for pharma or bio but they're not like none of them are attempting to answer big fundamental questions and maybe there's like different levels when i think about that and you seem to be, um, it seems like the future, how the, the focus of future house in Edison is much more towards like, you know, sort of R and D and sort of end run science.

59:08But, um, you know, I, I have some background in, you know, fundamental physics. Um, you know, it's like, is there any thought about like, how do you like take on, you know, dark matter candidates? And like, I just, you know, think the data to really give us a complete story is just not there yet. You know what? I'm sure everybody at every company is the biggest critic of their own product. So we think Cosmos is, we think it's great, but there's a very large amount of area for improvement. So with Cosmos, there's like an open access to everybody version. Do you provide access to other labs that is less open?

59:51um we have a version of cosmos that has like um bigger resources like it can run for longer it uses gpus um so like basically when it does data analysis it'll have a gpu so we use that for things like um like machine learning experiments you know if you want to know like this question about whether it's better to pre-train first on noisy data or not yeah um we have like pre-release models that are coming out and we try those. But yeah, so I guess like, yes, we do. And we do have like research partnerships with companies where we like build something specific for them. And that is something we think about.

1:00:28But broadly, I would say Cosmos that's on the website is pretty close to what is the best we have internally. Yeah. I have a question. So you previously have stated that you think that language is the natural, um, language, what is it? Language of chemistry. The future of chemistry is language. Yeah. Yeah. Um, okay. So I wonder, do you still believe that? Good question. I think, uh, I w I would say yes. I still believe that, um, that so, so in that article, that opinion article, my, my point was that, uh, you know, at the time when I wrote that article, I think maybe three years ago now or something, maybe 2023.

1:01:10It was that we have models for predicting solubility of compounds. We have like data about our large populations and we have like papers and we have code. And the only way to bridge all that information is natural language. And the argument was that like humans, like, you know, whenever we can't bridge information, like if I can't talk about my code or I can't talk about some idea to you, I will invent words until I can get the point across. Right. And that humans are always innovating on language to make it represent all known observations and people innovate in language to represent whatever code pattern they have.

1:01:44Right. Like this is like the only shared activity we've been doing for this long is like coming up with words to represent everything we know. And so I think that for that reason, natural language is the only possible way to connect all the different pieces of data we need in biology, medicine or any domain for that matter. I think there's some caveats to this of like, you know, you can make an argument. Like if Jan LeCun were here and he would make an argument about like, you know, world models or like vision or embodiedness, right? Like there's arguments against natural language that like, you know, that maybe there's something more that it's not the complete story.

1:02:18Or maybe natural language imposes limitations that cannot exceed because you're stuck in this abstract space that was invented by humans and you can't escape it until you can like touch something. Yeah, I mean, it is an abstraction, right? And like scientists basically work exclusively in abstractions to some degree. I just, I find, I found that interesting because it seems like most scientists, you're right. Like when they explain things, they explain things through language. But many conversations, maybe most at some point result in people drawing diagrams or something. Like, you know, chemistry, like biochemistry largely or medicinal chemistry is oftentimes a, it's a language of graphs, right?

1:02:56Or, you know, I mean, bonds are abstractions, yes, but, like, they're pretty good abstractions for most, for many cases. Or, like, you know, geometry, you know, thinking about, you know, protein is like the geometry of a protein. It's like, I think that that's how people will, a lot of scientists like to think about things. And so I find it interesting that, like, yeah, that you are focusing primarily in this language. Like, have you thought about essentially a multimodal version of this? Like where, you know, when it comes along a smile string, it doesn't just say, oh, this is a smile string, but like this is a graph.

1:03:31This is a representation of some higher like abstract object. You're absolutely right. And the problem with these is like, I don't know, Jacob's ladder or something, whatever you want to call it. It's like, yes, you can say that you can call a molecule by its name. You can show the graph. And if you go to a molecule like ferrocene, well, it doesn't really have bonds, but like part of it. And so then you're like, well, we need to draw it visually. and then you go to molecule like i don't know glycine betaine on this dihedral angle right and so like it's not actually this thing i drew it's actually an ensemble between this thing and this thing right then you go to benzene you're like well not only is it like a ensemble of these different conformers it actually has electron density and you can't really ignore the electron density in benzene you like need to treat it correctly and it's like well you can't actually represent the electron density that way you actually have to look at the correlation of the electrons individually right because you can't really model benzene with like dft right or functional, you have to actually look at the electron correlation.

1:04:25These are the electron correlation. Like, well, you know, you can model electron correlation, but, you know, actually these things, when they're in a solution, they have like, you know, relativistic effects because it's like there's a whole bunch of stuff around it. So you really got to have the relativity in there. And you're like, well, you got the relativity and you have the electron correlation. You can have the bonds and you have the conformers, but you really need to think about the cosmic radiation background because like, you know, it does actually impact everything and there is some energy there, right?

1:04:47And before you know it, you've ran out of, you know, You've ran out of compute or whatever resource you're using to model this. And so I think you have to draw the line somewhere. Natural language, like I said, is that humans have worked for a long time to make it be the, you know, what's the word? Like the least abstract or the, you know, it's somewhere on the border of like it's still abstract enough that you don't need to know all these details. But it's still granular enough or concretized enough that you actually can make use of it. there may be some other representation like multimodal might turn out the video or maybe i don't know there's some other like fusion that you can make i like natural language because we all work really hard to make it right at that boundary and i do agree sometimes sometimes ideas slip and they can't be in language you have to get out the whiteboard or ideas slip and you have to wave your hands around you know or maybe then then you need that that uh degree of freedom to communicate just digging in on this a little bit more like uh famously quantum mechanics is like undescribable, right?

1:05:50Like there's, there's an argument that you cannot understand quantum mechanics with words or in, in with our preconceived understanding of the physical world, because it doesn't behave like the macroscopic world. And so the only way to understand it is through mathematics. Right. Um, and I largely see language as the joint key of science as well, but I wonder if that's not true for many domains in quantum mechanics is just the one that hits you in the face i mean i don't know i actually i think the there's like seven principles of quantum mechanics or five or something like this that you can actually express pretty concisely in language i agree that like you need to actually look at the consequences of them you need some mathematics um i don't know i actually i don't know this is like a challenge i think you could actually describe a lot of quantum mechanics and language Sure, sure.

1:06:40But but I see your point. And yeah, I guess I'm a realist. Like I when I talk to my kids, you know, maybe I will be like, OK, let me draw for you. I don't I don't make sure in our house everything is described with natural language. So I agree with you there. I think maybe we can be a little a little flexible with with natural language and include equations and smiles strings in it. And I think we can get a little bit farther. So maybe that's OK. But some people, I think, like optionality. You know, like, oh, it could be this or it could be that. I'm somebody I like to take strong opinions and see how much farther they can get me.

1:07:17And I think in my career, it's actually been better for me to take strong opinions, which in my deepest of hearts, I know that are maybe not correct or not fully correct. But once you take these strong opinions, it just, you can sort of move many steps down the road once you take these strong opinions. And like, for example, at Future House, we took the opinion that scientific agents are the future. And that allows us to skip a lot of steps because a lot of other people were like, we need to build a foundation model for X. Yeah. And we just skipped all that. Right. And I think if you also were unopinionated and you had optionality, like I can think of a famous example of a different company that liked the optionality and they wasted a lot of time on foundation models or something.

1:07:53Then I think you get stuck. So that's one of my strong opinions is that natural language is a way to join all these different domains. It may not be a correct opinion. It may be more subtle or more complicated, but it's allowed me to get very far. I'll drop it someday and maybe find a new one. Yeah. Not yet, though. That's my meta opinion on the matter. The Ether Zero story on your blog, I find hilarious and kind of awesome. Yeah. You know, when I was a kid, I loved the like genie slash monkey paw concept of be careful what you wish for, but you just might get it. Yes. Maybe just like quick story.

1:08:32Can you just talk about that? That was just a really fun... EtherZero was a hell of a project because conceptually it was a very short project of like, hey, people have made a lot of progress and verifiable rewards in math and code. Let's say we can do it in chemistry. So chemistry is like not a verifiable field, right? Like, of course, you can go test something in the lab, but then we had to think about all these ways that we can make chemistry verifiable. And one of the ones we settled on was like, make a molecule that has like three nitrogens, two oxygens, ten hydrogens or something. And we thought that was like a pretty verifiable question.

1:09:09But every time we would train a model, it would find some new, insanely weird trick to generate these molecules. And I'll just tell you, one of the examples was that it would make these molecules and we would do some checks to make sure it had the right bonds, the right number of electrons, the right number of atoms and stuff like that. But it would just solve the problem in any way possible. So it would just put all the nitrogens over here, put all the auctions over here just like things that don't look good yeah so we started coming up with these rules of like oh let's check to make sure it followed these good practices or these good practices and we found ourselves into this like you know it's like the opposite of the bitter lesson like i don't know the boutique lesson where you like try to make everything custom but one of the things it kept doing is it kept putting these nitrogens in a row and it put like one nitrogen two nitrogen three nitrogen all in a chain and this is like you know if you have three nitrogens it's like explosive you know two nitrogens it's like bad and like four nitrogens you can't make and i kept telling everyone like it would make these like six nitrogen compounds and they're just they're just literally impossible and they're not possible and many of the people on the team were like computer scientists like on this team and one of them like one day sent me that like this is on the cover of nature today on nature's website somebody made a six nitrogen compound and this is like somebody's like career to deliver this compound because this is the most unstable like insane compound you can make it's some ridiculous setup and like the spectroscopy to get that proven was like very difficult.

1:10:26And this, I don't know how they did it. It was amazing accomplishment. Like, look, Andrew, like it's not actually impossible. And it was so funny to me that like our model was sitting here, spitting out these six nitrogen compounds in like, you know, 2024 or 2025. And like the paper just happened to come out that year. Like mankind had finally made a six nitrogen compound. Do you think that those were actually synthesizable even under these extreme circumstances? No, our model was just, it was just reward hacking. Okay. it was just the the model was so creative and ways to reward hack like one of the one another one we did was um you know we wanted it to make sure that the when it would propose a reaction like make this compound tell me how to make this compound we would try to make it sure that all the reagents were purchasable like you could purchase them they were not like made up yeah um and the reason we came with that is that originally we just like take the end compound and then like remove one atom and be like here's you buy this and then put the atom on it's like okay i wish it was like that um so they have to be purchasable and then we'll be like we thought it might be hard if they're all purchasable because sometimes you actually order things custom or something so we'll just make sure one purchasable so the first thing it starts doing is putting nitrogen in there because nitrogen is purchasable and it like has no participation in the reaction right like oh my god okay so they're like okay it has to be purchasable it has to participate in the reaction they started putting like acid base chemistry we'll just put an acid here acids are purchasable and it'll move one atom and they're like okay fine can't be that everything has to be purchasable then we find ourselves and i'm like sitting there one day building this like ridiculous catalog of purchasable compounds and bloom filter so i can go fast enough on our training loop and i'm like why am i doing this how did i get here and i don't know it was really funny because um pre-training or training transformers you know on just data like just supervised training where you just have the inputs and the outputs directly very nice relaxing you know like things are always robust you know things go pretty smoothly when we do these verifiable rewards where you have to like write a bulletproof verifier it is really difficult and we had so many models trained only to find out they were hacking some other like random thing in our setup it's really hard and i and i i don't envy the frontier labs that have to do this at a very massive scale because we had a lot of adventures in ether zero and you guys I should read the blog post.

1:12:41Definitely read the blog post. It's very fun. It's a great read. GRPO? We did make some modifications to GRPO. Yeah. I actually, I used to know all the names of these modifications, but I think it's like DAPO is one modification and like the clipping we did was special. And we explored a lot of that stuff. Yeah. And it was also one of these things where like you think the hypers are wrong, the algorithm is wrong, and then you find out it's just because like you had some. somehow sorted the reagents when you made your training data but when you made your test data you didn't sort them alphabetically and the model was just like barfing because its whole strategy was to exploit something in the way you sort of things so yeah we explored a lot of different methods and it was um i learned a lot about chemistry a lot about nomenclature um and actually there's a i learned a lot about medicinal chemistry as well more than i ever wanted to awesome if you want to do some like engineering just check out edison scientific and they have I think they're hiring with lots of interesting things, everything from scientists to infrastructure engineer.

1:13:44Thanks, Andrew, again. Yeah, thank you very much for joining us.