Where Is All the A.I.-Driven Scientific Progress?

0:00So there's a lot of noise about AI, but time's too tight for more promises. So let's talk about results. At IBM, we work with our employees to integrate technology right into the systems they need. Now, a global workforce of 300 ,000 can use AI to fill their HR questions, resolving 94 % of common questions. Not noise. Proof of how we can help companies get smarter by putting AI where it actually pays off, deep in the work that moves the business. Let's create smarter business. IBM. Cynthia Erivo is the best singer in the world. She's incredible. I don't know what it is about her voice, but it brings me to tears every single fucking time I hear it.

0:39She has the most incredibly emotional voice. I was trying to figure out what it was, but she's just the best. Obviously, she has the power, but there's all these textures in there. Did you see the design-to-go-viral clip of her visiting her old school? Yes. I obviously lost my shit I was the absolute best is that the students start singing and they just sound like shit

1:10it was like my nightmare just imagine you're one of these kids you're not that it's just like after school class you know it's a little club you're just doing it for a little bit of enrichment and like you're just kind of plodding along trying to get through the day and then fucking Cynthia Eriva shows up and they're like alright kid you're up next what do you got No thanks No it was so sweet I would throw up It was so sweet

1:35I'm Kevin Roos A tech columnist At the New York Times I'm Casey Noon From Platformer And this is Hard For This week Future House CEO Sam Rodriguez Joins us in the studio To separate the hype From the reality Of AI science

2:01Well, Casey, it's time for some science. Yeah, give me a second, Kevin. I'm just going to put on my lab coat here, get up my Bunsen burner, and see what you've got cooking for us today. So I have been obsessed with this question of what AI is and isn't doing for science and scientific discovery. Obviously, this is something we hear a lot about from the leaders of the big AI companies, people like Dario Amadei, Sam Altman, Demis Asabas, they have all been saying things in recent months about how close they believe we are to solving new scientific problems and curing diseases and fixing the climate with all of these new AI tools that they're building.

2:39And some of that is obviously hype, or at least has the sort of markings of hype. But there's actually a lot of real stuff going on in AI and science that I just do not feel personally qualified to evaluate. Yeah, and I would also say that science has become one of the main ways that the leaders of these tech companies want us to evaluate them. Because whenever one of their models does something horrible, the message we basically get back in response is, don't worry, we're about to cure cancer. Just hang on tight. I know that this chatbot might be driving you to madness, but if you could just give us a few more releases, we're going to do some really good stuff.

3:12Yes, and this is something that we're also hearing now from the U.S. government. The Genesis mission was announced by the White House just before Thanksgiving. That is what they're calling a dedicated, coordinated national effort to unleash a new age of AI, accelerate innovation and discovery that can solve the most challenging problems of the century. I thought the Genesis mission was just them trying to get Phil Collins to play the White House Christmas party. I guess not. And so today we have brought in a bona fide scientist to help us understand which of the sort of scientific discoveries and possibilities out there are real and which are not.

3:47We need an expert with a broad focus, someone tracking the impact of AI, not just on biotech or drug discovery, but across the different sciences. And Casey, we have found the perfect person. Let's hear about him. Sam Rodriguez is the co-founder and CEO of Future House and Edison Scientific, which is a San Francisco-based, I guess it's both a nonprofit and a for-profit. Future House is the nonprofit. Where have I heard that before? Yes, come back when he has his board coup. Future House is the nonprofit. Edison Scientific is the for-profit that spun out of it. I've been to their office in Dogpatch.

4:21It's really fun. It sort of feels like a kind of wacky, mad scientist lab. They've got all these, like, you know, sort of lab machines that I don't understand. You know, people running around in lab coats, and they're all talking about AI. And it just feels like kind of a cool place to be. And they are building what Sam calls an AI scientist, which is an AI agent that can do sort of parts of the process of scientific research. And Sam is also himself a scientist. He has a PhD in physics from MIT. And before he launched Future House, he spent several years running an applied biotech lab. So he has sort of seen this stuff happening from a couple different angles.

4:56Yeah, and today we want to talk to him about what he is up to, but also kind of get his vision of the entire landscape. Tell us what is working, what isn't, where's the hype, where's the real stuff. Sam has a lot to say about it. Yes, and I think it's fair to say that Sam is on the more optimistic end of the spectrum of beliefs about what AI will do for science. But as you'll hear in our conversation, he's more skeptical than some of the most optimistic people who are claiming that we'll cure all disease in five or 10 years. Yeah, if you've been craving a little bit of cold water for the wildest projections, he has some of that to offer you.

5:29So let's bring him in. When we come back, we'll be joined by Sam Rodriguez.

5:52Starting a tech company is thrilling. It's also really messy. Just ask anyone who's done it or anyone who's tried. Better yet, ask EY. They've seen startups at their best moments. EY knows you don't start a company to burn cycles on regulatory hoops, discounted cash flows, or the fine print of SEC Form S1. You can't ignore it. That's how risk compounds. What you can do is work with EY. They'll help you get it right early so you can stay in builder mode. Learn more at ey.com slash tech startups. The thing about AI for business, it may not automatically fit the way your business works. At IBM, we've seen this firsthand.

6:35But by embedding AI across HR, IT, and procurement processes, we've reduced costs by millions, slash repetitive tasks, and freed thousands of hours for strategic work. Now we're helping companies get smarter by putting AI where it actually pays off, deep in the work that moves the business. Let's create smarter business, IBM. Fast is only right when it's not reckless. Right now, pressure to adopt AI quickly is real. Your competitors feel it. Your board feels it. Moving fast without the right governance in place isn't a strategy. It's a risk. OneTrust provides the visibility into what's moving across your organization, the intelligence to understand what it means, and the guardrails to keep everything in bounds.

7:22So teams move with speed and confidence. That's AI-ready governance. Governance that helps you go. Visit OneTrust.com backslash AI.

7:40Sam Rodriguez, welcome to Hard Fork. Hello, thank you. So we have brought you here today to be our science expert, our guide to the biggest recent AI-powered breakthroughs that are happening in science. This is an area that I sort of understand in an ambient way is important, and there are big things happening, but neither of us are scientists, although I did make a killer baking soda volcano in elementary school. So we have so much to talk about today, but before we get into some of the particulars, I want to ask you about your project that you've been working on. Last month, the commercial arm of your nonprofit, which is called Edison Scientific, launched a new AI scientist called Cosmos that you say can accomplish work equivalent to six months of a PhD or postdoctoral scientist in a single run of this model.

8:27Tell us about how Cosmos works and where that six-month number comes from. Yeah, exactly. And actually, I will just start out by saying that when I got that six-month number, my reaction originally was like, there is no way that this is true, right? And we've now measured it in a bunch of different ways. I can walk you guys through that. But basically, just to take a step back, so we've been working for two years on figuring out how to build an AI scientist. And the concept here is there's so much more science that we can do than we have scientists, right? And so how do we scale up science? And the thing that happened with Cosmos that is pretty cool is Cosmos is like the first thing that I think that we've made that actually really feels like an AI scientist when you're working with it, right?

9:11Which is to say that you go in, you give it a research objective, it goes away and it comes back with insights that are actually really deep and interesting and sometimes wrong. but about 80 % of the time right, which is like kind of similar to like if you ask a human to go away and do something, comes back like similar percentage of the time is right. And it's like, it's a kind of new experience working with it. So that's very exciting. The six month number specifically, the way that we measured this was we had a bunch of academic collaborators, you know, scientists who had done a bunch of science previously that they had not published yet.

9:45And we basically gave the same research directive and the same data set to the AI, to Cosmos. And we ask it, you know, to go away and just make new discoveries. And it would come back and it had found the same things that the researchers had found overnight. And then you go and you ask the researchers, you know, how long did it take you to find this in the first place? And they would say like three months, five months, like six months, whatever. And so that's where it comes from. And it's like, that's the amount of time that it took them to come up with the finding. So let me just ask a couple of questions so I can ground myself here.

10:18Is this tool kind of a box you type into like the other chatbots? And if so, what is powering it? Did you guys sort of build your own model from scratch? Did you sort of make fine tunings to another company's model? Yeah, so it is indeed a box that you basically type into. You ask a research objective. It's not a chatbot, right? Like it runs for 12 hours or so before eventually coming back to you with its findings. In terms of how it's built, We build on top of a bunch of different language models from OpenAI, from Google, from Anthropic. Like in any given run, we use models from all the different providers.

10:55We also have like our own models for specific tasks that we've trained internally, where those models are like much better for the specific tasks that we train them on than the models that the frontier providers make. Got it. And then the key insight in Cosmos is basically this use of what we call like a structured world model. So one of the main limitations with AI systems today is that they're just limited in the length of the task and the sophistication of the task they can carry out before they kind of go off the rails. They, like, you know, forget what they're doing. They no longer are on task.

11:25and what we figured out was a way to have them contributing to this world model that gets built up over time that basically describes the full state of knowledge about the task that they're working on, which then means that we can orchestrate hundreds of different agents running in parallel, running in series, and have them all working towards a coherent goal. And that was the real unlock. Right. Another thing that I found interesting about Cosmos is the cost. This model costs$200 per prompt. Yeah. So every time you give it a task, you're paying$200. Why is it so expensive? I mean, it uses a lot of compute.

12:01I mean, that's like the fundamental answer is it uses a lot of compute, right? Give us a sense of how much. Well, so an individual run from Cosmos will write 42 ,000 lines of code and read 1 ,500 research papers on average. Like if you run Claude, it might write like a few hundred lines of code, right? So that gives you some sense. It's like there's a lot of compute that is going into this. Have you ever had like a scientist whose cat walks across the keyboard and accidentally hits enter and all of a sudden spends like$600? This is a problem. This is a problem. And we were like, right, so the thing that you have to understand, right, is that if you are a scientist and you go and do an experiment, you get some data back.

12:40You're going to spend$5 ,000 or$10 ,000 gathering that data. And so what scientists want is they want the absolute best performance that they can get. And like scientists who have used Cosmos generally come back to me and are like, they can't believe we're only charging$200 for it. Right. And, you know, I will say like, you know,$200 right now is a is a promotional price. We actually have to eventually charge more. It's going it's going up. So get those prompts in before Christmas. Exactly. But like but really, you know, it's like if you have to spend thousands of dollars gathering the data, like the cost at the end of the day is not the limitation.

13:13We do have to be very generous with refunds because people have, you know, make mistakes. I made a typo. Right, yeah, exactly. That sucks, yeah. So what you just mentioned about the sort of the tests that you all ran to figure out how long this thing could run for, how much time it was saving scientists, that's about like sort of replicating existing research that's out there. But a lot of what we hear from the people who are running these big AI labs is the possibility that pretty soon AI will start making novel scientific discoveries. We'll start doing things that existing scientific methods and processes can't do.

13:47How close are we to that? That's already happening, actually. So if you go and you read the paper that we put out about Cosmos, we put out seven conclusions that it had come to, three of which were replications of existing findings, four of which are net new contributions to the scientific literature, like new discoveries. And of those, what's the most impressive? So one of the ones that we really like, the human genome contains millions of genetic variants, right? These are differences between different people's DNA that are associated with disease. And for the most part, we know that a variant is associated with a disease, but we have no idea why.

14:26And so we asked Cosmos, we gave it a bunch of raw data about a huge number of different genetic factors. So like what the variants are, what proteins bind near the variants, right? Like all these kinds of things. And just asked it for type 2 diabetes to go and, you know, identify a mechanism associated with one of these variants. And it came back and it identified this was a variant that was not in a gene. And Cosmos identified that this is actually somewhere where a different protein binds. It was able to identify what protein binds and what gene is being expressed and connected that to the actual mechanism of that gene, SSR1, which is involved in the pancreas in secreting insulin.

15:08one, right? Okay, so in this case is what I'm hearing that your model was able to do some very fancy reasoning over some existing data and identify something that sort of no other human scientist had gotten around to and might not have for a really long time. Yeah, that's right. Okay. And I think science generally consists of deciding what data to gather, gathering that data, and then drawing conclusions. And so at this point, basically, it's like step number three that Cosmos is aimed at, you know, and there's more work on - You left out step zero, which was getting the Trump administration to unfreeze your funding, but everything else was right.

15:43So what happens when you get a discovery like this from Cosmos? Do you have to then go validate it? Do you hand it to like a team of researchers who then have to like make sure it works or like what happens next? Yeah, absolutely. You have to go and validate it. And so that's actually one of the things also, you know, in the paper, actually, we describe how we went and validated that particular variant. In general, when people are using it, yeah, you go in, I mean, actually, literally when When you run a Cosmos run, the first thing you have to do is you have to understand what it's telling you.

16:07Because it has just done something that scientists think is like six months worth of work. And you're going to sit there for a long time just like reading and understanding it. Once you've read it and understood it, then yes, indeed, you're going to go and you're going to run, you know, various experiments, do your own analysis, cross-reference to try to like convince yourself that this is true. And then based on what your research objective is, you'll decide next steps, right? In this case, I think it's probably low likelihood there's a new drug target from this particular finding, right? But you could go and you could run this on other findings, and then eventually maybe you find a new drug target, you start a drug program, that's, you know.

16:43So one concern that I've heard people express about models like Cosmos is that this is just like sort of not where the roadblocks are, that the sort of reason that we don't have more AI discover drugs and design drugs out there curing diseases is not actually because like we don't have the research methods to discover those. It's because there's like, you got to go to trials and you got to recruit human subjects and you got to get FDA approval. Like all that stuff just takes a lot longer than the actual discovery of the drug. So what problems are models like these helping to solve in our scientific process right now?

17:20So absolutely. I actually like, you know, I really agree that like the bottleneck at the end of the day in solving medicine is basically, you know, clinical trials. I mean, and the easiest way to see this is if you look at the number of diseases that we like know how to cure in mice, right? It's like astronomical because obviously you can just like run experiments. And in humans, things are just slow. That said, if you think that every experiment that is being run right now by pharma companies, like every clinical trial that's being run is like optimally planned and optimally, you know, conceived given the full state of knowledge, you are off your rocker, right?

18:00There's like no way. And those experiments cost hundreds of millions of dollars. And so the question is like, we do at the end of the day have to run clinical trials. How do we make sure that those experiments are the best experiments we could possibly be running given all the knowledge that we have, given all the data we have? There's so much data that we have that has insights in it that are waiting to be found where we just like do not have people to go and find them. And that's ultimately going to feed into better experiments, better trials, right? Well, so then I'm curious how you see like your tool fitting into the workflow of today's scientist?

18:33Is it the sort of thing where like I have completed my experiments and now I want some help doing some analysis? Is it, I have all these old experiments that I only did a little bit of analysis on and I'm curious if I can like sort of squeeze any more juice out of them? Or like what other ways are you seeing the AI being like really good right now for a working scientist? Yeah, yeah. Great question. So going back to me in 2019, which is when I was wrapping up my PhD, right, I had this gigantic data set and I wanted to graduate because I was a PhD student, which meant that I was making like, you know,$40 ,000 a year or something on.

19:07And like there were a ton of great opportunities to go out and like don't be a PhD student anymore. Or, okay, so I spent six months literally just like sitting at my desk, like trying to analyze the data and drawing conclusions, reading papers, right? For right now, that's what, that's where Cosmos fits in. It's like, you know, you would just take that data that you give it to Cosmos. It comes up with a lot of findings. Right now, you need to go and do a bunch of manual work to validate those findings and so on. Pretty soon, it's going to come with findings and you're going to be like, great. Sam, I'm curious if you could help sort of give us and our listeners There's a state of the world of AI science right now.

19:43Recently, the White House announced what it's calling the Genesis Mission, which is a federal effort to kind of corral and harness all of these data sets that the federal government is sitting on and use them to do new scientific exploring. We also have lots of efforts, including yours, but lots of things going on in and around the tech industry, the biotech industry, people doing AI for materials science. Give us a sense of like the lay of the land of like what's hot right now in AI science. Where is the effort and money going? Right. In order to understand the landscape of AI and science, the first thing like fundamentally that you have to understand is that AI is about building models, right?

20:28So for example, right, like a language model, like what is a language model? A language model is fundamentally a model of human language. It just so happens that when you build a model of human language, it learns how to think like a human in some sense because humans encode their thoughts in language. This is one of the greatest discoveries, certainly of the 21st century, maybe of all time. So similarly, when we talk about AI in science, what you have to think about is that you are modeling things. That is what AI does. And there are kind of two fundamental categories. There's modeling the natural world.

21:03Right. And there's modeling the process of doing science. These things are fundamentally different. And the reason to make this distinction is because, you know, what we are doing, right, we are modeling the process of doing science. The other side of the AI for science world is building models that can, for example, predict the structure of proteins, that can generate a new antibody, that can create a new organism from scratch, which are all things that have kind of like happened in 2025, where there's just a huge amount of momentum. Yeah, that makes sense. I mean, of the things that are happening in the part of the sort of process of modeling the natural world, you mentioned protein folding, novel organisms, like what has most excited you as a scientist that you've seen?

21:51So it's absolutely what's most exciting right now, I think, without a doubt, is this trend towards what we call generative models. So these are things where these are models that can produce examples of proteins or antibodies or whatever that have desired characteristics basically from scratch. This is a new capability that we have never had before, and it's huge. I'm curious about the reliability piece as you're running all of these experiments. I saw this going around on social media this week. I reproduced it myself. If you asked, Google, is 2026 next year? It said, no, 2026 is not next year.

22:26It is the year after next. So in such a world, Sam, some people might get concerned at the idea that we're now entrusting the AI with all of our data analysis. So how much time are scientists having to spend go back and essentially rechecking the work of the AIs? And what kind of tax does that place on their work? Yeah, this is very funny. I mean, look, you have to spend a lot of time going back and checking. Yeah. But like, to be clear, this is true regardless of whether or not an AI does it or whether you ask a friend to do it. If you're going to publish a paper, you damn well better go back and check it and like be sure that you are confident.

23:04And it's never going to be 100%, right? The best you're going to do is you're going to get to a place where it is similarly good to if you were doing it yourself, which is not 100 % because you're not infallible, right? and checking the work is like always going to be faster than producing it in the first place. Got it. Right. By a lot. A lot of our biggest scientific breakthroughs in history have come from these kind of strange accidents, these moments of serendipity, you know, penicillin starts growing in a Petri dish. Oh my God, it's, you know, it's, it's, this is great. Does AI preserve that kind of serendipity, those kinds of accidents, or do they sort of optimize it away?