Exploratory Causal Inference in SAEnce

0:00Welcome back to the Deep Dive. Today we're looking at something that is really starting to shift how science gets done. We're moving away from just testing a single idea to pure data-driven exploration. This is about the future of discovery itself. It's a shift we absolutely have to make. I mean, the amount of information we can generate now is just, it's astronomical. Right. Think about the data sets we have now. You've got these planetary scale maps of genomes. You have labs imaging the inside of a cell under, you know, thousands of different chemical perturbations. Or sequencing 33 different types of human cancer.

0:36Exactly. We have these incredible atlases of information. And that's a big change from the traditional way of doing things, the classical rationalist view, where you come up with a very specific testable hypothesis, something like treatment A decreases reaction time in species B. Right. Then you design the experiment, collect only the data you need, and you try to falsify your idea. It's a very targeted scalpel-like approach. It is a very disciplined scalpel approach. But when you're looking at petabytes of raw video, that scalpel starts to feel a little inadequate. And that's where the empiricist view comes in.

1:13The emerging empiricist view basically says, collect the whole map first, the comprehensive map, and then explore it. We're sort of moving from the scalpel to the satellite. Which brings us to the core challenge, really the mission of this deep dive, exploratory causal inference, ECI for short. Right. So you run a classic scientific trial. You have a treatment, let's call it T, but the outcome Y is buried somewhere inside just tons of raw data X. Like video footage or complex medical images. Exactly. How do you find the causal effect when you haven't even decided what the effect is supposed to look like?

1:49What if you don't even know the right question to ask yet? And that's our mission today. We want to give you the shortcut to understanding this whole ECI pipeline. We'll look at a critical bias that gets in the way of discovery. A really surprising paradox that can make even the best experiments fail. And then the novel solution, something called neural effect search, or NES, which is designed to, you know, rescue this kind of modern exploratory science. Okay, so let's start with that bias. It's a problem that's actually rooted in the traditional rationalist approach. It is. It's called the Matthew effect.

2:22You might have heard of it as the rich get richer bias. People use that to talk about society, but how does it apply in a science lab? Well, in science, it's about what gets studied. It basically suggests that scientists are, you know, they're biased towards studying outcomes they already know about. Things that have been successful in prior research. Exactly. When they set up a new experiment, they have to decide what to annotate a priori. Before they even see the data, they have to define their variables up front. So let's take the ant behavior example from the source material. A biologist has a new chemical treatment.

2:56they might decide to manually label things they've always labeled. Running speed, feeding time. Aggression levels, the classics. Right. But what if the treatment's only real effect is on some super subtle, you know, previously unknown type of social grooming? Like a very specific mouth-to-mouth liquid exchange that no one's ever really documented before. If the researcher never thought to look for that or it's just too subtle to label by hand. Then that crucial causal link gets completely missed. The rich, the known hypotheses get richer with more attention. And the poor, these unknown subtle effects, they just stay undiscovered.

3:33So ECI has to be the counter approach. It's about generating data-driven hypotheses from the observations themselves. It's meant to enrich the rationalist view, not replace it. The goal isn't to get rid of the domain expert. It's to give them, what, dozens, maybe thousands of data-driven candidates to look at. Instead of forcing them to commit to just one possibly biased hypothesis from the get-go, it flips the whole script. Instead of the expert defining the measurement, the AI learns what the measurement is. Okay, let's unpack that. How do we actually do this? If you just have raw, unstructured video data, X, how do you turn that into something testable?

4:11Into something a statistician can work with, yeah. This is where the AI acts as a learned measurement device, right? Right. It's a two-step process, really, that relies on modern machine learning. Step one is a foundation model, an FM. You take your raw observations, like a single video frame, X, and you feed it into one of these huge pre-trained models, things like Siglip or Dyno V2. And these models are so powerful because they've, what, they've seen billions of images already. They've learned what things in the world look like. Precisely. They turn raw pixels into mathematical concepts, things like smooth edges, movement, circular shape.

4:48And the really critical assumption here is something called sufficiency. Meaning? Meaning you're assuming the foundation model preserves all the important information about your unknown outcome, Y, that was in the original data X. If the treatment really did affect some subtle behavior, the math to prove it has to be in those features. That makes sense. But hold on. Doesn't this just replace one kind of human bias, the preannotation, with another? The bias of the pre-trained bottle. That is a fantastic question and a very important constraint the sources point out. How do we know the FM isn't just hiding the most interesting novel effects?

5:24Because its worldview was shaped by training data that, you know, focused on common everyday things. It's a real risk. We try to mitigate it by using FMs trained on these massive generalized data sets to minimize those domain-specific biases. But you're right. For now, the FM is kind of a necessary evil. It does the impossible job of turning raw data into features. If the true causal effect isn't captured in those features, well, then ECI fails before it's even begun. Okay, fair enough. So we've gone from messy raw data X to these dense mathematical features H. Now what? Step two. Step two is the sparse autoencoder, or SAE.

6:04This is a different kind of network. It takes those dense features H. All ones from the foundation model. Right, and it reparameterizes them into a sparse interpretable measurement dictionary. We'll call it Z. Okay, why sparse? Why is that the key? Because density is opaque. You can't interpret it. Sparsity gets us closer to an ideal called monosemanticity. Which means? The goal is for each coordinate in that dictionary, each neuron, to be a detector for one single simple thing. If the treatment affects ant head movement, then only the neuron for ant head movement should light up. That's how we build our learned measurement device.

6:38That's the dream, a clean one-to-one mapping. But this is where it gets really interesting, because you almost never get that clean separation in practice. You don't. Not really. In reality, these SAE codes suffer from leakage and entanglement. That leads to polysemanticity. So one neuron, it might be mostly about ant head movement, but it could also respond a little bit to fur color or shadow position. Exactly. Because of tiny overlaps in the data or just imperfections in the training. And that little bit of messiness, that entanglement, leads us directly to the big technical problem. the paradox of exploratory causal inference.

7:14Because when even that minimal entanglement exists, all our traditional statistical tests, they just fail dramatically. And here's the core issue. The problem actually gets worse the better your experiment is. Wait, that sounds completely upside down. If I run a more powerful trial with more data, clearer effects, I should be getting clearer results, not worse ones. Logically, you'd think so. But statistically, no. Here's an analogy. Imagine you're trying to hear a very, very faint whisper. That's the real small causal effect tau. And you're trying to hear it in a huge stadium where thousands of people are just subtly humming.

7:52Those are your thousands of entangled neurons. Now, you increase the microphone's gain. That's like increasing your sample size, Anne. I amplify the whisper. Makes sense. But you also amplify every tiny bit of electrical static, every hum from the microphone itself, every sound leak. That's the entanglement. As you keep turning that gain up and up, eventually the static is so loud that every signal, real or not, crosses your detection threshold. So I can't tell the real whisper from the random hum anymore. You can't. The power of the experiment becomes its own liability. So the paradox, as the sources define it, is that as your sample size or your effect magnitude grows, standard multiple testing, even with really strong corrections, ends up flagging all of the outcome entangled neurons as being independently significant.

8:40Wow. Mathematically, the statistical power, it just grows relentlessly with the sample size and it completely overwhelms the corrections that are meant to stop false positives. So any neuron, it doesn't matter how minimally entangled it is, as long as it has some non-zero effect, it'll eventually be declared significant. The consequence of that for a scientist is catastrophic. They run this perfect massive experiment with their AI measurement device, and they get back a list of thousands of significant effects. And most of them aren't real distinct discoveries. They're just weak leakage signals, statistical artifacts.

9:14So interpretation just collapses. You can't tell the signal from the noise. We need a system that gets it, a system that understands that lots of these little signals aren't unique causes. They're just echoes of the main one. We have to filter out that noise and find only the principal effects. So what does this all mean? We need a system that disentangles as it goes. We need neural effects search. Precisely. Neural effects search, or NES, is this new causally principled algorithm that solves the paradox using something called recursive stratification. Okay, stratification. That just means controlling for a variable to remove its influence, right?

9:51Yes, exactly. And recursive just means it does it over and over, peeling away layers. So how does it work, step by step? It's an iterative search. So first, NES finds the most prominent effect in the data, the loudest whisper in the stadium. It finds the single most representative neuron for that effect. Okay, it's got the main signal. Right. Once it's confident about that primary effect, NES uses the information from that one neuron to stratify or statistically residualize the test for all the other entangled neurons. It's basically saying, okay, we found the main signal for running speed. Now let's go back and test every other neuron again.

10:27But first we're going to mathematically subtract the influence of running speed from all of their measurements. You got it. That stratification process kills the spurious signal that was just leakage from the first effect it found. It forces the remaining neurons to prove they have genuinely new information, that they aren't just echoes. And that's why it's so robust where the standard methods fail. It's not just testing. It's actively disentangling as it goes. It is. It's a very different approach. All right. Let's take this out of the theory and into the real world. You mentioned an application, the Istent Ecological Experiment.

11:02It was studying something called social immunity in ants. Yeah. This was a perfect test case. You have these ant triplets. They're randomly assigned to a treatment or a control substance, and the researchers just filmed them. The biologists wanted to find subtle behaviors that were affected, but without, you know, the huge labor of manually annotating hours and hours of video. And crucially, they only had 44 videos, a pretty small sample size, which is typical for these kinds of experiments. Very typical. So they ran the whole ECI pipeline. They pushed the video frames through the foundation model, DiEMV2, to get the features.

11:35Then the sparse auto-encoder to create the measurement dictionary. And then they ran neural effects search on it. And what did it find? N.E.S. found two statistically significant treatment-sensitive codes. The first one was Code 394. When the experts looked at it, they interpreted it as the grooming event. Grooming. And this was a fascinating result because that exact finding, that the treatment affects ant grooming, had already been confirmed by the original researchers using the old-school slow, manual, rationalist analysis. Wow, that's a perfect validation. the AI finds the known, real scientific signal all on its own, completely unsupervised.

12:15It validates the whole pipeline. But this is where it gets even more interesting. The second significant finding was code 550. And this code was interpreted as the palette background. Yeah, it was basically detecting a black color mark in the top left corner of the video frame. And statistically, this code had a strong correlation with the treatment group. Wait a minute. The color of the background was affected by the ant treatment. That sounds like a failure. It's a false correlation. It is a false correlation scientifically, but its discovery is actually seen as a huge strength of this whole method.

12:48How so? NES accurately identified a statistically significant experimental artifact. Because the sample size was so small, the treatment condition just happened to be imbalanced with respect to that background mark. Maybe the control group was filmed more often on one side of the dish, or a corner got dirty over time. Ah, I see. So in a normal experiment, that subtle background bias might have gotten tangled up with a real behavioral signal, and it would have confused the whole interpretation. Precisely. But here, the experts get two clean signals. Code 394 is the real science, the grooming. And code 550 is the experimental flaw, the background bias.

13:26They can confidently separate the two, and they can use that to clean up their experimental setup next time. It turns a potential data flaw into really useful feedback. This feels like a huge shift. This ECI pipeline with foundation models and SAEs, it basically creates these learned measurement devices. It offers these dramatic efficiency gains for that first exploratory phase of science. It's a massive step forward, absolutely. But we have to be clear about the limitations. The source material is very upfront about this. The whole approach is still really dependent on this assumption of good identifiability from the SAEs.

14:04Meaning if that sparse autoencoder is just too messy, if the neurons don't map cleanly to real world concepts, then the causal analysis downstream is going to struggle, even with NES doing the heavy lifting. So the goal isn't to replace the rigorous rationalist approach. Not yet. Not yet. For now, the method is really being used as a, I think the quote was a rescue system for hypotheses they may have missed. It speeds up exploration. It generates strong candidates that you can then go and confirm manually. But that expert domain knowledge, you know, to interpret the codes, that remains absolutely critical.

14:36That's a vital distinction. It's an AI-powered telescope for finding things you weren't even looking for, but it's not a replacement for the microscope. Exactly. And that brings us to our final thought for you to chew on this week. Let's think about the power of neural effect search to fight this entanglement problem, its ability to peel away those leakage signals and separate them from real unique causal factors. How many weak yet truly distinct causal effects might have been completely missed in past experiments, across all fields of science, simply because our conventional analysis just dismissed them as statistical noise or as unavoidable measurement leakage?