


25 Jun 2025 · 22m · 8 chapters

Argues that causal inference (interventions and counterfactuals) can be fully handled within standard probabilistic modeling, using the principle “write down the probability of everything” (David McKay’s rule), rather than requiring a separate causal math framework.
No guest names or bios are provided; the episode is a host/interviewer style discussion with two speakers.
(1) Interventional queries can be solved by a “twin model” that places observed and intervened worlds in one joint distribution, sharing parameters (e.g., C) while breaking the intervention’s dependence (e.g., T independent of Z). (2) Classical do-notation/causal graphs are syntactic shortcuts; Markov-equivalent graphs can yield different intervention results. (3) Counterfactuals can be handled by sharing relevant individual variables (e.g., Z for a specific person) and using Bayesian updating; SCMs’ abduction-action-prediction is equivalent to the probabilistic twin approach.
Aspirin dosage (T) vs headache duration (Y) with confounding by initial severity (Z), showing Simpson’s paradox; RCTs as physical do-interventions; mediator vs confounder and hidden confounder pitfalls.
Written by AI. May contain mistakes.
Discussion on the paper claiming probabilistic modeling suffices for causal inference.
“This paper just boldly steps in and says, hold on a minute, maybe not.”
Exploration of the disagreement between Judea Pearl and Andrew Gelman regarding causal questions.
“Now, it's crucial to clarify something here because the paper isn't saying causal tools are useless, right?”
Explanation of interventional causal questions using aspirin's effect on headaches as an example.
“In other words, if we make someone take aspirin, what happens to their headache, not what happens if they choose to take it?”
Discussion on how naive approaches to causal inference can lead to incorrect conclusions, illustrated with Simpson's paradox.
“The math then clearly shows that the effect of a higher dose T is directly determined by that parameter C.”
Introduction and explanation of the twin model approach to answer interventional questions.
“you might accidentally block the very causal pathway you're trying to study.”
Learn about the concept and function of Structural Causal Models and their relevance to counterfactuals.
“If that a y represents something unique and persistent about your friend, like her specific physiology not captured by z, then yes, sharing it makes sense.”
Explore the advantages of using a probabilistic framework for causal inference and modeling.
“So the probabilistic approach covers it too, potentially with less restrictive assumptions about function forms and noise sharing.”
Discuss the semantic differences between causal and statistical inference, aiming for clarity and unification in definitions.
“It's more about how we define our terms than a deep mathematical difference.”

Get the gist, explore an idea or ask about something said.
On Best AI papers explained, what are the key points of this episode?
Automatic transcript. May contain errors.
0:00Welcome to the Deep Dive, where we cut through the noise to get you informed fast. Today, we're tackling one of machine learning's biggest what ifs. Can we really answer questions about cause and effect with just standard probability and statistics? Or do we need something entirely new, a completely different framework? Yeah, it's a fundamental question. We've been absolutely gripped by a fascinating new research paper, Probabilistic Modeling is Sufficient for Causal Inference, which just came out at ICML 2025. And honestly, it really challenges some, you know, long-held beliefs in the field. What was your biggest takeaway from its core argument?
0:36You know, it's a huge one. There's this idea that's really common in the ML community right now that to even touch causal questions, you need bespoke tools, new notation, a whole different language almost. This paper just boldly steps in and says, hold on a minute, maybe not. So its core claim is essentially that you absolutely can answer any causal inference question purely within the realm of probabilistic modeling. No need for entirely new math, no jumping ship to a different discipline. That sounds, well, almost too good to be true, doesn't it? It does sound radical, but they make a compelling case.
1:11They argue you just need to follow David McKay's like beautifully simple overarching rule. Always write down the probability of everything. And when you actually do that, it turns out all these causal problems fit right in. It's quite elegant. Okay. So our mission today is to guide you through that argument, showing how to approach these complex causal problems by doing exactly that, writing down the probability of everything. And what's really cool is we'll see how those specialized causal tools you might have heard of aren't, you know, replacements for probability. Right. But rather emerge naturally from standard probabilistic methods.
1:45Okay, let's unpack this. Let's do it. This brings us directly to a central disagreement that has really shaped the field for decades. The great debate between two foundational researchers, Judea Pearl and Andrew Gelman. Pearl famously stated that there is no way to answer causal questions without snapping out of statistical vocabulary. He believes we fundamentally need to enrich our language with a do operator. It's about a distinct causal language for him. Exactly. He really pushes for that separation. And then you have Gelman, who on the other side finds it baffling that Pearl and his colleagues keep taking statistical problems and, to my mind, complicating them by wrapping them in a causal structure.
2:24He sort of sees it as making things unnecessarily convoluted. Yeah, it sounds like a pretty fundamental disagreement. So what does this new paper say about this alleged insufficiency of standard statistical tools? Well, the paper argues that these claims of insufficiency are not only really common, but also, frankly, incredibly confusing for anyone new coming into the field. The authors firmly assert that standard tools of probabilistic modeling are sufficient for causal inference. by rigorously applying McKay's rule. They show it can all be done within the probabilistic framework. Right. Now, it's crucial to clarify something here because the paper isn't saying causal tools are useless, right?
3:07It's not throwing the baby out with the bathwater. Absolutely not, and that's a really important point. The paper acknowledges their immense utility and the profound insights they've provided, especially Pearl's work. The point is they aren't necessary. It's not proposing a brand new framework. It's more illustrating how causal problems fit perfectly into an existing, well-understood one. The goal is really to make causal inference more accessible, more flexible, maybe. That makes sense. Less a new blueprint, more like detailed instructions for tools you already have. Exactly. That's a great way to put it.
3:38Okay, so let's dive into the first type of causal question. Interventional queries. You know those, what would happen if I did something? Questions. Precisely. They ask, what would happen in a hypothetical setting where we have altered the mechanism by which something is determined? So it's like, if I force something to happen, what's the outcome on average? And the paper grounds its discussion with a very relatable running example, aspirin's efficacy on headache duration. You've got three main ingredients here. Z for initial headache severity, T for the aspirin dosage someone takes, and Y for how long their headache lasts.
4:14Simple setup. Right. And in their model, these are all distributed in a specific way, using log-normal distributions. They introduce parameters like C, which captures the direct strength of aspirin's effect on headache duration. So if C is large, aspirin is really effective. Then A and B are there to link how that initial headache severity might influence both the dosage people choose and how long their headache lasts naturally. Okay. So the core question they're trying to answer with this setup is, what is the expected effect of intervening to assign someone a given dose of aspirin tea on their headache duration on average?
4:48In other words, if we make someone take aspirin, what happens to their headache, not what happens if they choose to take it? Exactly. And this is where the naive approach, which is basically just simple supervised learning, completely falls apart. If you just tried to estimate the expected headache duration given a certain dosage, like EYTT, you'd likely get the wrong answer. Why? What's the issue there? The fundamental issue is confounding. In the real world, initial headache severity, Z, affects both how much aspirin people decide to take and how long their headache lasts. You can see how that messes things up if you're just looking at correlations.
5:24And what's really surprising, or maybe not surprising if you know about these things, is that the paper shows even if aspirin had no effect on headaches, say C was zero, the naive approach might still show that people who take higher aspirin doses have longer headaches on average. That's totally counterintuitive. Why would that happen? Because the people taking more aspirin typically started with worse headaches. It's not the aspirin causing the longer headache in that scenario. It's the more severe initial headache that led to both taking more aspirin and having a longer headache. It's a classic, beautiful example of Simpson's paradox.
5:57The paper illustrates how, while duration seems to increase with dose overall, if you zoom in on any group with a narrow range of initial severities, the trend actually reverses. Aspirin helps. I remember when I first encountered Simpson's paradox, my mind was just blown, thinking, wow, correlation really doesn't imply causation. And that inadequacy of the naive approach, that's kind of the cornerstone of arguments that machine learning techniques are not sufficient for causal inference, isn't it? It absolutely is. It's the go-to example. It highlights that causal questions often require you to generalize beyond the observed data distribution.
6:33And that means making significant modeling assumptions about how the data was generated. But the paper's critical insight is that these assumptions don't need a special language. They can all be specified within probabilistic modeling. Okay, and this is where their twin model approach really comes in. The core idea, as I understand it, is that to answer an interventional question, you have to explicitly model both worlds, the observed world and the hypothetical intervened world within a single joint distribution. Is that right? Exactly. It's about writing down the probability of literally everything you're interested in, side by side.
7:06Imagine two parallel universes in one model. One showed how things naturally unfold, severity influences dose, dose and severity influence duration. In the other, this twin world, we explicitly set the aspirin dose, T. And what connects these two worlds? Crucially, the underlying mechanisms and parameters, like C, aspirin's effectiveness, are shared. So observed data informs the hypothetical, and, importantly, in the intervened world, the aspirin dose T is modeled as being independent of the initial headache severity Z. This reflects the intervention you're assigning a dose, regardless of how bad the headache was.
7:41We also assume the distribution of initial severity Z is the same, and the way Y depends on Z and T is the same physiological process. So by writing down this full joint distribution, probability theory pretty much just, guides you to find the expected headache duration in that intervened world, EYD. That's the idea. The math then clearly shows that the effect of a higher dose T is directly determined by that parameter C. If C is greater than zero, a higher dose shortens the headache. It isolates the causal effect. The beauty of this approach is its transparency. All the assumptions are right there in the model specification.
8:17If they're correctly specified, it's valid, and you didn't need any, you know, Biscoe causal syntax. Okay, so if this twin model approach is so powerful and transparent, where do all those well-known classical causal tools fit in? Like the do operator and causal graphs, are they just redundant now? Not at all. The paper doesn't dismiss them. It argues they're more like syntactic sugar for this underlying probabilistic framework. Take the formal intervention operation in Bayesian networks. It's essentially a specific, well-defined procedure for getting a new joint distribution for an intervened setting.
8:49You're basically modifying the original graph, like snipping the arrows going into the intervened node and setting its value or distribution. Ah, okay. And the famous do notation, like PXOXKC, is just a shorthand for the result of that whole graph snipping distribution setting operation. Precisely. It's a very compact, very useful way to express what is underneath a modification of a joint probability distribution. And confounding then gets this really neat definition. It's when the observational conditional probability, PIT, is not equal to the interventional probability, PIT. Makes sense. And the due calculus.
9:26That's basically a set of rules, like algebraic rules, derived from probability theory for manipulating these due expressions. Super useful for figuring out if you can estimate a causal effect from observational data. But again, built on probability. Now, there's a tricky point here you mentioned earlier, Markov equivalence classes. Can you walk us through that again? Why is that a potential pitfall with the classical graph approach? Right. This is subtle but really important. You can have different Bayesian network graphs, different structures of arrows that perfectly describe the same set of observed conditional independencies.
9:59You can't tell them apart just by looking at observational data. But when you apply the do operator to these seemingly equivalent graphs, they can apply totally different outcomes for the intervention. Oh, wow. So the observational data alone doesn't tell you which graph structure is right for predicting interventions. Exactly. Which means if you're using these classical tools, you either need strong prior knowledge to pick the correct graph structure, or you have to state your assumptions explicitly in a joint model, which is exactly what the twin model approach forces you to do anyway. It avoids this ambiguity.
10:32That really clarifies the value of the explicit joint modeling. And you mentioned RCTs before. Yeah, randomized controlled trials. They are essentially physical implementations of these atomic do interventions. You randomly assign the treatment, like aspirin dose, breaking the lane from confounders, like headache severity, to the treatment choice. So there's a very direct intuitive connection. But modeling these interventions isn't always straightforward, is it? The paper mentions challenges. Definitely. It's nuanced. For instance, distinguishing mediators from confounders. If Z included something affected by aspirin, like prostaglandin levels, which then affect the headache that's a mediator, if you just blindly remove the arrow from Z to T in your intervened model, you might accidentally block the very causal pathway you're trying to study.
11:19The model has to reflect the actual physical mechanisms correctly. And the biggest elephant in the room? Hidden confounders. That's the toughest one. What if there's something you didn't measure, like maybe a fever that affects both how much aspirin someone takes and the type or duration of their headache? If it's not in your model, your estimates can be completely wrong. This is where Pearl's work has been incredibly valuable, showing just how easily you can be misled if you don't account for confounding properly. It really forces you to think hard about your assumptions. Absolutely. Okay, let's shift gears then to the second type of causal question, counterfactuals.
11:57These are those really intriguing, what if I had acted differently questions, but crucially, asked about a specific past event. Right. It's not about averages anymore. It's about what would have happened in this one specific instance if something had been different. And they use the friend's headache example. Your friend had a headache, took dose T, it lasted Yao hours. Now you wonder, had she taken a larger dose 10, would her headache have gone away faster? For that specific friend, that specific headache. Exactly. So how does the probabilistic twin model handle that? It's slightly different from the interventional twin model.
12:32Instead of just sharing parameters between the observed and hypothetical worlds, this approach shares specific individual variables. Which variable here? In the friend's headache example, the key assumption is that her initial headache severity, Z is the same specific value in both the reality that happened and the counterfactual scenario where she took dose T. It's her underlying severity. Okay, so the Bayesian network looks different again. Observed world, Z leads to T leads to Y, counterfactual world. That same Z leads to T, the hypothetical dose, and Y, the hypothetical outcome. Precisely.
13:05The goal now is to compute PYTYD. That's the probability of the hypothetical headache duration Y, given what actually happened, Dose T, Duration Y and your general background data, D. The way you do it involves a neat Bayesian trick. First, you infer the posterior distribution over your friend's specific headache severity, Z given the T and Y you observed. Basically, how bad was her headache likely given what happened? Then, you use that specific posterior for Z as the prior for Z in the counterfactual model when you calculate the effect of T. That's quite elegant. It connects the observed reality to the hypothetical one through that shared, inferred variable z.
13:44But this brings up a deeper point the paper digs into, sharing latent or noise variables. If the model has randomness, like y, f, c, d, key, y, should that random bit y also be shared between the real and counterfactual friend? Yeah, that's a fascinating debate. And the paper argues, it depends, it's contextual. If that a y represents something unique and persistent about your friend, like her specific physiology not captured by z, then yes, sharing it makes sense. You want to know what happened to her with all her quirks? But if AOI represents, say, random variation in the potency of the specific pill she took, maybe sharing that exact same random fluctuation doesn't make sense for a different hypothetical pill.
14:25So the context of what the noise means is critical. Exactly. And this contrasts with some classical causal literature, particularly using structural causal models, SCMs, which often requires all sources of randomness to be shared by default. The paper critiques this as potentially overly restrictive. Maybe you don't need to share all the noise for the specific question you're asking. Okay, so let's talk about those structural causal models, SCMs. We sometimes hear claims that SCMs are necessary for counterfactuals, that simpler causal graphs aren't rich enough. How does the paper tackle that? Right.
14:57So what is an SCM, roughly? It's like a Bayesian network, but with an extra layer. Each variable is defined as a specific deterministic function of its parents and some unique latent noise variable. So, X, parents, X, noise, X. This functional definition means SCMs contain, as they say, strictly more information than just the graph structure. But crucially, any SCM implies an underlying Bayesian network structure. You can think of SCMs as a special, more restricted class of Bayesian networks that explicitly includes these noise terms and functional forms. Got it. So they're more specific. How do they typically handle counterfactuals than in that classical approach?
15:34The standard SCM procedure for counterfactuals usually involves three steps, sometimes called abduction-action-prediction. First, abduction. You observe the actual situation, like T and Y, and infer the posterior distribution of all the latent noise variables, and the muster produced this observation. Second, action. You modify the SCM equations to reflect the counterfactual intervention, like changing the equation for T to T. Third, prediction. You use the inferred noise variables end from step one and the modified equations from step two to compute the counterfactual outcome. Why? Okay, that sounds like a very defined process.
16:09So what's the paper's take on this? Here's the kicker. The paper argues and demonstrates mathematically that this whole three-step SCM procedure is mathematically equivalent to the probabilistic twin model approach for counterfactuals that we just discussed. The twin model, where you share the relevant variables like Z and infer posteriors achieves the same result. So the conclusion is, SCMs are sufficient for counterfactuals. Yes, they work, but they're not necessary. You don't need to specify everything in terms of deterministic functions and latent noise if your probabilistic twin model captures the necessary dependencies.
16:46Wow. Okay. So the probabilistic approach covers it too, potentially with less restrictive assumptions about function forms and noise sharing. Exactly. It offers more flexibility in how you specify the model based on what you actually need for your question. Beyond just proving sufficiency, the paper highlights some other real benefits of sticking within this unified probabilistic framework. You mentioned flexibility. Yeah, huge flexibility. For example, you can model things that are tricky in other frameworks like backtracking counterfactuals, asking questions like, Given my headache went away fast after taking aspirin, was it likely because my initial headache wasn't that severe?
17:20It lets you reason backwards about causes. Also, you're not tied to just graphical models. Probabilistic programming languages like STAN or PyMC or Pyro let you build much more complex, nuanced models than you can easily draw as a graph. Think hierarchical models, complex dependencies. And combining data types? Yes. It provides a natural way to combine data from different sources. Maybe you have some observational data and data from a small clinical trial and intervention. You can use both within the same probabilistic model to learn the shared parameters more effectively. That sounds incredibly practical.
17:56And it avoids those rigid steps you mentioned with SCMs. Right. The SCM abduction action prediction is one way to compute things. But the probabilistic approach you just says, specify the full joint distribution over everything. Observed, counterfactual. Then use the rules of probability to compute whatever conditional probability or expectation you need. It's less prescriptive. And you think this lowers the barrier to entry? I think so, definitely. The machine learning community is already deeply familiar with probabilistic modeling, Bayesian inference, tools like PPLs. Framing causal inference as an extension of these familiar methods, rather than a whole separate field with its own syntax, could make it much more accessible to many more people.
18:37And you said it's already having an impact. Yeah, the paper points to work, for instance, in identifiability research and deep learning. Researchers are using these ideas to show formally that certain causal queries about learned representations are actually equivalent to, or reducible to, purely probabilistic queries about the model's distributions. It lends strong support to this unified view. Okay, so let's tie this all together. The paper addresses alternative views, particularly this idea of a causal-statistical dichotomy. What's the main point there? The authors really believe the disagreement about whether causal and statistical inference are fundamentally distinct is primarily semantic.
19:17It's more about how we define our terms than a deep mathematical difference. They argue strongly that terminology insisting they are distinct is deeply misleading and unnecessarily confuses newcomers. How does it confuse them? Well, often claims that statistics is insufficient for causation rely on a narrow definition of statistics, like it's only about summarizing observed data correlations, or it always assumes IED data. But modern statistics, especially Bayesian statistics, is perfectly capable of incorporating subjective assumptions, modeling complex dependencies, and handling non-static situations like interventions that change the data distribution.
19:56It's much broader than that narrow view. And Perl's position evolved slightly on this. Yes. The paper notes that Perl later refined his distinction to be between associational and causal concepts, which is clearer. But it still tends to label a whole class of concepts derived from interventions or counterfactuals as uniquely causal, separate from the probabilistic machinery used to compute them. So the legacy of that distinction persists. It seems so. The language used in a lot of current machine learning literature still reflect this idea of a fundamental split. it. This paper is making a strong push for clarity and unification.
20:32Okay, so let's recap the main point for everyone listening. The core takeaway seems to be causal inference can be comprehensively addressed through probabilistic modeling. The key is to explicitly define a probabilistic model over all the settings you care about, observed, interventional, counterfactual, all within one framework. That's it, exactly. This approach is clear, it's useful, it's general. It potentially resolves a lot of the confusion about whether that you need specialized causal framework beyond probability theory. It's like realizing your standard toolkit, if used thoughtfully, can handle way more than you initially thought.
21:06So if there's one key takeaway we want you to mull over building on McKay's rule, it's this. Always write down the joint probability over all the settings you're interested in. That single principle seems to unlock a unified, robust way to think about cause and effect. It's a powerful guiding principle. So maybe consider how this shift in perspective could simplify how you think about or tackle causal questions in your own work or even just in understanding the world around you. It's definitely food for thought.
This position paper argues that probabilistic modeling is sufficient for causal inference, directly challenging the prevalent idea that specialized causal frameworks or notation, such as Pearl's "do-operator," are necessary. The authors demonstrate through concrete examples like aspirin's effect on headaches how interventional and counterfactual questions can be answered by explicitly defining joint probability distributions across observed and hypothetical "intervened" or "counterfactual" worlds. They present the "twin model approach" using Bayesian Networks to illustrate this, emphasizing that conventional causal tools can be reinterpreted as convenient "syntactic sugar" within a broader probabilistic framework. Ultimately, the paper advocates for a more accessible and flexible approach to causal problems within the machine learning community, asserting that the perceived "causal-statistical dichotomy" is primarily a semantic issue.