Causal Discovery Is Not Causal Inference

Why mapping system mechanics is fundamentally different from A/B testing a black box — and why conflating them quietly wastes years of experimentation budget.

TL;DR — The industry treats causality as a validation exercise: running an A/B test to prove a local effect exists. My research treats it as an engineering blueprint: algorithmically discovering the system’s structural topology before you ever intervene. Causal inference estimates the magnitude of an effect given a causal structure you already assume. Causal discovery recovers the engine’s structure itself from data. The first is an estimation problem over parameters; the second is a search problem over a super-exponential space of graphs. You cannot optimize a complex, interconnected system by running isolated tests on a black box — you must first map the glass engine.

Below, I make the distinction formal, demonstrate with runnable code why the same dataset yields effect estimates ranging from -0.12 to +0.92 depending on which structure you assume, and describe how simulation-based discovery — using frameworks like PyCausalSim and CarryNet — replaces physical guesswork with algorithmic system mapping.

1. The Conflation: The Black Box vs. The Glass Engine

Ask ten data scientists what “causal analysis” means and you will get two incompatible answers wearing the same name.

The first group means interventional causality: “We ran a randomized A/B test, measured the lift, and computed a confidence interval.” The second group means mechanistic causality: “We have observational history across forty continuous metrics, delayed carryover effects, and complex dynamics, and we need to map the exact topology of what drives what to optimize the entire system.”

Both call it causality. But the first treats the business as a black box, seeking only to prove that pulling Lever A moves Metric B by a certain amount. The second seeks to build a glass engine, mapping the actual economic and causal flow so the global system can be optimized.

The practical cost of this confusion is specific and expensive. It shows up as a team that runs a well-powered experiment on the wrong variable. The randomization is clean, the p-value is honest, and the result is structurally worthless, because the variable being tested was chosen from a correlation matrix. The true driver was three edges upstream, unmeasured in the test design.

The goal of causal discovery is not to prepare a single variable for an A/B test. The goal is to map the system mechanics. When you treat causality purely as an interventional testing problem, you are flying blind — trying to optimize an engine by randomly swapping parts and listening to the noise.

Full article with detailed steps and instruction available to my paid subscribers on substack: https://brianjamescurry.substack.com/p/causal-discovery-is-not-causal-inference

About the Author

Brian Curry is a Kansas City–based Quantitative Data Science Leader, AI Strategist, and Applied Researcher. He is the founder and board chair of KC AI Lab, LLC, and operates Vector1 Research as a personal project entity dedicated to advancing open-source frameworks in marketing economics, causal inference, and cognitive AI systems.

He has led analytics strategy, causal modeling, and data science initiatives across major enterprises including Vail Resorts, Tractor Supply Company, Koch Ag & Energy Solutions, Hallmark Cards, and Garmin.

Research & Open-Source IP:

• PyCausalSim — Python framework for causal discovery through in silico simulation.
• Papilon — Open-source platform for complex-system analytics and simulation.
• CarryNet — Algorithmic framework for modeling delayed causal carryover effects.
• Daedalus — Platform for money-flow mapping and economic causal analysis.
• Memory-Node Encapsulation (MNE) — Cognitive AI architecture for artificial episodic memory.

Contact: brian at vector1.ai | Vector1 Research | GitHub: @Bodhi8