The Direction of Money: Information-Theoretic Causality for Media Measurement

Part two of a series on digital media optimization. Before you fit a single regression, transfer entropy can tell you which channels actually drive revenue, which ones are just chasing demand, and which “high-ROI” channels are an illusion. A model-free, nonlinear, directional approach to media causality — with runnable code and a fully worked example.

The question regression cannot answer

In part one we built response curves, optimized budget under constraints, and flagged the causal problem at the end: regression conflates E[revenue | spend] with the interventional E[revenue | do(spend)). When spend reacts to demand, a naive model credits the demand's lift to the spend.

But there is an even more basic question that correlation-based methods physically cannot answer:

Which way does the money flow — does spend drive revenue, or does revenue drive spend?

Correlation is symmetric. corr(spend, revenue) == corr(revenue, spend). A Pearson coefficient of 0.7 looks identical whether your paid search genuinely generates sales or whether your budget-pacing rules simply pour money into channels after sales spike. These are opposite causal stories with identical correlations, and the second one — reverse causation, "budget chases sales" — is the single most common way media ROI gets inflated. Branded search and retargeting are the usual offenders: spend rises because intent already exists, then the model hands the credit to the spend.

You cannot see direction with a symmetric measure. You need something asymmetric, and ideally something that also catches nonlinear relationships (saturation curves are nonlinear) and makes no distributional assumptions. That tool exists, it predates most of modern MMM, and it comes from physics and communication theory rather than econometrics: transfer entropy.

This article builds it from the ground up — entropy, mutual information, then transfer entropy — shows how to estimate it and test it for significance, works a full example with real computed numbers, and lays out where it fits in a measurement stack (spoiler: as a discovery and screening layer before the MMM, not a replacement for it).

Why information theory

Three properties make information-theoretic measures uniquely suited to this problem.

Model-free. Mutual information and transfer entropy make no assumption of linearity, Gaussianity, or any functional form. They operate directly on probability distributions estimated from data. A regression assumes you already know the shape; information theory measures dependence without one.

Nonlinear. Pearson correlation captures only linear association — it can be exactly zero for a perfect nonlinear relationship. Media response is saturating and threshold-driven by nature, so a linear screen misses real signal. Information measures capture dependence of any shape.

Directional (transfer entropy only). This is the payoff. Mutual information is still symmetric, but transfer entropy is explicitly asymmetric in time — it asks whether the past of one series reduces uncertainty about the future of another, beyond what that series’ own past already explains. That temporal asymmetry is what encodes direction.

The building blocks

Everything rests on Shannon entropy, the amount of uncertainty (in bits) in a random variable:

H(X) = − Σ_x  p(x) · log₂ p(x)

From it, the joint and conditional entropies:

H(X, Y) = − Σ  p(x, y) · log₂ p(x, y)
H(Y | X) = H(X, Y) − H(X)

H(Y|X) is the uncertainty remaining in Y once you know X. In code, all of these reduce to counting:

import numpy as np

def discretize(x, bins):
    """Quantile binning -> roughly uniform marginal occupancy."""
    edges = np.quantile(x, np.linspace(0, 1, bins + 1))
    edges[0] -= 1e-9; edges[-1] += 1e-9
    return np.clip(np.digitize(x, edges[1:-1]), 0, bins - 1)

def entropy(*arrays, bins):
    """Joint Shannon entropy (bits) of one or more discretized arrays."""
    idx = np.zeros(len(arrays[0]), dtype=np.int64)
    for a in arrays:
        idx = idx * bins + a
    counts = np.bincount(idx, minlength=bins ** len(arrays)).astype(float)
    p = counts[counts > 0] / counts.sum()
    return -np.sum(p * np.log2(p))

Mutual information: dependence without a model

Mutual information measures how much knowing one variable reduces uncertainty about another:

I(X; Y) = H(X) + H(Y) − H(X, Y) = H(Y) − H(Y | X)

It is zero if and only if X and Y are statistically independent, and unlike correlation it detects relationships of any shape. It is the right tool for a first-pass dependency screen: does this channel carry any information about revenue, linear or not?

def mutual_information(x, y, bins=8):
    xd, yd = discretize(x, bins), discretize(y, bins)
    return entropy(xd, bins=bins) + entropy(yd, bins=bins) - entropy(xd, yd, bins=bins)

But I(X; Y) == I(Y; X). Mutual information is symmetric. It tells you that two series are related, never which drives which. For direction we need the temporal generalization.

Transfer entropy: putting an arrow on the dependence

Transfer entropy (Schreiber, 2000) asks a sharper question: how much does the past of X reduce uncertainty about the next value of Y, beyond what Y’s own past already tells us? That conditioning on Y’s own history is the crucial move — it strips out Y’s autocorrelation so you measure genuinely new information arriving from X.

TE(X → Y) = Σ  p(y_{t+1}, yₜ^(k), xₜ^(l))
            · log₂  [ p(y_{t+1} | yₜ^(k), xₜ^(l)) / p(y_{t+1} | yₜ^(k)) ]

where yₜ^(k) = (yₜ, …, y_{t−k+1}) is the k-step history of Y and xₜ^(l) the l-step history of X. Equivalently, and more usefully for implementation, transfer entropy is a conditional mutual information:

TE(X → Y) = I( Y_{t+1} ; Xₜ^(l) | Yₜ^(k) )
          = H(Y_{t+1} | Yₜ^(k)) − H(Y_{t+1} | Yₜ^(k), Xₜ^(l))

The entropy-difference form is four counting operations:

def transfer_entropy(x, y, bins=4):
    """TE(X -> Y): directional information flow from X's past to Y's future."""
    xd, yd = discretize(x, bins), discretize(y, bins)
    yf, yp, xp = yd[1:], yd[:-1], xd[:-1]          # y_{t+1}, y_t, x_t  (k=l=1)
    #  TE = H(yf|yp) - H(yf|yp,xp)
    return ((entropy(yf, yp, bins=bins) - entropy(yp, bins=bins))
            - (entropy(yf, yp, xp, bins=bins) - entropy(yp, xp, bins=bins)))

The defining property: transfer entropy is asymmetric. TE(X → Y) ≠ TE(Y → X). The difference between the two directions is the net information flow, and its sign is the answer to "which way does the money flow":

net flow(X → Y) = TE(X → Y) − TE(Y → X)

Positive net flow from spend to revenue is the signature of a genuine causal channel. Negative net flow — information running from revenue to spend — is the signature of budget chasing sales.

The connection to Granger causality

If you have an econometrics background, transfer entropy should feel familiar: it is the formalization of Wiener-Granger causality in information-theoretic terms. Barnett, Barrett, and Seth (2009) proved that for jointly Gaussian variables, transfer entropy and Granger causality are equivalent — transfer entropy equals half the Granger F-statistic. So transfer entropy is not exotic; it is the model-free, nonlinear generalization of Granger causality. Where Granger causality assumes a linear vector-autoregression, transfer entropy drops the linearity and the distributional assumptions and measures the same directional predictability directly from the data.

Estimation in the real world

Two practical issues separate a toy implementation from a trustworthy one.

The estimator. The histogram/plug-in estimator above is intuitive but biased for continuous data and suffers badly from the curse of dimensionality as you add history length or conditioning variables. For production on continuous marketing series, the Kraskov–Stögbauer–Grassberger (KSG) estimator — a k-nearest-neighbor method — is the standard, because it avoids binning entirely and is far more sample-efficient. The libraries below implement it.

Significance. A nonzero transfer entropy estimate is not automatically meaningful — finite-sample noise produces positive values even for independent series. You test against a surrogate null: repeatedly shuffle (or block-permute / time-shift) the source series to destroy its temporal coupling while preserving its marginal distribution, recompute transfer entropy each time, and build an empirical null distribution. The effective transfer entropy subtracts the surrogate mean to debias the estimate, and a z-score or empirical p-value against the surrogate distribution gives significance:

def effective_transfer_entropy(x, y, bins=4, n_surrogates=200):
    te = transfer_entropy(x, y, bins)
    null = np.array([transfer_entropy(np.random.permutation(x), y, bins)
                     for _ in range(n_surrogates)])
    ete = te - null.mean()                       # debiased
    z   = (te - null.mean()) / (null.std() + 1e-12)
    return ete, z

A fully worked example (real computed outputs)

To see all of this fire at once, here is a 6,000-week simulation with three channels of deliberately different causal character, plus a latent demand process Z. All numbers below are actual program output.

Channel A genuinely drives revenue at a 1–2 week lag (the honest channel).
Channel B is set by a pacing rule that spends 70% of last week’s revenue — budget chases sales (reverse causation, zero true effect).
Channel C is driven by latent demand Z, which separately drives revenue — so C is correlated with revenue but has no direct effect (pure confounding).

Effective transfer entropy in both directions, with z-scores against a 200-sample surrogate null:

Channel A  (true cause)
   TE(spend  -> revenue)   ETE = +0.3820 bits   z = 397.0
   TE(revenue -> spend )   ETE = −0.0006 bits   z =  −0.6
   net flow = +0.3826      -> money flows spend -> revenue ✓

Channel B  (budget chases sales)
   TE(spend  -> revenue)   ETE = +0.0012 bits   z =   1.3
   TE(revenue -> spend )   ETE = +0.8748 bits   z = 914.8
   net flow = −0.8736      -> money flows revenue -> spend ✗ (reverse!)

Channel C  (demand-confounded)
   TE(spend  -> revenue)   ETE = +0.0499 bits   z =  47.8
   TE(revenue -> spend )   ETE = +0.0167 bits   z =  16.2
   net flow = +0.0332      -> looks causal... but isn't

Read what just happened. Channel A shows a large, highly significant flow from spend to revenue and essentially nothing backward — a textbook genuine driver. Channel B shows the flow reversed: 0.87 bits running from revenue to spend, net flow strongly negative. A correlation analysis would have reported a strong positive relationship and a flattering ROI; transfer entropy reveals the arrow points the wrong way. Channel C is the trap — it shows a significant-looking forward flow (z = 47.8) even though it has zero direct effect, because it shares the demand driver with revenue.

Conditional transfer entropy: blocking the confounder

Channel C is exactly the demand-confounding problem from part one, and transfer entropy has a direct fix. Conditional transfer entropy measures the flow from X to Y while conditioning on the past of a third variable Z, blocking any path that runs through Z:

CTE(X → Y | Z) = I( Y_{t+1} ; Xₜ | Yₜ , Zₜ )
               = H(Y_{t+1} | Yₜ, Zₜ) − H(Y_{t+1} | Yₜ, Xₜ, Zₜ)

def conditional_transfer_entropy(x, y, z, bins=3):
    """CTE(X -> Y | Z): flow from X to Y after blocking paths through Z."""
    xd, yd, zd = discretize(x, bins), discretize(y, bins), discretize(z, bins)
    yf, yp, xp, zp = yd[1:], yd[:-1], xd[:-1], zd[:-1]
    a = entropy(yf, yp, zp, bins=bins) - entropy(yp, zp, bins=bins)
    b = entropy(yf, yp, xp, zp, bins=bins) - entropy(yp, xp, zp, bins=bins)
    return a - b

Conditioning Channel C on the demand process Z:

TE(C -> revenue)        unconditioned  =  0.0471 bits
CTE(C -> revenue | Z)   conditioned    =  0.0076 bits

The apparent flow collapses by ~84% once demand is accounted for. The information was never coming from C — it was coming from the demand both shared. That collapse is a quantitative confounding test: if a channel’s transfer entropy to revenue mostly disappears after conditioning on a demand proxy, its standalone “contribution” is largely an artifact.

The honest caveat: this is Granger, not Pearl

Transfer entropy detects directional predictive information flow — Wiener-Granger causality. That is not the same as Pearl’s interventional causality, and conflating them is the field’s most common mistake. Three limits worth stating plainly:

Predictiveness is not intervention. Transfer entropy tells you X’s past helps predict Y’s future. It does not guarantee that setting X via do(X) changes Y. A channel that is a slightly-earlier proxy of an unobserved driver will show forward flow it does not deserve.
Conditioning only blocks what you observe. Conditional transfer entropy neutralizes a confounder if you measured it. Unobserved demand is still invisible. The Channel C fix worked because we had Z; in production you need a real demand proxy (search query volume, category trend, web sessions).
It is data-hungry and estimator-sensitive. Reliable transfer entropy — especially the conditional, multivariate form — needs long series and careful choice of history length and estimator. Histogram estimators degrade fast as dimensions grow; use KSG and test against surrogates.

The correct posture: transfer entropy is a discovery and screening layer, not a verdict. It generates and prioritizes causal hypotheses cheaply; experiments and structural causal models confirm them.

Where it fits: a pre-flight layer for your MMM

This is the part that makes it operationally valuable rather than merely elegant. Run transfer entropy before and alongside your marketing mix model, where it does four jobs no regression does:

Reverse-causation flagging. Compute net flow for every channel. Any channel where revenue → spend dominates (negative net flow) is a budget-pacing artifact — quarantine it or model the pacing explicitly before trusting its ROI. This is the diagnostic Channel B failed.
Channel screening. Channels with no significant directional flow to revenue (after surrogate testing) are carrying correlational noise. Letting them into a joint MMM lets them steal credit from real drivers via collinearity. Screen them first.
Lag discovery for adstock. Sweep the source history offset and find the lag at which transfer entropy to revenue peaks. That peak is data-driven prior information for your adstock window and delay parameter — better than guessing l_max.
Causal-graph skeleton. Pairwise and conditional transfer entropy across all channels and controls builds the skeleton of a causal DAG — which variables inform which — that seeds the structural model instead of hand-drawing it. This is precisely the bridge from screening into causal MMM.

That last point is where this connects to the rest of the stack. The transfer-entropy skeleton feeds a structural causal model, which is then calibrated against geo experiments — the triangulation that makes media measurement trustworthy.

Tooling

The estimators in this article are intentionally from-scratch and runnable with numpy alone, so the mechanics are transparent. For production:

IDTxl (Python) — the most complete option: KSG estimators, multivariate and conditional transfer entropy, built-in surrogate significance testing, and network inference. This is the one to reach for when building the causal skeleton.
JIDT (Java, with a Python bridge) — the reference Kraskov-estimator toolkit.
PyInform, pyEntropy — lighter-weight libraries for the core measures.
Papilon — my own framework exposes these directly in its core module (shannon_entropy, mutual_information, transfer_entropy, plus conditional/multivariate forms), wired into the causal-discovery pipeline so the information-theoretic skeleton flows straight into structural causal inference. The companion library PyCausalSim uses agent-based simulation to stress-test that recovered structure under interventions you cannot run in the real world.

The takeaway

Correlation tells you two series move together. Transfer entropy tells you which one is moving the other — and in media measurement, that arrow is the whole question. A symmetric statistic cannot distinguish a channel that generates revenue from one that merely follows it, and that single blind spot is responsible for an enormous share of overstated marketing ROI.

Before you fit a response curve or optimize a budget, spend an afternoon measuring the direction of the money. Flag the channels where revenue leads spend. Screen out the ones carrying no directional signal. Find your adstock lags in the data instead of assuming them. Then hand a clean, direction-aware causal skeleton to the MMM. The model you build on top of that foundation will be measuring effects that actually point the right way.

In part one we optimized the allocation. Here we made sure the curves we are optimizing reflect causes, not echoes. Next in the series: turning that directional skeleton into a full structural causal model and calibrating it against geo experiments.

All estimators are runnable as written with numpy; the worked-example figures are actual program output from a 6,000-period simulation. Transfer entropy detects Granger-style directional information flow, which is necessary but not sufficient for interventional causality — pair it with structural causal models and experiments before acting on it.

Brian Curry is a Kansas City–based cognitive systems architect and applied AI researcher working at the intersection of AI architecture, knowledge engineering, and intelligent-system design. He is the founder of Vector1 Research, an applied AI lab publishing on agent systems, AI evaluation, cognitive architecture, and the macroeconomics of AI-driven transformation. He has led applied AI and data initiatives across major enterprises including Koch Industries, Tractor Supply, Vail Resorts, Hallmark, Garmin, AT&T, and McClatchy, and founded KC AI Lab, a 1,500-member AI/ML community. His open-source work includes Papilon (causal inference and counterfactual simulation), PyCausalSim (causal discovery via agent-based simulation), MeaningFlow (semantic modeling and ontology), and Memory-Node Encapsulation (episodic memory for agentic systems). Contact: brian at vector1.ai · LinkedIn