When You Can’t See the Confounder: Instrumental Variables (Part 4)
This is Part 4 of a series on causal inference. Parts 1–3 covered the core workflow: define the causal question, draw the DAG, and estimate the effect using propensity scores, matching, and weighting. That workflow assumes you can observe and measure every confounder. This article is about what happens when you can’t.
A New Problem at the Same Company
You’re still at the same e-commerce company. The marketing team ran another email campaign, and this time the question is different: did customers who engaged with the email (opened it, clicked through) actually spend more as a result?
This is a harder question than “did receiving the email affect spend?” because engagement is a choice. Customers who open promotional emails are different from those who don’t — they’re more interested in buying, more engaged with the brand, more likely to be in “shopping mode.” That underlying purchase intent drives both engagement and spending, and we can’t measure it directly.
In Parts 1–3, we had loyalty score as an observable confounder and could adjust for it. Here, the main confounder — purchase intent — lives inside the customer’s head. No column in your database captures it. Loyalty score is correlated with it but doesn’t fully capture it.
This is the situation that breaks the methods from Part 3. You can’t match on what you can’t see. You can’t weight by what you can’t measure. The backdoor criterion from Part 2 says: if there’s an unblockable backdoor path, you need a different strategy.
That strategy is called instrumental variables.
The Lucky Break: A Server Outage
Here’s what happened. During the campaign, your company’s email delivery system experienced a server outage that affected roughly 40% of customers. These customers received their emails hours late — well after the peak engagement window. The outage was caused by a hardware failure, completely unrelated to any customer characteristics.
When the data team investigates, they find that customers affected by the outage were much less likely to engage with the email. Not because they were different kinds of customers, but because a delayed email is easy to miss — it arrives buried under newer messages.
This outage is an instrument. It gives you a back door to estimate the causal effect of engagement on spending, even though you can’t observe purchase intent.
The Logic of Instrumental Variables
An instrument is a variable that affects the treatment but has no direct effect on the outcome — it only affects the outcome through its effect on the treatment.
Three conditions must hold:
1. Relevance. The instrument must actually affect treatment. The outage must reduce email engagement. (Testable.)
2. Independence (as-if random). The instrument must be unrelated to the unmeasured confounder. The outage must be unrelated to purchase intent. (Plausible if the outage was truly random hardware failure.)
3. Exclusion restriction. The instrument must not directly affect the outcome. The outage must not affect spending through any channel other than email engagement. (Untestable — requires domain knowledge and argument.)
The exclusion restriction is the critical assumption. If the outage somehow directly affected spending — maybe delayed emails caused some customers to lose trust in the brand and spend less, independent of whether they engaged — the instrument is invalid. You need to argue, based on context and domain knowledge, that this didn’t happen.
The DAG makes the structure clear. Purchase intent (dashed, unobserved) confounds the relationship between engagement and spending — that’s the backdoor path we can’t block with adjustment. But the server outage only connects to spending through engagement. It’s our way in.
Building the Scenario
Let’s simulate this dataset. As always, simulation lets us know the true effect and verify our methods against it.
import numpy as np
import pandas as pd
np.random.seed(42)
n = 10_000
# Purchase intent — the UNOBSERVED confounder
# High-intent customers engage more AND spend more
intent = np.random.normal(50, 15, n)
# Loyalty score — observed, but only partially captures intent
loyalty = 0.6 * intent + 0.4 * np.random.normal(50, 15, n)
# Server outage: randomly affected ~40% of customers
# This is our INSTRUMENT
outage = np.random.binomial(1, 0.4, n) # 1 = affected by outage
# Email engagement (the treatment)
# Depends on: intent (high-intent customers engage more)
# + outage (delays reduce engagement)
engage_propensity = 1 / (1 + np.exp(-(intent - 50) / 12 + 1.5 * outage))
email_engaged = np.random.binomial(1, engage_propensity)
# TRUE causal effect of email engagement on spending
TRUE_EFFECT = 15
# Spending depends on intent (strongly) and engagement (causal effect)
spend = 20 + 1.2 * intent + TRUE_EFFECT * email_engaged + np.random.normal(0, 20, n)
spend = np.clip(spend, 0, None)
df = pd.DataFrame({
"intent": intent, # we pretend we can't see this
"loyalty": loyalty, # observed but incomplete
"outage": outage, # our instrument
"email_engaged": email_engaged, # treatment
"spend": spend, # outcome
})
print(f"Customers: {n:,}")
print(f"Outage affected: {outage.sum():,} ({outage.mean():.1%})")
print(f"Engaged with email: {email_engaged.sum():,} ({email_engaged.mean():.1%})")
print(f"True causal effect: ${TRUE_EFFECT}")
Customers: 10,000
Outage affected: 3,984 (39.8%)
Engaged with email: 4,005 (40.1%)
True causal effect: $15Notice the key structural difference from Parts 1–3: intent drives both engagement and spending, but we’ve labeled it as unobservable. In practice, you wouldn’t even have this column. We include it in the simulation so we can verify that our instrument recovers the truth.
Why the Standard Methods Fail
Before reaching for IV, let’s confirm that the Part 3 methods can’t help us here.
# Naive comparison
naive = df[df.email_engaged == 1].spend.mean() - df[df.email_engaged == 0].spend.mean()
print(f"Naive: ${naive:.2f}")
# OLS controlling for loyalty (the best we can observe)
from numpy.linalg import lstsq
X_ols = np.column_stack([np.ones(n), email_engaged, loyalty])
beta_ols, _, _, _ = lstsq(X_ols, spend, rcond=None)
print(f"OLS controlling for loyalty: ${beta_ols[1]:.2f}")
print(f"True effect: ${TRUE_EFFECT}")
Naive: $30.62
OLS controlling for loyalty: $19.97
True effect: $15The naive estimate doubles the true effect. Controlling for loyalty improves things — from $31 to $20 — but it’s still off by $5 because loyalty doesn’t fully capture purchase intent. The residual confounding persists, and no amount of propensity score adjustment fixes it when the confounder is unobserved.
Checking the Instrument
Before using the outage as an instrument, we need to verify that it actually works as one.
Relevance: Does the outage affect engagement?
engage_by_outage = df.groupby('outage')['email_engaged'].mean()
print("Engagement rates:")
print(f" No outage: {engage_by_outage[0]:.1%}")
print(f" Outage: {engage_by_outage[1]:.1%}")
print(f" Difference: {(engage_by_outage[0] - engage_by_outage[1]):.1%}")
Engagement rates:
No outage: 50.6%
Outage: 24.1%
Difference: 26.4%
The outage cuts engagement by more than half. This is a strong first stage — the instrument clearly moves the treatment.
The right panel confirms that the outage is uncorrelated with customer characteristics. Loyalty distributions look identical across outage groups — exactly what we’d expect from a random hardware failure.
Independence: Is the outage random?
r = np.corrcoef(outage, intent)[0, 1]
print(f"Correlation between outage and intent: {r:.4f}")
# Also check loyalty, since that's what we can actually observe
r_loyalty = np.corrcoef(outage, loyalty)[0, 1]
print(f"Correlation between outage and loyalty: {r_loyalty:.4f}")
Correlation between outage and intent: -0.0057
Correlation between outage and loyalty: -0.0045Essentially zero. The outage was random, as expected.
Exclusion restriction: Does the outage affect spending only through engagement?
This one we can’t test statistically. We have to argue it. The outage delayed email delivery by a few hours. It’s plausible that this delay only matters because it reduces the chance of engagement — a buried email gets fewer opens. It’s hard to see how a delayed email would directly change spending behavior independently of whether the customer engaged with it.
Is this argument ironclad? No. Maybe some customers noticed the delay and it eroded their trust. Maybe the delay caused them to purchase from a competitor instead. In a real project, you’d discuss these threats with your team and decide whether the exclusion restriction is defensible. For our purposes, we’ll proceed — but with the caveat that this assumption is doing real work.
The Wald Estimator: IV in Its Simplest Form
With a binary instrument (outage yes/no) and a binary treatment (engaged yes/no), the IV estimate takes a beautifully simple form called the Wald estimator:
IV = (Mean outcome for Z=0 − Mean outcome for Z=1) / (Mean treatment for Z=0 − Mean treatment for Z=1)
The numerator is the reduced form: how much does the instrument move the outcome? The denominator is the first stage: how much does the instrument move the treatment?
The ratio gives you: for each unit of treatment change caused by the instrument, how much does the outcome change?
spend_by_z = df.groupby('outage')['spend'].mean()
engage_by_z = df.groupby('outage')['email_engaged'].mean()
numerator = spend_by_z[0] - spend_by_z[1] # reduced form
denominator = engage_by_z[0] - engage_by_z[1] # first stage
wald_iv = numerator / denominator
print(f"Reduced form (spend difference by outage): ${numerator:.2f}")
print(f"First stage (engagement difference by outage): {denominator:.3f}")
print(f"Wald IV estimate: ${numerator:.2f} / {denominator:.3f} = ${wald_iv:.2f}")
print(f"True effect: ${TRUE_EFFECT}")
Reduced form (spend difference by outage): $4.34
First stage (engagement difference by outage): 0.264
Wald IV estimate: $4.34 / 0.264 = $16.43
True effect: $15$16.43 — within about $1.50 of the true $15 effect. Compare that to the naive estimate of $30.62 or even the OLS-with-loyalty estimate of $19.97.
The intuition: the outage reduced engagement by 26.4 percentage points and reduced spending by $4.34. If the only way the outage affects spending is through engagement, then each percentage point of engagement is “worth” $4.34 / 0.264 = $16.43 in spending.
Two-Stage Least Squares: The General Version
The Wald estimator works with a single binary instrument. For continuous instruments or when you want to include control variables, the standard approach is Two-Stage Least Squares (2SLS).
Stage 1: Regress the treatment on the instrument (and any controls). Save the predicted values.
Stage 2: Regress the outcome on the predicted treatment (and the same controls).
The key insight: the predicted treatment from Stage 1 only contains variation in engagement that was caused by the outage. All the variation driven by purchase intent has been stripped out. So Stage 2’s coefficient is free of confounding.
# Stage 1: Predict engagement from outage
X_stage1 = np.column_stack([np.ones(n), outage])
beta_stage1, _, _, _ = lstsq(X_stage1, email_engaged, rcond=None)
email_hat = X_stage1 @ beta_stage1 # predicted engagement
print("Stage 1: Email_hat = {:.3f} + {:.3f} * Outage".format(*beta_stage1))
print(f" (Outage reduces predicted engagement by {abs(beta_stage1[1]):.1%})")
# Stage 2: Regress spend on predicted engagement
X_stage2 = np.column_stack([np.ones(n), email_hat])
beta_stage2, _, _, _ = lstsq(X_stage2, spend, rcond=None)
print(f"\nStage 2 coefficient: ${beta_stage2[1]:.2f}")
print(f"True effect: ${TRUE_EFFECT}")
Stage 1: Email_hat = 0.506 + -0.264 * Outage
(Outage reduces predicted engagement by 26.4%)
Stage 2 coefficient: $16.43
True effect: $15Same answer as the Wald estimator — which makes sense, because with a single binary instrument and no controls, 2SLS reduces to the Wald formula.
Now let’s add loyalty as a control to improve precision:
# 2SLS with loyalty as a control variable
X_stage1_ctrl = np.column_stack([np.ones(n), outage, loyalty])
beta_s1, _, _, _ = lstsq(X_stage1_ctrl, email_engaged, rcond=None)
email_hat_ctrl = X_stage1_ctrl @ beta_s1
X_stage2_ctrl = np.column_stack([np.ones(n), email_hat_ctrl, loyalty])
beta_s2, _, _, _ = lstsq(X_stage2_ctrl, spend, rcond=None)
print(f"2SLS with loyalty control: ${beta_s2[1]:.2f}")
print(f"2SLS without controls: ${beta_stage2[1]:.2f}")
print(f"True effect: ${TRUE_EFFECT}")
2SLS with loyalty control: $16.08
2SLS without controls: $16.43
True effect: $15Adding loyalty as a control nudges the estimate slightly closer to the truth. Loyalty isn’t the full confounder (intent is), but it captures part of it, so including it reduces residual variation and improves precision.
The Weak Instrument Problem
Everything above worked because our instrument was strong — the outage cut engagement by 26 percentage points. But what if the instrument barely moved the treatment?
This is the weak instrument problem, and it’s one of the biggest practical concerns with IV. When the first stage is weak, the denominator of the Wald estimator is close to zero, and the IV estimate becomes wildly unstable.
The standard diagnostic is the first-stage F-statistic. A rule of thumb from Staiger and Stock (1997): if the F-statistic is below 10, your instrument is too weak and the IV estimate is unreliable.
# F-statistic for our instrument
from scipy import stats
# Restricted model (no instrument)
X_restricted = np.column_stack([np.ones(n)])
beta_r, _, _, _ = lstsq(X_restricted, email_engaged, rcond=None)
SSR_restricted = np.sum((email_engaged - X_restricted @ beta_r) ** 2)
# Unrestricted model (with instrument)
SSR_unrestricted = np.sum((email_engaged - X_stage1 @ beta_stage1) ** 2)
F_stat = ((SSR_restricted - SSR_unrestricted) / 1) / (SSR_unrestricted / (n - 2))
print(f"First-stage F-statistic: {F_stat:.1f}")
print(f"Rule of thumb: F > 10 means the instrument is strong enough")
First-stage F-statistic: 749.8
Rule of thumb: F > 10 means the instrument is strong enoughOur F-statistic is 750 — extremely strong. But let’s see what happens when the instrument is weak:
The left panel shows bootstrap IV estimates with our strong instrument (F=750): they cluster tightly around the true effect. The right panel simulates a weak instrument (F≈1): the estimates are scattered across a huge range, including negative values and estimates above $100. A weak instrument doesn’t just give you a noisy answer — it gives you a misleading one, because weak-instrument bias pushes the IV estimate toward the biased OLS estimate.
The takeaway: before reporting an IV estimate, always check the first-stage F-statistic. If it’s below 10, stop and find a better instrument.
What IV Actually Estimates: The LATE
There’s a subtlety about IV that’s important to flag. IV doesn’t estimate the Average Treatment Effect (ATE) for the whole population. It estimates the Local Average Treatment Effect (LATE) — the causal effect for compliers.
Who are the compliers? They’re the people whose treatment status was changed by the instrument. In our case, compliers are customers who would have engaged with the email if there was no outage but didn’t engage because of the outage. The outage moved them from “engaged” to “not engaged.”
There are three other groups the instrument doesn’t help with:
- Always-takers: Would engage regardless of the outage. The instrument doesn’t change their behavior.
- Never-takers: Wouldn’t engage regardless. The instrument doesn’t change their behavior either.
- Defiers: Would engage only because of the outage. We typically assume these don’t exist (the monotonicity assumption).
The LATE is useful when the compliers are a policy-relevant group. In our case, compliers are customers on the margin of engagement — the ones who engage when it’s easy but don’t when there’s friction. If the marketing team is considering investments to reduce email delivery friction (better infrastructure, send-time optimization), the LATE is exactly the right quantity: it tells you the value of engagement for the customers who would be moved by such investments.
But if you want to know the effect of engagement on everyone, including die-hard fans who always engage and disengaged customers who never open emails, IV doesn’t give you that directly.
When Can You Use IV? Finding Instruments in the Wild
The server outage was convenient. In practice, finding good instruments is the hard part. Here are some common sources in marketing and business:
Geographic or temporal variation in exposure. Different regions received the campaign at different times due to rollout schedules. If the timing was driven by logistical constraints (not customer characteristics), it can serve as an instrument.
Technical failures and glitches. Our server outage example. Also: email deliverability issues, app crashes, A/B test assignments that were supposed to be random.
Policy changes. A new regulation changed who was eligible for a promotion. Eligibility is the instrument for receiving the promotion.
Physical distance. Customers closer to a store are more likely to visit. Distance is an instrument for store visits when estimating the effect of in-store shopping on spending.
The hard part isn’t finding something that correlates with treatment — it’s arguing convincingly that the exclusion restriction holds. Every instrument you propose, ask: “Is there any way this could affect the outcome other than through the treatment?” If you can think of a plausible channel, the instrument might be invalid.
Where We Stand
We’ve now added a critical tool to our kit. When the confounder is unobserved and the backdoor criterion can’t be satisfied through adjustment:
Find an instrument — a variable that moves the treatment but doesn’t directly affect the outcome.
Check the first stage — make sure the instrument is strong (F > 10).
Estimate via 2SLS or the Wald formula — and remember you’re estimating the LATE, not the ATE.
Argue the exclusion restriction — this is where the real work happens, and it’s a domain knowledge argument, not a statistical test.
For our marketing problem, the IV estimate of $16.08 is much closer to the true $15 effect than either the naive ($30.62) or the OLS-with-loyalty ($19.97) estimates. The server outage gave us a window into the causal effect that no amount of observable data could have provided.
What’s Coming Next
Part 5: Difference-in-Differences — The company rolls out a new loyalty program in some regions but not others. With before-and-after data for both groups, we’ll use the parallel trends assumption to estimate the program’s causal effect — no instrument required.
Part 6: Regression Discontinuity — Customers who cross a spending threshold get automatically enrolled in a premium tier. The sharp cutoff creates a natural experiment we can exploit.
All code from this series is available as a notebook on GitHub. Built with Python 3.11, numpy, pandas, scipy, and matplotlib.
Part 5 drops next week. Different method, same company, same question: did this thing actually work?
Brian Curry is a Kansas City–based data scientist, AI researcher, and founder of Vector1 Research, an independent lab advancing marketing economics, causal inference, and cognitive AI systems. He has led data and analytics initiatives at enterprises from $4B to $125B, including Koch Industries, Tractor Supply, Vail Resorts, Hallmark, Garmin, and AT&T. He is the creator of MeaningFlow (semantic content modeling), Papilon (causal inference and complex systems), PyCausalSim (causal discovery through simulation), and the Memory-Node Encapsulation (MNE) architecture for cognitive AI.
