When Your Marketing Agent Optimizes the Wrong Number

Autonomous optimizers are only as good as the signal they chase. Here is an analytical and simulated demonstration that an agent trained on correlational measurement doesn’t just misallocate budget — it does so in a way more data cannot fix.

Two things are happening in marketing measurement at the same time, and the interaction between them is getting almost no attention.

The first is that marketing mix modeling (MMM) is back. After a decade in which user-level attribution was treated as ground truth, the erosion of cookies and identifiers, tighter privacy rules, and budget fragmentation across walled gardens have pushed teams back toward aggregate, privacy-durable measurement. The catch is that adoption has outrun usefulness: in recent industry surveys, fewer than a third of marketers say their organization is effective at turning MMM output into actual decisions.

The second is that execution is going agentic. The product pitch across the category has converged on a single sentence — humans set goals, systems act — with budget optimization recast as something an autonomous agent does continuously rather than something an analyst does quarterly.

Put those two trends in one room and a problem appears that the current enthusiasm glosses over. An autonomous optimizer is only as good as the signal it optimizes against. Point an agent at a number that reflects correlation rather than causation, and it will not merely make one bad decision. It will make that decision thousands of times, at machine speed, with no analyst in the loop to apply the skeptical discount that experienced marketers apply by reflex.

This article makes that claim precise and then tests it. We build a ground-truth market, generate the kind of observational data a real advertiser would have, and let two agents loose on it: one that optimizes against a correlational estimate of channel performance, and one that optimizes against a causally identified estimate. We measure how much money each leaves on the table. The headline: the correlational agent loses about 80% of the achievable incremental revenue, and — this is the part that matters — giving it more data does not help, because its error is bias, not variance.

All code is included at the end and runs in a few minutes with nothing but NumPy and SciPy. No MMM library, no proprietary data.

The question, stated precisely

Suppose an advertiser spends a fixed weekly budget B across K channels. An autonomous agent observes historical data, estimates each channel’s response to spend, and reallocates the budget to maximize predicted incremental revenue. We want to know: how much does it matter whether the agent’s response estimates are causal or merely correlational?

To answer that without hand-waving, we need three things — a model of how spend actually turns into revenue, a model of where the data comes from, and two estimators that differ in exactly one respect.

A model of marketing response

We use the standard functional forms from the Bayesian MMM literature (Jin et al., 2017): geometric adstock to capture carryover, and a Hill curve to capture saturation.

Adstock says this week’s effective exposure carries over from last week with decay θ:

The saturating response of revenue to effective exposure is a Hill function with ceiling β, half-saturation point κ, and shape α:

Total weekly sales are a baseline driven by latent demand u, plus the sum of channel contributions, plus noise:

Here u is the part of demand that marketing does not create — seasonality, brand equity, macro conditions, a viral moment. It is the villain of the piece, because it moves sales on its own and, as we’ll see, it moves spend.

In code, the ground truth is exactly this and nothing more:

def adstock(spend, theta):
    a, prev = np.zeros_like(spend, float), 0.0
    for t in range(len(spend)):
        prev = spend[t] + theta * prev
        a[t] = prev
    return a

def hill(a, beta, kappa, alpha):
    a = np.maximum(a, 1e-9)
    return beta * a**alpha / (a**alpha + kappa**alpha)

# sales_t = mu + gamma * u_t + sum_i hill(adstock(spend_i)) + noise

Where the confounding comes from

Real spend is not assigned at random. It is set by an incumbent policy — increasingly an autobidder — and that policy chases demand. When demand is high, automated bidding pours money into the channels that are best at intercepting people who are already converting: brand search, retargeting, app re-engagement. We model the incumbent policy as coupling spend to the latent demand state, with a coupling strength ρ that is large for demand-harvesting channels and near zero for demand-generating ones:

This single line is where causality goes to die. Because brand-search spend rises exactly when demand (and therefore baseline sales) is high, the two are strongly correlated in the historical record — even though the spend caused almost none of those sales. This is not a contrived worry: it is precisely the mechanism behind the eBay field experiments of Blake, Nosko, and Tadelis (2015), who found that brand-keyword search ads had no measurable short-term effect despite looking highly productive in observational data, and behind the 15-experiment Facebook study of Gordon et al. (2019), who found that observational methods routinely fail to recover the experimental truth even with rich user-level features.

In our simulation, two channels (brand search, retargeting) are demand-harvesting with high coupling; two (video, prospecting display) are demand-generating with low coupling. By construction, the demand-generating channels have the higher true incremental return — they are the ones actually worth funding.

Two estimators that differ in one thing

Both estimators fit the same adstock-and-Hill model to the same data, with adstock decay and Hill shape fixed at their true values. They differ in exactly one respect:

The correlational estimator regresses sales on adstocked spend and an intercept. It does not account for the demand state — the analyst-out-of-the-loop case.
The causal estimator conditions on the demand state. This stands in for the information a proper causal design recovers: geo holdouts, incrementality experiments, or back-door adjustment on the confounder (Pearl, 2009). We are not claiming you get u for free; we are modeling what successful identification buys you.

The bias is not subtle, and it is not random. Omitting a confounder that drives both spend and sales produces classic omitted-variable bias, and its sign is determined by the coupling:

For any demand-harvesting channel (ρ > 0), the bias term is positive: the correlational estimator systematically overstates the marginal return of exactly the channels that are merely present for the sale rather than causing it. More data shrinks the variance of this estimate, but not the bias — it converges confidently to the wrong number.

Both estimators are regularized with weakly-informative priors on the saturation parameters, as is standard in Bayesian MMM to handle the well-known β/κ identifiability problem (Jin et al., 2017). The priors are generic — set from the spend scale, never from the true parameters.

The agent: closed-loop allocation

Given its estimated response curves, the agent solves a constrained optimization — put the budget where estimated marginal returns are highest, until they equalize (the standard interior optimum):

def optimize_allocation(beta, kappa, alpha, theta, B):
    neg_val = lambda s: -sum(hill(s_i/(1-th), b, k, al)
                             for s_i, b, k, al, th in zip(s, beta, kappa, alpha, theta))
    cons = [{"type": "eq", "fun": lambda s: s.sum() - B}]
    bnds = [(0, B)] * len(beta)
    # multistart SLSQP; return best feasible allocation

Then we score honestly. We take whatever allocation the agent chose and evaluate it against the true response model — the agent’s allocation, reality’s revenue. Regret is the gap between that and the oracle allocation a perfectly-informed planner would pick:

The correlational agent believes it is optimal. We grade it on the world, not on its own beliefs.

Results

The setup: four channels, a fixed weekly budget equal to the historical total, three years of weekly data (T = 156). Results are reproducible from the code below.

First, what each estimator thinks each channel is worth. The correlational estimator inflates the marginal ROAS of brand search by roughly 22× and retargeting by roughly 17×, while collapsing the genuinely effective video and prospecting channels toward zero. The causal estimator tracks the truth.

Estimated marginal ROAS by channel (log scale). The correlational estimator overstates exactly the demand-harvesting channels and understates the demand-generating ones. Multipliers show correlational / true.

Second, what each agent does with that belief. The oracle spreads budget toward the high-incremental video and prospecting channels. The causal agent broadly reproduces that. The correlational agent does the opposite — it pours the entire budget into the two demand-harvesting channels and starves the channels that actually move sales.

Budget allocation. The correlational agent funds the channels that merely correlate with high-demand periods.

The bottom line in dollars: against an oracle-achievable incremental revenue of about 210 units, the causal agent realizes ~174 (17% regret) while the correlational agent realizes ~37 (82% regret). Same model class, same data, same optimizer — the only difference is whether confounding was handled.

Third, generality. Regret grows with the strength of the confounding for the correlational agent, while the causal agent stays low across the range.

Budget regret vs. demand–spend coupling, mean ± SD over repetitions.

Fourth — the result that should change how people deploy these systems. As you give the agents more historical data, the causal agent’s regret falls steadily toward single digits. The correlational agent’s regret does not move. It sits at roughly 75–82% whether it has one year of data or ten, because its problem is a biased target, not an under-powered one.

Budget regret vs. training data (log scale). More data is not the fix.

That last figure is the whole argument in one picture. The industry’s instinct when an autonomous system underperforms is to feed it more data and more compute. If the measurement layer is correlational, that instinct makes the agent more confidently wrong, faster.

Why this matters for agentic marketing

In the human-in-the-loop era, the gap between correlation and causation was patched informally. An experienced analyst receiving a model that loved brand search would quietly discount it — they know brand search harvests demand created elsewhere. That discount lived in their head, and it never had to be written down.

Autonomous optimization removes the head that held it. An agent inherits structure, not intuition. So the causal correction that used to be implicit now has to be made explicit — encoded into the signal the agent optimizes against. That correction is a distinct layer, sitting between measurement and execution, and it is the layer almost no one is naming:

Execution — the agent that bids, budgets, launches, pauses.
Causal identification — the layer that guarantees the numbers handed to execution represent incremental, counterfactual impact.
Measurement — the model that produces those numbers.

The components of a real causal layer are not exotic; they are the established toolkit. Geo-based holdouts and incrementality experiments to anchor estimates in randomized variation. Geo-level hierarchical modeling to tighten estimates and, as Sun et al. (2017) put it, protect against unsound reallocation. Explicit confounder adjustment via the back-door criterion (Pearl, 2009) when experiments aren’t available. What’s new is not the science but the requirement: in a closed loop with delegated budget authority, this is no longer optional hygiene. It is the thing standing between an autonomous optimizer and an expensive, self-reinforcing mistake.

What to do about it

A practical sequencing rule falls out of the analysis:

Do not hand budget authority to an autonomous optimizer until its objective has been validated causally. Anchor the response estimates in at least one real experiment per major channel.
Treat any channel that resists incrementality testing with suspicion, not deference. The channels that look most efficient in observational data — brand search, retargeting — are exactly the ones most likely to be demand-harvesting.
Stop expecting scale to fix bias. If an autonomous system underperforms a holdout, more data and more compute will not close a causal gap.
Start narrow. One channel, one causally-instrumented objective, widen only as identification proves out.

Limitations

This is a simulation, and an idealized one. The causal estimator is handed the information that identification would recover, rather than having to run the experiments itself; in practice that identification is costly and imperfect, so real-world causal estimates will sit somewhere between the two agents shown here — which only strengthens the directional conclusion. We fix adstock and Hill shape parameters to isolate the confounding channel, hold the budget level constant, and assume a stationary demand process; relaxing any of these adds variance but does not remove the bias mechanism. The point is not the exact regret number. It is the structure: confounded measurement produces sign-definite bias, autonomous optimization amplifies it, and data volume does not touch it.

Reproduce it

The full, self-contained script — ground-truth model, data generation, both estimators, the allocation agent, and all four experiments — is here:

Reach out for full code.

It runs in a few minutes on a laptop and is seeded for exact reproducibility.

References

Blake, T., Nosko, C., & Tadelis, S. (2015). Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment. Econometrica, 83(1), 155–174.
Gordon, B. R., Zettelmeyer, F., Bhargava, N., & Chapsky, D. (2019). A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook. Marketing Science, 38(2), 193–225.
Jin, Y., Wang, Y., Sun, Y., Chan, D., & Koehler, J. (2017). Bayesian Methods for Media Mix Modeling with Carryover and Shape Effects. Google Inc.
Sun, Y., Wang, Y., Jin, Y., Chan, D., & Koehler, J. (2017). Geo-level Bayesian Hierarchical Media Mix Modeling. Google Inc.
Lewis, R. A., & Rao, J. M. (2015). The Unfavorable Economics of Measuring the Returns to Advertising. Quarterly Journal of Economics, 130(4).
Chan, D., & Perry, M. (2017). Challenges and Opportunities in Media Mix Modeling. Google Inc.
Pearl, J. (2009). Causality: Models, Reasoning, and Inference (2nd ed.). Cambridge University Press.
Imbens, G. W., & Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences. Cambridge University Press.
Angrist, J. D., & Pischke, J.-S. (2009). Mostly Harmless Econometrics. Princeton University Press.
Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction (2nd ed.). MIT Press.

Written at Vector1, an independent research lab working on causal inference, marketing science, and autonomous systems.

About the Author

Brian Curry is a Kansas City-based data scientist and applied AI researcher with two decades of enterprise experience across retail, agriculture, energy, hospitality, telecommunications, and media. He is the founder of Vector1 Research, an independent lab focused on applied AI, causal inference, agentic systems, and the economics of intelligent organizations. His work centers on building systems that surface the causal and structural drivers of performance — bridging advanced analytics, machine learning, and executive decision-making. He writes regularly on Medium about applied AI, knowledge engineering, decision science, and the engineering disciplines that make machine learning actually work in production.