The Calculus of When: Media Spend as an Optimal Control Problem

Part three of a series on digital media optimization. Static allocation answers “where to spend.” This answers “when.” Treating advertising goodwill as a capital stock and spend as a control trajectory turns budget planning into an optimal control problem — and yields a single equation that governs how much goodwill to hold, why fast-decaying channels are expensive to rent, and the first-principles reason flighting beats even spend. With full derivations and runnable code.

The dimension the first two parts left out

Part one optimized where the money goes — a constrained split across channels. Part two established which way causality runs — transfer entropy to separate genuine drivers from demand-chasers. Both treated spend as a knob you set: a static allocation, solved once.

But spend is not a knob. It is a trajectory through time, and the moment you take adstock seriously, the static framing breaks down. Adstock means today’s dollar produces revenue not just today but across many future weeks — so the optimal spend this week depends on what you will spend next week, and the week after, all the way to the planning horizon. Set this week’s budget without solving the whole path and you are optimizing a single variable in a problem that is fundamentally about a function of time.

The right mathematical object is optimal control: choose a spend trajectory u(t) to maximize discounted profit, subject to the dynamics of how advertising goodwill accumulates and decays. This is not a new idea — it is one of the oldest rigorous treatments of advertising, predating modern MMM by decades — but it has almost vanished from contemporary measurement writing, which is a shame, because it answers questions static allocation cannot even pose: When should a channel go dark? How much goodwill is it optimal to hold? Why does pulsing sometimes crush even spend, and sometimes not?

This article rebuilds that machinery from the ground up and grounds every result in real computed numbers.

Adstock is a dynamical system

Start by writing adstock in continuous time. Let aₖ(t) be channel k's goodwill stock — the accumulated, still-active effect of past advertising. It grows with spend and leaks away through decay:

daₖ/dt = − θₖ · aₖ(t) + uₖ(t)

Here uₖ(t) is the spend rate (the control) and θₖ > 0 is the continuous decay rate. This is exactly the geometric adstock from part one, written as a differential equation: the discrete carryover λ corresponds to e^(−θ) per period. The stock integrates spend and forgets it exponentially.

This formulation is the Nerlove–Arrow goodwill model (1962), and its cousin the Vidale–Wolfe model (1957) — advertising-as-capital and advertising-as-control are ideas from operations research’s golden age. The reframing they introduced is the key one: advertising goodwill is a capital stock. Spend is investment. Decay is depreciation. And once you see goodwill as capital, the entire neoclassical theory of optimal investment becomes available — including, as we will see, its central result.

The control problem

A planner chooses the spend trajectory u(t) = (u₁, …, u_K) over a horizon [0, T] to maximize discounted profit — revenue from goodwill minus the cost of spend:

maximize   J = ∫₀ᵀ e^(−ρt) [ Σₖ gₖ(aₖ(t)) − Σₖ uₖ(t) ] dt
   u(·)

subject to   daₖ/dt = − θₖ aₖ + uₖ        (goodwill dynamics)
             0 ≤ uₖ(t) ≤ uₖᵐᵃˣ            (spend-rate bounds)
             aₖ(0) = aₖ⁰                   (starting goodwill)

gₖ(·) is the saturating goodwill→revenue response (a Hill curve, from part one), ρ is the discount rate, and price is normalized so a unit of spend costs one unit. An optional total-budget constraint ∫₀ᵀ Σₖ uₖ dt ≤ B can be added; we will use it to derive the flighting result.

Notice what makes this hard: the control u appears in the constraint (the dynamics), and the state a it drives appears in the objective with a lag structure stretching across the whole horizon. You cannot solve it pointwise. You need the calculus of variations — specifically, Pontryagin.

Pontryagin’s Maximum Principle

Introduce a costate (adjoint) variable λₖ(t) for each goodwill stock — the shadow price of goodwill — and form the current-value Hamiltonian:

Ĥ = Σₖ [ gₖ(aₖ) − uₖ ] + Σₖ λₖ ( − θₖ aₖ + uₖ )

Pontryagin’s Maximum Principle gives three conditions the optimal solution must satisfy.

1 — Costate (adjoint) dynamics. The shadow price evolves by

dλₖ/dt = ρ λₖ − ∂Ĥ/∂aₖ = (ρ + θₖ) λₖ − gₖ′(aₖ)

2 — Transversality. With no salvage value of goodwill at the end of the horizon,

λₖ(T) = 0

3 — Maximization of the Hamiltonian. Because Ĥ is linear in each control uₖ, the optimal spend is determined by the sign of a switching function σₖ = ∂Ĥ/∂uₖ = λₖ − 1:

uₖ(t) =  uₖᵐᵃˣ    if λₖ > 1     (shadow value of goodwill exceeds its unit cost → build)
         0        if λₖ < 1     (goodwill not worth its price → stay dark)
         singular if λₖ = 1     (hold the stock — maintenance spend)

This is a bang–singular–bang structure, and it already tells the whole qualitative story. The costate λₖ is the present value of all future revenue that one more unit of goodwill now will generate along the optimal path:

λₖ(t) = ∫ₜ^∞ e^(−(ρ+θₖ)(s−t)) · gₖ′(aₖ(s)) ds

You invest in a channel exactly when the asset value of its goodwill exceeds the cost of building it. That is the entire logic of when to spend, compressed into one inequality.

The central result: the user cost of goodwill

The singular arc — where λₖ = 1 is held over an interval — is where the action is, because it defines the optimal steady-state goodwill level. Holding λₖ = 1 means dλₖ/dt = 0, so the costate equation collapses to a strikingly clean condition:

gₖ′(aₖ*) = ρ + θₖ

At the optimum, the marginal revenue product of a channel’s goodwill equals the discount rate plus its decay rate. Anyone who has seen neoclassical investment theory will recognize this immediately: ρ + θₖ is the Jorgensonian user cost of capital (discount + depreciation). Advertising goodwill obeys the same law as any capital stock — you hold it up to the point where its marginal product equals its user cost. Media planning is capital budgeting.

The maintenance spend that holds goodwill at aₖ* simply replaces what decays:

uₖ* = θₖ · aₖ*

The practical implications are not obvious from static MMM, and they are large. Solving gₖ′(aₖ*) = ρ + θₖ numerically for three channels that differ in their decay rate:

channel        θ      a* (goodwill)     u* maintenance      rev rate
paid_search   0.80         163.5             130.8            844.9
paid_social   0.40         355.3             142.1           1369.8
display       0.20         500.0             100.0           1076.2

Read down the decay column. Fast-decaying goodwill is expensive to rent. Paid search (θ = 0.80) optimally holds the least goodwill (163) yet needs high maintenance spend (131/week) to sustain it — you are replacing 80% of the stock every week, paying continuously for something that evaporates. Display (θ = 0.20) holds the most goodwill (500) on the least maintenance (100/week), because slow decay means each dollar’s effect compounds across many periods before it leaks away. The static equal-marginal-returns rule from part one is silent on this entirely; it has no notion that a dollar in a high-decay channel buys a shorter-lived asset. Optimal control prices that difference exactly, through the ρ + θ user cost.

import numpy as np
from scipy.optimize import brentq

def hill(a, V, K, n):        return V * a**n / (K**n + a**n)
def hill_prime(a, V, K, n):
    a = max(a, 1e-9)
    return V * n * K**n * a**(n-1) / (K**n + a**n)**2

def turnpike_goodwill(theta, rho, V, K, n):
    """Optimal steady-state goodwill: solve  g'(a*) = rho + theta."""
    f = lambda a: hill_prime(a, V, K, n) - (rho + theta)
    grid = np.linspace(0.1, 500, 20000)
    s = [f(x) for x in grid]
    for i in range(len(grid) - 1):
        if s[i] > 0 and s[i+1] < 0:          # marginal product crosses user cost
            a_star = brentq(f, grid[i], grid[i+1])
            return a_star, theta * a_star     # (goodwill, maintenance spend)
    return None, None

The optimal trajectory: approach, maintain, coast

The steady state tells you the destination; Pontryagin tells you the path. The bang–singular–bang structure produces a universal three-phase trajectory:

1. Approach — if you start below aₖ*, the shadow price exceeds one, so spend goes to maximum to build goodwill toward the turnpike as fast as the bounds allow.
2. Maintain — once at aₖ*, ride the singular arc with maintenance spend uₖ* = θₖ aₖ*, holding goodwill exactly where its marginal product equals user cost.
3. Coast — as the horizon approaches, transversality (λₖ(T) = 0) drives the shadow price below one, so spend cuts to zero and the remaining goodwill is harvested as it decays. There is no point paying for an asset you will not live long enough to use.

Solving the finite-horizon problem numerically (discretizing time and optimizing the full control sequence) reproduces this exactly. The optimal spend trajectory for the fast-decay search channel:

early weeks :  163, 131, 131, 131, 131, 131    <- pulse up, then settle
middle weeks:  131, 131, 131, 131, 131, 131    <- maintenance = θ·a* = 130.8 ✓
final weeks :  131, 131, 110,   0              <- coast: shut off near horizon

The maintenance level the optimizer discovers — about 131/week — is the analytic turnpike spend uₖ* = θ · aₖ* = 0.80 × 163.5 ≈ 130.8 to the decimal. The end-of-horizon shutoff is transversality made visible. The model was never told the three-phase structure; it falls out of the mathematics.

from scipy.optimize import minimize

def finite_horizon_control(channels, T, rho, u_max=200):
    """Discrete-time optimal control: maximize discounted profit over a horizon."""
    theta = np.array([c["theta"] for c in channels])
    beta, Kc = 1 / (1 + rho), len(channels)

    def unroll(U):
        U = U.reshape(T, Kc); a = np.zeros((T, Kc))
        for t in range(1, T):
            a[t] = (1 - theta) * a[t-1] + U[t-1]   # discrete goodwill dynamics
        return a, U

    def neg_profit(Uflat):
        a, U = unroll(Uflat); disc = beta ** np.arange(T)
        rev = np.array([[hill(a[t, k], channels[k]["V"], channels[k]["K"],
                              channels[k]["n"]) for k in range(Kc)] for t in range(T)])
        return -np.sum(disc[:, None] * (rev - U))

    res = minimize(neg_profit, np.full(T * Kc, 5.0), method="L-BFGS-B",
                   bounds=[(0, u_max)] * (T * Kc), options=dict(maxiter=2000))
    return unroll(res.x)

When flat is optimal, and when to pulse

Here optimal control settles a decades-old media debate — even spend versus pulsing/flighting — and shows the answer depends entirely on the shape of the response curve.

Concave response (diminishing returns from the first dollar). When gₖ is concave everywhere, the singular arc is a genuine maximum and steady maintenance is optimal. In the three-channel concave example above, the full optimal trajectory beats a flat-spend baseline by only +0.7% — almost all of the gain is in the initial build and the end-of-horizon coast, not the middle. This is itself a useful and underappreciated finding: for concave channels, roughly constant "always-on" spend is close to optimal, and the turnpike condition tells you the right level. Don't over-engineer the flighting of a saturating search channel.

S-shaped response (a threshold before the channel “turns on”). When gₖ has a convex region — an S-curve where small spend does little until goodwill crosses an inflection — the singular arc is not optimal, and the solution concentrates spend into bursts. This is the classic result of Sasieni (1971) and Mahajan–Muller (1986): pulsing dominates when returns to advertising are increasing over some range. With the same total budget spread over 26 weeks against an S-shaped channel (n = 3), optimal control delivers a dramatic improvement:

even spend across all 26 weeks :  profit 10,552
optimal control (pulsed)       :  profit 17,489      lift = +65.7%
active weeks: 13 of 26   ->   spend pattern:
   0  0  0  0  76 38 38 38 38 38 38 38 38 38 38 38 29  0  0  0  0  0  0  0  0  0

The optimal policy stays dark for the first four weeks, fires a burst (76) to drive goodwill past the response threshold, maintains at ~38 while the channel is “lit,” then shuts off and harvests — going dark again for the final nine weeks. Spreading the same money evenly wastes most of it below the activation threshold the entire time. The +65.7% gap is the mathematical case for flighting: when a channel has a threshold, concentrate; when it saturates from the first dollar, sustain. Static MMM, which fits a single response curve and allocates a single number, cannot represent this distinction at all.

The Hamilton–Jacobi–Bellman view (and stochastic demand)

Pontryagin gives an open-loop trajectory — the optimal plan from a fixed starting state. Its dynamic-programming twin, the Hamilton–Jacobi–Bellman (HJB) equation, gives a feedback policy: optimal spend as a function of the current goodwill state, which is what you need when demand is uncertain. Let V(a) be the value function — the maximum discounted profit achievable from goodwill state a. It satisfies:

ρ V(a) = max  { Σₖ gₖ(aₖ) − Σₖ uₖ + Σₖ V_{aₖ}(a) · (− θₖ aₖ + uₖ) }
          u

The envelope condition ties the two views together cleanly: V_{aₖ}(a) = λₖ — the gradient of the value function is the costate. The shadow price of goodwill from Pontryagin is literally the marginal value of the goodwill state in Bellman's value function. Both say the same thing: spend on channel k iff the marginal value of its goodwill exceeds one.

The HJB form is what you take to production under uncertainty. Real demand is stochastic; goodwill dynamics get a noise term and you solve a stochastic control problem, or — far more practical — you run model-predictive control (MPC): at each week, observe the realized goodwill state, re-solve the finite-horizon problem from there, execute only the first step, and repeat next week with updated data. MPC turns the open-loop plan into a closed-loop policy that self-corrects as demand surprises arrive, and it slots directly into the weekly Snowflake retraining loop from part one — the optimizer just gains a time dimension and a goodwill state.

Where this connects

Across the series the same structure keeps reappearing in three forms. Part one’s equal-marginal-returns rule, dRₖ/dsₖ = μ, is the static shadow-price condition. This article's turnpike, gₖ′(aₖ*) = ρ + θₖ, is its dynamic generalization — the same marginal logic, now pricing the time value and the decay of goodwill as capital. And part two's transfer-entropy screen ensures the gₖ you are differentiating reflects a real causal effect, not a demand echo, so the control problem optimizes something true.

In Papilon this lives where the optimization and dynamics modules meet: response curves estimated under causal constraints feed a finite-horizon controller that returns not a single allocation but a spend schedule — when to build each channel, where to hold it, and when to let it coast. The trilogy answers the three questions a media plan actually faces: where to spend (allocation), which way the causality runs (measurement), and when to spend (control).

The takeaway

Treat advertising goodwill as what it is — a depreciating capital stock — and budget planning stops being a one-shot allocation and becomes an investment problem with a known optimal law. Hold each channel’s goodwill where its marginal revenue product equals the user cost ρ + θ. Build fast to that level, maintain it, and coast off at the end. Spread spend evenly on channels that saturate from the first dollar; pulse hard on channels with a threshold, where in the worked example concentration was worth two-thirds more profit on identical budget. None of this is visible to a model that collapses a time-extended investment into a single number.

The dollar you spend today is not a knob setting. It is a deposit into a leaky asset whose value you can price exactly. Optimal control is how you read the price.

All results are runnable as written with numpy and scipy; the turnpike levels, the three-phase trajectory, and the +65.7% pulsing lift are actual program output. The continuous-time model is the Nerlove–Arrow goodwill formulation; the user-cost result is the advertising analog of Jorgenson's neoclassical investment theory; the pulsing-versus-even result follows Sasieni and Mahajan–Muller. Optimal control assumes you know the response curves — estimate them causally (parts one and two) before optimizing the schedule.

Brian Curry is a Kansas City–based cognitive systems architect and applied AI researcher working at the intersection of AI architecture, knowledge engineering, and intelligent-system design. He is the founder of Vector1 Research, an applied AI lab publishing on agent systems, AI evaluation, cognitive architecture, and the macroeconomics of AI-driven transformation. He has led applied AI and data initiatives across major enterprises including Koch Industries, Tractor Supply, Vail Resorts, Hallmark, Garmin, AT&T, and McClatchy, and founded KC AI Lab, a 1,500-member AI/ML community. His open-source work includes Papilon (causal inference and counterfactual simulation), PyCausalSim (causal discovery via agent-based simulation), MeaningFlow (semantic modeling and ontology), and Memory-Node Encapsulation (episodic memory for agentic systems). Contact: brian at vector1.ai · LinkedIn