Deep Learning That Respects the Theorems: Arbitrage-Free Neural Networks for Commodity Futures

How CARRY-Net compiles the theory of storage into architecture — Working curves, projection layers, and four instructive failures

Ask a generic deep learning model to forecast a corn futures curve and it will happily hand you a curve where the July contract trades 8% above May. Any grain trader will tell you what’s wrong in one second: you could buy May delivery, pay two months of storage and interest (~0.8%), deliver into July, and pocket 7% risk-free. The curve is not a forecast; it’s a machine that prints money for whoever trades against it.

This is the embarrassing open secret of ML in commodity markets. LSTMs and transformers fit wiggles beautifully and violate atheoremwhile doing it. Meanwhile, the structural models economists have used since the 1990s — Schwartz two-factor and friends — respect the theorems but are shackled to linear-Gaussian dynamics that can’t represent a drought.

So I built CARRY-Net, a model where the economics is not a feature of the training data but a property of the architecture. Three claims from the theory of storage become three structural components:

The Working curve (Working, 1949): convenience yield is a decreasing, convex function of inventory. → A network layer that can only represent decreasing convex functions.
Cost-of-carry arbitrage: calendar spreads are capped at “full carry.” → A projection layer that makes violations architecturally impossible. Not penalized. Impossible.
Storage-theoretic dynamics: scarcity mean-reverts toward its inventory-implied level; risk premia load on the convenience yield. → A structured rollout where the neural network only models the one genuinely nonlinear object: inventory flows with harvest cycles and drought regimes.

Constraints on the exercise: no PyTorch, no JAX, no internet. Everything — including a reverse-mode autodiff engine — is pure NumPy, and the full experiment runs in under ten minutes on one CPU core. The complete code accompanies this article.

I’ll show you the architecture, the math, the results — and, because this is where the real education lives, the four ways the elegant end-to-end version failed before the robust version worked.

Part 1: Thirty lines of economics, eighty years old

Three facts about storable commodities, all of which your model should know before it sees a single data point.

Fact 1 — the cash-and-carry bound. If you can buy the near contract, store the grain, and deliver it into the far contract, then for maturities τ₁ < τ₂:

F(t, τ₂) ≤ F(t, τ₁) · exp((r + w)(τ₂ − τ₁))

where r is the interest rate and w the storage cost. Contango is capped at "full carry." There's no matching lower bound — you can't sell short physical corn you don't have — so backwardation can be arbitrarily violent. Real grain curves in glut years literally flatline against this cap: the famous kinked full-carry curve.

Fact 2 — Schwartz pricing. With a mean-reverting convenience yield δ (speed κ), log futures prices are affine in the states:

ln F(t, τ) = ln S_t + (r + w)·τ − δ_t · B(τ),      B(τ) = (1 − e^{−κτ}) / κ

The loading B(τ) says: convenience-yield shocks matter less for distant contracts because δ mean-reverts before they expire. (That one loading is also why near contracts are more volatile — the Samuelson effect — for free.)

Fact 3 — the Working curve. The convenience yield is what physical possession is worth. When silos are empty, possession is gold; when they’re overflowing, it’s nothing. Empirically, across eight decades and many commodities:

δ = g(I),    g′ < 0,    g″ ≥ 0

Decreasing and convex in inventory I. That shape is the theory of storage.

Everything that follows is a way of making a neural network unable to disagree with these three lines.

Part 2: The architecture

2.1 A layer that can only draw Working curves

Here’s the whole trick. Take softplus bases ζ(x) = log(1+eˣ) — each one convex — flip them so they're decreasing, and combine with non-negative weights:

class WorkingLayer:
    """delta = g(i): monotone-DECREASING and CONVEX for EVERY parameter value."""
    def __init__(self, rng, K=4):
        self.rw  = Param(rng.normal(-0.7, 0.3, (1, K)))  # -> w_k = softplus(rw)/K >= 0
        self.rb  = Param(rng.normal( 0.0, 0.3, (1, K)))  # -> b_k = softplus(rb)   >= 0
        self.h   = Param(rng.normal( 0.0, 0.5, (1, K)))
        self.th0 = Param(np.array([[-0.02]]))

    def __call__(self, i):
        w = self.rw.softplus() * (1.0 / self.K)      # nonnegative weights
        b = self.rb.softplus()                        # nonnegative slopes
        z = (-(i * b) + self.h).softplus()            # convex & decreasing bases
        return (z * w).mm(T(np.ones((self.K, 1)))) + self.th0

The proof fits in a tweet: each basis φ(i) = ζ(−b·i + h) has φ′ = −b·σ(·) ≤ 0 and φ″ = b²σ(1−σ) ≥ 0; non-negative combinations preserve both properties. Gradient descent can move this curve around all it wants — it can never make it slope upward, never make it concave. An economic law, enforced by parameterization. (Same spirit as input-convex neural networks, specialized to the decreasing-convex class.)

2.2 A projection layer that makes arbitrage a compile-time error

The decoder first prices the curve the Schwartz way — level s, seasonal term, carry slope, minus δ·B(τ) — and then passes it through a full-carry projection: walk out the curve and clamp every tenor at the previous tenor plus full carry.

# smooth cost-of-carry curve
yraw = s + seas + c * tau - delta * B(tau)

# full-carry projection: exact, differentiable, sequential min
rows = [yraw.row(0)]
for j in range(1, M):
    cap = rows[j-1] + (c + 0.002) * (tau[j] - tau[j-1])
    rows.append(cap - (cap - yraw.row(j)).relu())     # min(a,b) = b - relu(b-a)
yhat = T.cat(rows, axis=0)

min(a, b) = b − relu(b − a) is exact and autodiff-friendly. Two things I love about this layer. First, admissibility is now a proposition about the architecture — every output satisfies the bound between every pair of tenors, provably, forever. Second, it's not a hack: it is literally the storage-arbitrage operator, and it spontaneously reproduces the kinked full-carry curves you see on real grain terminals whenever the seasonal upslope tries to exceed carry.

Penalty terms can be traded away by an optimizer having a bad day. Projections can’t.

2.3 Dynamics: neural networks only where nonlinearity lives

The forecasting state is (i, u): latent inventory plus a small context vector (think: drought regime, demand conditions). A gated cell filters them from each day's observations. The rollout — how you forecast — is where the discipline pays:

def roll(self, i, u, dlt, s, x):
    kpp = self.kp.sigmoid() * 3.0                    # OU pull speed, bounded (0,3)/yr
    tgt = self.working(i)                            # Working-curve target
    d2  = dlt + kpp * (tgt - dlt) * DT               # delta -> its storage-implied level
    i2  = i + self.m_i(T.cat([i, u, xt])).tanh() * 0.02   # NEURAL inventory flow,
                                                          # bounded at physical scale
    s2  = s + (self.a_s + self.b_s * dlt) * DT       # risk-premium drift, LINEAR in delta
    return i2, u, d2, s2

Read it as economics: the convenience yield mean-reverts toward what inventory says it should be (that’s Working’s theory as a difference equation); the spot drifts with a premium that loads on scarcity (that’s Gorton–Hayashi–Rouwenhorst as a drift term); and the only neural network in sight models inventory flows — the one object with genuinely nonlinear structure (harvest pulses every autumn, droughts that persist, consumption that doesn’t care).

Notice the * 0.02. Inventory in this market moves about 0.018 standardized units a day. An earlier version allowed ±0.1/day and the rollout ratcheted the latent inventory into the stratosphere. Bounding dynamic terms at physically sensible scales turned out to be non-negotiable — more on that in the failure museum.

Part 3: Estimation — where I ate humble pie

My first build was the fashionable thing: an end-to-end variational deep Kalman filter. Amortized Gaussian posterior, ELBO, KL warm-up, truncated backprop through time. It failed four separate ways, each instructive enough that I’ve kept them in the paper’s appendix. The production version is a two-stage estimator, and it’s better and simpler.

Stage A: invert the pricing map, exactly

For a fixed κ, look at the decoder again: ln F = s + Δφᵀγ + cτ − δ·B(τ). Given the global seasonal loadings γ, each day's states (s_t, δ_t) enter linearly — a two-parameter regression per day. And given the states, γ is a global least-squares problem. Jointly, it's one big sparse linear least-squares problem, and you can solve it exactly with a Frisch–Waugh concentration: project each day's seasonal features through that day's residual-maker, solve for γ in closed form, back out the states, and profile κ on a one-dimensional grid.

def _concentrated_gamma(y, dphi, kappa, mask):
    B = b_load(TAU, kappa)
    A = np.zeros((4, 4)); rhs = np.zeros(4)
    for t in range(len(y)):
        X = np.column_stack([np.ones(mask[t].sum()), -B[mask[t]]])
        Q, _ = np.linalg.qr(X)                    # day-t residual-maker via QR
        D  = dphi[t, mask[t]]
        yy = y[t, mask[t]] - CBAR * TAU[mask[t]]
        MD = D - Q @ (Q.T @ D)                    # Frisch-Waugh projection
        My = yy - Q @ (Q.T @ yy)
        A += MD.T @ MD;  rhs += MD.T @ My
    return np.linalg.solve(A, rhs)

Why the mask? Because of the kinks. When a curve sits at full carry, those tenors tell you nothing about δ except an inequality — the spread equals the cap whether the glut is mild or biblical. So clipped tenors are treated as censored: masked from estimation, and on days when the entire curve is at full carry, δ is imputed from neighboring days, capped at the bound-consistent level. That's an honest identification boundary of any curve-based method, and it's better to model it than to pretend it isn't there.

On noiseless data drawn from the model family, Stage A recovers κ and every state to machine precision (a unit test in the repo). On the actual noisy, clipped, regime-switching data: correlation 0.999 with the true log-spot, 0.88 with the true convenience yield — a variable that is never observed — at full dynamic range. It runs in 0.4 seconds.

There was one identification landmine: the net carry c and the level of δ are nearly collinear (both tilt the curve), so learning both let c wander and drag the projection cap with it. Solution, straight from the storage literature: treat c = r + w as observed. Rates come from the bond market and storage tariffs are published. Degeneracy gone.

Stage B: train on the objective you’ll be judged by

The dynamics train on 44-day teacher-forced windows (in random order — see failure #2) followed by a fully differentiable 20-step rollout, with two losses. An anchor, which pins the Working layer to the measured convenience-yield path:

loss += (model.working(i) - delta_measured[t]).sq() * (8.0 / L)

— this is what makes the flat-curve collapse impossible: g(i) must reproduce a target with real variance. And the anchored-curve forecast loss: decode the rolled-out states into a curve, anchor it to today’s observed curve, and penalize against the actual future curve. The training loss is the evaluation metric, which also gets the s-vs-δ error weighting exactly right without hand-tuning (a spot error hits all twelve tenors; a δ error is weighted by B(τ)).

Anchoring deserves its own sentence, because it’s a two-line trick that cut my error floor by 40%:

ŷ(t+h) = y(t) + [ fit(ẑ(t+h|t), t+h) − fit(ẑ(t), t) ]     # then project

Forecast thechangeof the fitted curve, add it to today’s observed curve. Any persistent decode misfit — finite-sample κ̂ error, imputation error at the kinks — appears in both fit terms and cancels. Forecasters have known this forever; state-space modelers forget it constantly.

Finally, state forecasts are shrunk toward persistence with a per-horizon weight chosen on a validation window (the last 280 training days). Boring, standard, and — as you’re about to see — the most honest hyperparameter in the whole project.

Part 4: The test market

You can’t validate structure recovery on real data — nobody observes daily convenience yields or true inventories. So the testbed is a synthetic corn-like market with known ground truth and deliberately hostile physics: ~11 crop years of daily 12-tenor curves; autumn harvest pulses; a 22%-per-year drought regime that cuts the harvest 45% and jumps the price on onset; convenience yield OU-reverting to a true Working curve g*(I) = −0.01 + 0.13·e^{−I}; an 18% risk premium; 6% seasonality; and every curve clipped at full carry, then observed with noise. Prices range $2.40–$10.10/bu, 48% of days are backwardated, corr(δ, I) = −0.68.

Crucially, the deck is not stacked: droughts, jumps, harvest pulses, and the clipping are outside every model’s parametric family — including CARRY-Net’s. Train on the first 2,200 days, test on the last 550 (~2.2 crop years).

Figure 1 — The testbed. Top: 1m vs 12m futures (~11 crop years; dotted line = train/test split). Middle: convenience yield spikes exactly when inventory craters — the theory of storage at work. Bottom: sample curves; note the kinked full-carry contango.

Baselines: the random walk (the assassin of forecasting papers), per-tenor AR(1), dynamic Nelson–Siegel, a Schwartz-style linear twin (identical measurement stage, linear dynamics — this isolates what the Working layer + neural inventory buy), and a GRU-Seq2Seq with the same capacity and training protocol but zero economic structure (this isolates what structure buys).

Part 5: Results

Out-of-sample curve RMSE, log-price %:

Model                  h = 1  h = 5  h = 20
---------------------  -----  -----  ------
Random walk            1.67   3.49   7.23
AR(1)                  1.67   3.48   7.17
Dynamic Nelson–Siegel  2.49   3.98   7.60
Schwartz linear twin   1.79   4.44   8.88
GRU-Seq2Seq            11.35  11.73  12.53
CARRY-Net              1.68   3.50   7.16

And the column that would actually get you hired or fired — the fraction of forecast calendar spreads violating the cash-and-carry bound:

RW    AR1   DNS   Schwartz twin  GRU    CARRY-Net
----  ----  ----  -------------  -----  ---------
6.2%  5.6%  1.3%  0              17.7%  0

Figure 2 — The two tables as one picture. Left: the unstructured GRU (orange) towers over everything; CARRY-Net (red) is lowest or tied at every horizon. Right: only the projected models sit at exactly zero violations — by theorem, not by luck.

Read the two tables together:

CARRY-Net is best-or-tied at every horizon — best at one month — while compressing twelve prices through a two-state economic bottleneck plus measurement noise. Matching the random walk at h=1 through that bottleneck means the structural representation is essentially lossless.
Structure beats its absence by 43%. The GRU with identical data, capacity, and training budget is 4–7× worse than persistence — the classic faceplant of unstructured deep forecasters on a single nonstationary series. The economic skeleton is what prevents it.
Learning beats linearity by 19%. Against the Schwartz twin — same measurement, same decoder, same anchoring, only the dynamics differ — the Working layer plus neural inventory flow is worth 19% at h=20. That’s the cleanly isolated value of the ML.
Zero violations, by theorem. For the two projected models the 0% is not an empirical result; it’s Proposition 2. The random walk violates 6% of the time simply because the seasonal position of the curve rotates over 20 days; the GRU violates 18% of the time because nothing tells it not to.

Interpretability comes standard: the daily curve-implied convenience yield tracks the unobservable truth at 0.88 correlation; the learned Working curve comes out decreasing and convex (it must) and matches the true curve’s shape along inventory percentiles; and the designated latent inventory state correlates 0.37 with true inventoryon the test set — a variable the model has never seen, recovered from prices alone. (A post-hoc probe of the GRU’s latents reaches 0.97, but you need the answer key to pick which latent — that’s an oracle, not interpretability.)

Figure 3 — Stage A measurement. The curve-implied convenience yield (red) tracks a variable that is never observed (blue), out of sample, at 0.88 correlation — extracted in 0.4 seconds of linear algebra.

Figure 4 — What the constraints bought. Left: the learned Working curve (red) is decreasing and convex because it cannot be anything else, and matches the true shape (dashed). Right: the designated latent inventory state co-moves with true inventory on held-out data.

The result I didn’t want, and why I’m publishing it anyway

The validation-selected shrinkage weights came out α = 1.0 at h=1, 0.25 at h=5, and 0.0 at h=20: at one month, the model chose pure structured persistence over its own learned dynamics. I tried neural dynamics, the linear twin, even closed-form OLS drift regressions fitted directly to the measured states. Out of sample, at h=20, nothing beat structured persistence.

The arithmetic says this is not a bug. Over 20 days, the predictable components in this market — the OU pull (~11% of the δ-gap), the risk-premium drift (±1.5% of spot), seasonal inventory flow — add up to a few percent of variance against a 6.7% unpredictable spot innovation. That’s below the learnability floor of any flexible model on 8.7 crop years, and the 2.2-year test window adds regime luck on top. The deterministic predictable component (seasonal curve rotation) is captured analytically by the structured decode — which is exactly why anchored structural persistence edges out the raw random walk. The stochastic predictable component is real but unlearnably small at this sample size.

So the honest headline is not “deep learning found alpha in corn.” It’s better: economic structure bought admissibility, interpretability, and immunity to the unstructured model’s catastrophic failure — at zero cost in accuracy against the strongest benchmark that exists. In production terms: same accuracy as the random walk, but the curves are legal, the states mean something, and the model can’t blow up the way the GRU did.

Part 6: The failure museum

Everything above is the third architecture. The first two produced the most useful artifacts of the project. All four exhibits are reproducible.

Exhibit 1: The model that turned up its own deafness. The end-to-end variational version learned its observation noise σ_y — and used it as an escape hatch. Early misfit inflated σ̂_y to 2.3% (truth: 0.4%), which flattened the likelihood by 33×, which removed the gradient pressure to use the convenience-yield channel, which let the Working layer collapse to a constant, which kept the misfit high. A perfect self-sealing local optimum. Lesson: a learnable decoder variance is a variational model’s cheapest way to stop caring; anchor structural channels to targets with real variance.

Exhibit 2: The loss that lied. Training printed loss −20. A frozen-parameter forward pass over the same data with the same final weights: +274. Sequential TBPTT windows with one optimizer step each meant the network was surfing — re-tuning its weights to each 44-day regime, scoring well locally, converging to no single parameter vector that fit anything globally. The printed loss was measuring adaptation speed, not fit. Random window order plus a persistent state buffer killed it. Lesson: with sequential windows on one nonstationary series, always re-score with frozen parameters. Always.

Exhibit 3: The filter that resonated. The Kalman-style innovation encoder fixed level-tracking beautifully — then training pushed its base gain above 1.0, prior noise scales grew 4×, and the closed loop between “decode at prior mean” and “update on innovation” started oscillating. Lesson: learned feedback gains need architectural bounds, the same way the rollout’s dynamic terms need physical bounds (remember the inventory ratchet from Part 2.3 — that was Exhibit 3b).

Exhibit 4: The two parameters wearing one trenchcoat. Net carry c and the level of δ both tilt the curve; jointly free, they're near-collinear, and c wandered off — taking the projection cap with it. Treat carry as observed (it is) and the degeneracy vanishes. Lesson: identification analysis isn't optional just because your model has a neural network in it.

The meta-lesson, and the actual thesis of this article: when the elegant end-to-end estimator fights you on identification, replace the fragile parts with exact linear algebra and keep the learning where it earns its keep. Stage A is 1960s econometrics executed flawlessly in 0.4 seconds. Stage B is a small neural network doing the one job only it can do. The combination beat the monolith on every axis — accuracy, stability, interpretability, and debuggability.

Part 7: Where this goes

Real CME corn/soybean/wheat curves are the obvious next step — with USDA quarterly Grain Stocks entering as sparse observation equations on the latent inventory, four days a year (the filtering formulation eats mixed-frequency data natively). Sharing the Working layer across commodities would test whether auniversalscarcity–convenience-yield shape exists — a genuinely open empirical question this architecture was built to ask. And the recipe itself is portable to any domain with hard-won qualitative laws and scarce data: electricity forward curves, freight rates, emissions markets.

If you take one thing away: name your latents, constrain your links, decode through the theory, and train on the objective you’ll be judged by. The neural network should be the smallest part of your model that still does the impossible bit.

The complete implementation — autodiff engine, market simulator, model, baselines, and every experiment in this article — is ~1,100 lines of pure NumPy, no framework required, and runs end-to-end in under ten minutes on one CPU core. See the accompanying technical paper for propositions, proofs, and the full failure-mode appendix. Github repo: https://github.com/Bodhi8/CarryNet

About the Author

Brian James Curry is a Kansas City–based data scientist and applied AI researcher working at the intersection of economic modeling, causal inference, and machine learning — with a standing preference for models where the economics is load-bearing. He is the founder of Vector1 Research, an independent lab exploring marketing economics, cognitive AI architecture, and AI-driven enterprise transformation, with research contributions including Papilon, PyCausalSim, and Memory-Node Encapsulation. His twenty-plus years of enterprise work span Koch Industries, Tractor Supply, Vail Resorts, Hallmark, Garmin, and AT&T.

He grew up in a farming town and was working the crop fields by age twelve — so the theory of storage is not, strictly speaking, something he learned from a paper. CARRY-Net’s code, technical paper, and full failure museum live at [github.com/yourhandle/carrynet]. Views are his own; nothing here is investment advice.