The Oldest Decision in Agriculture, Made With a New Instrument
Using a curve-implied convenience yield to decide when to sell the harvest — a worked economic problem, a backtest, a risk dial, and one drought that proves the point
Authors Note: CARRY-Net is an algorithm I developed at Vector1 Research: a structure-informed machine-learning model that embeds the Theory of Storage directly into its neural architecture. This is the second article in the series — the first article covers how the model was built, and the full implementation, technical paper, and every experiment in this piece are open source at https://github.com/Bodhi8/CarryNet.
Every harvest begins the same recurring economic decision: sell the crop immediately, or pay to store it for a later market.
Every October, anyone who holds grain confronts the same decision, and the basic problem has not changed since commercial silos were invented: sell at harvest, when the market is flooded with new supply, or pay to store the crop and sell it later.
The decision appears simple because agricultural prices often follow a recognizable seasonal pattern. Harvest concentrates supply into a narrow period, storage absorbs part of that supply, and prices may strengthen as inventories are consumed during the following winter and spring. Yet storage is not a free option on higher prices. It requires capital, incurs physical costs, and exposes the owner to the possibility that the expected recovery never arrives.
Interest continues to accrue against the value of the inventory. Elevators charge storage and handling fees. Grain is exposed to shrink, spoilage, quality deterioration, and basis risk. Capital that might otherwise reduce debt or finance the next production cycle remains tied up in the crop. In favorable years, the eventual price increase more than compensates for these costs. In unfavorable years, the owner pays to store an asset whose value is falling.
Commodity economics has had a theoretical answer to this problem for nearly a century. The answer comes from the Theory of Storage and, more specifically, from an economic quantity known as the convenience yield. Holbrook Working’s research on grain markets established that the price of storage is related nonlinearly to physical inventories, while later work formalized the role of convenience yield in connecting spot prices, futures prices, carrying costs, and scarcity.
The operating principle can be expressed plainly. When convenience yield is low, physical inventories are abundant and the market may compensate owners for carrying them forward. When convenience yield is high, immediate possession has become valuable, the market is signaling scarcity, and releasing inventory is often more attractive than continuing to store it.
The difficulty has never been the economic logic. The difficulty is that convenience yield is not directly observable. It does not appear on an exchange screen, an elevator bid sheet, or a standard market-data terminal. It is embedded inside the futures curve together with interest rates, storage costs, seasonality, mean-reversion structure, delivery effects, and, during surplus regimes, a censoring mechanism created by the full-carry arbitrage boundary.
This article applies CARRY-Net to that measurement problem. CARRY-Net is a structure-informed machine-learning model that embeds the Theory of Storage directly into its architecture. Its measurement stage inverts an arbitrage-consistent pricing map to estimate a daily curve-implied convenience yield, while explicitly accounting for seasonality, maturity-dependent mean reversion, and futures tenors that have flattened against the full-carry boundary. The implementation, technical paper, synthetic market generator, figures, metrics, and complete reproduction workflow are available in the open-source CARRY-Net repository.
The experiment presented here uses the same reproducible corn-like synthetic market used to develop and validate the model. The simulated history contains approximately eleven crop years, seasonal harvest cycles, stochastic inventory consumption, a constant storage cost, an 18 percent spot risk premium, and drought regimes that reduce harvest size by 45 percent. Because the market is synthetic, the true latent inventory and convenience yield are known, which makes it possible to test whether the model is measuring the intended economic quantity rather than merely producing a plausible-looking signal. The results are therefore a methodological demonstration, not a claim about live CME corn performance.
The objective is narrower than predicting the future price of grain and more useful than it may initially appear. The question is whether a measured convenience yield can transform the Theory of Storage from an explanatory framework into a daily decision rule.
The Hidden Variable Behind the Storage Decision
Convenience yield is commonly defined as the non-cash benefit of holding a physical commodity rather than a contract promising future delivery. That definition is accurate, but it can sound abstract. A more operational interpretation is that convenience yield represents the market’s current valuation of immediate physical availability.
When inventories are abundant, another bushel of grain provides relatively little additional operational value. A processor can source replacement inventory easily, an exporter can satisfy commitments through several channels, and a temporary delivery disruption can be absorbed from existing stocks. Under those conditions, the convenience yield is generally low.
As inventories decline, the value of physical possession rises. Grain on hand can prevent a plant shutdown, satisfy an unexpected order, preserve delivery flexibility, or protect against a temporary disruption in transportation or production. Near the inventory floor, those benefits can increase very rapidly. The Theory of Storage therefore predicts a negative and convex relationship between inventory and convenience yield: convenience yield decreases as inventory rises, but it increases at an accelerating rate as inventory approaches scarcity. This is the classical Working curve.
Recent research continues to find evidence for this relationship. A 2024 working paper by Yining Zhu and Calum Turvey of Cornell examined convenience yields and implied storage prices across wheat, soybeans, corn, crude oil, and natural gas from 2018 through 2023, finding support for Working’s storage-supply relationship in all of the examined markets except wheat. The result is important because it shows that the underlying inventory-scarcity mechanism remains empirically relevant across both agricultural and energy commodities.
The same mechanism also explains why the physical condition of the market matters more than the headline supply shock alone. A poor harvest does not produce a fixed price response. Its effect depends on the inventory buffer that exists when the shock arrives. A well-stocked market can absorb part of the loss by drawing down inventories. A depleted market cannot. Once the buffer is exhausted, the price of immediate possession becomes nonlinear, convenience yield rises sharply, and the nearby futures curve may invert.
That inventory-dependent response is central to the drought example later in this article. It is also the reason a storage policy based only on spot-price direction or a raw calendar spread can fail. Those observables show what prices did, but they do not necessarily identify the economic state that produced the move.
Figure 2. The Working curve in this experiment. Convenience yield is a decreasing, convex function of inventory — scarcity becomes disproportionately valuable as stocks approach exhaustion. The red curve is what CARRY-Net learned; the dashed blue curve is the true relationship in the simulated market. The model is architecturally incapable of learning any other shape.
Why a Futures Spread Is Not the Convenience Yield
The traditional cost-of-carry relationship provides the starting point for understanding commodity futures prices. In a simplified market, the price of a futures contract reflects the spot price plus financing and storage costs, minus the economic benefit of holding the physical commodity. In its standard form:
Futures Price
=
Spot Price x exp[(Interest Rate + Storage Cost - Convenience Yield) x Time]This equation is frequently rearranged to infer convenience yield from one spot price and one futures contract. The rearrangement is useful as an accounting identity, but it becomes unreliable when treated as a complete measurement model. Real commodity curves contain more structure than the simple formula acknowledges.
Agricultural production is strongly seasonal. A December-to-March spread is influenced not only by carrying costs and scarcity, but also by the position of both contracts within the crop cycle. Convenience-yield shocks are mean reverting, so their effect is not loaded equally across maturities. Delivery locations, storage constraints, quality differences, and basis conditions can influence parts of the curve differently. During severe surplus conditions, deferred contracts may press against the maximum spread justified by interest and storage costs, creating a full-carry ceiling that hides how deep the underlying glut has become.
This means that the raw calendar spread contains information about convenience yield but is not identical to it. Contemporary research reaches a similar conclusion. The futures basis is commonly used as a proxy for convenience yield, but it is confounded by financing and storage costs that may be persistent and difficult to observe precisely. Ming Gu, Wenjin Kang, Dong Lou, and Ke Tang have proposed a “relative basis” measure — the difference between a near-term basis and a similarly defined distant basis — precisely to purge those persistent cost components and isolate the faster-moving scarcity information.
The problem is especially acute at harvest. Seasonal pressure is strongest when the storage decision is first made, so the very moment when an owner most needs a clean scarcity signal is also the moment when the raw curve is most contaminated by predictable crop-cycle effects.
A naive spread rule may therefore see a harvest-related curve shape and conclude that storage is not being compensated, even when the underlying convenience yield remains below the economically relevant threshold. Conversely, a spread that appears to pay storage may conceal a developing scarcity premium if maturity effects and seasonality are not removed correctly.
CARRY-Net addresses this by treating convenience yield as a latent state to be measured from the whole curve rather than a number to be calculated mechanically from one spread.
CARRY-Net Is a Measurement Instrument Before It Is a Forecasting Model
Most machine-learning applications in financial markets begin with a predictive objective. They take a history of prices, fit a flexible function, and attempt to forecast the next price, return, or curve. CARRY-Net begins from a different premise: before asking what the market will do next, it asks what economic state is producing today’s curve.
That distinction changes the design of the model. A generic recurrent neural network is free to discover any representation that reduces its forecasting loss, even if that representation has no economic interpretation and produces futures curves that violate storage arbitrage. CARRY-Net instead “compiles” several established properties of commodity economics into the architecture itself.
The first component is the Working layer, which maps a latent inventory state to convenience yield. The layer is constructed so that its output can only be a decreasing and convex function of inventory. Training can change the precise shape, level, and curvature, but it cannot learn the economically nonsensical result that convenience yield rises with abundant stocks or becomes less sensitive as inventories approach zero.
The second component is the full-carry projection layer. For storable commodities, a deferred futures contract cannot remain arbitrarily expensive relative to an earlier delivery contract, because a market participant could buy the earlier delivery, finance and store the commodity, and deliver it against the later contract. CARRY-Net applies a differentiable projection that caps each adjacent calendar spread at the relevant carry boundary. Every decoded curve is therefore arbitrage-admissible by construction rather than merely encouraged to be admissible through a soft penalty.
The third component is a structured state transition. Convenience yield is pulled toward the level implied by latent inventory, while the spot drift loads on scarcity through a risk-premium channel. The neural network is reserved for inventory flows, where nonlinearities such as harvest pulses, consumption variability, and drought persistence are most plausible.
The fourth component is a two-stage estimation procedure. Stage A measures spot and convenience-yield states by inverting the cross-section of futures prices. Stage B learns how those measured states and the latent inventory evolve through time. The repository implements the complete model in pure NumPy, including a custom automatic-differentiation engine, and reproduces the experiment on a CPU without an external deep-learning framework.
Figure 3. CARRY-Net converts an observable futures curve into economically interpretable state estimates before those states are used for forecasting or decision-making.
The distinction between measurement and forecasting is important because the storage rule in this article does not require a precise prediction of the spot price months into the future. It requires a reliable estimate of the current convenience yield and a defensible threshold against which to compare it.
In the synthetic testbed, Stage A reconstructs the observed curve with a 1.55 percent log-price root-mean-square error and produces a curve-implied convenience-yield estimate whose correlation with the unobserved true state is approximately 0.88. The estimated spot state correlates with the true spot state at 0.999. These results matter because the trading decision is driven by the measured state, not by a black-box classification label.
Stage A: Extracting Scarcity From the Curve
The measurement stage begins with a cross-section of twelve monthly futures tenors. For each trading day, the model represents the curve using a spot-level state, a convenience-yield state, seasonal calendar features, known net carry, and a maturity loading that reflects the mean reversion of convenience yield.
In plain text, the uncensored pricing map can be summarized as:
Log Futures Price
=
Log Spot State
+ Seasonal Effect
+ Net Carry x Maturity
- Convenience Yield x Mean-Reversion LoadingThe mean-reversion loading is critical. A scarcity shock affects the nearby curve more strongly than distant contracts because the market expects storage, consumption, substitution, and production responses to reduce the imbalance over time. Subtracting the same convenience yield linearly from every maturity would therefore misread the shape of the curve.
Stage A estimates the global seasonal coefficients and the daily spot and convenience-yield states through concentrated least squares. For a fixed mean-reversion parameter, the daily states can be eliminated analytically, allowing the remaining seasonal regression to be solved efficiently. The mean-reversion parameter is then selected by profiling the concentrated objective.
The method also treats full-carry contracts as censored observations. A tenor that sits at the carry boundary no longer reveals the exact depth of the glut. It tells the analyst only that unconstrained contango would have been at least as steep as the arbitrage limit. Including such a tenor as though it were an ordinary linear observation would bias the inferred convenience yield.
CARRY-Net identifies those clipped tenors and removes them from the state regression. On days when the entire curve is pinned at full carry, the convenience yield is only set-identified, so the implementation uses bounded interpolation from adjacent identified days. This is not a convenient modeling trick; it is an explicit acknowledgment of an information boundary that affects any curve-only measurement method.
The result is a daily scarcity gauge that has already removed the components most likely to confuse a raw-spread heuristic. The gauge is not perfect, but it is tied to an economic pricing map, is available at market frequency, and can be validated against known ground truth in the simulated market.
Formalizing the Harvest Decision
Consider a producer, elevator, cooperative, or merchant holding grain at harvest on trading day h. Selling immediately realizes the harvest spot price. Continuing to store preserves exposure to future price changes but incurs interest and physical storage costs.
If the grain is sold on a later day t, its net excess value relative to selling at harvest can be represented as:
Excess Value at Sale
=
ln(Spot Price at t)
- ln(Spot Price at harvest)
- (Interest Rate + Storage Cost) x (t - h) / 252The synthetic market assumes that the expected annualized spot drift is the interest rate plus a risk premium, minus the current convenience yield:
Expected Spot Drift
=
Interest Rate + Risk Premium - Convenience YieldSubtracting the financing and storage expense of carrying the physical inventory gives the expected net return from storing for one additional unit of time:
Expected Net Storage Gain
=
Risk Premium - Storage Cost - Convenience YieldThe interest rate disappears because it influences both the expected spot drift and the financing expense of holding the commodity. What remains is an economically intuitive comparison between the premium earned for bearing commodity risk, the physical cost of storage, and the current scarcity value of immediate possession.
Storage has a positive expected incremental return while:
Convenience Yield < Risk Premium - Storage Cost
The break-even threshold is therefore:
Break-even Convenience Yield = Risk Premium - Storage Cost
In the synthetic market, the annualized risk premium is 18 percent and the physical storage cost is 2 percent. The break-even convenience yield is consequently 16 percent:
delta* = 0.18 - 0.02 = 0.16
This derivation turns the Theory of Storage into a decision rule. Continue storing while the measured convenience yield remains below 0.16. Sell when the measured convenience yield reaches or exceeds 0.16.
The rule formalizes the old trading maxim that one should not store an inverted market, but it is more precise than the maxim. Instead of relying on the visible slope of one calendar spread, it compares a model-based estimate of the underlying scarcity value with an economically derived threshold.
A simple implementation is:
def carrynet_sell(harvest_day, delta_hat, threshold=0.16, window=170):
"""
Store while the estimated convenience yield remains
below the break-even threshold. Sell when scarcity
makes immediate inventory more valuable.
"""
final_day = min(harvest_day + window, len(delta_hat) - 1)
for day in range(harvest_day, final_day + 1):
if delta_hat[day] >= threshold:
return day
return final_dayThe 170-trading-day cap approximates a storage window from autumn harvest into late spring or early summer. It also prevents the rule from becoming an indefinite inventory strategy when the scarcity threshold is never reached.
Figure 4. The storage policy is an optimal-stopping rule driven by a measured economic state. Shown here on a real window from the experiment: the estimated convenience yield crosses the 0.16 break-even line days after harvest, and the rule sells. The threshold is derived from the risk premium and physical storage cost.
Four Policies Across Ten Harvests
The backtest evaluates four policies at each of ten usable harvest dates. All four are exposed to the same simulated spot prices, carrying costs, and maximum storage horizon.
The first policy sells at harvest. It is the benchmark and has an excess return of zero by construction.
The second policy stores to the end of the spring window every year. This is a passive seasonal strategy that seeks to collect the commodity risk premium and benefit from the typical post-harvest recovery. It never responds to the state of the futures curve or the inventory system.
The third policy uses a traditional spread heuristic. It continues storing while the three-month calendar spread pays at least 80 percent of full carry, then sells when the observable curve appears to stop compensating storage. The rule uses only the raw spread and does not adjust for seasonality, mean-reversion loadings, or censored tenors.
The fourth policy uses CARRY-Net’s estimated convenience yield. It stores while the daily estimate remains below 0.16 and sells on the first day the estimate crosses the break-even line.
Across the ten crop years, passive spring storage generated the largest average excess return at 4.52 percent, but that average came with a 20.1 percent standard deviation and a worst year of negative 30.5 percent. The strategy collected the risk premium in favorable years but retained essentially all of the downside associated with continuing to store through a scarcity regime and subsequent normalization.
The naive spread rule produced an average excess return of negative 0.16 percent, a worst year of negative 3.3 percent, and a standard deviation of 1.2 percent. It sold almost immediately in nearly every year, with an average holding period of only 0.3 days. The policy avoided large losses by taking almost no storage exposure, but it also failed to earn the premium the rule was intended to capture.
The CARRY-Net policy generated a mean excess return of 1.94 percent, a standard deviation of 7.3 percent, and a worst year of negative 4.8 percent. It retained approximately 43 percent of the passive strategy’s average return while reducing the worst loss by more than a factor of six. These storage-policy results come from the worked experiment in the original analysis and should be interpreted as synthetic-market evidence rather than a live trading result.
Figure 5. Annual excess value relative to selling at harvest. The CARRY-Net rule gives up part of the passive strategy’s upside but substantially reduces its most damaging loss. Drought years are shaded.
The comparison illustrates why average return alone is an incomplete measure of storage performance. An elevator or producer does not hold an infinitely diversified portfolio of independent crop years. It must survive each year’s financing requirements, margin pressure, operating expenses, and debt obligations. A strategy with a positive long-run mean can still be commercially unacceptable if one adverse season creates a loss large enough to threaten liquidity.
The passive policy earns a premium precisely because it bears that risk. CARRY-Net does not eliminate uncertainty or manufacture a free return. It changes the shape of the exposure by exiting when the market’s implied scarcity value indicates that immediate inventory has become more valuable than continued storage.
Why the Naive Spread Rule Failed
The naive spread policy is especially informative because it appears, at first, to use the same economic intuition as CARRY-Net. Both policies inspect the futures curve and attempt to determine whether the market is paying for storage. Their very different behavior shows why measurement design matters.
At harvest, the raw three-month spread contains a large seasonal component. Futures prices reflect the transition from immediate harvest pressure into later parts of the crop cycle, and that seasonal shape can dominate the scarcity signal. A rule that compares the observed spread directly with full carry may therefore conclude that storage is unattractive even when the underlying convenience yield remains low.
CARRY-Net attempts to remove that confounding effect by modeling the maturity-date seasonality explicitly. It also applies the appropriate mean-reversion loading, recognizing that a unit of convenience yield affects a nearby contract more than a distant contract. Finally, it excludes tenors that have been clipped at full carry and therefore provide only inequality information about the latent state.
The naive rule is not wrong because spreads contain no economic information. It is wrong because it assigns the entire observed spread to the variable of interest. The difference is analogous to separating a signal from a structured set of known distortions rather than treating the raw observation as the signal itself.
This failure has a broader implication for commodity analytics. Calendar spreads, basis measures, carry ratios, and roll yields are often useful predictors, but they should not automatically be interpreted as clean estimates of convenience yield. The relative-basis research cited earlier makes the same argument from a different direction: the traditional basis mixes transient scarcity information with persistent cost components, and separating those components sharpens the relationship with physical inventories.
The Threshold Is a Risk Dial, Not a Magic Number
The 0.16 threshold comes from the known parameters of the synthetic market. In a real implementation, the risk premium would have to be estimated and allowed to vary, while storage costs would need to reflect the actual facility, financing arrangement, commodity, location, and time period.
Even within the synthetic market, the threshold should not be interpreted as a uniquely optimal constant for every objective. It is better understood as a risk dial.
A lower threshold causes the model to sell sooner. As the threshold approaches zero, the strategy converges toward selling at harvest. Average storage returns decline, but exposure to adverse storage years also contracts.
A higher threshold permits storage through more elevated scarcity conditions. As the threshold increases, the policy approaches passive spring storage. It captures more of the commodity risk premium and more of the positive outlier years, but it also accepts a larger left tail.
Sweeping the threshold therefore traces a frontier between average performance and worst-case loss. In this experiment, the theory-derived threshold lies near the knee of that frontier, where additional expected return begins to require a disproportionate increase in downside exposure.
Figure 6. Sweeping the convenience-yield threshold traces an explicit trade-off between average storage return and downside exposure. The economic break-even point lies near the frontier’s knee in the synthetic experiment.
This framing is useful operationally because it separates measurement from risk preference. CARRY-Net supplies an estimate of the current economic state. Management chooses how aggressively to respond to that state.
A highly leveraged operation with limited working capital might select a lower threshold and prioritize loss containment. A well-capitalized merchant with diversified storage capacity might accept a higher threshold. A cooperative could vary the threshold by inventory tranche, selling part of the crop at the break-even point while retaining another portion for later opportunities.
The instrument does not dictate a single universal decision. It makes the trade-off measurable and auditable.
The Honest Cost of Discipline
The CARRY-Net rule does not dominate passive storage in every year. It cannot, because exiting when scarcity is high sometimes forfeits a subsequent rally.
In one crop year, the estimated convenience yield opened near 0.36, well above the 0.16 threshold. The rule sold at harvest because the curve indicated deep scarcity and a negative expected incremental return from continued storage. The spot market nevertheless rallied approximately 31 percent afterward.
A backwardated market can become more backwardated, and a scarce market can become scarcer. Convenience yield is a state estimate, not a crystal ball. The threshold rule is designed to reduce exposure to an unfavorable conditional expectation, not to predict every realized price path correctly.
This is the unavoidable cost of risk discipline. A policy that truncates large losses will also truncate some large gains, particularly when a market that already appears expensive experiences an additional positive shock. The correct question is not whether the rule wins in every crop year, but whether its return distribution is better aligned with the owner’s capital constraints and operating objectives.
In this experiment, the rule exchanges some of the passive strategy’s lottery-like upside for a materially smaller drawdown. That is the behavior one should expect from a risk instrument.
The Two Droughts That Explain the Mechanism
The most economically revealing result comes from two consecutive drought years in the simulated history. Each drought reduces the harvest by 45 percent, so both represent severe physical supply shocks. Yet the futures curves respond differently because the inventory conditions at the time of each shock are different.
During the first drought, the spot price rises approximately 16.7 percent at onset, but the estimated convenience yield remains near 0.02. The futures curve shifts upward without developing a strong scarcity inversion. The reason is that the market enters the drought with a substantial inventory buffer. The failed harvest reduces stocks, but existing grain absorbs much of the physical imbalance.
This is a price-level event rather than a full scarcity event. The commodity becomes more valuable, but immediate possession does not yet become extraordinarily valuable relative to future delivery. The market can satisfy consumption by drawing down stored inventory.
The second drought arrives after the first has already depleted that buffer. With inventories close to their floor, the next harvest loss cannot be absorbed in the same way. The nearby physical commodity acquires a large operational premium, the curve inverts, and the estimated convenience yield crosses the 0.16 threshold within several days of harvest.
The CARRY-Net policy sells on day three and finishes the storage year at approximately negative 3.7 percent relative to selling at harvest. The passive strategy continues holding grain through the inversion and subsequent normalization, eventually losing 30.5 percent.
Figure 7. The first drought is absorbed by inventory and primarily shifts the price level. The second drought strikes an exhausted buffer, causes a scarcity inversion, and triggers the storage exit. The red dot marks the rule’s sale, three days after harvest.
The comparison demonstrates the mechanism described by the Theory of Storage. Inventory is a shock absorber. When the buffer is large, a supply disruption can be accommodated through stock withdrawal, and the curve may move largely in parallel. When the buffer is depleted, the same physical disruption produces a nonlinear increase in the value of immediate possession.
This distinction is difficult to infer from the spot-price move alone. Both droughts are bullish, and both cause an immediate price increase. The convenience yield provides different information: it measures whether the shock has changed the structure of physical scarcity.
The example also explains why the latent inventory state belongs in the architecture. The estimated convenience yield identifies the market’s current scarcity condition, while the Working layer encodes how sensitive that condition should be to further inventory loss. At high stocks, a demand or supply shock produces a relatively modest change in convenience yield. Near the inventory floor, the same shock produces a much larger response.
That convexity is not merely a statistical pattern discovered after the fact. It is imposed as a structural property of the model because it represents the economic mechanism the model is intended to measure.
What the Model Contributes Beyond the Storage Rule
The storage backtest uses only one output from the broader CARRY-Net system: the daily estimated convenience yield. The full model also produces a latent inventory representation, an admissible reconstructed futures curve, and structured multi-horizon forecasts.
In the repository’s held-out forecasting experiment, CARRY-Net achieves curve root-mean-square errors of approximately 1.68 percent, 3.50 percent, and 7.16 percent at one-, five-, and twenty-day horizons. At twenty days, it is the most accurate evaluated model, narrowly outperforming an autoregressive baseline and a random walk while beating dynamic Nelson-Siegel, a Schwartz-style linear twin, and an architecture-matched unstructured GRU by wider margins.
The more important result is not that a neural model dramatically defeats every simple forecasting benchmark. It does not. Commodity curves are persistent and noisy, and the experiment finds that multi-week dynamic predictability is limited. The validation procedure assigns full weight to structured dynamics at one day, partial weight at five days, and no incremental dynamic weight at twenty days, where structured persistence performs best.
This negative result strengthens rather than weakens the case for structure. CARRY-Net matches strong persistence benchmarks while preserving interpretability and economic admissibility. The unstructured GRU, trained with a comparable architecture and budget, performs substantially worse and violates the cash-and-carry bound on 17.7 percent of adjacent twenty-day forecast spreads. CARRY-Net and the structural linear twin produce zero violations because both use the projection layer.
For an operational storage system, this distinction matters. A forecast curve that contains an obvious cash-and-carry arbitrage is not merely aesthetically imperfect. It can corrupt hedge valuation, spread-risk reporting, scenario analysis, and downstream optimization.
CARRY-Net’s broader contribution is therefore not a claim that economic structure creates effortless forecasting alpha. Its contribution is that structure can produce curves and latent states that remain interpretable, admissible, and competitive with strong benchmarks even when the available dynamic signal is weak.
Why Synthetic Data Is Necessary at This Stage
A synthetic market is sometimes treated as a lesser substitute for real data. For this stage of the research, it serves a different purpose.
Real markets reveal futures prices, but they do not reveal the true daily convenience yield. Physical inventory data are often incomplete, released with delays, reported at a regional or aggregate level, and measured at a different frequency from market prices. Even when inventory data are available, they do not necessarily correspond to the exact deliverable supply relevant to a particular futures curve.
A real-data model can produce a plausible convenience-yield series, but the analyst cannot directly verify whether the estimated state is correct. Synthetic data make that verification possible because the data generator exposes every latent variable.
The CARRY-Net generator simulates 2,750 trading days, or roughly eleven crop years. It includes a harvest pulse centered in autumn, steady consumption, demand noise, annual drought regimes, spot-price jumps, seasonality, measurement noise, and full-carry clipping. The final 550 days are held out for testing. Importantly, drought jumps, clipping, and the physical regime process are not fully contained within the parametric family used by the model, so the evaluation is not simply an exercise in fitting the exact data-generating equation.
The synthetic design answers three foundational questions. It tests whether the measurement stage can recover convenience yield from a distorted curve, whether the Working layer can recover the correct decreasing-convex inventory relationship, and whether the full system remains competitive out of sample while respecting arbitrage.
Only after those questions are answered does it make sense to move to real CME curves, where the latent truth is unavailable.
What Generalizes to Real Commodity Markets
The exact numerical threshold and backtest returns do not generalize automatically, but the workflow does.
First, an analyst specifies an economically coherent pricing map for the commodity and delivery system being studied. Interest rates, storage tariffs, seasonal effects, maturity conventions, and relevant arbitrage bounds enter explicitly.
Second, the model inverts the observed curve to estimate the latent convenience yield while accounting for censoring and identification limits.
Third, the analyst estimates a risk-premium process rather than assuming the constant 18 percent value used in the synthetic market. This estimate might incorporate historical returns, commercial hedging pressure, option-implied information, inventory conditions, or regime-dependent macroeconomic variables.
Fourth, the measured convenience yield is compared with the operation’s actual carrying costs and risk tolerance. The result can be a binary sale decision, a staged marketing schedule, or a hedge adjustment.
Fifth, management evaluates the threshold frontier rather than selecting a strategy solely on mean historical return. The relevant objective may include worst-year loss, liquidity at risk, earnings volatility, basis exposure, borrowing constraints, or storage-capacity utilization.
This workflow applies most directly to grains and oilseeds, where storage and crop seasonality are central. With appropriate physical modifications, the same framework can extend to natural gas storage, crude oil and refined products, base-metal warehousing, and other commodities for which physical inventory creates an operational benefit.
The architecture’s design also supports mixed-frequency information. The technical paper proposes using daily CME curves with quarterly USDA Grain Stocks releases as sparse inventory anchors. The observation equation for inventory would become active only on report dates, while the curve-based state measurement would continue daily.
What Does Not Generalize Yet
The synthetic storage experiment benefits from information that a real analyst would not possess.
The first limitation is the risk premium. The decision threshold uses the true 18 percent premium embedded in the generator. A live model must estimate a premium that is likely time varying, regime dependent, and difficult to distinguish from expected spot-price movements. Estimation error in the premium translates directly into threshold error.
The second limitation is the simplified storage cost. A flat 2 percent annual physical storage charge does not capture commercial reality. Actual costs include facility-specific tariffs, loading and unloading fees, shrink, drying, quality discounts, insurance, variable financing rates, and opportunity costs that differ by producer and location.
The third limitation is basis. Futures curves represent standardized deliverable commodities at defined locations, while a producer owns grain at a local cash market. The economically relevant sale decision depends on both the futures component and the local basis, which can move independently because of transportation constraints, regional supply, processor demand, and quality.
The fourth limitation is transaction design. Selling the physical crop is only one response to a scarcity signal. A commercial operator may instead hedge the spread, exchange futures, roll a short hedge, sell a portion of inventory, purchase optionality, or reallocate storage capacity across grades and locations.
The fifth limitation is model uncertainty. Stage A can produce state-standard-error estimates, and Stage B can be ensembled, but the current storage rule acts on a point estimate. A production system should incorporate confidence bands, parameter uncertainty, stress scenarios, and sensitivity to the carry-cost and risk-premium assumptions.
The sixth limitation is sample size. Ten usable harvest decisions are enough to illustrate a mechanism but not enough to support a broad empirical claim. Real validation will require longer histories, multiple commodities, alternative delivery periods, rolling estimation, realistic execution assumptions, and careful control for look-ahead bias.
These limitations do not invalidate the demonstration. They define the work required to transform the instrument from a validated synthetic prototype into a deployable market system.
A More Realistic Decision System
The binary rule used in the backtest is deliberately simple. A more realistic system would treat convenience yield as one input to a continuous inventory policy.
One approach would divide the crop into tranches. The first tranche could be sold when the estimated convenience yield reaches the break-even threshold. Additional tranches could be released as the estimate moves farther into scarcity or as uncertainty around the estimate narrows.
Another approach would incorporate a buffer around the threshold. Instead of responding to every crossing, the system could require convenience yield to exceed the threshold by a confidence margin or remain above it for several days. This would reduce turnover caused by measurement noise.
A third approach would use the threshold to manage hedges rather than physical sales. If the market continues to pay carry, the operator might lock in a favorable calendar spread while retaining basis exposure. If scarcity emerges, the operator might accelerate cash sales or shorten the hedge horizon.
A fourth approach would optimize the decision jointly across inventory, storage capacity, financing limits, and expected basis. The convenience yield would remain the scarcity state, but the action would be selected by a broader operational model.
Finally, the latent inventory and admissible forecast curves could support stress testing. An analyst could shock the inventory state, move the system along the learned Working curve, and decode a futures curve that remains consistent with the carry bound. This would produce economically coherent drought, demand-surge, or storage-capacity scenarios rather than arbitrary parallel price shocks.
The value of the measured state is therefore not confined to a single trading rule. It creates a bridge between futures-curve information and physical inventory decisions.
Reproducing the Experiment
The CARRY-Net repository contains the technical paper, narrative article, market generator, model implementation, automatic-differentiation engine, evaluation metrics, and plotting scripts. The model is written in pure NumPy and distributed under the MIT License.
The repository’s standard workflow installs NumPy and Matplotlib, verifies the custom gradient engine, trains the structured dynamics, trains the unstructured GRU ablation, evaluates all baselines, and regenerates the figures. Training is checkpointed, so runs can be resumed without starting over.
pip install -r requirements.txt python3 gradcheck.py python3 run2.py dyn 40 python3 run2.py gru2 40 python3 run2.py eval python3 make_figs2.py
The repository should be read as part of the research rather than as an appendix to it. The technical report documents the mathematical propositions and failure modes, while the code makes the assumptions inspectable. The generator shows exactly how inventory, droughts, spot dynamics, seasonality, and full-carry clipping are implemented. The metrics file exposes the reported results directly.
This degree of reproducibility is especially important for structure-informed machine learning. Claims such as “arbitrage-free by construction” or “monotone and convex by construction” should be verifiable properties of the implementation, not descriptions applied after training.
The Broader Lesson for Machine Learning
The storage application demonstrates a general modeling principle. When a field has reliable qualitative laws but limited data, those laws should not be supplied merely as additional features. They can often be encoded into the set of functions the model is allowed to represent.
CARRY-Net names the economically meaningful latent variables, constrains the relationships that theory identifies, decodes those variables through a pricing map, and allows flexible learning only where the domain suggests genuine nonlinearity.
The Working layer cannot learn the wrong inventory-scarcity shape. The projection layer cannot emit a curve above the carry boundary. The state dynamics cannot use an unconstrained neural network to replace every part of the economic mechanism. These restrictions reduce flexibility, but they remove flexibility the model should never have needed.
The machine-learning literature on commodities is increasingly emphasizing theory-grounded signals and interpretable structures rather than treating futures as generic time series. Tony Guida’s chapter on machine learning in commodity futures, in the CFA Institute Research Foundation’s 2025 volume on AI in asset management, makes the same case for investment practice: models built from features grounded in commodity economics remain transparent and interpretable rather than becoming black boxes.
The relevant comparison is not “economics versus machine learning.” The more productive question is which parts of the problem should be dictated by economics and which parts should be learned from data.
Conclusion
For nearly a century, commodity economics has described convenience yield as the implicit benefit of physical ownership and the variable connecting inventory scarcity with futures-curve structure. The theory explained why storage becomes less valuable during scarcity, why backwardation emerges when stocks are low, and why identical supply shocks can produce different market responses depending on the available inventory buffer.
What the theory did not provide was a daily instrument for measuring that variable in a market where the observable curve is distorted by seasonality, mean reversion, carrying costs, and full-carry censoring.
CARRY-Net is an attempt to build that instrument. It treats convenience yield as a latent economic state, extracts it from an arbitrage-consistent futures-curve model, links it to inventory through a decreasing-convex Working layer, and ensures that every decoded curve respects the storage-arbitrage boundary.
Applied to the harvest decision, the measured state produces a transparent rule: continue storing while convenience yield remains below the risk-premium-minus-storage-cost threshold, and release inventory when scarcity pushes the estimate above it.
In the synthetic experiment, that rule does not maximize the average return and does not win every year. It preserves 43 percent of passive storage’s mean excess return while reducing the worst annual result from negative 30.5 percent to negative 4.8 percent. It avoids the most damaging drought year not through an unexplained prediction, but because the second drought arrives after the inventory buffer has already been depleted and the curve-implied convenience yield responds exactly as the Theory of Storage predicts.
The result is not proof of a live commodity strategy. It is proof of a method.
An invisible economic variable can be measured. A qualitative theory can be turned into an operational threshold. A neural model can remain flexible without being allowed to violate the economics of the market it represents. Most importantly, the storage decision can be expressed as an explicit trade-off between expected carry and scarcity rather than a seasonal habit or an eyeballed spread.
The instrument existed in the equations for eighty years. CARRY-Net gives it a gauge.
References and Further Reading
Holbrook Working, “The Theory of Price of Storage,” American Economic Review, 1949.
Eugene F. Fama and Kenneth R. French, “Commodity Futures Prices: Some Evidence on Forecast Power, Premiums, and the Theory of Storage,” Journal of Business, 1987.
Yining Zhu and Calum G. Turvey, “Convenience Yields, Implied Price of Storage and the Working Storage Curve,” SSRN working paper, Cornell University, November 2024.
Ming Gu, Wenjin Kang, Dong Lou, and Ke Tang, “Relative Basis and the Expected Returns of Commodity Futures” (previously circulated as “Relative Basis: A Better Measure of the Convenience Yield”), working paper, 2024.
Tony Guida, “Machine Learning in Commodity Futures: Bridging Data, Theory, and Return Predictability,” in AI in Asset Management: Tools, Applications, and Frontiers, CFA Institute Research Foundation, 2025.
CARRY-Net technical report, source code, simulation, metrics, and reproducibility materials.
The market in this article is synthetic and the numbers are method demonstrations, not investment advice. The real-data extension is the next piece in this series.
Brian James Curry is a Kansas City–based data scientist and applied Machine Learning and AI researcher working at the intersection of economic modeling, causal inference, and machine learning — with a standing preference for models where the economics is load-bearing. He is the founder of Vector1 Research, an independent lab exploring marketing economics, cognitive AI architecture, and AI-driven enterprise transformation, with research contributions including Papilon, PyCausalSim, and Memory-Node Encapsulation. His twenty-plus years of enterprise work span Koch Industries, Tractor Supply, Vail Resorts, Hallmark, Garmin, and AT&T.
