Vector 1
Forecasting & Simulation

Forecasting Corn Futures: A Modeling Ladder for the Foundation-Model Era

Brian J. Curry
Brian J. Curry
May 21, 2026 · 10 min read

Why the modeling ladder remains essential — and what the AI-enabled version looks like.

Context

Predictive modeling in 2026 sits in an awkward middle. The floor — fit a model in scikit-learn, ship it — has commoditized to the point where any technically literate user with a chat interface can produce a passable gradient-boosted model on clean data. The ceiling — designing decision-grade systems that handle uncertainty, causality, and deployment honestly — has appreciated, not depreciated.

A useful exercise for senior practitioners is to walk the full modeling ladder end to end on a real, messy problem, evaluated honestly with multi-horizon, distributional metrics on walk-forward cross-validation. This piece is the methodology and design write-up for one such exercise: forecasting front-month corn futures using a ladder that spans naive baselines through statsforecast through LightGBM through neural and foundation models.

The code lives on GitHub; a Part 2 will publish results, including the cases where modern methods lose.

Why corn futures

The benchmark matters more than most articles admit, so the choice deserves defense.

Corn front-month futures (CME ZC) are well-suited to a methodology study for four reasons:

  1. Real economics. ~$80B annual US market, deep futures liquidity, real hedging demand from farmers, processors, ethanol producers, and feed operators.
  2. Estimable structure. Strong seasonality from the crop calendar — planting in April–May, pollination in July, harvest in October–November — gives classical methods something honest to chase.
  3. Exogenous drivers that matter. USD strength, energy prices (ethanol consumes roughly 40% of US corn), substitute-crop dynamics (soy, wheat), and weather in the growing belt. The problem is genuinely multivariate.
  4. Forgiving but not trivial. Sufficient daily history to train on, sufficient volatility (regime shifts in 2012, 2020, 2022) to expose weak methods, but not so adversarial that every method is hopeless.

Overly clean benchmarks make every method look the same. Overly adversarial benchmarks make every method look the same. Corn sits in the middle.

The thesis the ladder defends

A particular flavor of contemporary modeling discourse argues: point Chronos at it, the job is done. This deserves direct engagement.

Foundation forecasting models — Chronos, TimesFM, Moirai — are real and impressive. On standard benchmarks, zero-shot, they often beat naive baselines, sometimes beat classical methods, and occasionally beat tuned ML. They constitute a new tier in the stack and they belong there.

But “belonging in the stack” is different from “replacing the stack.” The only honest way to determine what foundation models actually buy is to compare them against a properly built ladder. Skipping the ladder leaves no way to know whether a zero-shot foundation model is actually better than a seasonal naive baseline that takes three lines of code. Production forecasting systems that lose to seasonal naive in walk-forward CV are not uncommon in industry, and the failure mode is consistent: nobody checked.

The ladder is not legacy infrastructure. It is the only mechanism for calibrating what newer tools are worth.

Project architecture

The project is structured as clean modules around a unified model interface:

corn-forecast/
├── run_baseline.py          # orchestrator
└── src/
    ├── data.py              # data pulls, caching
    ├── features.py          # lag / rolling / calendar / return features
    ├── splits.py            # walk-forward CV iterator
    ├── models.py            # unified model interface
    ├── metrics.py           # point + distributional metrics
    ├── evaluate.py          # walk-forward evaluation harness
    └── plotting.py          # diagnostic visuals

Every model exposes the same interface:

class Model:
    def fit(self, train: pd.DataFrame) -> "Model": ...
    def predict(self, test: pd.DataFrame) -> pd.DataFrame: ...

Predictions return the same shape across model families:

  • unique_id, ds, y_hat — point forecast
  • Optional y_q0.1, y_q0.5, y_q0.9 — quantile forecasts
  • Optional y_lo_80, y_hi_80, y_lo_95, y_hi_95 — interval bounds

This common interface allows the evaluation harness to treat naive, classical, ML, neural, and foundation models identically. Adding a new model requires roughly thirty lines.

Data: corn futures plus drivers

Daily front-month corn futures from Yahoo Finance (ZC=F), 2012 onward, plus a small set of exogenous drivers also pulled from yfinance to minimize the dependency tree:

Symbol Variable Rationale DX-Y.NYB USD index Corn is dollar-denominated globally; USD strength suppresses export demand CL=F WTI crude Energy costs propagate to fertilizer, transport, and ethanol economics NG=F Natural gas Direct input to nitrogen fertilizer production ZS=F Soybean futures Substitute crop — acreage competes; spread carries information ZW=F Wheat futures Cross-commodity feed substitution

This is not a Kaggle-winning feature set. A competitive feature set would add USDA WASDE supply/demand reports, NOAA growing-belt weather, ethanol production from the EIA, and CFTC commitment-of-traders positioning. The deliberate choice here is to hold feature engineering constant across the ladder so the comparison is honest. Additional features tend to help every method roughly equally; the question this study addresses is which modeling family wins on a fair playing field.

Walk-forward evaluation done right

This is where most forecasting studies quietly cheat. K-fold CV on time series, single train/test splits, evaluation only at the end of the test set — each produces a number that misrepresents how the model will actually perform in production.

The harness uses expanding-window walk-forward CV:

@dataclass
class WalkForwardSplit:
    initial_train_size: int   # 504 days (~2 years)
    horizon: int              # 21 days (~1 month)
    step: int                 # 21 days, non-overlapping
    
    def split(self, df):
        n = len(df)
        train_end = self.initial_train_size
        while train_end + self.horizon <= n:
            yield (
                df.iloc[:train_end],
                df.iloc[train_end:train_end + self.horizon],
            )
            train_end += self.step

This produces roughly 150 folds across the dataset. Each model refits from scratch in each fold. Metrics are aggregated across folds as full distributions rather than point averages, exposing variability of model performance rather than just central tendency. Whether a model is consistently better is typically more interesting than whether it has a marginally better mean RMSE.

Multi-horizon evaluation matters: this study reports at horizons of 1, 5, 10, and 21 trading days. Different methods win at different horizons. ARIMA-class models often dominate at h=1 and decay rapidly. Foundation models tend to hold up better at longer horizons. Feature-rich ML methods often peak in the middle. Single-horizon reporting hides this entirely.

The metric stack

Point metrics (MAE, RMSE) are necessary but not sufficient. The full stack:

  • MAE, RMSE — scale-dependent; useful only for within-study comparison.
  • MASE — mean absolute scaled error, normalized against a naive baseline on the training set. Scale-free and comparable across studies. MASE > 1 indicates worse-than-naive performance.
  • Pinball loss at multiple quantiles — proper scoring rule for quantile forecasts. Asymmetric penalty matching the quantile level.
  • CRPS (continuous ranked probability score) — proper scoring rule for full distributional forecasts. Approximated here by averaging pinball loss across a quantile grid.
  • Empirical coverage — for a nominal 80% interval, does it actually cover 80% of test points? Most “probabilistic” models are badly miscalibrated out of the box.

One point that practitioners consistently underweight: an overconfident model is dangerous in a way that an inaccurate-but-honest model is not. A point forecast that is 5% off but reports a 30% interval is useful. A point forecast that is 1% off but reports a 0.5% interval — and then fails when the next observation lands 3% away — is dangerous. In commercial contexts where positions or commitments are sized to forecast confidence, calibration of uncertainty matters more than sharpness of the point estimate.

Tier 0: baselines

class SeasonalNaive:
    def __init__(self, season_length=252):
        self.season_length = season_length
    
    def fit(self, train):
        self.history_ = train["y"].values
        return self
    
    def predict(self, test):
        n, h = len(self.history_), len(test)
        preds = [self.history_[n - self.season_length + i] for i in range(h)]
        return test[["unique_id", "ds"]].assign(y_hat=preds)

Seasonal naive on a 252-trading-day lag captures the crop-cycle reflex. A model that fails to beat this in walk-forward CV is not doing useful work. The frequency with which sophisticated models fail this test is the standard motivation for the ladder approach: “this LSTM achieves RMSE of 4” is uninterpretable without the seasonal-naive RMSE for comparison.

Tier 1: classical (statsforecast)

The Nixtla ecosystem is the modern home for statistical forecasting in Python. AutoETS and AutoARIMA perform model selection internally and produce uncertainty intervals by default.

from statsforecast import StatsForecast
from statsforecast.models import AutoARIMA, AutoETS

sf = StatsForecast(
    models=[AutoARIMA(season_length=252), AutoETS(season_length=252)],
    freq="B", n_jobs=1,
)
sf.fit(train_df)
forecasts = sf.predict(h=21, level=[80, 95])

Classical methods offer principled uncertainty intervals (conditional on residual assumptions holding), efficient estimation on small data, and interpretability. They do not natively leverage exogenous features without explicit specification and struggle with nonlinear interactions.

A reasonable prior on corn is that ETS dominates at h=1 — the seasonal exponential smoother is difficult to beat on near-term moves in a seasonal series — and decays through h=21 as the structural assumption is stretched.

Tier 2: ML with engineered features (LightGBM)

The bulk of production forecasting actually lives here, despite the discourse. LightGBM with lag, rolling-window, calendar, and return-volatility features handles nonlinearity, captures interactions, leverages exogenous variables natively, and trains in seconds.

Two specifics matter.

First, all rolling and lag features must be properly shifted to avoid leakage:

def add_rolling_features(df, target="y", windows=(5, 21, 63)):
    df = df.copy()
    shifted = df[target].shift(1)  # critical: shift before window op
    for w in windows:
        df[f"{target}_rollmean{w}"] = shifted.rolling(w).mean()
        df[f"{target}_rollstd{w}"]  = shifted.rolling(w).std()
    return df

A 5-day rolling mean computed without the shift includes today’s value in today’s feature. Walk-forward CV does not save against this — the leakage is within-row, not across time. This error appears in published academic forecasting work.

Second, quantile regression provides distributional output from LightGBM without additional modeling infrastructure:

for q in (0.1, 0.5, 0.9):
    params_q = {**params, "objective": "quantile", "alpha": q}
    models[f"q{q}"] = lgb.train(params_q, dataset, num_boost_round=500)

Three trained models, one per quantile, yield a respectable distributional forecast. Less principled than a fully Bayesian model, but fast and operationally honest about uncertainty.

Tier 3: modern deep (neuralforecast)

N-BEATS, NHITS, TFT via the neuralforecast library. The pitch: less feature engineering, better long-horizon performance, principled probabilistic outputs when paired with a quantile loss.

Empirically, deep methods on a single commodity series are roughly competitive with tuned LightGBM. They begin to dominate when multiple related series — multiple commodities, grades, or regions — allow parameter sharing. For a single-series benchmark, they remain competitive but not transformative.

Stubbed in the repo, with the harness ready to ingest them.

Tier 4: foundation models

The new tier, and where the AI-enabled angle becomes concrete. Chronos (Amazon) and TimesFM (Google) are pretrained on billions of time series points and produce zero-shot forecasts on series they have never seen. Given history, they return a forecast distribution.

from chronos import ChronosPipeline
import torch

pipeline = ChronosPipeline.from_pretrained(
    "amazon/chronos-t5-small",
    device_map="cuda" if torch.cuda.is_available() else "cpu",
)
forecast = pipeline.predict(
    context=torch.tensor(train["y"].values),
    prediction_length=21,
    num_samples=100,
)

The interesting question is not whether Chronos works in a basic sense — it does. The questions worth answering:

  1. Does zero-shot Chronos beat tuned LightGBM with proper features on corn?
  2. Does fine-tuning Chronos on the target series close any remaining gap?
  3. Does ensembling Chronos with classical and ML methods beat any individual method?
  4. At which horizons does each method dominate?

This kind of head-to-head comparison does not appear in foundation-model marketing material. Producing it cleanly is what makes the study worth publishing.

The AI-enabled overlay

“AI-enabled predictive modeling” is sometimes treated as marketing language. The substance, made concrete:

  1. LLMs collapse implementation cost. A scaffold of this scope, which would have been a week of work in 2020, is now an afternoon with LLM-assisted coding. The bottleneck shifts from coding to design judgment — the work that senior practitioners specialize in.
  2. Foundation models are baselines and ensemble components. Chronos belongs in the modeling ladder by default. It costs nothing to run zero-shot, and any signal it adds in an ensemble earns it a weight.
  3. LLMs as evaluation critics. Running methodology through an LLM specifically tasked with finding leakage, statistical errors, and unjustified assumptions catches issues that the original author misses — not because the LLM is smarter, but because it lacks anchoring on prior design choices.
  4. Structured data extraction. USDA WASDE reports are PDFs of unstructured text. Pulling supply/demand estimates from them was previously a parsing nightmare. LLMs reduce it to a one-shot prompt. This adds features the old workflow could not economically produce.

The framing is not “LLMs replace modeling.” It is “LLMs enable modeling that is better and faster than what is possible without them.”

Scope of the study

The full evaluation produces:

  • A ranked comparison of five tiers of forecasting methods on corn futures, with confidence intervals on relative performance
  • Multi-horizon evaluation showing where each method peaks
  • Calibration diagnostics indicating which methods are honestly uncertain
  • A clean GitHub repository that runs end to end

The study does not answer:

  • Whether corn futures should be traded on the basis of these forecasts (they should not, on this alone)
  • Whether the methods generalize to other commodities (likely, with caveats; testing requires a separate study)
  • Whether feature expansion changes the ordering (almost certainly, on the margin — this study deliberately holds features constant)

The follow-up piece reports results, surprises, and methodological revisions, with the GitHub link.

Conclusion

The premise of this work is that predictive modeling — performed with full uncertainty quantification, honest evaluation, and AI-native tooling — is more valuable in 2026 than it was in 2020, not less.

The competitive advantage for a modeler is no longer speed of fitting a model. That is now a free good. The advantage is judgment: knowing what to predict, how to evaluate, what to trust, and what to do with the answer. The corn ladder is one vehicle for sharpening that judgment. The same scaffolding extends naturally to natural gas, power load, freight rates, insurance claim severity, and other prediction problems with commercial stakes.

Part 2 — results, surprises, and what the ladder produced — forthcoming.

About the Author

Brian Curry is a Kansas City-based Data Scientist, AI expert, and founder of Vector1 Research, where he explores the frontier of Artificial Intelligence, Causal Inference, Agentic Systems, and Economic Modeling.

His work focuses on building intelligent systems that reveal the causal structures driving performance across marketing, content, and economics. He is the creator of PyCausalSim (causal discovery through simulation), Papilon (complex system optimization), Daedalus (causal and economic analysis), and Memory-Node Encapsulation (data structures for artificial episodic memory).

Related research