Modeling Crop Patterns and Yields Under Input and Climate Shocks
Testing hybrid crop models, agent-based crop switching, causal discovery, and conformal intervals against known ground truth, using PyCausalSim
Everything here is reproducible. Paid subscribers get the complete code, the simulated crop and supply models, and all figures: https://brianjamescurry.substack.com/
On February 28, 2026, war closed the Strait of Hormuz. Within weeks urea traded above $850 a tonne, up about 80 percent from February. By June it was back to $453. Somewhere in the middle of that swing, several hundred million farmers made planting decisions.
At the same time, El Nino conditions returned. NOAA now puts the odds of a very strong event peaking between November 2026 and January 2027 at 63 percent. GEOGLAM’s largest concern is Southern Africa, where the main maize season lines up with the forecast peak. In the strong events of 1997/98 and 2015/16, South African maize yields came in about 20 percent below trend and Zimbabwe’s up to 40 percent below.
Markets have already moved. The FAO Food Price Index averaged 133.3 points in August 2026, the highest reading since November 2022, with cereals at their highest since May 2024. The September WASDE cut the US corn yield forecast by 2.2 bushels to 178.5.
Every crop model in production right now learned its parameters from a world where fertilizer supply was boring and El Nino was a tail risk in a footnote. This article is about what breaks when both of those assumptions fail at once, and which modeling choices survive.
I build synthetic worlds where the true causal structure is known, then run the standard methods against them: gradient boosting, hybrid crop models, Markov transition matrices, agent-based models, doubly robust effect estimation, and conformal prediction. Every method gets scored against ground truth. The forward scenarios and the structural interventions use PyCausalSim, my open-source causal simulation library.
The pattern question is harder than the yield question
Most attention goes to yields, because yields are what forecasting competitions score. But supply comes from area times yield, and area is a decision. Farmers reallocate between nitrogen-hungry crops like corn and wheat and nitrogen-fixing crops like soybeans and pulses, and they do it in response to prices that models rarely include.
Look at what actually happened in the US. Corn plantings fell to 95.3 million acres, down 3 percent, and soybeans rose to 85.4 million, up 5 percent. That is a shift, but a small one. Corn area was essentially unchanged from the March intentions report, and it came in above the 94.99 million acres analysts expected. It was still the fourth-largest corn area since 1944.
Two things explain why the switch was smaller than the fertilizer story implies:
The nitrogen was already in the ground. Most US nitrogen is applied the previous fall, six to eight months before planting. The price spike arrived after that decision was made. The acreage response belongs to 2027 and 2028.
Fertilizer was not the only driver. The record 98.8 million corn acres planted in 2025 pulled 2026 down through ordinary rotation. Expanding soybean crush capacity and a US and China trade agreement pulled the other way.
So a model that treats the fertilizer shock as the only cause of the 2026 switch will get the direction right and the magnitude wrong, in the direction of overreaction. Design against that failure mode. It recurs in every section below.
Part 1: what breaks when the world moves
Trees do not extrapolate. They go flat.
Start with a county panel: 400 counties, 12 seasons, nitrogen rates clustered between 140 and 190 pounds per acre, the range a stable fertilizer market produces. True yield follows a Mitscherlich nitrogen response, an exponential heat penalty above 29 C, and a quadratic water response. Those shapes come from agronomy, not from the data.
I fit two models. The first is gradient-boosted trees on nitrogen, soil, precipitation and heat. The second is a hybrid: a declared mechanistic backbone whose coefficients are fitted, plus a boosted-tree model of the backbone’s residuals. That residual structure is the approach a July 2026 review of 143 crop-model-plus-ML studies found in exactly one paper, even though more than 94 percent of the studies that reported a comparison found integration improved accuracy. The review also notes that only 35 of the 143 reported comparable baselines at all.
Then I test three conditions: new seasons in the same regime, a 30 percent nitrogen cut, and a transfer to a drier western region with poorer soils.
RMSE (bu/acre) Pure ML Hybrid (misspecified) Hybrid (correct form) Same regime 6.7 6.2 6.2 Nitrogen cut ~30% 8.4 6.3 5.9 New region (east to west) 29.5 6.2 6.3
In the same regime, the models are indistinguishable. Under the nitrogen shock, the tree model carries a +4.6 bushel bias: it predicts too much yield, because a tree cannot produce a value below the lowest leaf it learned. Below 140 pounds of nitrogen, its prediction is a horizontal line. Move to a new region and the error quadruples.
A fair objection: I gave the hybrid the true functional form. So I also fit a deliberately wrong backbone, a linear-plateau nitrogen response instead of the Mitscherlich curve. It gets 6.3 against the correct form’s 5.9, still far ahead of pure ML. The mechanism has to be roughly right and correctly signed. It does not have to be exact.
This matches what the newest benchmark shows on real data. A 2026 Stanford and Corteva study tested Google DeepMind’s AlphaEarth embeddings on county corn and soybean yields. Within a familiar region the embeddings reached an R-squared around 0.80, matching purpose-built remote sensing pipelines. Trained in the eastern Corn Belt and tested in the western one, R-squared fell to between -0.28 and 0.07. Applied to Argentina, the model failed in every year while traditional remote sensing features held up. The authors also note the annual embeddings cannot support in-season prediction. That limitation bites hardest right now. The US harvest is running as I write this, the WASDE yield estimate moved 2.2 bushels in September, and the Southern Hemisphere season that El Nino threatens has not been planted yet. A model that only produces an annual answer after the fact cannot inform a procurement decision, a stocking policy, or an import tender in the weeks when those decisions get made. In-season updating is not a nice feature for this class of problem. It is the requirement.
Declaring the extrapolation instead of hiding it
Models have to extrapolate. The problem is that they do it silently. A boosted tree asked about 100 pounds of nitrogen returns a number with no indication that it has never seen such a field.
PyCausalSim’s futures module inverts the usual setup: the graph is declared rather than discovered, and every edge can carry a stated validity range. I built a small national model of the 2026 to 2030 seasons with five endogenous nodes:
urea price -> effective nitrogen cost (0.6 x last year + 0.4 x this year, for fall application) effective cost -> nitrogen rate -> corn yield effective cost + rotation + crush demand -> corn share -> planted area -> production
Coefficients are priors with uncertainty, not point estimates: the long-run corn share, the rotation alternation term, the acreage response to fertilizer cost, the crush pull, the nitrogen response curvature, and the trend yield. Three scenarios run on the same parameter draws so the comparisons are paired: a pre-war path with crush demand only, the 2026 Hormuz shock, and a prolonged disruption through 2027 planting.
The two nitrogen edges carry declared ranges, so the model reports its own reach. Under the prolonged scenario:
Support: edges extrapolated outside their declared range corn_yield ~ n_rate 2026-2030 up to 97% of draws (flag) effective_urea ~ urea 2027-2028 up to 100% of draws (flag) Conclusions downstream of these edges rest on declared mechanism, not on observed data.
Any 2027 crop forecast published this autumn should carry a statement like that one. A model can be wrong in a way readers can audit, or wrong in a way they cannot.
Does it retrodict?
A forecasting engine that cannot reproduce what already happened has no standing. Backtesting against the three real 2026 anchors:
anchor observed predicted error corn planted 95.3 93.7 -1.7% inside 90% interval soybeans planted 85.4 87.0 +1.9% inside 90% interval corn yield 178.5 181.1 +1.4% inside 90% interval
All three land inside the intervals, and the model makes the same error the market made: it expected a slightly larger switch away from corn than happened. With only three anchors, that is a sanity check rather than evidence of calibration, and the library says so in its own output.
Because the scenarios share parameter draws, the model can also decompose the 2025 to 2026 acreage change by driver:
rotation pullback after the record 2025 corn year -2.8 million acres soybean crush and trade pull -1.5 fertilizer shock -0.9
Fertilizer was the smallest of the three in 2026. In 2027 it becomes the largest: -1.3 million acres in the base shock scenario and -2.8 million if the disruption persists. Anyone attributing this year’s soybean shift mainly to Hormuz has the timing wrong.
Claims you can check
The module can also emit dated, falsifiable claims with probabilities attached, which is unusual for scenario work and is the part I would most like other forecasters to copy:
Claim Pre-war path Hormuz shock Prolonged disruption 2027 US corn plantings below 94.0M acres 0.17 0.40 0.67 2027 US corn yield below 180 bu/acre 0.25 0.40 0.61
These resolve against USDA’s June 2027 Acreage report and its January 2028 crop production summary, and they score with a Brier score when the data arrive. The exported claim file is in the companion code.
Part 2: crop patterns are decisions, not transitions
A transition matrix cannot see a price
The standard way to model crop patterns is a Markov transition matrix estimated from history: given corn this year, what is the probability of corn next year? These matrices are embedded in land-use models, in rotation-aware crop classifiers, and in the pre-season crop maps that reach about 88 percent accuracy on a six-class subset of the USDA Cropland Data Layer.
The matrix contains no prices. Whatever rotation behavior held while it was estimated is assumed to hold forever. To show what that costs, I built 1,500 farms on a small-world network using PyCausalSim’s agent-based module. Each farm has soil quality and a memory of last year’s crop. Each season, a farm compares corn against soybeans using four terms: soil quality, a rotation penalty for corn after corn, the effective nitrogen cost weighted toward the prior fall, and what neighbors did. Farms on poorer land feel the nitrogen cost more, because their yield response to nitrogen is flatter.
I ran it twice on identical random draws: once with stable fertilizer, once with the prolonged shock. Then I estimated a Markov matrix from the pre-shock years and projected it forward, which is what a conventional model would do.
corn share no shock shock fixed Markov 2026 0.516 0.504 0.520 2027 0.521 0.465 0.513 2028 0.510 0.451 0.516 2029 0.521 0.490 0.515
The Markov projection tracks the no-shock world almost exactly and misses the shock entirely. The agents drop 1.2 points in 2026 and 5.6 points in 2027, the same lagged pattern the fall-application timing produces in the real data.
The distribution matters as much as the average. Farms on the poorest third of land cut corn by 7 percentage points. Farms on the best land cut 3. Marginal land switches first, because that is where nitrogen buys the least yield. An aggregate model gets the national number roughly right for the wrong reason and gives you nothing about where the change lands.
What rotation is actually worth
Crop switching raises an immediate follow-up: what does rotation do to yield? Field trials say the benefit is large. Satellite-based causal studies say something more complicated. Kluger, Di Tommaso and Lobell applied causal forests to crop maps across four countries and found that the gain from a simple two-crop rotation averaged about 3.7 times the extra gain from diversifying further. Their estimates are net of changes in fertilizer, since farmers who rotate apply less nitrogen.
Mediation like this is easy to get wrong, so I simulated it with known truth: 8,000 fields where poorer soils rotate more often (selection), rotated fields receive 28 fewer pounds of nitrogen (mediation), and the true agronomic effect of rotation is 8.5 percent. Using PyCausalSim’s doubly robust estimator:
naive difference in means +3.0 bu/acre doubly robust, net effect +12.9 (truth +12.9) DR controlling for nitrogen +16.5 (true agronomic effect +16.8)
The naive comparison is off by a factor of four, because rotation happens on worse land. The doubly robust estimate recovers the net effect, which is what a farmer books. Adding nitrogen to the controls recovers the agronomic effect, which is what a trial measures. Both numbers are correct answers to different questions. Quoting a trial number in a supply model double-counts the nitrogen the farmer never applied.
Part 3: shocks that arrive together
Independence is the most expensive assumption in food security
Global supply models often treat regional harvests as roughly independent, which makes a simultaneous failure across several breadbaskets look nearly impossible. Atmospheric science says otherwise. Kornhuber and colleagues found that specific jet-stream wave patterns raise the odds of simultaneous extreme heat across major growing regions by up to 20 times, and that two or more weeks in those patterns are associated with about 4 percent lower crop production in the affected regions.
I modeled eight regions with production shares, ENSO loadings, and fertilizer import dependence. Each region’s harvest has an idiosyncratic weather component and a teleconnected component. The interesting part is structural: when the ENSO index exceeds 1.5, a structural rule severs the idiosyncratic channel, so the teleconnected component drives the harvest. An edge gets deleted because another variable crossed a threshold, which ordinary do-style interventions cannot express. In the strong scenario, all eight rules fire in every draw.
2027 global production index (1.0 = trend) mean P(shortfall > 4%) Neutral ENSO, pre-war fertilizer 1.000 2% Strong El Nino 0.987 32% El Nino + fertilizer shock 0.955 54% Same marginals, correlations removed 11%
The fourth row carries the argument. Take the exact same regional distributions and shuffle them so the correlations disappear, and the probability of a serious global shortfall falls from 32 percent to 11 percent. The means barely move. The correlation structure barely moves the expected harvest. It moves the size of the shortfall you should plan around.
When the graph itself changes
Effect estimators such as causal forests and doubly robust methods assume you already know which variables to control for. Discovery algorithms work the other way around: they learn the structure from data. The PC algorithm and its latent-variable relative FCI are the standard constraint-based choices, and they matter here because a strong El Nino does not merely shift the distribution of rainfall. It changes which variables drive which.
I simulated one growing region with six variables: the ENSO index, a jet-stream pattern, rainfall, temperature, soil moisture and yield. In neutral years, local weather drives the harvest and the ENSO index barely registers. In a strong event, the index drives rainfall and temperature directly. Running PC on 1,500 seasons from each regime, using the discovery module in PyCausalSim:
neutral years 4 edges found, 4 of 6 adjacencies recovered, 0 involving ENSO strong El Nino 14 edges found, 7 of 8 adjacencies recovered, 4 involving ENSO
A model that learned its graph on neutral years contains no path from ENSO to anything. Under the event, four ENSO-linked edges appear. If your feature set, your control variables or your transfer assumptions were built on the first graph, they are wrong in the second.
The orientation errors are worth dwelling on. PC recovers the skeleton well in both regimes but reverses several arrows, which is expected behavior: constraint-based methods identify a graph only up to an equivalence class, and the remaining direction has to come from time ordering or domain knowledge. Discovery tells you the structure changed. It does not tell you the whole structure, and any pipeline that treats its output as a finished causal graph is overreading it.
Intervals that fail exactly when you need them
If the model output is uncertain, the honest response is an interval. Split conformal prediction is the usual tool, and it comes with a guarantee: calibrate on held-out data and your 90 percent intervals cover 90 percent of the time. The guarantee assumes exchangeability between calibration and test data. A regime shift breaks it.
I trained a regional yield model on years that are mostly ENSO-neutral, calibrated 90 percent conformal intervals on held-out neutral years, then tested in a strong El Nino regime:
neutral test years 87.9% coverage strong El Nino 75.5% strong El Nino, regime-conditional 90.8%
Coverage drops by 12 points in the shifted regime. The fix is straightforward: calibrate separately by regime, which is Mondrian conformal prediction. That restores coverage, and it requires having enough strong-event years to calibrate on, which is exactly the constraint that makes rare events hard.
What I would ask of any crop or supply model right now
Report where the model left its data. Every model extrapolates. The ones worth trusting say where and how far. A support report costs nothing and changes how a forecast should be read.
Declare the mechanism on the edges that matter. A roughly right nitrogen response curve beat a flexible model with no structure under every kind of shift I tested. Getting the shape exactly right mattered far less than having one.
Test transfer, not just held-out accuracy. Report performance in a region the model never saw. The 2026 benchmark results make the case: R-squared of 0.80 in place and below zero across space is a normal result, not an outlier.
Model planting as a decision. Transition matrices encode past behavior as if it were physics. When input prices move outside their historical range, agents that weigh costs, soil and neighbors will track the change and a matrix will not.
Name the estimand. The agronomic effect of rotation and the net effect a farmer realizes differ by 30 percent in my simulation, and both are correct answers to different questions.
Re-learn the graph when the regime changes. Discovery algorithms recovered four ENSO-linked edges in a strong event and none in neutral years. A control set or feature pipeline fixed on the neutral graph is misspecified in the season it matters.
Correlate the shocks. An independence assumption cut the probability of a serious global shortfall by two-thirds in my model without moving the mean. In a strong El Nino year, that is the difference between a stocking policy that holds and one that does not.
Calibrate uncertainty by regime. A 90 percent interval that covers 75 percent of the time in the year that matters is worse than no interval, because it looks like diligence.
Limits
These are synthetic worlds. Their nitrogen curves, teleconnections and farmer decision rules are ones I wrote, which means every method is being tested against my assumptions, not against agronomic reality. They show something relative: which methods degrade gracefully when the world moves, and which fail silently. That ordering has held in every real benchmark I have seen, including the AlphaEarth transfer results.
Several sources are recent working papers and preprints that have not completed peer review, including the transfer benchmark and the rotation study. The fertilizer price path mixes monthly averages with reported spot peaks. The claims in Part 1 are dated, which means they will be either right or wrong by early 2028, and I will publish the score either way.
The 2026 season is not finished as I write this. Harvest is coming in, the WASDE numbers are still moving, and the El Nino event has not peaked. So the case for in-season updating and stated uncertainty is not theoretical this year.
Code
The complete companion code covers all three pillars: the out-of-distribution yield experiment, the forward scenarios with declared extrapolation and backtesting, the agent-based crop switching model with its Markov comparison, the doubly robust rotation analysis, the correlated breadbasket simulation with structural interventions, the PC discovery runs across climate regimes, and the conformal coverage tests. It runs on PyCausalSim 0.2.0 with numpy, scipy, scikit-learn, pandas and matplotlib.
Paid subscribers get the full scripts, the figure files and the exported claim set at https://brianjamescurry.substack.com/. PyCausalSim itself is open source and free: github.com/Bodhi8/pycausalsim.
About the Author
Brian Curry is a Kansas City-based Machine Learning & AI Strategist, Marketing Economist, and Applied Researcher. He drives Vector1, an independent research studio working at the frontiers of cognitive AI, causal inference, marketing economics, and knowledge engineering to build simulation-driven intelligence systems.
Across more than twenty years, Brian has led advanced data science and analytics initiatives at enterprises ranging from $4B to $125B in revenue, including Koch Industries, Tractor Supply, Vail Resorts, Hallmark, Garmin, and AT&T. His work systematically applies economic thinking and causal inference to connect marketing investment to structural outcomes, operational decisions to economic value, and AI adoption to permanent competitive advantage.
He is also the founder of KC AI Lab, a 1,500-member AI/ML education community in the Midwest.
Key Research & Open-Source Projects
GitHub: @Bodhi8
Substack: https://brianjamescurry.substack.com/
Causal Inference & Marketing Economics
- PyCausalSim: A Python framework for simulation-driven causal discovery and evaluation, mapping counterfactual scenarios to isolate true structural drivers.
- Papilon: A complex-system simulation and optimization library optimized for Media Mix Modeling (MMM) and adaptive architectures.
- Daedalus: A causal analytics framework designed for money-flow mapping and macro-economic tracking.
Algorithmic Engineering & Applied Forecasting
- Regimecast: Applied research and tooling for time-series regime detection, structural breaks, and economic forecasting.
- CarryNet: Open-source research and financial modeling focused on commodity carry trade analysis.
- Foodmesh-ml: A machine learning, optimization, and simulation framework engineered for modeling distributed food systems.
Cognitive AI & Semantic Knowledge Architecture
- Memory-Node Encapsulation (MNE): A cognitive AI architecture and novel data structure for artificial episodic memory (NeoCortex-M).
- The Autonomy Economy: A macroeconomic framework projecting the dual impacts of massive productivity growth and extreme structural price deflation in an AI-automated world.
- MeaningFlow: A semantic intelligence framework focused on content modeling, embeddings, clustering, and automated coverage gap analysis.
Contact: brian@vector1.ai | GitHub | Substack
This article may be shared with attribution under Creative Commons Attribution 4.0 International (CC BY 4.0).
References
FAO (2026). FAO Food Price Index, August 2026. Food and Agriculture Organization.
GEOGLAM Crop Monitor (2026). Special Report: El Nino and the 2026/27 season.
Jiang, F., and Hu, Y. (2026). Beyond data-driven: a review of process model and machine learning integration for crop yield prediction. European Journal of Agronomy.
Kluger, D., Di Tommaso, S., and Lobell, D. (2025). Precrop payoffs: causal machine learning reveals large but variable yield benefits of crop rotation in major breadbaskets. arXiv:2506.02384.
Kornhuber, K., et al. (2023). Global emergence of regional heatwave hotspots and concurrent crop failures. Nature Communications.
NOAA Climate Prediction Center (2026). ENSO Diagnostic Discussion, September 2026.
Stanford University and Corteva Agriscience (2026). Harvesting AlphaEarth: benchmarking geospatial foundation model embeddings for crop yield prediction. Preprint.
USDA National Agricultural Statistics Service (2026). Acreage, June 30, 2026.
USDA World Agricultural Outlook Board (2026). World Agricultural Supply and Demand Estimates, September 2026.
World Bank (2026). Commodity Markets Outlook and Pink Sheet monthly prices, 2026.
