EnviroFit — Environmental Intelligence for Hybrid Positioning

Demo MVP · public-data only · k-NN on 18-feature contract · trial sites simulated where G2F metadata is missing ·

EnviroFitenvironmental intelligence for hybrid positioning

Trial Sites—

Site-Years—

Hybrids—

CLIMATE SIMILARITY

0%25%50%75%100%

novelanalog

DEMO DATA · Public sources only · G2F (CC BY 4.0) · WorldClim 2.1 · SoilGrids · NASA POWER · basemap: Natural Earth

Honest methodology

How predictions are made

The honest two-paragraph version

Analog / Migration mode (Pillar I).EnviroFit projects every G2F maize trial site into an 18-dimensional climate & soil feature space, normalises against the training-set mean and std, then for any query (a site, a point, or every cell of the global grid) computes a weighted Euclidean distance over those features with PRD-locked weights summing to 1.0. Similarity is 1 − d / d_max, clamped to [0, 1].

Variety mode (Pillar II — G×E). Per EnviroFit Scientific Foundation §5.5, every hybrid gets a fitted linear reaction norm (Jarquín et al. 2014) of the form y = β₀ + β · x_norm + ε with hierarchical shrinkage toward a population-level β_main. The prior is the temperate-maize (CIMMYT ME-5) average response, fit on 164k G2F observations — not a global maize prior. Hyperparameters (λ values, sparse threshold) are tuned via 5-fold cross-validation; see the Validation section below for what we actually measured.

Honest scope — what this model is and isn't for

The model is useful for relative ranking within the training population (modern North American commercial dent maize, CIMMYT ME-5 temperate climate). It is not a substitute for field validation or breeder judgement.

✅ Relative ranking of regionsfor one hybrid (the algorithm's design goal)
✅ Climate-analog discovery— “where in the world has Iowa-like climate?”
✅ Hypothesis generation for breeders/agronomists
❌ Absolute yield predictionoutside the training envelope — linear extrapolation produces unrealistic values; we calibrate to G2F's observed yield distribution but the upper tail is still extrapolation
❌ Tropical / highland / subtropical maize — the prior was fit on temperate data; predictions outside ME-5 are untrustworthy
❌ Field-level planting decisions — our resolution is ~10 km; real fields need sub-field analysis with local agronomy

A separate methodological caveat: Mahony et al. (2017) Climatic Change 142:495–507 shows climate-analog distance is a weak predictor of ecological function. Iowa-climate-like cells in the Yangtze Delta have different soils, pathogens, pollinators, and local adaptation — a hybrid that thrives in Iowa may fail there for reasons our 18 features cannot capture.

Variety mode — the reaction-norm formula

# Stage 1 — temperate-maize population prior (β_main) on all 164k rows
β_main = (XᵀX + λ I)⁻¹ Xᵀ y    (ridge with λ = 1.0)

# Stage 2 — per-hybrid shrinkage fit, one per hybrid
β_hybrid = (XᵀX + λR)⁻¹ (Xᵀy + λR · β_main)
    where R = diag(0, 1, ..., 1)    # intercept stays unregularised
    λ tuned by 5-fold CV  (see Validation section)

# Predict on the global grid with closed-form prediction interval
y_hat       = β₀ + β · x_norm
Var(y* − ŷ) = σ² · (1 + x*ᵀ · (XᵀX + λR)⁻¹ · x*)

# Quantile-mapped to G2F observed yields (Maraun 2016 bias correction)
y_calibrated = inv_CDF_obs( CDF_pred(y_hat) )

The side panel surfaces each hybrid's centered coefficients (β − β_main) as a bar chart so you can read off what's specific to this variety vs the temperate-maize average response. Every predicted yield is accompanied by a 95 % prediction interval — when the interval is wide, the cell is either data-sparse or out-of-envelope.

Validation — what we actually measured

Per docs/REACTION_NORM_METHODOLOGY.md (Vol 1) and Vol 2's 21-item gap audit, we tune hyperparameters and measure honest predictive performance via leave-one-environment-out (LOEO) cross-validation per Jarquín 2014 standard practice.

LOEO pooled RMSE — error when predicting on a site-year that was held out from the fit. This is the ground-truth number to quote.
Median Pearson r per LOEO fold— the engine's ranking ability. Jarquín 2014 reports 0.45–0.60 for maize. Our number is lower because we use raw G2F yields (not BLUPs) and 18 collinear features. See Vol 2 issues #1 and #7 for the fix path.
Median CV r² per hybrid— honest predictive r² from 5-fold CV. Often substantially worse than the in-sample r² because the latter is the fit's optimisation target. Use the CV value, not the in-sample value, to judge any individual hybrid's fit quality.

The exact numbers from this engine instance are surfaced in /health under validation . They are re-computed via engine/scripts/validate_loeo.py whenever the training data or hyperparameter grid changes.

What we DON'T do

Random forests, neural nets, gradient boosters. PRD §2.2 locks the analog algorithm to weighted Euclidean k-NN. Variety mode adds the reaction-norm regression above. We deliberately use the simplest model family that gives an interpretable distance and an interpretable slope.
Predicting on novel climates. Both modes refuse to recommend on grid cells outside the training distribution. Variety mode flags per-cell out-of-range count(how many of the 18 features lie beyond the hybrid's training range) and restricts the “top regions” list to cells with ≤4/18 extrapolated features. Coverage mode tells the same story globally.
Future-climate prediction. Migration mode applies a simple linear delta scenario and is honest about being a placeholder.

Confidence model (PRD §2.3)

We find the 5 nearest G2F sites to the query, take their mean distance, then compute a proximity score and a variance penalty:

proximity     = 1 - mean_top5_distance / d_max
variance_pen  = min(ensemble_variance / 10.0, 1.0)
score         = proximity × (1 - 0.3 × variance_pen)

>= 0.75 → "high"
>= 0.50 → "medium"
>= 0.30 → "low"
<  0.30 → "novel_climate"

Data sources (all public, all v-locked)

G2F 2024 Prediction Competition · 271 site-years across 46 unique locations, 2014-2023, 5,205 hybrids (CC BY 4.0).
NASA POWER daily weather (shipped with the G2F distribution, not re-fetched).
WorldClim 2.1 · 19 BIO variables + elevation at 2.5' (~5 km).
SoilGrids 2.0 · pH, SOC, clay, sand, bulk density — point-sampled via VRT for G2F sites, server-side aggregated via ISRIC WCS for the global grid.

Caveats we want you to read

22 G2F site-years have metadata_source = sibling_year:* because the G2F metadata CSV was incomplete for those rows. Coordinates were copied from the same site code's other-year records.
1 site (WIH1_2015) has soil values copied from the nearest non-missing G2F neighbour (soil_fill_method = nearest_site).
6 POWER-derived features are linearly calibrated from WorldClim proxies on the 271 site-years; outside the training distribution the calibration extrapolates. Coverage mode is the right place to look for "where this matters".
The global grid is grid_features.zarr at ~10 km, ~2.15M valid land cells (Antarctica below -56° is dropped because SoilGrids has no coverage there).

Performance

Engine boots in ~4 s on a M1 laptop — adds ~1 s on top of the previous ~3 s for fitting 5,027 per-hybrid reaction norms. Holds ~1.2 GB RSS (compact grid, no precomputed site×grid matrix). Each /analogs/* response is one matmul over the (2.15M, 18) compact grid → ~55 ms compute, ~250 ms end-to-end. Variety mode is now a single matmul plus an OOR scan → faster than the old k-NN aggregation (no per-site loop), ~120 ms end-to-end.

References

Jarquín, D., et al. (2014). A reaction norm model for genomic selection using high-dimensional genomic and environmental data. Theoretical and Applied Genetics 127:595–607.
Heslot, N., et al. (2014). Integrating environmental covariates and crop modeling into the genomic selection framework to predict genotype by environment interactions. TAG 127:463–480.
Crossa, J., et al. (2017). Genomic selection in plant breeding: methods, models, and perspectives. Trends in Plant Science 22(11):961–975.

See docs/data-dictionary.md for the column-level contract.

Auto-generated · simulated

Decision memos

Each week the engine compares its top recommendations against any newly-arrived G2F observations and drafts a short memo. Memos are non-binding and reviewed by a human before sending to partners. The five memos below are simulated — the wiring for the live job lands in a follow-up.

2026-W21 · B73/PHN82
Carpathian foreland, RO
high
Recommended
10.9 t/ha
Observed
11.4 t/ha
Δ
+0.5
Pannonian-basin analog held up. Recommend expanding trial network into Moldova for 2027.
2026-W20 · PHK56/DK888
Black Sea coast, UA
medium
Recommended
9.8 t/ha
Observed
7.2 t/ha
Δ
-2.6
Cooler than analog predicted. Drought stress likely. Re-fit recommended with 2026 POWER updates.
2026-W19 · LH123/PHB47
Hokkaido, JP
high
Recommended
10.3 t/ha
Observed
10.0 t/ha
Δ
-0.3
Within tolerance. Tighten the latitude_abs weight? Open issue.
2026-W18 · B73/PHN82
Kazakh steppe, KZ
low
Recommended
8.4 t/ha
Observed
4.1 t/ha
Δ
-4.3
Confidence flag worked: we flagged 'low' and partner ignored. Don't extend.
2026-W17 · PHK56/DK888
Po valley, IT
high
Recommended
11.7 t/ha
Observed
11.5 t/ha
Δ
-0.2
Calibration good. Memo highlighted Po valley as adjacent analog basin worth expanding into.

Memo generator: PRD §6 (planned, not yet built)

Multi-agent council · scripted

Council debate

DECISION UNDER REVIEW

Should we expand B73/PHN82 distribution into Kazakhstan in 2027?

Three personas, three positions — convened automatically when the engine confidence flag is anything other than 'high'.

🌾Agronomist
expand
Our nearest analog — the Pannonian basin — delivered 11.4 t/ha last season, beating the engine's 10.9 t/ha point estimate. Kazakh steppe climate is one σ further out but the soil pH 6.7 and bio15 36% are inside the training envelope. I'd run a 10-site pilot, not commit the whole product line.
ΣStatistician
hold
The engine flagged 'low confidence' for the Kazakh cells in our last memo — mean distance to top-5 G2F sites is 1.4 σ, ensemble variance among contributing yields is 4.7 (t/ha)². That score is below the 'medium' threshold. I'd rather get 5 actual Kazakh site-years into the training set before extrapolating to thousands of hectares.
⚖Risk Officer
kill
Worst-case PnL from the memo retrospective: B73/PHN82 in similar 'low' cells dropped 4.3 t/ha vs forecast (week 18, 2026). At 200 ha and €240/t that's a €207k under-delivery per partner contract. Until coverage improves I want this gated on a signed performance clause.

CONSENSUS

Hold — pilot first. 10-site pilot in Kazakhstan with explicit performance gating before full distribution. Re-convene Q2 2027 with three site-years of ground truth.

Scripted demo · live multi-agent council planned for Phase 4