A gradient-boosted classifier trained on eight seasons and evaluated on the entire 2024-25 season, held out and never touched during development. The headline is not the accuracy — it is that the accuracy is reported against a baseline, and that every layer of the 71-feature set was ablated to see what it actually contributes.
12.0 percentage points of accuracy over always picking the home team. An accuracy figure without its baseline is uninterpretable, so the baseline ships beside it.
One feature — the Elo differential — reaches AUC 0.7098. All 71 reach 0.7250. The other 70 features are worth +0.0152 AUC, which is smaller than the ±0.021 spread between my own cross-validation folds.
Read that as a reason to prefer the simpler model, not as proof the features are useless: the Brier score does improve with rolling form (0.2166 → 0.2112), so the extra features buy calibration rather than ranking. Ablation variants are tuned at 30 Optuna trials each for comparability; the headline model above uses 60, which is why its AUC differs slightly from the "all 71" row here.
Bubble-era games carry a 0.6 sample weight in training — a domain judgement, made because the target's base rate genuinely shifted.
Permutation importance under-credits correlated features, and this set is collinear by construction (home / away / diff for ~12 stats). That is exactly why the ablation above measures the layers directly instead of trusting this chart alone.
Worst decile sits 8.4 percentage points off the diagonal. This is what the Brier score summarises — plotted, you can see whether “70% confident” really means 70%.
On the 254 games it called closest to a coin flip it scores 49.6% — chance, correctly. On the 377 it called clear favourites, 80.9%. It knows when it does not know.
Mean absolute error 4.3 wins per team over 82 games. Biggest miss: OKC.
Weekly means on one shared axis. Elo is computed forward through the season, so a rating on any given week uses only games before it.
Every rolling and EWM window is computed after a
.shift(1), so a game's features can never contain that game's own result:
shifted = grp[stat_cols].shift(1) roll = shifted.rolling(window, min_periods=3).mean()
Validation is a temporal holdout — train seasons 1–8, test season 9 — with TimeSeriesSplit(7) for tuning, so no fold ever trains on its own future. Cross-validated AUC 0.682 ± 0.021.
CalibratedClassifierCV uses stratified internal folds, so
calibration peeks across time even though the outer holdout does not.An earlier version of this project included a wagering backtest reporting +35.8% ROI. It was deleted rather than repaired: it priced every game at a constant −110 instead of real market odds, so the "edge" was measured against a market that does not exist. The model is the asset; that layer was not true.