Past forecasts, checked
Explore how the models performed against alpine weather-station observations. Compare events, change the measure, isolate models and sort the underlying scorecards. These are event studies, not a permanent league table: a model that handled one warm storm well may behave differently in the next cold outbreak.
Continuous record · 9–25 August
A warm rain-on-snow event. ECMWF IFS was best on rainfall, ECMWF AIFS came closest on phase and UKMO tracked the snow level best.
One warm event with uneven archive depth. GDPS and ARPEGE have limited coverage; phase is inferred from wet-bulb temperature.
The missing five-day bridge was almost dry. ECMWF AIFS made the smallest rainfall false alarm, followed by JMA and ECMWF IFS.
Only 1.0 mm was observed across the five verification gauges. Rainfall error is useful here; phase and snow-level rankings are exploratory only.
A mostly warm three-day event inside a full week with dry margins. ECMWF IFS was the best all-round weekly model; GDPS tracked snow level best.
The event-only view admits shorter-range UKMO and ICON runs that cannot cover the whole week. ARPEGE has too few event runs to rank.
A warm, windy rain event with no snow at any station. Every model was far too dry; UKMO made the smallest volume error and GFS was alone in forecasting substantial snow.
Snow level and phase are not ranked — the observed snow fraction was 0.0 at every station and every threshold, and the wet-bulb-zero regression solves ~1,550 m above the highest gauge. Rainfall, bias and gusts are ranked on a closed, fully observed window.
One span · no nested events
17 consecutive days
Two different questions, not a fairer-vs-unfairer pair: the broad view is the clean span for the five models present throughout; the matched view adds UKMO and ICON by holding every model to the same operational lead range instead.
The clean span view: one precipitation score and a choice of two snow-level scores per deterministic model across the four adjacent primary periods, for the five models present throughout. Use the all-hours view to give the dry bridge equal standing. The nested 19–21 August and 24–25 August events are excluded, so no date is counted twice. Lower is better.
This stitches non-overlapping verification blocks; it is not one 17-day forecast and it can hide day-to-day error cancellation inside a block.
Block-total error per day
Snow level is a deterministic-only thermal proxy evaluated around precipitating times, not an all-hours temperature score.
Snow level covers 9–22 August only (three of four periods). Period · 23–25 August: Nothing fell as snow at any station at any threshold in the sweep, and the observed wet-bulb-zero height solves at 3,403 m — about 1,550 m above the highest contributing gauge, reaching 6,829 m in the flattest hours. The target is an extrapolation, so this period carries no snow-level weight in the whole-period view.
245 exact observed hours · R² ≥ 0.70 · |lapse| ≥ 0.3 °C/100 m. This gives the dry spell a real voice, but remains a vertical thermal proxy rather than an all-hours 2 m temperature verification.
Same 9–22 August coverage as the precip-weighted view: the R² and lapse gates reject an incoherent profile, not a coherent one that is warm at every station and so solves its zero height above the network.
Precipitation: For models present in every primary period, sum the four published window-total precipitation MAEs and divide by the 17 calendar days. The nested 19–21 August and 24–25 August lenses are excluded.
Precip-weighted snow level: Combine each eligible primary period’s precipitation-weighted snow-level MAE, weighting the periods by the observed precipitation attached to their Victorian snow-level regressions. The near-dry bridge therefore carries very little weight, and a period whose observed level is an extrapolation carries none at all.
All-hours snow level: Compare archived deterministic snow levels with exact-hour observed wet-bulb-zero regressions whether or not precipitation occurred. Keep only regressions with R² at least 0.70 and a cooling rate of at least 0.3 °C per 100 m; do not interpolate across rejected profiles.
| Model | Precip MAE | Precip bias | Wet level MAE | All-hours level MAE | All-hours bias | Thermal cases | Runs |
|---|---|---|---|---|---|---|---|
| ECMWF IFS | 4.95 mm/day | −2.72 mm/day | 307 m | 249 m | −57 m | 8,088 | 281 |
| GDPS | 5.66 mm/day | −3.67 mm/day | 228 m | 201 m | −80 m | 3,153 | 118 |
| ECMWF AIFS | 6.01 mm/day | −5.14 mm/day | 336 m | 246 m | −91 m | 8,088 | 281 |
| GFS | 6.11 mm/day | −3.08 mm/day | 416 m | 322 m | −224 m | 8,220 | 287 |
| JMA | 7.78 mm/day | −6.46 mm/day | 319 m | 275 m | −110 m | 4,462 | 196 |
Adds UKMO and ICON — both capped around five days — by matching all seven models to the same daily lead-day bucket instead of joining every archived run. For each of the fourteen local days, this finds the lowest lead-day bucket at which every model has a run retained across that whole day at every one of the five precipitation resorts, and scores only those runs. 15 days at lead day 0 · 1 day at lead day 1 · 1 day at lead day 2.
This is common lead-day matching across daily blocks, not exact-initialisation matching, and it is not presented as fairer than the five-model broad-archive view above -- the two answer different questions. Matching by lead day still leaves initialisation times, cycle counts and archive depth free to differ between models within a bucket.
Matched-lead-day error per day
Victorian resorts only (buller, falls-creek, hotham); Snow level is a deterministic-only thermal proxy evaluated around precipitating times, not an all-hours temperature score. Scored across 14 of the 17 days in the span, on the same eligibility rule as the broad view.
0.70 R² minimum · |lapse| ≥ 0.3 °C/100 m. This gives the dry spell a real voice, but remains a vertical thermal proxy rather than an all-hours 2 m temperature verification.
Lead-day matching: For each local day, find the lowest lead-day bucket at which every one of the seven models has a run retained across that whole day at every one of the five precipitation resorts, using the earliest archived variant of each initialisation. Average forecast cycles within each model/date/resort cell before combining cells: precipitation combines its 70 date/resort cells with equal weight; the precip-weighted snow level combines its 42 Victorian-resort cells using the observed precipitation weight, the same convention as the broad view; the all-hours snow level averages its cells with equal weight regardless of precipitation. Both snow-level measures are restricted to the dates inside a period whose observed level is a usable target; precipitation uses every date in the span.
Not exact initialisation: Initialisations shared exactly by all seven models at the same site exist but are too thin to rank on: several of the fourteen dates would reuse a single 10 August initialisation out to roughly 134 hours, which is not a meaningful sample.
| Model | Precip MAE | Precip bias | Wet level MAE | All-hours level MAE | All-hours bias | Thermal cases | Runs |
|---|---|---|---|---|---|---|---|
| ECMWF IFS | 4.49 mm/day | −3.16 mm/day | 149 m | 140 m | −41 m | 1,209 | 165 |
| 4.57 mm/day | −1.35 mm/day | 147 m | 136 m | −46 m | 1,167 | 150 | |
| UKMO | 5.04 mm/day | +0.73 mm/day | 103 m | 143 m | −92 m | 1,209 | 165 |
| ECMWF AIFS | 5.13 mm/day | −4.22 mm/day | 165 m | 140 m | −15 m | 1,341 | 180 |
| GFS | 5.15 mm/day | −3.56 mm/day | 165 m | 166 m | −121 m | 1,341 | 180 |
| GDPS | 6.69 mm/day | −5.57 mm/day | 156 m | 147 m | −49 m | 735 | 90 |
| JMA | 8.89 mm/day | −8.48 mm/day | 178 m | 178 m | −102 m | 942 | 125 |
Scorecard explorer
Selected measure
All published measures
Ranks are within each window · click a heading to sort
The badge in each Period cell is the model’s rank within that scoring window for the selected chart measure; it changes when you change the measure above. Bias is forecast minus observed. Phase “miss” is the distance from the observed snow fraction to the model bracket, so smaller is better. A high bracket-hit rate can still come from a very wide bracket. “Limited” and “insufficient” samples remain visible unless you select ranked samples only. In the near-dry 11–15 August bridge, phase and snow-level values remain visible but carry no selected-measure rank.
Read the rankings carefully
Primary periods are consecutive; nested lenses are not. The 19–21 August view sits inside the 16–22 August week and is labelled separately so it cannot read as a fourth period.
Snow level and phase are inferred. The station network does not measure them directly; the verification derives them from wet-bulb temperature and elevation.
Ensembles are precipitation-only here. Their native grid cells sit well below the alpine stations, so their snow fractions are deliberately not ranked.