Summary of AI vs Physics-Based Weather Forecast Models

AI Weather Forecast Models vs. Physics-Based Models

  • Study compares machine learning (ML) and numerical weather prediction (NWP) models in forecasting Storm Ciarán.
  • ML models (FourCastNet, Pangu-Weather, GraphCast, FourCastNet-v2) accurately capture the synoptic-scale structure of the cyclone.
  • ML models underestimate peak wind amplitudes and have mixed ability in resolving detailed structures for weather warnings.

Numerical Weather Prediction (NWP)

  • NWP transformed atmospheric science in the 20th and 21st centuries.
  • Relies on physical/mathematical understanding, high-performance computing, and Earth system observations.
  • NWP is integral to transport, agriculture, healthcare, and recreation.

Machine Learning (ML) in Weather Prediction

  • ML techniques are increasingly applied to weather prediction.
  • Recent advances in ML and GPUs have enabled a 'new dawn' in ML/AI for weather and climate prediction.
  • WeatherBench dataset and ECMWF's 10-year ML roadmap have boosted ML model development.

ML Model Architectures

  • FourCastNet: Fourier Neural Operators (FNO) with a vision transformer architecture.
  • FourCastNet version 2: Uses spherical FNOs.
  • Pangu-Weather: Earth-specific transformer with hierarchical temporal aggregation.
  • GraphCast: Graph neural networks.

Challenges and Limitations of ML Models

  • ML models primarily produce deterministic forecasts, with probabilistic forecasts under development.
  • ML models are computationally efficient on GPUs/TPUs.
  • Limited studies compare ML and NWP models in capturing impactful weather events.
  • No published studies examine ML model forecasts of extratropical windstorms.

Storm Ciarán Case Study

  • Storm Ciarán affected Europe in November 2023.
  • It serves as a valuable out-of-sample test for ML models.
  • The study compares ML and NWP models' ability to capture the storm's physical structure and impacts.

Storm Ciarán Development

  • Ciarán originated south of Newfoundland around 00 UTC on October 31, 2023.
  • Explosively deepened from 988 hPa to 954 hPa between November 1-2.
  • The lowest recorded pressure was 953 hPa at 06 UTC on November 2, a record for November in England.
  • Deepening rate, 34 hPa in 24 h means that Ciarán was an extratropical cyclone “bomb” Δp=34hPa\Delta p = 34 hPa.

Impacts of Storm Ciarán

  • Maximum wind speed of 65 knots recorded on the Normandy coast.
  • Gusts exceeded 100 knots (51 m s-1) in Brittany, with a max of 111.7 knots (57.5 m s-1) at Pointe du Raz.
  • Across Northern Europe, at least 16 deaths were reported.
  • Widespread transport disruptions, power outages (1.2 million households in France), and mobile network outages occurred.
  • A T6 tornado in Jersey had estimated winds of 161-186 mph (71–83 m s-1).

Model Performance

  • Track of Ciarán was well forecast by both IFS HRES and ML models.
  • ML models underestimated the peak amplitude of winds, with forecasts being too weak.
  • Economic loss scales as the cube of normalised wind gust speed ∝v3\propto v^3, so underestimation is significant.

Dynamical Structure

  • ML models accurately captured the upper-level jet streak's position and the minimum MSLP associated with Ciarán.
  • There was consensus among ML models regarding the general shape of the cyclone.
  • The shape of the warm sector was well captured.
  • ML models struggled to resolve sharp across-front temperature gradients and the bent-back front.

Key Findings

  • ML models capture the large-scale dynamical drivers of Storm Ciarán well.
  • They accurately represent the synoptic-scale structure, including the cloud head and warm sector.
  • ML models have mixed ability in resolving detailed storm structures.
  • All ML models underestimate the magnitude of the strongest winds.

Discussion

  • ML models accurately forecast rapid MSLP deepening and track, comparable to NWP models.
  • ML models reproduce upper-level flow and many important dynamical features.
  • ML models had significant errors and poorer performance than NWP models in forecasting damaging winds.

Methods

  • Study compared forecasts from four ML models with ECMWF's operational analysis and high-resolution model (CY48R1).
  • ML models were trained on ERA5 dataset.
  • Comparison included forecasts from IFS HRES and ERA5-based forecasts, as well as control members of ensemble forecasts from multiple models.