2022

Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast

Bi, Kaifeng, Xie, Lingxi, Zhang, Hengheng et al.

Understand

In this paper, we present Pangu-Weather, a deep learning based system for fast and accurate global weather forecast.

For this purpose, we establish a data-driven environment by downloading $43$ years of hourly global weather data from the 5th generation of ECMWF reanalysis (ERA5) data and train a few deep neural networks with about $256$ million parameters in total.
The spatial resolution of forecast is $0.25^\circ\times0.25^\circ$, comparable to the ECMWF Integrated Forecast Systems (IFS).
More importantly, for the first time, an AI-based method outperforms state-of-the-art numerical weather prediction (NWP) methods in terms of accuracy (latitude-weighted RMSE and ACC) of all factors (e.g., geopotential, specific humidity, wind speed, temperature, etc.) and in all time ranges (from one hour to one week).

Built on

A. Betts, “Non-precipitating cumulus convection and its parameterization,” Quarterly Journal of the Royal Meteorological Society , vol. 99, no. 419, pp. 178–196, 1973
1973
Earlier work this paper cites.
H.-L. Kuo, “Further studies of the parameterization of the influence of cumulus convection on large-scale flow,” Journal of Atmospheric Sciences , vol. 31, no. 5, pp. 1232–1240, 1974
1974
Earlier work this paper cites.
H. Ritchie, C. Temperton, A. Simmons, M. Hortal, T. Davies, D. Dent, and M. Hamrud, “Implementation of the semi-lagrangian method in a high-resolution version of the ecmwf forecast model,” Monthly Weather Review , vol. 123, no. 2, pp. 489–514, 1995
1995
Earlier work this paper cites.
F. Molteni, R. Buizza, T. N. Palmer, and T. Petroliagis, “The ecmwf ensemble prediction system: Methodology and validation,” Quarterly journal of the royal meteorological society , vol. 122, no. 529, pp. 73–119, 1996
1996
Earlier work this paper cites.
M. R. Allen, J. Kettleborough, and D. Stainforth, “Model error in weather and climate forecasting,” in ECMWF Predictability of Weather and Climate Seminar . European Centre for Medium Range Weather Forecasts, Reading, UK, 2002, pp. 279–304
2002
Earlier work this paper cites.
E. Kalnay, Atmospheric modeling, data assimilation and predictability . Cambridge university press, 2003
2003
Earlier work this paper cites.
D. Randall, M. Khairoutdinov, A. Arakawa, and W. Grabowski, “Breaking the cloud parameterization deadlock,” Bulletin of the American Meteorological Society , vol. 84, no. 11, pp. 1547–1564, 2003
2003
Earlier work this paper cites.
R. Pincus, H. W. Barker, and J.-J. Morcrette, “A fast, flexible, approximate technique for computing radiative transfer in inhomogeneous cloud fields,” Journal of Geophysical Research: Atmospheres , vol. 108, no. D13, 2003
2003
Earlier work this paper cites.
A. Arakawa, “The cumulus parameterization problem: Past, present, and future,” Journal of climate , vol. 17, no. 13, pp. 2493–2525, 2004
2004
Earlier work this paper cites.
W. C. Skamarock, J. B. Klemp, J. Dudhia, D. O. Gill, D. M. Barker, W. Wang, and J. G. Powers, “A description of the advanced research wrf version 2,” National Center For Atmospheric Research Boulder Co Mesoscale and Microscale …, Tech. Rep., 2005
2005
Earlier work this paper cites.
T. Palmer, G. Shutts, R. Hagedorn, F. Doblas-Reyes, T. Jung, and M. Leutbecher, “Representing model uncertainty in weather and climate prediction,” Annual Review of Earth and Planetary Sciences , vol. 33, no. 1, pp. 163–193, 2005
2005
Earlier work this paper cites.
P. White, “Newsletter no. 102 - winter 2004/05,” 01 2005. [Online]. Available: https://www.ecmwf.int/node/14623
2005
Earlier work this paper cites.
P. Lynch, “The origins of computer weather prediction and climate modeling,” Journal of computational physics , vol. 227, no. 7, pp. 3431–3444, 2008
2008
Earlier work this paper cites.
D. J. Stensrud, Parameterization schemes: keys to understanding numerical weather prediction models . Cambridge University Press, 2009
2009
Earlier work this paper cites.
I. M. Navon, “Data assimilation for numerical weather prediction: a review,” Data assimilation for atmospheric, oceanic and hydrologic applications , pp. 21–65, 2009
2009
Earlier work this paper cites.
D. Richardson, J. Bidlot, L. Ferranti, A. Ghelli, C. Gibert, T. Hewson, M. Janousek, F. Prates, and F. Vitart, Verification statistics and evaluations of ECMWF forecasts in 2008-2009 . ECMWF Reading, UK, 2009
2009
Earlier work this paper cites.
P. Bougeault, Z. Toth, C. Bishop, B. Brown, D. Burridge, D. H. Chen, B. Ebert, M. Fuentes, T. M. Hamill, K. Mylne et al. , “The thorpex interactive grand global ensemble,” Bulletin of the American Meteorological Society , vol. 91, no. 8, pp. 1059–1072, 2010
2010
Earlier work this paper cites.
K. R. Knapp, M. C. Kruk, D. H. Levinson, H. J. Diamond, and C. J. Neumann, “The international best track archive for climate stewardship (ibtracs) unifying tropical cyclone data,” Bulletin of the American Meteorological Society , vol. 91, no. 3, pp. 363–376, 2010
2010
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems , 2012
2012
Earlier work this paper cites.
E. P. Diaconescu and R. Laprise, “Singular vectors in atmospheric sciences: A review,” Earth-science reviews , vol. 113, no. 3-4, pp. 161–175, 2012
2012
Earlier work this paper cites.

Similar

P. Bauer, A. Thorpe, and G. Brunet, “The quiet revolution of numerical weather prediction,” Nature , vol. 525, no. 7567, pp. 47–55, 2015
2015
Cited alongside, same era.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature , vol. 521, no. 7553, pp. 436–444, 2015
2015
Cited alongside, same era.
X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-c. Woo, “Convolutional lstm network: A machine learning approach for precipitation nowcasting,” Advances in neural information processing systems , vol. 28, 2015
2015
Cited alongside, same era.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Computer Vision and Pattern Recognition , 2016
2016
Cited alongside, same era.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017
2017
Cited alongside, same era.
X. Shi, Z. Gao, L. Lausen, H. Wang, D.-Y. Yeung, W.-k. Wong, and W.-c. Woo, “Deep learning for precipitation nowcasting: A benchmark and a new model,” Advances in neural information processing systems , vol. 30, 2017
2017
Cited alongside, same era.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018
Original
2018
Cited alongside, same era.
H. Hersbach, B. Bell, P. Berrisford, G. Biavati, A. Horányi, J. Muñoz Sabater, J. Nicolas, C. Peubey, R. Radu, I. Rozum et al. , “Era5 hourly data on pressure levels from 1979 to present,” Copernicus climate change service (c3s) climate data store (cds) , vol. 10, 2018
2018
Cited alongside, same era.
——, “Era5 hourly data on single levels from 1979 to present,” Copernicus Climate Change Service (C3S) Climate Data Store (CDS) , vol. 10, 2018
2018
Cited alongside, same era.
K. R. Knapp, H. J. Diamond, J. P. Kossin, M. C. Kruk, C. Schreck et al. , “International best track archive for climate stewardship (ibtracs) project, version 4,” NOAA National Centers for Environmental Information , 2018
2018
Cited alongside, same era.
S. Scher and G. Messori, “Weather and climate forecasting with neural networks: using general circulation models (gcms) with different complexity as a study ground,” Geoscientific Model Development , vol. 12, no. 7, pp. 2797–2809, 2019
2019
Cited alongside, same era.
J. A. Weyn, D. R. Durran, and R. Caruana, “Can machines learn to predict weather? using deep learning to predict gridded 500-hpa geopotential height from historical weather data,” Journal of Advances in Modeling Earth Systems , vol. 11, no. 8, pp. 2680–2693, 2019
2019
Cited alongside, same era.

Then

H. Hersbach, B. Bell, P. Berrisford, S. Hirahara, A. Horányi, J. Muñoz-Sabater, J. Nicolas, C. Peubey, R. Radu, D. Schepers et al. , “The era5 global reanalysis,” Quarterly Journal of the Royal Meteorological Society , vol. 146, no. 730, pp. 1999–2049, 2020
2020
Later among the works it cites.
H. Olafsson and J.-W. Bao, Uncertainties in Numerical Weather Prediction . Elsevier, 2020
2020
Later among the works it cites.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020
Original
2020
Later among the works it cites.
C. K. Sønderby, L. Espeholt, J. Heek, M. Dehghani, A. Oliver, T. Salimans, S. Agrawal, J. Hickey, and N. Kalchbrenner, “Metnet: A neural weather model for precipitation forecasting,” arXiv preprint arXiv:2003.12140 , 2020
Original
2020
Later among the works it cites.
P. Malakar, A. Kesarkar, J. Bhate, V. Singh, and A. Deshamukhya, “Comparison of reanalysis data sets to comprehend the evolution of tropical cyclones over north indian ocean,” Earth and Space Science , vol. 7, no. 2, p. e2019EA000978, 2020
2020
Later among the works it cites.
M. G. Schultz, C. Betancourt, B. Gong, F. Kleinert, M. Langguth, L. H. Leufen, A. Mozaffari, and S. Stadtler, “Can deep learning beat numerical weather prediction?” Philosophical Transactions of the Royal Society A , vol. 379, no. 2194, p. 20200097, 2021
2021
Later among the works it cites.
J. A. Weyn, D. R. Durran, R. Caruana, and N. Cresswell-Clay, “Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models,” Journal of Advances in Modeling Earth Systems , vol. 13, no. 7, p. e2021MS002502, 2021
2021
Later among the works it cites.
H. Bao, L. Dong, and F. Wei, “Beit: Bert pre-training of image transformers,” arXiv preprint arXiv:2106.08254 , 2021
Original
2021
Later among the works it cites.
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning . PMLR, 2021, pp. 8748–8763
2021
Later among the works it cites.
Q. Jiang, W. Li, Z. Fan, X. He, W. Sun, S. Chen, J. Wen, J. Gao, and J. Wang, “Evaluation of the era5 reanalysis precipitation dataset over chinese mainland,” Journal of hydrology , vol. 595, p. 125660, 2021
2021
Later among the works it cites.
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 10 012–10 022
2021
Later among the works it cites.
S. Ravuri, K. Lenc, M. Willson, D. Kangin, R. Lam, P. Mirowski, M. Fitzsimons, M. Athanassiadou, S. Kashem, S. Madge et al. , “Skilful precipitation nowcasting using deep generative models of radar,” Nature , vol. 597, no. 7878, pp. 672–677, 2021
2021
Later among the works it cites.
B. Fildier, W. D. Collins, and C. Muller, “Distortions of the rain distribution with warming, with and without self-aggregation,” Journal of Advances in Modeling Earth Systems , vol. 13, no. 2, p. e2020MS002256, 2021
2021
Later among the works it cites.
L. Magnusson, S. Majumdar, R. Emerton, D. Richardson, M. Alonso-Balmaseda, C. Baugh, P. Bechtold, J.-R. Bidlot, A. Bonanni, M. Bonavita, N. Bormann, A. Brown, P. Browne, H. Carr, M. Dahoui, G. D. Chiara, M. Diamantakis, D. Duncan, S. English, R. Forbes, A. J. Geer, T. Haiden, S. Healy, T. Hewson, B. Ingleby, M. Janousek, C. Kuehnlein, S. Lang, S.-J. Lock, T. McNally, K. Mogensen, F. Pappenberger, I. Polichtchouk, F. Prates, C. Prudhomme, F. Rabier, P. de Rosnay, T. Quintino, and M. Rennie, “Tropical cyclone activities at ecmwf,” no. 888, 10 2021. [Online]. Available: https://www.ecmwf.int/node/20228
2021
Later among the works it cites.
R. Keisler, “Forecasting global weather with graph neural networks,” arXiv preprint arXiv:2202.07575 , 2022
Original
2022
Closest in time.
J. Pathak, S. Subramanian, P. Harrington, S. Raja, A. Chattopadhyay, M. Mardani, T. Kurth, D. Hall, Z. Li, K. Azizzadenesheli et al. , “Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators,” arXiv preprint arXiv:2202.11214 , 2022
Original
2022
Closest in time.
Y. Hu, L. Chen, Z. Wang, and H. Li, “Swinvrnn: A data-driven ensemble forecasting model via learned distribution perturbation,” arXiv preprint arXiv:2205.13158 , 2022
Original
2022
Closest in time.
T. Nakaegawa, “High-performance computing in meteorology under a context of an era of graphical processing units,” Computers , vol. 11, no. 7, p. 114, 2022
2022
Closest in time.
S. Bourdin, S. Fromang, W. Dulac, J. Cattiaux, and F. Chauvin, “Intercomparison of four tropical cyclones detection algorithms on era5,” EGUsphere , pp. 1–43, 2022
2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…