KILOS

KILOS: Kilometer-scale nonhydrostatic global weather forecasting with IFS-FVM 

PI: Heini Wernli (ETH Zurich)

Co-PIs: Christian Kühnlein, Alexandru Calotoiu, Hanna Joos 

July 1, 2021 – June 30, 2024

Numerical weather prediction and climate research call for improved numerical models, which can better represent small-scale physical processes (e.g., thunderstorm convection) and interactions with complex topography. This requires the design of kilometer-scale, non-hydrostatic global models for weather and climate research, operational weather forecasting, and future climate projections. Given the immense computational costs, this inevitably requires the clever co-design of efficient numerical techniques and of the coupling to physical parameterizations for emerging heterogeneous supercomputing systems at (pre-)exascale. This project KILOS aims at making an important step in this direction, by focusing on the new weather prediction model IFS-FVM of the European Centre for Medium-Range Weather Forecasts (ECMWF), one of the world-leading global weather prediction centers. The project will further develop and implement IFS-FVM at CSCS and LUMI for the use in highly ambitious projects in atmospheric dynamics and predictability at ETH Zurich and beyond, and at the same time provide broad synergies with plans of the ECMWF to use the IFS-FVM for kilometer-scale operational weather prediction in the coming decade.   

The Finite-Volume Module of the Integrated Forecasting System (IFS-FVM) at the ECMWF represents a next-generation non-hydrostatic dynamical core, solving the fundamental fluid flow equations. The IFS-FVM is particularly suitable for extremely high global resolutions and adaptation to increasing multi-level parallelism. This new model IFS-FVM, which is based on a compact-stencil discretization, complements the established, and since decades highly successful, spectral-transform dynamical core of the IFS. In addition to becoming the next-generation dynamical core for the operational application in assimilation and forecasting at the ECMWF, the IFS-FVM will serve the weather and climate community as an openly available, versatile, high-resolution simulation tool for research also outside of the ECMWF. The horizontally unstructured grid of the IFS-FVM enables efficient quasi-uniform resolution meshes over the surface of the assumed spherical Earth and optionally adaptive resolution in sensitive regions (e.g., storm tracks) for later applications. The parallel communication associated with the finite-volume spatial discretization occurs with the nearest neighbors via thin halos, which offers high scalability across a wider supercomputing network.   

Given the need, potential, and ambition of the IFS-FVM, the ECMWF, CSCS, and ETH Zurich have already launched a joint project, referred to here as FVM2GT, with the objective to develop a coding implementation of the IFS-FVM dynamical core based on GT4Py – a Python framework that includes a high-level embedded DSL to write stencil computations – which will enable access to heterogeneous supercomputing architectures, performance portability and scientific productivity. This project KILOS will build upon and collaborate with FVM2GT with the broad objective to further develop the GT4Py implementation of the IFS-FVM dynamical core into a full research model that can be used for studies on atmospheric dynamics and predictability at ETH Zurich and beyond. Reaching this objective requires a range of important steps, in particular the testing of numerical options, the coupling of the dynamical core to physical parameterizations, the development of pre- and post-processing tools, and the careful testing of the IFS-FVM in specific research applications. The project will work with variants of the IFS-FVM of increasing complexity, starting with a limited-area model of the dry atmosphere, then extending the model by coupling simplified parameterization schemes for turbulence and cloud microphysics, and eventually using a full-physics global model, as it can be used for numerical weather prediction.   

More specifically, the project has three work packages to address its main objectives. The central work package P1 will consolidate and perform basic validation of the new GT4py implementation established in the FVM2GT project. It will refine important numerical aspects of the dynamical core and of the coupling to physical parameterizations, extend the model infrastructure in terms of IO, diagnostics, and pre- and postprocessing, and last but not least it will evaluate the performance of the different model variants (limited area, global) on different hardware architectures at CSCS and LUMI, in close collaboration with experts at CSCS. Essential work in P1 will prepare the model for application at kilometer-scale resolution and for specific research at ETH Zurich.  Among the most relevant numerical topics are model level specification, incorporation of resolved orography, and adjustment of the coupling to physical parametrizations with respect to super-stepping and treatment of deep convection. For the work in P1, which will be supervised mainly by co-PI C. Kühnlein, funding is requested for a Postdoc with expertise in numerical model development on HPC systems. The second work package P2 explores the efficient coding implementation of the IFS physical parametrizations together with dynamical core on heterogeneous supercomputing systems. The work here will be informed by the recent achievements in the Scalable Parallel Computing Laboratory (SPCL) at ETH Zurich, based on the Data-Centric programming environment (DaCe) that enables efficient mapping to various hardware architectures. In close collaboration with co-PI A. Calotoiu from SPCL, P2 will augment DaCe with a Fortran front-end (IFS parameterizations are written in Fortran) and explore this approach for selected IFS physical parametrizations starting with the cloud microphysics scheme. A significant part of the work in P2 will be the integration of DaCe code with GT4Py, as well as performance evaluation and comparison to complementary approaches for the translation of the IFS physical parametrizations pursued in FVM2GT. KILOS requests funding for a software developer to support this DaCe-based implementation. The third work package P3 is at the level of research applications in the group of the PI and co-PI H. Joos, using variants of the IFS-FVM to test the physical performance of the model and to explore novel aspects of important atmospheric flows. The limited-area version of IFS-FVM will be used to study idealized model setups of downslope windstorms and orographic precipitation, whereas with the full-physics global version of IFS-FVM a main focus will be on so-called warm conveyor belts and general aspects of cloud-dynamics coupling – one of the main research themes of the PI’s research group. The work in P3 will be performed by a PhD student, co-funded by KILOS and the PI’s resources from ETH Zurich. The overall project will also profit from substantial in-kind contributions from ETH Zurich and the ECMWF.