Handbook of European HPC projects

MAELSTROM

MAchinE Learning for Scalable meTeoROlogy and cliMate

To develop Europe’s computer architecture of the future, MAELSTROM will co-design be-spoke compute system designs for optimal application performance and energy efficiency, a software framework to optimise usability and training efficiency for machine learn-ing at scale, and large-scale machine learning applications for the domain of weather and climate science. 

The MAELSTROM compute system designs will benchmark the applications across a range of computing systems regarding energy consumption, time-to-solution, numerical precision and solution accuracy. 

Customised compute systems will be de-signed that are optimised for application needs to strengthen Europe’s high-performance computing portfolio and to pull recent hardware developments, driven by general machine learning applications, to-ward needs of weather and climate applications. 

The MAELSTROM software framework will enable scientists to apply and compare ma-chine learning tools and libraries efficiently across a wide range of computer systems. A user interface will link application developers with compute system designers, and auto-mated benchmarking and error detection of machine learning solutions will be performed during the development phase. Tools will be published as open source. 

The MAELSTROM machine learning applications will cover all important components of the workflow of weather and climate predictions including the processing of observations, the assimilation of observations to generate initial and reference conditions, model simulations, as well as post-processing of model data and the development of fore-cast products. For each application, bench-mark datasets with up to 10 terabytes of data will be published online for training and ma-chine learning tool-developments at the scale of the fastest supercomputers in the world. MAELSTROM machine learning solutions will serve as blueprint for a wide range of ma-chine learning applications on supercomputers in the future. 

PROJECT’S CONTACT:

Peter Dueben

Extreme scale computing and data driven technologies

Call:
EuroHPC-01-2019

Coordinating Organization:
ECMWF European Centre for Medium-range Weather Forecasts, United Kingdom

Project Timespan
2021-04-01 – 2024-03-31

Other Partners:
  • 4Cast GmbH & Co. KG, Germany
  • E4 Computer Engineering SpA, Italy
  • ETHZ – Eidgenössische Technische Hochschule Zürich, Switzerland
  • FZJ – Forschungszentrum Jülich GmbH, Germany
  • Meteorologisk institutt, Norway 
  • Université du Luxembourg, Luxembourg