Handbook of European HPC projects

eFlows4HPC

Enabling dynamic and Intelligent workflows in the future EuroHPC ecosystem

Today, developers lack tools that enable the development of complex workflows involving HPC simulation and modelling with data analytics (DA) and machine learning (ML). The eFlows4HPC project aims to deliver a work-flow software stack and an additional set of services to enable the integration of HPC simulation and modelling with big data analytics and machine learning in scientific and industrial applications. 

The software stack will allow to develop innovative adaptive workflows that efficiently use the computing resources and also innovative data management solutions. In order to attract first-time users of HPC systems, the project will provide HPC Workflows as a Service (HPCWaaS), an environment for sharing, reusing, deploying and executing existing workflows on HPC systems. The project will leverage workflow technologies, machine learning and big data libraries from previous open-source European initiatives. Specific optimizations of application kernels tackling the use of accelerators (FPGAs, GPUs) and the European Processor Initiative (EPI) will be performed. To demonstrate the workflow software stack, use cases from three thematic pillars have been selected. 

Pillar I focuses on the construction of DigitalTwins for the design of complex manufactured objects integrating state-of-the-art adaptive solvers with machine learning and data-mining, contributing to the Industry 4.0 vision. Pillar II develops innovative adaptive workflows for climate and the study of Tropical Cyclones (TC) in the context of the CMIP6 experiment. Pillar III explores the modelling of natural catastrophes – in particular, earth-quakes and their associated tsunamis shortly after such an event is recorded. Leveraging two existing workflows, Pillar III will work on integrating them with the eFlows4HPC soft-ware stack and on producing policies for urgent access to supercomputers. The results from the three Pillars will be demonstrated to the target community CoEs to foster adoption and receive feedback. 

PROJECT’S CONTACT:

Rosa M. Badia

HPC and data centric environments and application platforms

Call:
EuroHPC-02-2019

Coordinating Organization:
BSC Barcelona Supercomputing Center, Spain

Project Timespan
2021-01-01 – 2024-02-29

Other Partners:
  • CIMNE – Centre Internacional de Mètodes Numèrics a l’Enginyeria, Spain
  • FZJ – Forschungszentrum Jülich GmbH, Germany
  • Universitat Politècnica de València UPV, Spain
  • Atos (Bull SAS), France
  • DtoK Lab S.r.l., Italy
  • Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici, Italy
  • INRIA – Institut National de Recherche en Informatique et Automatique, France
  • SISSA – Scuola Internazionale Superiore di Studi Avanzati di Trieste, Italy
  • Instytut Chemii Bioorganicznej Polskiej Akademii Nauk – Poznań Supercomputing and Networking Center (PSNC), Poland
  • University of Málaga, Spain
  • INGV – Istituto Nazionale di Geofisica e Vulcanologia, Italy
  • AWI – Alfred-Wegener-Institut, Helmholtz-Zentrum für Polar- und Meeresforschung, Germany
  • Eidgenössisches Department des Innern, Switzerland
  • SIEMENS AG, Germany
  • NGI – Norwegian Geotechnical Institute, Norway