The PROCESS demonstrators will pave the way towards exascale data services that will accelerate innovation and maximise the benefits of these emerging data solutions. The main tangible outputs of PROCESS are very large data service prototypes, implemented using a mature, modular, generalizable open source solution for user friendly exascale data. The services will be thoroughly validated in real-world settings, both in scientific research and in advanced industry deployments.

To achieve these ambitious objectives, the project consortium brings together the key players in the new data-driven ecosystem: top-level HPC and big data centres, communities – such as LOFAR – with unique data challenges that the current solutions are unable to meet and experienced e-Infrastructure solution providers with an extensive track record of rapid application development. In addition to provide the service prototypes that can cope with very large data, PROCESS addresses the work programme goals by using tools and services with heterogeneous use cases, including exascale learning on medical images, airline revenue management and agricultural simulations based on Copernicus data, also enabling a SME to use the PROCESS ecosystem. This diversity from academic and industry ensures that in addition to supporting communities that push the envelope, the solutions will also ease the learning curve for broadest possible range of user communities. In addition, the chosen open source strategy maximises the potential for uptake and reuse, together with mature software engineering practices that minimise the efforts needed to set up and maintain services based on the PROCESS software releases.

The PROCESS project has achieved its objectives and accelerated the development in the use case’s communities. Implementations for LOFAR will also be valuable for the upcoming Square Kilometre Array (SKA) project, which increases the underlying challenge in the next years to Exa- and Zettabytes. The PROCESS solutions are deployed in three supercomputing centres across Europe, in Munich, Amsterdam and Krakow, enabling already today a testbed for exascale applications of tomorrow.

The figure shows the detailed PROCESS architecture, with the main components end-user portal, orchestration, data and compute service. The heart of the data service is LOBCDER and its Micro-Infrastructure-Containers, enabling a low-overhead solution for the data processing challenge. The already connected centres allow PROCESS to include applications based on HPC, Cloud and Accelerator-systems, and enable deployments also on resources available through the European Open Science Cloud.