DARE: Delivering Agile Research Excellence on European e-Infrastructures

The University of Edinburgh was awarded funding for the EU H2020 project DARE, which will deliver new methods and tools for data-powered collaborative research at extreme scales

Edinburgh will lead the architectural design and contribute significant advances in its fine-grained data-streaming workflow system, dispel4py. This builds on the sustained collaboration between EPCC and the School of Informatics initiated in the e-Science era and on EU projects ADMIRE and VERCE. These built close working relationships with geoscientists, particularly computational seismologists. DARE will deliver new power and agility, rapid paths between R&D and production, to climate-impact modellers and to solid-Earth scientists. Understanding how to improve our ability to handle challenging research data for multi-organisation, multi-national, multi-discipline campaigns is a key goal, part of a longer-term mission to advance our understanding of how to apply computing and data sciences effectively. The three-year, nine-partner project is led by The National Centre for Scientific Research, “Demokritos” (http://www.demokritos.gr), is funded by EINFRA-21-2017 (2,957,500 Euros) and will start in January 2018. Malcolm Atkinson, in CISA will lead the architectural design with a focus on sustainable abstractions and repeatability, EPCC will develop the data-intensive elements based on dispel4py. Alessandro Spinuso, a CISA p/t PhD student, at KNMI will lead the development of tools based on consistent provenance handling.

Official DARE abstract

DARE (Delivering Agile Research Excellence on European e-Infrastructures) aims to provide scientific communities with a unifying hyper-platform and development context to allow for user-friendly and reproducible carrying out of huge data-driven experiments, and rapid prototyping. DARE specifically addresses the requirements of innovating teams of research developers and scientists, who work on the intersection of software engineering and scientific domains, and on data, complexity and computing extremes. The size and complexity of scientific data, as well as the difficulty in formulating domain-specific solutions in reproducible and re-usable ways, may often lead to throw-away, unsustainable end-user products, or long release cycles. This complexity increases exponentially with the size and diversity of input and produced data. Furthermore, widely used big-data technologies and analytics, while they are known to lead to increased productivity in commercial settings, they are often not taken advantage of in scientific. The requirement to deal with diverse exascale data resources dictates the need to ensure and increase productivity through the controlled disruption of the current modus operandi of European RIs. DARE aims to be the technological pivot for this transition, while providing transparent, traceable and developer-friendly bridges over existing infrastructures and services. Building on extensive experience in research e-infrastructures, semantification and the handling of metadata, and on big-data technologies and domain applications, DARE will equip teams of innovators with meaningful abstractions and tools allowing for rapid prototyping of reproducible and efficient research solutions. DARE will improve further and integrate tried and tested programmatic dataflow specification APIs, big-data technologies and provenance/data-lineage solutions to address the requirements of European RIs, initially of EPOS, on Earth science, and IS/ENES2, on climate.