dispel4py: Agility and scalability for data-intensive methods using HPC

Rosa Filgueira, Malcolm P. Atkinson, Amrey Krause

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

Today’s data bonanza and increasing computational power provide many new opportunities for combining observations with sophisticated simulation results to improve complex models and make forecasts by analyzing their relationships. This should lead to well-presented actionable information that can support decisions and contribute trustworthy knowledge. Practitioners in all disciplines: computational scientists, data scientists and decision makers need improved tools to realize such potential. The library dispel4py is such a tool. dispel4py is a Python library for describing abstract workflows for distributed data-intensive applications. It delivers a simple abstract model in familiar development environments with a fluent path to production use that automatically addresses scale without its users having to reformulate their methods. This depends on optimal mappings to many current HPC and data-intensive platforms.

Original languageEnglish
Title of host publicationConquering Big Data with High Performance Computing
EditorsRitu Arora
PublisherSpringer
Pages107-136
Number of pages30
ISBN (Electronic)9783319337425
ISBN (Print)9783319337401
DOIs
Publication statusPublished - 2016

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'dispel4py: Agility and scalability for data-intensive methods using HPC'. Together they form a unique fingerprint.

Cite this