Adaptive-Compi: Enhancing Mpi-based applications' performance and scalability by using adaptive compression

Rosa Filgueira, David E. Singh, Jesús Carretero, Alejandro Calderón, Félix García

Research output: Contribution to journalArticlepeer-review

13 Citations (Scopus)

Abstract

This paper presents an optimization of MPI communication, called Adaptive-CoMPI, based on runtime compression of MPI messages exchanged by applications. The technique developed can be used for any application, because its implementation is transparent for the user, and integrates different compression algorithms for both MPI collective and point-to-point primitives. Furthermore, compression is turned on and off and the most appropriate compression algorithms are selected at runtime, depending on the characteristics of each message, the network behavior, and compression algorithm behavior, following a runtime adaptive strategy. Our system can be optimized for a specific application, through a guided strategy, to reduce the runtime strategy overhead. Adaptive-CoMPI has been validated using several MPI benchmarks and real HPC applications. Results show that, in most cases, by using adaptive compression, communication time is reduced, enhancing application performance and scalability.

Original languageEnglish
Pages (from-to)93-114
Number of pages22
JournalInternational Journal of High Performance Computing Applications
Volume25
Issue number1
DOIs
Publication statusPublished - Feb 2011

Keywords

  • adaptive system
  • cluster architecture
  • compression algorithm
  • heuristics
  • MPI library
  • parallel technique

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Adaptive-Compi: Enhancing Mpi-based applications' performance and scalability by using adaptive compression'. Together they form a unique fingerprint.

Cite this