Exploiting data compression in collective I/O techniques

Rosa Filgueira; David E. Singh; Juan C. Pichel; Jesús Carretero

doi:10.1109/CLUSTR.2008.4663811

Exploiting data compression in collective I/O techniques

Rosa Filgueira^*, David E. Singh, Juan C. Pichel, Jesús Carretero

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

4 Citations (Scopus)

Abstract

This paper presents Two-Phase Compressed I/O (TPC I/O,) an optimization of the Two-Phase collective I/O technique from ROMIO, the most popular MPI-IO implementation. In order to reduce network traffic, TPC I/O employs LZO algorithm to compress and decompress exchanged data in the inter-node communication operations. The compression algorithm has been fully implemented in the MPI collective technique, allowing to dynamically use (or not) compression. Compared with Two-Phase I/O, Two-Phase Compressed I/O obtains important improvements in the overall execution time for many of the considered scenarios.

Original language	English
Title of host publication	2008 IEEE International Conference on Cluster Computing
Publisher	IEEE
Pages	479-485
Number of pages	7
ISBN (Print)	9781424426393
DOIs	https://doi.org/10.1109/CLUSTR.2008.4663811
Publication status	Published - 31 Oct 2008
Event	2008 IEEE International Conference on Cluster Computing - Tsukuba, Japan Duration: 29 Sept 2008 → 1 Oct 2008

Conference

Conference	2008 IEEE International Conference on Cluster Computing
Abbreviated title	ICCC 2008
Country/Territory	Japan
City	Tsukuba
Period	29/09/08 → 1/10/08

ASJC Scopus subject areas

Software
Hardware and Architecture
Signal Processing

Access to Document

10.1109/CLUSTR.2008.4663811

Cite this

@inproceedings{7df78f528e7943ffbf804a480548a907,

title = "Exploiting data compression in collective I/O techniques",

abstract = "This paper presents Two-Phase Compressed I/O (TPC I/O,) an optimization of the Two-Phase collective I/O technique from ROMIO, the most popular MPI-IO implementation. In order to reduce network traffic, TPC I/O employs LZO algorithm to compress and decompress exchanged data in the inter-node communication operations. The compression algorithm has been fully implemented in the MPI collective technique, allowing to dynamically use (or not) compression. Compared with Two-Phase I/O, Two-Phase Compressed I/O obtains important improvements in the overall execution time for many of the considered scenarios.",

author = "Rosa Filgueira and Singh, {David E.} and Pichel, {Juan C.} and Jes{\'u}s Carretero",

year = "2008",

month = oct,

day = "31",

doi = "10.1109/CLUSTR.2008.4663811",

language = "English",

isbn = "9781424426393",

pages = "479--485",

booktitle = "2008 IEEE International Conference on Cluster Computing",

publisher = "IEEE",

address = "United States",

note = "2008 IEEE International Conference on Cluster Computing, ICCC 2008 ; Conference date: 29-09-2008 Through 01-10-2008",

}

TY - GEN

T1 - Exploiting data compression in collective I/O techniques

AU - Filgueira, Rosa

AU - Singh, David E.

AU - Pichel, Juan C.

AU - Carretero, Jesús

PY - 2008/10/31

Y1 - 2008/10/31

N2 - This paper presents Two-Phase Compressed I/O (TPC I/O,) an optimization of the Two-Phase collective I/O technique from ROMIO, the most popular MPI-IO implementation. In order to reduce network traffic, TPC I/O employs LZO algorithm to compress and decompress exchanged data in the inter-node communication operations. The compression algorithm has been fully implemented in the MPI collective technique, allowing to dynamically use (or not) compression. Compared with Two-Phase I/O, Two-Phase Compressed I/O obtains important improvements in the overall execution time for many of the considered scenarios.

AB - This paper presents Two-Phase Compressed I/O (TPC I/O,) an optimization of the Two-Phase collective I/O technique from ROMIO, the most popular MPI-IO implementation. In order to reduce network traffic, TPC I/O employs LZO algorithm to compress and decompress exchanged data in the inter-node communication operations. The compression algorithm has been fully implemented in the MPI collective technique, allowing to dynamically use (or not) compression. Compared with Two-Phase I/O, Two-Phase Compressed I/O obtains important improvements in the overall execution time for many of the considered scenarios.

UR - http://www.scopus.com/inward/record.url?scp=57949117133&partnerID=8YFLogxK

U2 - 10.1109/CLUSTR.2008.4663811

DO - 10.1109/CLUSTR.2008.4663811

M3 - Conference contribution

AN - SCOPUS:57949117133

SN - 9781424426393

SP - 479

EP - 485

BT - 2008 IEEE International Conference on Cluster Computing

PB - IEEE

T2 - 2008 IEEE International Conference on Cluster Computing

Y2 - 29 September 2008 through 1 October 2008

ER -

Exploiting data compression in collective I/O techniques

Abstract

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this