Poster: Reservation-Based I/O Performance Guarantee for MPI-IO Applications Using Shared Storage Systems

Yusuke Tanimura; Rosa Filgueira; Isao Kojima; Malcolm Atkinson

doi:10.1109/SC.Companion.2012.204

Poster: Reservation-Based I/O Performance Guarantee for MPI-IO Applications Using Shared Storage Systems

Yusuke Tanimura, Rosa Filgueira, Isao Kojima, Malcolm Atkinson

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

While optimized collective I/O methods are proposed for MPI-IO implementations, a problem in concurrent use of the shared storage system is raised. In order to prevent performance degradation of parallel I/O due to such I/O conflict, we propose an advance reservation approach, including possible integration with existing batch scheduler on HPC clusters. In this work, we use Dynamic-CoMPI as a MPI-IO implementation and Papio as a shared storage system which implements parallel I/O and performance reservation. Then we have been developing the ADIO layer to connect these systems and to evaluate the benefits of the reservation-based performance isolation. Our prototype implementation, Dynamic-CoMPI/Papio, was evaluated using the MPI-IO Test benchmark and the BISP3D application. In our preliminary evaluation, total execution time increased 3~12% with the Dynamic-CoMPI/PVFS2 and 6~40% with the Dynamic-CoMPI/Lustre, under the situation where additional workload affected on MPI execution, however there was no obvious time increase with Dynamic-CoMPI/Papio.

Original language	English
Title of host publication	2012 SC Companion: High Performance Computing, Networking Storage and Analysis
Publisher	IEEE
ISBN (Electronic)	9780769549569
ISBN (Print)	9781467362184
DOIs	https://doi.org/10.1109/SC.Companion.2012.204
Publication status	Published - 11 Apr 2013

Access to Document

10.1109/SC.Companion.2012.204

Cite this

@inproceedings{7f79f802b12c4c59a41d5fb24c318c8c,

title = "Poster: Reservation-Based I/O Performance Guarantee for MPI-IO Applications Using Shared Storage Systems",

abstract = "While optimized collective I/O methods are proposed for MPI-IO implementations, a problem in concurrent use of the shared storage system is raised. In order to prevent performance degradation of parallel I/O due to such I/O conflict, we propose an advance reservation approach, including possible integration with existing batch scheduler on HPC clusters. In this work, we use Dynamic-CoMPI as a MPI-IO implementation and Papio as a shared storage system which implements parallel I/O and performance reservation. Then we have been developing the ADIO layer to connect these systems and to evaluate the benefits of the reservation-based performance isolation. Our prototype implementation, Dynamic-CoMPI/Papio, was evaluated using the MPI-IO Test benchmark and the BISP3D application. In our preliminary evaluation, total execution time increased 3~12% with the Dynamic-CoMPI/PVFS2 and 6~40% with the Dynamic-CoMPI/Lustre, under the situation where additional workload affected on MPI execution, however there was no obvious time increase with Dynamic-CoMPI/Papio.",

author = "Yusuke Tanimura and Rosa Filgueira and Isao Kojima and Malcolm Atkinson",

year = "2013",

month = apr,

day = "11",

doi = "10.1109/SC.Companion.2012.204",

language = "English",

isbn = "9781467362184",

booktitle = "2012 SC Companion: High Performance Computing, Networking Storage and Analysis",

publisher = "IEEE",

address = "United States",

}

TY - GEN

T1 - Poster: Reservation-Based I/O Performance Guarantee for MPI-IO Applications Using Shared Storage Systems

AU - Tanimura, Yusuke

AU - Filgueira, Rosa

AU - Kojima, Isao

AU - Atkinson, Malcolm

PY - 2013/4/11

Y1 - 2013/4/11

N2 - While optimized collective I/O methods are proposed for MPI-IO implementations, a problem in concurrent use of the shared storage system is raised. In order to prevent performance degradation of parallel I/O due to such I/O conflict, we propose an advance reservation approach, including possible integration with existing batch scheduler on HPC clusters. In this work, we use Dynamic-CoMPI as a MPI-IO implementation and Papio as a shared storage system which implements parallel I/O and performance reservation. Then we have been developing the ADIO layer to connect these systems and to evaluate the benefits of the reservation-based performance isolation. Our prototype implementation, Dynamic-CoMPI/Papio, was evaluated using the MPI-IO Test benchmark and the BISP3D application. In our preliminary evaluation, total execution time increased 3~12% with the Dynamic-CoMPI/PVFS2 and 6~40% with the Dynamic-CoMPI/Lustre, under the situation where additional workload affected on MPI execution, however there was no obvious time increase with Dynamic-CoMPI/Papio.

AB - While optimized collective I/O methods are proposed for MPI-IO implementations, a problem in concurrent use of the shared storage system is raised. In order to prevent performance degradation of parallel I/O due to such I/O conflict, we propose an advance reservation approach, including possible integration with existing batch scheduler on HPC clusters. In this work, we use Dynamic-CoMPI as a MPI-IO implementation and Papio as a shared storage system which implements parallel I/O and performance reservation. Then we have been developing the ADIO layer to connect these systems and to evaluate the benefits of the reservation-based performance isolation. Our prototype implementation, Dynamic-CoMPI/Papio, was evaluated using the MPI-IO Test benchmark and the BISP3D application. In our preliminary evaluation, total execution time increased 3~12% with the Dynamic-CoMPI/PVFS2 and 6~40% with the Dynamic-CoMPI/Lustre, under the situation where additional workload affected on MPI execution, however there was no obvious time increase with Dynamic-CoMPI/Papio.

U2 - 10.1109/SC.Companion.2012.204

DO - 10.1109/SC.Companion.2012.204

M3 - Conference contribution

SN - 9781467362184

BT - 2012 SC Companion: High Performance Computing, Networking Storage and Analysis

PB - IEEE

ER -

Poster: Reservation-Based I/O Performance Guarantee for MPI-IO Applications Using Shared Storage Systems

Abstract

Access to Document

Fingerprint

Cite this