Dimensioning scientific computing systems to improve performance of Map-Reduce based applications

Gabriel G. Castañé, Alberto Núñez, Rosa Filgueira, Jesús Carretero

Research output: Contribution to journalConference articlepeer-review

5 Citations (Scopus)
24 Downloads (Pure)

Abstract

Map-Reduce is a programming model widely used for processing large data sets on scientific clusters. Most of the efforts and research are focused on enhancing and alleviating the drawbacks of the model proposed by Google. The requirements of Map-Reduce based applications are often unclear because of the difficulty in satisfying the overall system throughput, as well as exploring alternatives to obtain a good tradeoff between the performance of basic systems such as storage, networking and CPU. In this paper we present an evaluation of the compared performance of scaling up scientific computing systems using a Map-Reduce application model. This work is specifically focused on medium-size multi-core systems, frequently used by researchers to compute scientific applications. The scaling process is oriented towards the three main resources: computing power, communications and storage. By performing an extensive set of simulations using iCanCloud simulator, we also show that main bottlenecks of those kinds of applications executed in cluster systems are found in storage and network systems. Thence, in order to increase the overall performance of those applications, the computing power must be scaled up proportionally along the network and storage system.

Original languageEnglish
Pages (from-to)226-235
Number of pages10
JournalProcedia Computer Science
Volume9
DOIs
Publication statusPublished - 2012
Event12th Annual International Conference on Computational Science 2012 - Omaha, United States
Duration: 4 Jun 20126 Jun 2012

Keywords

  • Map-Reduce applications
  • Modeling and simulation
  • Performance prediction
  • Scientific applications
  • Scientific clusters

ASJC Scopus subject areas

  • General Computer Science

Fingerprint

Dive into the research topics of 'Dimensioning scientific computing systems to improve performance of Map-Reduce based applications'. Together they form a unique fingerprint.

Cite this