Variable-sized map and locality-aware reduce on public-resource grids

Yen Liang Su, Po Cheng Chen, Jyh Biau Chang, Ce-Kuen Shieh

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely, Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.

Original languageEnglish
Pages (from-to)843-849
Number of pages7
JournalFuture Generation Computer Systems
Volume27
Issue number6
DOIs
Publication statusPublished - 2011 Jun 1

Fingerprint

Scheduling
Scheduling algorithms
Wide area networks
Data transfer
Application programming interfaces (API)
Costs

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

Su, Yen Liang ; Chen, Po Cheng ; Chang, Jyh Biau ; Shieh, Ce-Kuen. / Variable-sized map and locality-aware reduce on public-resource grids. In: Future Generation Computer Systems. 2011 ; Vol. 27, No. 6. pp. 843-849.
@article{bd93f50216ff413ca9f2eda44f2b9a5d,
title = "Variable-sized map and locality-aware reduce on public-resource grids",
abstract = "This paper presents a grid-enabled MapReduce framework called {"}Ussop{"}. Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely, Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.",
author = "Su, {Yen Liang} and Chen, {Po Cheng} and Chang, {Jyh Biau} and Ce-Kuen Shieh",
year = "2011",
month = "6",
day = "1",
doi = "10.1016/j.future.2010.09.001",
language = "English",
volume = "27",
pages = "843--849",
journal = "Future Generation Computer Systems",
issn = "0167-739X",
publisher = "Elsevier",
number = "6",

}

Variable-sized map and locality-aware reduce on public-resource grids. / Su, Yen Liang; Chen, Po Cheng; Chang, Jyh Biau; Shieh, Ce-Kuen.

In: Future Generation Computer Systems, Vol. 27, No. 6, 01.06.2011, p. 843-849.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Variable-sized map and locality-aware reduce on public-resource grids

AU - Su, Yen Liang

AU - Chen, Po Cheng

AU - Chang, Jyh Biau

AU - Shieh, Ce-Kuen

PY - 2011/6/1

Y1 - 2011/6/1

N2 - This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely, Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.

AB - This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely, Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.

UR - http://www.scopus.com/inward/record.url?scp=79953212189&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79953212189&partnerID=8YFLogxK

U2 - 10.1016/j.future.2010.09.001

DO - 10.1016/j.future.2010.09.001

M3 - Article

AN - SCOPUS:79953212189

VL - 27

SP - 843

EP - 849

JO - Future Generation Computer Systems

JF - Future Generation Computer Systems

SN - 0167-739X

IS - 6

ER -