Variable-sized map and locality-aware reduce on public-resource grids

Yen Liang Su, Po Cheng Chen, Jyh Biau Chang, Ce-Kuen Shieh

研究成果: Article

10 引文 (Scopus)

摘要

This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely, Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.

原文English
頁(從 - 到)843-849
頁數7
期刊Future Generation Computer Systems
27
發行號6
DOIs
出版狀態Published - 2011 六月 1

指紋

Scheduling
Scheduling algorithms
Wide area networks
Data transfer
Application programming interfaces (API)
Costs

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

引用此文

Su, Yen Liang ; Chen, Po Cheng ; Chang, Jyh Biau ; Shieh, Ce-Kuen. / Variable-sized map and locality-aware reduce on public-resource grids. 於: Future Generation Computer Systems. 2011 ; 卷 27, 編號 6. 頁 843-849.
@article{bd93f50216ff413ca9f2eda44f2b9a5d,
title = "Variable-sized map and locality-aware reduce on public-resource grids",
abstract = "This paper presents a grid-enabled MapReduce framework called {"}Ussop{"}. Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely, Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.",
author = "Su, {Yen Liang} and Chen, {Po Cheng} and Chang, {Jyh Biau} and Ce-Kuen Shieh",
year = "2011",
month = "6",
day = "1",
doi = "10.1016/j.future.2010.09.001",
language = "English",
volume = "27",
pages = "843--849",
journal = "Future Generation Computer Systems",
issn = "0167-739X",
publisher = "Elsevier",
number = "6",

}

Variable-sized map and locality-aware reduce on public-resource grids. / Su, Yen Liang; Chen, Po Cheng; Chang, Jyh Biau; Shieh, Ce-Kuen.

於: Future Generation Computer Systems, 卷 27, 編號 6, 01.06.2011, p. 843-849.

研究成果: Article

TY - JOUR

T1 - Variable-sized map and locality-aware reduce on public-resource grids

AU - Su, Yen Liang

AU - Chen, Po Cheng

AU - Chang, Jyh Biau

AU - Shieh, Ce-Kuen

PY - 2011/6/1

Y1 - 2011/6/1

N2 - This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely, Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.

AB - This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely, Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.

UR - http://www.scopus.com/inward/record.url?scp=79953212189&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79953212189&partnerID=8YFLogxK

U2 - 10.1016/j.future.2010.09.001

DO - 10.1016/j.future.2010.09.001

M3 - Article

AN - SCOPUS:79953212189

VL - 27

SP - 843

EP - 849

JO - Future Generation Computer Systems

JF - Future Generation Computer Systems

SN - 0167-739X

IS - 6

ER -