Variable-sized map and locality-aware reduce on public-resource grids

Po Cheng Chen; Yen Liang Su; Jyh Biau Chang; Ce Kuen Shieh

doi:10.1007/978-3-642-13067-0_27

Variable-sized map and locality-aware reduce on public-resource grids

Po Cheng Chen, Yen Liang Su, Jyh Biau Chang, Ce Kuen Shieh

Department of Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Citations (Scopus)

Abstract

This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely: Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of the map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.

Original language	English
Title of host publication	Advances in Grid and Pervasive Computing - 5th International Conference, GPC 2010, Proceedings
Pages	234-243
Number of pages	10
DOIs	https://doi.org/10.1007/978-3-642-13067-0_27
Publication status	Published - 2010 Jun 25
Event	5th International Conference on Advances in Grid and Pervasive Computing, GPC 2010 - Hualien, Taiwan Duration: 2010 May 10 → 2010 May 13

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	6104 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	5th International Conference on Advances in Grid and Pervasive Computing, GPC 2010
Country/Territory	Taiwan
City	Hualien
Period	10-05-10 → 10-05-13

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-642-13067-0_27

Cite this

Chen, P. C., Su, Y. L., Chang, J. B., & Shieh, C. K. (2010). Variable-sized map and locality-aware reduce on public-resource grids. In Advances in Grid and Pervasive Computing - 5th International Conference, GPC 2010, Proceedings (pp. 234-243). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6104 LNCS). https://doi.org/10.1007/978-3-642-13067-0_27

Chen, Po Cheng ; Su, Yen Liang ; Chang, Jyh Biau ; Shieh, Ce Kuen. / Variable-sized map and locality-aware reduce on public-resource grids. Advances in Grid and Pervasive Computing - 5th International Conference, GPC 2010, Proceedings. 2010. pp. 234-243 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{ef9d4279e81343269abcd66ee13bdb38,

title = "Variable-sized map and locality-aware reduce on public-resource grids",

abstract = "This paper presents a grid-enabled MapReduce framework called {"}Ussop{"}. Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely: Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of the map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.",

author = "Chen, {Po Cheng} and Su, {Yen Liang} and Chang, {Jyh Biau} and Shieh, {Ce Kuen}",

year = "2010",

month = jun,

day = "25",

doi = "10.1007/978-3-642-13067-0_27",

language = "English",

isbn = "3642130666",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "234--243",

booktitle = "Advances in Grid and Pervasive Computing - 5th International Conference, GPC 2010, Proceedings",

note = "5th International Conference on Advances in Grid and Pervasive Computing, GPC 2010 ; Conference date: 10-05-2010 Through 13-05-2010",

}

Chen, PC, Su, YL, Chang, JB & Shieh, CK 2010, Variable-sized map and locality-aware reduce on public-resource grids. in Advances in Grid and Pervasive Computing - 5th International Conference, GPC 2010, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6104 LNCS, pp. 234-243, 5th International Conference on Advances in Grid and Pervasive Computing, GPC 2010, Hualien, Taiwan, 10-05-10. https://doi.org/10.1007/978-3-642-13067-0_27

Variable-sized map and locality-aware reduce on public-resource grids. / Chen, Po Cheng; Su, Yen Liang; Chang, Jyh Biau; Shieh, Ce Kuen.

Advances in Grid and Pervasive Computing - 5th International Conference, GPC 2010, Proceedings. 2010. p. 234-243 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6104 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Variable-sized map and locality-aware reduce on public-resource grids

AU - Chen, Po Cheng

AU - Su, Yen Liang

AU - Chang, Jyh Biau

AU - Shieh, Ce Kuen

PY - 2010/6/25

Y1 - 2010/6/25

N2 - This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely: Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of the map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.

AB - This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely: Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of the map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.

UR - http://www.scopus.com/inward/record.url?scp=77953750708&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77953750708&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-13067-0_27

DO - 10.1007/978-3-642-13067-0_27

M3 - Conference contribution

AN - SCOPUS:77953750708

SN - 3642130666

SN - 9783642130663

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 234

EP - 243

BT - Advances in Grid and Pervasive Computing - 5th International Conference, GPC 2010, Proceedings

T2 - 5th International Conference on Advances in Grid and Pervasive Computing, GPC 2010

Y2 - 10 May 2010 through 13 May 2010

ER -

Chen PC, Su YL, Chang JB, Shieh CK. Variable-sized map and locality-aware reduce on public-resource grids. In Advances in Grid and Pervasive Computing - 5th International Conference, GPC 2010, Proceedings. 2010. p. 234-243. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-13067-0_27

Variable-sized map and locality-aware reduce on public-resource grids

Abstract

Publication series

Other

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this