TY - GEN
T1 - Variable-sized map and locality-aware reduce on public-resource grids
AU - Chen, Po Cheng
AU - Su, Yen Liang
AU - Chang, Jyh Biau
AU - Shieh, Ce Kuen
PY - 2010/6/25
Y1 - 2010/6/25
N2 - This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely: Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of the map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.
AB - This paper presents a grid-enabled MapReduce framework called "Ussop". Ussop provides its users with a set of C-language based MapReduce APIs and an efficient runtime system for exploiting the computing resources available on public-resource grids. Considering the volatility nature of the grid environment, Ussop introduces two novel task scheduling algorithms, namely: Variable-Sized Map Scheduling (VSMS) and Locality-Aware Reduce Scheduling (LARS). VSMS dynamically adjusts the size of the map tasks according to the computing power of grid nodes. Moreover, LARS minimizes the data transfer cost of exchanging the intermediate data over a wide-area network. The experimental results indicate that both VSMS and LARS achieved superior performance than the conventional scheduling algorithms.
UR - http://www.scopus.com/inward/record.url?scp=77953750708&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77953750708&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-13067-0_27
DO - 10.1007/978-3-642-13067-0_27
M3 - Conference contribution
AN - SCOPUS:77953750708
SN - 3642130666
SN - 9783642130663
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 234
EP - 243
BT - Advances in Grid and Pervasive Computing - 5th International Conference, GPC 2010, Proceedings
T2 - 5th International Conference on Advances in Grid and Pervasive Computing, GPC 2010
Y2 - 10 May 2010 through 13 May 2010
ER -