Speed-based load balancer for scheduling reduce tasks to process intermediate data of MapReduce applications on cloud computing

Tzu Chi Huang; Kuo Chih Chu; Ce Kuen Shieh; Ming Fong Tsai

doi:10.1145/2818869.2818880

Speed-based load balancer for scheduling reduce tasks to process intermediate data of MapReduce applications on cloud computing

Tzu Chi Huang, Kuo Chih Chu, Ce Kuen Shieh, Ming Fong Tsai

電機工程學系

研究成果: Conference contribution

1 引文斯高帕斯（Scopus）

摘要

MapReduce is a programming model used to develop applications on cloud computing. However, MapReduce strongly relies on the runtime system to handle issues of task managements in order to get a better performance. While most research works focus on scheduling Map tasks to improve performances, MapReduce is identified by this paper to have the potential for performance improvements through the Speed-based Load Balancer (SLB) for scheduling Reduce tasks. According to observations on experiments of Inverted Index, Radix Sort and Word Count, MapReduce can use SLB to outperform the native scheduler used by Hadoop in the runtime system.

原文	English
主出版物標題	Proceedings of the ASE BigData and SocialInformatics 2015, ASE BD and SI 2015
發行者	Association for Computing Machinery
ISBN（電子）	9781450337359
DOIs	https://doi.org/10.1145/2818869.2818880
出版狀態	Published - 2015 10月 7
事件	ASE BigData and SocialInformatics, ASE BD and SI 2015 - Kaohsiung, Taiwan 持續時間: 2015 10月 7 → 2015 10月 9

出版系列

名字	ACM International Conference Proceeding Series
卷	07-09-Ocobert-2015

Other

Other	ASE BigData and SocialInformatics, ASE BD and SI 2015
國家/地區	Taiwan
城市	Kaohsiung
期間	15-10-07 → 15-10-09

All Science Journal Classification (ASJC) codes

軟體
人機介面
電腦視覺和模式識別
電腦網路與通信

存取文件

10.1145/2818869.2818880

其它文件與鏈接

引用此

Huang, T. C., Chu, K. C., Shieh, C. K., & Tsai, M. F. (2015). Speed-based load balancer for scheduling reduce tasks to process intermediate data of MapReduce applications on cloud computing. 於 Proceedings of the ASE BigData and SocialInformatics 2015, ASE BD and SI 2015 [a49] (ACM International Conference Proceeding Series; 卷 07-09-Ocobert-2015). Association for Computing Machinery. https://doi.org/10.1145/2818869.2818880

@inproceedings{0ec07cc48485412dbe794a7c9c8fc306,

title = "Speed-based load balancer for scheduling reduce tasks to process intermediate data of MapReduce applications on cloud computing",

abstract = "MapReduce is a programming model used to develop applications on cloud computing. However, MapReduce strongly relies on the runtime system to handle issues of task managements in order to get a better performance. While most research works focus on scheduling Map tasks to improve performances, MapReduce is identified by this paper to have the potential for performance improvements through the Speed-based Load Balancer (SLB) for scheduling Reduce tasks. According to observations on experiments of Inverted Index, Radix Sort and Word Count, MapReduce can use SLB to outperform the native scheduler used by Hadoop in the runtime system.",

author = "Huang, {Tzu Chi} and Chu, {Kuo Chih} and Shieh, {Ce Kuen} and Tsai, {Ming Fong}",

year = "2015",

month = oct,

day = "7",

doi = "10.1145/2818869.2818880",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

booktitle = "Proceedings of the ASE BigData and SocialInformatics 2015, ASE BD and SI 2015",

note = "ASE BigData and SocialInformatics, ASE BD and SI 2015 ; Conference date: 07-10-2015 Through 09-10-2015",

}

Huang, TC, Chu, KC, Shieh, CK & Tsai, MF 2015, Speed-based load balancer for scheduling reduce tasks to process intermediate data of MapReduce applications on cloud computing. 於 Proceedings of the ASE BigData and SocialInformatics 2015, ASE BD and SI 2015., a49, ACM International Conference Proceeding Series, 卷 07-09-Ocobert-2015, Association for Computing Machinery, ASE BigData and SocialInformatics, ASE BD and SI 2015, Kaohsiung, Taiwan, 15-10-07. https://doi.org/10.1145/2818869.2818880

Speed-based load balancer for scheduling reduce tasks to process intermediate data of MapReduce applications on cloud computing. / Huang, Tzu Chi; Chu, Kuo Chih; Shieh, Ce Kuen 等.
Proceedings of the ASE BigData and SocialInformatics 2015, ASE BD and SI 2015. Association for Computing Machinery, 2015. a49 (ACM International Conference Proceeding Series; 卷 07-09-Ocobert-2015).

研究成果: Conference contribution

TY - GEN

T1 - Speed-based load balancer for scheduling reduce tasks to process intermediate data of MapReduce applications on cloud computing

AU - Huang, Tzu Chi

AU - Chu, Kuo Chih

AU - Shieh, Ce Kuen

AU - Tsai, Ming Fong

PY - 2015/10/7

Y1 - 2015/10/7

N2 - MapReduce is a programming model used to develop applications on cloud computing. However, MapReduce strongly relies on the runtime system to handle issues of task managements in order to get a better performance. While most research works focus on scheduling Map tasks to improve performances, MapReduce is identified by this paper to have the potential for performance improvements through the Speed-based Load Balancer (SLB) for scheduling Reduce tasks. According to observations on experiments of Inverted Index, Radix Sort and Word Count, MapReduce can use SLB to outperform the native scheduler used by Hadoop in the runtime system.

AB - MapReduce is a programming model used to develop applications on cloud computing. However, MapReduce strongly relies on the runtime system to handle issues of task managements in order to get a better performance. While most research works focus on scheduling Map tasks to improve performances, MapReduce is identified by this paper to have the potential for performance improvements through the Speed-based Load Balancer (SLB) for scheduling Reduce tasks. According to observations on experiments of Inverted Index, Radix Sort and Word Count, MapReduce can use SLB to outperform the native scheduler used by Hadoop in the runtime system.

UR - http://www.scopus.com/inward/record.url?scp=84959934545&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84959934545&partnerID=8YFLogxK

U2 - 10.1145/2818869.2818880

DO - 10.1145/2818869.2818880

M3 - Conference contribution

AN - SCOPUS:84959934545

T3 - ACM International Conference Proceeding Series

BT - Proceedings of the ASE BigData and SocialInformatics 2015, ASE BD and SI 2015

PB - Association for Computing Machinery

T2 - ASE BigData and SocialInformatics, ASE BD and SI 2015

Y2 - 7 October 2015 through 9 October 2015

ER -

Huang TC, Chu KC, Shieh CK, Tsai MF. Speed-based load balancer for scheduling reduce tasks to process intermediate data of MapReduce applications on cloud computing. 於 Proceedings of the ASE BigData and SocialInformatics 2015, ASE BD and SI 2015. Association for Computing Machinery. 2015. a49. (ACM International Conference Proceeding Series). doi: 10.1145/2818869.2818880

Speed-based load balancer for scheduling reduce tasks to process intermediate data of MapReduce applications on cloud computing

摘要

出版系列

Other

All Science Journal Classification (ASJC) codes

存取文件

其它文件與鏈接

指紋

引用此