Smart partitioning mechanism for dealing with intermediate data skew in reduce task on cloud computing

Tzu Chi Huang, Kuo Chih Chu, Guo Hao Huang, Yan Chen Shen, Ce Kuen Shieh

研究成果: Conference contribution

4 引文 斯高帕斯(Scopus)

摘要

MapReduce greatly alleviates the burdens of programmers and gradually becomes an application programming standard on cloud computing nowadays, because the run-time system of cloud computing can automatically handle the issues of paralleled and distributed programming on behalf of programmers at run time. Although MapReduce can strongly benefit programmers on developing cloud computing applications, intermediate data skew inevitably hurts application performances. MapReduce can use the Smart Partitioning Mechanism (SPM) proposed in this paper as an alternative solution to deal with intermediate data skew in Reduce tasks on cloud computing. With the capability of averagely distributing intermediate data over Slave nodes in SPM, MapReduce no longer suffers from the performance penalty resulting from the intermediate data skew problem in Reduce tasks on cloud computing.

原文English
主出版物標題Proceedings - 31st IEEE International Conference on Advanced Information Networking and Applications, AINA 2017
編輯Tomoya Enokido, Hui-Huang Hsu, Chi-Yi Lin, Makoto Takizawa, Leonard Barolli
發行者Institute of Electrical and Electronics Engineers Inc.
頁面819-826
頁數8
ISBN(電子)9781509060283
DOIs
出版狀態Published - 2017 5月 5
事件31st IEEE International Conference on Advanced Information Networking and Applications, AINA 2017 - Taipei, Taiwan
持續時間: 2017 3月 272017 3月 29

出版系列

名字Proceedings - International Conference on Advanced Information Networking and Applications, AINA
ISSN(列印)1550-445X

Other

Other31st IEEE International Conference on Advanced Information Networking and Applications, AINA 2017
國家/地區Taiwan
城市Taipei
期間17-03-2717-03-29

All Science Journal Classification (ASJC) codes

  • 工程 (全部)

指紋

深入研究「Smart partitioning mechanism for dealing with intermediate data skew in reduce task on cloud computing」主題。共同形成了獨特的指紋。

引用此