Workload Alleviation Scheduling Framework to Alleviate Negative Performance Impact of Intermediate Data Skew in Small-Scale MapReduce Cloud

Tzu Chi Huang, Kuo Chih Chu, Jia Huei Lin, Guo Hao Huang, Ce-Kuen Shieh

研究成果: Conference contribution

摘要

A MapReduce cloud becomes the essential platform in the cloud computing infrastructure today. Because applications may process input data with different algorithms and logics to produce intermediate data, a MapReduce cloud may suffer intermediate data skew by unevenly distributing intermediate data among nodes at run time. When intermediate data skew happens, a MapReduce cloud not only idles nodes to waste computation resources but also prolongs the application execution progress to hurt user experiences in cloud computing. Instead of the existing solutions that assume many available idle nodes and use computation resources in a loose way, a MapReduce cloud can use the Workload Alleviation Scheduling Framework (W ASF) proposed in this paper to alleviate the negative performance impact of intermediate data skew in a small-scale MapReduce cloud by smartly utilizing computation resources. Besides, a MapReduce cloud is verified with popular applications in experiments to have the outstanding performance improvement with W ASF when intermediate data skew happens.

原文English
主出版物標題2018 International Conference on System Science and Engineering, ICSSE 2018
發行者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子)9781538662854
DOIs
出版狀態Published - 2018 十一月 1
事件2018 International Conference on System Science and Engineering, ICSSE 2018 - New Taipei City, Taiwan
持續時間: 2018 六月 282018 六月 30

出版系列

名字2018 International Conference on System Science and Engineering, ICSSE 2018

Other

Other2018 International Conference on System Science and Engineering, ICSSE 2018
國家Taiwan
城市New Taipei City
期間18-06-2818-06-30

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Control and Optimization

指紋 深入研究「Workload Alleviation Scheduling Framework to Alleviate Negative Performance Impact of Intermediate Data Skew in Small-Scale MapReduce Cloud」主題。共同形成了獨特的指紋。

  • 引用此

    Huang, T. C., Chu, K. C., Lin, J. H., Huang, G. H., & Shieh, C-K. (2018). Workload Alleviation Scheduling Framework to Alleviate Negative Performance Impact of Intermediate Data Skew in Small-Scale MapReduce Cloud. 於 2018 International Conference on System Science and Engineering, ICSSE 2018 [8520003] (2018 International Conference on System Science and Engineering, ICSSE 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICSSE.2018.8520003