Abstract
A MapReduce cloud becomes the essential platform in the cloud computing infrastructure today. Because applications may process input data with different algorithms and logics to produce intermediate data, a MapReduce cloud may suffer intermediate data skew by unevenly distributing intermediate data among nodes at run time. When intermediate data skew happens, a MapReduce cloud not only idles nodes to waste computation resources but also prolongs the application execution progress to hurt user experiences in cloud computing. Instead of the existing solutions that assume many available idle nodes and use computation resources in a loose way, a MapReduce cloud can use the Workload Alleviation Scheduling Framework (W ASF) proposed in this paper to alleviate the negative performance impact of intermediate data skew in a small-scale MapReduce cloud by smartly utilizing computation resources. Besides, a MapReduce cloud is verified with popular applications in experiments to have the outstanding performance improvement with W ASF when intermediate data skew happens.
| Original language | English |
|---|---|
| Title of host publication | 2018 International Conference on System Science and Engineering, ICSSE 2018 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| ISBN (Electronic) | 9781538662854 |
| DOIs | |
| Publication status | Published - 2018 Nov 1 |
| Event | 2018 International Conference on System Science and Engineering, ICSSE 2018 - New Taipei City, Taiwan Duration: 2018 Jun 28 → 2018 Jun 30 |
Publication series
| Name | 2018 International Conference on System Science and Engineering, ICSSE 2018 |
|---|
Other
| Other | 2018 International Conference on System Science and Engineering, ICSSE 2018 |
|---|---|
| Country | Taiwan |
| City | New Taipei City |
| Period | 18-06-28 → 18-06-30 |
Fingerprint
All Science Journal Classification (ASJC) codes
- Control and Systems Engineering
- Control and Optimization
Cite this
}
Workload Alleviation Scheduling Framework to Alleviate Negative Performance Impact of Intermediate Data Skew in Small-Scale MapReduce Cloud. / Huang, Tzu Chi; Chu, Kuo Chih; Lin, Jia Huei; Huang, Guo Hao; Shieh, Ce-Kuen.
2018 International Conference on System Science and Engineering, ICSSE 2018. Institute of Electrical and Electronics Engineers Inc., 2018. 8520003 (2018 International Conference on System Science and Engineering, ICSSE 2018).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
TY - GEN
T1 - Workload Alleviation Scheduling Framework to Alleviate Negative Performance Impact of Intermediate Data Skew in Small-Scale MapReduce Cloud
AU - Huang, Tzu Chi
AU - Chu, Kuo Chih
AU - Lin, Jia Huei
AU - Huang, Guo Hao
AU - Shieh, Ce-Kuen
PY - 2018/11/1
Y1 - 2018/11/1
N2 - A MapReduce cloud becomes the essential platform in the cloud computing infrastructure today. Because applications may process input data with different algorithms and logics to produce intermediate data, a MapReduce cloud may suffer intermediate data skew by unevenly distributing intermediate data among nodes at run time. When intermediate data skew happens, a MapReduce cloud not only idles nodes to waste computation resources but also prolongs the application execution progress to hurt user experiences in cloud computing. Instead of the existing solutions that assume many available idle nodes and use computation resources in a loose way, a MapReduce cloud can use the Workload Alleviation Scheduling Framework (W ASF) proposed in this paper to alleviate the negative performance impact of intermediate data skew in a small-scale MapReduce cloud by smartly utilizing computation resources. Besides, a MapReduce cloud is verified with popular applications in experiments to have the outstanding performance improvement with W ASF when intermediate data skew happens.
AB - A MapReduce cloud becomes the essential platform in the cloud computing infrastructure today. Because applications may process input data with different algorithms and logics to produce intermediate data, a MapReduce cloud may suffer intermediate data skew by unevenly distributing intermediate data among nodes at run time. When intermediate data skew happens, a MapReduce cloud not only idles nodes to waste computation resources but also prolongs the application execution progress to hurt user experiences in cloud computing. Instead of the existing solutions that assume many available idle nodes and use computation resources in a loose way, a MapReduce cloud can use the Workload Alleviation Scheduling Framework (W ASF) proposed in this paper to alleviate the negative performance impact of intermediate data skew in a small-scale MapReduce cloud by smartly utilizing computation resources. Besides, a MapReduce cloud is verified with popular applications in experiments to have the outstanding performance improvement with W ASF when intermediate data skew happens.
UR - http://www.scopus.com/inward/record.url?scp=85057622088&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85057622088&partnerID=8YFLogxK
U2 - 10.1109/ICSSE.2018.8520003
DO - 10.1109/ICSSE.2018.8520003
M3 - Conference contribution
AN - SCOPUS:85057622088
T3 - 2018 International Conference on System Science and Engineering, ICSSE 2018
BT - 2018 International Conference on System Science and Engineering, ICSSE 2018
PB - Institute of Electrical and Electronics Engineers Inc.
ER -