Improving speculative execution performance with coworker for cloud computing

Sheng Wei Huang, Tzu Chi Huang, Syue Ru Lyu, Ce Kuen Shieh, Yi Sheng Chou

研究成果: Conference contribution

7 引文 斯高帕斯(Scopus)

摘要

MapReduce is an important programming model for large-scale parallel applications. It divides a job into several parallel tasks and completes the job by sequential phases, i.e. map phase and reduce phase. The job completion time will be delayed when a task, called straggler, consumes more time than others. The main reason that a straggler occurs is the imbalance resource distribution among computing nodes in the cloud. Speculative execution is a solution for dealing with stragglers. Duplicate tasks are launched on other nodes to process the same data as the straggler does. Any completion of these tasks implies that this task is finished and other duplicate tasks can be aborted. However, aborting tasks misspends resources. In this paper, we propose an idea of using coworkers to help a straggler. According to the processing rate of the straggler and the coworker, the amount of data parceled out from the straggler to the coworker should be determined. Different from speculative execution, coworkers finish tasks with stragglers and do not misspend computing resources. Experimental results show that coworkers can reduce the task completion time by 37% and the network traffic by 64% when comparing with speculative execution.

原文English
主出版物標題Proceedings - 2011 17th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2011
頁面1004-1009
頁數6
DOIs
出版狀態Published - 2011 12月 1
事件2011 17th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2011 - Tainan, Taiwan
持續時間: 2011 12月 72011 12月 9

出版系列

名字Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS
ISSN(列印)1521-9097

Other

Other2011 17th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2011
國家/地區Taiwan
城市Tainan
期間11-12-0711-12-09

All Science Journal Classification (ASJC) codes

  • 硬體和架構

指紋

深入研究「Improving speculative execution performance with coworker for cloud computing」主題。共同形成了獨特的指紋。

引用此