Computation capability deduction architecture for MapReduce on cloud computing

Tzu Chi Huang, Kuo Chih Chu, Guo Hao Huang, Yan Chen Shen, Ce-Kuen Shieh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

MapReduce gradually becomes the de facto programming standard of applications on cloud computing. However, MapReduce needs a cloud administrator to manually configure parameters of the run-time system such as slot numbers for Map and Reduce tasks in order to get the best performance. Because the manual configuration has a risk of performance degradation, MapReduce should utilize the Computation Capability Deduction Architecture (CCDA) proposed in this paper to avoid the risk. MapReduce can use CCDA to help the run-time system to distribute appropriate numbers of tasks over computers in a cloud at run time without any manual configuration made by a cloud administrator. According to experiment observations in this paper, MapReduce can get great performance improvement with the help of CCDA in data-intensive applications such as Inverted Index and Word Count that are usually required to process big data on cloud computing.

Original languageEnglish
Title of host publicationProceedings - 18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017
EditorsShi-Jinn Horng
PublisherIEEE Computer Society
Pages368-375
Number of pages8
ISBN (Electronic)9781538631515
DOIs
Publication statusPublished - 2018 Mar 27
Event18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017 - Taipei, Taiwan
Duration: 2017 Dec 182017 Dec 20

Publication series

NameParallel and Distributed Computing, Applications and Technologies, PDCAT Proceedings
Volume2017-December

Other

Other18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017
CountryTaiwan
CityTaipei
Period17-12-1817-12-20

Fingerprint

MapReduce
Deduction
Cloud computing
Cloud Computing
Runtime Systems
Configuration
Degradation
Count
Programming
Architecture
Experiments
Experiment

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Theoretical Computer Science
  • Computer Science Applications

Cite this

Huang, T. C., Chu, K. C., Huang, G. H., Shen, Y. C., & Shieh, C-K. (2018). Computation capability deduction architecture for MapReduce on cloud computing. In S-J. Horng (Ed.), Proceedings - 18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017 (pp. 368-375). (Parallel and Distributed Computing, Applications and Technologies, PDCAT Proceedings; Vol. 2017-December). IEEE Computer Society. https://doi.org/10.1109/PDCAT.2017.00067
Huang, Tzu Chi ; Chu, Kuo Chih ; Huang, Guo Hao ; Shen, Yan Chen ; Shieh, Ce-Kuen. / Computation capability deduction architecture for MapReduce on cloud computing. Proceedings - 18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017. editor / Shi-Jinn Horng. IEEE Computer Society, 2018. pp. 368-375 (Parallel and Distributed Computing, Applications and Technologies, PDCAT Proceedings).
@inproceedings{262749ad3be342099ff7ea888ee3bd09,
title = "Computation capability deduction architecture for MapReduce on cloud computing",
abstract = "MapReduce gradually becomes the de facto programming standard of applications on cloud computing. However, MapReduce needs a cloud administrator to manually configure parameters of the run-time system such as slot numbers for Map and Reduce tasks in order to get the best performance. Because the manual configuration has a risk of performance degradation, MapReduce should utilize the Computation Capability Deduction Architecture (CCDA) proposed in this paper to avoid the risk. MapReduce can use CCDA to help the run-time system to distribute appropriate numbers of tasks over computers in a cloud at run time without any manual configuration made by a cloud administrator. According to experiment observations in this paper, MapReduce can get great performance improvement with the help of CCDA in data-intensive applications such as Inverted Index and Word Count that are usually required to process big data on cloud computing.",
author = "Huang, {Tzu Chi} and Chu, {Kuo Chih} and Huang, {Guo Hao} and Shen, {Yan Chen} and Ce-Kuen Shieh",
year = "2018",
month = "3",
day = "27",
doi = "10.1109/PDCAT.2017.00067",
language = "English",
series = "Parallel and Distributed Computing, Applications and Technologies, PDCAT Proceedings",
publisher = "IEEE Computer Society",
pages = "368--375",
editor = "Shi-Jinn Horng",
booktitle = "Proceedings - 18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017",
address = "United States",

}

Huang, TC, Chu, KC, Huang, GH, Shen, YC & Shieh, C-K 2018, Computation capability deduction architecture for MapReduce on cloud computing. in S-J Horng (ed.), Proceedings - 18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017. Parallel and Distributed Computing, Applications and Technologies, PDCAT Proceedings, vol. 2017-December, IEEE Computer Society, pp. 368-375, 18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017, Taipei, Taiwan, 17-12-18. https://doi.org/10.1109/PDCAT.2017.00067

Computation capability deduction architecture for MapReduce on cloud computing. / Huang, Tzu Chi; Chu, Kuo Chih; Huang, Guo Hao; Shen, Yan Chen; Shieh, Ce-Kuen.

Proceedings - 18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017. ed. / Shi-Jinn Horng. IEEE Computer Society, 2018. p. 368-375 (Parallel and Distributed Computing, Applications and Technologies, PDCAT Proceedings; Vol. 2017-December).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Computation capability deduction architecture for MapReduce on cloud computing

AU - Huang, Tzu Chi

AU - Chu, Kuo Chih

AU - Huang, Guo Hao

AU - Shen, Yan Chen

AU - Shieh, Ce-Kuen

PY - 2018/3/27

Y1 - 2018/3/27

N2 - MapReduce gradually becomes the de facto programming standard of applications on cloud computing. However, MapReduce needs a cloud administrator to manually configure parameters of the run-time system such as slot numbers for Map and Reduce tasks in order to get the best performance. Because the manual configuration has a risk of performance degradation, MapReduce should utilize the Computation Capability Deduction Architecture (CCDA) proposed in this paper to avoid the risk. MapReduce can use CCDA to help the run-time system to distribute appropriate numbers of tasks over computers in a cloud at run time without any manual configuration made by a cloud administrator. According to experiment observations in this paper, MapReduce can get great performance improvement with the help of CCDA in data-intensive applications such as Inverted Index and Word Count that are usually required to process big data on cloud computing.

AB - MapReduce gradually becomes the de facto programming standard of applications on cloud computing. However, MapReduce needs a cloud administrator to manually configure parameters of the run-time system such as slot numbers for Map and Reduce tasks in order to get the best performance. Because the manual configuration has a risk of performance degradation, MapReduce should utilize the Computation Capability Deduction Architecture (CCDA) proposed in this paper to avoid the risk. MapReduce can use CCDA to help the run-time system to distribute appropriate numbers of tasks over computers in a cloud at run time without any manual configuration made by a cloud administrator. According to experiment observations in this paper, MapReduce can get great performance improvement with the help of CCDA in data-intensive applications such as Inverted Index and Word Count that are usually required to process big data on cloud computing.

UR - http://www.scopus.com/inward/record.url?scp=85046823204&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046823204&partnerID=8YFLogxK

U2 - 10.1109/PDCAT.2017.00067

DO - 10.1109/PDCAT.2017.00067

M3 - Conference contribution

T3 - Parallel and Distributed Computing, Applications and Technologies, PDCAT Proceedings

SP - 368

EP - 375

BT - Proceedings - 18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017

A2 - Horng, Shi-Jinn

PB - IEEE Computer Society

ER -

Huang TC, Chu KC, Huang GH, Shen YC, Shieh C-K. Computation capability deduction architecture for MapReduce on cloud computing. In Horng S-J, editor, Proceedings - 18th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2017. IEEE Computer Society. 2018. p. 368-375. (Parallel and Distributed Computing, Applications and Technologies, PDCAT Proceedings). https://doi.org/10.1109/PDCAT.2017.00067