Context-aware task assignment for MapReduce in heterogeneous clouds

Wei Tsung Su, Wei Fan Pan, Chao-Chun Chen

研究成果: Article

摘要

The MapReduce programming model is designed to process large data sets based on parallel computing among multiple computer nodes (CNs). Because the data size is considerably increased (data are collected from sensors in most cases), the optimization problem of task assignment becomes important to improve the performance of MapReduce. Unfortunately, this problem is even more difficult in heterogeneous clouds in which the CNs have different capabilities and available resources. In this paper, the context-aware task assignment (CATA) approach is proposed to improve the performance of MapReduce in a twofold manner. First, CATA takes the resource demands for different types of jobs into account. Second, CATA can assign tasks to CNs according to their capabilities and available resources in a resource-proportional manner. The experimental results show that CATA can efficiently reduce the job execution time by 10 to 40%.

原文English
頁(從 - 到)1497-1512
頁數16
期刊Sensors and Materials
29
發行號11
DOIs
出版狀態Published - 2017 一月 1

指紋

resources
Parallel processing systems
Sensors
programming
optimization
sensors

All Science Journal Classification (ASJC) codes

  • Instrumentation
  • Materials Science(all)

引用此文

Su, Wei Tsung ; Pan, Wei Fan ; Chen, Chao-Chun. / Context-aware task assignment for MapReduce in heterogeneous clouds. 於: Sensors and Materials. 2017 ; 卷 29, 編號 11. 頁 1497-1512.
@article{6a6a446c367742ae97fbc1dcbb757e4e,
title = "Context-aware task assignment for MapReduce in heterogeneous clouds",
abstract = "The MapReduce programming model is designed to process large data sets based on parallel computing among multiple computer nodes (CNs). Because the data size is considerably increased (data are collected from sensors in most cases), the optimization problem of task assignment becomes important to improve the performance of MapReduce. Unfortunately, this problem is even more difficult in heterogeneous clouds in which the CNs have different capabilities and available resources. In this paper, the context-aware task assignment (CATA) approach is proposed to improve the performance of MapReduce in a twofold manner. First, CATA takes the resource demands for different types of jobs into account. Second, CATA can assign tasks to CNs according to their capabilities and available resources in a resource-proportional manner. The experimental results show that CATA can efficiently reduce the job execution time by 10 to 40{\%}.",
author = "Su, {Wei Tsung} and Pan, {Wei Fan} and Chao-Chun Chen",
year = "2017",
month = "1",
day = "1",
doi = "10.18494/SAM.2017.1659",
language = "English",
volume = "29",
pages = "1497--1512",
journal = "Sensors and Materials",
issn = "0914-4935",
publisher = "M Y U Scientific Publishing Division",
number = "11",

}

Context-aware task assignment for MapReduce in heterogeneous clouds. / Su, Wei Tsung; Pan, Wei Fan; Chen, Chao-Chun.

於: Sensors and Materials, 卷 29, 編號 11, 01.01.2017, p. 1497-1512.

研究成果: Article

TY - JOUR

T1 - Context-aware task assignment for MapReduce in heterogeneous clouds

AU - Su, Wei Tsung

AU - Pan, Wei Fan

AU - Chen, Chao-Chun

PY - 2017/1/1

Y1 - 2017/1/1

N2 - The MapReduce programming model is designed to process large data sets based on parallel computing among multiple computer nodes (CNs). Because the data size is considerably increased (data are collected from sensors in most cases), the optimization problem of task assignment becomes important to improve the performance of MapReduce. Unfortunately, this problem is even more difficult in heterogeneous clouds in which the CNs have different capabilities and available resources. In this paper, the context-aware task assignment (CATA) approach is proposed to improve the performance of MapReduce in a twofold manner. First, CATA takes the resource demands for different types of jobs into account. Second, CATA can assign tasks to CNs according to their capabilities and available resources in a resource-proportional manner. The experimental results show that CATA can efficiently reduce the job execution time by 10 to 40%.

AB - The MapReduce programming model is designed to process large data sets based on parallel computing among multiple computer nodes (CNs). Because the data size is considerably increased (data are collected from sensors in most cases), the optimization problem of task assignment becomes important to improve the performance of MapReduce. Unfortunately, this problem is even more difficult in heterogeneous clouds in which the CNs have different capabilities and available resources. In this paper, the context-aware task assignment (CATA) approach is proposed to improve the performance of MapReduce in a twofold manner. First, CATA takes the resource demands for different types of jobs into account. Second, CATA can assign tasks to CNs according to their capabilities and available resources in a resource-proportional manner. The experimental results show that CATA can efficiently reduce the job execution time by 10 to 40%.

UR - http://www.scopus.com/inward/record.url?scp=85035336905&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85035336905&partnerID=8YFLogxK

U2 - 10.18494/SAM.2017.1659

DO - 10.18494/SAM.2017.1659

M3 - Article

AN - SCOPUS:85035336905

VL - 29

SP - 1497

EP - 1512

JO - Sensors and Materials

JF - Sensors and Materials

SN - 0914-4935

IS - 11

ER -