Novel scheduling algorithms for efficient deployment of mapreduce applications in heterogeneous computing environments

Sun Yuan Hsieh, Chi Ting Chen, Chi Hao Chen, Tzu Hsiang Yen, Hung Chang Hsiao, Rajkumar Buyya

Research output: Contribution to journalArticlepeer-review

10 Citations (Scopus)

Abstract

Cloud computing has become increasingly popular model for delivering applications hosted in large data centers as subscription oriented services. Hadoop is a popular system supporting the MapReduce function, which plays a crucial role in cloud computing. The resources required for executing jobs in a large data center vary according to the job type. In Hadoop, jobs are scheduled by default on a first-come-first-served basis, which may unbalance resource utilization. This paper proposes a job scheduler called the job allocation scheduler (JAS), designed to balance resource utilization. For various job workloads, the JAS categorizes jobs and then assigns tasks to a CPU-bound queue or an I/O-bound queue. However, the JAS exhibited a locality problem, which was addressed by developing a modified JAS called the job allocation scheduler with locality (JASL). The JASL improved the use of nodes and the performance of Hadoop in heterogeneous computing environments. Finally, two parameters were added to the JASL to detect inaccurate slot settings and create a dynamic job allocation scheduler with locality (DJASL). The DJASL exhibited superior performance than did the JAS, and data locality similar to that of the JASL.

Original languageEnglish
Article number7450666
Pages (from-to)1080-1095
Number of pages16
JournalIEEE Transactions on Cloud Computing
Volume6
Issue number4
DOIs
Publication statusPublished - 2018 Oct 1

All Science Journal Classification (ASJC) codes

  • Software
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Novel scheduling algorithms for efficient deployment of mapreduce applications in heterogeneous computing environments'. Together they form a unique fingerprint.

Cite this