IDP: An innovative data placement algorithm for hadoop systems

Chia Wei Lee, Horng Chyau Huang, Sun Yuan Hsieh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In this paper, we propose a data placement strategy to deal with the imbalanced workload problem on DataNodes. Basing on computing capability of each node in a heterogeneous Hadoop cluster, the proposed strategy can balance the data that was stored in the DataNode such that the cost of data transfer time can be tremendously reduced. As a result, the Hadoop overall performance can be greatly improved. Experimental results demonstrate that the proposed data placement strategy can highly decrease the execution time and thus improves Hadoop performance in a heterogeneous cluster.

Original languageEnglish
Title of host publicationIntelligent Systems and Applications - Proceedings of the International Computer Symposium, ICS 2014
EditorsWilliam Cheng-Chung Chu, Stephen Jenn-Hwa Yang, Han-Chieh Chao
PublisherIOS Press
Pages49-58
Number of pages10
ISBN (Electronic)9781614994831
DOIs
Publication statusPublished - 2015 Jan 1
EventInternational Computer Symposium, ICS 2014 - Taichung, Taiwan
Duration: 2014 Dec 122014 Dec 14

Publication series

NameFrontiers in Artificial Intelligence and Applications
Volume274
ISSN (Print)0922-6389

Other

OtherInternational Computer Symposium, ICS 2014
CountryTaiwan
CityTaichung
Period14-12-1214-12-14

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence

Fingerprint Dive into the research topics of 'IDP: An innovative data placement algorithm for hadoop systems'. Together they form a unique fingerprint.

Cite this