MC framework: High-performance distributed framework for standalone data analysis packages over Hadoop-based cloud

Chao Chun Chen, Nguyen Huu Tinh Giang, Tzu Chao Lin, Min Hsiung Hung

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The Hadoop MapReduce is the programming model of designing the scalable distributed computing applications, that provides developers can attain automatic parallelization. However, most complex manufacturing systems are arduous and restrictive to migrate to private clouds, due to the platform incompatible and tremendous complexity of system reconstruction. For increasing the efficiency of manufacturing systems with minimum efforts on modifying source codes, a high-performance framework is designed in this paper, called Multi-users-based Cloud-Adaptor Framework (MC-Framework), which provides the simple interface to users for fairly executing requested tasks worked with traditional standalone data analysis packages in MapReduce-based private cloud environments. Moreover, this framework focuses on multiuser workloads, but the default Hadoop scheduling scheme, i.e., FIFO, would increase delay under multiuser scenarios. Hence, a new scheduling mechanism, called Job-Sharing Scheduling, is designed to explore and fairly share the jobs to machines in the private cloud. Then, we prototype an experimental virtual-metrology module of a manufacturing system as a case study to verify and analysis the proposed MC-Framework. The results of our experiments indicate that our proposed framework enormously improved the time performance compared with the original package.

Original languageEnglish
Title of host publicationProceedings - 2013 IEEE International Conference on Granular Computing, GrC 2013
PublisherIEEE Computer Society
Pages27-32
Number of pages6
ISBN (Print)9781479912810
DOIs
Publication statusPublished - 2013 Jan 1
Event2013 IEEE International Conference on Granular Computing, GrC 2013 - Beijing, China
Duration: 2013 Dec 132013 Dec 15

Publication series

NameProceedings - 2013 IEEE International Conference on Granular Computing, GrC 2013

Other

Other2013 IEEE International Conference on Granular Computing, GrC 2013
CountryChina
CityBeijing
Period13-12-1313-12-15

All Science Journal Classification (ASJC) codes

  • Software

Fingerprint Dive into the research topics of 'MC framework: High-performance distributed framework for standalone data analysis packages over Hadoop-based cloud'. Together they form a unique fingerprint.

Cite this