MC framework: High-performance distributed framework for standalone data analysis packages over Hadoop-based cloud

Chao Chun Chen, Nguyen Huu Tinh Giang, Tzu Chao Lin, Min Hsiung Hung

研究成果: Conference contribution

摘要

The Hadoop MapReduce is the programming model of designing the scalable distributed computing applications, that provides developers can attain automatic parallelization. However, most complex manufacturing systems are arduous and restrictive to migrate to private clouds, due to the platform incompatible and tremendous complexity of system reconstruction. For increasing the efficiency of manufacturing systems with minimum efforts on modifying source codes, a high-performance framework is designed in this paper, called Multi-users-based Cloud-Adaptor Framework (MC-Framework), which provides the simple interface to users for fairly executing requested tasks worked with traditional standalone data analysis packages in MapReduce-based private cloud environments. Moreover, this framework focuses on multiuser workloads, but the default Hadoop scheduling scheme, i.e., FIFO, would increase delay under multiuser scenarios. Hence, a new scheduling mechanism, called Job-Sharing Scheduling, is designed to explore and fairly share the jobs to machines in the private cloud. Then, we prototype an experimental virtual-metrology module of a manufacturing system as a case study to verify and analysis the proposed MC-Framework. The results of our experiments indicate that our proposed framework enormously improved the time performance compared with the original package.

原文English
主出版物標題Proceedings - 2013 IEEE International Conference on Granular Computing, GrC 2013
發行者IEEE Computer Society
頁面27-32
頁數6
ISBN(列印)9781479912810
DOIs
出版狀態Published - 2013
事件2013 IEEE International Conference on Granular Computing, GrC 2013 - Beijing, China
持續時間: 2013 12月 132013 12月 15

出版系列

名字Proceedings - 2013 IEEE International Conference on Granular Computing, GrC 2013

Other

Other2013 IEEE International Conference on Granular Computing, GrC 2013
國家/地區China
城市Beijing
期間13-12-1313-12-15

All Science Journal Classification (ASJC) codes

  • 軟體

指紋

深入研究「MC framework: High-performance distributed framework for standalone data analysis packages over Hadoop-based cloud」主題。共同形成了獨特的指紋。

引用此