The Case of a Novel Operational Distributed Storage Service for Big Data in a Semiconductor Wafer Fabrication Foundry

Andy R.K. Chang, Yu Ling Chen, Yen Zhou Huang, Hung Chang Hsiao, Michael Hsu, Chia Chee Lee, Hsin Yin Lee, Wei An Shih, Huan Ping Su, Chia Ping Tsai, Kuan Po Tseng

研究成果: Conference contribution

摘要

We present in this paper a novel infrastructural service based on Hadoop for big data storage and computing in a Taiwan's semiconductor wafer fabrication foundry. The service is named Hadoop data service (HDS), which has been built and operated in production systems for 3.5 years. It evolves over time by incrementally accommodating users' requirements. HDS is a web-based distributed big data storage facility. Users simply rely on HDS to access data objects stored in Hadoop with the HTTP protocol. In addition, HDS is scalable and reliable. Moreover, HDS is efficient and effective by intelligently selecting either Hadoop distributed file system (HDFS) or database (HBase) for publishing data objects. Specifically, HDS is transparent to existing analytics and data inquiry applications, such as Spark and Hive. This paper discusses the design and implementation features for HDS. The performance metrics of HDS are also demonstrated.

原文English
主出版物標題Proceedings - 2018 IEEE 24th International Conference on Parallel and Distributed Systems, ICPADS 2018
發行者IEEE Computer Society
頁面1028-1033
頁數6
ISBN(電子)9781538673089
DOIs
出版狀態Published - 2019 二月 19
事件24th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2018 - Singapore, Singapore
持續時間: 2018 十二月 112018 十二月 13

出版系列

名字Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS
2018-December
ISSN(列印)1521-9097

Conference

Conference24th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2018
國家Singapore
城市Singapore
期間18-12-1118-12-13

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture

指紋 深入研究「The Case of a Novel Operational Distributed Storage Service for Big Data in a Semiconductor Wafer Fabrication Foundry」主題。共同形成了獨特的指紋。

  • 引用此

    Chang, A. R. K., Chen, Y. L., Huang, Y. Z., Hsiao, H. C., Hsu, M., Lee, C. C., Lee, H. Y., Shih, W. A., Su, H. P., Tsai, C. P., & Tseng, K. P. (2019). The Case of a Novel Operational Distributed Storage Service for Big Data in a Semiconductor Wafer Fabrication Foundry. 於 Proceedings - 2018 IEEE 24th International Conference on Parallel and Distributed Systems, ICPADS 2018 (頁 1028-1033). [8644546] (Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS; 卷 2018-December). IEEE Computer Society. https://doi.org/10.1109/PADSW.2018.8644546