The Case of Big Data Platform Services for Semiconductor Wafer Fabrication Foundries

Andy Rk Chang, Yu Ling Chen, Po Yu Chou, Yen Zhou Huang, Hung Chang Hsiao, Tsung Ting Hsieh, Michael Hsu, Chia Chee Lee, Hsin Yin Lee, Yun Chi Shih, Wei An Shih, Chien Hsiang Tang, Chia Ping Tsai, Kuan Po Tseng

研究成果: Conference contribution

1 引文 斯高帕斯(Scopus)

摘要

We present in this paper two novel infrastructural services based on Hadoop for big data storage and computing in a Taiwan's semiconductor wafer fabrication foundry. The two services include Hadoop data service (HDS) and distributed R language computing service (DRS), which have been built and operated in production systems for 3.5 years. They evolve over time by incrementally accommodating users' requirements. HDS is a web- based distributed big data storage facility. Users simply rely on HDS to access data objects stored in Hadoop with the HTTP protocol. In addition, HDS is scalable and reliable. Moreover, HDS is efficient and effective by intelligently selecting either Hadoop distributed file system (HDFS) or database (HBase) for publishing data objects. Specifically, HDS is transparent to existing analytics and data inquiry applications, such as Spark and Hive. While HDS is a unified storage for supporting sequential and random data accesses for big data in the wafer fabrication foundry, DRS is a distributed computing framework for typical R language users. R users employ DRS to enjoy data-parallel computations, effortlessly and seamlessly. Similar to HDS, DRS can be horizontally scaled up. It guarantees the completion of computational jobs even with failures. In particular, it adaptively reallocates computational resources on the fly, minimizing job execution time and maximizing utilization of allocated resources. This paper 1 discusses the design and implementation features for HDS and DRS.

原文English
主出版物標題9th International Conference on Information and Communication Technology Convergence
主出版物子標題ICT Convergence Powered by Smart Intelligence, ICTC 2018
發行者Institute of Electrical and Electronics Engineers Inc.
頁面41-45
頁數5
ISBN(電子)9781538650400
DOIs
出版狀態Published - 2018 十一月 16
事件9th International Conference on Information and Communication Technology Convergence, ICTC 2018 - Jeju Island, Korea, Republic of
持續時間: 2018 十月 172018 十月 19

出版系列

名字9th International Conference on Information and Communication Technology Convergence: ICT Convergence Powered by Smart Intelligence, ICTC 2018

Other

Other9th International Conference on Information and Communication Technology Convergence, ICTC 2018
國家Korea, Republic of
城市Jeju Island
期間18-10-1718-10-19

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications
  • Information Systems
  • Information Systems and Management
  • Artificial Intelligence

指紋 深入研究「The Case of Big Data Platform Services for Semiconductor Wafer Fabrication Foundries」主題。共同形成了獨特的指紋。

  • 引用此

    Chang, A. R., Chen, Y. L., Chou, P. Y., Huang, Y. Z., Hsiao, H. C., Hsieh, T. T., Hsu, M., Lee, C. C., Lee, H. Y., Shih, Y. C., Shih, W. A., Tang, C. H., Tsai, C. P., & Tseng, K. P. (2018). The Case of Big Data Platform Services for Semiconductor Wafer Fabrication Foundries. 於 9th International Conference on Information and Communication Technology Convergence: ICT Convergence Powered by Smart Intelligence, ICTC 2018 (頁 41-45). [8539541] (9th International Conference on Information and Communication Technology Convergence: ICT Convergence Powered by Smart Intelligence, ICTC 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICTC.2018.8539541