The Case of Big Data Platform Services for Semiconductor Wafer Fabrication Foundries

Andy Rk Chang, Yu Ling Chen, Po Yu Chou, Yen Zhou Huang, Hung Chang Hsiao, Tsung Ting Hsieh, Michael Hsu, Chia Chee Lee, Hsin Yin Lee, Yun Chi Shih, Wei An Shih, Chien Hsiang Tang, Chia Ping Tsai, Kuan Po Tseng

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We present in this paper two novel infrastructural services based on Hadoop for big data storage and computing in a Taiwan's semiconductor wafer fabrication foundry. The two services include Hadoop data service (HDS) and distributed R language computing service (DRS), which have been built and operated in production systems for 3.5 years. They evolve over time by incrementally accommodating users' requirements. HDS is a web- based distributed big data storage facility. Users simply rely on HDS to access data objects stored in Hadoop with the HTTP protocol. In addition, HDS is scalable and reliable. Moreover, HDS is efficient and effective by intelligently selecting either Hadoop distributed file system (HDFS) or database (HBase) for publishing data objects. Specifically, HDS is transparent to existing analytics and data inquiry applications, such as Spark and Hive. While HDS is a unified storage for supporting sequential and random data accesses for big data in the wafer fabrication foundry, DRS is a distributed computing framework for typical R language users. R users employ DRS to enjoy data-parallel computations, effortlessly and seamlessly. Similar to HDS, DRS can be horizontally scaled up. It guarantees the completion of computational jobs even with failures. In particular, it adaptively reallocates computational resources on the fly, minimizing job execution time and maximizing utilization of allocated resources. This paper 1 discusses the design and implementation features for HDS and DRS.

Original languageEnglish
Title of host publication9th International Conference on Information and Communication Technology Convergence
Subtitle of host publicationICT Convergence Powered by Smart Intelligence, ICTC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages41-45
Number of pages5
ISBN (Electronic)9781538650400
DOIs
Publication statusPublished - 2018 Nov 16
Event9th International Conference on Information and Communication Technology Convergence, ICTC 2018 - Jeju Island, Korea, Republic of
Duration: 2018 Oct 172018 Oct 19

Publication series

Name9th International Conference on Information and Communication Technology Convergence: ICT Convergence Powered by Smart Intelligence, ICTC 2018

Other

Other9th International Conference on Information and Communication Technology Convergence, ICTC 2018
CountryKorea, Republic of
CityJeju Island
Period18-10-1718-10-19

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications
  • Information Systems
  • Information Systems and Management
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'The Case of Big Data Platform Services for Semiconductor Wafer Fabrication Foundries'. Together they form a unique fingerprint.

  • Cite this

    Chang, A. R., Chen, Y. L., Chou, P. Y., Huang, Y. Z., Hsiao, H. C., Hsieh, T. T., Hsu, M., Lee, C. C., Lee, H. Y., Shih, Y. C., Shih, W. A., Tang, C. H., Tsai, C. P., & Tseng, K. P. (2018). The Case of Big Data Platform Services for Semiconductor Wafer Fabrication Foundries. In 9th International Conference on Information and Communication Technology Convergence: ICT Convergence Powered by Smart Intelligence, ICTC 2018 (pp. 41-45). [8539541] (9th International Conference on Information and Communication Technology Convergence: ICT Convergence Powered by Smart Intelligence, ICTC 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICTC.2018.8539541