A fast large-size production data transformation scheme for supporting smart manufacturing in semiconductor industry

In this paper, we propose a novel data transformation scheme over a big data platform, aiming at injecting production data from the local database in the factory side and transforming them into the workpiece-centric form that many manufacturing analytics systems need. The key idea is to blend big data processing techniques, including table composition with external distributed files, columnar storage, partition, massively parallel processing into the data transformation scheme for minimizing the data processing time. Our proposed scheme brings two main impacts to the smart manufacturing. First, our scheme plays the key component to develop data-driven manufacturing decision systems, since large-volume production data sources can be efficiently transformed into the workpiece-centric form that other smart manufacturing services require. Second, our proposed scheme provides a development exemplar to assist a manufacturing factory toward the Industry-4.0 realm, since big data techniques are ingeniously blended in building data-intensive manufacturing services. We finally implement the prototype of the proposed scheme on the Hadoop platform and apply the prototype to a semiconductor factory for conducting integrated tests. Testing results of a case study physically applying the proposed scheme to a semiconductor factory demonstrate the success of our work.