A Novel Big Data Processing Approach to Feature Extraction for Electrical Discharge Machining based on Container Technology

Denata Rizky Alimadji, Min Hsiung Hung, Yu Chuan Lin, Benny Suryajaya, Chao Chun Chen

研究成果: Conference contribution

摘要

EDM (Electrical Discharge Machining) is a process to remove metal from conductive materials using electrical sparks. To monitor the EDM process using virtual metrology (VM), we need to obtain the electrode's voltage and current signals of a machine tool. Due to the nature of EDM, the sensors installed on the machine tool acquire the signals at a high sampling rate and generate a vast amount of data in a short time, thereby raising the big-data processing issue. Our previous work proposed an efficient approach called BEDPS to process the EDM big data in a Hadoop distributed cluster. This paper presents a novel big data processing approach to feature extraction for EDM by using container technology (i.e., Docker and Kubernetes). We re-implement some Spark algorithms of BEDPS in Python (originally in Scala) and then run the refined BEDPS in containers in a Kubernetes cluster. Testing results show that the refined BEDPS developed in this study can reduce the execution time by almost half, compared to the original Scala version (9.6577 minutes vs. 19.2735 minutes). The adoption of Python in Spark is also shown to have similar performance with Scala, although there are some cases where Python performance falls short, for example, parallel processing using Python parallel processing library. The results also show that the Kubernetes cluster is promising to be an alternative way, other than the Hadoop, for processing big data. At the same time, it can bring some advantages to the big data processing applications, such as easy deployment, robustly running, load balance, self-healing, failover, and horizontal auto-scaling for containerized applications.

原文English
主出版物標題Proceedings - 22nd IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2021-Fall
編輯Her-Terng Yau, Roland Stenzel, Mei-Ling Shyu, Hsiung-Cheng Lin
發行者Institute of Electrical and Electronics Engineers Inc.
頁面142-147
頁數6
ISBN(電子)9781665404037
DOIs
出版狀態Published - 2021
事件22nd IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2021-Fall - Virtual, Taichung, Taiwan
持續時間: 2021 11月 242021 11月 26

出版系列

名字Proceedings - 22nd IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2021-Fall

Conference

Conference22nd IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2021-Fall
國家/地區Taiwan
城市Virtual, Taichung
期間21-11-2421-11-26

All Science Journal Classification (ASJC) codes

  • 人工智慧
  • 電腦網路與通信
  • 電腦視覺和模式識別
  • 硬體和架構
  • 軟體
  • 資訊系統與管理

指紋

深入研究「A Novel Big Data Processing Approach to Feature Extraction for Electrical Discharge Machining based on Container Technology」主題。共同形成了獨特的指紋。

引用此