A Novel Big Data Processing Approach to Feature Extraction for Electrical Discharge Machining based on Container Technology

Denata Rizky Alimadji, Min Hsiung Hung, Yu Chuan Lin, Benny Suryajaya, Chao Chun Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

EDM (Electrical Discharge Machining) is a process to remove metal from conductive materials using electrical sparks. To monitor the EDM process using virtual metrology (VM), we need to obtain the electrode's voltage and current signals of a machine tool. Due to the nature of EDM, the sensors installed on the machine tool acquire the signals at a high sampling rate and generate a vast amount of data in a short time, thereby raising the big-data processing issue. Our previous work proposed an efficient approach called BEDPS to process the EDM big data in a Hadoop distributed cluster. This paper presents a novel big data processing approach to feature extraction for EDM by using container technology (i.e., Docker and Kubernetes). We re-implement some Spark algorithms of BEDPS in Python (originally in Scala) and then run the refined BEDPS in containers in a Kubernetes cluster. Testing results show that the refined BEDPS developed in this study can reduce the execution time by almost half, compared to the original Scala version (9.6577 minutes vs. 19.2735 minutes). The adoption of Python in Spark is also shown to have similar performance with Scala, although there are some cases where Python performance falls short, for example, parallel processing using Python parallel processing library. The results also show that the Kubernetes cluster is promising to be an alternative way, other than the Hadoop, for processing big data. At the same time, it can bring some advantages to the big data processing applications, such as easy deployment, robustly running, load balance, self-healing, failover, and horizontal auto-scaling for containerized applications.

Original languageEnglish
Title of host publicationProceedings - 22nd IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2021-Fall
EditorsHer-Terng Yau, Roland Stenzel, Mei-Ling Shyu, Hsiung-Cheng Lin
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages142-147
Number of pages6
ISBN (Electronic)9781665404037
DOIs
Publication statusPublished - 2021
Event22nd IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2021-Fall - Virtual, Taichung, Taiwan
Duration: 2021 Nov 242021 Nov 26

Publication series

NameProceedings - 22nd IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2021-Fall

Conference

Conference22nd IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2021-Fall
Country/TerritoryTaiwan
CityVirtual, Taichung
Period21-11-2421-11-26

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Hardware and Architecture
  • Software
  • Information Systems and Management

Fingerprint

Dive into the research topics of 'A Novel Big Data Processing Approach to Feature Extraction for Electrical Discharge Machining based on Container Technology'. Together they form a unique fingerprint.

Cite this