A Fast and Scalable Cluster Simulator for Network Performance Projection of HPC Applications

Cheng Yueh Liu, Po Yao Huang, ChiaHeng Tu, Shih Hao Hung

研究成果: Conference contribution

1 引文 斯高帕斯(Scopus)

摘要

For high-performance computing (HPC) cluster to solve distributed computing problems with big data, the performance is usually bounded by communication ability of each node and the underlying network architecture of the cluster. Network simulations are commonly adopted to evaluate the impacts of network configurations before the deployment of such a high-cost system, and a fast and scalable network performance simulation with profiling mechanism is crucial to explore the design space in time effectively. This work presents Clusim, a cluster simulator for exploration of the performance of interplay among distributed HPC applications and network architectures, where three primary techniques are developed in Clusim·. 1) the full system emulators forming a virtual cluster for collecting the network communication traces, 2) window-based simulation, and 3) phase-Aware accelerated network simulation. The first technique helps to reduce the cost and efforts of establishing a physical cluster system for performance estimation. The second one enables cross-platform performance projection by thoroughly preserving the concurrency of the recorded packet flows during simulation. The last one efficiently reduces the simulation time by skipping the detailed simulations of repeated phases of network traffics. We validate Clusim against a physical cluster and show that the difference of the estimated end-To-end delays of the simulated network and those measured on the physical counterpart is within 15%. Meanwhile, thanks to the phase-Aware simulation scheme, Clusim achieves up to 18x speedups, compared with the packet-by-packet simulation by ns-3. Our results suggest that Clusim is a promising design and can rapidly provide the network performance projection of a large scale system with the appropriate trade-off between accuracy and time.

原文English
主出版物標題Proceedings - 2018 International Conference on High Performance Computing and Simulation, HPCS 2018
編輯Khalid Zine-Dine, Waleed W. Smari
發行者Institute of Electrical and Electronics Engineers Inc.
頁面970-977
頁數8
ISBN(電子)9781538678787
DOIs
出版狀態Published - 2018 十月 29
事件16th International Conference on High Performance Computing and Simulation, HPCS 2018 - Orleans, France
持續時間: 2018 七月 162018 七月 20

出版系列

名字Proceedings - 2018 International Conference on High Performance Computing and Simulation, HPCS 2018

Other

Other16th International Conference on High Performance Computing and Simulation, HPCS 2018
國家France
城市Orleans
期間18-07-1618-07-20

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Networks and Communications
  • Safety, Risk, Reliability and Quality
  • Modelling and Simulation
  • Instrumentation

指紋 深入研究「A Fast and Scalable Cluster Simulator for Network Performance Projection of HPC Applications」主題。共同形成了獨特的指紋。

引用此