Pruning Analog Over-the-Air Distributed Learning Models with Accuracy Loss Guarantee

Kailei Xu, Howard H. Yang, Zhongyuan Zhao, Wei Hong, Tony Q.S. Quek, Mugen Peng

研究成果: Conference contribution

1 引文 斯高帕斯(Scopus)


Analog over-the-air computing enables a swarm of end-user devices to efficiently conduct distributed learning, where the intermediate parameters of users, such as gradients, are modulated and transmitted via a group of orthogonal waveforms, and can be mixed directly at a server without individually detecting the feedback parameters of each user. Nonetheless, the scarcity of orthogonal waveforms, as well as communication resources of the end-user devices, are throttling this paradigm in adopting complex deep learning models. To balance the tradeoff between communication efficiency and accuracy performance, we study model pruning for analog over-the-air distributed learning in this paper. First, a model pruning scheme is proposed to improve the communication efficiency of analog over-the-air training. An importance measure for model parameter pruning is also designed based on the analog over-the-air aggregated gradient, which can characterize the contribution of each parameter without removing channel fading and electromagnetic interference. Second, an analytical expression of the training error upper bound is derived, which shows the proposed scheme is able to converge even when the aggregated gradient is corrupted by heavy-tailed electromagnetic interference with an infinite variance. Finally, several experimental results are provided to show the performance gains achieved by our proposed scheme, and also verify the correctness of analytical results.

主出版物標題ICC 2022 - IEEE International Conference on Communications
發行者Institute of Electrical and Electronics Engineers Inc.
出版狀態Published - 2022
事件2022 IEEE International Conference on Communications, ICC 2022 - Seoul, Korea, Republic of
持續時間: 2022 5月 162022 5月 20


名字IEEE International Conference on Communications


Conference2022 IEEE International Conference on Communications, ICC 2022
國家/地區Korea, Republic of

All Science Journal Classification (ASJC) codes

  • 電腦網路與通信
  • 電氣與電子工程


深入研究「Pruning Analog Over-the-Air Distributed Learning Models with Accuracy Loss Guarantee」主題。共同形成了獨特的指紋。