FedCorr: Multi-Stage Federated Learning for Label Noise Correction

Jingyi Xu, Zihan Chen, Tony Q.S. Quek, Kai Fong Ernest Chong

研究成果: Conference contribution

30 引文 斯高帕斯(Scopus)

摘要

Federated learning (FL) is a privacy-preserving distributed learning paradigm that enables clients to jointly train a global model. In real-world FL implementations, client data could have label noise, and different clients could have vastly different label noise levels. Although there exist methods in centralized learning for tackling label noise, such methods do not perform well on heterogeneous label noise in FL settings, due to the typically smaller sizes of client datasets and data privacy requirements in FL. In this paper, we propose FedCorr, a general multi-stage framework to tackle heterogeneous label noise in FL, without making any assumptions on the noise models of local clients, while still maintaining client data privacy. In particular, (1) FedCorr dynamically identifies noisy clients by exploiting the dimensionalities of the model prediction subspaces independently measured on all clients, and then identifies incorrect labels on noisy clients based on persample losses. To deal with data heterogeneity and to increase training stability, we propose an adaptive local proximal regularization term that is based on estimated local noise levels. (2) We further finetune the global model on identified clean clients and correct the noisy labels for the remaining noisy clients after finetuning. (3) Finally, we apply the usual training on all clients to make full use of all local data. Experiments conducted on CIFAR-10/100 with federated synthetic label noise, and on a real-world noisy dataset, Clothing1M, demonstrate that FedCorr is robust to label noise and substantially outperforms the state-of-the-art methods at multiple noise levels.

原文English
主出版物標題Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
發行者IEEE Computer Society
頁面10174-10183
頁數10
ISBN(電子)9781665469463
DOIs
出版狀態Published - 2022
事件2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022 - New Orleans, United States
持續時間: 2022 6月 192022 6月 24

出版系列

名字Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
2022-June
ISSN(列印)1063-6919

Conference

Conference2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
國家/地區United States
城市New Orleans
期間22-06-1922-06-24

All Science Journal Classification (ASJC) codes

  • 軟體
  • 電腦視覺和模式識別

指紋

深入研究「FedCorr: Multi-Stage Federated Learning for Label Noise Correction」主題。共同形成了獨特的指紋。

引用此