Clustered Federated Learning in Internet of Things: Convergence Analysis and Resource Optimization

Federated learning (FL) framework enables user devices to collaboratively train a global model based on their local data sets without privacy leak. However, the training performance of FL is degraded when the data distributions of different devices are incongruent. Fueled by this issue, we consider a clustered FL (CFL) method where the devices are divided into several clusters according to their data distributions and are trained simultaneously. Convergence analysis is conducted, which shows that the clustered model performance depends on cosine similarity, device number per cluster, and device participation probability. Besides, to quantify the training performance, the utility of clustered model training is defined based on the analysis results. Then, aiming at optimizing the system utility, a joint problem of resource allocation and device clustering is formulated, which is solved by decoupling it into two subproblems. First, given the results of device clustering, a low-complexity iterative algorithm based on the convex optimization theory is proposed to make the bandwidth allocation and the transmit power control. Then, according to the individual stability, a coalition formation algorithm is proposed for the device clustering. Finally, the real-data experiments on the classification tasks (e.g., MNIST, CIFAR-10, and CIFAR-100) validate the results of convergence analysis and advantages of the proposed algorithm in terms of the test accuracy.