CCSF: Clustered Client Selection Framework for Federated Learning in non-IID Data

Abstract

Federated Learning (FL) is a distributed approach where numerous devices train a shared global model for Machine Learning (ML) tasks. At every training round, the client devices must share their new local gradients with the central server to update the global model. Hence, FL requires high communication costs in terms of bandwidth and the number of messages exchanged between FL clients and the central server, leading to many issues, such as communication bottlenecks and scaling in the network. Consequently, having all devices participating in every training round is not practical. Moreover, the devices' local datasets are usually not Independent and Identically Distributed (IID), posing additional challenges for training the global model. In this sense, we introduce a Clustered Client Selection Framework (CCSF) to decrease the overall communication costs for training an ML model in the FL environment. CCSF clusters the client devices and employs a biased client selection strategy with two main objectives: (i) reducing the number of devices training at every round; and (ii) the number of rounds required to reach convergence. Our experimental evaluations, conducted on two well-known datasets, MNIST and MotionSense, show that CCSF is highly efficient, where the clients' local datasets can be grouped into homogeneous clusters. In MNIST, CCSF reaches an accuracy score above 60% in less than 50 rounds compared to FedAvg at 50% after 100 rounds. The performance gap is wider in the MotionSense data. CCSF reaches an accuracy score of 70% in a little more than 20 training rounds compared to FedAvg below 30% of accuracy in the first 100 FL rounds.