Federated matched averaging with information-gain based parameter sampling

Abstract

Federated learning allows edge devices to learn a shared global model from the client’s model parameters while keeping the training data on the device. However, for large models, transmitting all model parameters imposes considerable communication costs, which can be a significant bottleneck in bandwidth-constrained deployments. We present a federated matched averaging algorithm with information-gain based sampling that considerably reduces the number of parameters to be sent by all clients in a federated learning paradigm. Experiments across five benchmark datasets, encompassing symbolic, image and audio data, suggest that our algorithm significantly reduces communication overhead in federated learning settings, without reducing classification accuracy.