Accelerating federated learning based on grouping aggregation in heterogeneous edge computing

Abstract

Recently, edge devices such as mobile phones and smartwatches have become part of modern distributed systems, federated learning is an effectively distributed learning paradigm that can leverage these edge devices to collaboratively train models without sharing raw data. In federated learning, the device periodically downloads the model from the server, uses the local data for training, and uploads it to the server, while the servers aggregates params uploaded to update the global model. However, different devices are located in different network environments and have different communication and computation capability. Therefore, the model training speed depends on the slowest device, and the system between devices is heterogeneous. To effectively address these problems, we propose to group the devices, firstly use the synchronous method to aggregate model updates within a group, then aggregate updates between groups in an asynchronous way, and propose an algorithm based on weight update to aggregate models. We conduct extensive simulations on our proposed algorithms, and the results show that they can dramatically accelerate model training while achieving high accuracy.