Adaptive User Scheduling and Resource Allocation in Wireless Federated Learning Networks: A Deep Reinforcement Learning Approach

Abstract

Federated Learning (FL) is widely regarded as a leading distributed machine learning paradigm, owing to its outstanding performance in preserving privacy and conserving communication resources. To use it efficiently in wireless communication networks, novel transmission schemes that jointly consider the model propagation and training features are required. In this paper, a novel joint user scheduling and resource allocation scheme is proposed to reduce the communication cost in terms of the weighted sum of energy and time consumption while ensuring the convergence of FL. The time-varying channels and unpredictable model loss in the system make it difficult to use conventional optimization methods for this problem. Furthermore, considering optimal transmission policy in FL is to train a qualified model in the dynamic iterative process, a deep reinforcement learning based Proximal Policy Optimization (PPO) approach is employed to train an automatic policy maker. Specifically, the dynamic policy is decided in each training round based on the observed model accuracy and the time-varying channel gains, aiming at minimizing the total cost. Simulation results verify the proposed scheme can reduce the defined communication cost and improve the training efficiency compared with the traditional greedy and random benchmarks.