Joint Multi-UAV Deployment and Resource Allocation Based on Personalized Federated Deep Reinforcement Learning

Abstract

Unmanned aerial vehicles (UAVs) are capable of serving as aerial base stations (BSs) for providing dynamic coverage and connectivity extension for the sixth-generation (6G) wireless networks. While flexibility is provided, the deployment of the UAV swarms and the associated resource allocation bring challenging issues due to dynamic nature of UAVs and difficulty in obtaining global user information. In this paper, we propose an adaptive and flexible joint UAV deployment and resource allocation scheme by exploiting a personalized federated deep reinforcement learning framework, called PFRL, with aim to maximize the long-term network throughput while enforcing user privacy and adapting to time-varying network states. To allow UAVs to make real-time decisions on resource allocation and position adjustment based on local observations while achieving a global optimal solution, we incorporate deep reinforcement learning (DRL) into federated learning framework. Specifically, we use DRL to train a local model and a personalized model on UAVs, and employ a two-level parameter aggregation scheme on a leading UAV to form a global model. The personalized model can adapt to specific environments, while exploiting the generalization of global model to accelerate the learning convergence. Numerical results show that the proposed PFRL scheme can achieve significant performance gain in terms of network throughput and convergence in comparison with some state-of-art solutions.