Real-Time Energy Harvesting Aided Scheduling in UAV-Assisted D2D Networks Relying on Deep Reinforcement Learning

Abstract

Unmanned aerial vehicle (UAV)-assisted device-to-device (D2D) communications can be deployed flexibly thanks to UAVs' agility. By exploiting the direct D2D interaction supported by UAVs, both the user experience and network performance can be substantially enhanced at public events. However, the continuous moving of D2D users, limited energy and flying time of UAVs are impediments to their applications in real-time. To tackle this issue, we propose a novel model based on deep reinforcement learning in order to find the optimal solution for the energy-harvesting time scheduling in UAV-assisted D2D communications. To make the system model more realistic, we assume that the UAV flies around a central point, the D2D users move continuously with random walk model and the channel state information encountered during each time slot is randomly time-variant. Our numerical results demonstrate that the proposed schemes outperform the existing solutions. The associated energy efficiency game can be solved in less than one millisecond by an off-the-shelf processor using trained neural networks. Hence our deep reinforcement learning techniques are capable of solving real-time resource allocation problems in UAV-assisted wireless networks.