Deep reinforce learning and meta-learning based resource allocation in cellular heterogeneous networks

Abstract

Heterogeneous networks (HetNets) can effectively increase system capacity and improve coverage and throughput through flexible deployment. However, it also suffers critical challenges in rational user allocation and resource allocation due to the increase of base stations, users and complex interference scenarios. To tackle these problems, we propose a multi-agent prioritized experience replay and Dueling double deep Q network (MAPD3QN) algorithm to ensuring users' quality of service (QoS) in HetNets by achieving optimal user association and resource allocation. Specifically, we introduced multi-agent reinforcement learning approach into the optimization problem, where optimal policy is learned through interacting with the environments rather than channel state information. Next, to cope with the large action space and faster convergence speed, the Dueling deep Q-network (DQN) architecture is employed. Moreover, double-network and Prioritized Experience Replay methods are explored in dueling DQN to prevent overestimation and increase the utilization of valuable experience samples, which further improves the system capacity. Experiments show that the proposed MAPD3QN method can achieve efficient user-associated base station and channel allocation with fast convergence, and high capacity while ensuring QoS.