Deep Reinforcement Learning Based Resource Allocation for Heterogeneous Networks

Abstract

This paper investigates the problem of distributed resource management (i.e., joint device association, spectrum allocation, and power allocation) in two-tier heterogeneous networks without any central controller. Considering the fact that the network is highly complex with large state and action spaces, a multi-agent dueling deep-Q network-based algorithm combined with distributed coordinated learning is proposed to effectively learn the optimized intelligent resource management policy, where the algorithm adopts dueling deep network to learn the action-value distribution by estimating both the state-value and action advantage functions. Under the distributed coordinated learning manner and dueling architecture, the learning algorithm can rapidly converge to the optimized policy. Simulation results demonstrate that the proposed distributed coordinated learning algorithm outperforms other existing learning algorithms in terms of learning efficiency, network data rate, and QoS satisfaction probability.