Deep Reinforcement Learning-Based Load Balancing Algorithm for Sliced Ultra-Dense Network

Abstract

With the deployment of end-to-end network slicing (NS), the flexibility of ultra-dense networks (UDN) can be enhanced to meet diverse requirements of various services. In sliced UDN, load balancing is an important factor affecting network performance and service quality. Especially, handoff strategies have a great influence on load balancing performance. In this paper, the handoff problem considering load balancing issue is modeled as a Markov decision process (MDP), which takes into account the load of each access point, service profit, outage penalty, and handoff cost. A deep reinforcement learning (DRL) based load balancing handoff algorithm is proposed and the double deep Q network (DDQN) is trained to maximize the cumulative reward. The proposed algorithm is proved to be converged by the numerical results and the form of state we set are convinced to improve convergence performance. The proposed algorithm can achieve better load balancing performance compared with traditional algorithms.