Dynamic Network Slice Reconfiguration by Exploiting Deep Reinforcement Learning

Abstract

It is widely acknowledged that network slicing can tackle the diverse usage scenarios and connectivity services that the 5G-and-beyond systems need to support. To guarantee performance isolation while maximizing network resource utilization under traffic uncertainty, network slice needs to be reconfigured adaptively. However, it is commonly believed that the fine-grained resource reconfiguration problem is intractable due to the extremely high computational complexity caused by the numerous variables. In this paper, we investigate network slice reconfiguration with aim of minimizing long-term resource consumption by exploiting Deep Reinforcement Learning (DRL). To address the curse of dimensionality of the problem, we propose to incorporate the Branching Dueling Q-network (BDQ) into DRL, to avoid some unnecessary calculations of Q-value by separating the Q-network into a shared value branch and a number of distributed advantage branches. Furthermore, the value branch and the advantage branch of each dimension are aggregated to derive the corresponding dimension’s sub-Q-value. Then the best reconfiguration action is composed of the subactions in individual dimensions which are selected by ϵ−greedy policy. Finally, we design an intelligent online network slice reconfiguration policy based on BDQ and extensive simulation experiments are conducted to validate the effectiveness of the proposed slice reconfiguration policy.