Two-Tier Resource Allocation for Multitenant Network Slicing: A Federated Deep Reinforcement Learning Approach

Abstract

Fifth-generation (5G) wireless networks enable gigabit-per-second data speeds, minimal latency, and reliable Internet of Things (IoT) connectivity. Thus, network slicing (NS) has gained enormous interest due to its ability to improve resource allocation. Due to the exponential growth of IoT data, it is difficult for the infrastructure providers (InPs) to determine the appropriate resource to allocate to mobile virtual network operators (MVNOs). In addition, MVNOs and IoT devices may use self-serving tactics that cause MVNOs to violate service level agreements (SLAs). Therefore, a fundamental problem in NS is capturing the interaction between MVNOs and IoT devices and ensuring efficient use of InP resources. This article proposes a two-tier resource allocation technique for NS involving a monopolistic market between an InP, multiple MVNOs, and IoT devices. First, we model the upper tier problem as a Markov decision problem (MDP) and design a federated deep reinforcement learning-based resource allocation algorithm (FDRL-RA) to explore the optimization solution. At the lower tier, we model a trading market between MVNOs and IoT devices as a two-stage Stackelberg game, where MVNOs set their unit prices and IoT devices set their purchase quantities. We use the backward induction method to analyze the proposed Stackelberg game under a competitive pricing scheme (CPS) and independent pricing scheme (IPS), which ensures high MVNOs' profit and users' utility at acceptable levels. Simulation results show that our proposed algorithm converges to the optimal solution and effectively maximizes utility under different pricing schemes while providing a high degree of privacy.