Multi-Agent RL for SDN-Based Resource Allocation in HAPS-Assisted IoV Networks

Abstract

The high-altitude platform station (HAPS) is a promising 6G network technology that can meet the stringent requirements for high reliability, ultra-reliable low latency, and large-capacity communications, particularly in vehicular networks. HAPS with aerial computing and intelligent aerial software-defined networks (A-SDN) is a prominent solution to empower vehicles with limited resources. It allows vehicles in any geographical area to offload tasks and allocate resources within the dynamic infrastructure. The traditional MEC-based Internet of Vehicles (IoV) network is suffering from offloading various high data-rate real-time applications to B5G and the upcoming 6G networks. To handle this issue, we propose an intelligent HAPS-enabled IoV network to provide network connectivity, allocate resources, and allow computation in IoV networks. The HAPS is equipped with an aerial computing server and SDN, connected to the backhaul network of satellites and the cloud. The main objective is to maximize the utility of HAPS by jointly optimizing the association and resource allocation strategies of vehicles and other mobile devices. We formulate the optimization problem as a Stackelberg game. However, the formulated problem is complex to solve directly due to dynamism and multi-objective problems. Therefore, we transform it into a stochastic game model and utilize a distributed multi-agent deep reinforcement learning (MADRL) approach. In the proposed MADRL-based HAPS-assisted IoV network, the HAPS and vehicles are intelligent agents. We utilize a multi-agent deep deterministic policy gradient (MADDPG) algorithm to manage the continuous state-action. The simulation results prove that the proposed framework maximizes the network's utility and optimizes the association and resource allocation.