QoS based Multi-Agent vs. Single-Agent Deep Reinforcement Learning for V2X Resource Allocation

Abstract

Autonomous driving requires Vehicle-to-Everything (V2X) communication as standardized in the 3rd generation partnership project (3GPP). Diverse use cases and service types are foreseen to be supported, including safety-critical use cases, e.g., lane merging and cooperative collision avoidance. Each service type's quality of service (QoS) requirements vary enormously regarding latency, reliability, data rates, and positioning accuracy. In this paper, we analyze and evaluate the performance of a QoS-aware decentralized resource allocation scheme using first, a single-agent reinforcement learning (SARL) and second, a multi-agent reinforcement learning (MARL) approach. In addition, the impact of multiple vehicular user equipments (V-UEs) supporting one and multiple services are considered. The QoS parameter considered here is the latency and the relative distance between the communicating V-UEs, which is mapped on the Priority to reflect the required packet delay budget (PDB). The goal is to maximize the throughput of all V2N links while meeting the V2V link's latency constraint of the supported service. The results based on a system-level simulation for an urban scenario show that MARL improves the throughput for V-UEs set up for single and multiple services compared to SARL. However, for latency SARL indicates advantages at least when multiple services per V-UE apply.