Deep-Reinforcement-Learning-Based AoI-Aware Resource Allocation for RIS-Aided IoV Networks

Abstract

Reconfigurable intelligent surface (RIS) is a pivotal technology in communication, offering an alternative path that significantly enhances the link quality in wireless communication environments. In this paper, we propose a RIS-Assisted internet of vehicles (IoV) network and consider the vehicle-To-everything (V2X) communication method. In order to improve the timeliness of vehicle-To-infrastructure (V2I) links and the stability of vehicle-To-vehicle (V2V) links, we introduce the age of information (AoI) and the payload transmission probability models. Thus, we aim to minimize the AoI of V2I links and prioritize transmission of V2V links payload. In this framework, the base station (BS) server acts as the agent responsible for resource allocation for vehicles and controlling the phase-shift of the RIS. We use the Soft Actor-Critic (SAC) algorithm to address this problem due to its gradual convergence and high stability in the optimization process. An AoI-Aware joint vehicular resource allocation and RIS phase-shift control scheme based on SAC algorithm is proposed and simulation results show that its convergence speed, cumulative reward, AoI performance, and payload transmission probability outperform those of proximal policy optimization (PPO), deep deterministic policy gradient (DDPG), twin delayed deep deterministic policy gradient (TD3) and stochastic algorithms.