Proximal Policy Optimization based sum rate maximization scheme for STAR-RIS-assisted vehicular networks underlaying UAV

Abstract

The consumer electronics industry is undergoing significant transformations due to the ongoing advancements in mobile Internet technology, 5G, Internet of Things (IoT), artificial intelligence (AI), and other emerging technologies. Additionally, the development of intelligent electronic products is accelerating. Higher communication quality is required as a result of the convergence of consumer electronics and developing technologies. The low cost and simple deployment of the Simultaneous Transmitting and Reflecting Reconfigurable intelligent surface (STAR-RIS) can show considerable possibilities. STAR-RIS is a well-known for potentially improving wireless network performance. STAR-RIS enables users positioned on different sides of the surfaces to simultaneously receive signals that are transmitted or reflected. In this article, we examines the difficulties of sum rate maximization in a STAR-RIS assisted downlink network with NOMA assistance, where the incident signal energy at STAR-RIS is divided into two halves for transmitting and reflecting. This dynamic nature of wireless networks makes it challenging to tackle the sum rate maximization problem using the conventional approach of convex optimization techniques. To overcome the difficulties of the sum rate, the proposed scheme uses the Proximal Policy Optimization (PPO) based algorithm based on Deep Reinforcement Learning (DRL) which optimizes the beamforming vectors at the base station and the coefficient matrices and symbol rate at the STAR-RIS. Finally, the performance evaluation demonstrates that the proposed scheme maximizes the system sum rate while considering time-varying channels into account, and the PPO-based algorithm performs better than the Deep Deterministic Policy Gradient (DDPG) algorithm. Also, the results shows that the proposed scheme has 22.05%, 35.12% and 48.9% higher sum rate as compared to DDPG, Zero forcing and random.