Deep Reinforcement Learning for Multi-Objective Resource Allocation in Multi-Platoon Cooperative Vehicular Networks

Abstract

Grouping vehicles into platoons is a promising cooperative driving scenario to enhance the traffic safety and capacity of future vehicular networks. However, fast changing channel conditions in multi-platoon vehicular networks cause tremendous uncertainty for resource allocation. In addition, the unprecedented proliferation of various emerging vehicle-to-infrastructure (V2I) applications may result in some service demands with conflicting quality of experience. In this paper, we formulate a multi-objective resource allocation problem, which maximizes the transmission success ratio of intra-platoon communications and the mean opinion score (MOS) of V2I communication links. To efficiently solve this multi-objective optimization problem, we resort to a deep reinforcement learning (DRL) framework. Specifically, we divide it into a set of scalar optimization subproblems based on the weighted sum approach and model each one as a partially observable stochastic game (P-OSG), where each platoon acts as an agent and the actions taken by all platoons correspond to the resource allocation solution. We further propose a contribution-based dual-clip proximal policy optimization (CD-PPO) algorithm to deal with each subproblem, which is a DRL algorithm based on the actor-critic framework. The network parameters of all subproblems are then optimized collaboratively by using the proposed training algorithm and the neighborhood parameter transfer strategy. The desired Pareto front is obtained when all subproblems are solved. Simulation results reveal that the proposed algorithm can outperform other algorithms in terms of the MOS and transmission success ratio.