Zero-Sum Game (ZSG) based Integral Reinforcement Learning for Trajectory Tracking Control of Autonomous Smart Car

Abstract

The ultimate aim of our research study is the development, practical implementation, and benchmarking of continuous-time, online reinforcement learning (RL) schemes for the trajectory tracking control (TTC) of fully autonomous vehicles (AVs) in real-world scenarios. The adaptive optimality and model-free nature offered by RL has a stronger promise against its model-based counterparts, such as MPC, against uncertainties related to the vehicle, road, tire-terrain and environmental dynamics. The existing studies on RL based AV control are mostly theoretical, often dealing with high-level TTC, and perform evaluations in simulations considering simplified or linear models with no disturbance and slip effects. The literature also demonstrates the lack of practical implementations in overall RL based autonomous vehicle control. Our ultimate goal is to fill these theoretical and practical gaps by designing and practically evaluating novel RL strategies that will improve the performance of TTC against uncertainties at all levels. This paper presents the simulation results of our preliminary studies in the online, longitudinal tracking control of a realistic AV (with uncertain nonlinear dynamics, as well as disturbance, and slip effects), which we treat as a Zero-Sum Game (ZSG) problem using an Integral Reinforcement Learning (IRL) approach with synchronous actor and critic updates (SyncIRL). The results are promising and motivate the practical implementation of the approach for combined longitudinal and lateral control of AV.