Bird’s Eye View Map for End-to-end Autonomous Driving Using Reinforcement Learning

Abstract

End-to-end autonomous driving is one of the hot research topics in the field of artificial intelligence. End-to-end autonomous driving system uses neural networks to take multi-sensor data as input and output relevant parameters such as acceleration, braking force, and steering angle to control the vehicle. With the increasing complexity of sensor configurations in most autonomous vehicles, integrating information from different sensors and representing features in a unified view has become crucially important. The bird's-eye view(BEV) has the advantage of friendly fusion of multiple sensor information into a single image and providing a comprehensive and intuitive understanding of the surrounding scene without the issues of occlusion and scale. It is particularly suitable for training decision-making tasks in autonomous driving systems. The BEV map is derived using the BEVFusion method, which leverages camera images and other sensor data captured by the multiple sensors installed on an ego vehicle. In this paper, we utilize BEV map as the input for end-to-end autonomous driving system and employ reinforcement learning algorithms to investigate the impact of the bird's-eye view on decision-making. The results demonstrate that instances of error frames from BEV perception can have a significant impact on the reinforcement learning in end-to-end autonomous driving system.