Robust Autonomous Navigation of Vehicles using Visual Reinforcement Q-learning

Abstract

Abstract Recent advances in the manufacturing technology of vehicles has led to a dramatic increase in their production which in turn has led to the need of efficient and robust autonomous navigation systems. Since the model of the environment in which the vehicle will move is unknown, this makes the task very challenging. In this work we have developed an unsupervised visual reinforcement Q-learning based system for autonomous navigation of vehicles. Here we use the fact that Q-learning is able to learn even without a model of the environment. Q-learning approaches try to predict an action-value function that gives the expected utility of taking a given action in a given state and then following a fixed policy. The testing of the algorithm was done on different simulated environments using an open source simulator TORCS. The input video from vehicle camera is first preprocessed and then edge detection is done. Then some strategically important regions are extracted from the image. The steering angle of the vehicle is predicted from the number of pixels in these strategic regions using Q-learning. Everything is done in real time which makes the proposed algorithm efficient. Simulation and experimental results show that the proposed algorithm can accurately, robustly and efficiently be used for autonomous navigation of vehicles in unknown environments. Keywords: Artificial Intelligence, Neural Networks, Computer Vision