Visual navigation for mobile robots based on deep reinforcement learning

Abstract

Since visual navigation algorithms based on deep reinforcement learning (DRL) suffer from insufficient extraction of spatio-temporal features of images and poor target point-oriented exploration, we propose a new visual navigation framework. The input of the framework consists of three components: the depth images generated by the depth camera at successive moments, the current state of the mobile robot, and the relative position of the target point, which outputs the control commands of the mobile robot in an end-to-end manner through DRL. In addition, we design a feature extractor based on convolutional long short-term memory network (ConvLSTM), which not only captures the correlation between the depth images and target point information at consecutive moments, but also better obtains the key spatial layout and corresponding temporal features from the time-series images, thus reducing the exploration time. The designed network is trained in an unknown simulation environment and random Gaussian noise is added to the depth images to enhance the generalization ability of the network. The results show the algorithm achieves better results in terms of learning rate, convergence and navigation success rate, and also shows good generalization ability across scenes.