Reinforcement learning based mapless robot navigation

Abstract

Navigation is the one of the most fundamental capabilities required for mobile robots, allowing them to traverse from a source to a destination. Conventional approaches rely heavily on the existence of a predefined map which is costly both in time and labour to acquire. In addition, maps are only accurate at the time of acquisition and due to environmental changes degrade over time. We argue that this strict requirement of having access to a high-quality map fundamentally limits the realisability of robotic systems in our dynamic world. In this thesis, we investigate how to develop practical robotic navigation, motivated by the paradigm of mapless navigation and inspired by recent developments in Deep Reinforcement Learning (DRL). One of the major issues for DRL is the requirement of a diverse experimental setup with millions of repeated trials. This clearly is not feasible to acquire from a real robot through trial and error, so instead we learn from a simulated environment. This leads to the first fundamental problem which is that of bridging the reality gap from simulated to real environments, tackled in Chapter 3. We focus on the particular challenge of monocular visual obstacle avoidance as a low-level navigation primitive. We develop a DRL approach that is trained within a simulated world yet can generalise well to the real world. Another issue which limits the adoption of DRL techniques for mobile robotics in the real world is the high variance of the trained policies. This leads to poor convergence and low overall reward, due to the complex and high dimensional search space. In Chapter 4, we leverage simple classical controllers to provide guidance to the task of local navigation with DRL, avoiding purely random initial exploration. We demonstrate that this novel accelerated approach greatly reduces sample variance and significantly increases achievable average reward. The last challenge we consider is that of sparse visual guidance for mapless navigation. In Chapter 5, we present an innovative approach to navigate based on a few waypoint images, in contrast to traditional video based teach and repeat. We demonstrate that the policy learnt in simulation can be directly transferred to the real world and has ability to generalise well to unseen scenarios with minimal description of the environment. We develop and test novel approaches towards the key issues of obstacle avoidance, local guidance and global navigation, towards our vision of enabling practical robotic navigation. We show how DRL can be used as a powerful model-free approach to tackle these issues.