Deep reinforcement learning for collision avoidance of autonomous ships in inland rivers

Abstract

Due to the characteristics of large inertia, large time delay and non-linearity in ship motion, the manoeuvrability of ships is an important issue related to the safety of ship navigation. The manipulation of ships significantly affects the safety of ship navigation, and its importance will gradually increase with the development of autonomous ships (ASs). Due to the complexity of inland waterways and the density of ships, collisions are common. In this work, an autonomous learning framework with deep reinforcement learning (DRL) was constructed for ASs. The state space, action space, reward function and neural network structure of DRL were designed based on the AS's manoeuvring characteristics and control requirements. A deep deterministic policy gradient (DDPG) algorithm was then used to implement the controller. Finally, some representative route segments of an inland waterway were selected for simulation research based on a virtual simulation environment. The designed DRL controller was able to quickly converge from the training and learning process to meet the control requirements. The effectiveness of the DDPG algorithm was verified by comparisons with the experimental results. This study provides a reference for future research on collision-avoidance technology for ASs in inland rivers.