Double-Layer Multi-robot Path Planner based on Deep Multiple Dueling Network Agent

Abstract

With its advantages of high efficiency and robustness, the multiple robots system has better application prospects in dynamic and complex environments, such as smart factories. However, when the number of robots increases, it is critical to efficiently coordinate congestion and jams among robots. To address this problem, we design the Double-Layer Multi-robot Path Planner to coordinate the robot conflicts. In the Collaboration Layer, we propose a Deep Multiple Dueling Network Agent( DMDNA) for node distribution based on the idea of forward node distribution to coordinate the path conflict problem of multiple robots. In DMDNA, we design the Multiple Dueling Network(MDN) model based on the DDQN network structure to address the high dimensional discrete action space problem in multiple robots. In addition, to overcome the characteristics of discrete and sparse rewards of reinforcement learning, we add the Hindsight Experience Replay(HER) experience replay strategy in the training of DMDNA. The experience of primary sequences is cut and reused to improve the utilization of samples. In the Motion Layer, we propose the Adaptive-DWA(A-DWA) algorithm, which adds the Target Function. According to the Target Function, the forward simulation time and weight of the evaluation function can be dynamically changed to improve the efficiency of local path planning and dynamic obstacle avoidance. The final experimental results show that the Double-Layer Multi-robot Path Planner based on DMDNA has better success rate and time cost than traditional reinforcement learning methods in solving multi-robot path planning problems. Relevant experimental code: https://github.com/WILL-ZHAO-1/DMDNACODE.git.