Multi-agent reinforcement learning for deadlock handling among autonomous mobile robots

Abstract

This dissertation explores the application of multi-agent reinforcement learning (MARL) for handling deadlocks in intralogistics systems that rely on autonomous mobile robots (AMRs). AMRs enhance operational flexibility but also increase the risk of deadlocks. Existing approaches often neglect deadlock handling in the planning phase and rely on rigid control rules that cannot adapt to dynamic operational conditions. To address these shortcomings, this work develops a methodology for integrating MARL into logistics planning. It introduces reference models that explicitly consider deadlocks in multi-agent pathfinding (MAPF) problems. The thesis compares traditional deadlock handling strategies with MARL-based solutions, focusing on PPO and IMPALA under different training and execution modes. Findings reveal that MARL-based strategies, particularly when combined with centralized training and decentralized execution (CTDE), outperform rule-based methods in complex, congested environments.