Decentralized Multi-Agent Reinforcement Learning for Traffic Signal Control

Abstract

Multi-Agent reinforcement learning (MARL) is a promising method for traffic signal control (TSC) to optimize the traffic efficiency of a road network. However, existing MARL algorithms for multi-intersection TSC are mostly based on the centralized training and decentralized execution (CTDE) framework, which suffers from a heavy communication burden and the single-point failure problem. This paper proposes a fully decentralized MARL framework for the multi-intersection traffic signal control problem, and the main challenge is how to optimize the TSC model performance with only communication among neighbors. For this problem, we propose a neighboring information fusion method to represent traffic flows' spatial coupling and temporal dependency, and design a decentralized framework to aggregate local models with only communication among neighbors. The simulation study demonstrates that the proposed neighboring information fusion and decentralized learning method can greatly improve agents' performance for the traffic signal control problem, and the proposed NIF-Decentralized method exhibits superior performance against state-of-the-art methods.