An effective deep reinforcement learning approach for adaptive traffic signal control

Abstract

Intelligent traffic signal timing is critical to reduce traffic congestion and vehicle delay. Recent studies have shown promising results of deep reinforcement learning for traffic signal control. However, existing studies have only focused on selecting which direction (phase) to let vehicles go, not on phase duration. In this paper, we propose a deep reinforcement learning algorithm that automatically learns an optimal policy to adaptively determine phase duration. To improve algorithm performance and stability, we propose a phase sensitive neural network structure based on the deep deterministic policy gradient (DDPG) model, i.e. we design a deep neural network controller for each specific traffic signal phase with DDPG; we develop some interesting training techniques to improve training efficiency, i.e. dividing the training process into three stages and introducing the episode-break mechanism. We test the proposed methods on an isolated intersection under diverse traffic demands. Experiments show that our method is more effective.