An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement Learning

Abstract

Many studies on the application of deep reinforcement learning (DRL) in the field of traffic signal control do not fully consider the influence of vehicles approaching the intersection on traffic flow. In this paper, the convolutional block attention module (CBAM) is incorporated on the basis of the Dueling Double Deep Q Network (D3QN) method to improve the sensitivity of the model to the traffic situation, which can help the model to focus more on the distribution and dynamics of vehicles near intersections. To further improve the model performance, this paper introduces the traffic light phase variable time interval based on the original D3QN method, which helps the model to take into account the traffic requirements in all directions of the intersection. In addition, Double Deep Q Network (Double DQN) and Dueling Deep Q Network (Dueling DQN) technologies are used to further improve the performance of the model. The simulation experiments using the urban traffic simulator SUMO show that the proposed method has significant advantages over D3QN, Maximum Pressure algorithm and Fixed Timing Strategy for key indicators such as mean vehicle delay time, mean queue length and average number of stops. This shows that the method proposed in this paper has great potential in practical traffic signal control applications.