Urban Traffic Control Using Adjusted Reinforcement Learning in a Multi-agent System

Abstract

Dynamism, continuous changes of states and the necessity to respond quickly are the specific characteristics of the environment in a traffic control system. Proposing an appropriate and flexible strategy to meet the existing requirements is always an important issue in traffic control. This study presents an adaptive approach to control urban traffic using multi-agent systems and a reinforcement learning augmented by an adjusting pre-learning stage. In this approach, the agent primarily uses some statistical traffic data and then uses traffic engineering theories for computing appropriate values of the traffic parameters. Having these primary values, the agents start the reinforcement learning based on the basic calculated information. The proposed approach, at first finds the approximate optimal zone for traffic parameters based on traffic engineering theories. Then using an appropriate reinforcement learning, it tries to exploit the best point according to different conditions. This approach was implemented on a network in traffic simulator software. The network was composed of six four phased intersections and 17 two lane streets. In the simulation, pedestrians were not considered in the system. The load of the network is defined in terms of Origin-Destination matrices whose entries represent the number of trips from an origin to a destination as a function of time. The simulation ran for five hours and an average traffic volume was used. According to the simulation results, the proposed approach behaved adaptively in different conditions and had better performance than the theory-based fixed-time control.