Multiagent Reinforcement Learning Algorithm for Distributed Dynamic Pricing of Managed Lanes

Abstract

Priced managed lanes are increasingly being used to manage congestion on urban freeways. This research looks at a distributed model of dynamic pricing for managed lanes with multiple entrances and exits with a toll agent controlling the toll at each diverge point. We formulate the problem as a Markov decision process for each agent and use a multiagent reinforcement learning algorithm to find toll policies that maximize revenue over a finite time horizon. We also propose a local policy search method which explores the continuous action space without the need to discretize tolls. We compare the performance of the distributed control against other heuristics used in practice. Experiments conducted on the test networks show promising results. The proposed algorithm generates 70-86% more revenue than the other heuristics which assume no coordination and produces toll policies which reduce the violation of free flow travel on managed lanes to only 5% of the times. Despite showing lack of convergence within a reasonable computation time, the proposed algorithm generates toll policies which perform better than the existing heuristics and provides a viable alternative for dynamic tolling of managed lanes with multiple toll locations.