Multi-Agent Reinforcement Learning for Mobile Crowdsensing Systems with Dedicated Vehicles on Road Networks

Abstract

Vehicle fleets with on-board sensors hold promise for cost-effective mobile crowdsensing in urban areas. How such a vehicle fleet navigate collectively through a road network is critical for ensuring sufficient spatial-temporal coverage of the sensors to meet domain-specific requirements. In this paper, we develop multi-agent reinforcement learning algorithms (MARL) for centralized vehicle routing on road networks to optimize the spatial-temporal coverage. We construct an environment that is capable of incorporating user-defined weightings in a space-time domain to be covered by mobile sensing. We train the routing policy in the environment with two RL algorithms: proximal policy optimization and deep Q network in a multi-agent setting. Numerical tests on two grid networks (of sizes 20 × 20 and 30 × 30) show the proposed MARL algorithms can improve the performance by at most 56% compared with a heuristic random routing policy. Furthermore, the sensitivity analysis against different fleet sizes implies that a small number of dedicated vehicles is able to approach the limit of coverage for squared road networks. The codes for numerical experiments can be accessed at https://github.com/SpartanBin/mobile_crowd_sensing.