Virtual-Action-Based Coordinated Reinforcement Learning for Distributed Economic Dispatch

Abstract

A unified distributed reinforcement learning (RL) solution is offered for both static and dynamic economic dispatch problems (EDPs). Each agent is assigned with a fixed, discrete, virtual action set, and a projection method generates the feasible, actual actions to satisfy the constraints. A distributed algorithm, based on singularly perturbed system, solves the projection problem. A distributed form of Hysteretic Q-learning achieves coordination among agents. Therein, the Q-values are developed based on the virtual actions, while the rewards are produced by the projected actual actions. The proposed algorithm deals with continuous action space and power loads without using function approximations. Theoretical analysis and comparative simulation studies verify algorithm's convergence and optimality.