Multi-Agent Deep Reinforcement Learning-based Volt-VAR Control in Active Distribution Grids

Abstract

With increasing penetration of distributed energy resources, voltage fluctuations have become a critical challenge in modern distribution grids. This paper proposes a centrally trained and decentrally executed multi-agent deep reinforcement learning (DRL)-based Volt-VAR Control (VVC) for distribution grids with high penetration of photovoltaics (PVs) generation. Unlike existing DRL-based VVC frameworks, the proposed approach does not depend on system modeling in both training and execution phases. The proposed multi-agent soft actor-critic (MASAC) approach uses historical data to effectively learn the optimal coordinated control policy by applying counter-training on local policy networks and central critic networks. The agents control optimal set-points of the reactive power output of PV inverters to improve the voltage profile. The performance of the proposed approach is tested on a modified version of the IEEE 34-bus test case with different load and PV profiles and the results are compared with a base case scenario, i.e., no action is taken by the agents. The results show that the proposed multi-agent deep reinforcement learning (MADRL) framework can effectively improve the voltage profile of the network under any PV generation or loading scenario.