Supplementary Primary Frequency Control Through Deep Reinforcement Learning Algorithms

Abstract

This work presents the implementation of deep reinforcement learning (DRL) agents as supplementary primary frequency controllers. To achieve this, the primary frequency regulation problem is formulated in a DRL framework, where an actor-critic algorithm, for continuous actions space, is used to change the frequency reference of traditional governors. By modifying this reference, the DRL agent effectively reduces the magnitude of the frequency nadir and rate of change of frequency, thereby enhancing the power grid frequency response. Two DRL algorithms including Deep Deterministic Policy Gradient (DDPG) and Twin Delayed DDPG (TD3) are employed for the frequency regulation. The supplementary control using these two algorithms is tested on a 14-bus, 5-machine test system. The results show that the frequency stability of the grid can be improved by using DRL algorithms as supplementary controllers in the primary frequency regulation.