Grid-Supportive Load Frequency Control using Deep Reinforcement Learning

Abstract

System frequency is one of the main essential variables along with voltage to ensure reliability, stability, and security of power system operation. Currently, maintaining system frequency becomes a concern because of: (i) the integration of renewable resources (i.e., solar, wind) in mixed energy, (ii) the requirement of real-time techniques to tune classical controller's gain, and (iii) the requirement of knowing the complex model of new participants. Therefore, the need for a model-free scheme for frequency control is the primary driver to propose a frequency controller based on deep reinforcement learning. In this paper, primary and secondary frequency controllers are investigated. An infinite state-action agent, namely deep deterministic policy gradient (DDPG), is proposed for secondary control along with considering demand response (DR) for primary control. The proposed framework considers DR to enhance primary response in terms of response speed and frequency nadir. The impact of two typical grid-supportive loads is analyzed. The results demonstrate that the proposed framework can effectively respond to various system conditions. The DR can support the grid and allow more renewable energy resources (RES) integration.