Intelligent Grid Shepherd: Towards a Resilient Distributed Energy Resources Control System

Abstract

The recent flourish of diversified distributed energy resources (DERs) such as generators, consumers, and prosumers brings indispensable cybersecurity challenges for the smart grid controller. Therefore, to assure a resilient smart grid operation, in this paper, we study the problem of continuous-time consensus policy-based DERs control mechanism for the smart grid controller. In particular, we propose an intelligent grid shepherd for the smart grid controller in the power grid framework. That can autonomously detect the abnormal behavior of the received status message from each DER and apply control decisions into the smart grid controller. To do this, first, we propose a continuous-time Markov decision process problem by formulating a resilient control system for the intelligent grid shepherd. Second, we design a data-informed policy-based model-free reinforcement learning framework to find the optimal consensus policy for each DER control decision (i.e., remain connected with the main grid or disconnected). Thus, we devise a distributed energy resources control algorithm for the intelligent grid shepherd. Particularly, we design an advantage actor-critic scheme under the continuous-time domain with the shared neural network mechanism. Finally, experimental results show the efficiency of the proposed intelligent grid shepherd in terms of accuracy and robustness towards a resilient DERs control.