Deep reinforcement learning-driven bidding strategy for wind-storage systems in energy and frequency regulation markets

Abstract

In the power market environment, participation of wind-storage system in both the energy market and the frequency regulation market is essential to enhance economic efficiency and support grid frequency regulation and peak shaving. However, key issues such as formulating bidding strategies for wind-storage systems in energy-frequency regulation dual markets need to be addressed. A bidding model driven by deep reinforcement learning is proposed in this paper to formulate bidding strategies in an incomplete information market environment. Firstly, a framework for wind-storage systems participating in the energy and frequency regulation markets is established to clarify the bidding operation strategies of each market entity. Then, a real-time frequency regulation performance scoring model is introduced to address the differences in frequency regulation response capabilities among various resources. Based on this, a bidding model for wind-storage systems is developed. Finally, a multi-agent deep reinforcement learning method with strong model-free learning capabilities is employed to solve the stochastic game problem in an incomplete information market environment and to handle the multi-agent bidding game relationship. Simulation results indicate that the proposed method can effectively formulate bidding strategies for wind-storage systems participating in the energy and frequency regulation markets. The method achieves high returns while ensuring high convergence stability. As a result, the economic efficiency of wind-storage systems is enhanced, and grid frequency regulation and peak shaving are effectively supported.