Deep Reinforcement Learning Based Real-Time Renewable Energy Bidding With Battery Control

Abstract

Recently, various renewable energy sources and large-scale batteries have been integrated into power grids, and renewable energy bidding and battery control become critical problems in the real-time energy market. However, bidding and control problems have been studied separately while these two problems simultaneously influence the total profit of renewable producers. In this paper, we propose a novel strategy where renewable energy bidding and battery control are collectively investigated. First, unlike the previous studies where bidding is simply the forecasted value, the proposed methods determine the bidding values considering the error compensability of the battery by switching the objective of forecasting from reducing errors to making errors compensable. After the error compensation, additional battery control is applied to utilize the energy arbitrage process considering the energy price. As there are energy price and renewable generation uncertainties, we propose a deep reinforcement learning based bidding combined with control, called DeepBid, for sequential decision making under uncertainty. Our extensive simulations with real solar and wind generation data show that the proposed DeepBid strategy substantially increases the total profit compared to existing bidding strategies by achieving as high revenues as the arbitrage strategy and as low deviation penalties as the error compensation strategy.