A Reinforcement Learning Based Large-Scale Refinery Production Scheduling Algorithm

Abstract

Refinery production scheduling is a mixed-integer programming problem, which exists the issue of combinational explosion. Thus, solving a large-scale refinery production scheduling problem is time-consuming. This article proposes an approximate solution framework based on reinforcement learning (RL) for large-scale long-time refinery production scheduling problems to rapidly obtain a satisfactory solution. In the proposed algorithm, the Proximal Policy Optimization algorithm is used to process the continuous action. To address the cold start issue of RL in refinery scheduling problem, we present an initialization method for the actor of agent, which utilizes the operation knowledge of tractable small-scale problems to initialize the actor network, and the agent is trained in the environment of large-scale problems. Hence, the convergence of the RL algorithm is greatly accelerated. In addition, the product flowrate concept is used to express the state, making the scheduling agent scalable in terms of scheduling horizon. Experimental studies show, to large-scale refinery scheduling problems, the proposed algorithm can obtain better solutions than that of the CPLEX solver and the existing evolutionary algorithm in a much shorter solving time of the two methods. Note to Practitioners —Scheduling is a link between planning and execution, and it can bring huge economic benefits to the refinery enterprises. With the enlargement of scheduling horizon, the scale of scheduling problems increases dramatically. How to deal with this large-scale scheduling problem caused by a long scheduling horizon is a significant problem. In this paper, the proposed method learns a decision-maker by reinforcement learning and applies to large-scale problems to obtain a good solution quickly. The proposed method is essentially a heuristic algorithm, and it is easy to implement in practice. At present, more and more things will be integrated into one model, leading to the traditional solver cannot meet the application needs. The fast solution method is necessary to be used to solve this problem in the new era.