Dynamic flexible job shop scheduling based on deep reinforcement learning

Abstract

The unpredictable dynamic events in smart factory seriously influence the scheduling schemes and production efficiency. To minimize the total tardiness of orders, this paper proposes a Deep Reinforcement Learning (DRL) method to solve the Dynamic Flexible Job Shop Scheduling Problem (DFJSP) with random job arrival. In the scheduling process, the intelligent agent can select the operations to be processed on the available machines according to the job shop state at each scheduling point by transforming DFJSP into a Markov Decision Process (MDP). For reflecting the job shop environment, eight state features are extracted, and six composite dispatching rules are designed as the action space. Moreover, a reward function is developed to enhance the learning efficiency and solution quality of the intelligent agent, the Soft-max selection strategy is applied to balance exploration and exploitation. The Dueling Double Deep Q-Network (Dueling DDQN) algorithm is introduced to train the intelligent agent, which effectively alleviates the overestimation problem. Numerical simulation results show that the proposed Dueling DDQN method effectively solves the DFJSP. Compared with existing scheduling strategies, the proposed approach not only exhibits superior performance but also maintains its advantages among different scale instances.