Digital Twin Enabled Dual-System Reinforcement Learning Method

Abstract

The emergence of digital twins provides a new technical means for enabling cyber-physical manufacturing systems. When digital twins are applied in workshops, a major problem is how to use virtual-real mapping to connect the cyber and the physical world to improve decision-making and management in the workshop, to enable the self-adaptive scheduling automation of flexible workshops in multi-disturbance and uncertain production environments. This paper studies the adaptive scheduling of flexible workshops and combines the digital twin technology to establish a digitaltwin-based prototype system of self-adaptive scheduling of flexible workshop. Targeting the real-time and adaptability requirements of workshop scheduling in dynamic production environments, and based on the digital twin environment of workshops, this paper selects the average delivery time of workpieces as the optimization target, and proposes a Deep Q-network (DQN) method driven by the perception-cognitive dual system. The perception system generates workshop states using digital twins and knowledge graphs; the cognitive system abstracts the scheduling process into a process sequencing agent to optimize the completion time, and uses a workshop state matrix to describe the constraints of the problem, introducing order selection and workpiece selection actions step by step into the scheduling decision-making process; then a reward function is designed to evaluate the decision. The superiority of the proposed method is verified by the example verification and algorithm comparison analysis in the sewing workshop of a garment factory.