Federated Deep Reinforcement Learning for Task Offloading in Digital Twin Edge Networks

Abstract

Digital twin edge networks provide a new paradigm that combines mobile edge computing (MEC) and digital twins to improve network performance and reduce communication cost by utilizing digital twin models of physical objects. The construction of digital twin models requires powerful computing ability. However, the distributed devices with limited computing resources cannot complete high-fidelity digital twin construction. Moreover, weak communication links between these devices may hinder the potential of digital twins. To address these issues, we propose a two-layer digital twin edge network, in which the physical network layer offloads training tasks using passive reflecting links, and the digital twin layer establishes a digital twin model to record the dynamic states of physical components. We then formulate a system cost minimization problem to jointly optimize task offloading, configurations of passive reflecting links, and computing resources. Finally, we design a federated deep reinforcement learning (DRL) scheme to solve the problem, where local agents train offloading decisions and global agents optimize the allocation of edge computing resources and configurations of passive reflecting elements. Numerical results show the effectiveness of the proposed federated DRL and it can reduce the system cost by up to 67.1% compared to the benchmarks.