Edge Cloud Resource Scheduling with Deep Reinforcement Learning

Abstract

Designing optimal scheduling algorithms for task allocation in edge cloud clusters presents significant challenges due to the constantly changing workloads and service requests in edge cloud data center environments. These challenges stem from the need to manage the vast amounts of information transmitted by IoT devices, as well as the necessity of offloading computational tasks to cloud data centers. To tackle this issue, we propose a novel deep reinforcement learning-based resource allocation method called Decima#, which offers an effective resource optimization solution for edge cloud data centers. We utilize a transformer architecture to capture resource states on directed acyclic graphs (DAGs), accelerating the aggregation speed of the Graph Neural Network (GNN). Moreover, we develop innovative reward functions and concurrent processing mechanisms to minimize training time. Furthermore, we enhance the Proximal Policy Optimization (PPO) algorithm to improve adaptability, increase the accuracy of likelihood ratio estimation, identify a more suitable activation function, and impose constraints on gradient updates. In simulation environments, Decima# reduced the average job duration by 19% compared to the Decima algorithm, while also achieving a 56% increase in training convergence speed. Code has been made available at https://github.com/limengzhaolihai/spark-decimasharp-ppog.