Parameterized Deep Reinforcement Learning With Hybrid Action Space for Edge Task Offloading

Abstract

Multi-access edge computing (MEC) has emerged as a promising solution that can enable low-end terminal devices to run large complex applications by offloading their tasks to edge servers. The task offloading strategy, determining how to offload tasks, remains the most critical issue of MEC. Traditional offloading approaches either suffer from high computational complexity or poor self-adjustability to dynamic changes in the edge environment. Deep reinforcement learning (DRL) provides an effective way to tackle these issues. However, most existing DRL-based methods solely consider either a continuous or a discrete action space, where the limited action space results in accuracy loss and restricts the optimality of offloading decisions. Nevertheless, the edge task offloading problem in practice often confronts both discrete and continuous actions. In this paper, we propose a tailored Proximal Policy Optimization (PPO)-based method, named Hybrid-PPO, enhanced by the parameterized discrete-continuous hybrid action space. Assisted with Hybrid-PPO, we further design a novel DRL-based multi-server multi-task collaborative partial task offloading scheme adhering to a series of specifically built formal models. Experimental results prove that our approach achieves high offloading efficiency and outperforms the existing state-of-the-art offloading schemes in terms of convergence rate, energy cost, time cost, and generalizability under various network conditions.