Deep Deterministic Policy Gradient-Based Algorithm for Computation Offloading in IoV

Abstract

The continuous evolution of cellular networks has resulted in the rapid increase in both mobile applications and devices in the Internet of Vehicles. The introduction of the multi-access edge computing method makes it possible for vehicles in remote areas to offload their computational tasks, which can effectively relieve the computing pressure of local devices and reduce the computational delay as well. Tasks offloading for multi-user is a resource competition problem, especially in dynamic environments, which is difficult to be solved by traditional algorithms. In this article, we propose a two-layer hybrid system with local and edge computing, providing convenient computing and offloading services for vehicle users in dual dynamic scenarios of task generation and vehicle mobility. The delay and queuing situations are considered comprehensively in the formulated optimization problem, which can be solved by the proposed deep deterministic policy gradient-based computation offloading algorithm. The offloading process of the vehicle tasks in dynamic scenarios is transformed into a Markov decision process to obtain the offloading strategy. Simulation results demonstrate the performance advantages of two-tier computing architecture. Compared with random offloading, deep Q network-based offloading, and local computing, the algorithm proposed in this article gains the highest average reward of tasks. Besides that, numerical results also prove that our algorithm has the lowest average delay under different computing capabilities of edge servers.