Multi-Agent Deep Reinforcement Learning Based Computation Offloading Approach for LEO Satellite Broadband Networks

Abstract

Conventional computation offloading approaches are originally designed for ground networks, and are not effective for low earth orbit (LEO) satellite networks. This paper proposes a multi-agent deep reinforcement learning (MADRL) algorithm for making multi-level offloading decisions in LEO satellite networks. Offloading is formulated as a partially observable Markov decision process based multi-agent decision problem. Each satellite as an agent either conducts a received task, forwards it to neighbors, or sends it to ground clouds based on its own policy. These agents are independent and their deep neural networks to make offloading decisions share identical parameter values and are trained by using the same replay buffer. A centralized training and distributed executing mechanism is adopted to ensure that agents can make globally optimized offloading decisions. Comparative experiments demonstrate that the proposed MADRL algorithm outperforms the five baselines in terms of task processing delay and bandwidth consumption with acceptable computational complexity.