Deep Reinforcement Learning for Energy Minimization in Multi-RIS-Aided Cell-Free MEC Networks

Abstract

In this paper, we investigate the computation offloading problem in a distributed reconfigurable intelligent surface (RIS)-aided cell-free network, where users offload computing-intensive tasks to their associated base stations with the aid of multiple RISs. To minimize the long-term energy consumption of all users under the constraints of latency tolerance, we formulate a long-term non-convex problem by jointly optimizing the offloading strategy, transmit power at users, reflection matrix at the RIS, receive beamforming at the BS, and CPU resources at both users and the server. To solve this intractable time-varying problem with multiple coupling variables, we propose a two-layer distributed proximal policy optimization (DPPO) algorithm for solving the problem with a high-dimensional decision-making space. Simulation results show that the proposed algorithm effectively reduces the long-term energy consumption of all users while completing the computing task within a given time limit.