Federated Reinforcement Learning for RIS-Aided Non-Orthogonal Multiple Access MEC

Abstract

A novel reconfigurable intelligent surface (RIS) aided non-orthogonal multiple access mobile edge computing (NOMA-MEC) framework is proposed to release the heavy transmission delay of the edge devices (EDs) in next-generation wireless communication networks. We formulate a stochastic optimization problem that jointly optimizes the phase-shifter design of the RIS, the task offloading decision and the computing resource allocation of MEC, to minimize the overall transmission delay of all EDs in a long-term manner. The mathematical solution for the formulated optimization problem is a long-term offline policy which is non-trivial for conventional optimization approaches due to high computational complexity and stringent delay constraint. Therefore, we propose a federated reinforcement learning (FRL) approach for the formulated optimization problem to obtain the optimal solution taking advantages of the computing resource of all EDs. Moreover, a reputation-enabled ED selection scheme is proposed in the FRL approach that takes the task offloading history into consideration. The proposed RIS-aided NOMA-MEC framework is capable of outperforming conventional orthogonal multiple access (OMA) enabled RIS-MEC networks. The proposed FRL scheme achieves a near-optimal performance when the computional task is image classification in the MNIST and the IRIS dataset.