Multi-Agent Deep Reinforcement Learning Based Incentive Mechanism for Multi-Task Federated Edge Learning

Abstract

Federated edge learning (FEL) is capable of training large-scale machine learning models without exposing the raw data of edge devices (EDs). Considering that the learning performance heavily depends on the active participation of EDs, it is essential to motivate the resource-limited EDs to contribute their efforts to learning tasks. In this paper, a learning-based multi-task FEL mechanism is proposed to design the economic incentive and participation contribution strategy jointly. Specifically, the incentive-based interaction between the edge servers and EDs is formulated as a multi-leader multi-follower Stackelberg game. Then, the theoretical analysis is provided to prove the existence and uniqueness of the Stackelberg equilibrium. To obtain the equilibrium solution under the incomplete information, a Markov decision process is formulated for the two-stage Stackelberg game. Considering the high dimensionality of the continuous action space, a multi-agent double actors deep deterministic policy gradient algorithm is employed to achieve the optimal training-ratio of EDs and the payment policies of edge servers. Numerical results validate the effectiveness and efficiency of our proposed incentive mechanism.