Distributed Reinforcement Learning for Privacy-Preserving Dynamic Edge Caching

Abstract

Mobile edge computing (MEC) is a prominent computing paradigm which expands\nthe application fields of wireless communication. Due to the limitation of the\ncapacities of user equipments and MEC servers, edge caching (EC) optimization\nis crucial to the effective utilization of the caching resources in MEC-enabled\nwireless networks. However, the dynamics and complexities of content\npopularities over space and time as well as the privacy preservation of users\npose significant challenges to EC optimization. In this paper, a\nprivacy-preserving distributed deep deterministic policy gradient (P2D3PG)\nalgorithm is proposed to maximize the cache hit rates of devices in the MEC\nnetworks. Specifically, we consider the fact that content popularities are\ndynamic, complicated and unobservable, and formulate the maximization of cache\nhit rates on devices as distributed problems under the constraints of privacy\npreservation. In particular, we convert the distributed optimizations into\ndistributed model-free Markov decision process problems and then introduce a\nprivacy-preserving federated learning method for popularity prediction.\nSubsequently, a P2D3PG algorithm is developed based on distributed\nreinforcement learning to solve the distributed problems. Simulation results\ndemonstrate the superiority of the proposed approach in improving EC hit rate\nover the baseline methods while preserving user privacy.\n