QoE-Driven Content-Centric Caching With Deep Reinforcement Learning in Edge-Enabled IoT

Abstract

Edge-enabled Internet of Things (IoT) services for users are subject to intelligent management of content-centric caching. Although managing edge caching can reduce storage cost and transmission latency, maintaining a high Quality of Experience (QoE) of caching is still a crucial challenge. In this environment, we study QoE-based content-centric caching. To evaluate the qualities of edge-enabled IoT, we introduce a QoE model which can grasp the influencing factors: (1) storage cost, based on available bandwidth, and (2) transmission latency, depending on the Signal-to-Interference-plus-Noise Ratio (SINR) and caching capacity. As the requirements and signals are stochastic, we use a Reinforcement Learning (RL) architecture to jointly determine the Q-value. Estimating the Q-value, constrained by a maximum QoE, can be conducted in a Deep Neural Network (DNN) approximator, as the states and action spaces are on a large scale. Unfortunately, training DNN models can lead to RL instability. To address this issue, fixed target network, experience replay, and adaptive learning rate methods are proposed to balance the Q-value accuracy and accelerate stability in Deep RL (DRL). Experimental results indicate that our approach can gain a higher value of QoE, compared to existing methods.