Q-Learning for Content Placement in Wireless Cooperative Caching

Abstract

Caching during off-peak times can bring popular contents closer to users, and hence improves quality of experience (QoE) of users in wireless networks. We formulate an optimization problem of cooperative content caching, with the aim of maximizing the sum mean opinion score (MOS) of all users in the network. To solve the challenging content caching problem, we cluster users by global K-means (GKM), based on content popularity. For improving the effectiveness of caching, we propose a low complexity ε-greedy Q-learning based content caching algorithm which obtains a near-optimal solution. To characterize the performance of the proposed cooperative caching algorithms, sum MOS of users is used to define the reward function in Q- learning. The proposed Q-learning algorithm is capable of assisting the network to efficiently utilize the caching resource of the BSs. Simulation results reveal that: The proposed low complexity ε-greedy Q-learning based content caching algorithm achieves a near-optimal performance and is capable of outperforming GKM based caching.