Decentralized Multi-Agent Multi-Armed Bandit Learning With Calibration for Multi-Cell Caching

Abstract

This paper investigates online decentralized cache strategy design in multi-cell networks without the knowledge of user preference. The goal is to minimize the cumulative transmission delay of multi-cell networks within a finite time interval. Each small base station (SBS) aims to decide on its own cache action autonomously based on its past observations and limited information transmitted from other SBSs without a central controller. To coordinate the cache actions of different SBSs in a decentralized manner, we first propose an ϵ-calibration learning algorithm for each SBS to predict the cache strategy of other SBSs in real-time, which can progressively improve the accuracy in cache action prediction. Then a decentralized multi-agent multi-armed bandit (MAMAB) algorithm is developed for each SBS to decide its own cache strategy based jointly on its past observations and estimated upcoming cache action of other SBSs. This decentralized MAMAB algorithm with ϵ-calibration enables multiple SBSs to converge to a reasonable joint cache action and realize a cooperative cache decision making in a decentralized manner with limited information exchange. Simulation results demonstrate that our proposed decentralized caching algorithm outperforms other decentralized caching algorithms and can rapidly approach towards the centralized caching solutions.