Reinforcement Learning for Repeated Power Control Game in Cognitive Radio Networks

Abstract

Cognitive radio (CR) users are expected to be uncoordinated users that opportunistically seek the spectrum resource from primary users (PUs) in a competitive way. In most existing works, however, CR users are required to share the interference channel information and power strategies to conduct the game with pricing mechanisms that incur the frequent exchange of information. The requirement of significant communication overheads among CR users impedes fully distributed solutions for the deployment of CR networks, which is a challenging problem in the research communities. In this paper, a robust distributed power control algorithm is designed with low implementation complexity for CR networks through reinforcement learning, which does not require the interference channel and power strategy information among CR users (and from CR users to PUs). To the best of our knowledge, this research provides the solution for the first time for the incomplete-information power control game in CR networks. During the repeated game, CR users can control their power strategies by observing the interference from the feedback signals of PUs and transmission rates obtained in the previous step. This procedure allows achieving high spectrum efficiency while conforming to the interference constraint of PUs. This constrained repeated stochastic game with learning automaton is proved to be asymptotically equivalence to the traditional game with complete information. The properties of existence, diagonal concavity and uniqueness for the game are studied. A Bush-Mosteller reinforcement learning procedure is designed for the power control algorithm, and the properties of convergence and learning rate of the algorithm are analyzed. The performance of the learning-based power control algorithm is thoroughly investigated with simulation results, which demonstrates the effectiveness of the proposed algorithm in solving variety of practical CR network problems for real-world applications.