Bayesian Reinforcement Learning for Link-Level Throughput Maximization

Abstract

One intrinsic property of neural networks is making confident decisions because they do not capture uncertainty in training data. As a result, when Neural Networks (NN) are used in Deep Reinforcement Learning (DRL), agents cannot explore the action-space effectively. Bayesian Neural Networks (BNN) is one alternative that, instead of one value, assigns a probability distribution to the weights of NN. Using BNN as the policy network of an RL agent, the RL agent will have natural exploration capability. Recent studies demonstrate high potential for the application of RL methods in wireless networks. The inefficient exploration capability, however, limits their use cases. In this letter, we show how Bayesian RL agents can be used to solve complex wireless resource allocation problems. We consider the link-level throughput maximization that needs simultaneous power and Modulation/Coding Scheme (MCS) assignment to each user. We show that due to the large and sparse action-space, only Bayes-by-Backprop Q-network (BBQN) agents can find proper assignments. Simulation results show the performance of the proposed scheme in different network settings.