Performance Analysis of Multi-armed Bandit Algorithm with Negative Autocorrelation

Abstract

We analyze the effectiveness of a multi-armed bandit algorithm, which utilizes ideal spatiotemporal chaotic dynamics generated by an FIR filter. In the previous research on additive chaotic noise to heuristic searches for combinatorial optimization problems, it has been shown that the chaotic sequences with negative autocorrelation improve the performance of asynchronously updated algorithms. The effectiveness of chaos can be understood in terms of the conventional theory of the chaotic CDMA, which showed that the cross-correlation between the sequences with negative autocorrelation becomes lowest. The spatiotemporal chaotic searching dynamics with such lowest cross-correlation has been shown to be effective in improving asynchronously updated algorithms. In this paper, as an asynchronously updated multiarmed bandit algorithm, we apply the FIR filter to the softmax algorithm, and analyze the spatiotemporal dynamics of this method. Our numerical simulation results show that the cross-correlation of this method can be minimized and its performance can be improved by using negative autocorrelation.