Adaptive Multi-layer Attention Double Dueling Deep Q-Network for Muti-agent Reinforcement Learning

Abstract

Abstract. In multi-agent fields, traditional muti-agent DQN methods often suffer from overestimation bias and overestimation of unimportant actions, especially when state-action Q-value differences are slight. To deal with such issue, we present an adaptive Multi-layer Attention Double Dueling Deep Q-Network (MAD-D3QN) model, aiming to improve decision-making accuracy in complex multi-agent environments. The proposed model utilizes two attention layers that dynamically calculate state value and action advantage weights, facilitating more precise Q-value estimation and reducing the common overestimation bias. Related experiments carried out in StarWar II scenarios show that the MAD-D3QN model obviously outperforms traditional methods (IQL,DQN), achieving higher decision efficiency and robustness. Our findings demonstrates that the MAD-D3QN framework not only promotes the state-of-the-art in multi-agent reinforcement learning but also provides potential applications in real-world cooperative tasks. Future research will delve into the integration of advanced multi-agent communication structures to further enhance model adaptability.