Reinforcement Learning for Multiaircraft Autonomous Air Combat in Multisensor UCAV Platform

Abstract

Autonomous air combat has received significant attention from researchers working on artificial intelligence (AI) applications. Most previous research on autonomous air combat has focused on one-on-one air combat scenarios in which air combat situational information is considered to be precisely observable. However, most modern air combats are conducted in formations, where air combat situational information is obtained from multiple sensors. Therefore, we introduce a novel automated maneuver decision architecture for close-range multiaircraft air combat scenarios under the multisensor unmanned combat aerial vehicle (UCAV) platform that can handle air combat scenarios with variable-sized formations. Then, a multiagent reinforcement learning (MARL) algorithm is proposed to obtain the strategy. The training performance of the training algorithm is evaluated, the obtained strategy is analyzed in different air combat scenarios, and it is found that these formations exhibit effective cooperative behavior in symmetric and asymmetric situations. Finally, we give ideas for the engineering implementation of a maneuver control architecture. This study provides a solution for future multiaircraft autonomous air combat.