Intelligent Dynamic Spectrum Anti-Jamming Communications: A Deep Reinforcement Learning Perspective

Abstract

Wireless devices are vulnerable to malicious jammers due to the openness of the spectrum environment. However, traditional anti-jamming approaches work on predetermined patterns, which are unable to resist dynamic jamming attacks. To capture the dynamics of jamming environment and utilize spectrum resources effectively, dynamic spectrum anti-jamming (DSAJ) ability is urgently required. Unfortunately, the unknown jamming environment and high-dimensional spectrum state present great challenges to anti-jamming communications. In this context, intelligent DSAJ is discussed based on deep reinforcement learning (DRL) due to its remarkable advantages in solving sequential decision making problem. First, an overview of DRL-based DSAJ is introduced, and then the challenges of applying DRL algorithms to DASJ are discussed, including non-Markov state, imperfect information, slow convergence, and local optimum. Second, a framework of DRL-based DSAJ is proposed, and two phases are introduced. For the first phase, the anti-jamming sequential decision making process is formulated as a Markov decision process (MDP), and three elements – state, action, and reward – are discussed in detail. The second phase is the design of DRL-based algorithms, aiming to update anti-jamming policy and find the optimal policy. Then two DRL-based anti-jamming cases are presented for better understanding. Finally, several future research directions are recommended.