Reinforcement Learning-Assisted Secure Reliable Underwater Wireless Acoustic Communications

Abstract

In recent days, there has been an increasing demand for the deployment of autonomous underwater vehicles (AUVs) for tactical wireless acoustic communications. This requires secure and reliable AUV communications to protect sensitive data. However, existing methods such as cryptography and channel coding introduce extra overheads and computational complexity. This is primarily due to the inherent challenges posed by acoustic communication systems, such as limited bandwidth and low energy efficiency. To overcome these challenges, we propose using intelligent reflecting surfaces (IRSs) in conjunction with reinforcement learning (RL) techniques, resulting in what is termed as RL-assisted Buoyed-IRS-AUV (RL-BIA) links. The RL-BIA links facilitate simultaneous secure and reliable communications by dynamically adjusting its beam width and IRS’s depth in response to seawater turbulence induced by wind and tide. We introduce a comprehensive link model that accounts for pointing errors, path loss, interference, and noise. Additionally, we developed an RL model adaptable to BIA links. To integrate channel secrecy and outage probability, a non-convex Max-Min optimization problem is formulated and solved iteratively using Q-learning and State-Action-Reward-State-Action (SARSA) algorithms. Numerical results demonstrate that at a wind speed of 8.5 meters per second, the proposed approach significantly enhances channel secrecy, with the RL-BIA link achieving a remarkable 400% improvement compared to the RL-assisted buoyed-AUV (RL-BA) link.