Deep Reinforcement Learning Based TCP Congestion Control in UAV Assisted Wireless Networks

Abstract

In this paper, the problem of congestion control is studied for transmission control protocol (TCP) in an unmanned aerial vehicles (UAVs) assisted wireless network (UAWN). In the studied model, transmitters transmit data packets to receivers over transmission links that must pass through two UAV base stations. The transmission link between the two UAVs, called bottleneck link, is highly prone to congestion due to the significant traffic. To control the system congestion, the number of packets passing through the bottleneck link during each time interval must be controlled. The problem is formulated as an optimization problem whose goal is to improve the average throughput and reduce the average latency by controlling the maximum number of packets passing through the bottleneck link. To solve this problem, an advantage actor critic (A2C) framework based reinforcement learning algorithm is proposed. The proposed algorithm effectively solves the congestion control problem and enables the trained model to quickly adapt to dynamic network environments. Simulation results demonstrate that, compared to the Q-learning based congestion control algorithm, the proposed approach yields an 18.9% improvement in the average throughput and a 2.8% reduction in average latency.