Deep Reinforcement Learning based Aggressive Collision Avoidance with Limited FOV for Unmanned Aerial Vehicles

Abstract

As the novel lightweight sensors become more capable of providing high-level perception abilities to the small unmanned aerial vehicles for a cluttered environment, they require more capable navigation and control methodologies to push the operational limits. Such methods are bounded due to computational power requirements, limits of conservative deterministic methods, or the algorithms utilizing heuristics built upon specialized cases. Considering this discussion, in this work, we develop a trajectory re-planning algorithm for collision avoidance, combining machine learning and deterministic trajectory generation methodologies allowing the small UAVs to navigate in clutter environments aggressively. First, we utilize the deferentially flat model description of the UAVs and define the output flight trajectories through high-order B-splines. The local support property of the B-spline allows us to generate these evasive maneuvers without regenerating the whole B-spline trajectory, thus granting aggressiveness. Then, assuming the small UAVs have a limited field of view, to provide instantaneous trajectory segment re-planning, we trained deep reinforcement learning agent for optimal control point reallocation through knot insertion to avoid the sensed obstacles, which also considers the feasibility of the newly generated trajectory segment. The deep reinforcement learning agent can generate an optimal solution in 1.2 ms, which offers the fastest solution in the literature and allows it to be utilized on highly agile vehicles. Finally, we focus on a real-time platform implementations of the algorithm in order to show the performance and to build and perform aggressive collision avoidance maneuvers in highly cluttered environments can be seen in the following video link: https://youtu.be/8IiLQFQ3V0E.