Spacecraft Proximity Maneuvering and Rendezvous With Collision Avoidance Based on Reinforcement Learning

Abstract

The rapid development of the aerospace industry puts forward the urgent need for the evolution of autonomous spacecraft rendezvous technology, which has gained significant attention recently due to increased applications in various space missions. This article studies the relative position tracking problem of the autonomous spacecraft rendezvous under the requirement of collision avoidance. An exploration-adaptive deep deterministic policy gradient (DDPG) algorithm is proposed to train a definite control strategy for this mission. Similar to the DDPG algorithm, four neural networks are used in this method, where two of them are used to generate the deterministic policy, whereas the other two are used to score the obtained policy. Differently, adaptive noise is introduced to reduce the possibility of oscillations and divergences and to cut down the unnecessary computation by weakening the exploration of stabilization problems. In addition, in order to effectively and quickly adapt to some other similar scenarios, a metalearning-based idea is introduced by fine-tuning the prior strategy. Finally, two numerical simulations show that the trained control strategy can effectively avoid the oscillation phenomenon caused by the artificial potential function. Benefiting from this, the trained control strategy based on deep reinforcement learning technology can decrease the energy consumption by 16.44% during the close proximity phase, compared with the traditional artificial potential function method. Besides, after introducing the metalearning-based idea, a strategy available for some other perturbed scenarios can be trained in a relatively short period of time, which illustrates its adaptability.