Safe Spacecraft Inspection via Deep Reinforcement Learning and Discrete Control Barrier Functions

Abstract

While reinforcement learning (RL) offers high-performance solutions to complex tasks, its trial-and-error paradigm presents safety concerns when exploring unsafe states may be unacceptable. Safety filtering or run-time assurance (RTA) approaches can be used to monitor the RL agent’s desired control and prevent the agent from taking unsafe actions by satisfying safety constraints during and after training. This study investigates the problem of on-orbit inspection of a chief spacecraft using a deputy spacecraft controlled by an RL agent with safety constraints. Several constraints on the state space are enforced using discrete control barrier functions and an optimization-based RTA system. This assures the safety of the deputy spacecraft and its sensors even with a neural-network-based primary controller. A detailed analysis of RL agent performance for eight separate experiments is presented, including each safety constraint individually, all constraints simultaneously, and no constraints. Results demonstrate that the RL agent can complete the inspection task while adhering to safety constraints in a simulated RL environment. These results also show that the RTA system can aid the RL agent in learning a successful policy over fewer time steps but may lead to higher fuel usage, depending on the constraint(s) enforced.