Reinforcement Learning with Temporal Logic Constraints

Abstract

Reinforcement learning (RL) is an agent based AI learning method, where learning and optimization are combined. Dynamic programming is then performed iteratively, based on reward and next state observations from the system to be controlled. A brief survey of RL is given, followed by an evaluation of a recently proposed method to include temporal logic safety and liveness guarantees in RL, here combined with classical performance optimization. RL is based on Markov decision processes (MDPs), and to reduce the number of observations from the system, a modular MDP framework is proposed. In the learning process, it is then assumed that some parts of the system are represented by known MDP models, while other parts can be estimated by observations from the real system. Local information from the modular system may then be used to reduce the computational complexity, especially in the handling of safety properties.