Adaptive state aggregation for reinforcement learning

Abstract

State partition is an important issue in reinforcement learning, because it has a significant effect on the performance. In this paper, an adaptive state partition method is presented for discretizing the state space adaptively and makes use of decision trees effectively. The proposed method splits the state space according to the temporal difference generated by the reinforcement learning. Consequently, the reinforcement learning uses the state space partitioned by the decision tree to learn the policy simultaneously. For avoiding a trivial partition, sibling nodes are pruned according to the Activity and the Reliability. A Monte-Carlo Tree Search (MCTS) is also proposed to explore the policy. A simulation for approaching goal has been conducted to demonstrate that the proposed method can achieve the design goal.