Competitive Takagi-Sugeno fuzzy reinforcement learning

Abstract

This paper proposes a competitive Takagi-Sugeno fuzzy reinforcement learning network (CTSFRLN) for solving complicated learning tasks of continuous domains. The proposed CTSFRLN is constructed by combining Takagi-Sugeno type fuzzy inference systems with action-value-based reinforcement learning methods. The architecture of CTSFRLN is described and a fitting exploration strategy, i.e., max-min Boltzmann exploration, is developed to implement local competitions in rule consequents. Three competitive learning algorithms are derived, including the competitive Takagi-Sugeno fuzzy Q-learning, competitive Takagi-Sugeno fuzzy R-learning, and competitive Takagi-Sugeno fuzzy advantage learning. These learning methods lead to the so called Takagi-Sugeno fuzzy variable structure controller. Experiments on the double inverted pendulum system demonstrate the performance and applicability of the proposed schemes. The superiority of these methods with respect to other related reinforcement learning ones is also illustrated. Finally, the conclusion remark is drawn.