Motor Learning Model through Reinforcement Learning with Neural Internal Model

Abstract

The present paper proposes a learning control method for the musculoskeletal system of arm based on the reinforcement. An optimization for the hand trajectory and muscle's force distribution is needed to acquire the reaching motion. The proposed architecture can acquire an optimized motion through learning the task. However, the biological control system composed of muscluloskeletal system is not able to sense the state without time delay. The time delay causes instability of learning. The proposed scheme consists of reinforcement part and neural internal model. Neural internal model is employed to compensate for the time delay. Then, there must be a modeling error if the muscle noise is assumed. Thus we introduce the minimum modeling error criterion for reinforcement learning. The minimum modeling error criterion gives not only reduction of total muscle level but also smoothness of the hand trajectory. The effectiveness and the biological plausibility of the present model is demonstrated by several simulations.