Motor learning model using reinforcement learning with neural internal model

Abstract

The present paper proposes a learning control method for the musculoskeletal system of arm based on reinforcement learning. An optimization for the hand trajectory and muscle's force distribution is needed to acquire the reaching motion. The proposed architecture can acquire an optimized motion through learning the task. However, the biological control system composed of musculoskeletal system is not able to sense the state without time delay. The time delay causes instability of learning. The proposed scheme consists of the reinforcement learning part and neural internal model. Neural internal model is employed to compensate for the time delay by estimating the state of musculoskeletal system. Then, there must be a modeling error if some noise is included. Thus we introduce the minimum modeling error criterion for reinforcement learning, which gives not only the reduction of total muscle level but also the smoothness of the hand trajectory. The effectiveness and the biological plausibility of the present model is demonstrated by several simulations.