The Improved Training Algorithm of Back Propagation Neural Network with Self-adaptive Learning Rate

Abstract

This paper addresses the questions of improving convergence performance for back propagation (BP) neural network. For traditional BP neural network algorithm, the learning rate selection is depended on experience and trial. In this paper, based on Taylor formula the function relationship between the total quadratic training error change and connection weights and biases changes is obtained, and combined with weights and biases changes in batch BP learning algorithm, the formula for self-adaptive learning rate is given. Unlike existing algorithm, the self-adaptive learning rate depends on only neural network topology, training samples, average quadratic error and error curve surface gradient but not artificial selection. Simulation results show iteration times is significant less than that of traditional batch BP learning algorithm with constant learning rate.