Exponentiated backpropagation algorithm for multilayer feedforward neural networks

Abstract

The gradient descent backpropagation learning algorithm is based on minimizing the mean square error. An alternate approach to gradient descent is the exponentiated gradient descent algorithm which minimizes the relative entropy. Exponentiated gradient descent applied to backpropagation is proposed for a multilayer feedforward neural network. The learning rules for changing weights in the output layer as well the hidden layer neurons in the network are developed. Simulations were performed to explore the convergence and learning of the backpropagation algorithm with exponentiated gradient descent. Accuracy obtained with exponentiated gradient descent back propagation was comparable to the gradient descent back propagation while convergence was faster. The results show that exponentiated gradient descent can be extended to a multilayer feedforward neural network and used in pattern classification applications.