Optimization of Neural Networks with Multi-Objective LASSO Algorithm

Abstract

This paper presents a bi-objective algorithm that optimizes the error and the sum of the absolute weights of a Multi-Layer Perceptron neural network. The algorithm is based on the linear Least Absolute Shrinkage and Selection Operator (LASSO) and provides simultaneous generalization and weight selection optimization. The algorithm searches for a set of optimal solutions called Pareto set from which a single weight vector with best performance and reduced number of weights is selected based on a validation criterion. The method is applied to classification and regression real problems and compared with the norm based multi-objective algorithm. Results show that the neural networks obtained have improved generalization performance and reduced topology.