MAPPED LEAST SQUARES SUPPORT VECTOR MACHINE REGRESSION

Abstract

This paper describes a novel version of regression SVM (Support Vector Machines) that is based on the least-squares error. We show that the solution of this optimization problem can be obtained easily once the inverse of a certain matrix is computed. This matrix, however, depends only on the input vectors, but not on the labels. Thus, if many learning problems with the same set of input vectors but different sets of labels have to be solved, it makes sense to compute the inverse of the matrix just once and then use it for computing all subsequent models. The computational complexity to train an regression SVM can be reduced to O (N 2 ), just a matrix multiplication operation, and thus probably faster than known SVM training algorithms that have O (N 2 ) work with loops. We describe applications from image processing, where the input points are usually of the form {(x 0 + dx, y 0 + dy) : |dx| < m, |dy| < n} and all such set of points can be translated to the same set {(dx, dy) : |dx| < m, |dy| < n} by subtracting (x 0 , y 0 ) from all the vectors. The experimental results demonstrate that the proposed approach is faster than those processing each learning problem separately.