Pre-extracting boundary vectors for support vector machine using pseudo-density estimation method

Abstract

Support vector machines (SVMs) have become useful and universal learning machines. SVMs construct a decision function by support vectors (SVs) and their corresponding weights. The training phase of SVMs definitely uses all training samples, which leads to a large computational complexity for a large scale sample set. Moreover support vectors could not be found until a quadratic programming (QP) problem is solved. Actually we know only SVs play a role in the decision function. Hence, pseudo density estimation (PDE) is presented to extract a set of boundary vectors (BVs) which may contain SVs. The PDE method is a variant of Parzen window method. Hyperspheres are considered as the window functions. In our method, for each sample we construct a hypersphere with an unfixed radius. The ratio of the number of samples contained in the hypersphere of a sample to the total training samples can be taken as the pseudo density of the corresponding sample. The set of BVs is taken as the training input to SVMs. In doing so, it speeds the training procedure of SVMs. It is convenient for PDE to determine its parameter. The experiments show that SVMs using PDE have the similar generalization performance to SVMs.