Locally Linear Embedding algorithm based on OMP for incremental learning

Abstract

Locally Linear Embedding (LLE) is a sort of powerful nonlinear dimensionality reduction algorithms. The basic idea behind the LLE method is that each data point and its neighbors lie on or close to a locally linear patch of the manifold if there is sufficient data. Then the local geometry of these patches is described by using linear coefficients which can reconstruct each data point from its neighbors. However, LLE operates in a batch way and its dimension reduction cannot be generalized to unseen samples. If a test sample arrives, LLE must run repeatedly and the former computational results are discarded. Thus, some incremental methods have been proposed for LLE to solve this problem. In these incremental methods, the neighbor number is globally fixed, which may result in selecting points from another linear space as neighbors. This paper presents LLE based on orthogonal matching pursuit (OMP) and applies it to classification tasks. In the classification tasks, dimensionality reduction on test samples is implemented by applying dimension reduction on training samples. The new LLE method could select a more appropriate neighbors from the selected neighbors. OMP is applied to not only LLE for training samples, but also the incremental learning of LLE for test samples. Compared with other linear incremental methods, experimental results show that the proposed method is promising.