Efficient Orthogonal Non-negative Matrix Factorization over Stiefel Manifold

Abstract

Orthogonal Non-negative Matrix Factorization (ONMF) ap- proximates a data matrix X by the product of two lower- dimensional factor matrices: X ≈ UVT, with one of them orthogonal. ONMF has been widely applied for clustering, but it often suffers from high computational cost due to the orthogonality constraint. In this paper, we propose a method, called Nonlinear Riemannian Conjugate Gradient ONMF (NRCG-ONMF), which updates U and V alterna- tively and preserves the orthogonality of U while achiev- ing fast convergence speed. Specifically, in order to update U, we develop a Nonlinear Riemannian Conjugate Gradi- ent (NRCG) method on the Stiefel manifold using Barzilai- Borwein (BB) step size. For updating V, we use a closed- form solution under non-negativity constraint. Extensive experiments on both synthetic and real-world data sets show consistent superiority of our method over other approaches in terms of orthogonality preservation, convergence speed and clustering performance.