Global dynamics in principal singular subspace networks

Abstract

A left (resp. right) principal singular subspace of dimension p is the subspace spanned by the p left (resp. right) singular vectors corresponding to the p largest singular values of the cross-correlation matrix of two stochastic processes. We study the global dynamics of a system of nonlinear ordinary differential equations (ODEs) that govern the unsupervised Hebbian learning of left and right principal singular subspaces from samples of the two stochastic processes. In particular, we show that these equations admit a simple Lyapunov function when they are restricted to a well defined smooth, compact manifold, and that they are related to a matrix Riccati differential equation. Moreover, we show that in the case p=1, the solutions of these ODEs can be given in closed form.