Entropy-Randomized Projection

Abstract

We propose a new randomized projection method based on entropy optimization of random projection matrices (the ERP method). The concept of compactness indicator of a data matrix, which is stored in projection matrices, is introduced. An ERP algorithm is formulated in the form of a conditional maximization problem for an entropy functional defined on the probability density functions of the projection matrices. A discrete version of this problem is considered, and conditions are obtained for the existence and uniqueness of its positive solution. Procedures are developed for the implementation of entropy-optimal projection matrices by sampling the probability density functions.