Unified Framework to Regularized Covariance Estimation in Scaled Gaussian Models

Abstract

We consider regularized covariance estimation in scaled Gaussian settings, e.g., elliptical distributions, compound-Gaussian processes and spherically invariant random vectors. Asymptotically in the number of samples, the classical maximum likelihood (ML) estimate is optimal under different criteria and can be efficiently computed even though the optimization is nonconvex. We propose a unified framework for regularizing this estimate in order to improve its finite sample performance. Our approach is based on the discovery of hidden convexity within the ML objective. We begin by restricting the attention to diagonal covariance matrices. Using a simple change of variables, we transform the problem into a convex optimization that can be efficiently solved. We then extend this idea to nondiagonal matrices using convexity on the manifold of positive definite matrices. We regularize the problem using appropriately convex penalties. These allow for shrinkage towards the identity matrix, shrinkage towards a diagonal matrix, shrinkage towards a given positive definite matrix, and regularization of the condition number. We demonstrate the advantages of these estimators using numerical simulations.