Sparse covariance thresholding for high-dimensional variable selection

Abstract

In high-dimensions, many variable selection methods, such as the lasso, are often limited by excessive variability and rank deficiency of the sample covariance matrix.Covariance sparsity is a natural phenomenon in high-dimensional applications, such as microarray analysis, image processing, etc., in which a large number of predictors are independent or weakly correlated.In this paper, we propose the covariance-thresholded lasso, a new class of regression methods that can utilize covariance sparsity to improve variable selection.We establish theoretical results, under the random design setting, that relate covariance sparsity to variable selection.Real-data and simulation examples indicate that our method can be useful in improving variable selection performances.