High-dimensional graphs and variable selection with the Lasso

Abstract

The pattern of zero entries in the inverse covariance matrix of a\nmultivariate normal distribution corresponds to conditional independence\nrestrictions between variables. Covariance selection aims at estimating those\nstructural zeros from data. We show that neighborhood selection with the Lasso\nis a computationally attractive alternative to standard covariance selection\nfor sparse high-dimensional graphs. Neighborhood selection estimates the\nconditional independence restrictions separately for each node in the graph and\nis hence equivalent to variable selection for Gaussian linear models. We show\nthat the proposed neighborhood selection scheme is consistent for sparse\nhigh-dimensional graphs. Consistency hinges on the choice of the penalty\nparameter. The oracle value for optimal prediction does not lead to a\nconsistent neighborhood estimate. Controlling instead the probability of\nfalsely joining some distinct connectivity components of the graph, consistent\nestimation for sparse graphs is achieved (with exponential rates), even when\nthe number of variables grows as the number of observations raised to an\narbitrary power.\n