Variance estimation in high-dimensional linear regression via adaptive elastic-net

Abstract

Variance estimation in high-dimensional linear regression is a fundamental problem in statistical learning, and it plays a wide range of roles in signal processing, pattern recognition, and other fields. Because it is difficult to choose the true model precisely in high-dimensional regression, variance estimation remains a challenging problem, especially in scenarios where the true regression parameter has a large number of non-zero elements. In this paper, we develop a novel approach for variance estimation by solving a re-parameterized log-likelihood optimization problem with adaptive elastic-net regularization. It is called the natural adaptive elastic-net (NAEN). The relationship between NAEN and the naive adaptive elastic-net is established. The NAEN inherits the advantages of the naive adaptive elastic-net, that is, it can select and estimate the regression and variance parameters simultaneously. Moreover, we also give the asymptotic properties of NAEN for error variance. The simulation results show that the proposed NAEN is suitable for scenarios where the true regression parameter has many non-zero elements.