Penalized linear regression with high-dimensional pairwise screening

Abstract

In variable selection, most existing screening methods focus on marginal effects and ignore dependence between covariates.To improve the performance of selection, we incorporate pairwise effects in covariates for screening and penalization.We achieve this by studying the asymptotic distribution of the maximal absolute pairwise sample correlation among independent covariates.The novelty of the theory is in that the convergence is with respect to the dimensionality p, and is uniform with respect to the sample size n.Moreover, we obtain an upper bound for the maximal pairwise R squared when regressing the response onto two different covariates.Based on these extreme value results, we propose a screening procedure to detect covariates pairs that are potentially correlated and associated with the response.We further combine the pairwise screening with Sure Independence Screening [Fan and Lv, 2008] and develop a new regularized variable selection procedure.Numerical studies show that our method is very competitive in terms of both prediction accuracy and variable selection accuracy.