Consistent Variable Selection in Linear Models

Abstract

Abstract A method of estimating linear model dimension and variable selection is proposed. This new criterion, which generalizes the Cp criterion, the Akaike information criterion (AIC), the Bayes information criterion, and the phiv criterion and is consistent under certain conditions, is based on a new class of penalty functions and a procedure of sorting covariates based on t-statistics. In the course of introducing this method, we discuss the important role of the penalty function in the consistency of model dimension estimation and in variable selection. The proposed method requires less computation than resampling-based methods that search over all subsets of covariates for the true model. Simulation results show that the new method is superior to the Cp criterion and AIC in finite-sample situations as well.