Penalized Mallow’s model averaging

Abstract

This article proposes penalized Mallow’s model averaging (pMMA) in the linear regression framework given non nested candidate models. Compared to the MMA, additional constraints are imposed on model weights. We introduce a general framework and allow for non convex constraints such as SCAD, MCP, and TLP. We establish the asymptotic optimality of our proposed penalized MMA (pMMA) estimator and show that the pMMA can achieve a higher sparsity level than the classic MMA. A coordinate-wise descent algorithm has been developed to compute the pMMA estimator efficiently. We conduct simulation and empirical studies to show that our pMMA estimator produces a more sparse weight vector than the MMA, but with better out-of-sample performance.