Principled High-Dimensional Nonparametric Inference: Reconciling Bayesian Flexibility with Frequentist Guarantees

Abstract

This paper addresses the fundamental challenge of statistical inference in high-dimensional nonparametric settings by proposing a principled framework that reconciles the inherent flexibility of Bayesian approaches with the rigorous guarantees offered by Frequentist methods. High-dimensional data, characterized by a number of covariates exceeding or comparable to the sample size, necessitates robust and adaptive inference techniques. While Bayesian nonparametrics offers a natural framework for uncertainty quantification and model complexity adaptation, its Frequentist operating characteristics, such as coverage properties of credible sets and optimal convergence rates, are not always immediately evident or guaranteed without careful prior specification. Conversely, traditional Frequentist methods, though providing strong theoretical guarantees, often require explicit regularization or model selection, which can be less adaptive or computationally intensive in complex nonparametric landscapes. We develop a methodology that leverages the power of sparsity-inducing Bayesian nonparametric priors, such as spike-and-slab Gaussian processes or sparse Dirichlet process mixtures, and demonstrates how judicious prior construction can lead to posterior distributions with provable Frequentist properties. Specifically, we establish that our proposed framework achieves optimal or near-optimal Frequentist contraction rates for posterior distributions and that Bayesian credible sets exhibit valid Frequentist coverage. This reconciliation offers a powerful paradigm for robust and adaptable high-dimensional inference, providing both a comprehensive account of uncertainty and strong theoretical performance assurances.