Nonparametric Binary Regression: A Bayesian Approach

Abstract

The performance of Bayes estimates are studied, under an assumption of conditional exchangeability. More exactly, for each subject in a data set, let $\\xi$ be a vector of binary covariates and let $\\eta$ be a binary response variable, with $P\\{\\eta = 1\\mid \\xi\\} = f(\\xi)$. Here, $f$ is an unknown function to be estimated from the data; the subjects are independent, and satisfy a natural "balance" condition. Define a prior distribution on $f$ as $\\sum_kw_k\\pi_k/\\sum_kw_k$, where $\\pi_k$ is uniform on the set of $f$ which only depend on the first $k$ covariates and $w_k > 0$ for infinitely many $k$. Bayes estimates are consistent at all $f$ if $w_k$ decreases rapidly as $k$ increase. Otherwise, the estimates are inconsistent at $f \\equiv 1/2$.