Abstract: We consider a Bayesian setup in which we have a parametric family {pi_h, h in H} of potential prior densities on the parameter theta and we need to select a member of that family, i.e. select a value of h. Let m(h) be the marginal likelihood of the data (this is the likelihood of the data with theta integrated out with respect to the prior pi_h). The empirical Bayes estimate of h is, by definition, the value of h that maximizes m(h). Unfortunately, except for ahandful of textbook examples, analytic evaluation of argmax_h m(h) is not feasible. All existing procedures for estimating it don't scale well with either the dimension of h or the dimension of theta or both, or are potentially highly inaccurate. We present a method, based on Markov chain Monte Carlo, for estimating argmax_h m(h),which applies very generally and scales well with dimension. We provide theorems, based on empirical process theory, which enable us to obtain confidence sets for argmax_h m(h), and to determine the Markov chain length needed to estimate argmax_h m(h) to a preset level of accuracy. Let g be a real-valued function of theta, and let I(h) be the posterior expectation of g(theta) when the prior is nu_h. As a byproduct of our approach, we show how to obtain point estimates and globally-valid confidence bands for the entire family {I(h), h in H}, based on a single Markov chain run.
As an application, we consider a Bayesian model for variable selection in nonparametric regression, in which the unknown parameter includes a component that specifies the variables that go into the regression. The hyperparameters governing the prior have a big effect on variable selection, and we show how our methodology can be used for making inference about these hyperparameters. We also give an illustration on a real data set.
This is joint work with Antonio Linero, University of Texas at
Austin.
Zoom link: https://illinois.zoom.us/j/99868029055?pwd=WjRLWWFFSzIrc05vam1BSEt2QnJwZz09
Meeting ID: 998 6802 9055
Password: 526232
