BAYESIAN FRACTIONAL POSTERIORS

• 成果类型: Article
• 署名作者: Bhattacharya, Anirban; Pati, Debdeep; Yang, Yun
• 署名单位: Texas A&M University System; Texas A&M University College Station; University of Illinois System; University of Illinois Urbana-Champaign
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/18-AOS1712
• 发表日期: 2019
• 页码: 39-66
• 关键词: CONVERGENCE-RATES; Oracle Inequalities; DENSITY-ESTIMATION; convex regression; least-squares; RISK; bounds; distributions; complexities; aggregation

摘要

We consider the fractional posterior distribution that is obtained by updating a prior distribution via Bayes theorem with a fractional likelihood function, a usual likelihood function raised to a fractional power. First, we analyze the contraction property of the fractional posterior in a general misspecified framework. Our contraction results only require a prior mass condition on certain Kullback-Leibler (KL) neighborhood of the true parameter (or the KL divergence minimizer in the misspecified case), and obviate constructions of test functions and sieves commonly used in the literature for analyzing the contraction property of a regular posterior. We show through a counterexample that some condition controlling the complexity of the parameter space is necessary for the regular posterior to contract, rendering additional flexibility on the choice of the prior for the fractional posterior. Second, we derive a novel Bayesian oracle inequality based on a PAC-Bayes inequality in misspecified models. Our derivation reveals several advantages of averaging based Bayesian procedures over optimization based frequentist procedures. As an application of the Bayesian oracle inequality, we derive a sharp oracle inequality in multivariate convex regression problems. We also illustrate the theory in Gaussian process regression and density estimation problems.