General Bayesian loss function selection and the use of improper models

• 成果类型: Article
• 署名作者: Jewson, Jack; Rossell, David
• 署名单位: Pompeu Fabra University; Barcelona School of Economics
• 刊物名称: JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY
• ISSN/ISSBN: 1369-7412
• DOI: 10.1111/rssb.12553
• 发表日期: 2022
• 页码: 1640-1665
• 关键词: density; robust; computation; estimators; likelihood; inference; dimension; CHOICE; cancer

摘要

Statisticians often face the choice between using probability models or a paradigm defined by minimising a loss function. Both approaches are useful and, if the loss can be re-cast into a proper probability model, there are many tools to decide which model or loss is more appropriate for the observed data, in the sense of explaining the data's nature. However, when the loss leads to an improper model, there are no principled ways to guide this choice. We address this task by combining the Hyvarinen score, which naturally targets infinitesimal relative probabilities, and general Bayesian updating, which provides a unifying framework for inference on losses and models. Specifically we propose the Script capital H$$ \mathscr{H} $$-score, a general Bayesian selection criterion and prove that it consistently selects the (possibly improper) model closest to the data-generating truth in Fisher's divergence. We also prove that an associated Script capital H$$ \mathscr{H} $$-posterior consistently learns optimal hyper-parameters featuring in loss functions, including a challenging tempering parameter in generalised Bayesian inference. As salient examples, we consider robust regression and non-parametric density estimation where popular loss functions define improper models for the data and hence cannot be dealt with using standard model selection tools. These examples illustrate advantages in robustness-efficiency trade-offs and enable Bayesian inference for kernel density estimation, opening a new avenue for Bayesian non-parametrics.