Trace-class Gaussian priors for Bayesian learning of neural networks with MCMC

• 成果类型: Article
• 署名作者: Sell, Torben; Singh, Sumeetpal Sidhu
• 署名单位: University of Edinburgh; University of Cambridge
• 刊物名称: JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY
• ISSN/ISSBN: 1369-7412
• DOI: 10.1093/jrsssb/qkac005
• 发表日期: 2023
• 页码: 46-66
• 关键词: INVERSE PROBLEMS; contraction; algorithms; rates

摘要

This paper introduces a new neural network based prior for real valued functions. Each weight and bias of the neural network has an independent Gaussian prior, with the key novelty that the variances decrease in the width of the network in such a way that the resulting function is well defined in the limit of an infinite width network. We show that the induced posterior over functions is amenable to Monte Carlo sampling using Hilbert space Markov chain Monte Carlo (MCMC) methods. This type of MCMC is stable under mesh refinement, i.e. the acceptance probability does not degenerate as more parameters of the function's prior are introduced, even ad infinitum. We demonstrate these advantages over other function space priors, for example in Bayesian Reinforcement Learning.