ON THE APPROXIMATION ACCURACY OF GAUSSIAN VARIATIONAL

• 成果类型: Article
• 署名作者: Katsevich, Anya; Rigollet, Philippe
• 署名单位: Massachusetts Institute of Technology (MIT)
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/24-AOS2393
• 发表日期: 2024
• 页码: 1384-1409
• 关键词: Asymptotic Normality; inference; CONVERGENCE

摘要

The main computational challenge in Bayesian inference is to compute integrals against a high-dimensional posterior distribution. In the past decades, variational inference (VI) has emerged as a tractable approximation to these integrals, and a viable alternative to the more established paradigm of Markov chain Monte Carlo. However, little is known about the approximation accuracy of VI. In this work, we bound the TV error and the mean and covariance approximation error of Gaussian VI in terms of dimension and sample size. Our error analysis relies on a Hermite series expansion of the log posterior whose first terms are precisely cancelled out by the first order optimality conditions associated to the Gaussian VI optimization problem.