THE TAP FREE ENERGY FOR HIGH-DIMENSIONAL LINEAR REGRESSION

• 成果类型: Article
• 署名作者: Qiu, Jiaze; Sen, Subhabrata
• 署名单位: Harvard University
• 刊物名称: ANNALS OF APPLIED PROBABILITY
• ISSN/ISSBN: 1050-5164
• DOI: 10.1214/22-AAP1874
• 发表日期: 2023
• 页码: 2643-2680
• 关键词: spin; EQUATIONS

摘要

We derive a variational representation for the log-normalizing constant of the posterior distribution in Bayesian linear regression with a uniform spherical prior and an i.i.d. Gaussian design. We work under the propor-tional asymptotic regime, where the number of observations and the number of features grow at a proportional rate. Our representation holds when the variance of the additive noise is sufficiently large, which corresponds to a high-temperature condition in statistical physics. This rigorously establishes the Thouless-Anderson-Palmer (TAP) approximation arising from spin glass theory, and proves a conjecture of (In 2014 IEEE International Symposium on Information Theory (2014) 1499-1503 IEEE) in the special case of the spherical prior (at sufficiently high temperature).