A Bayesian information criterion for singular models

• 成果类型: Article
• 署名作者: Drton, Mathias; Plummer, Martyn
• 署名单位: University of Washington; University of Washington Seattle; World Health Organization; International Agency for Research on Cancer (IARC)
• 刊物名称: JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY
• ISSN/ISSBN: 1369-7412
• DOI: 10.1111/rssb.12187
• 发表日期: 2017
• 页码: 323-380
• 关键词: sequential monte-carlo; likelihood ratio test; stochastic complexity; AUTOREGRESSIVE MODEL; generalization error; marginal likelihood; DENSITY-ESTIMATION; learning coefficient; exponential-families; asymptotic-behavior

摘要

We consider approximate Bayesian model choice for model selection problems that involve models whose Fisher information matrices may fail to be invertible along other competing submodels. Such singular models do not obey the regularity conditions underlying the derivation of Schwarz's Bayesian information criterion BIC and the penalty structure in BIC generally does not reflect the frequentist large sample behaviour of the marginal likelihood. Although large sample theory for the marginal likelihood of singular models has been developed recently, the resulting approximations depend on the true parameter value and lead to a paradox of circular reasoning. Guided by examples such as determining the number of components in mixture models, the number of factors in latent factor models or the rank in reduced rank regression, we propose a resolution to this paradox and give a practical extension of BIC for singular model selection problems.