Skinny Gibbs: A Consistent and Scalable Gibbs Sampler for Model Selection

• 成果类型: Article
• 署名作者: Narisetty, Naveen N.; Shen, Juan; He, Xuming
• 署名单位: University of Illinois System; University of Illinois Urbana-Champaign; Fudan University; University of Michigan System; University of Michigan
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459
• DOI: 10.1080/01621459.2018.1482754
• 发表日期: 2019
• 页码: 1205-1217
• 关键词: bayesian variable selection; nonconcave penalized likelihood; generalized linear-models; regression; inference; binary; priors

摘要

We consider the computational and statistical issues for high-dimensional Bayesian model selection under the Gaussian spike and slab priors. To avoid large matrix computations needed in a standard Gibbs sampler, we propose a novel Gibbs sampler called Skinny Gibbs which is much more scalable to high-dimensional problems, both in memory and in computational efficiency. In particular, its computational complexity grows only linearly in p, the number of predictors, while retaining the property of strong model selection consistency even when p is much greater than the sample size n. The present article focuses on logistic regression due to its broad applicability as a representative member of the generalized linear models. We compare our proposed method with several leading variable selection methods through a simulation study to show that Skinny Gibbs has a strong performance as indicated by our theoretical work. Supplementary materials for this article are available online.