Mixtures, envelopes and hierarchical duality

• 成果类型: Article
• 署名作者: Polson, Nicholas G.; Scott, James G.
• 署名单位: University of Chicago; University of Texas System; University of Texas Austin
• 刊物名称: JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY
• ISSN/ISSBN: 1369-7412
• DOI: 10.1111/rssb.12130
• 发表日期: 2016
• 页码: 701-727
• 关键词: nonconcave penalized likelihood; variable selection; data augmentation; BAYES; estimators; models; distributions; algorithm; inference; sparsity

摘要

We develop a connection between mixture and envelope representations of objective functions that arise frequently in statistics. We refer to this connection by using the term hierarchical duality'. Our results suggest an interesting and previously underexploited relationship between marginalization and profiling, or equivalently between the Fenchel-Moreau theorem for convex functions and the Bernstein-Widder theorem for Laplace transforms. We give several different sets of conditions under which such a duality result obtains. We then extend existing work on envelope representations in several ways, including novel generalizations to variance-mean models and to multivariate Gaussian location models. This turns out to provide an elegant missing data interpretation of the proximal gradient method, which is a widely used algorithm in machine learning. We show several statistical applications in which the framework proposed leads to easily implemented algorithms, including a robust version of the fused lasso, non-linear quantile regression via trend filtering and the binomial fused double-Pareto model. Code for the examples is available on GitHub at .https://github.com/jgscott/hierduals.