A LAVA ATTACK ON THE RECOVERY OF SUMS OF DENSE AND SPARSE SIGNALS

• 成果类型: Article
• 署名作者: Chernozhukov, Victor; Hansen, Christian; Liao, Yuan
• 署名单位: Massachusetts Institute of Technology (MIT); University of Chicago; University System of Maryland; University of Maryland College Park
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/16-AOS1434
• 发表日期: 2017
• 页码: 39-76
• 关键词: square-root lasso; variable selection; model selection; M-ESTIMATORS; freedom; smoothness

摘要

Common high-dimensional methods for prediction rely on having either a sparse signal model, a model in which most parameters are zero and there are a small number of nonzero parameters that are large in magnitude, or a dense signal model, a model with no large parameters and very many small nonzero parameters. We consider a generalization of these two basic models, termed here a sparse + dense model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide a deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein's unbiased estimator for lava's prediction risk. A simulation example compares the performance of lava to lasso, ridge and elastic net in a regression example using data-dependent penalty parameters and illustrates lava's improved performance relative to these benchmarks.