Piecewise linear regularized solution paths

• 成果类型: Article
• 署名作者: Rosset, Saharon; Zhu, Ji
• 署名单位: International Business Machines (IBM); IBM USA; University of Michigan System; University of Michigan
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/009053606000001370
• 发表日期: 2007
• 页码: 1012-1030
• 关键词: nonconcave penalized likelihood; regression; shrinkage; selection

摘要

We consider the generic regularized optimization problem (beta) over cap(lambda) = arg min(beta) L (y, X beta) + lambda J (beta). Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407-499] have shown that for the LASSO-that is, if L is squared error loss and J(beta) = vertical bar vertical bar beta vertical bar vertical bar(1) is the if l(1) norm of beta-the optimal coefficient path is piecewise linear, that is, is piecewise constant. We derive a general characterization of the properties of (loss L, penalty J) pairs which give piecewise linear coefficient paths. Such pairs allow for efficient generation of the full regularized coefficient paths. We investigate the nature of efficient path following algorithms which arise. We use our results to suggest robust versions of the LASSO for regression and classification, and to develop new, efficient algorithms for existing problems in the literature, including Mammen and van de Geer's locally adaptive regression splines.