Some sharp performance bounds for least squares regression with L1 regularization

• 成果类型: Article
• 署名作者: Zhang, Tong
• 署名单位: Rutgers University System; Rutgers University New Brunswick
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/08-AOS659
• 发表日期: 2009
• 页码: 2109-2144
• 关键词: statistical estimation; DANTZIG SELECTOR; sparsity; larger; Lasso

摘要

We derive sharp performance bounds for least squares regression With L-1 regularization front parameter estimation accuracy and feature selection quality perspectives. The main result proved for L-1 regularization extends it similar result in [Ann. Statist. 35 (2007) 2313-2351] for the Dantzig selector. It gives an affirmative answer to an open question in [Ann. Statist. 35 (2007) 2358-2364]. Moreover, the result leads to an extended view of feature selection that allows less restrictive conditions than some recent work. Based on the theoretical insights, a novel two-stage L-1-regularization procedure with selective penalization is analyzed. It is shown that if the target parameter vector can be decomposed as the sum of a sparse parameter vector with large coefficients and another less sparse vector with relatively small coefficients, then the two-stage procedure can lead to improved performance.