ON CROSS-VALIDATED LASSO IN HIGH DIMENSIONS

• 成果类型: Article
• 署名作者: Chetverikov, Denis; Liao, Zhipeng; Chernozhukov, Victor
• 署名单位: University of California System; University of California Los Angeles; Massachusetts Institute of Technology (MIT); Massachusetts Institute of Technology (MIT)
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/20-AOS2000
• 发表日期: 2021
• 页码: 1300-1317
• 关键词: model selection; freedom

摘要

In this paper, we derive nonasymptotic error bounds for the Lasso estimator when the penalty parameter for the estimator is chosen using K-fold cross-validation. Our bounds imply that the cross-validated Lasso estimator has nearly optimal rates of convergence in the prediction, L-2, and L-1 norms. For example, we show that in the model with the Gaussian noise and under fairly general assumptions on the candidate set of values of the penalty parameter, the estimation error of the cross-validated Lasso estimator converges to zero in the prediction norm with the root s logp/n x root log(pn) rate, where n is the sample size of available data, p is the number of covariates and s is the number of nonzero coefficients in the model. Thus, the cross-validated Lasso estimator achieves the fastest possible rate of convergence in the prediction norm up to a small logarithmic factor root log(pn), and similar conclusions apply for the convergence rate both in L-2 and in L-1 norms. Importantly, our results cover the case when p is (potentially much) larger than n and also allow for the case of non-Gaussian noise. Our paper therefore serves as a justification for the widely spread practice of using cross-validation as a method to choose the penalty parameter for the Lasso estimator.