NECESSARY AND SUFFICIENT CONDITIONS FOR VARIABLE SELECTION CONSISTENCY OF THE LASSO IN HIGH DIMENSIONS

• 成果类型: Article
• 署名作者: Lahiri, Soumendra N.
• 署名单位: Washington University (WUSTL)
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/20-AOS1979
• 发表日期: 2021
• 页码: 820-844
• 关键词: nonconcave penalized likelihood; p-regression parameters; asymptotic-behavior; model selection; M-ESTIMATORS; p2/n

摘要

This paper investigates conditions for variable selection consistency of the LASSO in high dimensional regression models and gives necessary and sufficient conditions for the same, potentially allowing the model dimension p to grow arbitrarily fast as a function of the sample size n. These conditions require both upper and lower bounds on the growth rate of the penalty parameter. It turns out that a variant of the irrepresentable Condition (IRC) of (J. Mach. Learn. Res. 7 (2006) 2541-2563), herein called the lower irrepresentable Condition (or LIRC), is determined by the lower bound considerations while the upper bound considerations lead to a new condition, called the upper irrepresentable Condition (or UIRC) in this paper. It is shown that the LIRC together with the UIRC is necessary and sufficient for the variable selection consistency of the LASSO, thereby settling a conjecture of (J. Mach. Learn. Res. 7 (2006) 2541-2563). Further, it is shown that under some mild regularity conditions, the penalty parameter must necessarily tend to infinity at a certain minimal rate to ensure variable selection consistency of the LASSO and that the corresponding LASSO estimators of the nonzero regression parameters cannot be root n-consistent (even for individual parameters). Thus, under fairly general conditions, the LASSO with a single choice of tvhe penalty parameter cannot achieve both variable selection consistency and root n-consistency simultaneously.