THE DISTRIBUTION OF THE LASSO: UNIFORM CONTROL OVER SPARSE BALLS AND ADAPTIVE PARAMETER TUNING

• 成果类型: Article
• 署名作者: Miolane, Leo; Montanari, Andrea
• 署名单位: New York University; New York University; Stanford University; Stanford University
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/20-AOS2038
• 发表日期: 2021
• 页码: 2313-2335
• 关键词: message-passing algorithms; confidence-intervals; regression; neighborliness; RISK

摘要

The Lasso is a popular regression method for high-dimensional problems in which the number of parameters theta(1), ..., theta(N), is larger than the number n of samples: N > n. A useful heuristics relates the statistical properties of the Lasso estimator to that of a simple soft-thresholding denoiser, in a denoising problem in which the parameters (theta(i))(i <= N) are observed in Gaussian noise, with a carefully tuned variance. Earlier work confirmed this picture in the limit n, N -> infinity, pointwise in the parameters theta and in the value of the regularization parameter. Here, we consider a standard random design model and prove exponential concentration of its empirical distribution around the prediction provided by the Gaussian denoising model. Crucially, our results are uniform with respect to. belonging to l(q) balls, q is an element of [0, 1], and with respect to the regularization parameter. This allows us to derive sharp results for the performances of various data-driven procedures to tune the regularization. Our proofs make use of Gaussian comparison inequalities, and in particular of a version of Gordon's minimax theorem developed by Thrampoulidis, Oymak and Hassibi, which controls the optimum value of the Lasso optimization problem. Crucially, we prove a stability property of the minimizer in Wasserstein distance that allows one to characterize properties of the minimizer itself.