EARLY STOPPING FOR L2-BOOSTING IN HIGH-DIMENSIONAL LINEAR MODELS

• 成果类型: Article
• 署名作者: Stankewitz, Bernhard
• 署名单位: Bocconi University
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/24-AOS2356
• 发表日期: 2024
• 页码: 491-518
• 关键词: statistical inverse problems; Discrepancy Principle; regression

摘要

Increasingly high-dimensional data sets require that estimation methods do not only satisfy statistical guarantees but also remain computationally feasible. In this context, we consider L2-boosting via orthogonal matching pursuit in a high-dimensional linear model and analyze a data-driven early stopping time tau of the algorithm, which is sequential in the sense that its computation is based on the first tau iterations only. This approach is much less costly than established model selection criteria, that require the computation of the full boosting path, which may even be computationally infeasible in truly high-dimensional applications. We prove that sequential early stopping preserves statistical optimality in this setting in terms of a fully general oracle inequality for the empirical risk and recently established optimal convergence rates for the population risk. Finally, an extensive simulation study shows that at a significantly reduced computational cost, the performance of early stopping methods is on par with other state of the art algorithms such as the cross-validated Lasso or model selection via a high-dimensional Akaike criterion based on the full boosting path.