CONVERGENCE RATES OF LEAST SQUARES REGRESSION ESTIMATORS WITH HEAVY-TAILED ERRORS

• 成果类型: Article
• 署名作者: Han, Qiyang; Wellner, Jon A.
• 署名单位: University of Washington; University of Washington Seattle
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/18-AOS1748
• 发表日期: 2019
• 页码: 2286-2319
• 关键词: CENTRAL-LIMIT-THEOREM; Minimax Rates; risk bounds; moment; inequalities; EIGENVALUE; selection; tests

摘要

We study the performance of the least squares estimator (LSE) in a general nonparametric regression model, when the errors are independent of the covariates but may only have a pth moment (p >= 1). In such a heavy-tailed regression setting, we show that if the model satisfies a standard entropy condition with exponent alpha is an element of (0, 2), then the L-2 loss of the LSE converges at a rate O-P(n(-1/2+alpha) boolean OR n(-1/2+1/2p)). Such a rate cannot be improved under the entropy condition alone. This rate quantifies both some positive and negative aspects of the LSE in a heavy-tailed regression setting. On the positive side, as long as the errors have p >= 1 + 2/alpha moments, the L-2 loss of the LSE converges at the same rate as if the errors are Gaussian. On the negative side, if p < 1 + 2/alpha, there are (many) hard models at any entropy level alpha for which the L-2 loss of the LSE converges at a strictly slower rate than other robust estimators. The validity of the above rate relies crucially on the independence of the covariates and the errors. In fact, the L-2 loss of the LSE can converge arbitrarily slowly when the independence fails. The key technical ingredient is a new multiplier inequality that gives sharp bounds for the multiplier empirical process associated with the LSE. We further give an application to the sparse linear regression model with heavy-tailed covariates and errors to demonstrate the scope of this new inequality.