Benign overfitting and adaptive nonparametric regression

• 成果类型: Article
• 署名作者: Chhor, Julien; Sigalla, Suzanne; Tsybakov, Alexandre B.
• 署名单位: Universite de Toulouse; Universite Toulouse 1 Capitole; Toulouse School of Economics; Institut Polytechnique de Paris; Ecole Polytechnique; ENSAE Paris
• 刊物名称: PROBABILITY THEORY AND RELATED FIELDS
• ISSN/ISSBN: 0178-8051
• DOI: 10.1007/s00440-024-01278-0
• 发表日期: 2024
• 页码: 949-980
• 关键词: fractional brownian-motion; rough path; differential-equations; signature; driven

摘要

We study benign overfitting in the setting of nonparametric regression under mean squared risk, and on the scale of H & ouml;lder classes. We construct a local polynomial estimator of the regression function that is minimax optimal on a H & ouml;lder class with any given smoothness, and that is a continuous function interpolating the set of observations with high probability. The key element of the construction is the use of singular kernels. Moreover, we prove that adaptation to unknown smoothness is compatible with benign overfitting. Namely, we construct a continuous and interpolating local polynomial estimator attaining the minimax optimal rate in L2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L_2$$\end{document} adaptively to the unknown H & ouml;lder smoothness. Our results highlight the fact that interpolation can be fundamentally decoupled from bias-variance tradeoff in the problem of nonparametric regression.