RHO-ESTIMATORS REVISITED: GENERAL THEORY AND APPLICATIONS

• 成果类型: Article
• 署名作者: Baraud, Yannick; Birge, Lucien
• 署名单位: Universite Cote d'Azur; Sorbonne Universite; Centre National de la Recherche Scientifique (CNRS)
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/17-AOS1675
• 发表日期: 2018
• 页码: 3767-3804
• 关键词: model selection; maximum-likelihood; inequalities; CONVERGENCE; density

摘要

Following Baraud, Birge and Sart [Invent. Math. 207 (2017) 425-517], we pursue our attempt to design a robust universal estimator of the joint distribution of n independent (but not necessarily i.i.d.) observations for an Hellinger-type loss. Given such observations with an unknown joint distribution P and a dominated model Q for P, we build an estimator P based on Q (a rho-estimator) and measure its risk by an Hellinger-type distance. When P does belong to the model, this risk is bounded by some quantity which relies on the local complexity of the model in a vicinity of P. In most situations, this bound corresponds to the minimax risk over the model (up to a possible logarithmic factor). When P does not belong to the model, its risk involves an additional bias term proportional to the distance between P and Q, whatever the true distribution P. From this point of view, this new version of rho-estimators improves upon the previous one described in Baraud, Birge and Sart [Invent. Math. 207 (2017) 425-517] which required that P be absolutely continuous with respect to some known reference measure. Further additional improvements have been brought as compared to the former construction. In particular, it provides a very general treatment of the regression framework with random design as well as a computationally tractable procedure for aggregating estimators. We also give some conditions for the maximum likelihood estimator to be a p-estimator. Finally, we consider the situation where the statistician has at her or his disposal many different models and we build a penalized version of the rho-estimator for model selection and adaptation purposes. In the regression setting, this penalized estimator not only allows one to estimate the regression function but also the distribution of the errors.