OPTIMAL CROSS-VALIDATION IN DENSITY ESTIMATION WITH THE L2-LOSS

• 成果类型: Article
• 署名作者: Celisse, Alain
• 署名单位: Centre National de la Recherche Scientifique (CNRS); CNRS - National Institute for Mathematical Sciences (INSMI); Universite de Lille
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/14-AOS1240
• 发表日期: 2014
• 页码: 1879-1910
• 关键词: model selection; CHOICE; inequalities; penalties

摘要

We analyze the performance of cross-validation (CV) in the density estimation framework with two purposes: (i) risk estimation and (ii) model selection. The main focus is given to the so-called leave-p-out CV procedure (Lpo), where p denotes the cardinality of the test set. Closed-form expressions are settled for the Lpo estimator of the risk of projection estimators. These expressions provide a great improvement upon V-fold cross-validation in terms of variability and computational complexity. From a theoretical point of view, closed-form expressions also enable to study the Lpo performance in terms of risk estimation. The optimality of leave-one-out (Loo), that is Lpo with p = 1, is proved among CV procedures used for risk estimation. Two model selection frameworks are also considered: estimation, as opposed to identification. For estimation with finite sample size n, optimality is achieved for p large enough [with p/n = o(1)] to balance the overfitting resulting from the structure of the model collection. For identification, model selection consistency is settled for Lpo as long as p I n is conveniently related to the rate of convergence of the best estimator in the collection: (i) p/n -> 1 as n -> +infinity with a parametric rate, and (ii) p/n = o(1) with some nonparametric estimators. These theoretical results are validated by simulation experiments.