MINIMAX RATES FOR CONDITIONAL DENSITY ESTIMATION VIA EMPIRICAL ENTROPY

• 成果类型: Article
• 署名作者: Bilodeau, Blair; Foster, Dylan J.; Roy, Daniel M.
• 署名单位: University of Toronto; Microsoft
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/23-AOS2270
• 发表日期: 2023
• 页码: 762-790
• 关键词: maximum-likelihood estimator; Lower bounds; log-concave; nonparametric-estimation; global rates; CONVERGENCE; MODEL; INFORMATION; regression; CLASSIFICATION

摘要

We consider the task of estimating a conditional density using i.i.d. sam-ples from a joint distribution, which is a fundamental problem with applica-tions in both classification and uncertainty quantification for regression. For joint density estimation, minimax rates have been characterized for general density classes in terms of uniform (metric) entropy, a well-studied notion of statistical capacity. When applying these results to conditional density es-timation, the use of uniform entropy-which is infinite when the covariate space is unbounded and suffers from the curse of dimensionality-can lead to suboptimal rates. Consequently, minimax rates for conditional density es-timation cannot be characterized using these classical results.We resolve this problem for well-specified models, obtaining match-ing (within logarithmic factors) upper and lower bounds on the minimax Kullback-Leibler risk in terms of the empirical Hellinger entropy for the conditional density class. The use of empirical entropy allows us to appeal to concentration arguments based on local Rademacher complexity, which- in contrast to uniform entropy-leads to matching rates for large, potentially nonparametric classes and captures the correct dependence on the complexity of the covariate space. Our results require only that the conditional densities are bounded above, and do not require that they are bounded below or other-wise satisfy any tail conditions.