A CLASS OF RENYI INFORMATION ESTIMATORS FOR MULTIDIMENSIONAL DENSITIES

• 成果类型: Article
• 署名作者: Leonenko, Nikolai; Pronzat, Luc; Savani, Vippal
• 署名单位: Cardiff University; Centre National de la Recherche Scientifique (CNRS); Universite Cote d'Azur
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/07-AOS539
• 发表日期: 2008
• 页码: 2153-2182
• 关键词: limit-theorems; entropy; asymptotics; Consistency; functionals; distances

摘要

A class of estimators of the Renyi and Tsallis entropies of an unknown distribution f in R-m is presented. These estimators are based on the kth nearest-neighbor distances computed from a sample of N i.i.d. vectors with distribution f. We show that entropies of any order q, including Shannon's entropy, can be estimated consistently with minimal assumptions on f. Moreover, we show that it is straightforward to extend the nearest-neighbor method to estimate the statistical distance between two distributions using one i.i.d. sample from each.