ASYMPTOTIC PROPERTIES OF HIGH-DIMENSIONAL RANDOM FORESTS

• 成果类型: Article
• 署名作者: Chi, Chien-Ming; Vossler, Patrick; Fan, Yingying; Lv, Jinchi
• 署名单位: Academia Sinica - Taiwan; University of Southern California
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/22-AOS2234
• 发表日期: 2022
• 页码: 3415-3438
• 关键词: Consistency

摘要

As a flexible nonparametric learning tool, the random forests algorithm has been widely applied to various real applications with appealing empirical performance, even in the presence of high-dimensional feature space. Unveil-ing the underlying mechanisms has led to some important recent theoretical results on the consistency of the random forests algorithm and its variants. However, to our knowledge, almost all existing works concerning random forests consistency in a high-dimensional setting were established for various modified random forests models where the splitting rules are independent of the response; a few exceptions assume simple data generating models with binary features. In light of this, in this paper we derive the consistency rates for the random forests algorithm associated with the sample CART splitting criterion, which is the one used in the original version of the algorithm (Mach. Learn. 45 (2001) 5-32), in a general high-dimensional nonparametric regres-sion setting through a bias-variance decomposition analysis. Our new theoret-ical results show that random forests can indeed adapt to high dimensionality and allow for discontinuous regression function. Our bias analysis character-izes explicitly how the random forests bias depends on the sample size, tree height and column subsampling parameter. Some limitations on our current results are also discussed.