ENSEMBLE PROJECTION PURSUIT FOR GENERAL NONPARAMETRIC REGRESSION

• 成果类型: Article
• 署名作者: Zhan, Haoran; Zhang, Mingke; Xia, Yingcun
• 署名单位: National University of Singapore; University of Electronic Science & Technology of China
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/24-AOS2460
• 发表日期: 2025
• 页码: 194-218
• 关键词: greedy approximation; convergence-rates; networks

摘要

Projection pursuit regression (PPR) and artificial neural networks (ANNs), both introduced around the same time, share a similar approach to approximating complex structures in data. However, PPR has received far less attention than ANNs and other popular statistical learning methods such as random forests (RF) and support vector machines (SVM). In this paper, we revisit the estimation of PPR and propose an optimal greedy algorithm and an ensemble approach via feature bagging, hereafter referred to as ePPR, to improve its effectiveness. Compared to RF, ePPR has two main advantages. First, its theoretical consistency can be proved for more general regression functions, as long as they are L2 integrable, and higher consistency rates can be achieved. Second, ePPR does not split the samples, so each term of the PPR is estimated using the entire data, making minimisation more efficient and ensuring the smoothness of the estimator. Extensive comparisons on real data sets show that ePPR is more efficient in regression and classification than RF and other competitors.