Deep P-Spline: Theory, Fast Tuning, and Application

• 成果类型: Article
• 署名作者: Hung, Noah Yi-Ting; Lin, Li-Hsiang; Calhoun, Vince D.
• 署名单位: University System of Georgia; Georgia State University; University System of Georgia; Emory University; Georgia Institute of Technology; Georgia State University
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459; 1537-274X
• DOI: 10.1080/01621459.2026.2671447
• 发表日期: 2026-04-03
• 页码: 1051-1063
• 关键词: Difference penalty; feature selection; regularization; Smoothing and nonparametric regression; Surrogate modeling; maximum-likelihood; Cross-validation; CONVERGENCE; bounds

摘要

Deep neural networks (DNNs) have been widely applied to solve real-world regression problems. However, selecting optimal network structures remains a significant challenge. This study addresses this issue by linking neuron selection in DNNs to knot placement in basis expansion techniques. We introduce a difference penalty that automates knot selection, thereby simplifying the complexities of neuron selection. We name this method Deep P-Spline (DPS). This approach extends the class of models considered in conventional DNN modeling and forms the basis for a latent variable modeling framework using the Expectation-Conditional Maximization (ECM) algorithm for efficient network structure tuning with theoretical guarantees. From a nonparametric regression perspective, DPS is proven to overcome the curse of dimensionality, enabling the effective handling of datasets with a large number of input variables-a scenario where conventional nonparametric regression methods typically underperform. This capability motivates the application of the proposed methodology to computer experiments and image data analyses, where the associated regression problems involving numerous inputs are common. Numerical results validate the effectiveness of the model, underscoring its potential for advanced nonlinear regression tasks. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.