Domain-Specific Nonparametric Regression for Domain Generalization

• 成果类型: Article; Early Access
• 署名作者: Zheng, Siming; Lin, Yuanyuan; Zhou, Yong; Huang, Jian
• 署名单位: Southeast University - China; Chinese University of Hong Kong; East China Normal University; Hong Kong Polytechnic University
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459; 1537-274X
• DOI: 10.1080/01621459.2026.2658866
• 发表日期: 2026-06-25
• 关键词: Domain generalization; Domain-specific feature; Deep Neural Networks; Non-asymptotic error bounds; Nonparametric Regression; neural-networks

摘要

We propose a domain-specific regression approach for domain generalization, taking into account possible heterogeneity among the datasets from different sources. In the proposed model, the domain-specific features are characterized through linear functionals of the marginal source distributions. The predictors are combined with the domain-specific linear functionals as inputs in the model. Using the source datasets, we estimate the domain-index function and the regression function nonparametrically based on neural network approximation. The resulting estimated domain-specific regression function can be used for prediction when future unlabeled data from a target domain arrives. We establish the convergence rates at which the cross-domain prediction error of the estimated domain-specific predictive function converges to that of the true one under suitable conditions. Specifically, we derive a general nonasymptotic bound characterized by the hierarchical variation of the proposed model. In addition, we prove that the proposed method can mitigate the curse of dimensionality under some low-dimensional structure assumption. We demonstrate the performance of the proposed method through numerical experiments with simulated and real data. The numerical results show that, our method outperforms several existing methods in terms of the prediction accuracy or computational efficiency, and is robust to model assumptions. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.