Residual Importance Weighted Transfer Learning for High-dimensional Linear Regression

• 成果类型: Article; Early Access
• 署名作者: Zhao, Junlong; Zheng, Shengbin; Leng, Chenlei
• 署名单位: Beijing Normal University; Hong Kong Polytechnic University
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459; 1537-274X
• DOI: 10.1080/01621459.2026.2623997
• 发表日期: 2026-05-27
• 关键词: Density Estimation; High-dimensional linear models; Importance weighting; sample selection; Transfer Learning; selection

摘要

Transfer learning is an emerging paradigm for leveraging multiple sources to improve the statistical inference on a single target. In this article, we propose a novel approach named residual importance weighted transfer learning (RIW-TL) for high-dimensional linear models built on penalized likelihood. Compared to existing methods such as Trans-Lasso that selects sources in an (approximately) all-in-or-all-out manner, RIW-TL includes samples via importance weighting and thus may permit more effective sample use. To determine the weights, remarkably RIW-TL only requires the knowledge of one-dimensional densities dependent on residuals, thus overcoming the curse of dimensionality of having to estimate high-dimensional densities in naive importance weighting. We show that the oracle RIW-TL provides faster rate than its competitors and develop a cross-fitting procedure to estimate this oracle. We discuss variants of RIW-TL by adopting different choices for residual weighting. The theoretical properties of RIW-TL and its variants are established and compared with those of LASSO and Trans-Lasso. Extensive simulation and a real data analysis confirm its advantages. Supplementary materials for this article are available online.