SUPERVISED HOMOGENEITY FUSION: A COMBINATORIAL APPROACH

• 成果类型: Article
• 署名作者: Wang, Wen; Wu, Shihao; Zhu, Ziwei; Zhou, Ling; Song, Peter X. -K.
• 署名单位: University of Michigan System; University of Michigan; University of Michigan System; University of Michigan; Southwestern University of Finance & Economics - China
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/23-AOS2347
• 发表日期: 2024
• 页码: 285-310
• 关键词: VARIABLE SELECTION; REGRESSION SHRINKAGE; grouping pursuit; identification; RECOVERY; exposure; sparsity; subset

摘要

Fusing regression coefficients into homogeneous groups can unveil those coefficients that share a common value within each group. Such groupwise homogeneity reduces the intrinsic dimension of the parameter space and unleashes sharper statistical accuracy. We propose and investigate a new combinatorial grouping approach called L-0-Fusion that is amenable to mixed integer optimization (MIO). On the statistical aspect, we identify a fundamental quantity called MSE grouping sensitivity that underpins the difficulty of recovering the true groups. We show that L-0-Fusion achieves grouping consistency under the weakest possible requirement of the grouping sensitivity: if this requirement is violated, then the minimax risk of group misspecification will fail to converge to zero. Moreover, we show that in the high-dimensional regime, one can apply L-0-Fusion with a sure screening set of features without any essential loss of statistical efficiency, while reducing the computational cost substantially. On the algorithmic aspect, we provide an MIO formulation for L-0-Fusion along with a warm start strategy. Simulation and real data analysis demonstrate that L-0-Fusion exhibits superiority over its competitors in terms of grouping accuracy.