Nonsparse Learning with Latent Variables

• 成果类型: Article
• 署名作者: Zheng, Zemin; Lv, Jinchi; Lin, Wei
• 署名单位: Chinese Academy of Sciences; University of Science & Technology of China, CAS; University of Southern California; Peking University; Peking University
• 刊物名称: OPERATIONS RESEARCH
• ISSN/ISSBN: 0030-364X
• DOI: 10.1287/opre.2020.2005
• 发表日期: 2021
• 页码: 346-359
• 关键词: nonconcave penalized likelihood; principal-components-analysis; model selection; confidence-intervals; regression; Consistency; shrinkage; regularization; eigenstructure; asymptotics

摘要

As a popular tool for producing meaningful and interpretable models, large-scale sparse learning works efficiently in many optimization applications when the underlying structures are indeed or close to sparse. However, naively applying the existing regularization methods can result in misleading outcomes because of model mis-specification. In this paper, we consider nonsparse learning under the factors plus sparsity structure, which yields a joint modeling of sparse individual effects and common latent factors. A new methodology of nonsparse learning with latent variables (NSL) is proposed for joint estimation of the effects of two groups of features, one for individual effects and the other associated with the latent substructures, when the nonsparse effects are captured by the leading population principal component score vectors. We derive the convergence rates of both sample principal components and their score vectors that hold for a wide class of distributions. With the properly estimated latent variables, properties including model selection consistency and oracle inequalities under various prediction and estimation losses are established. Our new methodology and results are evidenced by simulation and real-data examples.