Inference for Low-Rank Models Without Estimating the Rank

• 成果类型: Article; Early Access
• 署名作者: Choi, Jungjun; Kwon, Hyukjun; Liao, Yuan
• 署名单位: University of Rhode Island; Princeton University; Rutgers University System; Rutgers University New Brunswick
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459
• DOI: 10.1080/01621459.2025.2538272
• 发表日期: 2025
• 关键词: false discovery rate; matrix completion

摘要

This article studies the inference about linear functionals of high-dimensional low-rank matrices. While most existing inference methods would require consistent estimation of the true rank, our procedure is robust to rank misspecification, making it a promising approach in applications where rank estimation can be unreliable. We estimate the low-rank spaces using pre-specified weighting matrices, known as diversified projections. A novel statistical insight is that, unlike the usual statistical wisdom that overfitting mainly introduces additional variances, the over-estimated low-rank space also gives rise to a non-negligible bias due to an implicit ridge-type regularization. We develop a new inference procedure and show that the central limit theorem holds as long as the pre-specified rank is no smaller than the true rank. In one of our applications, we study multiple testing with incomplete data in the presence of confounding factors and show that our method remains valid as long as the number of controlled confounding factors is at least as large as the true number, even when no confounding factors are present. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.