PARTIAL CORRELATION SCREENING FOR ESTIMATING LARGE PRECISION MATRICES, WITH APPLICATIONS TO CLASSIFICATION

• 成果类型: Article
• 署名作者: Huang, Shiqiong; Jin, Jiashun; Yao, Zhigang
• 署名单位: Carnegie Mellon University; National University of Singapore
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/15-AOS1392
• 发表日期: 2016
• 页码: 2018-2057
• 关键词: higher criticism; COVARIANCE ESTIMATION; feature-selection; ALTERNATIVES; features; tests; MODEL; graph; rare; BIAS

摘要

Given n samples X-1, X-2,...,X-n from N(0, Sigma), we are interested in estimating the p x p precision matrix Omega = Sigma(-)1; we assume Omega is sparse in that each row has relatively few nonzeros. We propose Partial Correlation Screening (PCS) as a new row -by -row approach. To estimate the ith row of Omega, 1 <= i <= p, PCS uses a Screen step and a Clean step. In the Screen step, PCS recruits a (small) subset of indices using a stage -wise algorithm, where in each stage, the algorithm updates the set of recruited indices by adding the index j that has the largest empirical partial correlation (in magnitude) with i, given the set of indices recruited so far. In the Clean step, PCS reinvestigates all recruited indices, removes false positives and uses the resultant set of indices to reconstruct the ith row. PCS is computationally efficient and modest in memory use: to estimate a row of Omega, it only needs a few rows (determined sequentially) of the empirical covariance matrix. PCS is able to execute an estimation of a large Omega (e.g., p = 10K) in a few minutes. Higher Criticism Thresholding (HCT) is a recent classifier that enjoys optimality, but to exploit its full potential, we need a good estimate of Omega. Note that given an estimate of Omega, we can always combine it with HCT to build a classifier (e.g., HCT-PCS, HCT-glasso). We have applied HCT-PCS to two microarray data sets (p = 8K and 10K) for classification, where it not only significantly outperforms HCT-glasso, but also is competitive to the Support Vector Machine (SVM) and Random Forest (RF). These suggest that PCS gives more useful estimates of Omega than the glasso; we study this carefully and have gained some interesting insight. We show that in a broad context, PCS fully recovers the support of Omega and HCT-PCS is optimal in classification. Our theoretical study sheds interesting light on the behavior of stage-wise procedures.