Covariance-regularized regression and classification for high dimensional problems

• 成果类型: Article
• 署名作者: Witten, Daniela M.; Tibshirani, Robert
• 署名单位: Stanford University
• 刊物名称: JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY
• ISSN/ISSBN: 1369-7412
• DOI: 10.1111/j.1467-9868.2009.00699.x
• 发表日期: 2009
• 页码: 615-636
• 关键词: maximum-likelihood-estimation; shrunken centroids; variable selection; model selection; lymphoma; microarrays; prediction; diagnosis; algorithm; survival

摘要

We propose covariance-regularized regression, a family of methods for prediction in high dimensional settings that uses a shrunken estimate of the inverse covariance matrix of the features to achieve superior prediction. An estimate of the inverse covariance matrix is obtained by maximizing the log-likelihood of the data, under a multivariate normal model, subject to a penalty; it is then used to estimate coefficients for the regression of the response onto the features. We show that ridge regression, the lasso and the elastic net are special cases of covariance-regularized regression, and we demonstrate that certain previously unexplored forms of covariance-regularized regression can outperform existing methods in a range of situations. The covariance-regularized regression framework is extended to generalized linear models and linear discriminant analysis, and is used to analyse gene expression data sets with multiple class and survival outcomes.