EFFICIENT AND ADAPTIVE LINEAR REGRESSION IN SEMI-SUPERVISED SETTINGS

• 成果类型: Article
• 署名作者: Chakrabortty, Abhishek; Cai, Tianxi
• 署名单位: University of Pennsylvania; Harvard University
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/17-AOS1594
• 发表日期: 2018
• 页码: 1541-1572
• 关键词: sliced inverse regression; principal hessian directions; Dimension Reduction; kernel estimation; DISCOVERY; records; samples

摘要

We consider the linear regression problem under semi-supervised settings wherein the available data typically consists of: (i) a small or moderate sized labeled data, and (ii) a much larger sized unlabeled data. Such data arises naturally from settings where the outcome, unlike the covariates, is expensive to obtain, a frequent scenario in modern studies involving large databases like electronic medical records (EMR). Supervised estimators like the ordinary least squares (OLS) estimator utilize only the labeled data. It is often of interest to investigate if and when the unlabeled data can be exploited to improve estimation of the regression parameter in the adopted linear model. In this paper, we propose a class of Efficient and Adaptive Semi-Supervised Estimators (EASE) to improve estimation efficiency. The EASE are two-step estimators adaptive to model mis-specification, leading to improved (optimal in some cases) efficiency under model mis-specification, and equal (optimal) efficiency under a linear model. This adaptive property, often unaddressed in the existing literature, is crucial for advocating safe use of the unlabeled data. The construction of EASE primarily involves a flexible semi-nonparametric imputation, including a smoothing step that works well even when the number of covariates is not small; and a follow up refitting step along with a cross-validation (CV) strategy both of which have useful practical as well as theoretical implications towards addressing two important issues: under-smoothing and over-fitting. We establish asymptotic results including consistency, asymptotic normality and the adaptive properties of EASE. We also provide influence function expansions and a double CV strategy for inference. The results are further validated through extensive simulations, followed by application to an EMR study on auto-immunity.