A Robust Optimization Approach to Reliable Statistical Inference with Variables Generated by

• 成果类型: Article; Early Access
• 署名作者: Schecler, Aaron; Li, Weifeng
• 署名单位: University System of Georgia; University of Georgia
• 刊物名称: INFORMATION SYSTEMS RESEARCH
• ISSN/ISSBN: 1047-7047; 1526-5536
• DOI: 10.1287/isre.2023.0340
• 发表日期: 2025-12-24
• 关键词: Robust Optimization; Machine Learning; regression; measurement error correction; BIAS; measurement error

摘要

Leveraging supervised machine learning (SML) algorithms to operationalize constructs from unstructured data such as text or images is becoming increasingly common in practice and research. As a result, variables generated through SML are now used in traditional regression models to test hypotheses. However, algorithms are imperfect, and thus, the variables produced by SML have measurement errors relative to the underlying construct, potentially leading to biased coefficients and faulty inference. In this paper, we propose using robust optimization to reduce the negative impact of these errors and enable more accurate hypothesis testing. We leverage robust optimization techniques to fit a linear regression model in the presence of measurement errors of different magnitudes. We theoretically demonstrate the bias, variance, and hypothesis testing performance of the robust approach and propose a correction term to effectively reduce bias. Through experiments on simulated data sets and a case study of Amazon reviews, we demonstrate the effectiveness of our approach and identify conditions in which robust optimization likely outperforms other methods. We make recommendations for researchers leveraging machine learning-generated variables in causal inference.