On the Impossibility of Statistically Improving Empirical Optimization: A Second Order Stochastic Dominance Perspective

• 成果类型: Article; Early Access
• 署名作者: Lam, Henry
• 署名单位: Columbia University
• 刊物名称: MANAGEMENT SCIENCE
• ISSN/ISSBN: 0025-1909
• DOI: 10.1287/mnsc.2024.04482
• 发表日期: 2025
• 关键词: empirical optimization; second order stochastic dominance; optimality gap; regularization; distributionally robust optimization

摘要

When the underlying probability distribution in a stochastic optimization is observed only through data, various data-driven formulations have been studied to obtain approximate optimal solutions. We show that no such formulations can, in a sense, theoretically improve the statistical quality of the solution obtained from empirical optimization. We argue this by proving that the first order behavior of the optimality gap against the oracle best solution, which includes both the bias and variance, for any data-driven solution second order stochastically dominates that from empirical optimization as long as suitable smoothness holds with respect to the underlying distribution. We demonstrate this impossibility of improvement in examples ranging across regularized optimization, distributionally robust optimization, parametric optimization, and Bayesian generalizations. We also discuss the connections of our results to other perspectives in statistics and data-driven optimization and illustrate practical implications in choosing among data-driven formulations.