Online Statistical Inference for Stochastic Optimization via Kiefer-Wolfowitz Methods

• 成果类型: Article
• 署名作者: Chen, Xi; Lai, Zehua; Li, He; Zhang, Yichen
• 署名单位: New York University; University of Chicago; Purdue University System; Purdue University
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459
• DOI: 10.1080/01621459.2023.2296703
• 发表日期: 2024
• 页码: 2972-2982
• 关键词: Approximation; CONVERGENCE

摘要

This article investigates the problem of online statistical inference of model parameters in stochastic optimization problems via the Kiefer-Wolfowitz algorithm with random search directions. We first present the asymptotic distribution for the Polyak-Ruppert-averaging type Kiefer-Wolfowitz (AKW) estimators, whose asymptotic covariance matrices depend on the distribution of search directions and the function-value query complexity. The distributional result reflects the tradeoff between statistical efficiency and function query complexity. We further analyze the choice of random search directions to minimize certain summary statistics of the asymptotic covariance matrix. Based on the asymptotic distribution, we conduct online statistical inference by providing two construction procedures of valid confidence intervals. Supplementary materials for this article are available online.