Online Statistical Inference for Contextual Bandits via Stochastic Gradient Descent

• 成果类型: Article; Early Access
• 署名作者: Chang, Xiangyu; Chen, Xi; Lai, Zehua; Li, He; Liu, Zhihong; Zhang, Yichen
• 署名单位: Xi'an Jiaotong University; New York University; University of Texas System; University of Texas Austin; Purdue University System; Purdue University
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459; 1537-274X
• DOI: 10.1080/01621459.2026.2621503
• 发表日期: 2026-03-23
• 关键词: bahadur representation; contextual bandit; quantile regression; online inference; stochastic gradient descent

摘要

With the fast development of big data, learning the optimal decision rule by recursively updating it and making online decisions has been easier than before. We study the online statistical inference of model parameters in a contextual bandit framework of sequential decision-making. We propose a general framework for an online and adaptive data collection environment that can update decision rules via weighted stochastic gradient descent. We allow different weighting schemes of the stochastic gradient and establish the asymptotic normality of the parameter estimator. Our proposed estimator significantly improves the asymptotic efficiency over the previous averaged SGD approach via inverse probability weights. We also conduct an optimality analysis on the weights in a linear regression setting. We provide a Bahadur representation of the proposed estimator and show that the remainder term in the Bahadur representation entails a slower convergence rate compared to classical SGD due to the adaptive data collection. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.