Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning

• 成果类型: Article; Early Access
• 署名作者: Srikant, R.
• 署名单位: University of Illinois System; University of Illinois Urbana-Champaign; University of Illinois System; University of Illinois Urbana-Champaign
• 刊物名称: MATHEMATICS OF OPERATIONS RESEARCH
• ISSN/ISSBN: 0364-765X
• DOI: 10.1287/moor.2024.0444
• 发表日期: 2025
• 关键词: stochastic-approximation; distance; bounds; time; gap

摘要

We prove a nonasymptotic central limit theorem (CLT) for vector-valued martingale differences using Stein's method, and we use Poisson's equation to extend the result to functions of Markov chains. We then show that these results can be applied to establish a nonasymptotic CLT for temporal difference learning with averaging.