Technical Note-Cyclic Variables and Markov Decision Processes

• 成果类型: Article
• 署名作者: Haviv, Avery
• 署名单位: University of Rochester
• 刊物名称: OPERATIONS RESEARCH
• ISSN/ISSBN: 0030-364X
• DOI: 10.1287/opre.2019.1913
• 发表日期: 2020
• 页码: 1231-1237
• 关键词: Markov decision process; structural dynamic models; infinite-horizon dynamic programming; econometrics; cyclic variables; seasonality

摘要

In this paper I develop a cyclic value function iteration, which is an adjustment to the standard value function iteration. When using this algorithm, the inclusion of cyclic variables of any size into the state space of an infinite horizon Markov decision process does not increase the computational complexity of solving for the value function. This result is proven theoretically and shown to closely hold in practice using Monte Carlo simulations.