Uniform Turnpike Theorems for Finite Markov Decision Processes
成果类型：
Article
署名作者：
Lewis, Mark E.; Paul, Anand
署名单位：
Cornell University; State University System of Florida; University of Florida
刊物名称：
MATHEMATICS OF OPERATIONS RESEARCH
ISSN/ISSBN：
0364-765X
DOI：
10.1287/moor.2017.0912
发表日期：
2019
页码：
1145-1160
关键词：
摘要：
A turnpike integer is the smallest finite horizon for which an optimal infinite horizon decision is the optimal initial decision. An important practical question considered in the literature is how to bound the turnpike integer using only the problem inputs. In this paper, we consider turnpike integers as a function of the discount factor. While a turnpike integer is finite for any fixed discount factor, we show that it approaches infinity in the neighborhood of a specific set of discount rates (for all but some exceptional finite Markov decision processes). We provide several examples that illustrate how this taboo set of discount factors may arise and find sufficient conditions for a set of turnpike integers to be unbounded. This finding provides a cautionary tale for practitioners using point estimates of the discount factor to manage the length of rolling horizons by pointing to potential singularities in the procedure.
来源URL：