Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

• 成果类型: Article
• 署名作者: Guo, Xianping; Piunovskiy, Alexei
• 署名单位: Sun Yat Sen University; University of Liverpool
• 刊物名称: MATHEMATICS OF OPERATIONS RESEARCH
• ISSN/ISSBN: 0364-765X
• DOI: 10.1287/moor.1100.0477
• 发表日期: 2011
• 页码: 105-132
• 关键词: chains

摘要

This paper deals with denumerable continuous-time Markov decision processes (MDP) with constraints. The optimality criterion to be minimized is expected discounted loss, while several constraints of the same type are imposed. The transition rates may be unbounded, the loss rates are allowed to be unbounded as well (from above and from below), and the policies may be history-dependent and randomized. Based on Kolmogorov's forward equation and Dynkin's formula, we remind the reader about the Bellman equation, introduce and study occupation measures, reformulate the optimization problem as a (primary) linear program, provide the form of optimal policies for a constrained optimization problem here, and establish the duality between the convex analytic approach and dynamic programming. Finally, a series of examples is given to illustrate all of our results.