Computational Methods for Risk-Averse Undiscounted Transient Markov Models

• 成果类型: Article
• 署名作者: Cavus, Ozlem; Ruszczynski, Andrzej
• 署名单位: Ihsan Dogramaci Bilkent University; Rutgers University System; Rutgers University New Brunswick
• 刊物名称: OPERATIONS RESEARCH
• ISSN/ISSBN: 0030-364X
• DOI: 10.1287/opre.2013.1251
• 发表日期: 2014
• 页码: 401-417
• 关键词: shortest-path problems; decision-processes; DISCRETE-TIME; sensitive control; stochastic-dominance; infinite-horizon; criteria; probability; variance; policies

摘要

The total cost problem for discrete-time controlled transient Markov models is considered. The objective functional is a Markov dynamic risk measure of the total cost. Two solution methods, value and policy iteration, are proposed, and their convergence is analyzed. In the policy iteration method, we propose two algorithms for policy evaluation: the nonsmooth Newton method and convex programming, and we prove their convergence. The results are illustrated on a credit limit control problem.