Risk-averse dynamic programming for Markov decision processes

• 成果类型: Article; Proceedings Paper
• 署名作者: Ruszczynski, Andrzej
• 署名单位: Rutgers University System; Rutgers University New Brunswick
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610
• DOI: 10.1007/s10107-010-0393-3
• 发表日期: 2010
• 页码: 235-261
• 关键词: stochastic-dominance; utility; criterion

摘要

We introduce the concept of a Markov risk measure and we use it to formulate risk-averse control problems for two Markov decision models: a finite horizon model and a discounted infinite horizon model. For both models we derive risk-averse dynamic programming equations and a value iteration method. For the infinite horizon problem we develop a risk-averse policy iteration method and we prove its convergence. We also propose a version of the Newton method to solve a nonsmooth equation arising in the policy iteration method and we prove its global convergence. Finally, we discuss relations to min-max Markov decision models.