MDP modeling for multi-stage stochastic programs

• 成果类型: Article
• 署名作者: Morton, David P.; Dowson, Oscar; Pagnoncelli, Bernardo K.
• 署名单位: Northwestern University; SKEMA Business School; Universite Cote d'Azur
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-026-02368-8
• 发表日期: 2026-05
• 页码: 43-78
• 关键词: Multi-stage stochastic programming; Markov decision processes; Policy graph; decision-dependent uncertainty; Statistical learning; decomposition methods; CONVERGENCE; algorithm

摘要

We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include decision-dependent uncertainty for one-step transition probabilities as well as a limited form of statistical learning. We focus on the expressiveness of our modeling approach, illustrating ideas with a series of examples of increasing complexity. As a solution method, we develop new variants of stochastic dual dynamic programming, including approximations to handle non-convexities.