Optimal Control of Probability on a Target Set for Continuous-Time Markov Chains

• 成果类型: Article
• 署名作者: Ma, Chenglin; Zhao, Huaizhong
• 署名单位: Shandong University; Durham University; Shandong University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2023.3278789
• 发表日期: 2024
• 页码: 1202-1209
• 关键词: Markov processes; optimal control; dynamic programming; PROCESS CONTROL; games; Aerospace electronics; safety; controlled Markov chains; dynamic programming principle (DPP); Hamilton-Jacobi-Bellman (HJB) equation; optimal controls; risk probability criteria

摘要

In this article, a stochastic optimal control problem is considered for a continuous-time Markov chain taking values in a denumerable state space over a fixed finite horizon. The optimality criterion is the probability that the process remains in a target set before and at a certain time. The optimal value is a superadditive capacity of target sets. Under some minor assumptions for the controlled Markov process, we establish the dynamic programming principle, based on which we prove that the value function is a classical solution of the Hamilton-Jacobi-Bellman (HJB) equation on a discrete lattice space. We then prove that there exists an optimal deterministic Markov control under the compactness assumption of control domain. We further prove that the value function is the unique solution of the HJB equation. We also consider the case starting from the outside of the target set and give the corresponding results. Finally, we apply our results to two examples.