On Reachability of Markov Chains: A Long-Run Average Approach

• 成果类型: Article
• 署名作者: Avila, Daniel; Junca, Mauricio
• 署名单位: Universite Catholique Louvain; Universidad de los Andes (Colombia)
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2021.3071334
• 发表日期: 2022
• 页码: 1996-2003
• 关键词: Long-run average; Markov decision processes (MDP); probabilistic constraints; reach avoid

摘要

We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we characterize the domain of attraction and escape set of the system, and a generalization called p-domain of attraction, using the aforementioned value function. Next, we solve the problem of maximizing the probability of reaching a set A while avoiding a set B. Finally, we consider a constrained version of previous problem, where we ask for the probability of reaching the set B to be bounded. In the finite case, we use linear programming formulations to solve these problems.