Constrained Control of Large Graph-Based MDPs Under Measurement Uncertainty

• 成果类型: Article
• 署名作者: Haksar, Ravi N.; Schwager, Mac
• 署名单位: Stanford University; Stanford University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2023.3242344
• 发表日期: 2023
• 页码: 6605-6620
• 关键词: filtering; Markov processes; network analysis and control; stochastic optimal control; variational methods.

摘要

We consider controlling a graph-based Markov decision process (GMDP) with a control capacity constraint given only uncertain measurements of the underlying state. We also consider two special structural properties of GMDPs, called anonymous influence and symmetry. Large-scale spatial processes such as forest wildfires, disease epidemics, opinion dynamics, and robot swarms are well-modeled by GMDPs with these properties. We adopt a certainty-equivalence approach and derive efficient and scalable algorithms for estimating the GMDP state given uncertain measurements, and for computing approximately optimal control policies given a maximum-likelihood state estimate. We also derive suboptimality bounds for our estimation and control algorithms. Unlike prior work, our methods scale to GMDPs with large state-spaces and explicitly enforce a control constraint. We demonstrate the effectiveness of our estimation and control approach in simulations of controlling a forest wildfire using a model with 10(1192) total states.