Discrete-Time Partially Observable Stochastic Optimal Control Problems of McKean-Vlasov Type and Branching Particle System Approximations

• 成果类型: Article
• 署名作者: Wan, Hexiang; Wang, Guangchen; Xiong, Jie
• 署名单位: Shandong University; Southern University of Science & Technology; Southern University of Science & Technology
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2024.3525256
• 发表日期: 2025
• 页码: 4376-4391
• 关键词: Optimal control; Stochastic processes; mathematical models; kernel; Filtering theory; Stochastic systems; Particle filters; COSTS; Particle measurements; Atmospheric measurements; Branching particle system; discrete-time system; dynamic programming principle (DPP); McKean-Vlasov (MKV) optimal control; measurable selection; Nisio semigroup

摘要

This article investigates a broad category of McKean-Vlasov type discrete-time partially observable stochastic optimal control problems. The first goal is to prove the dynamic programming principle (DPP) by means of the measurable selection argument, which provides a methodology for finding both the value function as well as the optimal control. Here, we employ the Nisio semigroup technology, which is an intrinsic characterization of the DPP. Then, we derive the recursive formula for the filter process, which enables us to clearly track the time evolution of the posterior distribution. Next, we approximate the posterior distribution utilizing branching particle systems (branching particle filters) and illustrate its convergence. Making use of branching particle system approximations and Bellman equations, we devise a numerical algorithm for addressing the optimal control problem. A numerical experiment serves as the final part of this article.