Solving Feynman-Kac Forward-Backward SDEs Using McKean-Markov Branched Sampling

• 成果类型: Article
• 署名作者: Hawkins, Kelsey P.; Pakniyat, Ali; Theodorou, Evangelos; Tsiotras, Panagiotis
• 署名单位: University System of Georgia; Georgia Institute of Technology; University of Alabama System; University of Alabama Tuscaloosa; University System of Georgia; Georgia Institute of Technology
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2023.3349173
• 发表日期: 2024
• 页码: 5695-5710
• 关键词: Least mean square methods; nonlinear control systems; optimal control; Partial differential equations; Stochastic processes; Trajectory optimization; tree graphs

摘要

We propose a new method for the numerical solution of the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. Using Girsanov's change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift term is appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of a space-filling tree consisting of trajectory samples. Moreover, a local entropy-weighted least squares Monte Carlo (LSMC) method is developed to concentrate function approximation accuracy in regions most likely to be visited by optimally controlled trajectories. The proposed methodology is numerically demonstrated for linear and nonlinear stochastic optimal control problems with nonquadratic running costs, which reveal significant convergence improvements over previous FBSDE-based numerical solution methods.