Policy Iteration Reinforcement Learning Method for Continuous-Time Linear-Quadratic Mean-Field Control Problems

• 成果类型: Article
• 署名作者: Li, Na; Li, Xun; Xu, Zuo Quan
• 署名单位: Shandong University of Finance & Economics; Hong Kong Polytechnic University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2024.3494656
• 发表日期: 2025
• 页码: 2690-2697
• 关键词: Optimal control; mathematical models; trajectory; COSTS; Stochastic processes; Symmetric matrices; STANDARDS; Reinforcement Learning; finance; CONVERGENCE; Linear-quadratic (LQ) problem; mean-field (MF) optimal problem; policy iteration; reinforcement learning (RL)

摘要

In this article, we employ a policy iteration reinforcement learning (RL) method to study continuous-time linear-quadratic mean-field control problems in infinite horizon. The drift and diffusion terms in the dynamics involve the states, the controls, and their conditional expectations. We investigate the stabilizability and convergence of the RL algorithm using a Lyapunov recursion. Instead of solving a pair of coupled Riccati equations, the RL technique focuses on strengthening an auxiliary function and the cost functional as the objective functions and updating the new policy to compute the optimal control via state trajectories. A numerical example sheds light on the established theoretical results.