Model-Free Offline Reinforcement Learning for Linear Quadratic Control

• 成果类型: Article
• 署名作者: Ma, Haoran; Zhao, Zhengen; Liang, Dingguo; Yang, Ying
• 署名单位: State Grid Corporation of China; China Electric Power Research Institute (CEPRI); Nanjing University of Aeronautics & Astronautics; Peking University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2025.3649291
• 发表日期: 2026
• 关键词: EXPERIMENT DESIGN; driven; systems

摘要

This article investigates the linear quadratic control problem with both process and measurement noise using reinforcement learning. Instead of requiring a system model or real-time controller updates, the proposed approach derives the optimal controller from offline and noise-corrupted data. A key contribution is the formulation of the Bellman optimality equation with a correction term that compensates for the bias introduced by process and measurement noise. This addresses the limitations of existing model-free reinforcement learning approaches, which may fail to achieve optimality in noisy environments. The analysis of the algorithm provides guarantees on robustness, control performance, and computational complexity. Simulations validate the effectiveness of the proposed method and demonstrate improvements in accuracy and runtime.