Low-Complexity Q-Value Iteration Learning for Linear Quadratic Regulators Without Initial Stabilizing Gain

• 成果类型: Article
• 署名作者: Huang, Longyang; Zhao, Lin; Zhang, Weidong
• 署名单位: Shanghai Jiao Tong University; National University of Singapore; Hainan University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2025.3632464
• 发表日期: 2026
• 关键词: ADAPTIVE OPTIMAL-CONTROL; DISCRETE-TIME-SYSTEMS; reinforcement

摘要

Reinforcement learning (RL) has demonstrated promising results in the data-driven design of linear quadratic regulator (LQR) controllers. However, existing RL-based LQR controller design methods face challenges regarding computational complexity, sample complexity, system stability, and the requirement for a stabilizing gain. To address these issues, this article proposes a novel low-complexity Q-value iteration algorithm. We establish the convergence and monotonicity of the iterative sequence generated by the proposed algorithm. Furthermore, an algorithmic stopping condition is designed to guarantee that the resulting control gain is stabilizing. In contrast to existing data-driven methods, the proposed algorithm achieves both significant improvements in computational and sample efficiency and removes the requirement for a stabilizing gain. Comparative simulation studies are conducted to demonstrate the effectiveness of the proposed algorithm.