Structured Output Feedback Control for Linear Quadratic Regulator Using Policy Gradient Method

• 成果类型: Article
• 署名作者: Takakura, Shokichi; Sato, Kazuhiro
• 署名单位: University of Tokyo
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2023.3264176
• 发表日期: 2024
• 页码: 363-370
• 关键词: Output feedback; gradient methods; CONVERGENCE; linear programming; Symmetric matrices; estimation; State feedback; Data-driven control; Gradient descent; linear quadratic regulator (LQR); model free control; Nonconvex Optimization; reinforcement learning (RL)

摘要

In this article, we consider the static output feedback control for linear quadratic regulator problems with structured constraints under the assumption that system parameters are unknown. To solve the problem in the model free setting, we propose the policy gradient algorithm based on the gradient projection method and show its global convergence to epsilon-stationary points. In addition, we introduce a variance reduction technique and show both theoretically and numerically that it significantly reduces the variance in the gradient estimation. We also show in the numerical experiments that the model free approach efficiently solves the problem.