Online Actuator Selection and Controller Design for Linear Quadratic Regulation With an Unknown System Model

• 成果类型: Article
• 署名作者: Ye, Lintao; Chi, Ming; Liu, Zhi-Wei; Gupta, Vijay
• 署名单位: Huazhong University of Science & Technology; Huazhong University of Science & Technology; Purdue University System; Purdue University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2024.3422105
• 发表日期: 2025
• 页码: 18-33
• 关键词: actuators; trajectory; regulation; Approximation algorithms; Perturbation methods; COSTS; Additives; Large-scale system; Multiarmed bandit problem; optimal control; Reinforcement Learning; stochastic system

摘要

In this article, we study the simultaneous actuator selection and controller design problem for linear quadratic regulation with Gaussian noise over a finite horizon of length $T$ and unknown system model. We consider episodic and nonepisodic settings of the problem and propose online algorithms that specify both the sets of actuators to be utilized under a cardinality constraint and the controls corresponding to the sets of selected actuators. In the episodic setting, the interaction with the system breaks into $N$ episodes, each of which restarts from a given initial condition and has length $T$. In the nonepisodic setting, the interaction goes on continuously. Our online algorithms leverage a multiarmed bandit algorithm to select the sets of actuators and a certainty equivalence approach to design the corresponding controls. We show that our online algorithms yield $\sqrt{N}$-regret for the episodic setting and $T<^>{2/3}$-regret for the nonepisodic setting. We extend our algorithm design and analysis to show scalability with respect to both the total number of candidate actuators and the cardinality constraint. We numerically validate our theoretical results.