Decoupled Data-Based Approach for Learning to Control Nonlinear Dynamical Systems

• 成果类型: Article
• 署名作者: Wang, Ran; Parunandi, Karthikeya S.; Yu, Dan; Kalathil, Dileep; Chakravorty, Suman
• 署名单位: Texas A&M University System; Texas A&M University College Station; Nanjing University of Aeronautics & Astronautics; Texas A&M University System; Texas A&M University College Station
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2021.3108552
• 发表日期: 2022
• 页码: 3582-3589
• 关键词: Heuristic algorithms; trajectory; Approximation algorithms; Stochastic processes; dynamic programming; data models; computational modeling; Reinforcement Learning; stochastic control; Nonlinear systems

摘要

This article addresses the problem of learning the optimal control policy for a nonlinear stochastic dynamical. This problem is subject to the curse of dimensionality associated with the dynamic programming method. This article proposes a novel decoupled data-based control (D2C) algorithm that addresses this problem using a decoupled, open-loop-closed-loop, approach. First, an open-loop deterministic trajectory optimization problem is solved using a black-box simulation model of the dynamical system. Then, closed-loop control is developed around this open-loop trajectory by linearization of the dynamics about this nominal trajectory. By virtue of linearization, a linear quadratic regulator based algorithm can be used for this closed-loop control. We show that the performance of D2C algorithm is approximately optimal. Moreover, simulation performance suggests a significant reduction in training time compared to other state-of-the-art algorithms.