Cautious Bayesian MPC: Regret Analysis and Bounds on the Number of Unsafe Learning Episodes

• 成果类型: Article
• 署名作者: Wabersich, Kim Peter; Zeilinger, Melanie N.
• 署名单位: Swiss Federal Institutes of Technology Domain; ETH Zurich
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2022.3209358
• 发表日期: 2023
• 页码: 4896-4903
• 关键词: Constrained control; nonlinear predictive control; predictive control for linear systems; reinforcement learning (RL)

摘要

This article investigates the combination of model predictive control (MPC) concepts and posterior sampling techniques and proposes a simple constraint tightening technique to introduce cautiousness during explorative learning episodes. The provided theoretical analysis in terms of cumulative regret focuses on previously stated sufficient conditions of the resulting Cautious Bayesian MPC algorithm and shows Lipschitz continuity of the future reward function in the case of linear MPC problems. In the case of nonlinear MPC problems, it is shown that commonly required assumptions for nonlinear MPC optimization techniques provide sufficient criteria for model-based reinforcement learning using posterior sampling. Furthermore, it is shown that the proposed constraint tightening implies a bound on the expected number of unsafe learning episodes in the linear and nonlinear case using a soft-constrained MPC formulation. The efficiency of the method is illustrated using numerical examples.